Categories
Configuration

Will it remain possible to upgrade laptops with SSD?

Last year, I upgraded my sister’s Thinkpad G500 with a SSD, which greatly increased its performance. The laptop also suffered from hardware issues because of its faulty DVD drive; removing the drive surprisingly cured it.

After this success, I was thinking about improving her boyfriend’s laptop, an HP machine that happened to be slower than the older Thinkpad. The presence of a McAfee virus scan, possibly running on top of Windows Defender, wasn’t helping much, but the 5200 RPM hard drive was definitely making startup slow.

Figure out if we can install the SSD

Since Apple started to make it harder and harder to upgrade their laptops, to the point it is now nearly impossible with their most recent MacBook, possibly other laptop makers can follow so it is important to verify if and how we can upgrade the hard drive before purchasing a drive!

The best way to figure this out is to search for hard drive replacement for the laptop model, on Google. The model was HP 15-bw028ca. However, this time, all I could find was the specifications of the laptop, and some YouTube videos showing how to disassemble other similar laptops but not that one! I searched for more than an hour to find out the maintenance manual of that laptop, and then I was able to get the information I needed.

The hard drive was installed behind a bottom cover that can be removed. However, according to the manual, only the battery and optical drive should be removed by the user. Everything under the bottom cover should be serviced only by an HP-authorized technician. Quite bad! But since that laptop wasn’t under warranty anymore, it was less of an issue. But this makes it more important to carefully evaluate if I can reliably remove that cover and put it back, without breaking it. Without the cover, the laptop may at best look ugly, at worst not hold together anymore so not work!

Besides assessing the risk of disassembling the laptop to reach the hard drive without making it ugly or non-working, I needed to figure out the type of drive to install. The machine supports SATA 2.5″ drives, but it also accepts M2 ones. HP used SATA hard drives but M2 SSD. However, M2 requires the replacement of the connector, which is specific to HP. Getting the M2 connector is likely to be problematic, so I decided to try with a SATA SSD, since both hard drives and SSDs can be SATA. I got a 500Gb SSD on Amazon.ca and ordered it.

Replacing the hard drive

When I got the laptop and the SSD, I first examined the laptop a bit and figured out it looked like the one referred to by the manual I found. There was a small difference, though: no optical drive, so the laptop wasn’t the same as the one in the manual.

First I put the laptop upside down and removed the battery, using the latches. That operation was easy as expected. Only MacBook and ultrabooks have soldered batteries that cannot be removed.

The HP laptop upside down
Latches holding the battery

After that, I had to find and remove all the screws holding the cover in place. There were screws pretty much everywhere, even under the four rubber pads and under the battery. Trying to pry the cover starting from the back near the battery slot without removing the screws was a risk of breaking the cover or the chassis, making the reassembly impossible. I was thus quite worried, and more and more concerned the cover couldn’t be removed without a special tool.

Screws can be anywhere, including center and sides

Some attempts at removing that cover caused concerning cracking sounds. I was seriously concerned about the possibility of breaking that laptop altogether. But at some point, the cover unclipped completely, showing up the inside of the machine.

Removed cover
Inside the machine

The hard drive is at the upper left corner of the above picture. It is held in place by a bracket screwed to the chassis, similar to the Thinkpad. I removed the screw and was able to disconnect the drive. Then I transferred the bracket on the SSD and put the SSD in there. Nice, it seemed to work. I was so sure it worked that I put back the bottom cover and the screws.

Unfortunately, when I turned on the laptop, I found out it didn’t detect the SSD at all. I first started Ubuntu from a USB key: that worked but couldn’t find the SSD. I booted up without the USB key and was told to install an OS or press F2 for diagnostic. I pressed F2, tried to start a test of the hard drive, and got a message telling there was no installed hard drive. Could it be that only HP-approved drives can be installed?

I thus had to remove the cover a second time. Before putting back the old hard drive and call it a day (my sister’s boyfriend would have to contact HP that would check his warranty, then advise him to buy a new laptop), I removed the SSD and checked the connection. First time, it was a bit too easy to connect the drive; the drive wasn’t aligned into the SATA connector! The second attempt, I felt a slight resistance showing that the drive engaged into the connector. I screwed it back in and tested, with the cover but not the screws yet. After I checked it worked and passed the auto-test from the HP diagnostic, I put back the screws; the drive was installed and detected!

If I had just put back the old drive, without investigating the connection further, maybe it would have failed again, and I would have been stuck, not able to get the SSD working but also unable to put the laptop back into its original state. Every hardware modification causes such a risk; this must be carefully considered before attempting this. This is why I decided not to try my luck on my Lenovo Ideapad Yoga 13, which contains a too small SSD; that one is trickier to access.

Another activation concern

Then comes the time to install Windows 10. My plan was to use my USB-based installation medium as I did on other laptops. However, will activation work? It was possible that it would not, asking me for a product key. I could not get the product key unless I put back the old hard drive and log in to the old Windows installation, which would either require the password of my sister’s boyfriend or a way to hack the installation in order to turn on the local administrator account. Even with that product key, activation could fail anyway, requiring me to call Microsoft and try by phone. This could have gone as far as requiring HP’s custom recovery partition, or the purchase of a new product key.

Fortunately, the simple installation worked like a charm. The installer didn’t ask me for any product key and after that, Windows was activated!

Some testing and post-installation steps

After that successful installation, I installed the drivers and tested the machine a bit. It was working and not crashing. I added Firefox and LibreOffice and I created the user account for my sister’s boyfriend, making sure it was set to Administrator and not Standard account.

Restoring Enigmail

My sister’s boyfriend is using Thunderbird and Enigmail to send encrypted messages to some of his friends. If we don’t fully restore his configuration, that means he will have to generate a new private key, notify all his friends about that new key and get back all their public keys. That is kind of annoying and inefficient, both for him and his friends. I thus wanted to restore this configuration but we didn’t know where it was stored.

I had to install Enigmail on my machine to test and figured out it’s using the GnuPG’s keyring. That keyring is located inside the Application Data folder under the gnupg directory. I wasn’t sure he found and backed up that directory, so I plugged his hard drive on a SATA to USB adapter and got the directory back. He would thus be able to copy it at the right location, after he installed Thunderbird and Enigmail.

Why couldn’t have I set this back up completely? Because that would require logging in to his account, which would have required his password. Then to set up his GMail account, I would need his GMail password. It is important that passwords remain secret, even if both of us knew I would not misuse the password afterwards.

Categories
Bug

A dying laptop

My Lenovo Ideapad Yoga 15 is dying.

It happened the second time today, and that’s quite concerning. Suddenly, the machine freezes, keyboard is not responding, the mouse is moving but clicking does nothing. There is no way out other than rebooting, and when I did it, the screen became black with an error message displayed in white on a blue background.

The machine was telling me that there was no boot device or boot failed. What? This Lenovo laptop has a builtin mSATA SSD. I didn’t disassemble the laptop, ever, so why suddenly either the SSD became faulty, or the connection between the SSD and the motherboard is now unstable? Fixing any of these would require disassembling the laptop, with the risk of not being able to put it together again afterwards. The keyboard that needs to be removed to reach the components is clipped, and clips can break when trying to remove.

As the last time, during the holidays, powering off the laptop and booting up again “fixed” it. But who knows if that will last. Maybe it will fail tonight, maybe the next week, I don’t know. And next time it fails, maybe it won’t boot up again.

Besides of that, the battery lasts not really more than 2 hours and the wi-fi is now so clunky that most of the time, I need to plug in a USB to Ethernet or 802.11 adapter.

I’m wondering if that is Windows 10 that is killing this machine. As soon as I upgraded, the laptop became slower. A few months later, the battery life was reduced. If I could make Ableton Live work on Linux, I would attempt a switch to Ubuntu, but nope, no Linux version. All major DAWs have just Windows or Mac builds, no Linux. Things may change in 10-20 years, but that doesn’t matter to me, my passion may not be there at that time.

I’m kind of hopeless now to get any type of working laptop in the 10-20 next years. They all suck. Any candidate, any, I can find bad reports on some forums. People are slowly but surely stopping to use computers and/or migrating to Mac, which I cannot do because of font size issues. Maybe home computing is coming to an end, replaced by “smart” but limited phones and tablets not offering any efficient way of typing text. People are working around by posting meaningless pictures with any description, that’s what I see every day on Facebook: XYZ added on his story, no text, just a picture. I came to the point of disabling Facebook notifications on my smartphone and debating the possibility of closing my Facebook account.

But I want a solution. As a computer scientist, I just cannot give up having a computer, that makes just no sense to me.

Categories
Configuration

Trying to salvage a lemon

My sister got a Thinkpad G500 from Lenovo pretty much at the same time I purchased my Ideapad Yoga 13 from the same brand. My Ideapad, while suffering from battery life and wi-fi issues and being stuck with a too small SSD, still performs reasonably. It still boots fast and doesn’t exhibit hardware issues. The upgrade to Windows 10 seems to be the change that made it worse, but it still works… compared to the G500 of my sister. That machine was painfully slow and, I discovered, suffered from various hardware issues I didn’t know about.

Assuming the only issue was slowness, I proposed my sister to replace the hard drive of the laptop with a SSD. Before doing so, however, I checked in the user’s manual of the laptop how to replace the drive. It was possible to do it by removing the back cover of the machine and unscrewing the drive, no need to disassemble the keyboard, remove the memory, the display, etc., like on my Ideapad or even worse, on a MacBook. If there is too high risk of breaking something while dissassembling the laptop to reach the hard drive, it is better to leave that machine alone and invest in a new one later on. This is what I chose to do for my Ideapad, because of brittle keyboard clips that could completely break, preventing the keyboard to hold on afterwards. I also needed to make sure a 2.5″ SATA SSD would fit, because maybe I could need a M2 instead. If something intermediate is needed, like a mSATA, upgrading is less worth it. A SATA or M2 SSD can at least be reused in another system if the target laptop fails later on.

The SSD doesn’t fit! WHY?

I started to work on the laptop January 5, 2018. I first removed the battery, unscrewed the back cover and located the hard drive. It was attached to the system using a bracket screwed into the chassis. I removed the screw and was able to disconnect and pull off the drive. I then needed to unscrew the bracket from the hard drive and screw it to the SSD instead. That part didn’t work well. No matter how hard I was trying, I wasn’t able to fit the screws. I thought the laptop had a 1.5″ hard drive instead of a 2.5″, but no, it ended up working. The SSD had something written on both sides so I was trying to install the bracket on the wrong side. After I was able to screw the bracket, I connected the SSD to the laptop.

After I installed the SSD, I put back the cover and the battery. Then I turned on the laptop and booted using a USB key containing Ubuntu Live. I used that to run a SMART check on the SSD, making sure it wasn’t flawed right from the start. This is more likely to happen with an hard drive, but it CAN happen with a SSD as well. Doing a self-test before installing anything can thus save a lot of time.

Stuck at Lenovo logo, requiring RMA

After that check, I inserted my USB-based Windows 10 installation medium. I got this medium from Microsoft, it can now be downloaded freely as opposed to previously, requiring the purchase of a CD or DVD. Getting the medium is now easy, the challenge is to get the activation working now.

But before tackling the activation issues, I had to make this laptop boot the USB medium. Instead of booting, the laptop just froze at the Lenovo logo. The only thing I could do is hit ctrl-alt-delete, get a blank screen, and the logo back again. I tried to power off the laptop, power it back on, to no avail.

I searched on Google and found other occurrences of this issue. Several people are experiencing that problem, with no other solution than contacting Lenovo’s technical support and get the laptop replaced… when it is under warranty. It seems more and more that when warranty is over, a laptop is now a piece of crap that is just good to be thrown away, which annoyed me quite a lot. Fortunately, after a couple of attempts, removing the battery, putting it back in, rebooting again, again and again, I got past the frozen Lenovo logo. After that hurdle, I was able to install Windows 10.

Windows 10 asked me to connect to a Microsoft account. I used mine in order to perform the installation, but I knew I would need to do something to hook up my sister’s account, so she could login without me having to give her my main password.

Activation working without efforts? Strange…

After I finished installing Windows 10, I noticed from the system properties it was activated. Cool. However, I found that without even asking me, the system used the same product key as my own Lenovo Idepad Yoga 13 ultrabook. I was worried that the Microsoft’s tool creating the USB medium customized the installation USB key with my product key, so I was attempting to activate Windows on multiple computers with the same key. Maybe at some point, Microsoft will detect that and deactivate one of the two copies, either mine or the one of my sister, after she got her laptop back.

After significant amount of wasted time searching on Google, I figured out that this situation happened to others. It seems that the product key is used by several laptops of a given brand. Either the key is hard-wired somewhere in a read-only store of the machine, maybe the trusted platform module, or there is a registry at Microsoft of OEM machine ids mapped to product keys. Anyhow, my sister got a fully activated Windows 10 without any effort from me.

I was worried I would need to have her purchase a new license key or reactivate her current license via phone. Or maybe even worse, the current license would work for Windows 8, not Windows 10, and my sister would prefer to get back Windows 8. All my concerns went away with this effortless activation, but keep in mind things will not always be as smooth.

Looping Windows Update

That one caused me a lot of wasted time. There was a bug with one of the updates that failed to install. However, the update was partially installed. The failure caused Windows to retry installing the update each and every time the computer was shut down or rebooted. This pretty much wiped out the benefits of having a SSD because of increased shut down, reboot and even boot time, since at boot, Windows was finalizing the installation of the faulty update and failing!

I searched a long time for that one, tried to manually install the update to no avail. Some forum posts were referring to that problem without a known solution. Sometimes, it worked at some point after a lot of attempts. Other times, it required reinstallation. I even read a post from a user who called Microsft, got a new ISO image of Windows 10, installed that and the update worked! But then, why is there a medium creation tool if we need to get an ISO from elsewhere? Will everybody reinstalling Windows 10 from scratch need to call Microsoft to get that alternate ISO? Really? That is a serious bummer in my opinion. I thought about asking a link to that ISO, but that would have given me the English version while my sister wanted the French one.

At some point, I got so pissed off that I started searching a way to disable the automatic updates. This is possible by changing group policies… on the Pro version of Windows 10. My sister had the Home version. But the problem didn’t happen on my systems, probably because I started from Windows 8, then got 8.1, then 10. Maybe I’d need to go that same route on that Thinkpad?

Fortunately, there is a tool from Microsoft called Show or hide updates. I installed that tool, and told it to hide the offending update. That fixed the issue without having to tentatively reinstall, call Microsoft or try to install Windows 8, then upgrade to 8.1, then upgrade to 10!

Short-term laptop to be replaced after warranty?

What happened after the installation of Windows 10 pretty much lead me to believe what the section’s title states. I was really shocked and annoyed and questioning my trust against Lenovo. A PC is not like a simple appliance you just plug in and start. It has settings, it has applications installed, there is sometimes even a physical configuration to get used to. I cannot afford having to fully replace that configuration every year or two! That’s a serious non-sense, and not counting the very bad ecological impact of such a short-sighted offering. I know, it was not my laptop, but my next laptop could very well suffer from these more and more common flaws.

The freezing at Lenovo logo was just the tip of the iceberg! I then discovered that only Windows 8 drivers were offered for that Thinkpad G500 on the Lenovo website, no Windows 10 drivers at all. However, the preinstalled drivers coming with Windows 10 allowed the machine to pretty much work. Later on, I also found out that the DVD drive was also broken, not detected at all by Windows. Moreover, the Webcam was broken, showing a black screen.

Wi-fi started to go bad, the mouse pointer started to move erratically, the machine just became totally unusable. I had to plug in that laptop to an external keyboard, mouse and even an Ethernet cable.

Again, for the DVD drive and Webcam, the only solution was to contact Lenovo and get replacements, IF the laptop was still under warranty. At this point, I was stuck. Without another idea, I would have had to give up and tell my system the best solution is to throw that laptop away.

Pulling some things off

I noticed a black tape at the place of the Webcam. Maybe that tape is not supposed to be there and can be removed. I removed it and started the Camera application again: I got an image, yeah!

I then found out that some people were having issues with both the DVD drive and the freezing at boot. Could these two be related? If the DVD drive is flaky, it can slow down or even prevent boot as the BIOS/UEFI will try to query it in order to figure out if a disk is inserted.

The solution was quick and simple: just remove the optical drive from the machine, yes, really, pull it off. After I did that, the laptop booted flawlessly. I tested it several times, also tried to reboot, without any freezing issue. It also seemed, although I didn’t benchmark it formally, that the boot was faster. There would be a hole in the laptop casing instead of the DVD drive, because I didn’t have any other drive or something dummy to go into the bay. But at least, it would work.

Transferring ownership without my password

A small but significant step remained: how to allow my sister to log in to her “new” laptop without giving her my personal password in order for her to create her Microsoft account, or me getting her password to connect to her account? There are two ways to solve this cleanly, and I implemented both to be sure.

  1. Create a new account based on a known email address linked to a Microsoft account. I knew my sister’s email address and was sure she used that to create her Microsoft account, so I just had to create the account with that address; no need to enter the password until logging in. There are two pitfalls though. Firstly, the person needs to be connected to Internet for the first login, and not sure wi-fi will work, maybe just wired Ethernet since you need to be logged in to set up wi-fi! Secondly, the created account is not Administrator by default; I had to fix that so my sister would be able to install new programs on her machine.
  2. It is still possible to create a local account, so I did it and set a dummy password my sister can change after, or remove the local account altogether. Again, I needed to make sure the local account was in the Administrators group; by default it is not!

My sister was amazed at the speed increase we achieved by replacing the hard drive with a SSD. She thought that laptop was good for the thrash can before I fixed it. Even funnier, later on her boyfriend got an HP 15-bw028ca that although more recent, happened to be slower than that old fixed Thinkpad!

Eventually that HP piece of junk will benefit from a similar treatment. Maybe that will deserve another post.

Categories
Technical analysis

The downsides of SSDs

What’s the point of having a SSD if both Windows 8 and Ubuntu 15.04 introduce artificial timeouts that increase the boot time, making this equivalent as having a standard hard drive? Well, I’m there, I reached that point.

Windows 8 often boots fast from EFI to login screen but after I enter my password, it sometimes reaches the desktop in five seconds, sometimes hangs for 30 to 45 seconds. There is no obvious reason why, no way to track this down and no obvious solution other than deleting my user account and creating a new one. I cannot spend my weekends doing, redoing, redoing and redoing that. This is just pointless and inefficient! I could try to reinstall, but then I will have trouble with reactivating Windows, reauthorizing Ableton’s Live and have to spend hours waiting for manual installation of countless drivers and software tools. Ninite can help with programs, not with drivers.

Some time later, I found out that uninstalling and reinstall the driver for my M-Audio interface fixed the slow boot up. There seems to be a conflict between the M-Audio’s Fast Track Pro and Novation’s UltraNova drivers. Windows 10 also seemed to stabilize things a bit.

Ubuntu, most of the times, boots quickly. However, starting from 15.04, it was taking almost a minute from splash screen to login screen. I had to spend more than half an hour looking at syslog to figure out that the swap partition changed UUID but the update script didn’t reflect that into /etc/fstab. Several people repeat that we shouldn’t do dist-upgrades and rather reinstall, but then, why is there a dist-upgrade option at first place? Fortunately, fixing the partition ID in /etc/fstab restored my boot time.

This is not SSD-specific issues, but they cause the SSD to be less useful. Another factor reducing usefulness of SSD is the never-ending size increase of OS and applications, especially when dealing with virtual machines. This ultimately fills any SSD, requiring time consuming reorganization of the layout (partition resizing, copying on a larger drive, etc.).

I don’t want to go backward, switching from SSD to an hard drive, but practice seems to tell me I should. This is disappointing and quite frustrating.

Categories
Configuration

One SSD for my HTPC

A bit more than a month ago, I successfully transferred my dual boot Windows 8.1 and Ubuntu 14.04 from two 120Gb Solid State Drives (SSD) to a single 240Gb drive.  I got several problems restoring the bootloaders of the two operating systems, thought many times I would have to reinstall, then figured out a way to make them boot.

But what happened to the two drives I removed from my main computer? Well, they sat still on the top of a shelf. But at least one drive will be repurposed: become part of A.R.D.-NAS, my HTPC. Sunday, October 26 2014, I finally got the time and courage to undertake the transfer operation. This time, the software part was pretty smooth, but the hardware part was a uselessly intricate puzzle. During the process, I wondered myself many times about the purpose of generic hardware if it doesn’t fit well together and pestered about the lack of any viable alternatives.

The sacrifice

Well, my NMedia HTPC case has six 3.5″ drive bays. This is quite nice for an HTPC case. This is possible, because I chose an ATX case, to get a motherboard with rear S/PDIF audio connectors rather than just headers accepting brackets I could get nowhere. This case is a bit bulky; I would build off a MicroATX case if I had to start from scratch.

So installing this SSD seemed obvious at start: just add the drive, transfer the Linux boot partition from the hard drive to SSD, remove the original boot partition, setup GRUB on SSD and tada. No, things are rarely as simple. I thought my motherboard had only 4 SATA ports, and they were all used: one 1Tb hard drive, a second 1.5Tb hard drive, a third 3Tb hard drive, then a blu-ray reader/writer. Why so many hard drives? Well, I am ripping and storing all my video disks, even the huge blu-rays, to avoid the need for searching for them on shelves.

Even if I remembered correctly I had six ports on the motherboard (two are free!), my PSU only had four SATA power connectors, so I would not be able to easily and reliably connect all my drives. I could try to find some splitter cables or molex to SATA adapters, but that would add a factor of failure. I could replace my PSU for one with more SATA power cables, but it would also have more molex cables, PCI Express connectors, etc. Unless I went with a more expensive modular PSU, all these cables would have cluttered my case.

Safest and cheapest solution was to sacrifice one of the hard drives, the 1Tb one of course, the smallest. I thus had to move files around to have less than 120Gb of stuff on the hard drive that would be moved to SSD. That process took a lot longer than I thought. My poor HTPC spent the whole Saturday afternoon copying files around! Fortunately, this is machine time so I had plenty of time to experiment music creation with my still new UltraNova synthesizer combined with Ableton’s Live multri-track ability.

Preparation

On Sunday, I first burned the Ubuntu 14.04 ISO on a DVD. Yes, Ubuntu is now large enough to fit only on a DVD. After that, I shut down my Minecraft server running on my HTPC and moved its files to another old PC. I started the server on the old PC and reconfigured the port mapping on my router. This way, if my friend wanted to kill a bit of zombies and creepers while I was installing my SSD, he would be able to do so and I would not be stressed if something bad made my HTPC out of service (like something stuck in the CPU fan breaking it).

I then removed the cover of my HTPC and spent quite a bit of time trying to figure out what was the 1Tb hard drive. Based on the position of the SATA connector on the motherboard, I presumed that was the left-most drive. I thus had to disconnect the drive in the middle bay, and use the freed up power and data connectors to hook up my SSD. I then booted up the machine.

Following picture shows the drive temporarily hooked up.

DSC03934

Then I remembered about my old 22″ LCD that I stopped using after purchasing my Dell touch screen. I went pick it up in my computer room, put it on my kitchen table and plugged it in. This way, I would be able to have the screen right in front of me, keyboard on the table, rather than in front of my 46″ LCD HDTV with the keyboard on my knees.

The SSD and the LCD hooked up, I booted up my HTPC and quickly sticked the Ubuntu DVD in the blu-ray drive. After an incredible amount of time, the machine finally booted up into Live Ubuntu DVD!

Data transfer

After Ubuntu started, I launched GParted and realized that I chose the wrong hard drive. The 1Tb drive containing my Ubuntu setup was disconnected. Oh no! So that means I will have to turn off the machine, connect the right drive and wait once again for this stupidly long, almost five minute, DVD-based boot? No, not this time! Feeling like a cowboy, I decided to try something: drive hot swapping. This is possible with SATA, so let’s see! I thus disconnected the 1.5Tb hard drive, starting with the SATA data cable, then the power cord, then hooked up the 1Tb drive. Hearing the old hard drive coming back to life was kind of thrilling. Everything went well, no cable stuck into my CPU or rear fans, and the PC didn’t freeze like it would do with IDE. The hot swap worked.

After that, this was relatively straightforward. As with my main PC, I used GParted to transfer my Linux boot partition and reconstruct the layout. I fortunately remembered, before, to reset the partition table. If I didn’t do that, the GPT that was on my SSD would have caused booting issues that would have drove me mad! I would probably have ended up reinstalling everything, angry against Ubuntu, the technology and probably the whole human kind. A single step, recreate the msdos partition table from GParted before the transfer, saved me that!

Following picture shows my LCD on which we can see the progress of the transfer.

DSC03935

See how bulky was this setup: HTPC on the floor, case opened, SSD hanging on top. Hopefully it was possible to make this setup clean once again after all this.

DSC03936

The home partition: too big to fit on the SSD

Unfortunately, GParted didn’t want to transfer my home partition to the SSD, because it was obviously too large. I could have shrunk it in order to copy it, but I wanted to avoid altering the hard drive in case something bad happened. I thus instructed GParted to simply create a blank Ext4 partition and used cp to perform the copy. The following terminal session shows how I managed to do it in such a way that all files metadata (timestamps, ownership, permissions) was preserved.

ubuntu@ubuntu:~$ mkdir /media/old-home
mkdir: cannot create directory ‘/media/old-home’: Permission denied
ubuntu@ubuntu:~$ sudo mkdir /media/old-home
ubuntu@ubuntu:~$ sudo fdisk -l /dev/sda

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e2c4d

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    40965749    20482843+  83  Linux
/dev/sda2        40965750    81931499    20482875   83  Linux
/dev/sda3        81931500  1953520064   935794282+   5  Extended
/dev/sda5        81931563  1943286659   930677548+  83  Linux
/dev/sda6      1943286723  1953520064     5116671   82  Linux swap / Solaris
ubuntu@ubuntu:~$ sudo mount -t ext4 /dev/sda5 /media/old-home/
ubuntu@ubuntu:~$ ls /media/old-home/
eric  lost+found  mythtv
ubuntu@ubuntu:~$ sudo fdisk -l /dev/sdg

Disk /dev/sdg: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders, total 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d9a0c

   Device Boot      Start         End      Blocks   Id  System
/dev/sdg1            2048    40968191    20483072   83  Linux
/dev/sdg2        40968192    81934335    20483072   83  Linux
/dev/sdg3        81934336   234440703    76253184    5  Extended
/dev/sdg5        81936384    92170239     5116928   82  Linux swap / Solaris
/dev/sdg6        92172288   234440703    71134208   83  Linux
ubuntu@ubuntu:~$ sudo mkdir /media/new-home
ubuntu@ubuntu:~$ sudo mount -t ext4 /dev/sdg6 /media/new-home/
ubuntu@ubuntu:~$ sudo cp -a /media/old-home/* /media/new-home
ubuntu@ubuntu:~$ ls -a /media/new-home/ -l
total 36
drwxr-xr-x  5 root root  4096 Oct 26 19:49 .
drwxr-xr-x  1 root root   100 Oct 26 19:44 ..
drwxr-xr-x 69 1000 1000 12288 Oct 25 22:52 eric
drwx------  2 root root 16384 Sep 26  2009 lost+found
drwxr-xr-x  3  122  130  4096 Jan 24  2011 mythtv

The main idea is to mount both the old and new partitions, then use cp with -a option and root access (with sudo) in order to preserve everything. The operation went smoothly.

The boot loader

Even after copying all Ubuntu-related data from my old hard drive, my SSD was still not bootable. To make booting off the SSD possible, I had to install GRUB. Unfortunately, reinstalling GRUB on Ubuntu is not as simple as it should be. If there is a package doing it, why isn’t it built into Ubuntu’s image? Maybe because for most setups, reinstalling from scratch takes 15 minutes. That’s true, but then how about tweaks to fix mouse pointer too small, make XBMC work with S/PDIF sound, reinstall MakeMKV, etc.? Each step is simple, at least when no unexpected difficulty creeps in, but the sum of small things to tweak makes it long.

So let’s avoid this by running the following!

ubuntu@ubuntu:~$ sudo mkdir /media/ubuntu
ubuntu@ubuntu:~$ sudo mount -t ext4 /dev/sdg1 /media/ubuntu/
ubuntu@ubuntu:~$ sudo mount --rbind /dev /media/ubuntu/dev
ubuntu@ubuntu:~$ sudo mount --rbind /sys /media/ubuntu/sys
ubuntu@ubuntu:~$ sudo mount --rbind /proc /media/ubuntu/proc
ubuntu@ubuntu:~$ sudo chroot /media/ubuntu
root@ubuntu:/# grub-install /dev/sdg
Installing for i386-pc platform.
Installation finished. No error reported.
root@ubuntu:/# update-grub
Generating grub configuration file ...
Warning: Setting GRUB_TIMEOUT to a non-zero value when GRUB_HIDDEN_TIMEOUT is set is no longer supported.
Found linux image: /boot/vmlinuz-3.13.0-37-generic
Found initrd image: /boot/initrd.img-3.13.0-37-generic
Found linux image: /boot/vmlinuz-3.13.0-36-generic
Found initrd image: /boot/initrd.img-3.13.0-36-generic
Found linux image: /boot/vmlinuz-3.13.0-35-generic
Found initrd image: /boot/initrd.img-3.13.0-35-generic
Found linux image: /boot/vmlinuz-3.2.0-61-generic
Found initrd image: /boot/initrd.img-3.2.0-61-generic
Found linux image: /boot/vmlinuz-3.0.0-17-generic
Found initrd image: /boot/initrd.img-3.0.0-17-generic
Found linux image: /boot/vmlinuz-2.6.38-12-generic
Found initrd image: /boot/initrd.img-2.6.38-12-generic
Found linux image: /boot/vmlinuz-2.6.32-25-generic
Found initrd image: /boot/initrd.img-2.6.32-25-generic
Found linux image: /boot/vmlinuz-2.6.31-21-generic
Found initrd image: /boot/initrd.img-2.6.31-21-generic
Found linux image: /boot/vmlinuz-2.6.28-16-generic
Found initrd image: /boot/initrd.img-2.6.28-16-generic
Found memtest86+ image: /boot/memtest86+.elf
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 14.04.1 LTS (14.04) on /dev/sda1
done

The main idea here is to create a chroot environment similar to my regular Ubuntu setup, then install GRUB from here. I ran update-grub to make sure any disk identifier would be updated, pointing to the SSD rather than the old hard drive. Unfortunately, a small glitch happened: update-grub detected the Ubuntu setup on my hard drive. To get rid of this, I had to unmount the old hard drive and unplug it! After rerunning update-grub, I got the correct configuration.

Updating mount points

Since I rebuilt the home partition rather than copying it, its UUID changed so I had to update the mount point in /etc/fstab. I thus had to run the following:

root@ubuntu:/# cd /dev/disk/by-uuid/
root@ubuntu:/dev/disk/by-uuid# ls /dev/disk/by-uuid/ -l | grep sdg6
lrwxrwxrwx 1 root root 10 Oct 26 15:37 fb543fcb-908a-463d-bc1f-896f1892e3ad -> ../../sdg6
root@ubuntu:/dev/disk/by-uuid# ls /dev/disk/by-uuid/ -l | grep sdg1
lrwxrwxrwx 1 root root 10 Oct 26 16:11 54f4cbd6-aed0-4b43-91c0-2f8d866f3ee3 -> ../../sdg1

After I figured out the UUIDs, I had to open up /media/ubuntu/etc/fstab with gedit and make sure the mount points were correct. I only had to update the UUID for the /home partition.

Test boot

After all these preparatory steps, it was time for a test! I thus powered off the computer and made sure my old 1Tb hard drive was unplugged and my new SSD was hooked up. I turned on the PC and waited for the forever lasting BIOS POST. Why is it so long to boot a desktop while a laptop almost instantly hands off control to OS? After BIOS handed off control to OS, I got a blank screen with a blinking cursor, nothing else. I tried a second time: same result.

So after all these efforts, do I really have to format and reinstall from scratch? It seems so. Before doing that, I rebooted my machine once again and entered into BIOS setup by hitting the DEL key. Once there, I looked at the hard drives and found that the SSD was hooked up but it was not at SATA port 0.

I turned off the machine and connected the drive into a different port, what seemed to be the first. Looking into the BIOS setup again, my SSD was now at port 0. Ok, let’s try that a last time!

After a blank screen lasting too many seconds for a SSD boot and making me fear for a frustrating reinstall, the Ubuntu logo appeared, and my desktop finally came up! A quick check confirmed me that all the hard drives were present, except of course the disconnected 1Tb one. The SSD was ready to be installed into the machine!

The hardware part

The downside of SSD is that they seem not to fit in any regular desktop cases, only in laptops! This is a very frustrating limitation. Why are these drives all 2.5″ or why cases don’t have 2.5″ bays? When I shopped for my computer case, only high end ones had the 2.5″ bays and that was coming with fancy mechanisms to make the drive removable without plugging any cables, something adding into problem factors. Maybe at the time I am writing this post, some cases with SSD bays are available, but that doesn’t matter; I won’t change my case unless I really need to!

Before installing the SSD, I first removed that old 1Tb drive. I just had to remove four screws from my drive cage and slide the drive out.

DSC03938

DSC03939

To help me install my SSD into my HTPC case, I had a bunch of screws as well as an OCZ bay bracket. Just screwing the drive into the adapter’s tray took me forever, because I had trouble finding screws that fitted, there was a screw in one of the SSD hole I don’t know exactly why and that took me almost five minutes to realize. I then had trouble aligning the screws with the hole, was getting more and more tired and prone to drop screws, etc. At least, the screw I dropped fell on my table so I didn’t have to search it on the floor forever.

Following picture shows the drive in the bracket.

DSC03937

Then I had to screw that assembly into the drive cage of my case. Unfortunately, the upper bays of the cage only offer bottom holes while the SSD adapter has only side screws! I thus had to screw the adapter in one of the bottom bays, which are definitely suited for hard drives with their rubber pads to absorb vibration. None of my screws fitted well. It seems that the SSD adapter has holes smaller than normal while the screws for the drive bay are larger than usual! I got it after more than 15 minutes of attempts. I thought many times I would have to postpone this job and wait for my father to come by with a drill and make some new holes into the SSD bracket or the case.

Following picture shows the drive in the cage.

DSC03941

I don’t know exactly how much time I spent on this installation, but at the end, I was tired and was asking myself if all this would be worth it in the end.

Well after the SSD was screwed and the drive cage back into my HTPC case, I realized I wouldn’t be able to hook up my four SATA drives! No matter what I tried, there was always one drive lacking power. This was because the SATA cable coming out of my power supply unit were too short to accommodate the drive layout I came up with! Ok, I’m at a dead end now.

Before giving up and bringing that beast to a computer store in the hope they would figure out a way to hook the four drives up (maybe with some extension cable I don’t have, or using a new PSU), I remembered that the 1Tb drive I removed was in the middle upper bay which was now empty. My only hope to get the drive powered this day was thus to move one of my hard drives there. Ok, so let’s remove the cage again and play with the screwdriver once more!

I moved my 3Tb drive from the side bay to the upper one and put the drive cage back into my case. I was then able to hook up power. Reaching the SSD drive in the side bay to hook up SATA cables was a bit tricky, but I finally got it. A last check confirmed that all my drives were hooked up, except my blu-ray writer. Ok, just a cable to plug in, and that was it!

Was this all worth it?

After all this hassle, I asked myself this question. When I booted up the machine, it seemed as slow as with the hard drive. What? Maybe the CPU is too slow, after all. But when I reached the desktop and started XBMC, I felt the system was more responsive.

More importantly, the machine became a lot more silent. Since a few weeks, this HTPC was making a lot of noise. I thought it was the CPU fan stressed out by the Minecraft server running on the system, but the 1Tb hard drive was contributing to the noise as well. I suspect it was emitting more and more heat, causing the temperature to raise inside the case and heating up my poor little CPU. The CPU fan was then reacting by spinning like crazy.

Even after I restarted my Minecraft server, the sound didn’t come back. I am still surprised by this effect which I didn’t expect.

This 1Tb hard drive is definitely getting old and emitted some suspect sounds a few times. I am wondering if it would have failed and died if I left it in the machine. This SSD move thus saved me an unexpected reinstall and will help me have a better time with this HTPC.

So yes after all it was worth it!

Categories
Configuration

One SSD instead of two: simpler or not?

My Core i7 machine, named Drake, had two 120Gb SSD drives. I purchased the first one with the machine and put Windows 7 and Ubuntu on it. Then I needed more space to get Mac OS X, so I added a second 120Gb SSD. Mac OS X became a pain, almost unusable because everything was too small. When I reached the point I had to lower screen resolution to get Thunderbird running comfortably, I got rid of Mac OS X. Then Windows 7, upgraded to Windows 8, started to eat up more space so I needed to move Ubuntu to the second SSD.

I ended up with a brittle configuration composed of the ESP (EFI system partition) on the second SSD, Windows 8.1 on the first drive and Ubuntu on the second. I was waiting for a special deal on a 240Gb SSD and finally got one on TigerDirect at the beginning of September 2014. However, purchasing the SSD is only the easy part. Migrating data from two SSD drives to a single one, with Windows 8.1, Ubuntu 14.04 and UEFI in the way, is an incredible source of headache. This page shows how I got it done.

The easy way: reinstall everything

That would have worked ten, maybe even five years ago. Yes, just reinstall Windows, a few drivers, a few programs, put back Ubuntu, perform some settings, fine tune a bit, and enjoy the rebirth of the system, coming back to life and full functionality. Things changed with years, not for good. Now that Microsoft and other hardware manufacturers assume people won’t install by themselves and rather purchase hardware with everything preinstalled and preconfigured, things became more and more time consuming to setup. Just installing Windows 8 takes more than 45 minutes, and although I could obtain a DVD with Windows 8.1, my Windows 8 serial number won’t work with it. I would have had to install Windows 8, then upgrade to Windows 8.1 again!

Then come the drivers. Since I purchased my motherboard before Windows 8 was released, all my motherboard CD has to offer is Windows 7 drivers. So I cannot use the easy auto-install tool performing an unattended setup. I rather have to download every driver separately from Asus, run them, wait, reboot, run the next one, etc. Then there is the NVIDIA driver, requiring 100 Mb of download and yet another installation taking more than five minutes, and yet another reboot. Maybe I chose the wrong motherboard. By sacrificing a few USB ports, S/PDIF audio and maybe some PCI Express slots, maybe I could get something simpler not requiring as many drivers, that would be able to make use of what is prepackaged within Windows. That’s still to be investigated.

Then come the programs. Yes, Ninite can install me many programs automatically but not GNU Emacs, GNU GPG, it won’t configure my Dropbox, resync my Firefox bookmarks, reinitialize my Thunderbird email settings. It won’t link back my Documents, Images, Music and Videos default folders to my data hard drive.

And then come the licenses. How Windows 8.1 activation will behave? Will it happen smoothly, or will Windows decide that this change of SSD is too much and require me to call Microsoft to perform activation by phone, forcing me to exchange, by voice, on a poor channel, tens of nonsensical digits? After Windows 8.1 activation, my DAW, Live from Ableton, also requires authorization. I’m not sure it will reauthorize, since I activated it on my main PC as well as my ultrabook. That means additional hassle.

Bottom line, reinstalling is a pain, and that is just the Windows side. Ubuntu installation is usually smooth, but when a single thing goes bad, it requires hours of Google searches.

This is why I wanted a better way. I was so tired of this tedious process I was considering giving up on this machine and use my ultrabook instead, if data transfer failed. But my ultrabook, with its 128Gb SSD, won’t have enough storage for editing music made of samples or recording/editing Minecraft videos.

Preliminary connection of the new SSD

Before installing the new 240Gb SSD into my system permanently, I wanted to be sure I would be able to transfer my two operating systems (Windows 8.1 and Ubuntu 14.04) and make them boot. I thus only plugged the disk rather than attaching it right away into my case. I fortunately had some free SATA power cables as well as an extra SATA cable and port. That allowed me to connect the new drive without disconnecting the others. This way, it would have been easy to roll back in case of difficulties forcing me to reinstall everything, and then think about another strategy or gather my courage and patience for the full reinstall.

I then booted from a USB stick with a Live installation of Ubuntu 14.04. This was necessary to perform the data transfer on a totally offline, clean, file system.

Before transferring anything on the drive, I ran a SMART self test. For this, I installed smartmontools with apt-get and ran sudo smartctl -t long /dev/sdb. At this time, /dev/sdb was the device of the drive. That took almost an hour, but I could leave this running and do something else.

The self-test found no defects. I learned to do this preliminary step the hard way when I assembled a machine for my parents. The hard drive failed short while I was configuring Windows and I had to RMA it. Performing a self-test may have avoided me a waste of time and some frustration.

The drive being clean from any defect, at least from the point of view of the self test, I moved to the next step: data transfer.

GParted is the king!

A long time ago, my only friend for partitioning and drive transfer was Parition Magic, from PowerQuest, now purchased by Symantec. That time is over, thanks to GParted, a free open source tool that comes with Ubuntu. But that time, my job was pushing GParted to the limits. Here are the operations I needed to perform with it:

  1. Create a GUID Partition Table (GPT) on the new SSD. This is because I want a pure UEFI-based system. But this is not strictly necessary since the drive is far from the 2Tb limit!
  2. Copy the first partition of my second SSD at the beginning of the new drive: this is the ESP.
  3. Copy the first partition of the first SSD: this is the 128Mb system reserved partition of Windows. That copy wasn’t possible, because GParted didn’t know the partition type. I thus left a 128Mb hole declared as Unformatted, to figure out a way out later on. I was hoping Windows could recreate the data on this partition.
  4. Copy the second partition of the first SSD: this was the Windows main partition.
  5. Copy the 40-ish Gb partition of my second SSD at the end of the new drive: this was my home drive from Ubuntu.
  6. Copy the 20-ish Gb partition of my second SSD at the bottom of the free space on new drive: this was my main Ubuntu installation.
  7. Create an extra 20 Gb partition on the new drive in case I would like to give a shot to a new Linux distribution.
  8. Create a 16Gb swap space on the new drive for Ubuntu’s use.
  9. Resize my Windows main partition to take the rest of the space.

Phew!This long sequence gathering pieces from different sources reminds me of infusion crafting in the Thaumcraft mod of Minecraft, where essentias and items are combined together on an altar to craft powerful magical objects.

I hoped that sequence would work, but that failed at step 5. For no obvious reason, GParted wasn’t able to copy my Ubuntu home drive at the end of the new SSD! I had to leave an 8Mb gap and then resize the partition to fill it. I then performed, one by one, the other operations. That was a quite tedious job, because the mouse pointer was too small and impossible to enlarge without a system hack (Ubuntu bug since 11.10! They chose to remove the option to resize mouse pointer rather than fixing the issue.) and sometimes clicking was opening the menu and closing it right away rather than leaving it open.

Following image gives the final layout. Isn’t that great? Not sure at all this is simpler with one drive than with two, after all…

gparted

After this transfer process, I tried to recreate the entries in my UEFI’s NVRAM, using efibootmgr, for Windows and Ubuntu. I then unplugged the SATA cables of my two 120Gb SSD drives from my motherboard and rebooted the PC. I won’t state the exact commands I used here, because that just failed. System wasn’t booting at all.

Fixing Ubuntu

Back to my Ubuntu live USB, after at least five attempts because my motherboard is apparently defective and misses the F8 key from time to time and the need to jump into Setup and change the boot order from there to boot the UEFI USB stick. Boot time with that Asus board is desperately long. Waiting 15 to 20 seconds from power up to boot loader is a shame when knowing it takes less than 1 second on a 300$ laptop! But the laptop lacks storage expandability I need, so I am always stuck on one end or another.

Then comes the fun part. I am pretty surprised there is no easier ways to restore GRUB than the following. I read about boot-repair, but it is just missing, probably yet another PPA to copy/paste and install. Anyway, I ended up getting it to work.

First I found the partition where Ubuntu was installed, /dev/sda5, and mounted it: sudo mkdir /media/ubuntu && sudo mount -t ext4 /dev/sda5 /media/ubuntu. I did the same with my ESP: sudo mkdir /media/efi && sudo mount -t vfat /dev/sda1 /media/efi.

Second step was to establish bindings:

sudo mount –rbind /dev /media/ubuntu/dev
sudo mount –rbind /proc /media/ubuntu/proc
sudo mount –rbind /sys /media/ubuntu/sys
sudo mount –rbind /media/efi /media/ubuntu/boot/efi

That caused some directories inside my Ubuntu mount to mirror exactly the top level directories.

Then I had to chroot into my Ubuntu, using

sudo chroot /media/ubuntu

After all this, system was behaving a bit the same way as if I started a shell on my Ubuntu setup. From this, I tried

sudo upgrade-grub2

That just updated GRUB’s entries, not the EFI one, so didn’t fix the boot.

Then I tried

sudo grub-install

If I remember well, no arguments were necessary, and that fixed my GRUB EFI and added back the Ubuntu entry to NVRAM. This worked only after /boot/efi was correctly referring to my ESP. Note however that for this to work fully, the USB live Ubuntu had to be booted in UEFI mode, not MBR default mode.

A reboot later, I was starting my Ubuntu setup, fully intact and working! Half of the transfer done! Not quite…

Windows was failing to boot and Ubuntu’s update-grub wasn’t detecting Windows anymore. Quite bad.

Windows: desperately dead

Windows, on the other hand, wasn’t booting at all. It was showing a blue screen suggesting me to use the repair tools from Windows DVD. Last time I did this, the tools ran for at least one minute and bailed out, so I had to do a complete refresh which ended up wiping everything and leaving only applications from the Windows store. If I have to choose between such a messed-up repair and a clean install, I would bet for the second option.

Before entering into this reinstall nightmare once again, I tried to recover the reserved partition. For this, I plugged back my Windows 120Gb SSD and booted from my live USB stick to make sure Windows would not kick in and see two copies of itself (one on the old, one on the new SSD). If Windows sees two copies of itself, it changes the disk ID of one copy. If the new drive is changed, everything is messed up and Windows cannot boot anymore, until a refresh is done (and then everything is messed up again!). Back to my live USB, I used DD to transfer the bytes of the old reserved partition to the new one. I also made sure the new /dev/sda2 reserved partition was marked as such in GParted, by modifying the flags. That changed nothing.

The post How to repair the EFI Bootloader in Windows 8 literally saved me hours of work! This gives a procedure that allows to fix the boot loader. The main idea is to log into console from Windows DVD and run bootrec /fixboot command from directory EFI\Microsoft\Boot\ of the ESP, followed by bcdboot  with a couple of arguments, again from the ESP. Luckily, I had my ultrabook, which was quite handy to check the page while I was running the commands on my primary PC.

That solved the issue and allowed me to boot into Windows 8.1! PHEW! Quite a nice step forward.

GRUB not detecting Windows

Now that my machine was able to boot into both Windows and Linux, one could wonder what was missing. Well, I had no easy way to choose which operating system to boot at startup. Originally, GRUB was offering me an option to boot into Windows or Ubuntu. After the transfer, it was only seeing Ubuntu.

I found procedures to manually add an entry for Windows but that involved finding and copy/pasting drive UUID and probably redoing the change on each kernel update. I didn’t want that. Another possibility was to install an alternative EFI boot loader like rEFInd, but these have tendency to display many unwanted icons doing nothing. I got enough trouble with this while fiddling with triple boot (Windows, Linux, Mac OS X).

There was absolutely no way out. People were doing the manual addition of Windows or that was working out of the box. I had to spend more than 45 minutes inspecting the os-prober script and walking through it! By looking at the script and its logs in /var/log/syslog, I manage to find out it was skipping my ESP because the partition was not flagged as Boot! I fixed that from GParted, reran sudo update-grub and tada! GRUB was seeing Windows!

This is NOT the end!

Then I had to proceed with the hardware installation of the new drive. Since I was too impatient to get a SSD, I ended up with an ill-designed system. If I had waited another year before purchasing my Core i7 PC, I would have got a superb case with support for SSD drives. Now I have a CoolerMaster kind of case with only standard 3.5″ drive bays and need to fiddle with SSD brackets. Screwing the SSD drive in this is a painful process of trial and error. Then the assembly doesn’t fit well with the screwless mechanism of the case. This somewhat holds in place, but that’s not smooth installation like a regular 3.5″ drive.

Some more fiddling later, my new SSD was plugged back into my PSU and motherboard, and I got rid of the extra two SATA cables. I stored them away; they will be useful sooner than later, because my two 120Gb SSD won’t remain unused.

I plan to put one of them into my HTPC, which will be another adventure of its own. My HTPC has only four SATA ports, all used up, so I will have to get rid of one hard drive.