Unsolvable and annoying VirtualBox slowdown

Since end of December 2016, I was using my personal computer to work from home. All my development environment was living in a VirtualBox CentOS virtual machine. That was working very nicely until March 13th. All of a sudden, the virtual machine became annoying slow. Starting the virtual machine took twice the time as before and everything was taking several seconds to start. It became impossible to start Thunderbird and connect to my corporate email through IMAP.

I tried to restart the virtual machine, reinstall VirtualBox, fiddle with the parameters of the virtual machine, nothing worked! The only workaround that helped a bit and at least allowed me to work was to increase the number of virtual CPUs from 2 to 4. However, even with that, when I was switching windows with Alt-Tab, it was taking several seconds for the window selector to pop up. Typing on the terminal was sluggish, sometimes as if I was connected through SSH on a slow server! But my machine was running locally! Running the unit tests of the main component I was developing used to take 4 minutes and a half on that machine. This bumped to more than 5 minutes, and even 6 minutes last week.

None of the suggestions on forums and nothing I thought about could solve this. I set the power plan to maximum performance so the CPU wasn’t throttled down. I fiddled with virtualization and paravirtualization options, disabled 3D acceleration, enabled it again, absolutely no change. Even the GRUB menu was taking a few seconds to display rather than appearing instantly as before. Maybe this is a CPU issue? I checked the CPU temperature using Speccy: everything seemed normal. VirtualBox was the mainly affected program, the rest was working fine.

This really seemed like a nasty Windows 10 update that screwed everything up. I could attempt a system restore or I could recover the last CloneZilla image I had, but if that was a Windows 10 update, the update would reinstall and thinks would screw up again. I would thus need to downgrade to Windows 7.

The issue persisted more than a month. Sometimes, I tried stuff without success: recreating the virtual machine, copying the VDI files somewhere and putting them back on my SSD (in case of filesystem corruption), checking my NVIDIA settings because the slowdown was mainly affected GUI rendering, etc.

The day before the first slowdown, two things happened: I ran CCleaner and Avast got updated. I tried to restore the registry backup CCleaner made before it cleaned registry: that failed once again. Windows couldn’t restore the registry backup. I tried to disable Avast temporarily: no effect. The hypothesis of a flaky Windows update was the main one, but no update were coming to fix it. Maybe they won’t fix it, because they offer Hyper-V, and they would love people switching from VirtualBox to Hyper-V. But that would involve reinstalling CentOS on a new virtual machine and recreating all my development environment. I could try VMWare Player instead, I think it can migrate a VirtualBox machine, but nothing told me it wouldn’t be affected as well by this issue.

Monday morning, April 17th, I tried all what I could to fix this. I was ready to attempt the system restore if needed. However, it was too late for a simple restore through Windows: I would need to use my CloneZilla image and take the risk Dropbox wouldn’t be smart enough to detect there is an old directory to update and not replace the contents in the cloud with the restored old directory. In case Dropbox screwed up, I backed up my Dropbox folder. I also backed up my virtual machines that I would need to restore after the Clonezilla recovery.

Before doing that, I tried to uninstall CCleaner: no success. I tried to remove NVIDIA Experience: no success. Two days ago, I removed the 3D vision drivers without effect. Then I removed Avast. At this point, I had little hope. I was ready to plug in the external hard drive with the Windows 10 image on it. But before, I rebooted, tried the virtual machine one more time and then, finally, it worked! It wasn’t slow anymore!

I pushed as far as starting my Ubuntu virtual machine and upgrading it from 16.10 to 17.04. This went well, without any issue. I’m writing this post from Ubuntu 17.04, running inside my VirtualBox environment which is finally fixed.

It is not the first time VirtualBox is hindered by an anti-virus program. It happened at work with Symantec Endpoint Connection. I had to downgrade to an old VirtualBox for a few releases until that finally got fixed. VirtualBox or Avast will have to fix something for this system to work again. For now, I am using the anti-virus built into Windows 10. I don’t know if I will retry Avast, reinstall AVG or finally give up and pay for Symantec’s Anti-virus or Kaspersky.


KeePass and user interface problems

This morning, I tried to install Ableton‘s Live on my Mac and got an endless sequence of issues. First, I needed to log in to my Ableton Live account in order to get the Mac OS X binaries. Unfortunately, the machine-generated password for that account is stored in a KeePass keyring, so I had to install KeePass 2 on my Mac to get access to the keyring and find the password. In order to install KeePass 2, I needed to install Mono.

After I installed Mono, I was able to unpack the KeePass ZIP and got an executable that I copied in my Applications folder. However, KeePass never showed up in the Launchpad; I had to open up the Finder, browse to Applications, then double-click on the KeePass icon there. Startup of KeePass took from thirty seconds to five minutes. After that, the menus were unresponsive, taking from five to ten seconds to open up the File menu, from five to ten seconds to find and expand the Open item, then was able to open my password file. The upper KeePass menu was unresponsive; I had to use menus from within the KeePass window, which not the Mac way. Keyboard shortcuts like Command-Q were failing. But at least I got the GUI working.

However, no matter how many times I tried, my master password started to suddenly fail. I ended up making the password visible and found out that everything was typed in upper case. CAPS LOCK AGAIN! This is so frustrating, so annoying, that this Razer keyboard is causing me to make repeat mistakes, inserting all sorts of junk through anything I write, then the CAPS LOCK that triggers all the times.  I had less issues with my Logitech non-mechanical keyboard, which is by itself an incredible non-sense for me. A mechanical keyboard should work better, be more reliable, but it is almost ending up to be the opposite for me! I just cannot give up and put back the old Logitech keyboard.

However, sorting out the CAPS LOCK issue wasn’t sufficient. Master password was failing AGAIN! I then noticed that EVERYTHING I was typing was upper-cased! I tried to repeatedly press caps lock, I tried with the Mac’s internal keyboard rather than external one, it was even WORSE! The internal keyboard is now spitting out numbers when I press letters and most keys do nothing! This is a total non-sense for me. I tried to switch keyboard layout with Windows+Space, without success. I ended up trying to log off, but I couldn’t log back in: the system was entering upper-case letters even for password input. I thus had to reboot, then that came back to normal.

After all these hurdles, I was able to connect to my KeePass keystore. However, when I right-click on the entry for Ableton’s Live, NOTHING happens. I tried right-clicking multiple times without any success. If I try enough times, KeePass shuts itself down at me.

I then tried with a command-line KeePass tool, KPCLI. However, it was totally impossible to install it. The tool requires an endless number of Perl dependencies. The KPCLI website indicates nothing about how to get these dependencies. Following another web site, I tried to install them using CPAN, but despite the effort, I am still getting error messages about missing Moveover, the above blog post refers to a broken Gettext that needs to be manually fixed, again! Moreover, KeePass home page refers to a bug in Mono that needs to be worked around.

Bottom line, most open-source libraries are bugged and needs manual recompilation/patching! I also experienced this issue when trying to build Extempore a few days before. This is becoming a quite concerning trend. I cannot believe the libraries are all low-quality pieces of software. I rather think developers keep patching the libraries rather than trying to use them as they are and library owners cannot keep up pulling and integrating the patches. Even if the library owners integrate patches, the integration sometimes isn’t what the patcher exactly intended, so the code relying on the original patch needs to be adapted for the revised patch. This trend is concerning, because if it continues like that, we will end up to need a virtual machine for each and every open-source application we install. Docker can be of some help there, but it doesn’t work so well for applications with graphical user interfaces. Docker is nevertheless a good tool for running services in isolation, with each service having its own funky dependencies.

For my Perl/KPCLI issue, I probably need to recompile/reinstall Perl itself, because I am using a too old Mac OS X version! Homebrew can be used to perform such tasks, but often, it starts failing, because the installation recipes are not correct and requires tweaking for the local Mac OS X! This is non-sense, all Mac’s should be equal, but this is not the case as far as I can see. Only newer 2015 Mac’s will work, so this is pointless!

For example, I tried to use Homebrew to install Extempore. The program ran for almost one hour just to compile pcre, then it tried to install a patched LLVM package. Download happened, but install failed, because the SHA1 key mismatched. The download TAR archive can be checked with tar -ztf without issues, so it doesn’t seem corrupt, but SHA1 mismatches, so I would need to copy/paste blobs of text from a web site just to tweak the Homebrew recipes, which are non-senselessly named “formulas”. Why use a non-standard terminology at first? This is not a “formula”, we are not dealing with chemistry, mathematics, even less with magic! This should be called a “package”. Anyway, it doesn’t work, so it doesn’t really matter.

Maybe I would have more luck with a Mac-friendlier password manager such as 1Password (don’t know, never tried), but that would require me to copy/paste all my KeePass passwords into the new tool, one entry at a time, and more than that, again, 1Password is just a trial! I don’t want a temporary solution that works just for 15 or 30 days, unless I toss my full name and address with credit card to somebody that can as well sell it to non-profit organizations that will then start calling me weekly for donations. I am already starting to have this issue, so just don’t want to make things worse. I want something that will solve password management once and for all! In addition, 1Password works just for Windows and Mac, so again, Linux is left out.

The only thing I could do is to download the binaries from my main computer and copy them over to my Mac, or reset my Ableton account’s password to something I can remember and just type it, as before I discovered KeePass. I just gave up because this is too much work and just too slow. Live is likely just to fail starting on this old MacCrapPro.

These are not the only issues I ran into this morning. Firefox now takes thirty seconds to one minute to start. Scrolling with the mouse wheel is unreliable. I have to give the mouse wheel a big swing for the scrolling just to start, then it is not smooth, rather choppy. Even scrolling with the arrow keys is choppy.

I just cannot get used to the funky Mac keyboard shortcuts. For me, switching applications is done with Alt-Tab, not Windows-Tab, but the Mac way is Windows-Tab. Yes, technically, it is Command-Tab, but that’s not what I have on my keyboard, and the internal keyboard just doesn’t work, missing critical keys such as Option and Fn. I always end up pressing Alt-Tab, then Windows-Tab. I always end up pressing Ctrl-K to put the search area in focus, then have to press Windows-K because Ctrl-K fails. Sometimes Windows-W will not close Firefox tab while Ctrl-W does. Sometimes, Ctrl-W won’t close the tab while Windows-W does. Sometimes, Windows-Q will close currently running applications, sometimes not, then I have to switch to the mouse, trying to find out the really small Red circle and click, oups the mouse just slipped while clicking because the screen elements are too small, try again, works. Even the builtin terminal suffers from this issue: Ctrl-+ to zoom in, oups, Windows-+! Ok, home to move at the beginning of current line, oups that moves at beginning of the whole command line history and nothing can go back except the mouse again! Moving at beginning of the line is done with Ctrl-A, not Windows-A. All is about inconsistency, making the keyboard almost useless.

Bottom line, each and every operation requires multiple attempts, making me nervous and mad each time I am using the Mac.


Desperately slow

This morning, I tried to install Razer Synapse on my Mac now equipped with Mac OS X 10.7 (Lion). That worked very well but brought absolutely NO improvement with respect to mouse smoothness. In fact, it does absolute nothing: no compensation or removal of the unwanted acceleration, and no handling at all of the side buttons. That program seems to only manage keyboard macros, completely ignoring the mouse. I thus had to reinstall SmoothMouse.

Then I started to have issues with the mouse wheel. Sometimes it wasn’t working at all, sometimes it was scrolling in the wrong direction, sometimes it seemed correct after a couple of attempts. I found an option in Mouse preferences to invert direction of scrolling and unchecked that. That may partly explain the erratic behavior.

The biggest disappointment came from performance. Boot up time is more than a minute and a half and no video playback of any kind is possible. I tried YouTube with Flash, with HTML5: choppy video with correct sound. I tried to play MPEG4 videos with QuickTime: choppy video with correct sound. It seems I would have to convert all my videos to QuickTime, probably with reduced resolution. This is kind of pointless and very time consuming. In fact, QuickTime won’t play AVI files as well: again, transcoding is required. That can of course be worked around with any good video player such as VLC, MPlayer, maybe even Kodi/XBMC, but then I slowly but surely come up to the conclusion that ALL Apple applications need to be supplemented or replaced with a cross-platform alternative that can also work well, even better, on a non-Mac environment!

Final Cut Express, while claimed in the documentation to support many input video formats such as MPEG4 and AVI, happens to support only QuickTime .mov files. As far as I read, QuickTime is a container: a .mov file can contain different kinds of video and audio streams. In practice, there is no tools capable of repackaging a MPEG4 or AVI file into a QuickTime movie: tools will slowly and wrongly reencode video, recompress audio, into some kind of probably proprietary, unknown, undocumented, Apple-specific format. Maybe only trial versions of commercial tools can do the repackaging, if at all possible.

Final Cut Express miserably failed at importing MPEG4 files from my Android phone, bailing out with useless error message. I tried with AVI files from video captures with Bandicam with a bit more success: Final Cut Express imported the clip, was able to preview sound but displayed no video.

Sometimes, more and more often, system response became unacceptable. The mouse started to be choppy, literally jumping from one place to another. Sometimes, the movement was smooth, sometimes the mouse would jump ahead, sometimes the mouse would stop moving. When that happened, I had to shut down applications, which took up to THIRTY seconds!

After that, keyboard stopped working properly. This started with CTRL-F2 not opening up the menu bar. I had to change the binding for CTRL-F10, for no obvious reason. That worked an hour or two, then ALL F1-F12 keys on my Razer external keyboard stopped working, no reason, reboot required to fix! Probably the USB stack has some quirks to flaw non-Apple devices but make people believe they CAN work. So I would have to pay 50$ for an Apple keyboard, 70$ for an Apple USB trackpad or mouse, and find some place to install all this or move all my stuff around when switching from my PC to my Mac.

Conclusion: this machine is almost unusable. I REALLY would have to downgrade to Snow Leopard, probably even better to Leopard, to do anything with this, and even then, what’s the point of using Final Cut Express if I need to preprocess all my input files into QuickTime, loosing quality during the lossy process, and then let Final Cut Express scramble the video information once more during its rendering phase?


The obsolescence road block

Last weekend, I got to a first dead end: Mac OS X 10.5, installed on my Mac, being deprecated and not accepting any new application, including Firefox, Google Chrome, SmoothMouse, the Razer Synapse driver and probably many others. Besides the difficulties installing applications, I had issues with YouTube playback in both Safari and Chrome (before I broke it). Playback was choppy, especially when going full screen.

I ended up ordering the Snow Leopard DVD from online Apple Store. That worked better than I expected. I got it on Wednesday. UPS left it at my door, so even if I wasn’t at home, I wouldn’t have to go pick it up somewhere I cannot reach or make a phone call to them to schedule another delivery.

I sticked that DVD in my Macbook on Friday, December 12. The installation process happened to be simpler than I was expecting. Inserting the disk poppped up a wizard proposing me to upgrade to Snow Leopard, directly from Leopard, no need to boot from the DVD. I did that, but the upgrade took more than 45 minutes, longer than any Ubuntu or Windows installation ever, at least for me. Then I had to download and install the combined upgrade that bumped the system up from 10.6.3 to 10.6.8, then another set of updates for iTunes, Java and security.

After this, I was able to access the App Store and connect to it using my Apple ID. When I did it, I felt I was binding this machine to me, this was becoming a bit more mine.

The day after, I was able to toss in Firefox, latest and greatest version. I installed Razer Synapse, but that failed to start. I had to uninstall and reinstall to get rid of the partial non-working installation done in Leopard, but that finally worked and got rid of this damned mouse acceleration that was starting to drive me mad and creating pains in my wrist. However, font size is still too small, making the use of this machine prone to awful pains in the neck. If I don’t have to schedule an appointment with my physiotherapist before the end of this nightmare, I will need to spend a significant amount of time thanking God.

However, as soon as I started to feel this great, things went south once more. The machine became incredibly slow, starting to load from the hard drive almost endlessly. Clicking on the menu bar, including the Apple menu, was freezing the machine for a few seconds. Switching tabs in Firefox was taking a couple of seconds. And the hard drive, almost put to the torture, was never never stopping to spin like crazy.

Yesterday evening, I decided this was enough and worth trying a clean fresh install. For this, I inserted the Snow Leopard DVD once more, hooked up a USB keyboard and turned off the machine. Then that’s when the fun starts. There IS a somewhat reliable way to reach the boot manager. You need to wait for BOTH the chime and a sign of activity on the external keyboard. So I powered the Mac on, and stared at my keyboard’s lower part. As soon as I heard the chime and the Razer logo lit up, I pressed the Alt key. Pressing the key before will not send the Alt KeyPress signal to the Mac, so it will NEVER know I pressed Alt! And pressing too late gives OSX the time to boot off from the hard drive. It also seems that sticking a disk in the DVD drive, any disk, will slow down the boot process enough to leave more time for the USB keyboard¬† to power up. This time, that worked, so I had the menu allowing to choose between the hard drive and the DVD drive; I selected the DVD drive and pressed Enter.

It took more than three minutes, if not five, to boot up from that DVD. After that, I was in the Mac OS X installer.¬† From there, I launched the Disk Utility, wiped the hard drive, exited the Disk Utility, and started the installation. This took more than half an hour, I don’t know exactly how much time. After that, I tossed the combined upgrade in once more, then the other updates. I discovered with desperation that Razer Synapse was only available for 10.7. Did this changed suddenly or did Google lead me to a web site that just offered me the download? So I was maybe installing something for 10.7 on 10.6 or 10.5. I thus tried with SmoothMouse instead, and that worked great. Mouse acceleration stopped driving me crazy, and I knew it would work for ANY mouse, not just Razer ones.

The formatted machine was faster, no doubt. It wasn’t freezing when clicking on menu items, Firefox was usable, but things went bad once again when I went to The hard drive started to spin like crazy, scrolling became choppy and playback was almost impossible, even without going full screen. I tried with the HTML5 video player instead of the Flash player; that helped a bit.

Then I got issues with LibreOffice which was working somewhat correctly at start, but froze completely when I was searching for a way to automatically format dollar amounts in Calc. Right-clicking on its icon in the Dock had no effect. I tried to access the menu: no response. The menu was locked up as well. I would thus have to switch to a different machine, search on the Web and find the ackward impossible to remember combination of keys to shut off a program forcibly. Tired of being blocked by all possible applications, even on a freshly installed OS, I yanked the plug and put that damn machine away.

This morning, I sent a new message on the Apple Support forum about the issue. What I got just confused and discouraged me quite a lot. Somebody replied to me that this October 2006 MacBook Pro wouldn’t accept more than 2Gb of RAM and would not upgrade past 10..6.8, and that would be pushing to limits. Ideally, it seems I should order yet another DVD, this time to downgrade to 10.5 and have no possibility of installing any application. This is as frustrating as if I got a 2006 laptop, would install Ubuntu 6.06 on it (the most recent at that time) and would end up not being able to upgrade past this!

According to MacTracker, the machine would accept up to 4Gb, although only 3Gb are addressable. It would also accept Mac OS X 10.7. Whether it would work well with 10.7, even with 3Gb of RAM, remains to be determined. Moreover, I’m not sure at all the MacBook Pro would happily work with the SO-DIMM chip I would find for it. What if I cannot get 667MHz DDR2 SO-DIMM? Would the MacBook Pro and/or Mac OS X live with a 800MHz DDR2 SO-DIMM, of course using it as a 667MHz one? I have no way to know, because Internet is not reliable anymore, sellers in local sellers all tell me they don’t support Mac and Apple is just helpless unless I schedule an appointment with a tech during work hours, bring up that machine to an Apple Store I will have trouble finding and pay an undetermined amount of money, possibly to be told one week later, maybe even after the upcoming Christmas holiday, that it is worthless to do anything with that old laptop.

I am also suspecting issues with the graphic card, because from my extremely bad YouTube experience, even on OSX 10.5, it seems there is no graphic acceleration at all. Maybe the video memory is flawed and the OS/graphic hardware works around in some clunky way.

The best way to know about this would be to boot from a Ubuntu Live DVD or USB. Ubuntu will not apply such workaround, so if the graphic hardware is faulty, it will crash, freeze, hang, etc. If Ubunt works correctly, that means that Mac OS X took the same bad route as Windows, becoming slower and slower with releases. A better CPU will just compensate bad programming, nothing more.

Same issue happened when switching from Windows 3.1 to Windows 95, from Windows 98 to Windows XP, etc. Ubuntu is less prone to this, because if the system becomes too slow for the machine, switching to a lighter UI will usually help (e.g., using LXDE instead of GNOME or Unity). Windows and Mac OS X not having this modularity, users are stuck.

With a UNIX base, Apple could have done differently from Microsoft, but they just misused that UNIX base, simply grabbing a BSD kernel to make publicity about illusion of openness.