Long-Term Backup Verification: Beyond Compare, in Windows and Ubuntu

Some time back, I had looked into software that would verify that I was not losing data without realizing it. Data could disappear, as I have discovered, when files become corrupted but continue to look the same (until you try to open them). Data could also disappear if files quietly vanish through unnoticed mistakes (e.g., hitting Delete when an archival folder is highlighted). I had a backup system, in other words, but I lacked a way of checking whether anything might be falling through the cracks. The programs I had examined in my previous investigation had not turned out to be quite what I was looking for, so I still had this need.

Then I became aware of Beyond Compare from Scooter Software. BC had gotten a lot of very positive reviews from programmers and other users here and there. It came with a 30-day free trial offer, after which it would cost me $30. Amid praises that sometimes seemed to come from BC's own friends and/or employees, there were also references to Araxis Merge, which some considered much superior. After expiration of the trial period, it was available for $169/259 (standard/professional), but there was supposedly an academic discount of about 70%. Araxis Merge didn't offer a Linux version. There were also a number of other file and folder comparison tools, some of which were free but few of which offered CRC checksum calculation, which I wanted. I decided to start with BC and see how that went.

First Try: Ubuntu Installation

There were versions of BC for Windows and for Linux. In the spirit of my gradual, long-term effort to move away from Windows, I decided to start by trying the Linux version of the program. I was running 64-bit Ubuntu 9.04 (Jaunty Jackalope). Beyond Compare was a 32-bit program.

The 32-bit version may have been very easy to install. But it seemed I would have to make some adjustments in order to run this program on my 64-bit system. I found two different sets of advice on how to make those adjustments. One was for running BC on 32-bit Kubuntu 8.04. I figured it would probably work, if I wanted to try it. But the other was for Ubuntu 9.04, so I decided to try that one.

Following the latter set of instructions, I downloaded the .tar.gz version of BC. Ordinarily, it seems, it would have been necessary to download ia32-libs and libqt3-mt; but in my way of installing Ubuntu, Synaptic showed that these packages were already installed. I unzipped (technically, I guess, I should say untarred) the file by using the "tar -zxf [filename]" format instead of the "tar -vxf [filename]" format that I had previously decided I should use. (I was definitely still in a learning mode for purposes of Ubuntu commands.) Just out of curiosity, I deleted the resulting folder, went into a WinXP VM, right-clicked on the .tar.gz file, and told 7Zip (one of my Windows XP utilities, with a right-click context menu option) to unzip it. It did. Once again, I had that folder containing a file called bcompare-3.1.4.10554.tar. So if, like me, you weren't smart enough to just right-click on the tar.gz file and select Open with Archive Manager > Extract, you could do it this other way. Actually, in this case, the Windows approach may have been superior, because when I did belatedly try the Archive Manager approach, I got "An error occurred while extracting files." So I went back and did it with Windows again after all. This gave me a much larger .tar archive. I used 7zip in WinXP again on this .tar file. Now I had a regular folder called bcompare-3.1.4.10554.

I decided to continue trying the GUI approach. In Ubuntu's File Browser, I went into the unzipped folder and double-clicked on install.sh. I got a dialog asking, "Do you want to run 'install.sh', or display its contents?" I chose Run. Nothing seemed to happen. I went back to Terminal, navigated into that folder, and (reverting to the instructions) typed "sudo ./install sh". Now it seemed to install. At the end of its various messages, it said, "Please place the following in your .bashrc or .cshrc (etc.): export PATH=/home/ray/bin:$PATH," where "ray" was my username. It also said, "Executable is /home/ray/bin/bcompare." It was apparently telling me that I had to add /home/ray/bin to my computer's path, so that the program would know where to look when I typed "bcompare" (or whatever) to start the program.

I tried just typing "bcompare" right where I was in Terminal, but no joy. So I navigated over to where it said it had installed itself: "cd /home/ray/bin." Sure enough, there was a file called "bcompare." But when I typed "bcompare" there, I just got "command not found." Double-clicking on bcompare didn't do anything either. The installation instructions cited above said nothing about this. Was I supposed to make it executable? I typed "chmod +x bcompare" and then typed "bcompare" again, but this still just gave me "command not found."

It seemed I would have to figure out how to add something to my path, though I didn't understand what good that would be, if the damn thing wasn't executable. I found instructions that seemed to work, or at least they got me to an open .bashrc file. I didn't find any path lines in that file to use as a model, so I gathered that I was just supposed to type exactly what they said. I added it at the end of the .bashrc file as follows:

# add Beyond Compare to path [this is a non-executing comment]

export PATH=/home/ray/bin:$PATH

I saved and closed .bashrc. I opened a new Terminal session and typed "bcompare" at the $ prompt. It said "bcompare: command not found." I navigated to /home/ray/bin and tried again. Same result. I decided to back up and try the approach from that other webpage, the one that was supposed to work in Kubuntu 8.04 with the .deb download. Given ia32-libs and libqt3-mt (above), it now appeared that I had assumed that the installation of these libraries meant that the files I needed were installed in the right places -- that, in other words, Synaptic had already taken care of putting those lib files in their proper places. But now it seemed that I should have copied over program files from one folder to another. Putting this Kubuntu approach on hold, I reverted to the instructions from the approach I had already worked through. Specifically, the first command I apparently needed to enter was:

dpkg-deb --extract libqt3-mt_3.3.8-b-5ubuntu1_i386.deb libqt3-mt

changing the specific libqt3-mt file name as needed. But at this point, not knowing what particular file that might be, I said to hell with it and downloaded the Windows version instead. I then looked for a way to uninstall this version of Beyond Compare from my Ubuntu installation; but since I had not used Synaptic to install it, I did not seem to be finding uninstallation instructions that applied. So it's still installed.

Second Try: Windows XP Installation

I downloaded and ran the WinXP installer. Interestingly, they had an option to create a "portable install," which could apparently be put on a removable USB drive or wherever, without making any changes to the registry. Presumably it would still fail to work after the 30-day trial period unless I bought a license. But if I was going to keep the program, this would definitely be a useful form for it. So I went with that approach. (One special advantage of this approach, for my purposes, was that I could put it on a drive other than drive C, within my computer, and could therefore make it available to all of my virtual machines under VMware Workstation, without having to reinstall it on each VM.)

The program seemed good. I was impressed with the comparisons. When I tried to use its help feature, I got an error message, "Navigation to the webpage was canceled." I searched the database and didn't find anything, so I sent Scooter Software an e-mail about that.

The folder comparison feature was very smooth. Folders were color-coded according to whether they matched or not. Black folders matched -- that is, they were identical. Other colors seemed to indicate some degree of mismatch. Lacking the help feature, I looked for a user's manual on the website. All they had was a bunch of knowledgebase articles. I'm sure these were very helpful for some purposes, but their titles revealed none on the subject of folder contents or colors. Nonetheless, when I clicked on a purple folder, I saw that only one of its subfolders differed. Eventually, I found that I could actually configure my own preferred colors for the following statuses: same (i.e., the folders in the two comparison panes are the same), orphan, older, newer, and different. There were also color options for file comparisons.

Cool feature: when you click on a folder on one side of the comparison screen, the problem automatically opens the parallel folder on the other side. In other words, I'm looking into a subfolder on drive F, and it's opening up that folder for me; but at the same time it's also opening the comparison subfolder on drive H. With just this much knowledge, within about a minute after starting to fool with the program for the first time, I was able to detect that there was one varying file, nested seven layers down, in a folder containing almost 30,000 files. As far as I could tell, the difference was that the filename for the one had been truncated.

One thing that I didn't find was the ability to reload an old comparison log and compare it against a new drive. For example, suppose that, on December 31, 2007, I burn a CD to archive some files. I compare that CD against the source folder on the hard drive. Everything looks good. Then, sometime during 2008, something happens and it appears that I may have lost some stuff from the hard drive. What did I lose? I don't know, because the hard drive has changed by now, and unfortunately I can't find the CD. Ah, but if I could run a comparison of the hard drive's current state against the previously saved log comparing the hard drive and the CD on December 31, 2007, at least I could know what might be missing and take appropriate steps to replace or compensate for it. Another scenario: I back up my data every year, and now I want to see a single list of all the changes in my folders since 2003.

About this time, I realized that I had not actually examined the support forums at Scooter Software. As it turned out, there were several hundred threads in those forums. Some of them appeared to be exchanges between the proprietor of Scooter Software and his chief programmer, but whatever; it was still good to see the effort and interest in the product.

There was a lot more that I wanted to try with Beyond Compare, but I was not quite set up for some of that, so this is where the matter stopped for the time being.