A Backup Arrangement with Beyond Compare

I had been using Beyond Compare (BC) for a year or two.  Over that period, I had settled into what seemed like a decent backup arrangement.  This post describes that arrangement.

For a while, I had a spare internal partition to which I would make backups.  The original concept there was that I would use rsync or some other program to make backups on an hourly basis.  That setup had fallen into disrepair, mostly because I didn't quite like how it was working.  So I didn't have an hourly backup at this point.  The arrangement described here is more on the longer-term (e.g., daily, weekly, monthly) level.

My backup took place on external drives.  I had an external enclosure that I would have to open up (removing several screws) to swap drives, and I also had an inexpensive dock that I could just plug an internal SATA drive into.  Both seemed to work equally well.  The enclosure was handy for unplugging the drive and taking it with me.  Now and then -- especially when the tornado alarms sounded -- I visualized myself grabbing it and running for the basement.  I wondered if that would be one of those fateful delays that would cost me my life.

The external enclosure had an eSATA connector, but my previous motherboard had not been able to accommodate eSATA on a hot-swappable basis.  In other words, I had to reboot in order to get the system to recognize it.  It also had a USB connector.  The external dock (i.e., not the enclosure) was also a USB device.  USB was slower but very adaptable.  That was almost always what I used.  Some partitions on the external drive were compressed, to save space.  I had the impression that this did not help with the USB connection -- that the CPU would unpack the file before shipping it across the slow USB cable to the computer, resulting in at least as much data moving along the wire -- but I hadn't verified that.

For my purposes, Beyond Compare offered two key concepts.  First was the workspace.  If I plugged in the external drive that I used for daily backups, then I would open up the DAILY workspace in BC.  If I plugged in a drive that I used for weekly backups, then I would choose the WEEKLY workspace.  I also had a SIMPLE COMPARE workspace that I would use for random tasks -- say, comparing two folders on a one-shot basis.  And I had a NETBOOK workspace that I would use to synchronize my laptop.  GoodSync might have been better for that if I had been using the laptop frequently, but at this point it was mostly a case of keeping the data on the laptop current with the desktop.  That is, I was mostly doing one-way updates, from desktop to laptop.

My workspaces differed in the tabs they made available.  In the DAILY workspace, I had a tab for each day of the week, plus whatever other comparisons I would want BC to make on a daily basis.  Likewise for the WEEKLY and the other workspaces.  In other words, I used a workspace as a place where I would be able to see tabs for each comparison that I wanted BC to make, whenever I plugged in the weekly drive or the laptop or whatever.

I found that the best approach was to start BC first, let the workspace load, and only then turn on or connect the external USB drive.  That way, BC would not try to do complete comparisons for all of the open tabs.  It would do its calculations for the relevant folders on the drives inside the computer, which were already available to it, but on the external drive it would have to wait until I gave it the go-ahead within a particular tab.

Focusing on the DAILY workspace, I was writing these notes on a Friday.  So to guide my remarks, I opened BC at this point.  Somehow, I had arranged for the DAILY workspace to come up by default; or maybe BC just defaulted to the last open workspace.  I wasn't sure how I had arranged that.  When BC was up and running, I turned on the USB drive.  It took that drive a moment to become available.  (I found that AntiRun was useful, not only for protecting my system from autorun malware and such, but also for telling me when a drive really was online or offline, and for giving me a functional way of taking external drives offline.)

I went to the Friday tab.  BC had stalled because the Friday folder on the external drive had been unavailable.  I told it to retry; and now that the USB drive was connected, BC ran its comparison.  (Details on the kinds of comparisons available, and other program capabilities, are available at Beyond Compare's website.  Their forums and other tech support had been very responsive, the few times I had contacted them.)

I had modified my BC toolbar to present the red Mirror button.  This said, basically, just overwrite whatever is in the backup space (in this case, the Friday folder on the USB drive) with whatever is on drive D in the computer.  Drive D was the one that I backed up daily.  So in this case, a number of files had changed since the previous Friday.  Sometimes I would take a look at them; sometimes not.  Usually not.  It seemed pretty rare that a file would be accidentally deleted.  Daily examination of all changing files had seemed to be overkill.

When I say that I would take a look, I mean that BC showed me two panes, one for each of the folders being compared.  To keep things organized, the left-hand pane was almost always the authoritative one.  The left-hand pane would correspond, that is, to a partition inside the computer.  So I was looking at the right-hand pane, corresponding to the backup device.  If I saw a file listed in the right-hand pane, but not in the left-hand pane, that would mean that it was on the system when I made my backup a week ago, but now it was no longer on the system.  BC would also alert me, with a red font, if the file in the right-hand pane was newer.  Generally speaking, that wasn't supposed to happen.

I had an alternating weekly folder on this backup drive.  I used that one on Saturdays.  That's the one I examined more closely.  If I found that something was missing on Saturday, and I didn't think it should be missing, I could then click on the tabs for the other days of the week until I found the last backed-up version, and I could restore it from there.

Drive D contained the things that were in more active use.  I also had a separate partition, drive E, for things that took up a lot of space and didn't change very often.  Videos were the main example.  Because there were so few changes there, it was easier to look at the differences identified in BC, and verify that additions and deletions were desired.

In net terms, I liked this arrangement because it gave me some flexibility to combine automated and manual processes.  I wasn't vulnerable to one of those black-box backup solutions that would seem like they were working just fine until the moment of crisis, when I would painfully discover that I had failed to adjust some essential setting, or that the drive was malfunctioning, or whatever.

In this arrangement, if I was worried that files were missing, I could look down through lists of what was being added and deleted.  If I was confident that everything was fine, I could just click the Mirror button and the backup would happen.  I could also combine both approaches within a single tab, by telling BC to mirror only the selected folder(s).  This would gradually reduce the number of things remaining on the screen (assuming I had BC set, as usual, to Show Differences rather than Show All).  When confronted with what looked like a mess, I could thus eliminate the parts that seemed OK, and focus on the files and folders that didn't seem like they should have been getting added or deleted.

Like most other computer-related matters, my backup approach continued to evolve.  But as I say, I had been using BC for a while, at this point, and I was pretty much satisfied with the combination outlined here.

Synchronizing Two Computers: GoodSync and Alternatives

I had a home network with two computers, consistent with my evolving concept of how many computers a person should have.  (The third was a notebook.  The fourth was now distributed, in pieces, across the universe.)  I found it was convenient to have the second of those two computers take care of backup, construct reliable file indexes for searching, hold certain programs open, run certain batch files at regular intervals, and remain available for moments when the primary computer was doing maintenance or for some other reason was unavailable.

I was writing these words at one such moment.  The primary computer had just become dysfunctional.  And this was OK.  I was able to turn to the secondary computer and keep on working, pretty much where I had left off, because I was using a synchronization program that kept the two computers in sync via ethernet cable.

Critique of GoodSync

Except that, unfortunately, now was the moment when my synchronization program became flaky.  That program was GoodSync.  More than a year earlier, I had examined reviews of synchronization programs and had chosen GoodSync over the others.  I had purchased the pro version and worked through the details of setting it up.

During this year, I had found GoodSync largely suited to the task.  There were two principal problems, one of which prompted my search for a replacement.  That was the problem of licensing.  GoodSync's representative told me that the license allowed me only two installations per year.  The problem was that GoodSync could not be deactivated.  So, in my setup, I installed it on the secondary computer, and used it to synchronize my laptop and my primary computer with the secondary computer.  But if Windows became dysfunctional, or if my hard drive crashed, or for any other reason I had to reinstall GoodSync, that would count as one of my two permitted installations.

So I had already had one go-round with the GoodSync people about that.  Now there was another licensing issue.  They promised me that, since I had bought my copy within a certain timeframe, I would qualify for a free upgrade from version 8 to version 9.  But when the time came, the licensing did not work.  So at the moment when I was writing these words, I was looking at a notice that said this:

You have exceeded the Free version lmitations.  To continue using GoodSync you should upgrade to GoodSync Pro version or reduce your jobs to 3 or files in a job to 100.

But when I tried to activate, I got this:

FAILED:  Our records indicate that you have only GoodSync 8 licenses in the order you specified, but you are trying to activate GoodSync 9, which requires a separate license.

My options in this regard were to get on the phone and complain about the broken promise, go through the steps to uninstall version 9 and reinstall version 8, or buy an upgrade to version 9.  I did contact them and get it sorted out.  Possibly it was even my mistake in the first place.  I was not fresh on the details when writing these notes.  Point is, the matter might have been settled there, if it hadn't been for the second problem.

The second problem, which had been more persistently troublesome, was that GoodSync was a tremendous resource hog.  It would interrupt or slow down my work very noticeably, to the point that the computer would become unusable and even unresponsive.  I emailed them about this too.  Their reply was that I should upgrade to version 9.  I did that.  But that didn't fix the problem.  Version 9 did have an option, which may or may not have been present in version 8, to impose a "speed limit" on file copying.  This was obviously not an ideal solution:  it would slow things down when the computer was free, and it might not slow things down enough when the computer was busy.  This would be one more thing I would have to tinker with, in hopes of finding the right value.  Why not just have the kind of "smart" arrangement found in other programs, where it drops back to a slow jog or a standstill, depending on computer resource usage?

The slowdown problem was exacerbated by uncertainty as to how to get the damned thing to just stop when I was finally tired of waiting on it.  Right-clicking the system tray icon would give me options to Stop All or Disable Auto Run.  Disable Auto Run seemed to be only temporary.  A while later, I would be sitting at the computer in a stupor, waiting for it to do something, and would then find that it had sneakily started doing auto synchronizations again.  On the other hand, if I clicked Stop All, a long time could go by before I would think to check up on my supposedly automatic software.  And Stop All did not work reliably anyway.  More than once, I had to go into Task Manager (Ctrl-Alt-Del) and just kill GoodSync immediately.

Those were the big issues.  There were some others.  I didn't like that I couldn't get a clear indication of how long a job would be running.  When a program does the same thing 20 times a day, eventually it would seemingly be able to give a pretty good estimate of how long it would take to complete a synchronization.  I didn't like that GoodSync needed to distribute _gsdata_ folders all over my computer.  I had looked on that as the price of progress, but for some reason those folders sometimes contained copies of files, which could lead to confusing and potentially disastrous results when doing a sweep for unnecessary duplicates or searching for the latest copy of a file.  I didn't like that GoodSync was not able to keep computers synced on an up-to-the-moment basis.  I would have thought that, with the aid of those _gsdata_ folders, the program would have been able to know, pretty quickly, when a file had been changed, and it would ordinarily seem pretty straightforward to get a copy of that file to the other machine.  What I actually got, unfortunately, was a nearly constant hard drive workout on the secondary machine.

A Search for Alternatives

Since I was at this point of having to deal with license issues again, for the third or fourth time in the past 12 months, it occurred to me that it might make sense to see if I could find a better alternative.  I had pretty good impressions of recommendations from Gizmo's Freeware, so I started with their list of Best PC Freeware.  Their choice for best folder synchronization utility was PureSync.  Oddly, Gizmo's link to a review of PureSync actually led to a November 2011 Gizmo article that named a different program, FreeFileSync, as the best free synchronizer.  FreeFileSync didn't sound too good -- no autosync, among other things -- but the article did mention Allway Sync as a commercial alternative. 

A search for these three (i.e., GoodSync, PureSync, and Allway Sync) led to an alternativeTo webpage listing a number of possibilities.  Among those possibilities, I decided that a programmatic solution like rsync would not work for me, partly because I had not tended to devote the time needed to maintain command-line file management solutions, and partly because I wanted to be able to see what was being overwritten.  I decided against using Beyond Compare, which I used and liked for backup verification purposes, because it did not seem to provide a suitably automated solution.  The only other item that stood out, for me, among the alternativeTo entries was Microsoft SyncToy.  I decided to reserve judgment on it for now.  (For reasons of speed and reliability, I was looking solely at ethernet-capable tools.)

Seeking guidance, I looked back at the previous year's post, to see what comparison sources I had consulted then.  This led to the "sync software" category at TopTenReviews and the "data transfer & sync software" category at CNET.  MajorGeeks didn't have a sync category, and at Softpedia and elsewhere, sync software was mixed into a general category.  The top six at TopTenReviews were Syncables 360, GoodSync, SugarSync, Laplink PCsync, ViceVersa, and Allway Sync.  At CNET, filtering for Windows 7 programs, there were essentially two lists.  First were the "sponsored" (i.e., bought) reviews.  These were iffy:  ViceVersa Pro got four stars from editors and users alike, but the sponsored BeyondSync got five stars from editors but only three stars from users.  Otherwise, among programs most downloaded during the preceding week (so as to avoid some that may have been popular in the olden tymes), the only one of note seemed to be Allway Sync.

From those sources and from general browsing, certain names appeared to be emerging, including GoodSync, Allway, and ViceVersa.  A search for comparisons of those three didn't yield obvious recent sources of insight.  (Later, I would find a Wikipedia page that compared many programs on a number of features.  It seemed to indicate that GoodSync was one of the most capable programs.  I wasn't sure, at that point, whether a closer look at the other programs reviewed there would have yielded a different outcome for my purposes.  My tentative conclusion from that webpage was that, along with the programs reviewed below, I probably would have looked at FreeFileSync.)  The TopTenReviews comparison suggested that, in terms of features, the top six just listed were mostly competitive for my purposes, except for SugarSync.  According to TopTenReviews, the top three (Syncables 360, GoodSync, and Laplink) pulled away from ViceVersa and Allway Sync in terms of ease of use, support, and documentation.

I looked at individual product reviews at Softpedia.  There were some contrasts.  GoodSync got 2.9 stars from 118 users.  ViceVersa Free got 2.5 stars from 23 users.  ViceVersa Plus got 3.0 stars from 25 users.  ViceVersa Pro got 3.6 stars from 28 users.  SugarSync got 3.1 stars from 25 users.  Syncables Desktop got 1.6 stars from 3 users.  Allway Sync got 3.3 stars from 112 users.  BeyondSync got 3.2 stars from 11 users.  There was no Softpedia entry for Laplink.  In short, according to voters at Softpedia, the two leading programs were ViceVersa Pro and Allway Sync.  The lackluster score given to GoodSync seemed to be relatively well based.  That is, there seemed to be a fair number of confirming votes.  I should mention that, in a presumably different class, Microsoft SyncToy averaged 3.9 stars from 54 voters.  Of these programs, it appeared that Softpedia's editors rated only SyncToy.  They gave it four stars.

I also checked program reviews at CNET.  GoodSync got four stars from 654 user reviews, and four stars from editors.  ViceVersa Plus got no votes; ViceVersa Free got 3.5 stars from 20 users; ViceVersa Pro got four stars from 30 users, and four stars from editors.  SugarSync got five stars from editors, and 2.5 stars from 82 users.  Syncables 360 got two stars from four users.  Allway Sync got 3.5 stars from 53 users, and 4.5 stars from editors.  BeyondSync got three stars from 14 users, and five stars from editors.  Laplink had one star from one user.  SyncToy got four stars from editors and 3.5 stars from 61 users.  These results suggested that, for CNET users, the worst programs were Laplink, Syncables, and SugarSync, and the best programs were GoodSync and ViceVersa Pro, followed by ViceVersa Free, Allway Sync, and SyncToy.  There was something odd about the fact that GoodSync got at least eight to ten times as many votes as the others.  Again, there were suspicious discrepancies between the votes of editors and of users.

Overall, the reviews provided by these three sources -- TopTenReviews, Softpedia, and CNET -- were only somewhat consistent.  Their sometime inconsistency may have been at least partly due to somewhat different target markets.  At TopTenReviews, the good score for GoodSync was fairly compatible with the results at the other two sources, though the mediocre score at Softpedia was interesting.  TopTenReviews' high ratings for Syncables and Laplink seemed incongruous.  ViceVersa Pro and Allway Sync were in the top or second tier at all three sources.  SyncToy was fairly highly rated by the two sites that reviewed it.  Among the others, ViceVersa Free also seemed to be potentially worth a look.

Although Allway Sync had a free version, its license page indicated that that version had a limit of 40,000 files per month.  Although I was not sure how that would be counted, it seemed likely that a couple of full backups per month would exceed that limit for many people.  The pro version was available for $20, in comparison with $30 for GoodSync and $60 for ViceVersa Pro.  I noticed that the CNET editors described Allway Sync as "one of the best tools we've tried."  I decided to try it myself.  Also, while I now saw that SyncToy had no scheduler, it appeared likely to be useful for some purposes, so I downloaded it too.

Microsoft SyncToy 2.1

I decided to start with SyncToy for two reasons:  I believed it would be simple to install and run, and I figured it would probably be pretty severely limited, and could therefore be disposed of quickly.  In other words, it seemed likely to be a handy tool to know about, for some purposes, but not my primary machine for keeping computers reliably synchronized.

The program's interface seemed pretty straightforward.  Nonetheless, I ran a search for sources of guidance and insight.  A How-to-Geek webpage said SyncToy would require me to set up a pair of folders that I wanted to synchronize, and would then ask me which kind of synchronization I desired.  There were three options:  Synchronize (copy and delete in both directions, between the two folders, so that the latest additions, changes, and deletions appeared on both), Echo (update the right folder so it matched the left), or Contribute (add to the right so that it contained everything on the left, but don't delete anything from the right).  The How-to-Geek webpage went on to explain how SyncToy tasks could be automated through Windows Task Scheduler.

A discussion thread pointed out some limitations with SyncToy:  no volume shadow copy, so locked files would not be copied (which also seemed to be the case with GoodSync); no way to back up your folder pairs, for reinstallation or for use on another computer, so they would all have to be recreated by hand; a nasty bug regarding timestamps when synchronizing NTFS to FAT drives.  Hints to a potential solution for the folder pair backup issue appeared in another thread, discussing the possibility of creating folder pairs via command line.

The "What's New" section in SyncToy's internal help file suggested that SyncToy 2.1 might have resolved some or all of these issues.  I ran another search and saw references to memory leaks and other bugs.  A discussion thread contained indications of some dissatisfaction.  I decided SyncToy was probably pretty good, but it was not the sort of limited but airtight and portable tool I had imagined.  It was more like a contender for the role of primary synchronization tool.  For that purpose, I had a sense that I would like Allway Sync more, so I turned in that direction.

Allway Sync

For some reason, after that brief look at SyncToy, I started getting cold feet.  A couple of days had passed since I had started the thread, and now, as I looked at alternatives to GoodSync, it seemed like I was getting myself into a lot of unnecessary hassle.  This was probably due to two things.  First, I was busy with other stuff and didn't want to deal with this at the time; and second, I was having an unrelated system issue that kept the secondary/server computer from seeing the primary/workstation computer.

I rebooted as an expression of my faith that this would make it all better.  It didn't.  I gave it a couple of hours and the let spiritual power of saying "to hell with it" do its magic.  When I came back to my existential mess, computer B was seeing computer A.  All was well with the world.

So I installed and ran Allway Sync -- an awkward name, by the way, referred to hereinafter as AS.  AS had distinct Analyze and Synchronize passes, like GoodSync.  I set it up to compare a folder on each machine.  The interface was OK.  I didn't like the huge chunk of real estate, at the top of the screen, devoted to nothing in particular, but found I could get rid of that with View > deselect Show Logo.  As it was running its analyze pass, it suddenly gave me this:

script error, Unspecified error., URL:
file:///C:/Program%20Files/Allway%20Sync/Skins/default/profileex.js, line: 5138.

It offered to send an error report.  I let it do that.  I wasn't sure what effect the error might have on that comparison across two machines.  There were 89 Important Messages.  The brief messages provided -- "Questionable file (manual review recommended)" -- were not as informative as GoodSync's left/right comparison messages.  I didn't like its interface.  While I was poking around with it, I got another script error.

I wasn't excited about Allway Sync.  Blame it on them, blame it on me, but for whatever reason, I decided to keep looking.

WinMerge

Around this time, I came across the freeware WinMerge program.  I hadn't seen much mention of it in TopTenReviews or other commercial sites.  Wikipedia said the WinMerge project was currently dormant.  The program's homepage indicated that the last stable version (2.12.4) had been released nearly three years earlier, in mid-2009.  But Softpedia contained an "experimental" v. 2.13.20, uploaded in October 2010, rated 4.5 by 139 users, and CNET boasted installed and portable versions of a version 2.12.4 (4.5 stars, 29 votes).

Wikipedia's comparison of file comparison tools listed Beyond Compare but not GoodSync.  I had purchased BC Pro.  It had worked well for backup purposes, where I manually eyeballed what was going on.  I hadn't wanted to use it for automated syncing of computers, which seemed to be a less critical operation in my system.  (If GoodSync had been randomly losing stuff, I would have noticed it when I did my Beyond Compare backups.  In other words, after some early watching and tinkering, I was comfortable with letting GoodSync keep the two computers aligned, taking only an occasional look at what it was doing.)  I might have been more inclined to use Beyond Compare for that purpose if I had learned its scripting language, which might not have been that hard; but I had gotten started with GoodSync through a free trial, and just went ahead with that.

Based on the Wikipedia comparison, it now appeared that what I got for my money, in buying Beyond Compare rather than using the free WinMerge, was the ability to do three-way comparisons (which I never did) and possibly the ability to do scripting (Wikipedia drew a blank on that).  WinMerge could not do "horizontal" something (that was all Wikipedia said:  horizontal or vertical, and BC could do both).  WinMerge would have given me a "moved lines" capability that Beyond Compare did not have.  WinMerge did not have FTP support.  These things did not seem crucial to me, though further research to verify what they meant would obviously have been advisable.  But there was one real problem:  WinMerge would not do CRC checks.  Granted, I had not actually been doing those checks in GoodSync either.  My reason was that GoodSync was already taking over my secondary computer; I could hardly bear the thought of making it even more demanding.  But I did want that capability.  It had been a while since memory or other system errors had noticeably corrupted numerous files, but I'd had that experience in the past, and preferred to have some warning if, perchance, one computer quietly began to wander off course.

Other programs mentioned above (e.g., Allway, SyncToy, Vice Versa) did not appear on the Wikipedia comparison list.  It seemed I was getting into an application for which a program like Beyond Compare or WinMerge was designed, and at which it might not excel.  For instance, the comparison did not include features, like scheduling and networking, that seemed essential for synchronizing computers. 
Nonetheless, I decided to try the portable version of WinMerge.  It was small, as befit a portable.  I could see right away that we weren't going to be coddling anybody.  Specifically, I couldn't figure out how to use it.  It didn't have Beyond Compare's user-friendly buttons and menus.  There were no tooltips, so I'd have to just learn what the various buttons meant.

There didn't seem to be a way to tell WinMerge to actually do anything.  I went to its online help manual.  The answer I was looking for was to go to File > Open.  This would seem obvious in another kind of program, but here I wasn't looking to do anything with any particular file.  I was looking for a way to compare folders.  But now that I did take that route, I saw a dialog allowing me to compare folders.  It was able to navigate to the other networked computer, so I compared the D:\Workspace folder on two different computers.  WinMerge seemed to be doing this comparison very slowly.  After a while, I saw that it had stalled on comparison of a particular item.  It didn't name the item; it just said it was item no. 7115.  That may have been a bug; when I killed it, I got a display showing 7142 items, which I think was the total number of items it had said it was comparing.

The WinMerge display was informative.  I could set it to show only the items that were different.  It was a plain-text display, without colors or graphics; but because it used the same tight font as Windows Explorer, with more narrowly spaced lines, it provided a lot more information per screenful than GoodSync did.  It had a collapsible tree mode, like GoodSync.  It was willing to generate a patch.  I didn't know what that was, but apparently it meant a code patch.  It seemed to be a way to help programmers offer suggestions that would improve WinMerge.  There weren't any scheduling features, so it appeared that, for my purposes, I'd be learning how to write command-line scripts and running them through Windows Task Scheduler.  But if I was going to do that, I thought it might make more sense to use Robocopy, which was already built into Windows.

Robocopy

In one of those uncanny cyclical events, exactly a year had passed since I had last worked with Robocopy.  Robocopy was a command-line tool in Vista and Windows 7 (and possibly elsewhere).

Back then, I had set up batch files to run relatively simple Robocopy commands to make backups.  A month later, I had tried to use Backup Maker instead, but that hadn't lasted long.  By this point, both of these tools had fallen into disuse, because I didn't like the backup configuration they were giving me, and I was too busy with other stuff to revisit the question.  I decided to revisit it now.

One thing I hadn't liked about Robocopy was that, every hour (or whenever), Windows Task Scheduler would pop up one of my old-style batch files to run my Robocopy script.  It was inelegant; it was distracting.  So I was interested in Robocopy front ends that might conceal the script execution.  There seemed to be several such front ends:  WinRoboCopy and Easy RoboCopy and Robocopy GUI and SH-Soft RoboCopy GUI and RichCopy, apparently a hybrid using some Robocopy and some other tools.  RichCopy offered ways of fine-tuning a copying process, but they didn't seem relevant to me.  When I ran across a critical report on RichCopy, I decided not to start with it.  Among the others, Robocopy GUI seemed to have the best pedigree, being a Microsoft product like Robocopy.

The link for the Robocopy GUI download was confusingly described as a link to the code for the Technet article discussing it.  I wasn't sure whether it was a link to the code for the article itself or was, rather, a link to the code (or, less nerdily and more accurately, the utility, including its accompaniments, e.g., User's Guide) discussed in the article.  Anyway, somehow I managed to get the utility itself.  I installed and ran the GUI.  It was a concise little thing that you could not possibly use safely without the aid of a Robocopy guide or manual, or a preexisting knowledge of Robocopy.  I say that because, for example, its Copy Options tab (one among seven available) included such options as /E and /ZB.  What did these mean?  The program did not say.  It did have a Help > Robocopy User's Guide menu option that led to a 35-page Word document which I promptely PDFd.  It seemed the principal value of the GUI would be to remind and organize, not really to reduce the need to understand Robocopy's options.

At this point, I started a separate post to discuss Robocopy options in detail.  I also considered using something like Beyond Compare, which would allow checksum comparisons rather than Robocopy's relatively simpleminded and potentially incorrect comparison of dates and times.  As noted in that other post, the main problem I was encountering, with both Robocopy and Beyond Compare (aside from the latter's sometime ability to bog down system processes), was that they weren't true automated synchronization solutions.

What I probably needed, for the checksum issue, was a distinct program that would do continual, low-priority background comparisons between the two computers, so as to distinguish files whose checksum inconsistency was accompanied by a change in file date and time (probably indicating a legitimate update of the file) from those whose checksums differed while timestamps remained the same (suggesting the possibility that one had become corrupt). Having this as a low-priority background function would allow this potentially resource-intensive file comparison function to run when the computer was not busy -- which, itself, would not necessarily be a good trait for a synchronizer. This background checksum comparison wouldn't ordinarily be an urgent function, though it could be an important one. I would be happy with a report, presented to me each morning, as to the results of checks during the last 24 hours, with links that would take me to individual files displaying expected and unexpected checksum inconsistencies. Ideally, such a stealth checker would (also, or perhaps instead) compare checksums against a database, which would be updated upon file creation or acquisition. In other words, as soon as I would PDF a Word document, this background tool would calculate and store that PDF's checksum, so as to facilitate future estimations of what had been happening to it (e.g., it had been subsequently edited in Acrobat, which would tend to indicate manual user verification and therefore an expected change in checksum).

Regarding the bogging-down of system resources, it seemed that, if I did find a way to use something like Robocopy, there was the question of how I could prevent the program from hogging system resources.  There seemed to be two parts to this question: how do I detect that the computer is busy, and how do I slow Robocopy in that case? Or was there maybe one tool that would let me slow down designated processes when computer resource usage reached a certain level? This led to a separate inquiry. At this point, that inquiry had yielded the impressions that something like Process Hacker or Process Explorer might put the brakes on how heavily a program would draw upon system resources during busy times. Those tools hadn't been especially successful against GoodSync, but possibly GoodSync had been programmed to demand priority, in a way that would not be a problem with something like Robocopy.

There was also the question of how I could run a command-line tool like Robocopy unobtrusively.  So far, I had used Robocopy in batch files that would pop up and require a manual click to get them out of the way.  Not a huge problem, but when it occurs multiple times per hour for each of several partitions, it could get to be an irritant.

Conclusion

At this point, I was not finding good alternatives to GoodSync.  The slowness problem seemed to become less pronounced, over the several days during which I was doing these investigations and writing up these notes.  I wasn't sure if that was just because we had not again encountered a heavy load, or if GoodSync 9 had perhaps made a huge improvement over version 8.

My investigation had turned up numerous possible replacements.  Among these, I had identified several programs with potential.  The leading one would be WinMerge; and if that didn't work, I would try Microsoft SyncToy.  I decided to give GoodSync 9 a while longer, to see if it continued to take over the secondary computer.  In that event, I thought I might first make a renewed search for tools that would reliably slow down GoodSync during times when the machine was not idle.

Windows 7: Setting and Maintaining Accurate System Time

I wanted to keep two computers' clocks set the same, for purposes of synchronization, so that they would have an accurate sense of whether the version of File X on computer A was newer than the version of File X on computer B.  I had previously installed (or, more accurately, just added a copy of) Judah Levine's portable NISTIME 32 in something of a rush, when installing Windows 7, and, later, had vaguely recognized that it was not working right and/or I had not set it right.  Now I decided to work out the kinks in this function.

NISTIME-32BIT.EXE

I started with the National Institute of Standards and Technology (NIST), from which programs like NISTIME 32 would draw the current time.  It developed that NIST had a program called nistime-32bit.exe.  It turned out to be the same as NISTIME 32, just slightly updated.  The webpage's instructions were to start by going into File > Select Server and then Query Server > Now.  Somewhere I saw advice to choose a server near me.  I was tempted to choose two different ones, one for each computer, so as to have accurate time in case there was some terrible disruption of the national timekeeping system.  Then I realized that this could have the effect of making rivers run upstream, where my files were concerned, to wit:  new could be replaced by old.  Being up-to-date on the latest developments in American chronology suddenly seemed less important than making sure I didn't accidentally overwrite today's crossword puzzle.

When I went to the Query Server > Now menu pick, I got a dialog indicating that NISTIME 32 was prepared to adjust my computer by 0.953 seconds.  I told it to go ahead.  I also went into Query Server > Periodically and told it to update the computer every 12 hours.  Query Server > Server Status confirmed these settings.  File > Help in Choosing Dirs told me to hit File > Save Config to save my settings.  This gave me "File Error:  Cannot open file to save configuration."  That problem may have been caused by nesting the program too deeply in a subfolder.  I moved it elsewhere and tried again. Now it seemed to confirm that it had saved my settings, and it created NISTIMEW.CFG in the same folder as the program's portable executable (nistime-32bit.exe).  I exited and restarted, and it remembered what I had told it.  But I had to remember to hit File > Save Config; it would not remember anything.

But then, when I did go into Query Server > Periodically, specified 12 hours, and hit File > Save Config and then File > Exit, I could not get it back.  The program refused to become visible.  I tried a couple of times, and then looked at Windows Task Manager (Start > Run > taskmgr.exe) > Processes tab.  Taskmgr showed four separate instances of "nistime-32bit.exe *32."  I selected them and clicked End Process, one by one, and then ran nistime-32bit.exe again.  It returned to taskmgr.exe, but not to the screen.  I minimized all windows, one by one, but, no, it was not lurking anywhere.  There didn't seem to be a taskbar or system tray icon for it.  It was here, and yet not here.  I killed the processes again, now that I had started one or two new ones.  I renamed NISTIMEW.CFG to be something else, and now it would start, and it saved new settings in a new NISTIMEW.CFG.  Apparently the config file had gotten corrupted.  I had originally created that file manually in lowercase (nistimew.cfg); possibly something about the program needed the uppercase filename.

But now, same thing again.  Exiting and restarting gave me a hidden program:  visible in Task Manager's Processes tab, but not visible onscreen.  When I right-clicked on nistime-32bit.exe *32 in Task Manager and selected Properties, I got an error:  "Windows cannot find [pathname] nistime-32bit.exe."  I ended the process again.  I created a shortcut to the .exe and tried starting it that way.  I had no reason to think that would make any difference, and in fact it didn't.  I tried moving all of the files from the folder where I had put nistime-32bit.exe, and placed them all instead in C:\Windows, with a shortcut to the executable in my Start Menu.  That wasn't the answer; I still got lurking program sessions that appeared in Task Manager but nowhere else.  I deleted the CFG again and tried again.  Now it ran.  I went directly to File > Save Config without making any changes.  It indicated that it had saved the config file.  I exited and restarted the program.  It ran.

Now I saw something that may have explained the config file problem.  The server list had changed.  The Colorado server that I had selected previously was no longer listed in File > Select Server.  I had previously gone into File > Update Server List, and that had generated a message:  "New server file is C:\Windows\NIST-SRV.LST."  It did that again now, when I designated a new server.  I hit File > Save Config and then File > Exit, and then restarted the program.  Now it was running normally.  I moved the three files (the exe, cfg, and nist-srv.lst files) from C:\Windows back to the folder where I really preferred to have them.  It seemed that the server list had not properly updated when the files were in that folder originally.  I restarted and went through the same steps -- update server list, choose a new server, save config -- and now I was exiting and restarting without a problem.

But no, I spoke too soon.  When I restarted, saved a 12-hour periodic refresh, and exited, it would not restart.  Deleting the config and moving the other files back to C:\Windows did not fix it.  The problem seemed to relate specifically to the attempt to set up recurrent time checks.  I was doing something wrong, or perhaps the program had a bug, or maybe it was not suited for 64-bit Win7.  I went to the NIST webpage cited in the program's Help > More Help and sent an email to the Webmaster link at the bottom of that page, pointing them here.

The Built-In Windows Time Sync Option

I decided to look for an alternative time-updating program.  I ran a search and discovered that there was apparently some kind of automatic time-updating arrangement built into Windows.  The advice there was, however, that "The W32Time service is not a full-featured NTP solution that meets time-sensitive application needs."  That was consistent with the fact that my two computers' timeclocks tended to be somewhat inconsistent with one another.  I had not tried to see how inconsistent they could be, or how long they could remain that way.  I did see an indication somewhere that Windows defaulted to a weekly time update, so maybe it would verify that it was accurate to within a minute, or something, every week or so.

That appeared to be steered by Control Panel > Date and Time.  That dialog could also be opened by right-clicking the clock in the system tray and choosing Adjust Date/Time.  Or, as I now learned from Eric Phelps, it could also run from the command line via "rundll32.exe shell32.dll,Control_RunDLL timedate.cpl."  The latter option would facilitate the option of opening the Date and Time dialog for manual adjustment via, say, a batch file that would open it automatically (to the correct tab) every day, week, or whatever.  (Later, I found a How-To Geek webpage that said I could just run "w32tm /resync" as administrator to resynchronize the clock without even going into the Date and Time dialog.  That, too, could be incorporated into a scheduled batch file.)

The Date and Time dialog > Internet Time tab > Change Settings option gave me a choice of synchronizing with time.nist.gov, which I understood to be the most accurate (though others in that list, not counting time.windows.com, appeared to be cousins of NIST).  I noticed that the dialog told me, here, that "This computer is set to automatically synchronize on a scheduled basis."  The previous sync site, as I saw on the other computer, was time.windows.com."  I wasn't sure how synchronizing with that site could have left my two computers with different times -- differing by seconds, that is, not by minutes -- unless maybe time.windows.com was just not that worried about the seconds.  Or maybe it was trying to synchronize when my router was doing its daily self-restart, and was therefore not getting access to the online clock?  I wasn't sure.  (Note:  Fouzan said that this whole process wouldn't work if the computer was on a domain.)

Curious about the timing, I went into Start > Run > taskschd.msc > Task Scheduler Library.  There were maybe 15 items in the list, and none of them were obvious time sync tasks.  So another possibility was that some bug or tweak, brought into my system somewhere along the line, was preventing the creation or execution of the scheduling function.  Another emerging possibility was that, as stated in a How-To Geek webpage, time.windows.com (which my systems had been using by default) had "a ton of problems with uptime."  So possibly I had already fixed my problem, just by switching the machines to use time.nist.gov in the Date and Time dialog.  (I did notice, as soon as I made that switch and clicked the update button, that both computers' clocks showed exactly the same time.)

Other Possibilities

I ran another search and found a Gizmo recommendation for Dimension 4 as a time correction utility.  It occurred to me, at this point, that possibly I had fixed my problem, just by switching away from time.windows.com (above), and that maybe I should just let things slide for a week or two.  I decided mostly just to record some notes, here, for possible future reference.  So instead of installing Dimension 4, I just dragged the icon for its webpage from my browser's Address bar over to the Time subfolder in my customized Start Menu.  If I ever needed it, I could follow the link at that time.

There also appeared to be more to know than I had realized, regarding Task Scheduler (taskschd.msc).  In Task Scheduler's left-hand pane, I went down the tree into Task Scheduler Library > Microsoft > Windows > Time Synchronization.  Now I saw that my machine was indeed set to synchronize time at 1 AM every Sunday.  I saw advice from Tina Sieber on a way to adjust and improve the scheduling via Task Scheduler.  Tina seemed to believe, however, that using a separate program might be the simpler and more accurate approach.  Tina pointed toward two other programs, Atomic Clock Sync and AtomTime.  The webpage for the latter seemed very old.  I was not sure how it would fare in a 64-bit Windows 7 world.

For now, the solution seemed to be simply to go into the system's clock and change its time source to NIST.  My monthly batch file brought up the NIST/USNO timepage on the first of every month, so I could observe, later, whether my two computers were again diverging from one another and/or from the time on that webpage.  If they didn't stay in line, I would have two options.  One would be to add one of the foregoing command lines to my daily or weekly batch files, to permit manual and/or automatic checking and/or resynchronization.  Another would be to try one of the several freeware utilities just mentioned, particularly Dimension 4 or Atomic Clock Sync.

Windows 7: RAID or Mirror Across Computers?

Where to put the data ... hmm.  I had a home network with two computers running Windows 7.  If the data I needed to work with was on one computer and it went down or had one of those frequent Microsoft maintenance or service interruption needs, I couldn't get to it from the other computer.  But if I put the data on a server, then (a) I had to buy and maintain the server, cables, routers, etc., (b) I had slower access times, (c) the data would then be unavailable to *both* machines (unless I wanted to swap out one or more hard drives) if the server went down, and (d) I had found that, if I accidentally corrupted or deleted the wrong file, a server might not be willing to undelete it.  Not a big deal, assuming you had good backup, but there were painful exceptions.

So it occurred to me:  can you put the data on one computer, so that it can function as a standalone, and also put the data on the other computer, so that it is a standalone too, but then have a constant RAID or mirror arrangement between the computers, so that whatever you do with the data on one computer is immediately duplicated on the other computer?  That way, you've got local speed, no server, and redundancy during downtime on either machine.  Basically, two-way mirroring:  when a file is modified, it checks the other computer, and the two of them figure out which version is newer, and it overwrites the older version on the other machine as well.  All you need is a router, if that.

I figured possibly everybody else already knew the answer to this.  But since I didn't, I started with a search.  Only six hits.  It looked like the concept of "RAID between computers" was a nonstarter.  Alright, a different search.  Wow, "mirror between computers" produced 13 hits.  But, OK, not to complain, it seemed most of those hits were for TreeNetCopy.  Take it out of the equation, and the search produced only five hits.  So TreeNetCopy seemed to provide the path forward.  But it didn't look like CNET, PCMag, or other big-name sites had reviewed it.  I went to the product's home website and found out why:  it was for systems using Windows NT or Windows 2000.

Apparently mirror and RAID were not the concepts I wanted.  How about incremental backup?  You couldn't have it running constantly; it would have to finish one scan of the system's files before it could start on the next one.  So maybe you'd set it to run every 15 or 30 minutes, or however a scan would take to finish, across the network connection.  This wouldn't be nearly as good as software that would detect and propagate changes as soon as they were made, but I wasn't seeing how to find anything like that.  With a 15- or 30-minute delay, you couldn't have someone being able to open the file on computer B as soon as someone else updated it on computer A, unless possibly if you had a script that would somehow be able to run the incremental backup manually for a given folder by just maybe choosing a right-click context menu option.

Alright, a different approach.  I had been using Beyond Compare, a file comparison tool.  It still looked like one of the more capable file comparison tools, so how about using it?  As I was thinking about that possibility, I realized that I didn't like the idea of having to do a right-click or other manual update.  The computer could crash before I got around to that, and then I wouldn't have the current data in the parallel folder on the other computer, and therefore really couldn't just keep right on working where I left off.  I had only used Beyond Compare as a manual comparison tool, where I would start it up and it would run for a while and compare directories and then show me what needed to be mirrored to my backup drive, and then I would click the buttons necessary to do that.  I knew it was possible to write scripts to automate some of this, but it seemed unlikely that scripts would help Beyond Compare remain up-to-the-minute on all of the file changes made on the system.  Most likely, I could set up scripts to run in some frequently used folders, and maybe even to automate the mirroring of those folders, but other folders would be left out in the cold.  Possibly I could have multiple scripts doing comparisons of more- and less-frequently used folders on different schedules, so as to increase the likelihood that most folders would be mirrored relatively often.  But with enough scripts running simultaneous file comparisons, I'd start to take a performance hit.

Lacking a better option, I did a search to learn more about Beyond Compare scripts.  The search came up with a number of interesting concepts, right there among the top ten hits.  One was the concept of automated synchronization.  Duh!  Of course.  Synchronization was the Windows term for what I wanted.  So I did a search for that, dropping Beyond Compare for the moment.  But the only thing that came of it was the discovery of Super Flexible File Synchronizer, which cost $60 for a two-year license (unless, for some bizarre reason, I would think that I could do without the pro version's ability to copy ZIP files!).  It did look like it might have some advantages over Beyond Compare, such as the ability to detect that I had moved a folder, so that it could just repeat the move rather than delete the folder from one location and create it in another (which might involve a lot of copying, if it was a large folder).  It had very good ratings on CNET.  I could download and try it out free for 30 days.  But it was ultimately still a backup program, running on a schedule, not a mirroring program, so I was still basically working with the same scenario:  design a set of backup scripts, profiles, or whatever, and set them to run at different frequencies, backing up what I would consider the most heavily used folders most frequently.

TopTenReviews ranked Super Flexible File Synchronizer eighth in its list of sync programs.  Their comparison page had a number of relevant criteria, including the ability to do bidirectional sync, to mirror files, and to operate across a network.  It actually looked like their number two program, GoodSync ($19.95), had better features for my purposes than their number one choice, Syncables 360 ($39.95).  Their review of GoodSync made it sound good indeed.  They said it couldn't sync or merge Microsoft Outlook files, which was OK because I was using Thunderbird.  (Later, I encountered a review by a user who said s/he was using it for this purpose, so I assumed they had updated the program.)  They said that working over networks could be complicated.  I wasn't sure if they meant that as a generic remark that would be relevant to all kinds of work over networks.  They seemed to rank it number two rather than one because it "lacks some of the advanced features professional users expect."  CNET's review likewise ranked it number two, but in the category of "file management."  I wasn't sure what they considered the number one program; their webpage didn't indicate which criterion they used for that ranking.  But GoodSync was the most frequently downloaded program during the prior week.  GoodSync's awards webpage mostly listed awards and positive reviews that were at least a couple of years old.  So apparently it had been created and was now coasting.  I did a search among its many reviews on CNET, looking for more info about using it on a network.  Unfortunately, CNET's links to specific reviews weren't working for me at that point, so I wasn't able to get details, but what I was able to read from the summary results was positive with the exception of one person for whom GoodSync did not work well.

There were hardware options.  SyncSharp offered a device that would synchronize via USB.  It sounded similar to The Tornado and to the Windows 7 Easy Transfer option.  I didn't want an additional device, and since ethernet was faster and was already in place, I didn't want to use USB.  For purposes of speed and also capacity, not to mention reducing dependency on external data sources, I was obviously not going to be interested in a cloud (i.e., web-based) solution, even if I had found one that offered constant, continuous, real-time synchronization.

It turned out that CNET had another category, for "data transfer & sync software."  As with some other CNET searches, I looked at the top 30 both in terms of downloads last week and user ratings.  Setting aside those that were for special purposes (e.g., Blackberry, Outlook), and focusing on those that were for Windows 7, I found that only two were free:  CopyTo Synchronizer, which had only two votes and which I therefore deemed insufficiently tested, and Microsoft Live Mesh, which had only one vote but which I was willing to assume was better developed.  The nonfree alternatives that came up in this search included BeyondSync, ViceVersa Pro, Easy Computer Sync, and Syncables 360 Premium.  I reduced this set to Live Mesh, Beyond Sync, and ViceVersa.  A search for further information on Live Mesh suggested it was web-based, more like DropBox, leaving me to focus on the other two.  A search led back to a TopTenReviews comparison -- it may have been the same one as before, but a couple of weeks had passed by thie time -- naming GoodSync (above) as No. 2, ViceVersa as No. 5, and BeyondSync as No. 10.  Of the ten, the ones offering bidirectional sync, network synchronizing, and Windows 7 support were these three plus Syncables 360, SugarSync, Laplink, and Super Flexible.  Most of the same names also appeared in a CEOWorld review.  I eliminated SugarSync as another cloud solution.  The TopTen review for Beyond Sync made it sound unappealing.  A dotTech review echoed that. 

I ran a search looking for comparisons of Syncables against the others that also sounded good for bidirectional synchronization.  A couple of reviews alerted me to the feature, evidently present in GoodSync but not all others, of being able to see which changes would overwrite.  I began to get a sense that Syncables was more of a glossy product, designed for people who wanted simple and trouble-free synchronization without necessarily having an option to scrutinize every step of what was happening.  Having been burned by the occasional backup program that would not save (and would also not tell me that it was not saving) files of a certain kind, or nested too deep, or had an umlauted character in their filenames, or were otherwise secretly exempt from what I thought was happening, I had become more inclined to use transparent software.  At least until I gained a lot of trust and experience with a program, I wanted to see what it was doing.  So this feature of GoodSync appealed to me.  I noticed, also, that a review described Liuxz Sync as being "of most use to users that need to carryout real time synchronizations over a network or between hard drives."  As I continued to look at other opinions, ViceVersa still sounded relatively good too.

On this basis, I decided to start with GoodSync, as described in a separate post.  After some days of using it, and comparing its results against an external backup drive via manual comparisons using Beyond Compare (as described in more detail in that other post), I concluded that GoodSync was a good product for this purpose.  I set its sync rules so as to check most frequently those partitions in which I was most likely to make changes.  For practical purposes, I could change files on one computer and I would see those changes on the other computer when I went looking for them.

In short, I wound up using GoodSync to synchronize files on two computers on a home network.  The files were generally available on the other computer within minutes.  I did this without using a server.  That is, the files were available locally on each computer, so that I could keep right on working if the other one went down.  I arranged backup via external drive, and I occasionally checked that external drive against the internal drive manually using Beyond Compare.  I had better performance than on a network, and was not very vulnerable to network problems; presumably I could have set up the same arrangement via crossover cable, without even having a router.  This really felt like a solution that I had been seeking for years.