Windows 7: Considering a RAID Array for Performance

It had been more than a year since I had last looked at adding a RAID array to improve performance.  I could see that my hard drive light was on very often, and the system slowed way down when that happened.  So it seemed advisable to take another shot at RAID.

I had a reasonably good backup arrangement.  As such, I wasn't too worried about the risk of losing data.  My motherboard would accommodate RAID 5 or 10, if I wanted to add more drives, but at this point hard drive supplies were still reeling from the floods in Thailand.  Prices were high on HDDs, and large solid-state devices (SSDs) were still prohibitively expensive.  For at least the near future, I was looking at a two-drive RAID 0.

The main problem with RAID 0 would be the amount of time required to restore my data from backup, and the increased likelihood that I would have to do so (because, in RAID 0, failure of either drive would mean failure of the entire array).  It made sense to make the RAID array as small as possible.  It would need to contain data that I used frequently, but there were a number of other things, less frequently accessed, that I could put onto a third drive that would not be part of the array.  A smaller RAID array would be quicker to restore and would also require smaller, less expensive drives -- which, in turn, would make it more feasible, sometime down the line, to add more drives and switch to a different RAID flavor.

The Moo0 System Monitor was informing me that, at present, my hard drive delays were coming from my existing data drive, not from drive C.  I had not been eager to reinstall Windows onto a new RAIDed drive C.  I was also not too keen on the prospect that Acronis backups and restores involving a RAIDed drive C could be complicated.  So there seemed to be some reason to put my programs partition (drive C) onto that standalone third drive as well.

While I was content with my backup arrangement on a day-to-day basis, I did not presently have an hourly backup.  If I decided to make that third drive a large one, I could use some of its space to store hourly backups.

The plan, then, was for the third drive to contain my drive C partition, my large and infrequently used data items, and perhaps an hourly backup partition.  It could also hold the partition where I stored my Acronis drive images, the partition containing my customized Start Menu and installation-related programs, and the partition that I used for various programs' cache folders and such.  The only partitions on this third drive that would need to be backed up would be drive C, the data partition, and the Start Menu partition.
At this point, there was a pause.  I found myself involved with a large spreadsheet task that was just dragging in its calculations. Those calculations were taking so long that I was forced to take considerable breaks from the project to do other things while Excel would grind away. The problem there was that, when I did come back to the spreadsheet, sometimes it would be several hours later. By that point, my head was no longer tuned in to the project; I was thinking about other things. As a result, I was forgetting and overlooking things that I would have handled much more efficiently if the computer had been able to keep up.

So before installing the RAID array, I upgraded my CPU.  To my surprise, although Moo0 had not seemed to indicate this, the faster CPU contributed noticeably to other tasks as well.  This performance boost took away some of the urgency behind the transition to RAID.

A second hardware upgrade made an even greater difference.  I had found that sometimes I would have to wait quite a while for the screen to keep up when I would switch from one program to another.  I had assumed that the hard drive was the reason.  As I could see from its activity light, it was keeping very busy, and Moo0 was indicating that it was the bottleneck.  Still, it occurred to me that some of the display issues might be due to limits in the motherboard's onboard graphics.  So I installed a 2GB video card.  Suddenly things were *much* better.  It seemed that the hard drive may somehow have been working overtime to compensate for inadequate video.

The speedup wrought by these two upgrades -- the faster CPU and the more capacious video card -- was so substantial as to eliminate, for now, the need for a RAID array.  In that sense, these other hardware upgrades paid for themselves.  I was pleased not to have to spend the time and endure the upheaval that would be involved in getting my data onto a RAID array, as well as the more-than-doubled risk that a single hard drive failure would wipe out all (not just some) of my data.

There were two other considerations favoring my decision to delay the RAID investment until a time of greater need.  First, as I could see, solid state drives (SSDs) were coming down in price.  Second, I was doing some housecleaning, deleting or compressing some materials and putting others onto separate partitions.  It seemed possible that, at some point, an SSD (or, conceivably, an SSD RAID array) containing a smaller set of files would be much more affordable.  Hence, for now, the RAID investment was once again on the back burner.

Windows 7: RAID or Mirror Across Computers?

Where to put the data ... hmm.  I had a home network with two computers running Windows 7.  If the data I needed to work with was on one computer and it went down or had one of those frequent Microsoft maintenance or service interruption needs, I couldn't get to it from the other computer.  But if I put the data on a server, then (a) I had to buy and maintain the server, cables, routers, etc., (b) I had slower access times, (c) the data would then be unavailable to *both* machines (unless I wanted to swap out one or more hard drives) if the server went down, and (d) I had found that, if I accidentally corrupted or deleted the wrong file, a server might not be willing to undelete it.  Not a big deal, assuming you had good backup, but there were painful exceptions.

So it occurred to me:  can you put the data on one computer, so that it can function as a standalone, and also put the data on the other computer, so that it is a standalone too, but then have a constant RAID or mirror arrangement between the computers, so that whatever you do with the data on one computer is immediately duplicated on the other computer?  That way, you've got local speed, no server, and redundancy during downtime on either machine.  Basically, two-way mirroring:  when a file is modified, it checks the other computer, and the two of them figure out which version is newer, and it overwrites the older version on the other machine as well.  All you need is a router, if that.

I figured possibly everybody else already knew the answer to this.  But since I didn't, I started with a search.  Only six hits.  It looked like the concept of "RAID between computers" was a nonstarter.  Alright, a different search.  Wow, "mirror between computers" produced 13 hits.  But, OK, not to complain, it seemed most of those hits were for TreeNetCopy.  Take it out of the equation, and the search produced only five hits.  So TreeNetCopy seemed to provide the path forward.  But it didn't look like CNET, PCMag, or other big-name sites had reviewed it.  I went to the product's home website and found out why:  it was for systems using Windows NT or Windows 2000.

Apparently mirror and RAID were not the concepts I wanted.  How about incremental backup?  You couldn't have it running constantly; it would have to finish one scan of the system's files before it could start on the next one.  So maybe you'd set it to run every 15 or 30 minutes, or however a scan would take to finish, across the network connection.  This wouldn't be nearly as good as software that would detect and propagate changes as soon as they were made, but I wasn't seeing how to find anything like that.  With a 15- or 30-minute delay, you couldn't have someone being able to open the file on computer B as soon as someone else updated it on computer A, unless possibly if you had a script that would somehow be able to run the incremental backup manually for a given folder by just maybe choosing a right-click context menu option.

Alright, a different approach.  I had been using Beyond Compare, a file comparison tool.  It still looked like one of the more capable file comparison tools, so how about using it?  As I was thinking about that possibility, I realized that I didn't like the idea of having to do a right-click or other manual update.  The computer could crash before I got around to that, and then I wouldn't have the current data in the parallel folder on the other computer, and therefore really couldn't just keep right on working where I left off.  I had only used Beyond Compare as a manual comparison tool, where I would start it up and it would run for a while and compare directories and then show me what needed to be mirrored to my backup drive, and then I would click the buttons necessary to do that.  I knew it was possible to write scripts to automate some of this, but it seemed unlikely that scripts would help Beyond Compare remain up-to-the-minute on all of the file changes made on the system.  Most likely, I could set up scripts to run in some frequently used folders, and maybe even to automate the mirroring of those folders, but other folders would be left out in the cold.  Possibly I could have multiple scripts doing comparisons of more- and less-frequently used folders on different schedules, so as to increase the likelihood that most folders would be mirrored relatively often.  But with enough scripts running simultaneous file comparisons, I'd start to take a performance hit.

Lacking a better option, I did a search to learn more about Beyond Compare scripts.  The search came up with a number of interesting concepts, right there among the top ten hits.  One was the concept of automated synchronization.  Duh!  Of course.  Synchronization was the Windows term for what I wanted.  So I did a search for that, dropping Beyond Compare for the moment.  But the only thing that came of it was the discovery of Super Flexible File Synchronizer, which cost $60 for a two-year license (unless, for some bizarre reason, I would think that I could do without the pro version's ability to copy ZIP files!).  It did look like it might have some advantages over Beyond Compare, such as the ability to detect that I had moved a folder, so that it could just repeat the move rather than delete the folder from one location and create it in another (which might involve a lot of copying, if it was a large folder).  It had very good ratings on CNET.  I could download and try it out free for 30 days.  But it was ultimately still a backup program, running on a schedule, not a mirroring program, so I was still basically working with the same scenario:  design a set of backup scripts, profiles, or whatever, and set them to run at different frequencies, backing up what I would consider the most heavily used folders most frequently.

TopTenReviews ranked Super Flexible File Synchronizer eighth in its list of sync programs.  Their comparison page had a number of relevant criteria, including the ability to do bidirectional sync, to mirror files, and to operate across a network.  It actually looked like their number two program, GoodSync ($19.95), had better features for my purposes than their number one choice, Syncables 360 ($39.95).  Their review of GoodSync made it sound good indeed.  They said it couldn't sync or merge Microsoft Outlook files, which was OK because I was using Thunderbird.  (Later, I encountered a review by a user who said s/he was using it for this purpose, so I assumed they had updated the program.)  They said that working over networks could be complicated.  I wasn't sure if they meant that as a generic remark that would be relevant to all kinds of work over networks.  They seemed to rank it number two rather than one because it "lacks some of the advanced features professional users expect."  CNET's review likewise ranked it number two, but in the category of "file management."  I wasn't sure what they considered the number one program; their webpage didn't indicate which criterion they used for that ranking.  But GoodSync was the most frequently downloaded program during the prior week.  GoodSync's awards webpage mostly listed awards and positive reviews that were at least a couple of years old.  So apparently it had been created and was now coasting.  I did a search among its many reviews on CNET, looking for more info about using it on a network.  Unfortunately, CNET's links to specific reviews weren't working for me at that point, so I wasn't able to get details, but what I was able to read from the summary results was positive with the exception of one person for whom GoodSync did not work well.

There were hardware options.  SyncSharp offered a device that would synchronize via USB.  It sounded similar to The Tornado and to the Windows 7 Easy Transfer option.  I didn't want an additional device, and since ethernet was faster and was already in place, I didn't want to use USB.  For purposes of speed and also capacity, not to mention reducing dependency on external data sources, I was obviously not going to be interested in a cloud (i.e., web-based) solution, even if I had found one that offered constant, continuous, real-time synchronization.

It turned out that CNET had another category, for "data transfer & sync software."  As with some other CNET searches, I looked at the top 30 both in terms of downloads last week and user ratings.  Setting aside those that were for special purposes (e.g., Blackberry, Outlook), and focusing on those that were for Windows 7, I found that only two were free:  CopyTo Synchronizer, which had only two votes and which I therefore deemed insufficiently tested, and Microsoft Live Mesh, which had only one vote but which I was willing to assume was better developed.  The nonfree alternatives that came up in this search included BeyondSync, ViceVersa Pro, Easy Computer Sync, and Syncables 360 Premium.  I reduced this set to Live Mesh, Beyond Sync, and ViceVersa.  A search for further information on Live Mesh suggested it was web-based, more like DropBox, leaving me to focus on the other two.  A search led back to a TopTenReviews comparison -- it may have been the same one as before, but a couple of weeks had passed by thie time -- naming GoodSync (above) as No. 2, ViceVersa as No. 5, and BeyondSync as No. 10.  Of the ten, the ones offering bidirectional sync, network synchronizing, and Windows 7 support were these three plus Syncables 360, SugarSync, Laplink, and Super Flexible.  Most of the same names also appeared in a CEOWorld review.  I eliminated SugarSync as another cloud solution.  The TopTen review for Beyond Sync made it sound unappealing.  A dotTech review echoed that. 

I ran a search looking for comparisons of Syncables against the others that also sounded good for bidirectional synchronization.  A couple of reviews alerted me to the feature, evidently present in GoodSync but not all others, of being able to see which changes would overwrite.  I began to get a sense that Syncables was more of a glossy product, designed for people who wanted simple and trouble-free synchronization without necessarily having an option to scrutinize every step of what was happening.  Having been burned by the occasional backup program that would not save (and would also not tell me that it was not saving) files of a certain kind, or nested too deep, or had an umlauted character in their filenames, or were otherwise secretly exempt from what I thought was happening, I had become more inclined to use transparent software.  At least until I gained a lot of trust and experience with a program, I wanted to see what it was doing.  So this feature of GoodSync appealed to me.  I noticed, also, that a review described Liuxz Sync as being "of most use to users that need to carryout real time synchronizations over a network or between hard drives."  As I continued to look at other opinions, ViceVersa still sounded relatively good too.

On this basis, I decided to start with GoodSync, as described in a separate post.  After some days of using it, and comparing its results against an external backup drive via manual comparisons using Beyond Compare (as described in more detail in that other post), I concluded that GoodSync was a good product for this purpose.  I set its sync rules so as to check most frequently those partitions in which I was most likely to make changes.  For practical purposes, I could change files on one computer and I would see those changes on the other computer when I went looking for them.

In short, I wound up using GoodSync to synchronize files on two computers on a home network.  The files were generally available on the other computer within minutes.  I did this without using a server.  That is, the files were available locally on each computer, so that I could keep right on working if the other one went down.  I arranged backup via external drive, and I occasionally checked that external drive against the internal drive manually using Beyond Compare.  I had better performance than on a network, and was not very vulnerable to network problems; presumably I could have set up the same arrangement via crossover cable, without even having a router.  This really felt like a solution that I had been seeking for years.

Windows 7: Native Virtual Hard Disk (VHD) Boot

I was interested in making Windows 7 Ultimate run faster.  One possibility that came to my attention was using native virtual hard disk (VHD) boot in connection with RAID0.  This post discusses that.

A virtual hard disk was just a hard disk in virtual form.  Normally hard disks were physical.  A virtual hard disk, like a virtual machine (VM), would exist as a file (or a set of files) on a physical hard drive, not as a physical device in itself.  In other words, the user would have a physical hard drive in his/her computer, and somewhere on that drive s/he would have a VHD.

The "native" part of "native VHD boot" meant that the user would have direct access to the VHD.  Ordinary VM software (e.g., VMware, VirtualBox, Virtual PC) would require the user to install an operating system (e.g., Windows 7, Ubuntu) to serve as host (also called "parent").  Then the user would have to install the VM software (e.g., VMware) to manage the guest, and then install another operating system (e.g., Windows XP) inside the VM software as guest.

The VHD concept seemed to be that the host layer would be removed from the equation.  There would be a virtual hard drive that could be loaded or unloaded, perhaps without rebooting the computer.  The computer would be running the "guest" machine (e.g., a Windows XP installation) in virtual form.  For example, a packaged Windows XP installation could run on computer A, or on computer B, without having to be reinstalled and without having to install a host and a VM program like VMware.  In addition to simplicity, this scheme would presumably have the advantage of improving performance by removing extra layers and tasks.

In Windows 7, native VHD boot was apparently available only in the Ultimate and Enterprise versions.  This meant that it could be out of the reach of ordinary users who were not inclined to spring for such expensive software, unless perhaps they had access to it through their jobs or through academic versions.  It did not appear that Ubuntu, for one, would be developing a comparable capability in the near future.  There were hypervisors like VMware's ESXi that apparently provided somewhat similar bare-metal functionality, but ESXi seemed unavailable for my purposes.

I was interested in the possibility of using native VHD boot with RAID0.  This interest arose from a desire to make Windows 7 and other operating systems run quickly.  New installations of operating systems would typically perform well, I found, but then time and the addition of more programs and other demands would slow them down.  So, for example, I had found that Windows XP running as a VM inside VMware Workstation on an Ubuntu host was much more stable than native WinXP, but its performance had slowed dramatically.  My attempts to fix it (even through reinstalling both Ubuntu and VMware) had failed.  After several years of experimentation, I had therefore recently given up on Ubuntu for my purposes.

VHD seemed to offer a good way to run Windows 7 in a RAID array.  I had found that Win7 did not, itself, support booting from the software RAID0 arrays that Win7 was capable of creating.  In other words, a Win7 user could use Win7 to create RAID0 arrays, and could speed things up somewhat by putting some programs and data files on such arrays; but the user could not boot Win7 from a Win7 software RAID0 array, and would therefore have to install and run some Win7 program files from a basic hard drive.

As an alternative, the user could create a RAID array using hardware, either on the motherboard or on a third-party controller.  Hardware RAID had potential advantages of performance, bootability, level support (i.e., the ability to choose versions other than RAID0 and RAID1, including some I had not previously heard of), multiple operating system access (i.e., the data would remain available regardless of how the machine is booted), and reliability.  The performance story was a bit controversial.  I had seen indications that motherboard RAID was no faster than software RAID, and I had heard some claim that RAID0 was no faster than non-RAID, but my own experience (in Ubuntu) was consistent with tests indicating that even a software RAID0 was markedly faster than non-RAID.  Cost was a consideration.  While RAID5 was perhaps ideal, a Newegg search for a SATA RAID5-capable controller would cost hundreds of dollars.  Removing the RAID5 constraint led me to a $30 Rosewill item.  I didn't need that myself; I had motherboard RAID 0, 1, and 10.  For reasons of cost, noise, and heat, I wasn't going to be installing four drives to have RAID 10 (a/k/a RAID 1+0, which Wikipedia (apparently backwardly) called a stripe of mirrors, as distinguished from RAID 0+1, a mirror of stripes).  So for my purposes, it was a choice between hardware RAID0 or software RAID0; and since I was now using Win7 rather than Ubuntu, the desire to boot in RAID0 meant I would be using motherboard RAID -- unless VHD gave me a good alternative.

At this point, I had a general sense of what VHD sounded like.  Among other things, it sounded great.  But it seemed that I wouldn't run into the actual drawbacks and impossibilities until I created my own VHD and tried to use it.  As described in another post, the first step in doing so was just to create and play around a bit with a VHD, within a regular, running Win7 system.

Windows 7: Scaling Back the Ambitions

I was installing Windows 7.  I had spent nearly two weeks learning about it and tinkering with some of its possibilities.  I had learned quite a bit, and was impressed with much of what I saw.

At the same time, I was sobered by the prospect of spending additional weeks in this exploration.  It did appear that a commitment of that kind would be required to follow through with the Windows 7 projects I was looking into at that point.  Although this period of almost two weeks of tinkering and exploring had been brief compared to some periods of computer-related investigation that I had experienced in the past, it was still a large chunk of time, some of which I would rather have spent focusing on other things.

I had pulled back from my ongoing Ubuntu exploration, near the start of this project, because of its time requirements and the drawbacks I had experienced with it.  During these past two weeks, I had found that Windows 7, while being a sophisticated operating system, was still not entirely stable.  The plan of building a whole structure of RAID arrays and bootable virtual hard disks (VHDs) on Windows 7 was beginning to look like another indulgence that would continue to demand large amounts of time, not only to learn more about it, but also to maintain and repair it.

Certainly I could work toward setting up a RAID5 array eventually, as need and conditions dictated.  I could stay attuned to developments in the area of VHDs.  No doubt I would continue to refine my system in various ways.  But for purposes of my primary scholarly work, as distinct from various other projects of secondary interest, it was not essential to have RAID0 performance for my data partition or, with the aid of a bootable VHD, for my Win 7 boot partition.

As I pondered these thoughts, it seemed that my pullback from Ubuntu, a few weeks earlier, had been a sign of a change of direction with the arrival of the new year.  I liked improving my system and learning about these things.  Indeed, I believed I would like those sorts of activities more, if I approached them less ambitiously, with more patience to learn about and explain an occasional system development.

I still had some issues to work through, to achieve the more ordinary functionality that I did need.  Even those issues had the potential to consume another week or more.  But in the big picture, I felt that the time had probably come to scale back my system plans.  Accordingly, I decided to focus, perhaps for the coming year, on improving aspects of my system that seemed most crucial, particularly in the areas of reliability and trouble-free functionality.

Windows 7: Upgrade Installation to Win7 Software RAID0 Array

I was trying to install an upgrade version of Windows 7 on a RAID0 array.  This post contains some notes on what I learned about the possibilities.

I had a new basic hard drive.  I started by installing Win7 on that drive.  The upgrade version of Windows 7 required a previous version of Windows to be installed.  It was not enough just to have the previous disc or serial number.  I was interested in upgrading from Windows XP.  To accomplish this installation, then, I had to install my copy of Windows XP and then upgrade from there.

Having done that, I used Disk Management (diskmgmt.msc) in Win7 to create a couple of Windows 7 software RAID0 arrays on two other empty hard drives.  Unlike other RAID solutions, Win7 was willing to create multiple arrays and single-drive partitions on a pair of drives being used in a RAID0 array.

I hoped to install Win7 into one of those arrays (which I called PROG-FUTURE), and to put my data into another.  Of course, since this was RAID0, I planned to have a good backup scheme for the data.

I went ahead and copied my data into that RAID0 data array.  Later, when it came time to try to install the Win7 upgrade to the PROG-FUTURE array, it seemed that this might have been a mistake.  An attempt to install WinXP to PROG-FUTURE got as far as the point where the installer recognized the various partitions on my drives.  It saw the entire hard drive as a single dynamic disk.  In other words, WinXP might have been willing to install to at least one of the two drives I was using for my RAID arrays.  It gave no sign that it would install itself in any array format to two drives simultaneously.

I was not sure whether an attempt to install WinXP, Win7, or any other operating system to a dynamic drive would run into problems.  There did exist a Dynamic Disk Converter program, and probably others like it, that would apparently be able to convert the dynamic disk to a basic disk format.  I could not say how well such programs would work.

It had occurred to me that perhaps I could use the Universal Restore feature of Acronis True Image Home 2011 (ATIH) to restore a working Win7 installation to the PROG-FUTURE array.  My attempts along those lines did not succeed.  As far as I could tell, ATIH was not capable of restoring a RAID0 array.

Another possibility was to use Ubuntu 10.10 to copy Windows 7 program files from a Win7 installation on a basic drive to the PROG-FUTURE array.  This did not appear feasible at this time, however, because Ubuntu evidently could not see the Win7 RAID0 array as such.  I also wasn't sure whether the resulting partition would actually boot.

An attempt to install directly from the Win7 upgrade CD to the PROG-FUTURE array failed early in the process, when I received this error message:

Windows cannot be installed to this hard disk space.  The partition contains one or more dynamic volumes that are not supported for installation.

It appeared, in other words, that Windows 7 could not be installed to a software RAID0 array created by Win7 itself.  I found a thread suggesting that there were ways to make it work, but it seemed that the process was tricky and prone to problems.  It appeared that the array would probably better be created from some other software or by using a RAID0 controller on the motherboard or on a separate controller card.  Another possibility that I had not heard of previously was native virtual hard disk (VHD) boot.

Windows 7: Setting Up RAID

The two potential advantages of setting up hard drives in a RAID array seemed to be performance and/or safety.  Among the many types of RAID arrays, some of the more frequently mentioned were RAID 0, RAID 1, and RAID 5.

RAID 0 used a "striping" approach where data was divided between hard drives.  This was intended to speed up data reading.  The drawback was that failure of either drive would mean loss of all data, since the data was stored on the drives in a puzzle-like form that could not be reconstructed without all of the pieces from both drives.  If someone had a RAID 0 array containing 100 drives, and if the whole array would fail as soon as any one drive failed, the safety of the whole array would be only as great as the safety of its weakest drive.  So the performance improvements of RAID 0 meant greater risk of data loss.

RAID 1 used a mirroring approach where everything was written onto disk 1, but simultaneously copied to disk 2.  This would not speed things up at all; if anything, it would slow things down a bit, as the system coped with twice as much data.  The purpose of RAID 1 was safety.  Either drive could fail, and yet no data would be lost.  If the computer was set up so as to let the user know that a drive had failed, then s/he could just replace the failed drive and let the RAID array fill in the new replacement with the current state of the data.

RAID 5 used a striping approach, for speed, combined with a distribution scheme for safety.   A RAID 5 array would have at least three drives.  Failure of one drive would reduce performance but would not result in data loss.

As of this writing, Microsoft was not providing clear information regarding which versions of Windows 7 offered which kinds of RAID.  Informal reports indicated that only RAID0 and RAID1 were available in user versions of Win7, and that RAID5 (and perhaps other kinds) were available in server versions of Win7.  Acronis said that RAID5 was available only in Windows Server 2000, 2003, and 2008.  The approach I planned to take, then, was to set up a RAID0 array with good backup.

To set up the array, I typed diskmgmt.msc in the Start button > Search programs and files box.  That opened up Disk Management.  The disks I wanted to put into my array were Disk 0 and Disk 1.  First, I wanted to set up a 50GB array called PROGRAMS to contain Windows program files.  I right-clicked on the gray part of the Disk Management window where it said "Disk 0" and chose "New striped volume."  This opened the New Striped Volume Wizard.  I added Disk 0 and Disk 1 to the Selected box, specified 25600MB space (on each drive, for a total of 50GB).  I looked into allocation unit size briefly and decided to stick with the default.  As I had gathered elsewhere, I didn't have to worry about the question of converting the disks from basic to dynamic because that was unavoidable with striping, and now I was at the point where the wizard was going to take care of it.

There were some other things I wanted to put into RAID0 arrays for speed.  Examples included virtual machines and files on which I would be doing video editing.  I set up arrays for those.  But there were also some things that I didn't want to put into a RAID0 array.  These were items that were big and bulky, and therefore would take a lot of time to back up, but were not performance-intensive.  General-purpose storage would fall into this category.  If I wanted performance in that area, I could move those items out of storage and into a RAID0 workspace.  For storage, I considered using a spanned volume that would put half of the data on each of the two hard drives, so as to reduce the amount of data that I would have to restore if either drive did fail.  But it sounded like restoring data from a spanned volume could be tricky, so I thought I might just divide the stored data into two separate partitions, one on each disk.  To do that, I used the "New simple volume" option in Disk Management.

There didn't seem to be any point in trying to set up a RAID volume for an Ubuntu dual-boot.  This was software RAID, governed by the Windows operating system.  Windows allowed the user to allocate only part of a drive to RAID, and as far as I knew Ubuntu set aside whole disks in a hardware RAID array.  I did set aside some space for Ubuntu program installation, however.

With the partitions set up, the next goal was to restore my Windows 7 installation to the PROGRAMS array.  That became a whole ordeal in itself, as described in a separate post.

Two Computers: Dividing the Workload

In this post, as part of my New Year's housecleaning, I introduce several investigations, involving various ways of setting up two computers in a workspace.

*  *  *  *  *

KVMs and Monitor-Sharing Options

I had developed the habit of working with two computers, using a KVM switch so I needed only one mouse and keyboard.  A KVM switch would let me use just one monitor with two computers:  it would be showing me what was happening on the computer that the mouse and keyboard were then connected to.

I had two monitors.  I didn't have them connected to the KVM.  Instead, I connected them directly to the two computers, one for each.  So the KVM switch would have me working on computer A or computer B, but monitor A would continue to show what was happening on computer A, and monitor B would continue to display the state of things on computer B.

Actually, one KVM had started malfunctioning, and I had replaced it with another.  In the process, I decided to disconnect the mouse from the KVM too.  So now the KVM just controlled the keyboard.  It made sense:  the keyboard was big.  I couldn't fit two keyboards on my desktop.  But I could fit two mice.  Mouse A, on the left side, was controlled by my left hand, and was controlling computer A, while mouse B was preoccupied with computer B on my right.  I wasn't sure if I liked this arrangement.  It had its advantages.  It gave my hands a more equal workout.  I could glance over at the other computer and click something, to make some process move ahead, without having to hit ScrollLock twice, which was how the KVM switched the keyboard back and forth.  The main problem was that it was hard to tell which computer the keyboard was controlling at any given time.  I was still occasionally hitting the Del key and deleting things on computer A and wondering why they weren't disappearing from monitor B.  It was a bit nutty.  But I was adapting and, like I say, it had its advantages, at least until I worked out something better.

The monitor situation was OK, too.  I had widescreen monitors, so I was able to accommodate a couple of different open windows at the same time on each of them.  But there were times when it would have been helpful to have both monitors attuned to just one computer.  I had previously used an arrangement where monitor A was dedicated to computer A, but monitor B was connected to a second KVM switch, so I could reach up and punch the button and switch monitor B back and forth between computer A and computer B.  This would let me use a multiple monitor approach on computer A, so as to spread out my work.  The problem with this was that, when there was no monitor turned on to computer B, I tended not to use it.  That was a problem because the purpose of having two computers was to let me continue to work when one of them would crash or need maintenance or get involved in some process, like scanning or rendering, that would make it virtually unavailable for other purposes.  I also used this arrangement to write these blog posts, tinkering with one machine and recording the steps in the other.  I found that, if I got too fixated on one computer, the other one tended not to be organized and capable enough to take over when I needed it.

There didn't seem to be a way to share one monitor between two computers simultaneously, with computer B running in a little window in the corner -- a real "monitor" as distinct from a big display.  There was the option of buying a little display, connecting it to the second DVI connector on computer B, and putting it off to the side, so I would always have that information about what was happening on computer B.  But that little monitor would cost more than $100.  I could probably go down to Goodwill and pick up an old CRT for less than $10, but the other problem was that it would still be more clutter, more heat, more power consumption.  And the desk didn't have room for it, so it would have to go onto a side table or shelf or something.  I would also need a video card, since my motherboard only supported one monitor.

A decision on that question of how to use multiple monitors would await the outcome of an investigation of various non-KVM strategies.

*  *  *  *  *

Dual-Booting and Virtual Machines

Over the past couple of years, my two computers had different missions.  Computer A was running Windows XP in a VMware virtual machine on Ubuntu.  Computer B was running WinXP natively.  Now I was moving away from Ubuntu and back to Windows.  I had planned to just install Windows 7 on both.  I'd been having problems with my WinXP installation on computer B, even when I restored from an image, so it seemed like a good time to upgrade rather than continue to fight it.

I liked the idea of dual-booting WinXP and Win7.  Actually, though, I rarely used my dual-boot setups.  I pretty much always went with one or the other.  Right now, for instance, I already had a dual-boot Ubuntu and WinXP setup on computer B, but I couldn't remember the last time I had gone into Ubuntu there.

Meanwhile, on computer A, it seemed like I should consider just replacing Ubuntu with Win7:  continue to run WinXP in a virtual machine (VM), that is, but run it on Windows 7 instead of Ubuntu.  This would give me access to the familiar way of doing things in WinXP, while easing the transition to Win7.  I looked into the VM option in more detail.  It looked like I would be able to use the Windows version of VMware Workstation as long as I wasn't using my Workstation license in Ubuntu.  So the tentative plan was to have WinXP running in VMware on Win7 on one machine, while just running Windows (either XP or 7) natively (i.e., without a VM) on the other.

Although I was moving away from Ubuntu, it occurred to me that this strategy would leave a door open for a possible return to Ubuntu.  Ubuntu had been far more stable than WinXP.  Assuming the virtualization setup worked, I could install Ubuntu as a VM on Win7, essentially reversing what I had tried to do previously, and I could thus continue to identify projects and programs that Ubuntu could handle just as well as Windows for my purposes.

*  *  *  *  *

Differences Between Machines

The decisions described above meant that, contrary to my assumptions, I probably should not set up identical hardware on both computers.  Computer A was probably going to need more muscle to handle the VMs.  WinXP on computer B had been struggling to run video editing software, though that may have been due to a flawed installation.

Part of the tradeoff here was simplicity versus options.  The more operating systems and hardware arrangements I had in play, the more different ways I had of configuring things, and the more time it would take.  I had always viewed the computer hobbying as a means to an end of being more productive, while having its own intrinsic interest too, and for that reason I had tended to think, in recent years, in terms of getting myself set up for the next six months or a year.  During that kind of timeframe, the problems of a setup would tend to emerge and sometimes become overwhelming.  The current Windows XP installation was a case in point.  I had been able to nurse it along despite its obvious dysfunctionality, occasionally replacing its current state with an Acronis image backup, but it was now crashing one or more times per day.  So, oddly, in this case it was the simpler installation that had become the more problematic.

The switch from Windows XP to Windows 7 probably meant that I would be spending quite a bit of time getting up to speed on the best way to configure the latter.  I was probably going to install it in a RAID array, for purposes of performance, with an external backup option.  There would be some things to figure out and play with.  Most likely, some hardware would be getting shuffled around.  So, contrary to my first thoughts during this particular housecleaning, it did not appear likely that the two computers would take on any clear definition:  they would not be mirror images of one another, in hardware terms, so as to simplify maintenance, and they would also not feature one being considerably more powerful than the other.  The plan for the coming six months to a year was just to make them both capable and to explore options without investing more time than I wanted to spare for this kind of project.

Since I was now in a period of computer housecleaning, it seemed likely that my posts over the next few weeks would explore these considerations in more detail.

Ubuntu 10.10: Streamlined RAID 0 Installation

I had previously installed Ubuntu 10.04 on a two-drive RAID 0 array.  I did that to make a Windows XP guest virtual machine (VM) run faster in VMware Workstation 7.1.  I had then run into some problems with that installation, and had abandoned it.  Now it was time to try again, but this time with Ubuntu 10.10.  This post describes the process in more streamlined terms, drawing from the previous post in which I logged the details of that earlier attempt.

This time, as before, I had two hard drives for the RAID 0 array, plus a third drive on which I had already installed Windows XP.  The two empty hard drives for RAID were each 320GB.  That third drive also held my /home partition (i.e., the contents of the /home partition from a previous Ubuntu installation), which contained many of my settings and adjustments for various Ubuntu programs.  In other words, my Ubuntu installation would not be like a Windows XP installation, where it would be necessary to reinstall all of my applications (except the portable ones) after reinstalling the operating system.  The third drive also held my Linux swap space, which I probably could have put into the array instead, along with a partition I called LOCAL, which would hold backup copies of the VMware virtual machines.  I was going to put the active VMs into the RAID 0 setup to make them run faster, but of course RAID 0 was riskier in the sense that failure of either of the two hard drives would mean the loss of everything in the RAID 0 array.

I started by downloading and burning the Ubuntu 10.10 alternate (or "alternative") CD.  I booted that CD and chose the "Check disc for defects" option.  This took five or ten minutes, and then it said, "Integrity test successful," and then rebooted.  So then I went through the "Install Ubuntu" option and took the basic steps (selecting my country, my keyboard type, etc.).  The meat of the RAID 0 process began about three minutes into that video by amzertech (speaking, here, of its Part 1, not Part 2), where it was time to partition the drives.  I went to Manual (i.e., not Guided), and this put me into the main "Partition disks" screen, the one beginning with "This is an overview."  I went down to the first of the two hard drives.  It referred to them as SCSI partitions, but it also recognized them as being sdb and sdc.  So it looked like I had correctly cabled that third drive to actually be the first in the system (i.e., sda, a/k/a SCSI3 according to the partitioner), so as to make Windows happy.

The general concept of the RAID setup process was that, first, you designate some free space on each drive as a physical volume for RAID, and then you combine those physical volumes from the two (or more) drives into a single software RAID device.  The following paragraphs provide the details.

First, following the video, I went down to the first of the empty 320GB drives that I was going to use for my RAID array.  In my case, unlike the video, there was not yet any "pri/log" line showing "FREE SPACE" that I could select, under the drive identification line on the screen, so I just highlighted the drive itself and hit Enter.  This gave me the option of creating a new empty partition table on the drive, and I went with that for each of the two drives.  Then I highlighted the pri/log line under the first 320GB drive, showing free space.  There, I hit Enter and chose "Create a new partition."  For its size, I typed "50GB" and made it a primary partition at the end of the drive.  I guessed that this meant the outside of of the physical disc, where I believed data transfers would be faster.  Instead of leaving "Use as" at the default ext4 setting, I highlighted and hit Enter and went down to select "physical volume for RAID" (enter) > "Done setting up the partition."  I went through the same steps with the second 320GB drive, which was sdc on my system.  So now, back on the Partition disks" screen, each of the two drives showed an entry that looked like this:

#1   primary   50.0GB   K   raid

So this would give me a total of 100GB for my Ubuntu program installation, and I would still have several hundred GB left over as free space.  Now, on the main "overview" screen, I went up to the line that said "Configure software RAID" > "Write the changes to the storage devices" > "Create MD device" > RAID0.  This put me at a list of "active devices."  I wanted sdb1 and sdc1 (i.e., I didn't want to use one of the partitions I had previously created on sda, my third hard drive).  These were the only partitions on drives sdb and sdc, so the choice was easy.  For some reason, they showed up here as being 49999MB rather than 50GB.  I selected sdb1 and sdc1.  I highlighted each of those two, hit spacebar to select them, and then tabbed to Continue > Finish.  This put me back in the "overview" screen, where I saw that I now had these new lines, near the top:

RAID0 device #0 - 100.0 GB Software RAID device
   #1       100.0 GB
              131.1 kB         unusable

I highlighted the line that began with #1 and hit Enter > Use as > ext3 (apparently still more reliable than ext4) > Mount point > "/ - the root file system" > "Done setting up the partition."  This put me back in the "overview" screen, where the line now looked like this:

   #1       100.0 GB     f   ext4      /

I decided to go ahead with the video's approach of putting the swap space on the RAID0 partition.  To do this, I went through the same steps as above, starting with the free space line on each of the two drives.  The only differences were:

(a) I allocated only 5GB on each drive for this partition.
(b) This time, I selected sdb2 and sdc2 (instead of sdb1 and sdc1) as my active devices for the array.
(c) Under "Use as," I chose "swap area" instead of ext3.

The result, back in the "overview" screen, was that I had these lines:

RAID0 device #0 - 100.0 GB Linux Software RAID Array
   #1       100.0 GB    F   ext3      /
RAID0 device #1 - 10.0 GB Linux Software RAID Array
   #1         10.0 GB     f  swap     swap
              131.1 kB unusable

At this point, I wanted to vary from the video by adding one more partition, where I would put my VMs and possibly other things.  I went through the same process as with the first RAID device (above), and I used all of the remaining space on the two drives except for about 1GB.  The active devices in this case (when I got to that point in the process) were, of course, sdb3 and sdc3.  Back in the "overview" screen, I saw that I now had RAID0 device #2 of 528GB.  I would never need all of that space for my VMs, but I had no other use for the space, and this RAID setup process was a one-shot deal:  designate the space in some useful form now, or leave it forever unallocated.

So now, the final step.  I needed to create a /boot partition on just one drive.  That was why I needed to save 1GB.  I could have made one of those last active devices (either sdb3 or sdc3) larger than the other, but there was no point:  as I understood it, RAID0 would use only the amount of space that they both had in common.  So I would wind up with 1GB unused on one of the two drives.  Anyway, to create the /boot partition, I selected that remaining free space on sdb (i.e., the first of my two RAID drives) and used it all up on another ext3 partition.  This time, I chose ext3 without first choosing "physical volume for RAID"; and after choosing ext3, I didn't go right to "Done setting up the partition."  Instead, I stopped first at the "Mount point" option, where I chose the "/boot" option.  Back in the "overview" screen, I saw that I now had my three RAID0 devices at the top of the list, and the /boot device as sdb4 down under the first of my two 320GB drives.

In the "overview" screen, I saw that there was too much information; some had scrolled off the bottom of the screen.  I arrowed down until I got to the very bottom of all that, where I chose the "Finish partitioning and write changes to disk" > Yes option.  This started me right into the Ubuntu installation process, where I just entered basic information (e.g., my name).  The installation was very straightforward, and it worked:  Ubuntu booted up.  I then went to System > Administration > System Monitor > File Systems tab.  There, I saw /dev/md0 as root directory and /dev/sdb4 as /boot.  (The video said that the swap would not be visible here, and it wasn't.)  So my next step, at this point, was to refine the basic installation to suit my preferences.  The description of that process appears in a separate post.

Ubuntu 10.10: Tweaked Installation

I had previously installed Ubuntu 10.04, and had arrived at a fast way of adjusting the basic installation to my needs. This post updates and simplifies the separate post in which I described that process.  Here, I was using Ubuntu 10.10, not 10.04.

This post begins with a basic Ubuntu setup already installed.  That basic installation process was very straightforward; but for those unfamiliar with Ubuntu, there were many webpages on how to install Ubuntu 10.10.  The question addressed in this post was, what additional steps did I need to take, in order to make Ubuntu look and act the way I wanted?  Of course, people will have various preferences.  This post does not go into that kind of individual detail.  It is more a matter of how to go about choosing and preserving one's desired setup.

In this case, I had installed Ubuntu 10.10 in a two-drive RAID0 setup.  I have written a separate post on that as well.  It looked and acted the same as a normal single-drive Ubuntu installation.  But the following discussion contains a few references to the setup described in that other post, for those who have come to this description from there.

The Home Partition

If I had been installing Ubuntu without using RAID, I probably would have decided, in the initial installation, to install everything in a single root ("/") partition.  But the RAID process had required the creation of a separate /boot partition, so I now had those two partitions instead of one.  Either way, though, my installation did not yet include a separate /home partition.  I did want a separate /home partition to be part of the mix, because /home was where all kinds of program settings were preserved.

I had retained a copy of my previous /home partition.  It was on a third hard drive.  That is, it was not part of the RAID array, and I did not plan to copy it into the RAID array.  RAID 0 is risky, in the sense that failure of any drive in the array means the end of the entire array and everything on it.  So I was just going to leave the /home partition on that separate drive, and back it up from there.  The question was, how should I get the new installation to recognize that separate /home partition?  I had struggled with this step previously, but now I wanted to write it up in simpler form.

On that third drive, I had a folder called Saved Settings.  In that folder, I had kept a copy of my old fstab (that is, the file called "fstab," from the /etc folder, viewable by typing "sudo gedit /etc/fstab").  I added lines from that copy into my current fstab, making sure that its lines referred to UUIDs (available via "sudo blkid") rather than to drive letters (e.g., sda, sdb), so that the commands in the fstab would still function if I rearranged partitions in my computer.  I also made sure it had a line referring to the correct UUID for the /home partition.  I saved and closed fstab.

To give /home a place to be mounted, I typed "sudo mkdir /media/home."  To prevent the newly installed (and nearly empty) /home partition from interfering with the selection and use of my preferred, preexisting /home partition, I typed "sudo mv /home /old_home."  I had meanwhile allowed Update Manager to install updates, and that process was done, so at this point I rebooted.  I got an error message, "Could not update ICEauthority file /home/ray/.ICEauthority."  I clicked through that, and after a moment my old, preferred desktop layout was there and seemed to be functioning normally.  (I worked through the ICEauthority problem as described in a separate post.  Basically, the solution was to make sure the user (i.e., ray, not root) had ownership of the home folder.)  The last step was to delete the temporary old_home folder I had created by typing "sudo rm -r /old_home."

Repositories

Having the home folder in place meant that about 90% of the work of installing and configuring programs was already done and saved.  This was one huge advantage over Windows XP installations, where the only programs that did not have to be reinstalled in the event of a new operating system installation were portable applications.  This section describes the relatively few steps that I did have to take to install applications and configure my Ubuntu system, as compared to the writeup in a separate post on Windows XP reinstallation.

I could not rely on my previously saved sources.list file to set up my repositories, since I was now dealing with a new version of Ubuntu.  Instead, I went into System > Administration and discovered that, unlike Ubuntu 10.04 (as discussed in a separate post), Ubuntu 10.10 no longer had a Software Sources option.  A search revealed that this change was made to make Ubuntu more user-friendly.  I could either edit the menu to add back the Software Sources option or use System > Administration > Synaptic Package Manager as an alternative.  Choosing the latter, I went into Synaptic's Settings > Repositories > Other Software and selected the non-source options.  In the Authentication tab, I saw a list of Trusted Software Providers, and the names on it were Ubuntu (archive, CD image, and extras), GetDeb, and Google.  I had no problem with any of these except maybe Ubuntu extras.  I wasn't sure if these had come from my previous installation or were pre-supplied with Ubuntu 10.10.  I could have hit "Restore Defaults" to find out, but then I would have had to figure out how to restore them.  I closed out of that and, back in Synaptic, clicked Reload.

In Terminal, I typed "sudo gedit /etc/apt/sources.list."  I saw that it contained a handful of repositories.  I wondered if there were others I should include, so I went to the Ubuntu Sources List Generator and got a list that was more concise and that also included a few third-party repos of interest (i.e., GetDeb, Google, Medibuntu, Wine, and X Updates).  I ran the commands in the "Getting the GPG Keys" list from the bottom of the page, one at a time.  In previous installations, I had saved such commands in a text file and had executed it, but that had made it easier to overlook error messages.  In this case, the Medibuntu command gave me "Unable to locate package medibuntu-keyring."  A search indicated that virtually nobody had gotten precisely that message.  Not a good sign.  A community documentation page gave me a different command to add Medibuntu:

sudo wget --output-document=/etc/apt/sources.list.d/medibuntu.list http://www.medibuntu.org/sources.list.d/$(lsb_release -cs).list && sudo apt-get --quiet update && sudo apt-get --yes --quiet --allow-unauthenticated install medibuntu-keyring && sudo apt-get --quiet update

... all on one line!  It also said that Medibuntu's repository is deactivated whenever you upgrade to a newer Ubuntu release, so this reinstallation would have to happen each time.  I copied and pasted that command and ran it.  It gave me an error message:

Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?

to which the answer was yes, of course, Synaptic is still running.  I shut down Synaptic and tried again.  This time, it ran.  I took another look at the Authentication tab in Synaptic (see above), and now Medibuntu was on the list of trusted software providers.  So the answer to my question was, this was my customized list.

I went back to the process of running commands generated by the Ubuntu Sources List Generator.  They ran without further difficulty.  I typed "sudo gedit /etc/apt/sources.list" again, and replaced its contents with the lines generated by that Generator.  I took another look at Synaptic's Authentication tab, and this produced an error indicating that I had duplicate entries for Medibuntu and, I think, something else.  But when I tried to figure out what that was about, the error went away, and I didn't seem to be able to get it back.  Anyway, the Authentication tab did show that I now had the five third-party repos that had interested me (above), so all that remained was to go into System > Administration > Update Manager > Check and download the additional updates that it detected.

Installing Programs & Other Adjustments

In Terminal, I navigated to my Saved Settings folder (basically, cd "/folder name/" -- using quotes because "folder name" has a space in it) and verified that I had a copy of the installed-software file that I had created, in my previous installation, by typing "dpkg --get-selections > installed-software."  Now that I wanted to restore the programs listed in installed-software, I entered these commands:

sudo dpkg --set-selections < installed-software
sudo apt-get install dselect
sudo dselect

That opened up dselect.  The dpkg command had provided the list of what I wanted to install, so all I had to do now, in dselect, was to arrow down and hit Enter at the Install option.  This gave me an option of installing a large quantity of stuff, and I said OK, do it.  That took an hour.  At the end, I declined to let it erase previously downloaded .deb files, when it asked.  The last time I used this installed-software approach, I got a bunch of errors after reboot, and had to work back through the process manually.  There were some errors this time, too (see below), but they did not appear to be related to the dselect process.

As before, I navigated to the folders containing other program downloads (with .deb, .gz, .bin, and .bundle extensions).  I typed "sudo sh [filename]" to install my .bin and .bundle downloads (e.g., GoogleEarthLinux.bin).  Double-clicking on the filename in Nautilus no longer installed my .deb downloads; instead, I had to right-click and choose "Open with GDebi Package Installer."  In the case of my Synology software, I typed "sudo sh install.sh."  I wasn't sure where to install it, so I told it to install in /home, just in case that would spare me from having to install it again.  But I should have said /home/ray.  When I was done with all installations, I went into Update Manager, and ran and reran it until I was all caught up.

GRUB2 menu edits were the same as before:  to get rid of the Memtest+ options, I typed "sudo chmod -x /etc/grub.d/20_memtest86+." To let Ubuntu remember which operating system it had used last, I typed "sudo gedit /etc/default/grub," changed the first line to be "GRUB_DEFAULT=saved," and added a second line that said "GRUB_SAVEDEFAULT=true."  To limit the number of Ubuntu kernels shown, I typed "sudo gedit /etc/grub.d/10_linux," added "GRUB_DISABLE_LINUX_RECOVERY=true" at the top, and changed two lines at the bottom to be three that read as follows:

list=`echo $list | tr ' ' '\n' | grep -vx $linux | tr '\n' ' '`
list=`version_find_latest $list`
done

I saved and closed that and typed "sudo update-grub."

I also needed to make some adjustments for VMware.  First, I typed "sudo vmware" and made some root adjustments there.  As before, I typed "sudo gedit [path][filename].vmx," for the .vmx file pertaining to this VM; and at the end of that file I added a line that said this:

bios.bootDelay = "10000"

and that bought me ten seconds instead of one or two, when that vmware logo came up.  In a variation from Ubuntu 10.04, the restricted drivers for my monitor were now at System > Administration > Additional Drivers.

Bugs and Other Problems

There were some problems.  First, BOINC would not suspend itself when the system was in use, so I just suspended it, period.  Also, as discussed in a separate post, there were some Firefox errors.  The solution there was to completely uninstall and reinstall Firefox, though possibly it would have been sufficient just to uninstall firefox-gnome-support and delete my profile.

Another problem was that Ubuntu was not clearly recognizing all local and network partitions.  The problem of mapping the drive in my Synology NAS required another separate post.  In that case, it came down to a problem with the line used to mount the drive in fstab.  It had seemed like there were other drive recognition problems, but evidently they sorted themselves out, or perhaps I was just mistaken.  At this point, the drives seemed to be recognized in good form.

There was also the problem that GParted wouldn't run.  It wasn't just in my installation; it wasn't running when I booted from the live CD either.  I guessed that this was some kind of brand-new bug in Ubuntu 10.10 that would be fixed shortly.  When I started GParted from System > Administration, it would start up, but then it would disappear after just a few seconds.  When I typed "sudo gparted" or "gksu gparted," it did the same thing, but it gave me an error message:

glibmm-ERROR **:
unhandled exception (type std::exception) in signal handler:
what: basicI_string::_S_create
aborting...

A search indicated that this was indeed a bug in Ubuntu.  It looked like a new release would be fixing the problem imminently.  Another problem:  Google Earth would not install.  I got this error message:

parser error : Document is empty
parser error: Start tag expected, '<' not found
Couldn't load 'setup.data/setup.xml'

The command I used was "sudo sh GoogleEarthLinux.bin."  The first response to it was "Verifying archive integrity... All good."  Just in case, I downloaded a replacement of GoogleEarthLinux.bin, but got the same result.  It looked like others had also had this problem.  It seemed to be another instance of Ubuntu 10.10 not yet having all the kinks worked out.  I started with a lengthy thread on the issue.  One post in that thread recommended a command-line alternative, which in full form went like this:

sudo apt-get install googleearth-package
sudo make-googleearth-package --force
sudo dpkg -i googleearth_5.2.1.1588+0.5.7-1_i386.deb

I tried that.  The make-googleearth-package command generated a lot of errors that included the sentence, "Can't extract name and version from library name."  I got that long package name shown in the last line (googleearth_5.2.1 etc.) from one of the last lines produced by the make-googleearth-package command:  it said this was the name of the package it was building.  (It also seemed to say that simply "googleearth" was the name of the package, but that didn't work.)  The third command (sudo dpkg etc.) seemed to run successfully.  There was also now a Google Earth icon in Applications > Internet > Google Earth.  I clicked on that, and it worked.

Another problem involved VMCI Sockets.  I have addressed that one in a separate post.  I ran out of time to continue this project at the time.  When I returned to it two months later, I had decided to stop trying to maintain a primarily Ubuntu machine, but instead to return to Windows 7 for the foreseeable future.

Ubuntu Do-It-Yourself Network Attached Storage (DIY NAS): A Preliminary Look

I had just done an enormous amount of work in an unsuccessful attempt to get a Synology DS109 network attached storage (NAS) device to work in my Ubuntu 10.04 system.  It was an expensive unit, on my budget, even if it was near the bottom of Synology's line of products, so I was hoping for good things from it.  But I was not able to figure out why it wasn't connecting with my computer.  So while waiting for a reply from Synology's tech support, I did a search for do-it-yourself (DIY) alternatives.  This post describes a bit of what I found.

What I was hoping to find, in a DIY alternative, may be summarized in this excerpt from the previous post:

I decided to go ahead with the Synology project for now; I could return or sell the device if it really wasn't what I wanted.  I probably could have assembled another computer at equal or lower cost, with far greater potential storage capacity, with more RAID options, with a more powerful processor (for e.g., checksum calculations) if needed, with what might prove to be more options in the choice of software packages and commands to manage and adjust it, and with more flexible hardware troubleshooting options (i.e., more than just fix it or replace it) in the event of malfunction.  Its drawbacks would include time and expense for software and hardware selection, learning, installation, maintenance, and troubleshooting; physical space requirements; power consumption; and noise and heat generation.

If the Synology unit had been easy to use, I wouldn't have been able to generate that list of potential advantages of a DIY alternative.  I accumulated that list during the process of writing up the hassles I was having.  So now it was a question of how true those observations really were.

One thing for sure:  if I had taken seriously the idea of building a NAS myself at the outset, I would not have bought the Synology unit.  I did, in fact, have an old computer sitting around, one that I rarely used, mostly just for troubleshooting random hardware problems.  I was willing to convert it to another function. So, right there, I did have much of the hardware that I would need.  Its case design would accommodate a number of drives, if I decided to build a RAID NAS that would require that.

That took care of some of the objections to a DIY NAS.  What about space, noise, heat, and power?  There was no comparison:  the Synology was such a cool, quiet, sleek little thing compared to that whole computer case and its monitor and keyboard.  Even if I put the peripherals in a drawer and managed the server remotely (assuming that was possible), there was still the noise from its fans and power supply.  The trade-up in expandability and flexibility (with e.g., RAID) would come at a cost.  And that was the real question:  did I want the additional capabilities badly enough to accept the drawbacks?

Then again, it occurred to me that I didn't absolutely have to have the server remain on constantly.  Couldn't it hibernate during slack periods?  For noise reduction, couldn't I park it in a closet?  This seemed like something worth experimenting with.

What about hardware hassles?  I could probably just plug in the drives and, optionally, a RAID controller and be done.  The Synology unit did not have a RAID option, and it did not come with a drive.  The DIY NAS would have more things that could fail, but failures did not happen often, and with an optional RAID setup it would be better equipped to absorb them.  In terms of hardware, the DIY NAS was the winner.

And software?  The Synology software package had a nice GUI that, for all its good looks, had managed to confuse me and had thus failed to deliver a working solution.  If the Synology people came through on my tech support request, I would be ahead of the game there; but my request had been so broad and confused that I doubted they would be able to help much.  If it was a matter of devoting another five or ten hours, I was leaning toward trying something new, rather than beating my head against the wall with more efforts to understand the Synology software.

I decided, though, that I had probably better take a closer look at the software I would be using.  Ubuntu came with software RAID, and I had already had a bit of experience with that.  But what about NAS -- what software would I use for that?  I had gotten the impression that NFS was a good alternative to Samba for a Linux network -- simpler and faster, but possibly not as secure and apparently not as good for interfacing with Windows machines.  The Synology unit used NFS.  But this didn't really answer the question.  What software -- really, what operating system -- would I use to manage the DIY NAS?  Sources cited on the Wikipedia page for FreeNAS made it sound like a great solution.  It was apparently UNIX-based, and thus was some kind of cousin to Ubuntu, but it seemed I would have to invest some additional time to learn FreeNAS.  Ubuntu Server seemed like a more familiar, better-supported, and more flexible alternative.  Openfiler was another possibility.  A lot of people swore by Windows Home Server; I just didn't want to pay $100 for the privilege of going back in the direction of relying completely on Microsoft software.  Going in a different direction, there also seemed to be routers with hard drive connections, though I wasn't fond of the USB connection offered by the one I saw.

Those thoughts led to a thread that made me ask whether I wanted an NAS or a home server.  The tradeoffs described above were echoed in another thread I found, but now the discussion had mutated a bit.  If I was going to use a computer as an NAS, it seemed I might as well make it a home server.  In fact, that's probably how I had been thinking of it anyway:  a machine that could do whatever a computer could do, including RAID as well as NAS.

By this point, a couple of things had happened.  I had downloaded a copy of Ubuntu Server and had decided to try setting it up on that old computer, to see how easy it would make things for me in terms of network storage and RAID options.  I had read enough webpages to persuade me that setting up a home server could be somewhat time-consuming and thus would ideally be approached, for my purposes, as a longer-term project:  install the operating system, tinker with the hardware, and gradually move toward comfort with its maintenance and other requirements.  And I had heard back from the Synology people via email, not once, but twice in the space of three hours in the evening, East Coast time.  They were offering to set up a TeamViewer session with me for the next day.  So my working plan, at this point, was to get the Synology going, if I could do so the next day.

Improving Performance in VMware Workstation 7.1

I reviewed the latest version of VMware's document, Performance Best Practices for VMware Workstation, to see what hardware purchases or sales it would suggest for my situation.  The document consisted of four main sections, pertaining to host system hardware, the host operating system (OS), VMware Workstation and virtual machines (VMs), and guest OSs.  I was particularly interested in information about running Windows XP on an Ubuntu host, since that was the setup I was using.  This post does not say much about Windows host systems.

Section 1:  Hardware

A.  CPUs

1.  Hyperthreading

VMware (p. 7) recommended using a CPU that would support hyperthreading (also called "logical processing").  (The OS and the BIOS would have to support it, and the user would have to make sure it was enabled in the BIOS.)  Patrick Schmid at Tom's Hardware said that the primary benefit of hyperthreading was to permit smoother responsiveness, but that it would not yield noticeable increases in performance otherwise, and certainly would not substitute for having multiple cores in the CPU.  Intel's own writeup of hyperthreading affirmed that responsiveness was a leading benefit.

AMD quoted VMware as saying, “Virtual machines are preferentially scheduled on two different cores rather than on two logical processors on the same core.”  That is, VMware tried to assign different VMs to different CPU cores, if available.  This seemed to imply that AMD CPUs would do better when the number of CPU cores matched or exceeded the number of VMs being run.  But AMD suggested that increasingly complex software (e.g., multithreading in Microsoft Excel 2007) could keep as many as 48 CPU cores busy, even if the number of VMs being run was much lower.

AMD's point in that particular article was that its Opteron CPU, with more cores, could significantly outperform Intel's Xeon, with hyperthreading and fewer cores.  Anandtech's comparison of state-of-the-art Intel and AMD CPUs in March 2010 found, however, that the Xeon did much better than the Opteron.  Recent observations suggested that AMD might be moving toward implementing hyperthreading after all.

A search on Newegg.com turned up 20 Intel CPUs with hyper-threading capabilities, starting at $115 and ranging above $1,000.  (The least expensive Intel CPU listed on Newegg at that point cost $41.)  Anandtech said that the AMD advantage was in terms of price, with good performance at much lower cost.  One Anantech commentator said, "The twelve-core AMD Opteron 6100 and six-core Xeon 5600 perform more or less the same," but suggested that Intel had two advantages at the enterprise level:  RAS (i.e., reliability, availability, and serviceability, including the ability of systems to heal themselves) and licensing.

2.  MMU Virtualization

VMware (pp. 7-8) also expressed a preference for second-generation hardware-assisted MMU virtualization, called rapid virtualization indexing (RVI) or nested page tables (NPT) in AMD processors or extended page tables (EPT) in Intel processors.  (Wikipedia indicated that NPT was used during development, but that RVI was the term currently used.)

VMware found that, in its ESX product, AMD's "RVI provides performance gains of up to 42% for MMU-intensive benchmarks and up to 500% for MMU-intensive microbenchmarks."  VMware found similarly dramatic performance improvements for Intel's EPT, provided the virtualization product made suitable adjustments -- which, VMware said, ESX did.  It was not clear that the same could be said for VMware Workstation.  Pending further research, this information made an AMD CPU with RVI the more certain performance boost for an ordinary user of Workstation.

At this writing, neither Newegg nor TigerDirect offered products featuring any of those CPU-related acronyms.  According to Wikipedia, MMU debuted in the third-generation AMD Opteron, and at Intel EPT debuted in the Nehalem architecture.  (That same Wikipedia page said that RVI was supported, at VMware, in ESX Server 3.5 and later -- and also, interestingly, in Oracle's VirtualBox 2.0 and later.)  At Newegg, at this writing, Opterons were available in the range of $190-1,300 (and would require motherboards in the $200-600 range).  The Nehalem appeared in the Core i7 line of CPUs, available at Newegg for $290-1,140.  (Newegg didn't list a canned search option for Nehalem or Westmere cores.)

I looked at some historical prices to get a rough idea of how processor pricing trends worked.  On the Intel side, the Core 2 Duo E6700, introduced in July 2006 for $530, was apparently available (in some form) for $316 in June 2007, around $212 in July 2008, $130 in September 2009, and $95 in July 2010.  These values suggested that prices dropped dramatically (perhaps 40%) in the first year, less dramatically (perhaps 20% of the original price) in the second year, and likewise (perhaps 10% of the original price per year) over the next couple of years.  (Intel apparently discontinued the E6700 (presumably meaning that manufacturing ceased) in February 2008.  At that time, the chip may have been selling for somewhat less than half the original price.)  On those data, the rate of discount from the original price was cut in half in each succeeding year, during the first several years of the product's life.

I took a particular interest in one of the Core i7 CPUs at the bottom of Newegg's list, pricewise.  The Core i7-870 that was available for $290 in July 2010 debuted at a list price of $562 in September 2009, representing a 49% drop in less than a year.  The data from the preceding paragraph suggest that the consumer might anticipate another 25% reduction from the original price (i.e., half of the previous year's price cut), for a price of around $145, by summer 2011.  On this basis, it seemed to me, personally, that I might thus save myself $150 (plus whatever price drop might apply to the corresponding motherboard) if I waited to implement these particular suggestions for VMware performance until summer 2011.

Intel described the Core i7-870 as having both hyper-threading and VT-x virtualization technology.  But VMware (p. 8) indicates that VT-x is the first-, not the second-, generation of virtualization technology.  Its potentially outdated status is reflected in a VirtualBox recommendation that VirtualBox has been designed to perform better without enabling this sort of hardware-assisted CPU virtualization at all.  As of late 2008, someone in a VMware Community post considered VT-x a major step forward, but noted that hardware-assisted virtualization in Workstation 7 was supported only on 64-bit hardware.  I did have 64-bit hardware, so that was not a concern for me.

But which Intel CPU would I have to be tracking, if I wished to get into the second-generation Intel EPT (or AMD RVI) MMU virtualization technology?  Intel characterized EPT as an "extension" of VT-x and, to revert to the (Wikipedia) observation offered above, that extension was apparently to be found on the Nehalem (45nm) or Westmere (32nm) architectures.  Evidently not all Core i7 CPUs employed that architecture, then, else the i7-870 would have it.  (I was not alone in being confused about this.)  It seemed that what I was looking for might be, in Intel-speak, VT-x2.  Further searches for insight led to a simple request for a list of VT-x2 features implemented in various Core i7 CPUs -- to which Intel provided the bizarre response that, no, actually, it was hard to provide any such list, and a pointer to lengthy software developer's manuals.  Indeed, it seemed that VMware was somewhat behind the curve:  while it was talking about EPT (as implemented in VT-x2), Intel was meanwhile moving on to VT-d and other technologies.  Then again, another Intel source seemed to say that VT-d was an older technology.

The message seemed to be that I, as a consumer, didn't need to know about this yet.  I decided to try a different approach.  I went back to Newegg's list of Core i7 processors and tried working my way up the list until I found one that did have VT-x2.  After the i7 860 and 870, next on Newegg's list was the 930.  My search regarding the 930 and VT-x2 led to an Intel Virtualization Technology List indicating, that, well, yes, a number of Intel CPUs did support VT-x.  I looked at them individually to see if perchance they supported VT-x2, that information unfortunately not being included in the alleged virtualization technology list.  Bottom man on this list was, again, the 860, and they confirmed that it did support both VT-x and VT-d.  At the top end of the set, we had the 970.  The 970's spec sheet didn't say anything about VT-d, so maybe it was indeed being phased out.  No mention of VT-x2 either; just VT-x.  Following some leads, I came around to the discovery that there was also something known as VT-i, referring to the Itanium processor.  It wasn't helpful information, but at least it was information.

Looking back at that page on the i7-970, I noticed that Intel said, in greyed-out letters, "No Datasheet Available."  But, hmm, did that mean there were datasheets for others on that virtualization technology list?  I tried the 920.  There, they had a "Download Datasheet" link that led to about a dozen Technical Documents.  I started with the 96-page Intel® Core™ i7-900 Desktop Processor Extreme Edition Series and Intel® Core™ i7-900 Desktop Processor Series Datasheet, Volume 1.  But no, according to Acrobat, there were no references to VT-x2 there.  How about VT-x?  Nope.  Alright, then, volume 2?  None!  Well, how about just plain old "virtual"?  Still nothing on what virtualization technology any particular CPU might have.  This was a contrast against another set of technical documents provided on that same page, for the i7-800 series.  Here, I found references to both VT-x and VT-d.  Volume 1 of that datasheet said, on page 29, that the i7-800 series did support EPT.  So that was pretty confusing.

I decided to try the Developer's Manuals.  The description mentioned virtualization only in connection with the Intel® Virtualization Technology FlexMigration (Intel® VT FlexMigration) Application Note and the Intel® 64 and IA-32 Architectures Software Developer's Manual Volume 3B: System Programming Guide.  The Application Note contained several references to VT-x, but did not distinguish it from VT-x2 or VT-d.  Volume 3B of the Software Developer's Manual contained no references to VT- of any type.  Both documents did refer to Virtual Machine Extensions (VMX), and the Manual contained lots of information on how virtualization works.  But I was not able to figure out, from this information, which CPU I should buy.  This was pretty strange, given the conclusion that Intel's whole reason for offering virtualization in only some CPUs was driven by marketing.

It occured to me that perhaps the people at VirtualBox would provide some insight into what they would recommend, if I opted for a VirtualBox-compatible CPU.  A search produced very meager results along these lines.  I went to the VirtualBox website and looked at their documentation.  They said that "the vast majority of today's 64-bit and multicore CPUs ship with hardware virtualization."  No distinction there between VT-x and VT-x2.  They also said, "The biggest difference between VT-x and AMD-V is that AMD-V provides a more complete virtualization environment."  The use of what they called "nested paging" (i.e., more advanced virtualization, apparently what others meant when they referred to VT-x2) could bring a "significant" performance improvement -- of up to 5%.  Five percent!  I was thinking we were talking about the difference between success and failure, and now it appeared this might be just one more incremental improvement.  Nested paging, they said, was standard on Intel's Core i7 (Nehalem) CPUs, and also on AMD's Barcelona CPUs.

I did finally find, at Tom's Hardware, a list of CPUs that would support "XP Mode" Virtualization.  XP Mode was the capability of running a near-perfect emulation of Windows XP within Windows 7 (which would enable people to use older applications on the newer operating system).  In March 2010, Microsoft altered Windows 7 so that it would no longer require hardware virtualization in order to provide XP Mode.  But the Tom's Hardware list dated from a year earlier, so it gave an idea of what Intel CPUs I would need to consider if I wanted hardware virtualization for purposes of improved performance in VMware.  The Tom's Hardware list actually drew from a list posted by Ed Bott on ZDNet.  Ed provided a list of Intel desktop and mobile CPUs.  His desktop list boiled down to the following, which I provide here, in ascending order according to their current prices according to Pricewatch.com (or, failing that, on Newegg or Amazon):

So, bearing in mind that these were approximate prices, a person dead-set on obtaining a virtualization-supporting Intel CPU for less than $100 would have more than a half-dozen to choose from.

It appeared, in other words, that we were no longer dealing in the rarified world of enterprise-level Xeon processors; we humble consumers were treating virtualization as a simple commodity.  Paying more would bring, not necessarily any improvements in virtualization per se, but rather in those other characteristics that people like in their CPUs, including hyperthreading.

In that case, I thought that perhaps I should take a look at AMD, just to be sure that I wasn't blowing off an already affordable option.  If we were forced to accept the simplistic conclusion that you should just be happy knowing you could get some kind of hardware virtualization with any Core i7 CPU, why not price any AMD CPU with AMD-V virtualization?  According to a simple statement from AMD, that meant almost any CPU that I would be looking at.  Here, comparable to the situation with Intel, a Newegg search for any desktop CPU with virtualization technology support gave me AMD Semprons for as low as $37.

I reflected on my current situation.  To improve VMware's performance, I was looking to replace an AMD Athlon 64 X2 5000+ CPU.  But that dual-core CPU, which was hot stuff four years earlier, did support virtualization already.  The question seemed to be, what kind of virtualization?  What they were offering now was AMD-V.  Seeing the amount of time I had already invested in this general line of questioning, I decided I should just assume that it was better than the virtualization of yesteryear, and that having it on a faster CPU would be better still.  It seemed, in short, that I might just upgrade to a somewhat more up-to-date CPU, without worrying much about understanding VMware's hardware suggestions.  VMware (p. 17) said that, if I did have hardware-assisted virtualization in my CPU, Workstation would typically set it up automatically, but there was the option of changing the default in VM > Settings > Hardware tab > Processors > Virtualization engine > Preferred mode.  They also said (p. 26) that, if the system was using MMU, performance would be best if VMI (i.e., software virtualization:  "virtual machine interface") was disabled (VM > Settings > Hardware tab > Processors > VMware kernel paravirtualization).  Mine was grayed out.  I assumed it was something I would have to set when the machine was powered down, or perhaps in root mode ("sudo vmware").  They also said, "No Microsoft operating systems support VMI," but I wasn't sure what the situation would be in the case of an Ubuntu host.

B.  Memory

VMware's recommentation on memory (p. 8) was just to make sure you had enough.

C.  Storage

VMware recommended (pp. 8-9) having enough disk storage space, but also emphasized making sure it was configured correctly.  They mentioned the potential for improved performance from RAID.  Browsing among various sources suggested, generally, that there could be significant performance improvements (and possibly greater performance smoothness) in a RAID 0 setup, where the program files were installed on two (or more) hard drives.  By contrast, it seemed to be generally agreed that a RAID array would make less of a performance difference in the handling of data files.

D.  Other Hardware

VMware offered suggestions about networking and hardware BIOS settings.  These recommendations were worth reviewing for some purposes, but did not seem to require any purchase decisions for my purposes.

E.  Summary

The Hardware section of Performance Best Practices for VMware Workstation left me with the impression that, all other things being equal, VMs will perform better on multiprocessor CPUs, and that hyperthreading is a plus.  I was not entirely able to penetrate the jargon about MMU virtualization; the general conclusion there seemed to be that I should shop for a CPU that supported a relatively recent generation of virtualization technology.  Assuming no bottlenecks due to inadequate RAM or disk storage space, the other main performance recommendation for my purposes was to use a striping RAID arrangement.

Section 2:  Host Operating System

This section contained virtually no relevant suggestions for Linux-based systems.

Section 3:  VMware Workstation and Virtual Machines

VMware said (p. 16) that most applications running in Workstation would perform nearly as well as in native Windows.  For the "small percentage of workloads" that would experience noticeable performance degradation, they had several CPU-related suggestions:

• Don't assign more of a load to the CPU than it can handle.
• Don't assign more CPU cores to a VM than it can use.
• Monitor CPU usage with the Linux "top" program.
• When using a single virtual CPU (vCPU), as I was likely to do, I would get better performance on an UP rather than SMP kernel or hardware abstraction layer (HAL).
• The guest operating system may not switch to the appropriate HAL if the CPU settings change later (p. 27).

They said that Windows operating systems (OSs) newer than XP would use the same HAL/kernel for both UP and SMP installations.  It sounded like that was not the case for WinXP, however.  Microsoft seemed to say that XP would detect the type of system and would install the correct HAL automatically.  They said this:

Microsoft does not support running a HAL other than the HAL that Windows Setup would typically install on the computer. For example, running a PIC HAL on an APIC computer is not supported. Although this configuration may appear to work, Microsoft does not test this configuration and you may have performance and interrupt issues. . . . Microsoft recommends that you switch HALs for troubleshooting purposes only or to workaround a hardware problem.

So the HAL issue seemed to be something to be aware of, in some situations, but not something of practical relevance for a user of Windows XP, Vista, or Windows 7.  I was curious which HAL was installed on my system, though.  As advised by Kelly's Korner, I went to Control Panel > System > Hardware > Device Manager > Computer.  On my native WinXP installation, it said ACPI Multiprocessor PC.  In a newly installed WinXP VM in Workstation set to use just one processor and one core, it said ACPI Uniprocessor PC.

VMware (p. 17) said that, if there were other VMs or programs running in the background, performance of a VM in the foreground would be noticeably better if the settings were changed in Workstation (i.e., not in any particular VM).  The advice was to go to Edit > Preferences > Priority, and set "Input grabbed" to High, and "Input ungrabbed" to Normal.  But Workstation gave me no such options.

According to VMware (p. 18), memory-related performance could be affected in several ways.  First, there needed to be enough RAM available to the host system for its own purposes.  My system had 6GB of RAM. Normally, some of that might have gone unrecognized by a 32-bit OS, but I was running a PAE-enabled kernel in 32-bit Ubuntu 10.04.  Ubuntu's Sysinfo reported that my system's total RAM was 6050 mebibytes (MiB) (i.e., about 6.3 billion bytes).  Running Workstation as root (i.e., "sudo vmware"), I had set RAM to 5000MB (by which Workstation presumably meant 5000 x 1 million), leaving more than 1GB of RAM for Ubuntu system operations and whatever programs I might be running in native Ubuntu.  I did not typically run many programs in Ubuntu.  So it seemed that I had allowed enough RAM for the system.  It did occur to me, though, that if I was going to run two distinct sessions of Workstation (as opposed to running two VMs within a single Workstation session), I might want to cut that 5000MB figure in half for each Workstation session.

VMware (p. 18) also advised that the best possible performance would come from requiring Workstation to "Fit all virtual machine memory into reserved host RAM" (Edit > Preferences > Memory tab > Additiaonal memory).  But they provided this caveat:

NOTE:  The setting described in this section affects only whether or not a virtual machine is allowed to start, not what happens after it starts . . . . After a virtual machine starts, other factors . . . [e.g., change of applications running in the host OS] can change.  In such situations, even if you selected the Fit all virtual machine memory into reserved host RAM option, virtual machine memory swapping might still occur.

Since I did most of my work in WinXP, the message to me seemed to be to make sure that there was enough RAM available to the Ubuntu host so that it would not need to be raiding the WinXP guests.  This was consistent with the advice of VMware (p. 19).  They warned particularly about host applications that lock memory.  While it did not apply to my configuration, it was also interesting that they recommended providing no more than 896MB to 32-bit Linux VMs.  To monitor what might be happening, they suggested checking for swap activity in the host and virtual machines.  Doing in this Linux, they said (p. 29), involved running "stat" to dispay the "swap counters," and verifying that both the si and so counters were near zero.  I wasn't sure that their remarks applied to the Ubuntu version of stat, though.  A search turned up a manual page that didn't say anything about swap.  That page said made me think that a different search, focusing on the bash shell, might be more illuminating.  But that turned up nothing.  This really did not seem to be something that the world was blogging about.  Eventually, it appeared that what we were really looking for was vmstat, for which a search produced a couple hundred hits.  Brian Tanaka recommended running "vmstat 5 10" to get an average impression of what was happening on the system.  That didn't work on my system, but the vmstat manpage led me to try "vmstat -a -n 5 10" and that gave me ten indications that si and so were at zero.  So I seemed to be OK there.

VMware (p. 29) also pointed toward a knowledgebase page about "excessive page faults generated by Windows applications."  To see if this was a problem, they suggested using Start > Run > perfmon. I tried that, inside a WinXP VM.  At the top center of the System Monitor graph, I clicked the + (Add Counters) button.  I got an error message:

System Monitor Control
At least one data sample is missing.  Data collection is taking longer than expected.  You might avoid this message by increasing the sample interval.
This message will not be shown again during this session.

I took that to be a statement that my VM was running very slowly, which was not surprising, because I had some very intensive processes going on elsewhere on the computer.  I okayed out of that message and, following the advice on that page fault webpage, proceeded to choose Memory as my performance object, selected Page Faults/sec as my counter, clicked Add.  To get an accurate sample, I considered the advice from their error message:  I clicked on the Properties icon along the top and thought about changing it to "Sample automatically every 2 [or 3] seconds."  But then I decided the one-second sample was ticking along OK, and left it at that.  I was seeing occasional spikes in page faults.  The webpage advised that I could trace this to a particular application by going back to the Add Counters button, making Process my performance object, and then choosing a process of particular interest.  I named one of the very intensive processes I had underway.  Sure enough, I got a line across the top of the graph, indicating that that process was accounting for 90-100 (percent?) of something related to page faults.  Very interesting.  So basically this seemed to be telling me that a process that I knew was soaking up a lot of system resources was, in fact, soaking up a lot of system resources.

VMware (pp. 20-21) discussed ways in which page sharing and memory trimming, intended to promote efficiency, could degrade performance in some instances.  My situation did not seem to fall into those kinds of situations, so I made no adjustments there.  They said that, of course, a local disk drive would be a faster home for a VM than would a network drive.  They provided other tips that they had also indicated somewhere during the VM setup process:  for best performance, use IDE rather than SCSI virtual disks, and preallocated rather than growable, and independent and persistent rather than nonpersistent, and don't use snapshots.  They also (p. 22) offered some suggestions that I hadn't encountered previously:  with the machine powered off, turn off debug mode (VM > Settings > Options tab > Advanced > Settings > Gather debugging information > None).  Other performance tips (p. 23):  run a general availability (GA) version of Workstation, not a debug or beta version.  Make sure you have designated the right operating system (VM > Settings > Options tab > General > Version).  Disconnect your optical drives from your VM until you need them (VM > Settings > Hardware > CD/DVD > uncheck Connect at Power On).

To sum up, section 3 of Performance Best Practices for VMware Workstation did provide a number of practical tips on how to adjust Workstation to run more efficiently.  I was not able to understand and apply all of them, and some (e.g., make sure you have enough RAM) were rather commonsense if not simply redundant.  What I derived from the discussion of cores was that, if I did get a new multicore processor, I should probably experiment, as I had done with my present CPU, to see how it performed with various numbers of cores assigned in Workstation.

Section 3:  Guest Operating System

In this section, VMware led off (p. 25) with suggestions:  make sure you're using a guest operating system that Workstation supports; keep VMware Tools updated; disable screen savers and animations; run backup and antivirus scans in off-peak hours; use a timekeeping utility suitable for the guest rather than the VMware Tools time-synchronization option.  VMware (p. 28) also referred to impacts on efficiency wrought by guest OS "idle loops."  It appeared that tweaking this would be painstaking and would likely yield minor effects.

VMware (p. 30) confusingly said, "It is best to use virtual SCSI hard disks in the virtual machine."  This differed from the installation process, which said (at least at one point) that IDE drives were recommended.  Bizarrely, VMware directed me to a Windows webpage dated December 4, 2001.  More promisingly, VMware also pointed toward their KB9645697 webpage regarding the splitting of large I/O requests into 64KB units.  The gist of their suggestion here was, "Changing the guest registry settings to issue larger block sizes can eliminate this splitting, thus enhancing performance" (p. 30).  The way to do that was sketched out on page 30 (section 2.2.6.1) of a PDF document entitled User's Guide: Fusion-MPT Device Management.  But in any case, this called for an edit of the registry setting HKLM\SYSTEM\CurrentControlSet\Services\Symmpi\Parameters\Device\MaximumSGList, and there was no such setting in my VM.

VMware (p. 30) also recommended that, if I did use IDE rather than SCSI virtual disks, I should make sure DMA access was enabled.  To do this, I went into Start > Run > devmgmt.msc > IDE ATA/ATAPI controllers > right-click on each channel > Advanced Settings tab > look at Current Transfer Mode.  If it says PIO, toggle the other box, Transfer Mode, between PIO and DMA to get Transfer Mode = DMA and Current Transfer Mode = DMA.

Another performance suggestion (pp. 30-31):  defragmentation.  Start by defragmenting the guest, then use VM > Settings > Hardware tab > Hard Disk > Utilities > Defragment, then defragment the host (not applicable in Linux hosts).  Defragment before creating linked clones or snapshots; afterwards is too late.  I was only creating independent clones, so this did not seem to apply.  Nonetheless, I did have a defrag utility in the WinXP guest.  Defragmentation in VMware itself had always been almost instant, when I had done it.

For network performance, VMware (p. 31) recommended using the VMXNET driver.  They noted, however, that that driver was installed automatically with VMware Tools.  There were a few other network performance suggestions in the document.  Since I was not having networking performance issues, I did not investigate these.  VMware (p. 32) also offered some other concluding, sensible suggestions (e.g., use general-availability software, not beta versions; make sure the latest version of VMware Tools is installed.  Here, again, the advice did not seem to apply.

Summary (for My Purposes) 

A single problem with hardware or software could seriously impair performance.  I did not attempt to scour the Performance Best Practices for VMware Workstation document for every possible thing that might be improved.  Rather, at least in this first pass through it, I was focused on big-picture items that sounded like they might have a great impact on the performance of my system.  The Hardware section led me to think especially about upgrading to a faster multiprocessor CPU, perhaps with hyperthreading, but in any event with a recent generation of virtualization technology, and also to switch to a striping RAID arrangement for my program files (presumably including both the Ubuntu host program partition and the partition on which I kept my VMs.  Other than that, improved performance in VMware Workstation appeared to be a matter of tuning a variety of settings, some of which were becoming obvious as I gained more experience, and some of which would come to mind only as I reviewed the pages of the document and/or of this post.