Windows 7 Installation: Chronology

On January 1, I began installing and learning about Windows 7 as an upgrade from Windows XP.  I installed just a few things -- antivirus, for example -- as I continued to research various items.  In the next several days, I reached decisions about assorted aspects of the emerging setup.  For example, I decided that Windows 7 would be my foundation layer, replacing Ubuntu, and that I would use VMware Player (or Workstation, if possible) to run Windows XP, and possibly Ubuntu or other operating systems, in virtual machines running on Windows 7.

On January 4, I awoke to find that Internet Explorer (IE) had frozen up and had to be restarted.  I was pleased to see that, when it restarted, it saved my open tabs and attempted to reopen them.  But the thought that Internet Explorer could crash when it was doing almost nothing on a new installation was a bit of a setback for the idea that Win7 was stable enough to serve as a foundation.  I did continue to use IE, but I also downloaded Google Chrome, and made that my default web browser.

I spent much of the days of January 5 and 6 trying to resolve networking problems.  January 7 and 8 were especially devoted to moving and rearranging partitions.  I was setting up two computers for somewhat different purposes.  Partitions were large, so these steps took a long time.  I had to be actively engaged to work with those network problems, but was more able to let the computer go for hours during the partitioning process.  I spent a lot of time on January 6 and 9 trying to make a RAID0 array work.

Some of these days were devoted to other projects, but this was all still very time-consuming.  As of January 11, I really had not gotten very far.  The soution to the networking problem, I hoped, would be to reinstall Win7 from scratch on one of the two computers I was trying to network.  I learned that it was not possible to restore an image of a Win7 installation to a Win7 software RAID0 partition.  Hence, on January 12 I looked into other RAID options.  That day, I also discovered the possibility of booting into native virtual hard disks (VHDs), and spent a lot of time learning about those.

On January 13, I reviewed progress in light of what I had learned and experienced to this point.  I decided to scale back to a simpler system.  This meant the end of the VHD pursuit, for at least the time being, and probably also the RAID0 arrangement, possibly in favor of a future RAID5 solution.  The Win7 system was performing much faster than WinXP had been, either natively or in VMware on Ubuntu, so I was not presently sure that I would even need RAID-like speed.

To consolidate my progress, I wrapped up the first comprehensive post describing my installation of Windows 7, and started a second post and a second installation.  After further networking hassles, that installation was done on January 17.  At this point, I moved beyond basic connectivity to network enhancement.  One step was to synchronize the computers.  I spent much of January 17 and 18 researching programs that would do that and putting those programs into effect.  This involved getting a parallel Windows installation running on the second computer.  By midday on January 18, and intermittently thereafter, I was working more on new additions, especially Input Director and Teamviewer, to tie the two computers together into one workspace.

Input Director seemed to produce unexpected problems requiring a reinstallation of a backup image on computer B on January 19.  I spent a considerable amount of time, that day, reconfiguring that reinstallation, refining it with additional tweaks (and putting those tweaks on computer A as well), and saving it to an updated image.  From then through January 21, I was gradually mixing in more of my other work while continuing to work through minor problems, further adjust the system, and (in one case) recover from an error involving the accidental deletion of a large number of files.

After January 21, I focused increasingly on getting my actual work done.  There continued to be tweaks and the occasional stray problem that needed fixing.  These actually consumed a fair amount of time, but not nearly as much as before.  By January 25, I had reached a point of being able to focus almost entirely on getting work done.  I hoped that this large investment of time would not only have been interesting (at times) for its own purposes, but would also have productivity payoffs, for me and others, for years to come.  I had been using Windows XP for years and, throughout 2010, had seen how much time I could lose to a slow operating system.  So far, Windows 7 seemed to be a significant improvement.

Windows 7: Native Virtual Hard Disk (VHD) Boot

I was interested in making Windows 7 Ultimate run faster.  One possibility that came to my attention was using native virtual hard disk (VHD) boot in connection with RAID0.  This post discusses that.

A virtual hard disk was just a hard disk in virtual form.  Normally hard disks were physical.  A virtual hard disk, like a virtual machine (VM), would exist as a file (or a set of files) on a physical hard drive, not as a physical device in itself.  In other words, the user would have a physical hard drive in his/her computer, and somewhere on that drive s/he would have a VHD.

The "native" part of "native VHD boot" meant that the user would have direct access to the VHD.  Ordinary VM software (e.g., VMware, VirtualBox, Virtual PC) would require the user to install an operating system (e.g., Windows 7, Ubuntu) to serve as host (also called "parent").  Then the user would have to install the VM software (e.g., VMware) to manage the guest, and then install another operating system (e.g., Windows XP) inside the VM software as guest.

The VHD concept seemed to be that the host layer would be removed from the equation.  There would be a virtual hard drive that could be loaded or unloaded, perhaps without rebooting the computer.  The computer would be running the "guest" machine (e.g., a Windows XP installation) in virtual form.  For example, a packaged Windows XP installation could run on computer A, or on computer B, without having to be reinstalled and without having to install a host and a VM program like VMware.  In addition to simplicity, this scheme would presumably have the advantage of improving performance by removing extra layers and tasks.

In Windows 7, native VHD boot was apparently available only in the Ultimate and Enterprise versions.  This meant that it could be out of the reach of ordinary users who were not inclined to spring for such expensive software, unless perhaps they had access to it through their jobs or through academic versions.  It did not appear that Ubuntu, for one, would be developing a comparable capability in the near future.  There were hypervisors like VMware's ESXi that apparently provided somewhat similar bare-metal functionality, but ESXi seemed unavailable for my purposes.

I was interested in the possibility of using native VHD boot with RAID0.  This interest arose from a desire to make Windows 7 and other operating systems run quickly.  New installations of operating systems would typically perform well, I found, but then time and the addition of more programs and other demands would slow them down.  So, for example, I had found that Windows XP running as a VM inside VMware Workstation on an Ubuntu host was much more stable than native WinXP, but its performance had slowed dramatically.  My attempts to fix it (even through reinstalling both Ubuntu and VMware) had failed.  After several years of experimentation, I had therefore recently given up on Ubuntu for my purposes.

VHD seemed to offer a good way to run Windows 7 in a RAID array.  I had found that Win7 did not, itself, support booting from the software RAID0 arrays that Win7 was capable of creating.  In other words, a Win7 user could use Win7 to create RAID0 arrays, and could speed things up somewhat by putting some programs and data files on such arrays; but the user could not boot Win7 from a Win7 software RAID0 array, and would therefore have to install and run some Win7 program files from a basic hard drive.

As an alternative, the user could create a RAID array using hardware, either on the motherboard or on a third-party controller.  Hardware RAID had potential advantages of performance, bootability, level support (i.e., the ability to choose versions other than RAID0 and RAID1, including some I had not previously heard of), multiple operating system access (i.e., the data would remain available regardless of how the machine is booted), and reliability.  The performance story was a bit controversial.  I had seen indications that motherboard RAID was no faster than software RAID, and I had heard some claim that RAID0 was no faster than non-RAID, but my own experience (in Ubuntu) was consistent with tests indicating that even a software RAID0 was markedly faster than non-RAID.  Cost was a consideration.  While RAID5 was perhaps ideal, a Newegg search for a SATA RAID5-capable controller would cost hundreds of dollars.  Removing the RAID5 constraint led me to a $30 Rosewill item.  I didn't need that myself; I had motherboard RAID 0, 1, and 10.  For reasons of cost, noise, and heat, I wasn't going to be installing four drives to have RAID 10 (a/k/a RAID 1+0, which Wikipedia (apparently backwardly) called a stripe of mirrors, as distinguished from RAID 0+1, a mirror of stripes).  So for my purposes, it was a choice between hardware RAID0 or software RAID0; and since I was now using Win7 rather than Ubuntu, the desire to boot in RAID0 meant I would be using motherboard RAID -- unless VHD gave me a good alternative.

At this point, I had a general sense of what VHD sounded like.  Among other things, it sounded great.  But it seemed that I wouldn't run into the actual drawbacks and impossibilities until I created my own VHD and tried to use it.  As described in another post, the first step in doing so was just to create and play around a bit with a VHD, within a regular, running Win7 system.

Windows 7: Introduction to Virtual Hard Disks (VHDs)

As described in another post in this blog, I was trying to figure out the best way to install Windows 7 Ultimate.  I had discovered native virtual hard disk (VHD) booting.  It sounded like I could use a native VHD to boot Win7.  I wasn't sure precisely how that worked, so I decided to give it a whirl.  This post describes that process.

I started with Win7 already installed and booted from a basic hard drive.  I had a two-drive software RAID0 array created by Win7.  Actually, it was a set of several RAID0 arrays, of which one was left empty and unformatted.  This, I hoped, would be the virtual partition on which I would install a bootable version of Windows 7.  I had not been successful in doing so natively (i.e., directly, without intermediary software).  I hoped to do so with a native VHD -- that is, a virtual drive that would create a Win7-boot-friendly world within the Win7 RAID0 partition.

The first step was to run Disk Management (Start > type "diskmgmt.msc").  (There were also command-line options.)  I could see my eagerly waiting empty RAID0 partition.  On the menu, I went into Action > Create VHD > Browse.  To my surprise, Disk Management seemed willing to create the VHD just anywhere.  It made sense -- the VHD was going to be contained in a single file that would function like other files -- but I had expected something like a partitioning program here.

I designated a location and filename:  D:\Current\Win7VHD.vhd.  Having heard nasty things about the dynamically expanding option, I went with the fixed size default setting.  Since I had the general idea that I was going to be installing Win7 inside this file, it seemed this file would have to be monstrously large, raising a question of backup:  would a single change anywhere in this file require another mega-backup?  A search led to a thread advising an allocation of 60GB to 100GB for a Win7 partition.  Informed posts in another thread echoed that; likewise a third.  I could spare 100GB on my existing setup without moving partitions, so I set it at 90GB (i.e., 92160MB) just in case there was some funky slippage that nobody was telling me about.  Win7 informed me that it was installing VHD device driver software.

I checked D:\Current.  There it sat, Win7VHD.vhd, size = 94,371,841KB = 92160MB.  Now what?  Apparently one option was to just treat this VHD like a data drive.  In Disk Management, I went into the Action menu pick, with the idea of clicking on the Attach VHD option.  But it was grayed out.  Apparently I was missing something there.  I figured the myster would be resolved eventually.  And then, right before my very eyes, Disk Management changed.  Now it was displaying a new, 90GB unallocated space that it described as Disk 3, Unknown, Not Initialized.  The minutes that had passed since I created the 90GB VHD file were apparently spent actually creating that file in D:\Current.  Evidently this was like the file copying process in Windows Explorer, where sometimes it would say that it had actually gone ahead and copied your 15GB video file to another drive, but what it actually meant was that it was just starting that process, and the new copy would not really exist for a while yet.

Alright.  So now I selected Disk 3, my VHD thing, and went into Action.  Now Attach VHD was not greyed -- it was ready to go.  I clicked on it, to see what would happen next.  Apparently selecting Disk 3 hadn't accomplished anything:  Attach VHD just opened up a dialog asking me where was the file that I wanted to attach.  That made sense -- but then why did Disk Management already show this 90GB unallocated partition?  It seemed like it would have been more logical to show it once I had designated the file that I wanted to attach.  Anyway, I browsed to and selected D:\Current\Win7VHD.vhd and clicked Open > OK.  This gave me an error message:  "The virtual disk is already attached."  So maybe I was right the first time:  as soon as I created the 90GB VHD file, Win7 set aside a logical partition for it, which I was seeing as Disk 3 in Disk Management.  Evidently the Attach VHD option had no real meaning for my present purposes; evidently it had been grayed out because Win7 was then in the VHD-creation process.

So did this mean that Windows Explorer would now display a 90GB unallocated disk?  No.  Well, alright then, could I right-click on the Unallocated area of Disk 3 and format it or something?  No, all options were grayed out except Properties.  Beggars can't be choosers:  I went into that.  There wasn't really anything to adjust there, unless I wanted to turn off write-caching.  Ah, but I could right-click on the left end of the Disk 3 bar, there in Disk Management, and there I did find options:  Initialize Disk, Offline, Detach VHD, Properties, or Help.  It looked all set to let me go ahead and initialize that disk.  So then apparently I would have a 90GB drive, identified with some Windows drive letter -- Q, maybe -- and this drive Q would exist within a 90GB file located in D:\Current.  Kind of an interesting idea.  Probably I could detach it, then, and move it around to some other folder, or maybe even move it without remembering to detach it and thereby crash the system and lose the file and everything in it.  But no, I had faith in Microsoft -- surely they had already anticipated that and would have various protections in place.

It then occurred to me that I had made the drive 90GB because I had an unused 100GB space on my hard drive.  But I was thinking wrong.  The VHD hadn't been created in that unused space.  It had been created in drive D.  I checked the properties for that drive in Windows Explorer.  Sure enough, drive D was a lot more full than it had been 15 minutes ago.  Wherever I was going to be creating VHDs, I needed to make sure I had enough space to play with.  Preferably I would have a space set aside for them, so that I would create and keep them in one place, instead of having to move these gigantic 100GB lumps around all the time, from one drive to another.  I guessed that they could be stored in much smaller spaces on a compressed backup drive, but that trying to run a VHD from a compressed drive would be an extremely slow affair, if such a thing was even possible.

Finally, detaching the VHD gave me the scary option of deleting the disk file.  I was surprised Microsoft put that there.  Oops, my mistake:  there goes 100GB of data.  In this case, I went ahead with it, since I had nothing in the VHD; and for that purpose it was convenient.

So we had the concept of a VHD.  Now, what if I wanted to use a VHD, not just as a storage unit, but as the home of my Windows 7 program files?  In other words, I wanted to boot the system from a VHD.  But how?  That is the topic of a separate post.

Windows 7: Creating a Bootable Virtual Hard Disk (VHD)

As described in other posts in this blog, I was in the process of installing Windows 7.  I had discovered virtual hard disks (VHDs).  I was at the point of wanting to create a VHD that would boot Windows 7.  This post describes that creation process.

There appeared to be several ways to create a bootable Win7 VHD.  I could:

• install it from scratch, using the Windows 7 installation DVD; or
• convert an existing physical Win7 installation into a bootable VHD; or
• convert a Windows Imaging format (WIM) file to a bootable VHD; or
• deploy a WIM file into a VHD without conversion, and then boot the WIM file.

The fourth option was interesting, but I did not discover it until later.  The first option was tempting, insofar as it, too, would avoid a conversion process.  My prior experience in converting physical Windows XP machines to VMware virtual machines (VMs) using VMware Converter suggested that a conversion process might leave me with a buggy, poor-functioning VHD.

I chose to start with a conversion option nonetheless because I felt it offered some advantages over the approach of installing Win7 from scratch in the VHD.  For one thing, I had heard that a bootable Win7 VHD would not be upgradeable.  In that case, I would probably want to have a reference source -- a physical installation, that is -- that I could go back to, upgrade, and repackage into a new VHD.  Maintaining a reference physical installation could also mean that I could start with a smaller fixed-size VHD and then create a new, larger one as the size of the total Win7 installation grew, instead of taking what sounded like the more problematic approach of using a dynamically expanding VHD.  I also thought it would probably be handy, at times, to have that physical installation readily available, in case something went wrong with the VHD, or if it turned out that there were limitations on what the VHD could do.  That concern came from working with Windows XP VMs, in VMware, that were not able to access USB devices that a physical WinXP installation could access.  In addition, of course, a successful conversion strategy would save me the whole process of doing yet another installation from scratch.

As I looked into the conversion options more closely, it seemed there were a number of useful tools for the purpose.  WinImage was said to be a capable shareware, command-line tool that could create images in VHD format.  WIM2VHD was a freeware tool from Microsoft for the same purpose.  It appeared that Acronis True Image might be able to restore an image to a VHD.  Disk2VHD, another Microsoft offering, provided a way to create a VHD on a Win7 machine while it was running.  I preferred a GUI, if I could get it, and Acronis had recently disappointed me, so I narrowed down my choice to the two Microsoft tools.  Now it was just a question of whether I wanted to use WIM2VHD to convert a WIM to a VHD, or Disk2VHD to convert a running machine to a VHD.  I decided the latter was the more straightforward approach, so I started with that.

To create the VHD with Disk2VHD, I downloaded, unzipped, and ran Disk2VHD.  It was a portable program, so it did not need to be installed.  It recognized the partitions on my system, and asked which ones I wanted to put into the VHD, and where I wanted the VHD to be created.  For the target location, I designated a Win7 software RAID0 partition.  The source was a 70GB partition of which 32GB was used.  The target was a 100GB partition.  The process of creating the VHD (with source and target on different hard drives) took about five minutes.  The resulting VHD file was 26GB.  This, unfortunately, was not the right size.  It would not be big enough to accommodate any growth in the size of the Win7 installation.  It appeared that there were ways to expand it, but the methods I saw seemed to promise another layer of hassle, complexity, and potential failure. 

Back at the drawing board, WIM2VHD was advertised as creating sysprepped VHDs for native VHD booting.  It turned out that WIM2VHD was an automated way of doing the "deploy" (fourth) option listed above, so there were really only three options.  To run WIM2VHD, I had to download and install the 1.7GB Windows Automated Installation Kit.  I found a Microsoft website with authoritative information about VHDs.  At this point, however, I decided to postpone this project in favor of other pursuits.

Windows 7: Scaling Back the Ambitions

I was installing Windows 7.  I had spent nearly two weeks learning about it and tinkering with some of its possibilities.  I had learned quite a bit, and was impressed with much of what I saw.

At the same time, I was sobered by the prospect of spending additional weeks in this exploration.  It did appear that a commitment of that kind would be required to follow through with the Windows 7 projects I was looking into at that point.  Although this period of almost two weeks of tinkering and exploring had been brief compared to some periods of computer-related investigation that I had experienced in the past, it was still a large chunk of time, some of which I would rather have spent focusing on other things.

I had pulled back from my ongoing Ubuntu exploration, near the start of this project, because of its time requirements and the drawbacks I had experienced with it.  During these past two weeks, I had found that Windows 7, while being a sophisticated operating system, was still not entirely stable.  The plan of building a whole structure of RAID arrays and bootable virtual hard disks (VHDs) on Windows 7 was beginning to look like another indulgence that would continue to demand large amounts of time, not only to learn more about it, but also to maintain and repair it.

Certainly I could work toward setting up a RAID5 array eventually, as need and conditions dictated.  I could stay attuned to developments in the area of VHDs.  No doubt I would continue to refine my system in various ways.  But for purposes of my primary scholarly work, as distinct from various other projects of secondary interest, it was not essential to have RAID0 performance for my data partition or, with the aid of a bootable VHD, for my Win 7 boot partition.

As I pondered these thoughts, it seemed that my pullback from Ubuntu, a few weeks earlier, had been a sign of a change of direction with the arrival of the new year.  I liked improving my system and learning about these things.  Indeed, I believed I would like those sorts of activities more, if I approached them less ambitiously, with more patience to learn about and explain an occasional system development.

I still had some issues to work through, to achieve the more ordinary functionality that I did need.  Even those issues had the potential to consume another week or more.  But in the big picture, I felt that the time had probably come to scale back my system plans.  Accordingly, I decided to focus, perhaps for the coming year, on improving aspects of my system that seemed most crucial, particularly in the areas of reliability and trouble-free functionality.