Installing Windows XP SP3 in VMware Workstation 7.1 on Ubuntu 10.04

I had decided to stay with VMware Workstation 7.1 for a while longer, adopting a wait-and-see strategy toward VirtualBox and whatever other virtualization developments might be underway.  A couple of years had passed since I had first installed Windows XP on VMware Workstation in Ubuntu.  My old virtual machines (VMs) were creaking and malfunctioning.  It was time to create a new WinXP VM.

I had obtained mixed results from efforts to use VMware Converter to build VMs from existing WinXP installations or other sources.  I decided to build a fresh installation, installing XP within a newly created VM.  I had already installed VMware Workstation.  Now I started it as root user by typing "sudo vmware," and then I went into Ubuntu's Applications > Accessories > Terminal.  In Workstation, I went into Edit > Preferences and adjusted the settings that would apply to all VMs, some of which could only be set by root.  In the Preferences > Workspace tab, I went with the default settings for the most part.  For the default location of my VMs, I chose a partition on a separate hard drive for better performance.  In the Display tab, I selected the three Autofit options.  I selected all of the Updates options.  In the Memory tab, I left about 1.5GB of RAM for the system; the rest was for VMs.

When I was done with the settings, I killed that session and restarted VMware as a normal user.  I created a new VM with these characteristics:

• I started with a 40GB independent, persistent SCSI VM, but ran into some problems when I tried to shrink it, and ultimately had to start over.  Since it was easier to grow a VM than to shrink it, I decided on 20GB to start.  Although I initially created this as a preallocated space, it seemed that clones (which I made from time to time as backup, as the installation and tweaking process went along) would default to being non-preallocated, and I decided that was actually better until things got settled, because non-preallocated drives were smaller and therefore easier to clone and to back up.
• A 4GB independent, persistent IDE virtual drive, within this VM, for the paging file (VM > Settings > Hardware > Add).  This, I discovered, was best created after WinXP was installed.  Otherwise, WinXP was quite capable of brainlessly installing itself into this little partition and leaving the 20GB partition unused, thereby providing yet another reason to start over from the beginning.  Also, this drive was best created when the VM was powered down; only a SCSI (not preferred) drive could be created while the machine was powered up.  Note also that, as soon as you start down this path, Workstation may create a miniature version of a file for such a hard drive, even if you then abort the process -- in which case your later attempts to create that file may trigger strange results, until you investigate and delete any such runt file.
• 1.5GB RAM.  I had opted for 32-bit Ubuntu and WinXP after numerous previous hassles with 64-bit systems.  The discovery of PAE-enabled kernels meant I could go above the ordinary 32-bit limit of 4GB of RAM, so I was comfortable with this 1.5GB allocation for a single VM.  I could have gone higher, but I had almost never reached the point of using even this much.
• One single-core CPU.
• Based on my own usage, I set "Don't automatically connect" for floppy, USB devices,  or printer.
• Power:  enter full screen mode after powering on; close after powering off.
• Shared folders:  always enabled, map as network drive, make read/write only those that needed to be written to in Windows.
• No AutoProtect.
• Guest isolation:  enable drag and drop; enable copy and paste; don't enable VMCI.
• VMware Tools Updates:  use application default (currently update automatically).

Then I inserted the Windows XP CD and installed Windows XP, using a slipstreamed CD with Service Pack 3 (SP3) on it.  In my first attempt at this, I had to deal with the problem of making the VM boot from the CD.  I had just learned how to do this.  First, in Ubuntu's Terminal, I typed "sudo gedit [path][filename].vmx," for the .vmx file pertaining to this VM, and then I went to the end of that file and added bios.bootDelay = "10000" and saved the edited file.  This gave me a ten-second delay on VMware's splash screen.  So then I had time, on reboot, to read the options and choose F2 for BIOS setup.  In BIOS setup, I chose Boot, moved the CD-ROM drive up to be first in the boot sequence, and then hit F10 to save and close.  Then I installed WinXP.  The process was completely automatic, this time; apparently the latest version of VMware Workstation was able to detect my settings from the underlying Ubuntu installation.

Once WinXP was done with its basic installation process, I right-clicked on the VMware Tools icon in the system tray (i.e., the lower-right-hand corner of the WinXP desktop), chose "Open VMware Tools," and selected all three items in the Options tab, and then closed that.  To get the Windows desktop to stretch all the way across the monitor, I used VMware's View > Stretch Guest (using the menu at the top of the screen), and then returned it to View > Autofit Guest, and for some reason that did it.  I had to map my network drives after listing them in Workstation's VM > Settings > Options > Shared Folders.  Then I turned to the process of tweaking my WinXP installation.  There are some additional VM-related notes in my post on that.  When I tried to open a PDF file from within the VM, I got a Default Host Application error message.  Another post discusses that problem.

I wanted to use Acronis TrueImage to make periodic images throughout the installation process, so as to capture each working state of the system in a single snapshot.  On a standalone WinXP installation, that would have been just a matter of inserting the Acronis CD and making the backup to a separate partition.  In the VM context, though, Acronis was only willing to recognize only those partitions that were defined as part of the VM.  What I did instead, then, was to power down the VM, use Nautilus to copy the entire VM's folder to an NTFS drive, boot Acronis, and make an image of that VM folder.  (I also made a .zip of it in 7zip, just in case.)

To get the system to pause before loading Windows within VMware, so that I would have time to make a decision to adjust the BIOS or choose boot devices, I edited the individual virtual machine as described in a previous post.  Basically, I set Workstation's VM > Settings > Hardware tab > CD/DVD > "Connect at power on" and "Use a physical device" (though I was intrigued by the "Use ISO image" option).  In Terminal, I typed "sudo gedit [path][filename].vmx," for the .vmx file pertaining to this VM; and at the end of that file I added a line that said this:

    bios.bootDelay = "10000"

and that bought me ten seconds instead of one or two, when that vmware logo came up.

*** NOTE ***

At this point, I stopped developing this post.  Several other system issues had to be taken care of first, and those were superseded by non-system tasks.  When I returned to this post several months later, my purpose was just to close it down.  I had decided, by that time, to stop working on VMware within Ubuntu.  The following fragments are the other notes I had left over, in incomplete form, when I ceased working on this post.

Fragmentary notes continue as follows:

*  *  *  *  *


At some point in the process, I started getting an error message when I was trying to shut down the WinXP VM:

End Program - VMwareUser.exe
This program is not responding.

I did a search and found that virtually nobody was having this problem.  That was not a good sign.  A different search suggested that lots of people were having problems of one sort or another with VMwareUser.exe.  That was not a good sign either.  I didn't have an answer for this problem at this time.

After adding another program, Windows Explorer began flashing, refreshing itself every couple of seconds, in what appeared to be a different network drive problem.  I have written up the process of solving that in a separate post.

After tweaking the WinXP installation, I began adding programs.  I installed Copernic Desktop Search as one of those programs.  VMware Workstation treated my data drives as network drives, and Copernic regarded network drives as a sign that I was in a workplace, so I could not use their free version with VMware.  I downloaded their professional version ($40 to buy) and used it on a trial basis.  It was not able to save its index to a network drive.  Its index, I heard somewhere, would be about 10% of the total size of the files being indexed.  So if I had 100GB worth of files, I would have a 10GB index.  I definitely did not want that kind of monster sitting on my drive C.  But there seemed to be no alternative.  I decided to add a third virtual hard drive (30GB max, not preallocated) to the VM, made it drive Q, and told Copernic to save its index there.  When Copernic was done with its indexing, that drive

The only tweak that didn't seem to work was the one where I went into Internet Explorer and tried to save its Temporary Internet Files folder to some drive other than C.  It wouldn't save to a network drive.  But it also wouldn't even save to the little drive that I had created to serve as a paging file.  Possibly there wasn't enough disk space available for it there.

In the interests of improving performance, I worked through a VMware document on that subject.

Ubuntu 10.04 Reinstallation: Another Go

I had previously installed Ubuntu 10.04 once or twice.  This post provides a synthesis of the process, done on a Windows XP dual-boot machine.

Things to Save

The first thing to think about was, What did I want to save from my previous installation?  I was already keeping my data in an NTFS partition that WinXP recognized as D:.  I had named that partition DATA in both operating systems (OSs).  So I did not have most of my data on the root (commonly referred to with a simple slash, "/") partition containing my Ubuntu program installation.  The main exception was what Ubuntu knows as the /home partition.  (This and other Ubuntu folders, being included on a partition formatted in ext3 or ext4, were invisible to Windows.)  The /home partition was the default location for user data files.  My own files -- documents, PDFs, etc. -- were no longer there, but Ubuntu would still save various settings and other things there.  So I decided to try create a new /home partition, and copy the old /home folder's contents to the new /home partition, before reinstalling Ubuntu.  That little effort became the subject of a separate post.

Just in case, I saved my open tabs in Firefox (Ubuntu) using an extension whose name was not visible to me at this writing -- called (I believe) CopyAllURLs.  I also used the FEBE addon to save my complete Firefox setup; and since FEBE had failed in the past, I used the InfoLister addon to save at least the list of addons that I had currently installed.  I had previously saved their settings separately, for those addons (e.g., Tab Mix Plus, Forecastfox) that allowed me to do so.  I saved all of these things to my data drive, not to the Ubuntu programs drive that I was about to wipe out.  All of these steps proved to be unnecessary:  the separate /home partition (below) saved my entire Firefox setup just as I had left it.

I started up Chrome and Opera, both of which I had installed on this machine, and saved a list of their open URLs.  This, too, was unnecessary, thanks to the separate /home partition.  I had already migrated my email and contacts from Thunderbird (Ubuntu) to the Windows portable version of Thunderbird.

There was another thing to save, but I forgot until too late.  I should have run the installed-software command to make a current list of the programs I had installed on Ubuntu.  Instead, I was going to have to use the older list that I had saved some months earlier, and any programs installed since then would have to be rediscovered and reinstalled one at a time.

I should also have saved /etc/fstab.  My set of hard drive partitions had mostly not changed.  I could have just edited that instead of starting over from scratch (below).

That was pretty much all that I needed to save from the old Ubuntu programs partition that I was about to wipe out.  Most of my work was done in Windows virtual machines running on VMware Workstation, and those were on a separate partition already.  So now it was time to begin the installation.

Installation from CD

This time, I installed Ubuntu 10.04 from scratch, using the CD.  Previous inquiries had suggested that the PAE-enabled kernel would be installed automatically when installing from the CD.  (PAE allowed 32-bit Ubuntu to access RAM well beyond the 4GB theoretical limit -- apparently up to 64GB.)  By right-clicking on individual items in the partition install screen during installation, I manually designated a 30GB root ("/") partition for programs.  The installer automatically detected my previously formatted 5GB swap partition.  It seemed that people were still having problems with ext4, so I formatted the root partition as ext3.  At reboot, GRUB did say that I was installing a PAE kernel, so that was good.

Installing Software

Now I ran the second half of the installed-software command (above) to restore the set of programs that I had installed on Ubuntu at various points.  This involved using Nautilus to put a copy of my installed-software list someplace accessible (I chose my Desktop), using "cd ~/Desktop" as the short way of getting to the desktop, and then typing the necessary commands:

sudo dpkg --set-selections < installed-software
sudo apt-get install dselect
sudo dselect

This gave me a menu, from which I chose option 3, "Install and upgrade wanted packages."  It did so.  It took about an hour.  During the process, it asked "Do you want to erase any previously downloaded .deb files?"  My search yielded no clear answer to this question.  I guessed, from a couple of things I saw in that search, that this was like the question that came up in some Windows installations (in Microsoft Office, perhaps), where you would have the option of keeping downloads on the hard drive to make future fixes or upgrades faster, at the expense of consuming additional disk space.  I tried the Yes option.  It paused for a minute and then put me back at the menu.  So I guess it deleted some .deb files.  Anyway, the installed packages included ntfs-config, so now I could run that ("sudo ntfs-config"), so that Ubuntu would recognize my NTFS (Windows) partitions and put appropriate entries in /etc/fstab.  So then /home would be recognized on reboot.

Unfortunately, something in the install-software process screwed up my system, leading to a "udevadm" error message and a whole separate post.  But then, when I rebooted, I had another problem.  I was looking at a nearly blank screen.  It just showed the name of Ubuntu across the top ("Ubuntu 10.04.1 LTS P4 tty1," where P4 was the name of my computer) and then gave me a simple login prompt:  "P4 login."  I wasn't sure what to search for, to solve this problem, so I booted with the alternative installation CD and chose the "Rescue a broken system" option.  Ultimately, it gave me two basic options:  either reinstall GRUB or execute a shell in my Ubuntu program partition (in my case, /dev/sdb8).  I had no idea what to type at the prompt, and I didn't seem to be having a GRUB problem, so I tried a search; but it turned up stuff dealing with monitor problems.  I tried another search and came across the idea that I could just type "startx" to open the graphical user interface (GUI), so I rebooted, entered my username and password at that prompt, and then typed startx.  But this gave me some error messages:  "Failed to open device," "No valid modes," and "Screen(s) found, but none have a usable configuration."

Basically, it seemed that the installed-software process had messed up my system.  So I started over, installing Ubuntu from scratch, and this time I didn't use the installed-software option.  I had to go back through the process of setting up my separate /home partition, and then I worked manually through the installed-software process.  That entailed setting up my repositories.  After that process, some programs were still not installed.  Drawing upon my previous installation and its follow-on comments (as well as notes in a few other posts), I used System > Administration > Synaptic Package Manager to install acroread, adobe-flashplugin, boinc, dvgrab, fdutils, gparted, mplayer, nautilus-open-terminal, ntfsprogs, p7zip-full, sysinfo, ubuntu-tweak, unetbootin, and webhttrack.  Then I installed several programs that I had downloaded separately, on the assumption they were not available via Synaptic.  These included VMware Workstation 7, which I installed, as I had learned to do, by typing "sh VMware-Workstation-Full-7.1.0-261024.i386.bundle" in the folder containing that downloaded bundle file.  The simple "sudo sh" prefix was also adequate for installing my separate .bin downloads (i.e., GoogleEarthLinux.bin).  I installed Beyond Compare, Opera, and Google Chrome by just double-clicking on the .deb downloads.

Settings and Adjustments

Creating the separate /home partition had preserved most of my Nautilus settings, but I still had to tweak a few of those manually.  It had not saved my monitor driver settings, so now I went into System > Administration > Hardware Drivers, searched for available drivers and, for my machine, found NVIDIA accelerated graphics driver (current version) > Activate. This, like other installation and activation steps, would work only when there was no other Synaptic or software installer running.  I rebooted to activate the hardware driver.  Unfortunately, I was still getting some bad ghosting.  Tightening my video cable connectors solved the problem.

Most of my newly installed programs did not require immediate configuration.  To configure BOINC, I went to Applications > System Tools > BOINC Manager > Next > Choose a Project > World Community Grid > Next, and then went into Advanced > Settings to adjust my preferences.

GRUB2 had lost the setting by which it would remember which operating system I had booted last, and would reboot that one instead of defaulting to Ubuntu, so I typed "sudo gedit /etc/default/grub"; I changed one line to say GRUB_DEFAULT=saved instead of GRUB_DEFAULT=0; I added another line, right after it, that said "GRUB_SAVEDEFAULT=true"; I saved and closed that file; and then I typed "sudo update-grub."

Following a previous post, I pruned the list of entries shown in the GRUB2 menu at bootup.  First, I typed "sudo gedit /etc/grub.d/10_linux" and added a line, immediately after the comments section at the top, that said "GRUB_DISABLE_LINUX_RECOVERY=true" (without quotes, as always).  That would prevent me from seeing the "recovery" versions of Linux kernels in that menu.  At the end of that same file, I searched for the place that had these two lines:

list=`echo $list | tr ' ' '\n' | grep -vx $linux | tr '\n' ' '`
done

and changed it by inserting another list line between those two:

list=`echo $list | tr ' ' '\n' | grep -vx $linux | tr '\n' ' '`
list=`version_find_latest $list`
done

(Note that backticks ( ` ) are not the same as single quotes ( ' ) in such files.)  This, I hoped, would show me just the one or two most recent Linux kernels.  Finally, I saved and closed 10_linux and modified its companion file with this command:  "sudo chmod -x /etc/grub.d/20_memtest86+."  This command would hopefully prevent me from seeing the memtest options in the GRUB2 menu.  I rebooted and confirmed that it worked.

Other settings did remain intact via the preserved /home partition.  Unlike the steps described in the previous installation and in another tweak post, I did not have to make manual adjustments to prevent icons for mounted drives from appearing on the desktop.  My previous list of programs that should run at startup remained intact.  Gedit was still my default crontab editor.  The permissions giving me access to various partitions as ordinary user, not root, were still in place.  My VMware settings seemed to have been preserved, except for what may have been a few new root-level settings in the new version of VMware Workstation that I had just installed.

When these steps were complete, I ran Update Manager one last time, rebooted, and confirmed that everything looked good.  Then I rebooted with a CD and made an Acronis disk image on a separate partition.  Done!

VMware Workstation 7: Improving Performance in 32-bit Ubuntu 9.10 (Karmic Koala)

I was in the process of tweaking an installation of x32 Ubuntu 9.10, and ran into some VMware performance issues.  Specifically, in VMware Workstation 7, I found that one of my virtual machines (VMs) performed much more slowly under 32-bit Ubuntu than it had performed under 64-bit.

To improve performance, I tried a couple of things.  I shut down the VM I was using, closed Workstation, typed "sudo vmware," and went into Workstation's VM > Settings > Hardware tab > Processors.  I had set it at 2, because I was using a dual-core processor, but I vaguely recalled that that had seemed slower at one point in the past, so I changed it to 1 processor.  I also increased Memory from 1000MB to 2000MB.  Also, while I was running Workstation as root, I went to Edit > Preferences > Memory.  I left Reserved Memory at 2500MB and changed Additional Memory to "Allow some virtual machine memory to be swapped," which as I recalled was the original setting.  I also thought that I might have been able to speed up the VM somewhat if it was smaller:  this particular one was 35GB, having been converted from a larger WinXP partition.  With these changes in place, I closed out of Workstation as root, started Workstation normally, and tried the VM again.  It was still very slow -- if anything, worse than before.  Some of this slowness could have been due to other hard drive activity.  I had recently installed Google Desktop, and it was indexing my hard drive throughout the time when I was writing this post.  But at this same time, another VM, set to 1 processor and only 1000MB of RAM, worked acceptably.

To speed things up further, there were numerous tweaks that would supposedly improve Ubuntu performance.  One speedup option was to buy a solid state drive (SSD).  This, it was said, would yield "a remarkable change in system performance - boot time and program startup times are drastically reduced. I highly recommend the investment - I have never before seen a single hardware change that affected performance as much as an SSD."  SSDs that would be large enough (30GB+) were currently at $150 and more, so that was something to think about for the future, as prices came down.  Getting an SSD would apparently call for some additional adjustments.

I ran a search to see if I could use the rest of my system's memory as a RAM disk.  This didn't seem to be a very common procedure.  But one post said it was simple.  The dominant approach seemed to be to tell Ubuntu to load its /tmp temporary files folder into RAM at startup.  There were different types of Ubuntu RAM disks.  The one called tmpfs seemed optimal.  The general idea seemed to be that you would set up tmpfs at a special location known as /dev/shm.  According to Kevin VanZonneveld, "The standard /dev/shm grows automatically as more space is needed, but is by default limited to half of your physical RAM. If you have 2GB, it can grow to 1GB at most."  There seemed to be ways to change that; but since I figured I'd typically have total RAM of 6GB to 8GB in my system, 50% of RAM seemed fine.  For future reference, there were ways to speed up Firefox by using tmpfs, but I hadn't noticed as much of a need for that, especially outside of the VMs.  What seemed most immediately promising, for my purposes, was to use /dev/shm for VMware specifically.  After some hunting, I found a website for VMware Server, but I wasn't sure whether I should try using its suggestions in Workstation.  I learned that KVM is an open source competitor with VMware but is still rather complex.  Then I noticed that someone said that a Physical Address Extension (PAE) aware kernel would allow 32-bit Ubuntu to use more than the (theoretically 4GB, but practically) 3GB RAM limitation.  I checked that out and concluded that 32-bit PAE performance was not going to be much better per se than plain 32-bit (and both would remain far below 64-bit performance), but neither would it hurt (unless I turned out to be one of those users who had problems with e.g., the mouse or graphics).  There was supposedly some performance reduction if you installed it in the standard way; the alternative was to build a custom kernel, which was beyond my time and abilities at that point.  I had just made an Acronis TrueImage backup, so I decided to try the standard approach, in its latest version.  First, I went to Synaptic and installed linux-generic-pae and linux-headers-generic-pae.  This did its thing for five minutes or so, and then said it needed to restart.  When the machine rebooted and got to GRUB, it showed me a new item:  "Ubuntu, Linux 2.6.31-19-generic-pae."  I selected that.  The BIOS told me, at boot, that I had 6GB of RAM.  When Ubuntu loaded, my previously opened windows were still in place (due to settings described in the previous post), but not VMware.  Instead, I had a dialog that said this:

VMware Kernel Module Updater

Before you can run VMware, several modules must be compiled and loaded into the running kernel.

So I went with that.  Meanwhile, I typed "free -m" and it reported that I did indeed have a total of 6050 of memory.  I wondered whether this would make any difference in VMware.  When the modules were done loading, I started VMware as root and increased the VM's allocation from 1000MB to 1500MB.  I also changed Additional Memory back to"Fit all virtual machine memory into reserved host RAM" and increased Reserved Memory to 5000MB of the 5472MB available to VMs.  Also, I changed the individual VMs to use 1500MB each.

Then I powered on the VM that had been working adequately before.  There was no question that it was much snappier.  With that one running, I powered up the other one, the one that had started the investigation.  So the additional memory definitely was available:  I had two machines running, where before I could only run one.  Now that I had two VMs up, the first VM, the better performer, was running more slowly.  The other one was still slow, but definitely was running better than before.  As before, the new Ubuntu kernel's audio started as muted, so I had to unmute it and turn it up, but then audio in both VMs worked.  There was stuttering in the second, larger VM.  To get the best performance, I turned off one VM and left just one running.  The memory enhancement had still not put me back to where I had been with 64-bit Ubuntu and 64-bit Workstation at their best, but if I got greater reliability and acceptable performance, that would be OK for now.  I did not pursue this further, though, because at about this time I began the process of upgrading to Ubuntu 10.04.