Tuesday, December 29, 2009

Installing Ubuntu 9.04: External USB Drive

I was installing Ubuntu 9.04 (Jaunty Jackalope) in a Compaq CQ60-420US laptop computer from which I had removed the original Windows Vista installation. Although people generally advised against removing Vista and its recovery partition, I had several reasons for doing so: (1) I wanted to use Ubuntu (I would use WinXP in a virtual machine when I needed to run Windows-compatible software); (2) given the limited size of my laptop's hard drive, I could use the extra 10GB or so that was used by the factory-installed Vista recovery partition; and (3) Vista had a reputation of being slow, which was especially undesirable on the relatively underpowered hardware of a cheap laptop.

After I repartitioned and reformatted the hard drive, I found that I could not get the Ubuntu system to recognize the external USB hard drive. (That drive was not a USB flash drive; it was a regular internal SATA hard drive in a Rosewill RX-358-S SLV external enclosure.) First, I took a few steps, recommended in various posts, that did not seem to make much difference:
  1. Following some users, I typed (in Terminal) "sudo gedit /etc/modules" and then added "usb_storage" as a separate line at the end of that file, exit, and reboot.
  2. When that didn't solve the problem (though, for all I know, it may have helped), I typed "sudo mkdir /mnt/OFFSITE," where OFFSITE was the name I wanted for my external drive. Then I typed "sudo mount -a." But the rest of those instructions failed because the external drive still wasn't mounted.
  3. I tried the advice to use System > Administration > NTFS Configuration Tool > Enable write support for external disk. Possibly this step takes place automatically if you type "sudo apt-get install ntfs-config" instead of installing ntfs-config through Synaptic. But it did not solve the problem.
I also tried the advice to install ntfsprogs (now included in the Synaptic list, above) and then type "sudo ntfsfix /dev/sdXX" where XX designates the correct drive (e.g., c1). This gave me "Volume is corrupt. You should run chkdsk." Given the NTFS orientation of ntfsprogs, this appeared to refer to the chkdsk program in Windows.

I didn't plan to install any version of Windows on this computer. My desktop computer was dual-boot, but for purposes of simplicity and to preserve disk space on the laptop's relatively small hard drive, I wanted to make this a Linux-only machine. So to run chkdsk, I tried restarting the computer with a Windows installation CD. But this introduced a new problem. Each time I booted the computer and hit the space bar in response to the CD's message, "Press any key to boot from CD," it went through the process of loading various drivers and then, when it reached the point of saying, "Setup is starting Windows" (in the bar at the bottom of the screen), it gave me a blue screen of death (BSOD): "A problem has been detected and Windows has been shut down to prevent damage to your computer," etc. The bottom line of the message referred to "STOP: 0x0000007B." (Note that I was getting this error before reaching the point of deciding whether I wanted to enter Recovery Console.)

Research on this problem led to a Microsoft troubleshooting page. It identified a number of possible problems, but I was not sure any of them applied precisely to me, and I hated to spend a lot of time on a troubleshooting wild goose chase (TWGC). I did run self-tests of hard drive and memory in the laptop's BIOS setup; that is, I rebooted the machine, hit the Esc key to go into BIOS setup, and ran its diagnostics. Those tests found no problems with those pieces of the laptop's hardware.

It occurred to me, at this point, that possibly I needed a BIOS upgrade. I went to the support webpage for this laptop and discovered, there, a flaw in my plan to make this a Linux-only laptop: Compaq was not distributing firmware upgrades for its laptops in Linux form. To upgrade the firmware, I needed to be able to run Windows.

Before addressing that problem, I tried rebooting the laptop from the Windows CD without the external USB drive connected. This could potentially defeat the purpose of booting from the Windows CD (i.e., I wanted to use it to run chkdsk on the external drive), but now I was curious. This, however, did not make a difference. Evidently the external USB drive was not the reason for this STOP 0x0000007B message.

In that case, the Windows CD appeared to be objecting that it did not find a working version of Windows already installed on the laptop's internal hard drive. But that seemed silly. Of course there would be no version of Windows installed on a hard drive, if the purpose of running the Windows XP CD was to install Windows on that drive. How did this error message expect me to "run chkdsk /f" if I could not even run Windows?

Well, I could install WinXP after all, make it a dual boot machine, and deal with the inevitable GRUB issues, as Windows and Ubuntu fought over which system would load first and then, just when I needed the machine most, refuse to load either. Or I could give up on the idea of running Windows diagnostics or firmware updates on this laptop. But I had previously experimented with booting Linux from a USB drive (a/k/a "thumb drive" or "jump drive"). Since then, I had become aware of another Linux USB drive approach known as Penlinux, where you could either buy one of their pre-configured Ubuntu 6.10 USB drives (for about $45, which was less than the cost of a 2GB USB drive back in 2007, when they apparently came up with this); or you could make one for yourself. I had heard of similar things in Windows; I wondered whether I could boot Windows XP from a USB flash drive and do these various maintenance tasks that way. The discussion of setting up a bootable WinXP USB flash drive appears in a separate post.

When I did boot the system with the bootable WinXP USB flash drive in TXT mode, I went into Recovery Console. There, I got a screen I had not received previously: "Setup has recognized the following mass storage devices in your computer: [none]." At this point, I did not have the external USB hard drive connected, so apparently this was OK. I pressed Enter and got the ordinary Recovery Console. There, a "dir" command produced "drive is not valid" messages for all drives other than C and D. For C, I got "There is no floppy disk or CD in the drive." D was recognized as the USB flash drive. So Recovery Console, started by the USB flash drive, appeared unable to recognize even the mere existence of the Ubuntu-formatted internal hard drive. I typed "exit" and rebooted the system without the USB flash drive inserted. Ubuntu still booted up OK. When I pressed a key to try to boot from the WinXP CD, I still got the BSOD and the STOP 0x0000007B message, as above. So merely starting the system with the USB flash drive, by itself, did not seem to have made any helpful change in the system. I booted with the USB flash drive again. This time, after booting in TXT mode, I chose the Setup option instead of the Recovery Console option in the Windows XP Professional Setup process. Unfortunately, this directed me to that same message: "Setup has recognized the following mass storage devices . . . ." But this time, when I pressed Enter to continue, I got the partitioner: WinXP was willing to set itself up in the jump drive. Again, though, it didn't see any other drives.

* * * * *

I thought that I might need to create (and, optionally, delete) a WinXP partition in order to completely rid this hard drive of the traces of Vista. (I assumed Vista was the culprit; I had not seen these problems previously in WinXP.) I booted my Gparted CD and inserted a 15GB NTFS partition before any other, where drive C would normally be. I had hoped to use my new bootable USB flash drive to install WinXP on it, but that drive was still not booting this laptop successfully, so I tried to install WinXP on that partition using a slipstreamed WinXP SP3 CD instead. That again gave me the "A problem has been detected" BSOD (above).

It began to appear that the process of deleting all partitions and starting over was more difficult than I had realized. That Recovery partition was no ordinary partition. There seemed to be a process by which it could be deleted from inside Vista itself; but once the Vista partition was gone, so was that option. At this point, I discovered Hiren's BootCD, another apparently famous funky tool that supposedly contained an unbelievable list of disk utilities. I downloaded a copy (version 9.9) and burned it to CD. I booted it, and it gave me four options: Boot from Hard Drive, Start BootCD, Start Mini Windows XP (which might well have done the same thing as the BartPE boot CD, as I now realized), and Windows Memory Diagnostic. I chose Start BootCD and used it to run PartitionMagic 8.05 Pro. PM did not see anything on the drive other than unallocated space. I tried running another program, but got a mouse error; apparently at least some of the programs on Hiren's BootCD were best run after a fresh reboot. I rebooted and tried Mini Windows XP. It was able to recognize a USB flash drive, so I used one to copy over the DiskPart.exe program I had previously downloaded. An AvaFind search of my drives revealed, at this point, that WinXP already contained DiskPart.exe, although in an apparently earlier (or at least smaller) version (i.e., 160KB rather than 191KB). With the larger one onboard, I moved the USB flash drive to the target computer and tried to run it from within Mini Windows XP. Unfortunately, this gave me an error message: "Error creating process [msiexec.exe /l diskpart.msi]. Reason: The system cannot find the file specified." I found msiexec.exe on the other (working) computer and copied it, and the older DiskPart.exe (i.e., same date as the msiexec.exe), to the flash drive. I tried again with this on the target computer. This did give me the DISKPART prompt and the options described in my other post. I typed "list disk." It showed that I had only Disk 0, which was right (if you don't count the jump drive). I typed "select disk 0" and then "clean." After a moment, it said, "DiskPart succeeded in cleaning the disk." I typed "list volume" and it did not show any volumes on the hard drive. So it seemed that it had indeed cleaned the drive. To test it, I tried something that had failed previously (see, again, my other post): I typed "exit" (to get out of DiskPart.exe), inserted the Windows XP installation CD, and tried to boot it. I got "NTLDR is missing" because, silly me, I had not yet removed the USB drive. I removed it and tried again. The disk churned for a long time and then gave me the familiar old BSOD. The older version of DiskPart.exe had failed to fix the hard drive. I restarted Hiren's BootCD and went into Hard Disk Tools > HDD Regenerator > Scan and repair. It ran for about an hour and detected no errors. (At this point, and repeatedly after this, I tried a number of other tools on Hiren's BootCD, not listed here.) Seagate SeaTools for DOS told me that it was a Seagate hard drive. I used that program's Advanced Features > Erase Track ZERO. I went into the Fujitsu low-level format tool, also on Hiren's BootCD, but it did not seem to recognize any drives. I tried rebooting the Windows XP installation CD again, but it still gave me a BSOD. In Hiren's BootCD, I went back into Mini Windows XP > BootCD WinTools, but didn't see anything that looked useful. I tried Start > Programs > Check Disk, but it did not find a volume to work on. Taking a different approach, I restarted the computer and chose Hiren's BootCD option (rather than Mini Windows XP) > Partition Tools > PartitionMagic Pro. There, I selected the unallocated space (i.e., the only thing listed) and created an unformatted primary partition that used up all of the unallocated space. I rebooted the WinXP installation CD, but still got a BSOD. Back in Hiren's BootCD, I tried the Active Partition Recovery program. It said this:
Logical C:

Detected file systems:
from partition table: FAT32 (LBA)
from BOOT sector: Unknown
Get file system from BOOT sector? [Y/N]
I said Yes. Next, it gave me an option to Perform Extended Disk Scan, with this note:
Extended Disk Scan may detect partitions being deleted even if you have created new ones instead, formatted and used them!
The screen also showed two partitions within HDD 80h. I arrowed down so that the first of them was highlighted: "Logical C." This gave me an error message: "Error reading sector # 0 or BOOT sector is invalid." When I highlighted the second one, "Unallocated," I got no error message, but it was weird that this partition was only 2.49MB. Anyway, I went back and ran the Extended Disk Scan. It was slow. After a while, it detected a 14.9GB NTFS logical drive. The program gave me the option of adding this partition to the drives list. I said yes. There was another one, a while later, a 35GB partition. Yes to that too. Both of these, I thought, were WinXP partitions that I had created, during the past month or so of fooling around with this laptop. As the scan reached the halfway mark, I was getting concerned that it hadn't yet found either the original Vista partition or the original Recovery partition. But then, about three-fourths of the way through, it did find a 10.9GB partition. The whole process took maybe six hours, on a 250GB drive. When it was done, I had three logical drives and two unallocated portions. Now what? I thought maybe I would take a look at these partitions in one of the other programs there on the Hiren's BootCD, so I hit Esc. This gave me an option of writing the detected partition information back to the hard drive. As I reflected on the matter, I decided that these former partitions were sort of like a former girlfriend: sure, things hadn't been so good with them, but then, things hadn't been so great without them either. So I decided sure, why not, let's take them back. Just like that, they were written to the hard drive, and I was rebooting. I went back into the partition tools, there on Hiren's BootCD, and took another look via PartitionMagic. Now PartitionMagic said it had "detected an error 116," and offered to fix it. I said sure. Now there was another error, partition table error #108. PartitionMagic was willing to format the single partition that it did find, so I went with that. It said it was "formatting partition *: (BADMBR)," but then it gave me Error #4, "Bad argument/parameter." Bad argument, indeed: this really was turning out like a reunion with an ex-girlfriend!

I decided to fight fire with fire. Vista had created this problem; maybe Vista could undo it. Probably the clearer reasoning would have been, they did this at a factory, so I need a factory to fix it. But I wasn't reasoning along those lines. I had gotten my hands on a copy of Microsoft Vista, and I was going to install it and see what would happen. The Vista DVD gave me an option of recovering my system, but I didn't have any previous images it could work from, so I went ahead with the installation option. Vista detected the five partitions or unallocated spaces that Active Partition Recovery had restored. I told Vista to delete each of those partitions. Now I had one big 232.9GB unallocated space. I went ahead with the installation of Vista into that big empty space; maybe I should have used the option to format that unallocated space first. But the main thing was that Vista installed, where WinXP had failed to do so. That was promising. Maybe it wouldn't even have been necessary to install Vista; maybe WinXP would have installed OK after I ran Active Partition Recovery. But maybe not; after all, PartitionMagic was still confused. Speaking of which, I decided to run it again, after Vista finished installing itself. This time around, when PartitionMagic on Hiren's BootCD detected an error, I didn't let it fix it; I now dimly recalled that PartitionMagic never got entirely comfortable with larger hard drives. Maybe if I hadn't let it fix the imaginary error the last time, it would have worked OK. But probably not: as before, it still thought we had a partition table error #105. I took out Hiren's BootCD and put in the Windows XP installation CD, and rebooted. But, wouldn't you know it, I still got a BSOD. Vista was happy with the hard drive; Ubuntu was happy with the hard drive; but WinXP couldn't be satisfied. Well, I had a Seagate Seatools CD from mid-2007. I assumed it was newer than the Seatools for DOS found on Hiren's BootCD, but perhaps not. What could this Seatools CD tell me? I rebooted with that CD, and I discovered that it could tell me, "No Hard Drives Found."

It seemed that Vista had its own special way with my hard drive. This called for a choice. I could leave Vista alone, giving it its own 15-20GB partition - that is, I could venture into the land of the Vista-Ubuntu dual boot. That might be the eventual outcome, but first I felt it was a good idea to screw around some more. I thought I might start by using Gparted to reduce the size of the Vista partition, as if I really were going to set up a dual-boot system after all, and see what happened then. So that's what I did. I booted with my Gparted CD, removed the 1MB unallocated space prior to the Vista partition, shrank the Vista partition to 30GB, inserted a 30GB ext3 and 5GB swap partition after that, and left the rest for an ext3 partition for data. Then I rebooted into Vista. I got Windows Boot Manager, telling me this:
Windows failed to start. A recent hardware or software change might be the cause. To fix the problem:

1. Insert your Windows installation disc and restart your computer.
2. Choose your language settings, and then click "Next."
3. Click "Repair your computer."
I did this, using the Vista DVD. I chose the "Repair and Restart" option, removed the DVD, and rebooted. Vista ran a disk check and then came up OK, but wanted to reboot because of hardware changes. After that second reboot, it seemed to be functioning normally. I rebooted with Gparted, to see what had changed. There was now a 6MB unallocated space after the Vista program partition. So possibly what screwed things up was that I removed that 1MB unallocated space that had been before the Vista partition. I ran a check on the two ext3 partitions in Gparted. They were OK. I rebooted with the WinXP CD and, as usual, I pressed a key to boot from the CD. This time, I got this:
Windows Boot Manager

Choose an operating system to start, or press TAB to select a tool:
(Use the arrow keys to highlight your choices, then press ENTER.)
Vista (recovered) was the only option. I restarted the computer and tried again to boot from the WinXP CD. Still got a BSOD. I could have uninstalled Vista and reformatted the drive, just to see if there was somehow a "right" way to remove Vista, thereby making the drive once again accessible from the WinXP installation CD. I decided instead to get my Ubuntu installation in place in a Vista dual-boot. Maybe I wouldn't need to go back to WinXP or worry about it anymore, and could perhaps even replace the Vista partition with an occasional Vista boot from a USB drive (for purposes of e.g., upgrading the laptop's firmware).

In that case, the final question was whether Vista and Ubuntu could work with the external USB hard drive. That, you will recall, was the question that triggered this expedition.  By this point, however, I had begun to suspect that the problem was with the external hard drive, which appeared to be defective.