Adding Bad Clusters to the Bad Clusters File - Second NTFS Boot Sector Is Unwriteable

I was using Windows 7 and, as such, was using hard drives formatted in NTFS and divided into several partitions.  I booted the system with an Acronis True Image Home 11 boot CD, made an image of one partition, and saved it on another.  When I rebooted Windows, the folder that I had told Acronis to create (named "2012-01-20 Backup") did not exist.  Windows Explorer showed it as being just a file, without an extension.

In Windows Explorer, I right-clicked on the drive and selected Properties > Tools > Check now.  I clicked both boxes (i.e., "Automatically fix file system errors" and "Scan for and attempt recovery of bad sectors") and then Start.  But the tool did not run.  It just disappeared.  That was probably because I had a paging file on that drive (Start > Run > SystemPropertiesAdvanced.exe > Advanced tab > Settings > Advanced tab > Virtual memory > Change).  I could have changed that, and then rebooted to make the change effective, and then gone back into Windows and tried again.

Instead, I rebooted with a Windows 7 installation DVD and went into Repair your computer > Use recovery tools > Next > Command Prompt.  At the prompt, I typed C: and then DIR, and then D: and then DIR, and so forth until I found the troubled partition.  I typed CHKDSK /R and let it run.  In stage 2 of 5, it deleted the index entry for the 2012-01-20 Backup folder (which it was calling "file 1045").  So it looked like I was going to have to redo my Acronis backup.  After finishing stage 5 of 5, it said this:

Free space verification is complete.
Adding 602 bad clusters to the Bad Clusters File.
CHKDSK discovered free space marked as allocated in the volume bitmap.
The second NTFS boot sector is unwriteable.
Failed to transfer logged messages to the event log with status 50.

That last message didn't bother me -- I was always getting that when I ran CHKDSK this way.  But I wanted to know more about the others.  This was the second time I'd had this kind of problem after running Acronis.  I wondered whether the problem was with the drive -- that perhaps I should get rid of it -- or whether it was instead something that Acronis was doing to the drive.

What I had done previously was to try to resize the partition using a GParted Live (i.e., bootable) CD -- or, actually, I had booted an Ubuntu Live CD (version 10.10 -- apparently not all versions had GParted built in) and had gone into its System > Administration > GParted option.  If I recalled correctly, GParted had been unable to resize the partition, apparently because of the partition's problems, so my next approach was to remove everything from the partition and then use GParted (or possibly Windows Start > Run > diskmgmt.msc) to delete and recreate it.  That had worked last time, so I decided to do it again.  This was the last partition on the drive, and the consensus seemed to be that the second NTFS boot sector was at the end of the partition.  I had used this Acronis CD for a long time, and had not otherwise been getting this problem.  So I thought this time I would create another small partition, after this one, and just let it sit.  If Acronis was getting confused when it wrote to the last partition on the drive, that little 1.5GB partition might provide a buffer and solve the problem.

So now I was going to try to find out whether it was indeed a hard drive issue.  (One other note:  as I continued in the process, it appeared that the Acronis backup may have come very close to filling the troubled partition.  I wondered whether that could somehow have corrupted it.)

I went into GParted again.  It showed an exclamation mark next to this partition.  When I right-clicked on it and looked at Information, it said several things, including these:

ERROR:  This software has detected that the disk has at least 602 bad sectors. . . . This means physical damage on the disk surface caused by deterioration, manufacturing faults or other reason.  The reliability of the disk may stay stable or degrade fast. . . . Unable to read the contents of this file system!  Because of this some operations may be unavailable.

This number of bad "sectors" matched the number of bad "clusters" reported by CHKDSK (above).  That is, it appeared that there might be no bad sectors other than those that were reported after the Acronis process.  I rebooted into Windows and continued to use the system as usual while moving my data off the troubled partition via Windows Explorer.  Then I rebooted into Ubuntu > GParted and, as just indicated, I created two partitions where there was formerly one.  As before, the first of the two was an NTFS partition.  The new addition was that 1.5GB partition, which I formatted as ext3 (invisible to Windows, thus causing no confusion).  Then I did another Acronis image of my programs drive (C:), as before, saving the image to the new NTFS partition.  When Acronis was done, I went back into Windows and saw that Acronis had created what appeared to be a valid image.

I ran System Information for Windows (SIW) to find out which drive this new partition was on, and who made that drive.  After consulting with Disk Management (Start > Run > diskmgmt.msc) to figure out whether I should be looking at disc 0 or disc 1, I saw that it was a Seagate ST31000520AS.  I went to Seagate's webpage and downloaded SeaTools for Windows.  (I already had a copy, but hadn't used it for a long time, and wasn't sure it was the latest version.)  I installed it and ran its Short Drive Self-Test (Short DST).  It passed.  I ran a S.M.A.R.T. test.  It passed that too.

It did not appear that this drive was failing.  I hibernated and then booted up with Ubuntu > GParted.  Its Information option saw no problems.  Neither did its Check option.  I rebooted with the Windows 7 DVD and ran CHKDSK /r on both of the new partitions (i.e., including the little 1.5GB trailer).  There were no statements of the kind I had gotten on the previous try -- nothing about deleting index entries, bad clusters, or an unwriteable NTFS boot sector.  What I got was, "Windows has checked the file system and found no problems" and "0 KB in bad sectors."  Of course, all of my data (except for the newly re-created Acronis backup) was still on the other drive, so I didn't expect file errors (and CHKDSK ran somewhat more quickly on the nearly empty drive).

It appeared that the previous warnings had been false alarms, triggered by the use of Acronis True Image Home 11 to save an image that nearly filled the last partition on the drive. 

To round out my investigation, I went back through the random webpages that I had opened up while searching for insight.  One post raised the possibility that I might have been able to use ntfsresize to resize the partition instead of removing its data, deleting it, and creating a new one in its place.  Ntfsresize (and ntfstruncate) were apparently Linux utilities, and it looked like using them could be risky.  There was a report that seemed to confirm that GParted would have reported an error again if it had found a bad sector, so getting no error messages in GParted's Information option apparently implied that the bad sectors were gone.  Evidently the repartitioning had reset them.  One webpage raised the question of whether the partition had been marked as a boot partition.  As I recalled from my glance in GParted, it hadn't.

Another thread alerted me to the thought that I should have run CHKDSK /B instead of /R.  To verify that, I went into a command window and typed CHKDSK /?.  It said, "NTFS only:  Re-evaluates bad clusters on the volume (implies /R)."  I wondered whether CHKDSK /B might have been a shorter response to the whole problem.  It also said, "The /I or /C switch reduces the amount of time required to run Chkdsk by skipping certain checks of the volume."  As Microsoft advised, I wouldn't have relied on those switches by themselves; but since I had already run /R, at this point I could have tried CHKDSK /I /C /B.  (Too bad there wasn't an /M option -- I'd have had a missile.)  Apparently some of these options were not available in earlier (pre-Vista) versions of Windows.

I had run a couple of searches and probably could have gone on indefinitely, but by this point it began to seem that I had already encountered most of the main recent lines of tinkering on the question.  One exception:  I had searched for, but never did find, a tool that would give me a visual representation or map of bad clusters on the drive.  I vaguely recalled that Norton had provided something along those lines, back in the DOS ages.  Defragmenters like Smart Defrag would show a nice, colorful map of disk clusters, but at this point I wasn't aware of a Windows 7 defragger (or other tool) that provided a depiction of bad clusters specifically, so I sent IOBit a suggestion to that effect.

Ubuntu 10.04: Sharing a Folder on an NTFS Partition

I had two computers running Ubuntu 10.04.  These machines were connected by ethernet cable through a router.  Both had Samba installed, and both had a shared folder set up as described in another post.  I had set up one of those shared folders on an ext3 partition.  As that other post indicates, I was able to see the contents of that shared folder from the other computer.  But without thinking about it, I had set up the other computer's shared folder on an NTFS partition.  Seeing a folder on that partition turned out to be a more complicated matter than I had expected.  This post describes my efforts in that regard.

Right away, I rediscovered that chown wouldn't work as expected with NTFS drives, and that it was therefore necessary or advisable to type "sudo gedit /etc/fstab" and change the lines for NTFS drives or partitions to something like this:

UUID=[UUID for the drive] /media/partitionname ntfs-3g rw,suid,dev,exec,auto,user,async,umask=000 0 0

and then unmount ("sudo umount /media/partitionname") and remount the partition.  Remounting would apparently force a look at the new contents of fstab, and could be done with "sudo mount /dev/sdc3 /media/drive1" or just "sudo mount -a" (assuming a folder named /media/drive1 had already been created (e.g., "sudo mkdir /media/drive1")).  That fstab line was one I had developed to replace the "defaults" word that appeared in some fstab lines previously:  it made all of the default settings explicit, and changed some of them.  Yet even with this change, when I tried to set the Sharing Options in Nautilus, I got this error message:

Folder Sharing

'net usershare' returned error 255: net usershare add: cannot share path /media/partitionname/folder2 as we are restricted to only sharing directories we own.

Ask the administrator to add the line "usershare owner only=false" to the [global] section of the smb.conf to allow this.

I was willing to go into smb.conf and make that change, but first I wanted to know why (as I confirmed with a right-click > Properties) root still owned that folder.  This led to the insight that NTFS filesystems (such as the one I was trying to share) did not remember ownership, so the partition would need to be reminded each time I mounted it.  One way to do this was to modify the fstab line to include user and group identifications:

UUID=[UUID for the drive] /media/partitionname ntfs-3g rw,suid,dev,exec,auto,user,async,umask=000,uid=username,gid=groupname 0 0

and then save fstab and unmount and remount the partition (above).  And yet that still didn't do it.  The better statement seemed to be, not that NTFS partitions needed to be reminded, but that they simply didn't have an ownership concept.  There seemed, then, to be no alternative but to type "sudo gedit /etc/samba/smb.conf" and add the line "usershare owner only=false" to its [global] section, as advised above.  I saved and closed that and tried Sharing Options again.  This time it worked, or at least it didn't give me an error message.

Unfortunately, the shared folder still wasn't showing up on the other computer's Places > Network list.  It sounded like the solution might be to try Sharing Options as root ("sudo nautilus").  Unfortunately, at this point "sudo nautilus" was giving me an error message on that computer:

(nautilus:23890): Unique-DBus-WARNING **: Error while sending message: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

I wondered what would happen if I stepped away from the NTFS situation for a moment.  I decided to try sharing a folder on an ext3 partition in that same computer.  I right-clicked on the ext3 partition in Nautilus, chose Create Folder, named the folder, went to the Share tab, and selected "Share this folder" and "Allow others to create and delete files in this folder."  This gave me an error message:

'net usershare' returned error 255: net usershare add:
cannot stat path /media/partitionname/foldername to ensure this is a directory.  Error was No such file or directory

I guessed that this might be because I had gone directly to the Share tab without first clicking Close in the Basic tab there in the P4 Share Properties dialog.  So, OK, I saved the new folder first, and then right-clicked on it in Nautilus and chose Sharing Options.  This time it went OK.  I went into System > Administration > Samba, as described in the other post, and added this folder there.  But I still wasn't able to see it from the other computer.  I rebooted this problematic machine and tried again.  In doing so, I noticed that GParted and some other things were still running on another desktop, so that may have explained why "sudo nautilus" worked OK after the reboot.  The sharing step just described also worked OK.  But Places > Network on the other computer still didn't show the problematic computer.  

I was just about to change some hardware and reinstall Ubuntu on that machine anyway, so I deferred further effort on this project until after that was done.  At that point, though, I found a better folder-sharing solution by using a Synology network-attached storage (NAS) unit.

Ubuntu 10.04 vs. Windows XP: Can't Delete a File

Many Windows users have encountered files that they cannot delete.  I had been able to handle most of those problems pretty easily.  The tools I used were Unlocker, to delete the problem file from within Windows conveniently, and Ubuntu Linux, when Unlocker failed.

I was able to switch to Ubuntu pretty quickly, if necessary.  I was using Ubuntu 10.04 on a dual-boot system.  I had installed VMware Workstation 7.1 in Ubuntu, and had created a virtual machine (VM) running Windows XP SP3 in Workstation.

But in this case, I wasn't having file deletion problems in Windows; I was having file deletion problems in Ubuntu.  Specifically, I was running into difficulties when trying to use an external USB hard drive.  For instance, when I tried to delete a file from the external hard drive, I first got this message:

Cannot move file to trash, do you want to delete immediately?

No problem there, to my knowledge; it was a 2.2GB .avi (i.e., video) file, so maybe it was too big for the trash.  So I clicked Delete.  This gave me "Error while deleting," and when I clicked on "Show more details," I got this:  "Error removing file:  Input/output error."  Similarly, when I tried to move some .avi files manually from the internal hard drive to the external USB drive, I got "Error while copying" and, in the details, "Error opening file [filename]:  Input/output error."

Through a relevant search, I came across a post that made me think I should try to approach this from the Windows side.  That seemed ironic at first, but then I reflected that the target drive was NTFS.  That is, it was formatted for use by Windows.  I wasn't having this problem with my Ubuntu ext3-formatted drives.

So I went into Windows and tried to do the same maneuvers I had just tried in Ubuntu:  connect to the USB drive; delete one file from the external (NTFS) USB drive; move files from an internal (ext3) drive to that external USB drive.  I didn't get too far.  Windows said the folder on the USB drive "is not accessible.  The file or directory is corrupted and unreadable."  Was that what "input/output error" in Ubuntu meant?

I ran Disk Management (Start > Run > diskmgmt.msc), right-clicked on the USB drive > Properties > Tools > Error-checking > click both boxes > Start.  I clicked on out of there and rebooted so that the error check would run.  But this did not repair the drive.  On a second try, I watched more closely.  The VM did reboot, but it did not run a disk check.  Apparently the problem was that the external USB drive did not become connected to the VM until after the operating system had already restarted, by which time it was too late for a disk check.  So I rebooted the VM again, this time with a WinXP installation CD in the CD/DVD drive, and ran CHKDSK /R from the Recovery Console.

CHKDSK ran for a while and then said, "The volume appears to contain one or more unrecoverable problems."  This particular external drive consisted of a hard drive in an external enclosure.  I took the drive out of the enclosure and put it directly in the computer.  This time, CHKDSK ran successfully.  I repeated CHKDSK until it reported no more errors, and then put the drive back in its enclosure.  Now I was able to delete the file.  Problem solved!

Error While Resizing Vista NTFS Partition with GParted

I was using the GParted 0.5.1 live CD to resize an NTFS partition.  (I had already backed up the data on the partition.)  When I tried to resize in GParted, I got an error:

An error occurred while applying the operations

See the details for more information.

IMPORTANT
if you want support, you need to provide the saved details!
See http://gparted.sourceforge.net/larry/tips/
save_details.htm for more information

I clicked OK.  Unfortunately, the details did not explain what had gone wrong.  I rebooted into Vista and then did a complete shutdown.  (Previously, I had just hibernated the system.)  I allowed a minute or so for the memory to clear, and then rebooted with the GParted live CD.  This time, it worked.  The problem was that I had hibernated instead of completely shutting down Vista before using GParted.

Ubuntu: Backup with Rsync

In a previous post, I got as far as concluding that rsync was the tool of choice for backing up my computer in Ubuntu 9.04. I didn't pursue it because I was short on time and patience for writing scripts at that point. But eventually the need for a regular backup system became acute. So this post logs the steps I took to make rsync and cron work for me.

First, here's what I wrote previously:

As an alternative to rdiff-backup, what people had actually mentioned more frequently was rsync. It did not have the incremental backup features of rdiff-backup, to my knowledge, but it seemed to be an established tool for backup purposes. So for now, at least, I thought I might try that instead. Once again, I did a Google search and got a package details page with no apparent link to any help files. Eventually I found what looked like the official rsync webpage and, after looking at their FAQs and some other pages, landed on their Examples page. It was intimidating.

This time, I went to their Documentation page. This gave me links to, among other things, Michael Holve's rsync tutorial. The tutorial said, "You must set up one machine or another of a pair to be an "rsync server" by running rsync in a daemon mode." I was curious, so I did a Google search for "what is daemon mode" and I got back, would you believe, exactly one page. One webpage in the entire known planet answered the question, "What is daemon mode?" Except it didn't really answer it. It just said, "It makes wget put standard output into a log file and not bug you while downloading." Accepting that as the best available answer (and ten points to the answerer!), I typed "rsync --daemon" in Ubuntu's Terminal and proceeded to the next step, "Setting Up a Server." After reading it, I decided it didn't seem to apply to me. It was for people who wanted to back up files between computers. I just wanted to back up to another drive.

So I went on to the tutorial's "Using Rsync Itself" section. Since I wasn't sure what daemon mode did, or if it was necessary, I killed that Terminal session and started another. I didn't know if that would shut off daemon mode, or if doing so was what I should do. I read the section and then checked another source of documentation, the rsync man page ("man" being short for "manual"). The man page would ordinarily be output in response to a Terminal command, but someone had put it here in html form, so that's what I used. It reminded me, first, to check Ubuntu's System > Administration > Synaptic Package Manager to make sure I had rsync already installed, here on my secondary computer. I searched Synaptic for rsync and got back a couple dozen listed programs; rsync was among them and was shown as being installed. I looked partway into the man page and got an answer to one question I had from the tutorial. So here's how I translated what the tutorial was telling me. First, the tutorial listed these lines:

rsync --verbose --progress --stats --compress --rsh=/usr/local/bin/ssh --recursive --times --perms --links --delete \
--exclude "*bak" --exclude "*~" \
/www/* webserver:simple_path_name

The first thing to know was that this all represented a single command line. It was too long to fit on one line, though, so apparently the trailing backslash said, "This line continues on the next line." The command would be typed into a file and saved as a script, not typed directly into Terminal. I didn't know why the first line didn't end with a backslash. I decided I would want to experiment with this in a relatively safe place -- with a junk directory on my secondary computer, perhaps -- to see what it was doing.

So as the tutorial explained it, the first line of this example told rsync how to proceed: verbosely (i.e., with lots of information about what it was doing), showing a progress report, with statistics. The rsh part was for encryption, to be used optionally if you were sending your stuff online to another computer. I wasn't, so I decided to try leaving that off. I also didn't want to compress the output, because that made the process slower and required more attention from the CPU.

The second line of the example, above, told rsync to recurse -- to work through all of my directories and subdirectories under the folder that I would be naming. It also told rsync to preserve file timestamps and file permissions -- so if, for example, a file was readable only by root on the source drive, it would be the same way on the target drive. The -- links command was an instruction to preserve symbolic links -- not sure what that meant -- and the --delete command, as I understood it, would tell rsync to delete anything on the target that wasn't on the source. So you'd have a mirror, and not just an accumulation of backups of files that you have deliberately trimmed out of your file collection.

The third line of the example told rsync not to bother copying some kinds of files. I liked the sound of that at first, but then I decided I would rather be able to do a Properties comparison of source and target and verify that both had exactly the same number of files. So I decided to leave out this line when I used rsync.

The fourth line of the example named the source and target locations. I wasn't going to be using it with a remote source or target, so mine was going to look somewhat different from this.

On that basis, here's what I assembled as a test version of the rsync example from above:

rsync --verbose --progress --stats \
--recursive --times --perms --links --delete \
/media/DATA/Source /media/DATA/Target

I created a Source subfolder in my DATA folder and put a TestFile.txt file into it. I also created a Target subfolder in DATA. Then I copied those three rsync lines into a file in Ubuntu's Text Editor (gedit) and saved it to Desktop as TestRun. To make it executable, I went into Terminal, typed "cd /home/ray/Desktop" and then "chmod +x TestRun" and then double-clicked on TestRun and said Run. And, you know, it worked. Just like that. Not exactly as intended -- I had not only TestFile.txt but also the whole Source folder underneath my Target folder -- but, yeah, there it was. I deleted the Target folder and ran it again and, sure enough, it created the Target folder and then inserted a copy of the Source folder into it. Excellent!

Now it was time to try something a little bolder. I wanted to see how it worked if I tried to copy the whole DATA folder to an external drive. This part was a little confusing. The external drive seemed to have two different names. If I looked in /media, its name was simply "disk." But if I hit the Computer icon in File Browser, it came up as "193.8 GB Media." I decided the latter sounded more specific, so I would try that first. So now the third line of my TestRun file read like this:

/media/DATA "/media/193.8 GB Media"

I used quotation marks because there were spaces in the name. I saved TestRun and double-clicked it again on the Desktop. It didn't seem to do anything. I realized that I had probably made a mistake in that line, and tried again like this:

/media/DATA "/193.8 GB Media"

That didn't do it either, so I tried again, without the leading slash:

/media/DATA "193.8 GB Media"

That still didn't work, so I tried the other approach:

/media/DATA /media/disk

Still nothing. I went into System > Administration > Partition Editor (GPartEd), wiped out the target partition, and recreated it as a FAT32 partition. Now the drive was totally invisible to Ubuntu. I went back into GPartEd and reformatted it as an ext3 partition. Then I realized: it was an IDE drive, so apparently it would not be recognized until I rebooted. I decided to reboot into Windows (I had a dual-boot system) and format it as NTFS. I named it 186GB (which seemed to be the net amount of space available in NTFS format) and rebooted into Ubuntu. I revised TestRun's last line again:

/media/DATA /media/186GB

and ran it again. This time, it seemed to be working -- the external 186GB drive was making noise -- but I wondered why I wasn't getting a verbose indication of what was going on. I guessed that, if I wanted the verbose information, I would have to execute rsync on the command line, not in an executable script. While I was rooting around for an answer to that question, I was reminded that I could also use the shorthand versions of these commands. So instead of typing --verbose into the script, I could just type -v and whatever other letters I needed. In this approach, the final contents of TestRun, which were as follows:

rsync --verbose --progress --stats \
--recursive --times --perms --links --delete \
/media/DATA /media/186GB

could instead be expressed like this, if I understood the man page's Options Summary section correctly:

rsync -vshlEPtrip --del --delete-excluded --force \
--exclude RECYCLER \
--exclude "System Volume Information" \
/media/DATA /media/186GB

In that version, I added a couple other options that seemed appropriate, and also told rsync to exclude (i.e., don't copy) those extra folders that Windows XP seemed to put on every drive. I revised TestRun along these lines and, when the external drive settled down and it looked like the foregoing TestRun process had ended, I ran it again. But it didn't seem to make any difference. The extra folders were still there. I had understood that it would delete them. Part of the problem seemed to be that I had not used the syntax correctly. I was supposed to use an equals sign: --exclude=RECYCLER. But another part of the problem was that it was not clear whether the "exclude" command was supposed to work with directories. It didn't seem so. The man page just referred to excluding files. I tried again with equals signs, but still no change. I posted a question on it, but the kind response was unfortunately not able to resolve the issue.

Next, I tried a modified version of TestRun on the primary computer. I went through several revisions and wound up with this version, which seemed to work:

rsync -vchlEtrip --progress --del --ignore-errors --force /media/CURRENT/ "/media/OFFSITE/P4 CURRENT"

The partition being backed up, in this case, was an ext3 partition named CURRENT, and the target to which it was being backed up was a USB external drive named OFFSITE. (Some weeks have passed since I started this post, so there may be some discontinuity in my writing at this point.)

I did not run this command as a script within a file called TestRun that I would start by double-clicking on it, because I discovered that you would only get the detailed output if you entered the rsync command on the command line. I was able to enter all of the foregoing rsync command on one line. I did not need the "exclude" commands because this was not an NTFS drive formatted by Windows. I still had ext3 "lost+found" and Trash folders that got copied over in this way, but they were small, so it was OK.

As I think I may have said before, I got the selected rsync parameters by typing "man rsync" at the command line. It did take some trial and error to get this particular set. The resulting backup, when checked by right-clicking and selecting Properties, seemed to be virtually identical.

When I ran that rsync command, it showed me lots of detail on what it was doing. It concluded with this message:

rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]

Eventually, however, I did figure out how to do it.  Here is an example of an rsync command that worked for me:

rsync -qhlEtrip --progress --delete-after --ignore-errors --force --exclude=/.Trash-1000/ --exclude=/lost+found/ /media/CURRENT/ /media/CURRBACKUP

This one would back up what Windows sees as drive D (named CURRENT) to a partition that Windows sees as drive G (named CURRBACKUP).  Both partitions had to be mounted in Ubuntu before this would work.  I used a similar command to copy a folder on CURRENT to a USB jump drive named KINGSTON.  That gave me a portable copy of the current state of that folder, ready to take along.

The next thing I needed to do was to back up my blogs.  I started by just wanting to be able to back up a webpage.  I had discovered that all of the posts on a Blogger (i.e., Blogspot) blog like this one could be displayed in a single webpage, at least if you had less than 1,000 posts.  To do that, you just needed to go to this URL:  http://blogname.blogspot.com/search?max-results=1000.  I wasn't sure what would happen if you entered a larger number than 1000.  So now that I had that webpage, I wanted to know how to save a copy of it automatically.  Strangely, at this point, Google searches for any of these sets of terms

ubuntu "back up a webpage"
rsync "back up a webpage"
rsync "copy a webpage"

produced zero hits.  Eventually, it started to look like this was because I was barking up the wrong tree.  As described in a separate post, it seemed that what I wanted for this purpose might be wget, not rsync.

Configuring 64-bit Ubuntu 9.04 with Vista Dual-Boot

I had a Compaq CQ60-420US laptop. It had come with Vista pre-installed. I didn't like that, so I wiped it off. This led to a whole ordeal in trying to get the hard drive to work with WinXP, for which I had developed lots of tricks and tweaks. That effort ultimately failed, and I wound up with Vista back on the thing after all. I had wanted to get away from dual booting, but I still needed some flavor of Windows for the occasional hardware interaction, e.g., updating the BIOS and other firmware. So for now, I was going to leave Vista in place and set up a 64-bit Ubuntu 9.04 (Jaunty Jackalope) dual-boot with it. This post describes the process of setting up that dual boot. A review of some guides gave me the general impression that installing a Vista-Ubuntu dual boot was much like installing a WinXP-Ubuntu dual boot, unless you used the wubi alternative. The approach I took was as follows:

1. With Vista already installed, insert the Ubuntu program CD, reboot, and go through the ordinary installation sequence. If you do nothing, the CD will pretty much take you right to the Install icon. If your BIOS isn't set to boot from CD before hard drive, hit F2 or Esc or Del or F8 or whatever key it is that gets you into your BIOS setup, promptly after the computer first starts up, and adjust the boot priority there.
2. You may find that the bootable Gparted CD provides a clearer view of hard drive partitions than does the partitioner in the Ubuntu installer. If the partitioning step leaves you dazed and confused, you may want to back up, download Gparted, burn yourself a CD, boot with that first, set up your partitions as you like, and then come back into the Ubuntu installation process. Note: if you're going to run Windows in a virtual machine, you may want to give it an NTFS partition somewhere, so you have a place to store data. Windows can't read Linux (e.g., ext3) partitions.
3. After the initial installation, make sure that Vista starts up OK. No point spending hours refining a system that isn't ready for prime time. Then restart and go into Ubuntu, and make various adjustments, including these: (a) Nautilus > View > Show Hidden Files. (b) Nautilus > Edit > Preferences > Behavior > Always open in browser windows.
4. Go through the steps described in my previous post on configuring Ubuntu 9.04 (including comments). That post updates the first part of an earlier post on how to configure 9.04. After running updates, type "sudo gedit /boot/grub/menu.lst" and put # symbols in front of each line (i.e., older Ubuntu kernels) that you don't want to appear in the GRUB menu.
5. Before continuing with items in that earlier post (as explained in more detail there), I initially thought that the next step would be to install and configure Thunderbird (just plain old thunderbird, not mozilla-thunderbird) and Lightning-extension via Synaptic. But then I decided I would just use web-based e-mail and calendaring (probably via Gmail) on the laptop, and would download and archive my e-mails solely on my desktop, thereby sparing the need to synchronize the two computers.
6. Now refer back to the earlier post, to install programs that came via individual downloads rather than through Synaptic. For me, these included Google Desktop for Linux, Adobe Reader, and VMware Workstation 6.5.3.
7. I installed .deb files by double-clicking on them in Nautilus, and .bin and .bundle files by running "chmod +x [filename]" and then "sudo ./[filename]." I didn't have any .tar.gz and .tar.bz2 files to install this time, but if I had, I would have moved them to my /home/[username] folder, navigated there in Terminal, and then used a tar unpack command (e.g., tar -vxf [or tar xvfz] filename.tar.gz, or tar xvf filename.tar, or tar yxf filename.tar.bz2).
8. To install Google Earth, continuing to follow my previous notes, I typed two lines: "wget http://dl.google.com/earth/client/current/GoogleEarthLinux.bin" and then "sh GoogleEarthLinux.bin." This gave me a Google Earth installation, but with flickering and basically nonfunctioning display. I tried System > Administration > Update Manager, and at first that program assured me that my system was up-to-date; but when I made it check again, it reported errors related to Wine and Opera. I ignored these for now, since they did not seem relevant. People seemed to be experimenting with the flickering video problem in Google Earth. There was a relatively complex tutorial that apparently fixed it in some cases, at the risk of messing up the system. Choosing instead an easy fix that seemed to work for some, I went to System > Preferences > Appearance > Visual Effects and downgraded from Extra to Normal. That didn't help. Following another tip, I downgraded further, from Normal to None (i.e., no visual effects), and also turned off the Atmosphere feature in Google Earth (View > Atmosphere). That fixed it.
9. To configure Firefox, I went to another computer and used the FEBE extension to make a full backup of that machine's Firefox installation. I copied the folder containing the FEBE backup to the target computer (i.e., the one where I've been doing all this installation stuff). I installed FEBE in Firefox on the target machine, restarted Firefox, started to watch the tutorial on restoring with FEBE, turned to the instructions on manually restoring with FEBE, and then took these steps on the target machine: Close Firefox. Go to the FEBE backup directory (i.e., the one where I put the FEBE backup folder that I copied over from the other computer). Copy its .fbu file (in my case, profileFx3{default}.fbu) and rename the copy as a .zip file. I called it FEBErestore.zip. Extract the contents of the .zip file (creating, in my case, a folder called FEBErestore). Move the contents of the FEBE restore folder to the Firefox profile folder, which I found in Nautilus at File System/home/ray/.mozilla/firefox/[random name].default. In my case, for example, I moved the contents of the FEBErestore folder to this .default folder. When it told me that a folder already exists, I said Merge All, and Replace All for the "file already exists" message. During this process, I got an "Error while copying 'febe.jar'" message. The details of the error said "Permission denied." I canceled and tried again as root (type "sudo nautilus" and then do the move in the Nautilus session that opens that way). That worked. Then I closed everything else and started Firefox and, yeah, it looked like all the extensions were there, configured and everything.

This was the extent of my Ubuntu configuration for now. During these processes, I came across some miscellaneous issues:

• I wanted to change login passwords. This, I thought, would be under System > Administration but no, eventually I found it instead under Applications > Accessories. Double-clicking on that did not work; I had to right-click and choose Change Password; but then the password that I changed it to did not work for login. I thought the problem might be that I hit Enter instead of clicking on the button after doing the change; that is, possibly the default was Cancel rather than Change. I could not tell; neither button seemed to be highlighted by default. So then it turned out that I had changed the password to unlock the keyring, rather than the password to log in. It looked like I had changed that properly, second time around; but still no.
• As in other Ubuntu installations, panels did not readily allow me to move icons to the locations I would designate. Sometimes they would not move; sometimes they would not go exactly where I indicated. On the bottom panel, for example, I could not rearrange them in the far right corner. It turned out to be easier to move things *out of* the corner (where the Windows system tray would be) than to move them *into* the corner. It turned out that I had to unlock every item that I wanted to drag another icon past. Tooltips came up, irritatingly enough, when I was trying to move icons, making it difficult to see what I was doing.
• The bottom panel failed to show icons or buttons for my currently running programs. The solution was to right-click on the bottom panel, choose Add to Panel, and choose Window List. But then, on reboot, it did not work again. I fixed it by right-clicking on the end of the Window List item on the bottom panel and checking Lock to Panel.
• When trying to view the contents of some hard drive partitions, such as a partition I called DATA, I got "Cannot mount volume. You are not privileged to mount the volume 'DATA'. Following my previous notes, in Terminal I typed "sudo mkdir /media/DATA," thinking that perhaps I had not yet created the mount point, but this gave me "cannot create directory `/media/DATA': File exists." I typed "sudo nautilus," went to File System > /media/DATA > right-click > Properties, and verified that I (i.e., user "ray", not just root) had full permissions. I typed "sudo fdisk -l" (that's an L, not a one) to get a list of devices. That showed that the DATA partition was being recognized as an NTFS device at /dev/sda4. I typed "sudo gedit /etc/fstab" and saw that there was no line in the fstab file for /dev/sda4. Following some notes from a few months earlier, I typed "sudo ntfs-config." (Note that this was one of the programs I had installed from Synaptic, above.) This detected the VISTA programs partition (i.e., drive C in Windows), but not the DATA partition. I ran ntfs-config again. This time, it didn't mention the VISTA drive, but as before it did give me the option to enable an internal drive. I accepted that. Now I saw that there was indeed a line for the DATA partition as well. While I was here, I used blkid to find the UUIDs for each partition (e.g., "sudo blkid") and replaced that portion of the relevant line in fstab. For example, the line that previously read "/dev/sda1 /media/VISTA ntfs-3g defaults,locale=en_US.UTF-8 0 0" now began with "UUID=19142FAA142F8D35" instead of "dev/sda1." Following the format of other lines in fstab, I preceded this one with a line that said, "# Entry for /dev/sda1 : " and followed a similar procedure for the DATA partition. I rebooted and was now able to view the NTFS-formatted VISTA and DATA partitions. On second thought, I went back into fstab and removed the line for the VISTA partition, since I didn't expect to need it normally in Ubuntu and didn't want to expose it to accidental deletions and such.

This seemed to give me a basic working dual-boot 64-bit system. The next step, for me, was to configure VMware, so that I could run any Windows program within Ubuntu.

Recognizing Partitions Automatically in Ubuntu 9.04

In Ubuntu 9.04 (Jaunty Jackalope), disk partitions normally remain unmounted until you mount them manually. One way to do this is to start Ubuntu's File Browser (also known as Nautilus). You can do this by choosing the Places > Computer menu option. In File Browser, just double-click on the drive you want to mount, and its name will appear under the File System > Media folder there in File Browser. You can also mount drives by typing commands into Ubuntu's command-line window (Applications > Accessories > Terminal). First, type "sudo mkdir /media/[drivename]" to create a mount point. After doing this for all unrecognized drives, type "sudo mount -a".

But instead of doing this sort of thing, I wanted the system to recognize my drives on bootup, so that I would not have to mount them manually each time I rebooted. In my first approach to this problem, I typed "sudo ntfs-config." This approach worked OK on one computer, where I was doing a clean installation -- where, that is, I had formatted the partition before installing 9.04 on it. But on another computer, I installed 9.04 on top of 8.04 without formatting the partition. This was advantageous in that it preserved lots of files and settings from before. The drawback was that I was running into funky problems with ntfs-config.

On this system, I got an error message when I ran ntfs-config. The message was, literally, "Error : An error occured when trying to configure [partition name], please retry. Thanks." (For those who are searching with the correct spelling, let me retype that error message: "Error: An error occurred when trying to configure . . . ." When I tried to examine the partition in GParted (i.e., System > Administration > Partition Editor (see above)), I got this: "Warning: Unable to read the contents of this file system!" The partition had been available a minute earlier, when I had booted into Windows XP, and I had shut down WinXP normally before rebooting into Ubuntu. So I didn't think the partition was truly hosed.

I found a very, very long thread on the subject, but post no. 2015 in that thread said the thread's advice was outdated for purposes of newer versions of Ubuntu. In another thread, someone said they got this error message after installing a newer version without formatting the partition, and solved it by deleting the mount points. I had a fresh backup of the partition in question, so I tried taking that approach. First, I opened File Browser and went to File System > Media > CURRENT. File Browser said that partition was empty. When I right-clicked on it, however, File Browser didn't give me an option of deleting it. Its Properties said it was owned by root, so I logged in as root (see above), typed "cd /media", typed "dir" to make sure CURRENT was there, typed "dir CURRENT" to make sure it was empty, and then typed "rm -r CURRENT" to delete it. I restarted the system, started File Browser, went to Computer (in the Tools bar), double-clicked on CURRENT, and it was there, with all its contents. CURRENT was formatted in ext3 (i.e., Linux). I had another partition that was having the same problem. It was formatted in NTFS (i.e., Windows). But now that I had rebooted, it too was available and looking fine.

I dropped this issue for the time being. Next day, when I rebooted, I found that, predictably, the various partitions were not mounted. In Terminal, I typed "sudo gedit /etc/fstab." My fstab file did not show the partitions I wanted to be mounted automatically. I reviewed one of my old posts on ntfs-3g. It seemed to say that ntfs-config was the solution. I tried running ntfs-config again. This time, it ran. It recognized two NTFS partitions. Now I just had two ext3 partitions to set up for automatic mounting. I went back into fstab and saw the lines that ntfs-config had added. I hadn't thought to save the fstab file from my previous installation; it would have been handy to copy over the relevant lines. From the previous post, I gathered that the trick was to write something like this in fstab:

/dev/sda3 /media/VMS ext3 defaults 0 0

for the VMS partition, and likewise for the other ext3 partition. I double-checked that I had the right device addresses by checking them in System > Administration > Partition Editor (i.e., GParted). I rebooted, but the ext3 partitions were still not automounted, though the NTFS partitions continued to be automounted nicely. I didn't know why the line just shown, for the VMS ext3 partition, did not work in fstab now; according to the old post, it had worked in Ubuntu 8.04. Ubuntu documentation said something about pysdm, so I looked into that. They suggested typing this into Terminal:

sudo apt-get install pysdm

but I saw it was available in Synaptic (i.e., Ubuntu's System > Administration > Synaptic Package Manager menu pick), so I used that instead to install pysdm. I typed "pysdm" at the Terminal prompt, but got an error message indicating that I had to have root privileges, so I tried again with "sudo pysdm." (Also available via System > Administration > Storage Device Manager.) That gave me a Storage Device Manager (SDM) window that listed some, but not all, of my partitions. I noticed that, if I started pysdm while I still had the foregoing /dev/sda3 line for my VMS partition in my fstab file, SDM would not give me an option of configuring the VMS partition; but if I deleted that line from fstab and then started pysdm and clicked on sda3, SDM would give me a "Configure now?" dialog with the message, "/dev/sda3 hasn't been configured. Do you want to configure it now?" I said OK. I told it to set the mount point to File System > Media > VMS. Then I clicked on the Assistant button and saw that the option was checked that said "The file system is mounted at boot time." I also checked the option that said, "Check file system at mount time" on the options dialog's Mounting tab. I didn't change any other options. When I checked OK to leave Assistant, I saw an error message in Terminal: "wrong fs type, bad option, bad superblock on /dev/sda3, missing codepage or helper program, or other error." I went back into Assistant and removed the "Check file system" option I had just checked. This time, when I left Assistant, I got Warning messages that said, "Unknown option: rw" and "nouser is not suitable for user" and "Unknown option: async." I clicked on the "Set defaults" button in SDM and then clicked Apply. This produced no additional Warning messages in Terminal. Trying once more, I just went into Assistant and immediately executed, and this did produce the same Warning messages, plus an additional one that said, "Unknown option: defaults." I tried just emptying out the Options box altogether in SDM. Then I clicked Apply. I did the same with the other ext3 partition that was not yet mounting automatically. I restarted the system.

It occurred to me that I may not have restarted the system previously, but may merely have logged out and logged back in. Ubuntu 9.04 seemed to require two steps to reboot, and it seemed that I had forgotten to take that second step last time around. This occurred to me right around the time when I tried to restart the system and found that it froze with a black screen. Evidently my fstab file was not presently in an acceptable form. I punched the computer's reset button and rebooted with the Ubuntu installation CD, chose the "Try Ubuntu" option, and edited fstab as above. (Note that there's an fstab on the CD too; make sure you're editing the one on the hard drive.) I saw that I had failed to add the preferred mountpoint for one of my ext3 partitions: it said "/media/sdc5" instead of "/media/CURRENT." I fixed that and rebooted without the CD. This time, the system booted OK. But those two ext3 partitions were still not automounted.

In File System > media (i.e., /media), I noticed that there was a mount point for the VMS partition, but it had nothing in it. I typed "dir" to make sure there was nothing there, and then typed "sudo rm -r VMS" to delete it. I also deleted the "sc5" mountpoint mentioned above. In File Browser, I clicked on the Computer button at the top and then double-clicked on each of these two partitions (i.e., VMS and CURRENT) to create mountpoints (i.e., /media folders) for them. I tried rebooting again. They still weren't automounting, and the mount points I had created for them were gone, as I should have expected. Incidentally, at this point I discovered that the custom was apparently to mount fixed hard drives at /mnt, and use /media for removable hard drives.

About this time, I discovered that I had commented out the lines added by SDM (i.e., I put # signs (also called hash marks) in front of each of those lines). I forgot. I had done that to troubleshoot fstab, and then didn't go that route after all, and forgot to remove them. So I removed the hash marks now and rebooted. But that wasn't the solution. I re-ran SDM, went through more or less the steps described above, and rebooted. This time, it worked: the partitions were automounted, and their contents were visible. Not sure exactly what was different this time, but something was. So I guess the conclusion is, if it doesn't work in SDM the first time, tinker around for a while, reboot a couple of times, try SDM again, and reboot.