Showing posts with label move. Show all posts
Showing posts with label move. Show all posts

Monday, April 2, 2012

Windows 7: Eliminating "This Folder Is Shared With Other People" Message

I had installed Windows 7.  Now, when I would try to move or delete some folders, I would get this message:

This folder is shared with other people.

If you move this folder, it will no longer be shared.
That message was correct:  I had, in fact, shared the parent folder (and all of its subfolders, including this one) with my other computer.  And I wanted existing and newly created subfolders to continue to be shared.  I just didn't want to receive this warning.

I tried the approach of going into Control Panel > HomeGroup > Leave the homegroup, because someone advised that this was the solution.  It wasn't.  I tried a search.  This led to a thread with some heartwarming rants about what a dumbass feature this was.  That thread also contained the suggestion that, at least for purposes of deleting (as distinct from moving) folders, a workaround was to cut and paste files and folders into Recycle Bin, rather than trying to delete them directly.  There was also a command line alternative that supposedly worked for wiping out lots of subfolders:
for /D %d in (*) do rmdir /S /Q "%d"
My guess was that the command line would also work for moving, in a pinch:  type MOVE /? for guidance.  But I didn't want a command-line solution.  The thread also offered a couple of other complicated workarounds with potential side effects.  Before going that route, I tried the suggestion to take ownership of the folder in question.  I had already added a Take Ownership context menu (i.e., right-click) option in Windows Explorer, so I used that now.  I used it at the level of the entire partition.  That didn't solve the problem.  I tried it again on the top-level folder.  That didn't do it either.  I verified that I had already given Full Control to Everyone in the Sharing context menu option, so that wasn't the solution either.

Another thread contained messages repeating the view that, as a deliberate feature of Windows, this couldn't be changed.  I took a look in TweakNow PowerPack 2012, in case they had a fix.  I hadn't really used TweakNow previously, but now I saw it wasn't really a tweaker in the sense that Ultimate Windows Tweaker (UWT) was.  So I ran UWT instead.  But no, UWT didn't seem to have a solution either.  So far, the only solution I had seen was that some people had repartitioned their drive and reinstalled Windows from scratch.  But that was a pretty draconian solution, and it didn't seem to guarantee against a recurrence of the problem.

Back in Windows Explorer, I right-clicked on the drive being shared and selected Properties > Security tab > Edit > Add > Everyone > Check Names > OK > Allow Full Control > OK.  That took a while, as it went through the drive, with a message that said, "Setting security information" various files and folders.  This was obviously not a very secure solution.  Nor was it logically related to sharing.  It was just a guess.

In another thread, someone said that they noticed this behavior began when they installed RC-1 (i.e., Windows 7, First Release Candidate).  As noted above, people had been saying it was a built-in Windows feature.  But there was at least the possibility that some program had triggered it.  In that spirit, someone else in that same thread offered the belief that it might be related to Windows Mail.  That was a possibility.  I had installed Windows Live Mail.  I didn't like it much -- it was very slow to start.  But I had already uninstalled it.  There didn't seem to be much more I could do on that front.  I tried running Glary Registry Cleaner, just in case that would make a difference.  It identified a lot of problems, but did not fix this particular one.  In another thread, someone suggested that the culprit might be the network sharing service in Media Player 11.

I saw recurrent references to the C:\Users\[username]\Appdata\Local\Temp\WPDNSE folder.  For username Ray, mine had nothing in it.  Evidently they came to that folder because its name was appearing in an error message.  I wasn't getting that.  The general idea seemed to be that one user might somehow be linked to another user at that folder.  That was interesting.  I looked into the possibility that this "shared folder" problem might be due to the existence of an unnecessary user account.  As described in another post, I pruned out some extraneous user accounts.  But that did not resolve this problem.

Over time, the problem appeared less frequently.  It seemed that there might have been an ownership or sharing event, occurring at one time, that became less relevant as my folders went through various processes of being renamed, moved, reshared, and so forth.  One post gave me the idea that the problem might be in Windows Explorer > select a folder > right-click > Properties > Sharing tab > Advanced Sharing > Permissions.  At this moment, the only group or user listed there was Everyone.  It seemed that, previously, there might have been additional entries there, for Ray, Administrators, and/or System.  The post made me think that, even though all roads seemed like they should lead to Mecca, the mere presence of multiple people here would confuse Windows.  Maybe the original warning that "This folder is shared with other people" was Windows' way of telling me that it was going to remove the others and leave only Everyone on the list.  I didn't test this; this was just an idea from that post.  At the time when I decided to close this post, it's not so much that the problem had disappeared as that it had become more rare.

Saturday, April 23, 2011

Using a Spreadsheet to Rename Thousands of Files - First Try

I had a list of a couple thousand files.  I got the list as an export from a program, but I could also have gotten it from a directory listing at a command prompt (e.g., DIR /a-d /b /s).  This post describes how I used that list to rename those files.  Needless to say, since I was working with the possibility of screwing up years' worth of information, I did make a backup of these files before proceeding.

This particular list was a list of emails that I had just exported from Thunderbird, as described in another message that I expect to post to this blog on the same day as this one.  I had exported, not only the list, but also the actual emails, in EML format.  I planned to use the list to rename those EMLs so that they would be in the format I wanted, which was like this:

2011-03-20 14.23 Email to John Doe re Tomorrow.pdf
It would require some massaging to get there. 

The Date and Time Segment

Starting with the date and time, here's what Thunderbird had given me in the list of files:
3/20/11 14.23
These were all in one field, in the .csv (i.e., Excel-readable) output from T-bird.  They might have been in two or more fields, in a text file produced by a directory listing (e.g., DIR /a-d /b /s > dirlist.txt), but to some extent the same techniques might prove useful.  These numbers were not in a format that would work as a Windows filename, and in fact the exported EMLs did not contain the date and time data in this format.

I was going to use the list to rename the emails the way I wanted, with data that existed in the list but not in the current EML filenames.  To do that, I would need to try to get the list so that it accurately represented the actual EML filenames.  Otherwise, if the list produced a command that said, Rename File 234A.eml to be "Message from Ray 234A," that command would not work if the file being renamed was actually called File_234A.eml (with an underscore).  My command would just get a "File not found" error.  So my first step was to find a way, in my Excel spreadsheet, to reproduce what the files were actually called, as I viewed them Windows Explorer.
First, I would have to extract each element into a separate column, there in the spreadsheet, so that I could format and rearrange them as desired.  For this, I started with FIND commands.  (Excel's internal help had good information on using these and other related commands. I was doing this in Excel 2003. There might have been other functions that would automate this process in newer versions.)  For instance, =FIND("/",A1) would locate the first occurrence of the forward slash in cell A1, if that's where I put the "3/20/11 14.23" item shown above.  So then I could search for the next slash (=FIND("/",A1,B1+1), starting one place after the results of the first FIND statement (which, in this example, I had put in cell B1).  I could do the same for spaces, colons, or whatever else might delimit various components of the date and time entry.  I'd then use RIGHT, LEFT, or MID statements to find those components.  For instance, a MID statement like =MID(A1,B1+1,C1-B1-1) might start one space after the first slash, continue to one space before the second slash, and thus give me the day of the month.

In the 3/20/11 14.23 example, the value of "3" for the month wouldn't give me the desired "03" so I would use IF and LEN statements to pad that out:
=IF(LEN(G1)=1,"0","")&G1
thereby inserting a zero in front of single-digit month values found in cell G1, but adding nothing to those month values if they were not single-digit.

I had gotten two different things from Thunderbird.  On one hand, as noted above, I had gotten a list of data about emails that I was going to export, with dates in the 3/20/11 format.  On the other hand, I had also gotten the actual emails, exported as individual EML files.  These were not named quite the same way.  The particular Thunderbird extension that I had used to produce this list of files, ImportExportTools, had produced files whose names were in this format:  Date-Time-Sender-Subject.  But in those filenames, the dates could not be rendered as 3/20/11, since slashes were not allowed in Windows 7 filenames.  Instead, the date and time came in the format of 20110320-1423.  So as hinted above, I would ultimately be producing a batch file that would automate the renaming of thousands of files.  That example, 20110320-1423, would instead begin with the more readable 2011-03-20 14.23.

Next:  Massaging the Sender Data

Before I could get to that point, I had to change some of the Sender data that T-bird had produced.  Again, the list of actual email data from T-bird contained characters (e.g., the > symbol) that could appear in email Subject lines but that were not allowed in Windows 7 filenames.  (The full set of forbidden characters:  \ / : * ? " < > | ).  For example, the spreadsheet might show a sender to be this:
John Doe <jdoe@xcom.com>
but the actual exported EML file's name would just have John Doe.  So I could get rid of some of these problems of verboten characters by just doing a FIND for < and then a MID or a LEFT statement for everything before that.  That wouldn't necessarily get rid of all the bad-character problems, but I wanted to clean those up just once, so I deferred that problem for the moment.  For right now, in a bid to match the format of the exported EMLs, I now had this much of the filename:
20110320-1423-John Doe
To get there, the actual command I used, for the Sender portion, was this:
=LEFT(B2,V2-1)
Cell B2 had the Sender's name as exported from Thunderbird, and cell V2 had the FIND location for the < symbol, which marked the end of the part of the Sender data that I planned to use.


At this point, there was a problem.  For some reason, ImportExportTools put underscores before and after some Senders' names, but not all.  So I might instead have this:
20110320-1423-_John Doe_
The filenames already used a hyphen ( - ) to delimit fields, so the combination hyphen-underscore seemed superfluous.  It needed to be fixed before I could continue with the main project here, so that I could be confident that Sender names were delimited consistently by a hyphen, and not by unpredictable choices involving underscores.

Detour:  Renaming Thousands of Files, So That I Could Proceed to Rename Thousands of Files

To fix this problem, I went to the Windows 7 command prompt (Start > Run > cmd) and did a directory listing to save the filenames to a file:  DIR /b > dirlist.txt.  I copied the contents of that file into column A of a new Excel spreadsheet.  I also copied those contents into Notepad.  In Notepad, I did two global replaces (changing both -_ and _- to just plain - ).  I copied those changed contents into column B of that Excel spreadsheet.  Next, I needed to combine these two columns, A and B, into a single DOS command that would rename the files.  To do that, I put this formula in column C:
="ren "&char(34)&A1&char(34)&" "&char(34)&B1&char(34)
The char(34) entries would introduce quotation marks:  34 was the ASCII code for double quotes.  I could have introduced any character that way -- Z, a semicolon, whatever -- but it was essential to use the char(34) approach for quotation marks specifically because otherwise Excel would have misunderstood them.  Quotation marks would be necessary for any DOS command involving filenames that contained spaces; DOS would otherwise think that the space marked the end of a part of the command.  Anyway, I copied the contents of column C back into Notepad and saved the file as a batch file (renamer.bat).  I saved renamer.bat in the folder where I had all those EML files I was going to rename.  At a DOS prompt in that folder, I typed "renamer.bat," without the quotes.  I could have just double-clicked on renamer.bat in Windows Explorer.  Either way, the files were renamed.

Later, I discovered that ImportExportTools would truncate the Subject field in the spreadsheet to 50 characters even if I had not asked it to do so.  Many emails in my set had Subject fields longer than that.  I initially tried to fix this by setting up a separate spreadsheet, using the process just outlined, but with a search for the last hyphen in the filename, which would hopefully identify the start of the Subject field in most cases.  Unfortunately, the calculation of 50 characters proved difficult, after taking into account the character substitutions described here.  Ultimately, I started over with a fresh export of EML files from Thunderbird, after setting the option at Tools > ImportExportTools > Options > Filenames tab > Cut subject at 50 characters (and also Cut complete file path length at 256 characters).

Returning to the Sender and Subject Segments

So now I could go back to work on the main spreadsheet.  I could assume, that is, that the first part of the EML filenames were going to look like this:
20110320-1423-John Doe
without underscores.  I could see that there were still a few underscores around names in the EMLs, and that was problematic.  Possibly ImportExportTools had introduced two underscores rather than one in a row, in some cases.  There were few enough that I figured I could fix those exceptions manually.

Now it was time to add the Subject part of the filename.  This was simple enough:  just use another ampersand (&) to combine it with the rest.  Sketching it out, the EMLs now just used hyphens to delimit the date, time, sender, and subject portions, so I just needed to use an Excel command of this format:
=[Date]&"-"&[Time]&"-"&[Sender]&"-"&[Subject]&".eml"
and that would give me a complete representation of how ImportExportTools seemed to have constructed the filenames for most of the exported EMLs.

Cleaning Up Unwanted Characters

As noted above, some characters (e.g., the colon, ":") were not acceptable as Windows 7 filenames, but were quite common in email subject lines.  It looked like ImportExportTools had replaced those characters with an underscore, and had removed spaces before and after the underscore.  So, for instance, a subject line of "Re: Tomorrow" was represented, in the EML filenames, as "Re_Tomorrow," without a space.

At this point, preparing to remove those unwanted characters, my spreadsheet had something like this:
20110320-1423-John Doe-Re: Tomorrow
Some of these fledgling combinations contained other unwanted characters (e.g., question marks), listed above.  I was almost ready to remove them.  But first, it was time to proofread my spreadsheet.  Having copied all of my various formulas down from the first line of the spreadsheet, where I was developing them, so that they were present in all rows, I now prepared to sort the spreadsheet.  First, I inserted an Index column as column A, and in that column I inserted numbers in ascending order, starting with 1.  The purpose of this column was to remember my original sort order, in case the date field or anything else did not accurately reproduce the order in which emails appeared in Thunderbird.  There were times when a person would want to check back and see if things were matching the source.  To insert these numbers, using Excel 2003, there were two options.  One was to enter a formula in each cell, adding 1 to the number above it, and then replace the formulas with fixed values.  The sequence for this was Edit > Copy, Edit > Paste Special > Values.  An alternative was Edit > Fill > Series.  Either way, I now had index numbers that wouldn't change if I sorted rows.

So now I did sort rows, sorting first on the column that concatenated (i.e., combined) the date, sender, and subject.  I sorted to highlight those rows in which the process had not worked as intended.  One problem, I saw, was that some Sender names did not include the <jdoe@xcom.com> part.  The sender of these emails was just listed as John Doe (or whoever), without the actual email address.  For these, I just copied the Sender straight over, skipping the unnecessary part of the spreadsheet calculation.  That cleared up the error messages in the spreadsheet.  So now I saved a version of the spreadsheet and moved the output column -- the one combining all of my work with these various fields -- and pasted it into Notepad.  I could have edited it right there in Excel, but there was a risk that I would accidentally have changed other columns in ways that I did not intend, or that I could not achieve what I needed to achieve.  For example, the ? symbol was a wildcard in Excel, so attempting to replace it with an underscore would either not work or create a mess.  I noticed that ImportExportTools had also converted commas into underscores, even though they were not forbidden characters.  Note:  now that I had parts of the spreadsheet in Notepad, and expected to paste those parts back into Excel in the same order, this would have been an especially bad time to sort the spreadsheet.

Over in Notepad, I did global find-and-replace operations for each of the special characters listed above, guided by the objective of matching as closely as possible the actual filenames of the EMLs I had exported from Thunderbird.  It was premature to do line-by-line editing of individual entries, though; those would be more obvious and perhaps more easily fixed later.  At this point, if I saw something that needed to be changed, I executed it as a global command, so that it would be changed wherever it occurred.  Obviously, a person can make serious mistakes with global changes so, again, this was not the time for fine-detail fixes of individual entries.  On the other hand, each of these global changes was a potential time-saver:  a single change here could save the need to make manual changes to 20 or 300 individual files later. 

After making these changes, I again had to replace the -_ and _- combinations with a simple hyphen ( - ), since some forbidden characters that I had just replaced with underscores had been adjacent to the delimiting hyphens that ImportExportTools seemed to convert into simple hyphens.  With these changes made, I copied the contents of that Notepad file back into the appropriate place in the spreadsheet.

Testing the Filename Match

How well did my spreadsheet now reflect the actual names of those EML files?  I decided to test it.  To do this, I first made a backup copy of the folder containing the EMLs.  I then created a subfolder in the EMLs folder, called Test.  In the spreadsheet, I worked up a MOVE command for each file, commanding it to move to the Test folder.  The command was like this:
="move "&CHAR(34)&Z2&".eml"&CHAR(34)&" Test"
I copied all those MOVE commands from the spreadsheet to Notepad and saved the Notepad file as MOVER.BAT in the EML folder.  Then I ran it.  It succeeded in moving less than half of the files, which meant that the spreadsheet had not yet captured hundreds of EML file names correctly.  I dir a DIR /b > dirlist.txt in the EMLs folder to capture the names of the ones that had failed.  I brought that list into the Excel spreadsheet and matched them up with my attempts.  This matching process was sufficiently time-consuming that I found myself grateful for the ones that I had managed to identify in a more automated fashion.  The time investment was acceptable.  As had been the case for me with spreadsheets going back almost 30 years, I figured that, once I identified the rules and became more familiar with this particular process, I would be able to use it again in similar tasks.

To match up the real filenames with the versions in the spreadsheet, I pasted the contents of dirlist.txt into a separate worksheet in Excel.  From both, I compared the date and time data, using VLOOKUP on shortened versions of the relevant colums (=LEFT(Z1,13)).  This revealed several things.  First, there was an error in how I had converted the date and time data, so I would have to fix that and re-run it to move more of the items from the EML folder into the Test folder.  Since I planned on doing this task again, I also decided to automate some of the find-and-replace operations described above.  This required using FIND and MID to divide the text string into parts appearing before and after the character I wanted to replace (e.g., colons) and then joining them together without the middle part.  I didn't do this for all possible combinations; I was basically interested in eliminating the first and/or most frequent occurrences of the forbidden characters that did seem to appear. 

Starting Over:  Revise the Filenames, Not the List

After hours of effort and a couple of retries, the approach described above was still identifying (i.e., successfully moving) only about 80% of the EMLs I had exported from Thunderbird.  Depending on the total number of emails involved, that could mean hundreds or even thousands of emails that would have to be manually renamed in order to insure that their filenames contained all of the desired data.

I decided that the better approach mgiht be to try to rename the files, and keep renaming them if necessary, until they conformed to certain basic rules in the file list exported into Index.csv.  There had already been some of that in the steps described above; the change now was to make that the primary effort.

The first step was to take a listing of the EMLs.  This was easy enough:  DIR /b > dirlist.txt.  Then I copied the contents of that text file into Excel and changed components of the filename to be the way I wanted.  The names of EMLs exported from Thunderbird did not have "To" fields, but I was able to match up most of the names automatically with what I had already prepared, thus borrowing To fields from the work described above.  (This is a cursory description.  At this writing, I had run out of time for this project, and was focused on getting it done.  But the basic techniques were as described above.)

Some filenames did not match up easily to support an automated renaming of the EMLs..  I renamed the rest, using a MOVE command to put them into a separate folder.  I copied those that did not rename into a separate folder too.  These I renamed using the Bulk Renamer (above), so as to have .txt extensions.  I did that so that I could view them in IrfanView, which enabled me to flip back and forth among them quickly, so as to identify the proper "To" information.

At this point, I started a new post to summarize more clearly the steps taken here.

Friday, January 21, 2011

Windows 7: The INSTALL Partition

For a long time, probably since the 1980s, I had kept my Windows installation on drive C and my data on drive D.  (Drives, or partitions of drives, can be readily created within Windows 7 and also by partition manager programs.  GParted, included in the downloadable Ubuntu CD, has been the most reliable partitioning programs for my purposes in recent years.)

Having the data on drive D had several advantages.  One was that I could back up drives C and D on different schedules.  Once I got a good Windows installation set up on drive C, I didn't need to back it up very often.  I'd just make a drive image (using Acronis True Image in the past few years), and I'd make an updated image backup just when I had done some significant new program installation or adjustment.  By contrast, I would want to back up my data on drive D on at least a daily basis.  Of course, when I made the image of drive C, I would need someplace to put it other than drive C.  An external drive was a possibility, assuming the bootable CD would recognize it, but it would tend to be slower, and this would be complete downtime, when it would not be possible to do other work.  It worked the other way, too:  having the data on a separate partition meant that I could completely reinstall Windows without affecting or even worrying about my data.

There were exceptions to that last statement.  Some program configurations were so detailed, time-consuming, and/or oft-changing as to constitute a sort of data.  Not the kind of data I was supposed to be working on, but data nonetheless.  An example:  Firefox add-ons.  It could take a half-hour or more to find, install, and configure my Firefox add-ons after doing a new Windows installation.  Some add-ons allowed me to export my saved settings, but obviously I would not want to store those on drive C; they'd be wiped out if I reinstalled Windows sometime down the line.

I also found it was handy to keep a local copy of the programs that I would install in Windows, after installing the Windows operating system itself.  I did not want to have to re-download all those programs and re-invent all of the things I had previously figured out about installing them.  So instead I had folders containing the programs to install, with installation notes in accompanying text files, and I named the folders in such a way as to guide me in the installation sequence that worked best (e.g., "01 Motherboard Drivers").  There were also quite a few program installers that I tried and uninstalled, or hadn't gotten around to installing.  Also, some ISOs -- ready-to-burn CD images that I had downloaded but hadn't burned to CD, or wanted to keep because it was a hassle to re-download a 700MB image.  I had collected these sorts of things, not only for Windows, but also for Ubuntu.

I accumulated about 75GB of this stuff.  Keeping it all on drive D meant that my daily data backups were swelling up with all this material that didn't need to be backed up every day.  So at some point I moved a bunch of it over to its own partition, with occasional backups.  What I kept on drive D was mostly stuff that I was actually using in my current installation.  One example:  those Firefox settings files.

Sometime in the late 1990s, I discovered that I could move my Start Menu to drive D.  This would have the advantages mentioned above, including especially the fact that my custom-arranged Start Menu (top-level folders:  Productivity, Online, Multimedia, Tools, Startup, Miscellany) would not have to be rearranged each time I installed Windows.  As long as I installed everything in its default installation location, the shortcuts in the Start Menu would come back to life as soon as I reinstalled the target program where the shortcut expected to find it.

A few months before writing this post, I came to realize that, of course, I could also install my portable applications in the Start Menu.  This would be ungainly in the sense that a bottom-level folder in the Start Menu might contain a slew of program files instead of a nice, orderly collection of shortcuts.  But it was handy for keeping everything that ran in one place, where I could copy it to a jump drive and use considerable parts of it on any other Windows machine.  The Start Menu, wherever located, could also be accessed and synchronized on a network, so that I only needed to configure the Start Menu once and would then have it available for any computer I would attach to my home network.  (Making it available did require a registry tweak.)

Putting the portable applications in the Start Menu had an unwanted side effect.  Many portable apps (especially those coordinated by PortableApps.com) used many of the same program files.  That was a problem because I liked to use DoubleKiller to delete duplicate files on drive D.  I couldn't do that anymore, at this point, because it would detect tons of duplicative portable program files that were supposed to be there.  I looked for a different duplicate remover, one that would allow me to exclude folders like the Start Menu folder, but ultimately decided to stay with DoubleKiller for now.  The reason was that I did not want to risk that, one fine day, I would forget to exclude the Start Menu from a DoubleKiller sweep, and (although this was unlikely) would punch the wrong button and delete that Start Menu from my hard drive.

What I decided to do, instead, was to move the Start Menu so that it would join those other program files -- programs to be installed, etc. -- in their own partition.  I called it INSTALL and gave it a letter of W, so that its location would not be affected by the connecting and disconnecting of various USB drives and whatnot.  So then hopefully the shortcuts in it would stay in place, and would require no further adjustment forever and ever.

Right now, unfortunately, they did require adjustment.  I modified the registry tweak to point toward drive W:\Start Menu rather than D:\Installation\Start Menu.  This caused almost no problems.  The main issue was just that a bunch of shortcuts were now dysfunctional, in that they pointed at executable files on D that were now on W.  I ran Glary Registry Repair 3.3.  It identified 165 new registry errors.  As I scrolled down the list, I noticed that a huge number of the problems identified by Glary were links to IrfanView on D.  I considered doing a global registry search and replace for the IrfanView location -- or, indeed, for all references, changing them from D to W, perhaps with the aid of a global registry search-and-replace tool like Registry Toolkit ($25) or Registry Replacer ($15) or Replace Registry Values (free).  But then I decided it might be safer to let Glary fix those dud links and then do a search in a registry editor for any remaining references to D:\Installation|Start Menu.  I used O&O RegEditor to do that search.  I was now down to a total of just 29 registry references to D:\Installation\Start Menu.  I was going to edit them in O&O, but then decided not to.  As long as they weren't hurting me, I was better off just leaving them alone.  One ill-advised registry edit could cost me an hour or more for recover or restoration.

So this was pretty much the end of the project, aside from some continuing cleanup, correction of shortcuts, etc.  I now had an INSTALL partition, labeled as drive W, containing my customized, shared Start Menu and installers for various programs.  I could now run DoubleKiller on drive D without worrying that it might knock out program files, and without having to do manual exclusions to focus it on the data duplicates that I was seeking.

Wednesday, January 5, 2011

Windows 7: A Futile Attempt to Move the Fonts Folder

In Windows 7, I tried to move the C:\Windows\Fonts folder to another location.  I did it because I didn't want to lose fonts, if I decided to install additional ones.  At the default C:\Windows\Fonts location, they wouldn't survive reinstallation of the system.  An alternative was to save copies of additional fonts at a different location and copy them to C:\Windows\Fonts after installing a new system.

I found no guidance on this process of moving the Fonts folder.  I copied everything from the C: folder to the X: folder.  Then I searched the registry, changed the references in my .reg file to the new location (X:\Cache\Fonts) in the two locations where I found references to C:\Windows\Fonts, and ran the reg file.  The system rebooted without a problem.  I tried to delete C:\Windows\Fonts, but that gave me a message:

Folder Access Denied
You need permission to perform this action.
You require permission from TrustedInstaller to make changes to this folder.
I searched on that and found advice to go into right-click Properties > Security tab > Advanced > Owner tab.  There was no such tab for C:\Windows\Fonts, but there was for C:\Windows.  I went to Edit, changed it to Administrator, and checked "Replace owner on subcontainers and objects."  When I clicked Apply, I got another message:
Windows Security
You do not have permission to read the contents of directory C:\Windows\CSC\v2.0.6.  Do you want to replace the directory permissions with permissions granting you Full Control?
All permissions will be replaced if you press Yes.
The answer seemed obvious, and that made me suspicious.  I searched on that and saw that Microsoft advised to just go ahead with this, in a couple of contexts, so I did.  It took a while to change permissions for everything in C:\Windows.  Next, I got a message telling me to close and reopen properties to see the updated situation, so I did that.  Still looking at Properties for C:\Windows, I went into Security tab > Edit.  I highlighted Administrators and then clicked Allow for Full Control.  Now I got a message saying, "You need permission to perform this action" -- permission from Administator, which I was.  I found other advice that seemed to recommend these two commands:
takeown /f C:\Windows\Fonts
cacls C:\Windows\Fonts /G Administrator:F
But when I then tried to delete, I got another error message:
Folder In Use
The action can't be completed because the folder or a file in it is open in another program.
Close the folder or file and try again.
Not surprising, since the system itself was obviously using fonts, though it would seem like it could have been just as happy using them from X: as from C:.  I thought I had rebooted since making the registry change, so this time I booted into Safe Mode and tried the deletion there.  No joy.  I selected all fonts and tried deleting them individually, and that got me down to a selected core of 81 fonts, including essentials like Sylfaen Regular and Utsaah.  I was tempted to boot into Ubuntu and wipe out the Fonts folder that way, but instead I gave up:  I went to X:\Cache\Fonts and copied all of the fonts back to C:\Windows\Fonts.  I couldn't remember exactly how many there had been, but the count didn't seem the same, so I rebooted into Normal mode to run System Restore.  My hunch was correct:  some fonts were not displaying correctly.  After System Restore, all was lovely, so I moved on to the next part of my life.

Tuesday, January 4, 2011

Windows 7: A Shared, Relocated Start Menu

As described in another post, in Windows XP I had moved my Start Menu to a separate location.  That location was on a networked drive, where it would be backed up regularly, would not be affected by Windows reinstallations (as long as I reinstalled programs on each computer back to the same places as before), and would be available to multiple machines as needed.  It had grown into a monster that included not only shortcuts to installed programs but also the full contents of portable programs.

Now that I was installing Windows 7, I wanted it to function the same way.  Unfortunately, Windows 7 was not set up to accommodate this desire.  Apparently there had been a NoSimpleStartMenu registry option in Vista that would have returned the system to the classic WinXP Start Menu, but I wasn't finding much guidance on that registry tweak in Win7.  In my own experimentation, it didn't work.

There were some third-party utilities that would give me back the classic Start Menu.  I installed and played briefly with one of them.  It was the Classic Start Menu option within Classic Shell.  (Apparently there was at least one other utility calling itself Classic Start Menu, but that one wasn't free.)  Classic Windows Start Menu was another option:  it would apparently give me a Windows 7 Aero style combined with classic functionality, which sounded great.  It seemed to be a bit clunky, though, in the sense that it added entries for Create Folder, Create Shortcut, and Sort in each submenu.  It didn't look like it offered as many settings as Classic Start Menu, but that initial impression could have been mistaken.

Classic Start Menu seemed to be the most developed utility for changing at least the looks if not also the functionality of the Win7 Start Menu.  In my brief introductory dabbling, it was actually better than the original WinXP Start Menu in terms of configurability, and it left the original Win7 Start Menu still available via Shift-Click.  Because of its features, I decided to stay with it for a while, if possible, rather than just use a relatively simple set of registry edits and adjustments that might have accomplished some of the same results.  The idea behind those registry edits was to rewire the Favorites option on the Win7 start menu so that it would point to C:\ProgramData.

That pointer to C:\ProgramData might have been useful, for my purposes, if C:\ProgramData had been customizable.  It just wasn't.  It appeared that, as I installed more programs, it (and therefore a Start Menu built around it) would get very cluttered.  I had the same concern about Classic Start Menu:  I couldn't tell how to redirect it to look at the network drive.  Their FAQs didn't have a question on exactly that.  It sounded like their main plan was that I should drag and drop items into their settings dialog.  This, I thought, would not work for a shared Start Menu:  the changes would take effect only on the computer where I made them.  There appeared to be some possibility of future expansion in their Settings > Customize Start Menu tab > Programs Menu > right-click > Edit Item > Link option, but presently that option was grayed out.  The Classic Shell people had a list of alternative utilities, but it sounded like they tended to have bugs.  I looked at their forum, but didn't see a question on relocating the Start Menu, so I posted a question on it.  The question got no response.

I tried to take a look at C:\Documents and Settings, which is where the Start Menu shortcuts were saved in WinXP.  (This was a hidden folder; I had already adjusted settings in Windows Explorer > Tools to see it.)  When I went to that folder in Windows Explorer, I got an error:  "Location is not available.  C:\Documents and Settings is not accessible.  Access is denied."  A search on that led to a page that seemed to be telling me that, actually, Documents and Settings was just a link to C:\Users.  Another page said C:\Documents and Settings was a "junction point" that existed for backwards compatibility.

So I tried C:\Users instead.  That opened up.  There wasn't a Start Menu under Ray (my original username) because I had deleted that username.  I was only using Administrator.  I saw a Start Menu under Administrator, and I also saw one under All Users, as well as one under Default.  There might also have been one under Default User too; but when I tried to go into there, or into any of these several other Users subfolders, I got the "not accessible" error again, even though I was logged in as Administrator.  To defeat that, I entered a command to turn off User Access Control (UAC):

C:\Windows\System32\cmd.exe /k %windir%\System32\reg.exe ADD HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /v EnableLUA /t REG_DWORD /d 0 /f
But that didn't do it.  The message was still basically the same:  "C:\Users\All Users\Start Menu is not accessible."  (Later, I saw that Win7 had a message for me, saying that I had to reboot to turn off UAC.)  I did another search and found a page that said that, actually, that was a junction point too.  Where I *really* wanted to go under each username there at C:\Users, it seemed, was the AppData\Roaming\Microsoft\Windows\Start Menu folder.  I found those folders for Administrator and All Users.  So -- wasn't there a registry location pointing the Start Menu to those folders, the way WinXP's registry would point to specific folders (and could be modified to point to others instead)?

As I thought about it, I wondered if I could use the approach (above) of doing a registry edit that would rewire the Favorites button in the classic Start Menu, and have it point to my shared Start Menu at D:\Installation\Start Menu.  That had already occurred to me, but the problem there was that, if I couldn't move items from C:\ProgramData, my Start Menu on D would be just ignoring the items on C.  But now I realized that I probably could create shortcuts to the items on C, and put those in my Start Menu on D.

So I started to do the registry tweak as described in the sources cited above.  But when I got to HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders, I saw that there was in fact not only an entry for Favorites, which is what the abovementioned tweak would use, but also for Start Menu itself.  Well.  So why not just redirect that?

My WinXP registry edits had redirected a number of items in that same folder, having to do with not only the Start Menu but also with other folders.  I decided to see if I could retain most of those redirections in the .reg file I was preparing to tweak new Windows 7 installations.  But for present purposes, I basically closed other programs, ran a .reg file with lines like these:
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders]

"Administrative Tools"="D:\\Installation\\Start Menu"
"Programs"="D:\\Installation\\Start Menu\\Programs"
"Startup"="D:\\Installation\\Start Menu\\Programs\\Startup"
"Start Menu"="D:\\Installation\\Start Menu"
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders]
"Programs"="D:\\Installation\\Start Menu\\Programs"
"Startup"="D:\\Installation\\Start Menu\\Programs\\Startup"
"Start Menu"="D:\\Installation\\Start Menu"
and sat back to see what would happen.  The registry changes were visible instantly in regedit.  On reboot, Win7 gave me an error message:
Location is not available
D:\Current refers to a location that is unavailable.
That was one of the locations I had specified in the reg file.  None of the registry edits had worked; I was still looking at the same Start Menu, or less.  Worse, the system wouldn't even run Windows Explorer.  I restarted in Safe Mode (i.e., I held F8 after the first BIOS bootup screen), but decided to try the System Recovery option instead.  But the last System Restore was even older than the most recent image backup, which seemed odd, so I went with the image.  (Apparently I was tired.  This was at day's end.  I didn't try checking the "Show more restore points" when I ran rstrui.exe.)  I probably could have just restored my backup of the affected registry keys, but I decided this would be a good first chance to test Win7's built-in system image feature.  I ran it and went to bed.

When I got up, the image restore had worked, so I tried some of those registry edits again, one at a time.  This time, though, I saved System Restore points along the way, using Control Panel > System > System protection > Create.  I saw that I had left out some steps.  First, as described in a separate post, I tried to move the fonts folder.  That failed.  It was OK; it was just an attempt to avoid having to save any new fonts in a separate location, to be restored as an extra step of any future system reinstallation. 

Some time passed, as I worked on other parts of the Win7 installation.  Installation of other programs added more icons to the Start Menu.  When those other installations were done, I came back to this draft post.

I decided to keep using the Classic Start Menu utility.  It worked for my purposes.  I saved its settings to a backup file.  With the registry edits shown in my post entitled "Windows 7 Installation - First Try," I was able to get the Start button to point to my shared Start Menu at D:\Installation\Start Menu.  With Classic Start Menu running, I clicked on Start > right-clicked on Programs > Properties > Location .  I specified the place on drive D where I kept my customized Start Menu > Programs folder.  I clicked Move and then closed out of that dialog.  I then went into C:\ProgramData\Microsoft\Windows\StartMenu and into the AppData\Roaming\Microsoft\Windows\Start Menu folder under each username at C:\Users, to see if there were any other program shortcuts that I would want to move to that location on D, or any other installed programs there to which I would want to link by shortcut.  There were none.

Other than some rearranging, that was it.  I had a Windows 7 Start Menu on drive D.  Drive D could be a network share, or it could be mirrored to other computers, as described in another post I was developing at about this same time.

Saturday, July 25, 2009

How to Find a Large Number of Random Files on Your Computer

I had a list of more than 1,600 old files that used to exist on my computer. I wanted to find out if these files were still there and in good condition, or if they had gotten lost or accidentally deleted over the years.

I was using Windows XP. If it had been just a few files, I could have done a search manually, one file at a time. (I use WinXP in classic mode, so for me the sequence would be Start > Search > For Files or Folders.) But with this large number of files, I needed to automate the process. Here's the approach I used.
1. Spreadsheet. Put the list of files in Excel. If you don't have Excel, one free alternative is the spreadsheet in OpenOffice. Sort the list of files alphabetically and view it to see if you have any duplicates. Duplicates will make your computer repeat an unnecessary search. These searches can take a long time if you have a lot of files. If you know how, you can use the spreadsheet to filter out unique records. In Excel 2003, that option is in Data > Filter > Advanced Filter. For me, this reduced my starting list of 1,600 file names to just a few hundred.
2. DOS. Write a DOS command for each file, in the spreadsheet column next to the filename column. The DOS command will search for the file. The command for this purpose is DIR. (Capital letters aren't necessary in DOS commands, but I'll use them here for clarity.) Most DOS commands come with additional options. In this case, I used the S, B, and -C options. To write the DOS command automatically for all of the files in my list, I figured out the command format in the first row of the spreadsheet and then just copied it down to all the other rows. Using my first row as an example, the command I wanted was DIR "FILE1.DOC" /S /B, with the quotation marks. Those options would (a) search all subdirectories for the file and (b) give me the location of the file on the same line as the filename. My list of files was in column G of my spreadsheet, so on line 2 of column H, under the column title (i.e., in cell H2), I wrote this:
="DIR "&CHAR(34)&G2&CHAR(34)&" /S /B"
with the quotation marks. CHAR() inserts a symbol; 34 is the number of the quotation mark symbol. That is, CHAR(34) allowed me to show a quotation mark in the result, without confusing Excel as to what that quotation mark was supposed to mean. Thus, when I was done entering those characters in cell H2, the cell looked like this:
DIR "FILE1.DOC" /S /B
These characters, if entered as a DOS command, would search for the file named FILE1.DOC and would tell me where it was, if it existed. The quotation marks were useful because some of my filenames contained spaces, such as "Letter to Joe.doc." Without the quotation marks, DOS would stop at the first space, assuming that the name of the file was simply "Letter," and the rest of the filename would confuse DOS and yield a failed search. When I had the formula for the search command I wanted to use in DOS, I copied it down to all rows in the spreadsheet. Thus, for instance, cell H3 now displayed a search for a file called FILE2.DOC, in this form:
DIR "FILE2.DOC" /S /B
3. Redirect. Indicate that you want to save the search results in a file. To do this, I needed to add a redirection sign to my commands. The redirection sign ">>" would create the output file if it didn't exist, and would append the results of the current command to the file if it did already exist. I went back to cell H2 and revised its formula to look like this:
="DIR "&CHAR(34)&G2&CHAR(34)&" /S /B >> "&CHAR(34)&"D:\File List.txt"&CHAR(34)
I needed the additional CHAR(34) items because my output file name, "File List.txt," contained a space. When I entered that revision in cell H2, the resulting command looked like this:
DIR "FILE1.DOC" /S /B >> "D:\File List.txt"
This would list all of the found files in an output file located at the root of my drive D. If I didn't know the exact filename I was searching for, I would use wildcards in my DIR command. For example, a search for FILEX*.* would give me all files whose names began with FILEX. Finally, copy this revised DIR command down to all rows in your filelist spreadsheet.
4. Batch. Copy all of these commands into a DOS batch file. To create a DOS batch file, open Windows Notepad and copy and paste from the spreadsheet to Notepad. In my case, the first two lines in Notepad now read as follows:
DIR "FILE1.DOC" /S /B >> "D:\File List.txt"
DIR "FILE2.DOC" /S /B >> "D:\File List.txt"
Save the DOS batch file, but don't close it yet. For simplicity, I saved mine to the same place as the output file (i.e., D:\), and I called it simply BATCH.BAT. The BAT extension is important; it makes the file executable. Without that, you've just got a text file that won't run. Also, in Notepad, be sure to save BATCH.BAT in ANSI format. You will find this option under Notepad's File > Save As > Encoding option.
My batch file was now sitting in D:\, but the place I wanted to search was on drive E. So I needed to add commands on the first two lines of BATCH.BAT. While I was still in Notepad, then, I went to the top of BATCH.BAT and inserted these two lines:
E:
CD \
These commands would tell DOS to go to drive E and then go to the root directory (which is also known, confusingly, as the "top" of the DOS subdirectory tree) before it starts its search. Now it was all set to run its searches across the entire drive E. Of course, I could go back into BATCH.BAT later and change its first line to refer to drive F instead, and then it would run the whole search again on drive F and would append the results to my D:\File List.txt file. (To edit a batch file, right-click on it and select Edit.) To automate the whole thing, of course, you could just repeat the contents of BATCH.BAT in a separate batch file for drives E, F, etc., and run them all at the same time (though you'd better do a global find-and-replace so they aren't all trying to write to File List.txt at the same time); or if you didn't want them all running at once, you could just copy and paste the list of commands again and again in BATCH.BAT, adding a line in front of each set to direct the computer to drive D, then E, then F, etc.
If you would like to test any lines from BATCH.BAT at any time, you can copy and paste them to a DOS command box. You can open a DOS command box by going to WinXP's Start > Run option and typing CMD into the space (and then hit Enter). You can also test the batch file as a whole by saving some of its lines into another batch file (let's call it X.BAT) and running that.
Note: when you're done with these batch files, delete them or rename them without the BAT extension. For instance, you might rename BATCH.BAT as BATCH.TXT. You don't want executable batch files sitting around, ready to go off and do all kinds of strange things if you accidentally click on them. You can put explanatory comments into batch files, but be sur eto precede each comment line with "::" or "REM" (short for "remark"). If you don't "comment out" your textual remarks in a batch file, DOS will try to run them as though they, too, were commands.
5. Run and Wait. Used to be known as "hurry up and wait," but then life sped up. (Kidding.) To run BATCH.BAT, save it with the changes just described, and then go to D:\ in Windows Explorer and double-click on it. The search can easily take hours if you are searching for a lot of files on several large hard drives.
6. Problem: Long Directory Names. As I watched the command run, I saw that I was getting an error message. It said, "The directory name [name of folder] is too long." It seemed to be doing this for one folder in particular. When I checked that folder in Windows Explorer, I found that it was buried so deep that I could not even view its contents. I killed the CMD window where BATCH.BAT was running, moved that deeply buried folder to a shallower location, deleted File List.txt (so that it would not contain repetitive information), and then re-ran the batch file. This fix told me that DOS commands in WinXP did not suffer from the much shorter file name limits found in original DOS. (If this hadn't worked, I was considering a revised batch file in which I would first have the system go to each subdirectory and then run its DIR command there. This approach might have involved printing out the directory tree and using the results as input to DIR.)
7. Further Operations. The foregoing steps resulted in a list showing that some of the files I sought did indeed exist on my computer, and showing where they were. That was the purpose of this blog post, so I won't detail additional steps. Briefly, though, in my case I wanted to gather those files in one location, so that I could take a look at them. I created a folder (D:\Gather) for the purpose. I could have copied this list of files into an Excel spreadsheet, and (using the same kind of formula as above) could have developed, for each of these files, a new batch file that used the DOS MOVE command to achieve the move. Each line in that batch file would use basically this format:
MOVE /-Y "D:\Folder\Subfolder\Filename.doc" D:\Gather
The Excel spreadsheet would give me a count of how many files I was expecting to find in the Gather folder, and of course Windows Explorer would show me how many actually made it. Before doing that in Excel, though, I realized it would be easier, with search and replace, to add the necessary ingredients to each line using Microsoft Word. Since not all filenames ended with ".doc," I had to do at least one search and replace involving ^p (Word's formula for the end of a line). Note that Word may default to using smart quotation marks, or may otherwise insert characters that won't run as plain text in a batch file, so be sure to do a final check (and run one or more search-and-replaces, copying samples of the smart quote and an actual, typed Notepad-style quotation mark from the text and pasting it into the search box, if needed) in Notepad. If you want to preserve the error messages (if any) produced by DOS in this process, you may want to run BATCH.BAT from the DOS command line instead of double-clicking on it in Windows Explorer.
As always, if you find this helpful, please enter a comment. Cheers!