Troubleshooting Some Options for Converting PDF to JPG or Other Image Formats

I had a bunch of PDFs.  I wanted to glance at just the first page of each.  This post describes some steps I took to find a way of extracting those first pages and making them easily available.

I believed it was going to be a simple matter to export from PDF to JPG using either Adobe Acrobat or IrfanView.  I had used both successfully in the past for this kind of purpose.  Unfortunately, for some reason Acrobat's Advanced > Document Processing > Batch Processing option was not working for me on this particular day.  Instead, I went into IrfanView > File > Batch Conversion/Rename, set the output format to JPG, selected the folder containing the newly created PDFs, clicked "Add All," set the output directory, and clicked Start Batch. Yet here, again, unknown causes were conspiring against me. IrfanView said, "Error! Can't load [filename]" for each of the PDFs. I tried exporting to PNG instead of JPG; same result. Was I mistaken in believing that IrfanView could produce JPGs from PDFs?  It had worked in a previous post.  Why not now?

A search led to a thread that raised the question of whether the source files were corrupt. I checked several of these newly created PDFs and they did seem to open OK in Acrobat.  So corruption didn't seem to be the issue.

At first, in that previous post, I had gotten that "Can't load" error.  The solution there was to download an upgrade to Ghostscript. Maybe I had updated IrfanView since then and, in the process, had somehow broken the Ghostscript update? I'd also gotten the error, another time, when trying to use IrfanView for audio conversions, which it apparently couldn't do.  So, ah, maybe what I was trying to do was confusing IrfanView.  I tried again with just one PDF.  There didn't seem to be much that could go wrong with that.  But it still didn't work.  When I tried to open a PDF with IrfanView, it gave me a message:

"Decode error! Can't load Ghostscript or Ghostscript error. Install Ghostscript from http://sourceforge.net/projects/ghostscript or http://sourceforge.net.

I went to the former. It pointed me to another page, and that one pointed me to yet another. It looked like Ghostscript had indeed been updated within the past month or so. I downloaded and installed the update. It didn't solve the problem, though possibly a reboot would have helped.

Was there another way to export JPGs or PNGs from PDFs? I tried XnView, which Irfan claimed was somehow built on code appropriated from him. Its interface certainly was very similar in this particular task. It didn't produce any JPGs either. A post reminded me that I hadn't tried IrfanView in command-line mode. The formula I had worked out previously was like this:

i_view32 D:\Folder1\File35.pdf /c=d:\TestFolder\File35.jpg

but now that was giving me Ghostscript errors too. A search led to suggestions to try ImageMagick and iTextSharp. The latter seemed beyond me. There were also a couple of suggestions on using scripting in GhostScript. They were a bit technical for my taste at this time.

It looked like ImageMagick had been around for a long time -- there were books about it -- and I had run across a number of references to it. So I decided to start there. The copy I downloaded from CNET was corrupted, so I got another from the ImageMagick FTP site. Unzipped, the ImageMagick program folder was 146MB and contained 5,694 files. This was no little alternative to IrfanView. This was a doctoral dissertation.  I immediately cast about for a GUI front end -- I couldn't even find its executables -- and alighted on Converseen.  I downloaded it from Softpedia and installed it.  Would it convert my PDFs to JPGs? I named three PDFs to try it. All three failed with this error:

Error: Magick: Postscript delegate failed [filename]: No such file or directory @ error/pdf.c/ReadPDFImage/664

Now, what do you suppose that meant? I tried a search and got a couple of very helpful sites in Chinese. I modified the search and, lo, it looked like that was an ImageMagick error, not a Converseen error. I gathered there could be a couple of possibilities. One was that, as someone reported, this error was likely (for some unknown reason) when converting PDFs that were mostly text, as mine were, rather than mostly image. Also, there seemed to be a regular cascade of programs: Converseen was a front end for ImageMagick which used GhostScript. So I couldn't necessarily be sure which one was responsible. For that matter, I didn't even know where to put ImageMagick; I wasn't seeing a manual. Further reading suggested that resolving problems in these programs could be challenging.

I tried another search. At the top of the list: Boxoft PDF to JPG Converter. Its installation process said, "Requires Ghostscript. Do you want to get a free GPL Ghostscript?" It seemed I was destined to have a computer full of copies of Ghostscript.  Or maybe this installation would be the answer to all of the problems described above.  I downloaded and installed the proffered Ghostscript 8.71.  The process didn't look familiar.  I wasn't sure what I had been doing with Ghostscript previously, but this wasn't it.  So would this perchance fix IrfanView?  I opened IrfanView and, wow, now it ran, no reboot necessary.  The way to fix IrfanView was to install Boxoft (or, possibly, to reinstall IrfanView).

But something was wrong.  Didn't IrfanView normally give me multiple pages, one PDF per page, when I used it to convert PDFs to JPGs?  This time, I was getting only one JPG per PDF, no matter how many pages it had.  I assumed that later Ghostscript installations replaced earlier ones, but now I saw that was not necessarily so.  I went into Control Panel > Programs and Features.  I saw three different Ghostscript items.  I wasn't sure if I could uninstall two of them without screwing up the third one.  I tried it, keeping only the Ghostscript 8.71 that I had just installed.  Boxoft and IrfanView both ran as before.  So cleanup accomplished, but IrfanView problem not fixed.

Since I now had Boxoft installed, I tried that.  Like IrfanView, it had a command-line mode and a GUI mode.  I tried the GUI.  It had multiple options.  I liked it.  It did produce multiple JPGs per PDF.  They looked good.  I ran it again, this time selecting only page 1 (Settings > Common tab > PDF Convert Range), and that's exactly what it gave me.  So I wouldn't have to take a separate step of deleting JPGs for the pages other than page 1 of each PDF.  IrfanView didn't have that option, though it did have image editing options (if e.g., I wanted to change the dots per inch, size, or coloring of the resulting JPGs).

I could have continued to look at other options appearing in that search, but it appeared a working conclusion was to go with Boxoft and/or IrfanView, depending on the kind of output desired, and to make sure I was using just one stable, recent version of Ghostscript.

Using Screenshots to Examine Many Files Quickly

I had a couple of projects that called for a way to examine a large number of files.  It seemed that screenshots could help in those projects.  This post describes the techniques I used.

EML Analysis

In one project, I was working with various email files that I had exported from Thunderbird.  These files had an EML extension.  Typically, if I viewed an EML file in Notepad, I would see various codes and other information that wouldn't be visible if I viewed it in an email program like Thunderbird.

I was interested in seeing the header codes in these EML files.  Those codes appeared at the tops of the files.  I felt that I could probably see what I needed to see in the first screenful of a Notepad session, opened maximized.

In other words, the concept was that I would open the EML file in Notepad; I would take a screenshot; I would save the screenshot; and then I would close the file and repeat the process with the next EML file on my list.  Then I would combine all those screenshots into one file, and flip through it or perhaps use other tools to analyze it further.  I wouldn't have to sit there, maintaining constant attention while the process continued in real time; I could just review the outcome afterwards.  (For some purposes, an alternative would have been to combine or select from the text, without a graphical view.)

The first step was to build the list of EML files that I wanted to examine.  I moved them all into a single folder and used DIR and Excel to give me the list and to convert it into a series of batch commands.  There was one such command for each such EML file.  Before running those commands, I had to open Notepad once, turn on its Format > Word Wrap option, and then close it.  The format of the command was as follows:

start /max notepad "D:\Folder Name\Email Name.eml"

That command was sufficient to open the EML file.  Next, I needed to pause the system for a moment, so that the file would have time to come onscreen. Among numerous suggestions, I favored a command involving PING ("ping 1.1.1.1 -n 1 -w 1500 > nul") because of its fine-tunable setting (in the example just given, 1500 milliseconds).  Unfortunately, that command's output component (" > nul") would have prevented me from adding more commands on the same line.  So I had to go with "TIMEOUT /T 1" for a one-second delay.

Next, I needed a command to take a snapshot.  It looked like there were multiple options here.  I had already installed NirCmd and had found it useful for other things, so I used this command:

start NirCmd savescreenshot "D:\Folder Name\Screenshots\Email Name.png"

NirCmd came with an option to copy its executable (nircmd.exe) to C:\Windows, so that this command could run without any need to specify the location of NirCmd, to put a copy of it in the current working folder, or to modify the computer's Path.  NirCmd wasn't saving to subfolders properly, so in the end I had to modify that part of the command.

Finally, I needed a command to close Notepad.  The advice that worked for me was:

taskkill /f /im notepad.exe

Note that this would close all currently open Notepad sessions.  These three steps (i.e., open the EML in Notepad, take a picture with NirCmd, close Notepad) would give me a screenshot of the first screenful's worth of the file's contents.  Collectively, those screenshots would give me a visual impression of the various kinds of codes appearing at the start of my EML files.

I used && to combine multiple commands on the same line, as a single (long) batch command. If that had failed, I could have added index columns next to the spreadsheet columns in which I built those two commands, with alternating even and odd numbers in those columns:  1 for the first Notepad command, 2 for the first NirCmd command, 3 for the second Notpad command, and so forth.  These index numbers would allow the various commands to be sorted into proper sequence in a single column, for copying and pasting into a batch file.

In short, for each EML file, I combined four commands with &&, into a single long command like this:

start /max notepad "D:\Folder Name\Email Name.eml" && timeout /t 1 && start NirCmd savescreenshot "Email Name.png" && taskkill /f /im notepad.exe

This gave me some PNGs.  Now there was the question of what to do with them.  One option was to simply stitch them together in a slideshow (using e.g., IrfanView) or a single PDF (using e.g., Acrobat).  I did a brief investigation of OCR software for that purpose.  Ultimately, I just used IrfanView, without even creating a slideshow, to arrow down through those PNGs, one at a time, at whatever pace I chose.  So I could look at whether each page came through OK.

PDF Analysis

In another project, I had a bunch of PDFs that I had created in a conversion process.  I wanted to check if the PDFs came through OK.  It would have been very slow to open them, one at a time, and page through them.  Combining them all into a single large PDF, which I could also page through, would have produced a huge file.  Also, if I was working with large PDFs or many PDFs (or both), I might have to look at huge numbers of pages.  Boredom or haste could lead me to flip past an important one-page document, while checking hundreds or thousands of less important pages.

Based on various factors (including the number of PDFs, their importance, and the time available), I decided to examine just the first page of each PDF.  I might not be able to tell if the whole document printed properly, but at least I could eliminate those instances where printing failed completely.

For this purpose, the process described in the previous section offered one possibility.  I could probably work up a set of commands to open a PDF, take a screenshot, and then close it, and then flip through the resulting screenshots.

I did not actually pursue that approach in this case, however.  Instead, I wanted to see if I could convert the PDF documents to JPG and then flip through just the first page from each such document.  If I had a hundred documents to check, I would have a hundred pages to look at -- not a thousand.  A separate post discusses that investigation.  The tool I chose was  Boxoft PDF to JPG Converter.  Another way to proceed might have been to split the PDFs first, using something like PDFsam, and then combine the PDFs of each resulting first page into a larger PDF that I could flip through.

Batch Merging Many Scattered JPGs into Many Multipage PDFs - Clarified

I had recently streamlined a spreadsheet-oriented technique for working up file names and batch commands to merge multiple JPGs into single- or multi-page PDFs.  I had yet another task calling for that process.  This post presents the same steps as in that previous writeup, but in clearer and more organized terms.  The various steps are described in different terms in the previous post and in its much more convoluted predecessor.  Note that the simplification of steps here yields a longer explanation.

Generate the Input PDF Filenames

In this discussion, the project had already been brought to the point where there was a spreadsheet containing the names of JPGs to be converted to PDF.  (A list of such files could come from a command like DIR D:\*.jpg /s /a-d /b > JPGLIST.TXT.)  In that spreadsheet, row 1 was reserved for column names (e.g., "Original Filename," "Path").  The name of the first file to be converted appeared in cell A2 (e.g., "Test File.jpg"), and the others appeared below it in column A.

Column B was used to calculate the endpoint of the path (e.g., "D:\Folder 1\Test File.jpg").  This calculation required or at least benefited from the use of a REVERSE function.  The REVERSE function was created by inserting the following text into a module:

Function Reverse(Text As String) As String
Dim i As Integer
Dim StrNew As String
Dim StrOld As String
StrOld = Trim(Text)
For i = 1 To Len(StrOld)
StrNew = Mid(StrOld, i, 1) & StrNew
Next i
Reverse = StrNew
End Function

To create the module, I closed all Excel files other than the one being worked on.  I went into Tools > Macro > Visual Basic Editor > Insert > Module.  I pasted the text there.  I went to to File > Close and Return to Microsoft Excel.  The formulas in the next several cells were then as follows:

B2:  =LEN(A2)-FIND("\",REVERSE(A2))+1
C2:  =LEFT(A2,B2)
D2:  =MID(A2,B2+1,LEN(A2))

I was reconstructing some of these steps after the fact; hopefully I was not misstating anything or leaving anything out of this description.  I focused on the formulas for row 2 because, when I was done filling out row 2, it would be easy to copy all of its formulas at once to fill the spreadsheet, so that the same formulas would be applied to the filenames in rows 3, 4, 5 et seq.

The next step was to identify which files I wanted to convert.  I was able to use the spreadsheet to help in this process.  For example, I had files with names like these:

Test File page 01.jpg
Test File page 02.jpg

I could determine that these belonged together by using a column that showed these filenames without the ending X characters.  That column would use a command like =RIGHT(D2,$E$1), where I would put a number in E1.  The number I used in this example was 12, because I would have to eliminate 12 characters from the right end of "Test File page 01.jpg" to get the name that I wanted to give to the new PDF, which would be simply "Test File."  I manually entered 12 in cell E2.  I filtered the rows so that the ones with numbers in column E would not show, so as to reduce clutter.  I repeated this process until each row had a number in column E, indicating how many characters I had to trim from the right end of the filename to get a common stem.  Then cell F2 contained =LEFT(D2,LEN(D2)-E2), to produce the stem filename.  So, for example, if there were two pages that had to go into Test File.pdf, then (assuming I had sorted my spreadsheet correctly), the contents of cells F2 et seq. would look like this:

F2:  Test File
F3:  Test File
F4:  Another File

and so forth, continuing down the list, so that each JPG would be assigned to a stem filename.  I discovered at this point that I had brought over a bunch of JPGs that did not belong in a multipage PDF, or at least I could not be sure of it.  I put these back where they came from, and thus had a smaller set of files to work on.

Now I could sort the spreadsheet to insure that the JPGs I wanted to merge into a specific PDF would be next to each other.  Once I had the files sorted into the desired order, I used column G as an Index column, numbering each row in that order.  This column would be useful to restore the desired order if I resorted the spreadsheet for some reason, and it would also provide the key ingredient in the unique filename that I was about to calculate.  (To generate an index number, I used either a +1 formula, incrementing the number in the preceding row by 1, followed by an Edit > Copy, Edit > Paste Special > Values sequence to convert those formulas to fixed numbers that would not be affected by resorting the spreadsheet; or, easier, I entered 1 in the first row, 2 in the second row, and then highlighted the cells to be filled and used Edit > Insert > Series to add the sequential numbers.)  Then I used columns G through I to calculate a simple, unique filename and produce a renaming command:

G2:  unique Index number
H2:  ="Input_"&REPT("0",4-LEN(G2))&G2&".pdf"
I2:  ="ren "&CHAR(34)&A2&CHAR(34)&" "&H2

Cell H2 would produce the file name of "Input_0001.pdf."  (More or fewer digits could be appropriate, depending on the number of files involved.)  This name worked for me because a search of my files verified that my computer contained no other files whose names began with Input_.  Otherwise, a different name might have been advisable.  Cell I2 would produce a command that would rename Test File page 01.jpg to be Input_0001.pdf.  (Of course, this wasn't a conversion.  IrfanView would be doing that.  This was just a translation of file names, to make subsequent steps easier.)

Then, as usual, I copied these commands from row 2 down to fill the remaining rows of the spreadsheet.  Next, I copied all of the commands from column I into Notepad, saved that file with a .bat extension (e.g., "Renamer.bat"), and ran it in Windows Explorer.  That renamed all of the JPGs that I hoped to merge into multipage PDFs.  (Obviously, I had a backup.)

Now I would use column J for commands to move all those JPGs to a single folder.  (I did the renaming before the moving in case I had two separate sets of incoming JPGs with the same name, such as D:\Folder A\Test File page 01.jpg and E:\Folder R\Test File page 01.jpg.)  In my actual example, the files were all moved to V:\Workspace, so my command in cell J2 looked like this:

J2:  ="move /-y "&CHAR(34)&C2&H2&CHAR(34)&" V:\Workspace"

I combined the commands in column J into another batch file, Mover.bat, and used it to move all those renamed JPGs to V:\Workspace.  Now that they were together, I could use IrfanView's File > Batch Conversion/Rename option to convert all those JPGs to single-page PDFs.  I output those PDFs to V:\Workspace and then removed the JPGs from V:\Workspace to V:\Workspace\Original JPGs, where they would probably not be needed further.

Generate the File Value Lines

So now I had files starting with Input_0001.pdf in V:\Workspace. The project now was to merge those single-page input PDFs into multipage output PDFs. In column K, I entered a number indicating where each input PDF should go. As I could see from column F (above), the files that used to be called Test File page 01.jpg and Test File page 02.jpg, which had now been converted to uniquely named single-page PDFs, were both going to end up in Output_0001.pdf, so both of those had the number 1 in column K.   I needed to pad out that number so that the resulting filename would sort correctly. So the next formulas were as follows:

K2: =LEFT(H2,11)&"pdf"
L2: =REPT("0",4-LEN(K2))&K2

But I wasn't ready, yet, to generate the ultimate Output PDF files.  I needed to make sure that PDFsam (below) would have all the instructions needed to insert multiple single-page PDFs into one Output PDF.  Those instructions would come from lines of XML code, one line per input PDF.  The lines of XML code would be combined into a smaller number of FileValueLines files.  In my example, the instruction lines for the erstwhile Test File page 01.jpg and Test File page 02.jpg files (now called Input_0001.pdf and Input_0002.pdf) would both go into a single FileValueLines file.  A FileValueLines file could contain any number of lines of XML code; it just depended on how many single-page PDFs were going to be combined into it.  What I needed, then, was the command that would shovel those XML code lines into the appropriate FileValueLines files.  The next formulas in my spreadsheet looked like this:

M2:  ="FileValueLines_"&L2
N2:  ="echo ^<file value="&CHAR(34)&"V:\Workspace\PDF\"&H2&CHAR(34)&" /^> >> "&M2

and cell N2 produced this command:

echo ^<file value="V:\Workspace\PDF\Input_0002.pdf" /^> >> FileValueLines_0001

Note that this command assumed that the PDFs created by IrfanView were in a folder called V:\Workspace\PDF.  I filled the lower rows of the spreadsheet with these formulas and copied the commands in column N into a new Notepad file called _ValueMaker.bat.  I ran _ValueMaker.bat in V:\Workspace. It produced 545 FileValueLines files in V:\Workspace, as expected.  (I added an underscore before the file's name so that it would appear near the top of the potentially cluttered list of files in V:\Workspace.)

This was a point of transition.  Until now, I had been working with a total of 3,894 spreadsheet rows, one for each of the 3,894 JPGs that I wanted to combine into multipage PDFs.  Each of those 3,894 JPGs (converted into single-page PDFs called Input_0001.pdf et seq.) was represented in exactly one File Value line.  Those 3,894 File Value lines were now contained within a total of 545 FileValueLine_ files.  In other words, we were transitioning from a focus on 3,894 individual JPGs to a focus on 545 multipage PDFs.

Building the XML Files

The 545 FileValueLines files (FileValueLines_0001 and so forth) needed more lines before they would function.  To add those lines, I needed a new spreadsheet.  I started this one with a list of the FileValueLines files I had just created.  (A command like "dir FileValueLines* /b /a-d > _FVLlist.txt" would produce that list.)  In other words, column A contained the list of the new FileValueLines files.  Next, I calculated the name of the resulting XML file, and then wrote the formula that would create those XML files:

B2:  =RIGHT(A2,4)
C2:  ="XMLfile_"&B2&".xml"
D2:  ="copy /b _Header.txt+"&A2&"+_Tailer.txt "&C2

That formula in D2 produced the command that I would need to create the complete XML files:  "copy /b _Header.txt+FileValueLines_0001+_Tailer.txt XMLfile_0001.xml."  This command called for me to create just one _Header.txt and one _Tailer.txt file that would be added to the start and end of each of my XML files.  The _Header.txt file read as follows:

<?xml version="1.0" encoding="UTF-8"?>
<filelist>

I saved those two lines in a new _Header.txt file.  It was important to make sure the file ended on a blank line.  In other words, if I moved my cursor down to the end of _Header.txt, it needed to be on the line below <filelist>, not at the end of the line containing <filelist>.  The same was true for _Tailer.txt, which contained this line:

</filelist>

Once _Header.txt and _Tailer.txt existed, I could run the commands contained in column D.  I copied those 545 commands into a new batch file called _Builder.bat and ran it.  That gave me 545 XML files, starting with XMLfile_0001.xml.  Again, V:\Workspace was filling up, so I moved the no longer needed FileValueLines files to their own archival subfolder.

Generating the PDFsam Commands

At this point, I would use a batch file to generate batch files.  This would call for the same kind of process as above:  a header text file (but no tailer) containing lines that would be standard in all of these batch files, combined with another text file containing unique commands.  In this new _HeaderB.txt file, I placed these lines:

@echo off

set JAVA=%JAVA_HOME%\BIN\JAVA

set JAVA_OPTS=-Xmx256m -Dlog4j.configuration=console-log4j.xml

set CONSOLE_JAR="C:\Program Files (x86)\pdfsam\lib\pdfsam-console-2.3.1e.jar"

@echo on

Apparently these first lines set up a few essential Java variables.  (In case of difficulty here, as elsewhere, see the preceding post.)  The second part of the batch file -- the unique command -- would tell PDFsam (below) what to do.  The command looked like this:

E2:  ="echo %%JAVA%% %%JAVA_OPTS%% -jar %%CONSOLE_JAR%% -l V:\Workspace\"&C2&" -o V:\Workspace\Merged\Output_"&B2&".pdf concat >> Command_"&B2&".txt"

Basically, this would produce a command that said, "Write a command that will create Output_*.pdf, and put that command in a file called Command_*.txt."  In other words, that spreadsheet formula in cell E2 produced a command that looked like this:

echo %%JAVA%% %%JAVA_OPTS%% -jar %%CONSOLE_JAR%% -l V:\Workspace\XMLfile_0001.xml -o V:\Workspace\Merged\Output_0001.pdf concat >> Command_0001.txt

Spreadsheet column E gave me 545 variations on that command.  I copied them all and ran them in a new batch file called _Texter.bat.  The resulting 545 Command_*.txt files would have to share V:\Workspace with my preexisting XMLfile_*.xml files; both would be needed shortly.

Now it was time to combine _HeaderB.txt and Command_*.txt into a batch file that would run PDFsam and produce an multipage Output PDF.  This was easy enough:

F2:  ="copy /b _HeaderB.txt+"&RIGHT(E2,16)&" Ready_"&B2&".bat"

The 545 versions of that command, run via _Commander.bat, created 545 Ready_*.bat files, each containing the header information from _HeaderB.txt plus the single command line from Command_*.txt.  Now I was done with the Command_*.txt files, and could archive them.

Final Run:  Creating the Merged Output PDFs

If I double-clicked on one of those Ready batch files, it would run and produce a final merged Output.pdf.  But I wanted to run them all at once.  So, back to the spreadsheet:

G2:  ="call "&RIGHT(F2,14)&" >> _Errorlog.txt"

That produced "call Ready_0001.bat >> _Errorlog.txt."  I copied the 545 CALL commands from column G into a file called _Runner.bat and -- you guessed it -- I ran it.  It took a few minutes, but it produced 542 merged PDFs.  Not 545.  I searched _Errorlog.txt for occurrences of "Error."  Sure enough, three occurrences.  In each case, the error was the same:  "Input_0838.pdf (and Input_0947.pdf, and Input_1090.pdf) not found as file or resource."  Apparenly PDFsam had not gone ahead to create the Output PDFs anyway, without the offending Input file.  I checked and, indeed, those three files were not included among my input.  I was not sure what happened to them.  They appeared to have disappeared during this process.  I made a note to retrieve them from backup.

Otherwise, though, the process seemed to have succeeded.  Spot checks indicated that I had working multipage PDFs.  Now it was time to rename them to be something other than Output_0001.pdf etc.  I went back to my first spreadsheet, saved it, and went to work on a temporary copy of it.  Specifically, I converted all of its formulas to numerical values.  (In Excel 2003, this involved going to the top left corner, selecting the whole spreadsheet, and using the Edit > Copy, Edit > Paste Special Values combination.)  With that done, I was free to delete and rearrange columns as needed, without having cells recalculate in undesirable ways.  I kept only columns C (showing the original source folder), F (showing the name I had chosen for the combined output PDF), and L (showing the output file number).  I rearranged those columns in reverse order, so that the output file number column could function as a rather redundant index.  I went to the second spreadsheet and, working likewise on a copy of that, deleted all columns except column B, containing the output file number.  In an adjacent column, I used VLOOKUP to look up the filenames, from the other spreadsheet, to which these Output_0001.pdf etc. files should be renamed.

Through such steps, I renamed these files.  Yet something went wrong.  Due to distraction and time delay during the last stages of this process, somehow the renaming did not bring me back to a coherent set of files.  Spot checks indicated that the ones that did rename were renamed correctly.  That is, if a file wound up being renamed as Letter to Joe, it did seem to be exactly that.  But I was left with 60 files -- more than 10% of the total -- that did not rename at all.  There may have been a way to fix them through the spreadsheet, but I had run out of time for the project.  Under the circumstances, I defaulted to putting those files aside into a separate folder, pending a future manual renaming effort.

The process described in this post did seem to work.  But its numerous steps continued to hold the threat of something going wrong, somewhere along the line, and being difficult to retrace.  This did not seem to be nearly the last word on batch merging scattered JPGs into multipage PDFs.

Batch Merging Many Scattered JPGs into Many Multipage PDFs - Streamlined

I was facing a task that I had undertaken before.  This presented an opportunity to revise and streamline the writeup that I had produced during that previous effort.

First, I had previously converted multiple JPGs to PDF, scattered among multiple folders, in some cases combining multiple JPGs into a single PDF.  The general approach taken there was to rename the JPGs to unique names, move them to a single folder, conduct my operations on them there, delete the JPGs, and move the new PDFs back to the folders where the JPGs had come from.

In a related effort, I had also previously converted MHTs to PDF.  Those were not multipage.  In that effort, I had taken the strategy of converting them in-place (i.e., without moving them to a single folder, transforming them there, and moving them back to the place of origin) and then deleting the MHTs.

Between these two strategies, I liked the former better.  Having all files in one folder made it easier to verify that my processes yielded the same number of output files as source files.  It was also easier to spot-check the output and make sure that the files looked like they should.

Now I wanted to clean up any remaining JPGs that should be converted and, in some cases, combined into PDFs.  Using techniques described in more detail in those previous posts, I ran a DIR to get a full list of JPGs.  I put that list into a spreadsheet, used a REVERSE function to identify paths, sorted the spreadsheet by path, and deleted those rows that contained images that I did not want to convert (e.g., photos).  This made it possible to reduce a starting set of thousands of JPGs into a list of a few hundred that actually needed to be converted.

Using my spreadsheet, I cooked up REN commands to run in a batch file.  This produced unique filenames, as described previously, so that I could use MOVE commands (or cut and paste them from a file finder) to pool them all into a single folder without fear of overwriting.  Of course, I needed to keep the spreadsheet, so that I would know what the original names and locations were, so that I could write MOVE commands to put the resulting PDFs back in the source folders.

Once the uniquely named JPGs were combined in one folder, I used IrfanView's batch capability -- again, as detailed in the previous posts -- to convert them to PDFs.  These were all one-page PDFs; I had one PDF per JPG.  I made sure the spreadsheet had a column associating these one-page PDFs (with names like ZZZ_0001.pdf) with their original filenames (e.g., Letter from Ed page 01.jpg).  That is, the current PDF name and the original JPG name would be on the same row.  Now I could use the original JPG name to calculate the name of the new multipage PDF (e.g., Letter from Ed.pdf).  So there would be spreadsheet rows like these:

ZZZ_0001.pdf   Letter to Ed.pdf
ZZZ_0002.pdf   Letter to Ed.pdf
ZZZ_0003.pdf   Letter to Jane.pdf
ZZZ_0004.pdf   Memo from ABCD.pdf
ZZZ_0005.pdf   Memo from ABCD.pdf

(Letter to Jane.pdf is there because I did a sweep for all JPGs, some of which would wind up having just one page.)  I sorted the spreadsheet alphabetically by the output filename and input PDF name (as shown, with the pages that would be going into Letter to Ed arranged in proper order, and with Letter to Ed coming before Memo from ABCD).  For convenience, I assigned a simple working name to the output filenames (e.g., Letter to Ed.pdf would be represented by YYY_0001.pdf, and Memo from ABCD.pdf by YYY_0002.pdf).  I expressed the relationship between the input (single-page) PDF filenames and the output (potentially multipage) PDF filenames with commands that would create the lines of the necessary XML files, in this format:

echo ^<file value="D:\Workspace\ZZZ_0001.pdf"/^> >> YYY_0001

That produced 460 batch file lines, each starting with "echo," that would generate most of the contents of 159 different XML files needed to produce 159 different single- or multi-page PDFs.  The XML files would each need to begin with these two lines:

<?xml version="1.0" encoding="UTF-8"?>
<filelist>

and end with "</filelist>" (without quotes).  I put those lines into Header.txt and Tailer.txt, respectively, and then combined them with a batch file containing 159 lines like this:

copy /b Header.txt+YYY_0001+Tailer.txt YYY_0001.xml

That batch file gave me 159 XML files, starting with YYY_0001.xml.  Now I needed 159 new batch files, one for each XML.  These batch files would tell PDFsam to do the actual work -- to merge the single-page PDFs listed in the XMLs into an appropriate output PDF.  So at this point, D:\Workspace contained those 159 XMLs and the 460 single-page PDFs that would soon be merged.  As above, each of these 159 batch files would begin with several lines.  Working in a separate folder, I saved those several lines in a new Header.txt file:

@echo off

set JAVA=%JAVA_HOME%\BIN\JAVA

set JAVA_OPTS=-Xmx256m -Dlog4j.configuration=console-log4j.xml

set CONSOLE_JAR="C:\Program Files (x86)\pdfsam\lib\pdfsam-console-2.3.1e.jar"

@echo on

Those lines made some assumptions about environment variables, which I had already set.  Only the final line of each of those 159 batch files would vary.  That final line would have two variables:  it would name the XML file that listed the PDFs to be combined, and it would name the output file that would contain those PDFs, as in this example:

%JAVA% %JAVA_OPTS% -jar %CONSOLE_JAR% -l D:\Workspace\YYY_00001.xml -o D:\Workspace\Merged\YYY_00001.pdf concat

I had to create the D:\Workspace\Merged folder to hold the output, and I had to write Excel spreadsheet formulas to mass-produce one final line, like the one just shown, for each of the 159 batch files.  The Excel formula I used was like this:

="echo %%JAVA%% %%JAVA_OPTS%% -jar %%CONSOLE_JAR%% -l D:\Workspace\"&A2&".xml -o D:\Workspace\Merged\"&A2&".pdf concat >> "&A2&".txt"

where cell A2 contained YYY_0001.  That gave me 159 batch commands that I put into Notepad, and saved and ran as Texter.bat.  That produced 159 files with names like YYY_0001.txt, each containing a single JAVA line.  Then, in the spreadsheet, I created another set of commands like this:

copy /b Header.txt+YYY_0001.txt YYY_0001.bat

to combine the header and the JAVA lines into 159 batch files, each of which would ask PDFsam to merge the single-page PDFs listed in the corresponding XML file into the appropriate single- or multi-page output PDF.  And finally, to run those 159 batch files and produce the desired PDFs, I used the spreadsheet to work up a batch file called Runner.bat that began like this:

@echo off
call YYY_00001.bat
call YYY_00002.bat

I had been doing some of this work in other folders, but at this point my simplistic approach would work only if I moved them all back to D:\Workspace before trying Runner.bat.

Altogether, the process worked for me, as it had before, and this time it took only a few hours to go through the foregoing steps, make the inevitable mistakes, and get the desired output.

The remaining step was to get these multipage PDFs back where they belonged.  I went back to the spreadsheet, prepared batch lines for that purpose, and finished the job.

JPG: Can't Read File Header - Unknown File Format or File Not Found

I was looking at various JPGs.  I noticed that a number of them produced this error message when I tried to view them in IrfanView.  I had checked the box to activate IrfanView's Unicode plug-in as suggested, and anyway these were not exotic file names.  So I didn't know what this error would mean for these JPGs.

A search indicated that numerous people had encountered this error.  I decided to start by verifying that this was not just a quirk of IrfanView.  It didn't seem to be; I also wasn't able to view these JPGs as icons in Windows Explorer, and when I tried to view them in Firefox, I got an error:  "The image [filename] cannot be displayed because it contains errors."  When I tried Internet Explorer, I got "Your web browser has blocked this site from using an ActiveX control in an unsafe manner."  Windows Photo Viewer said, "Windows Photo Viewer can't open this picture because the file appears to be damaged, corrupted, or is too large."  Chrome didn't show an error message; it just gave me a blank page.  Photoshop said, "Could not complete your request because an unknown or invalid JPEG marker type is found."  Microsoft Paint said, "Paint cannot read this file.  This is not a valid bitmap file, or its format is not currently supported."

Eliminating the Easy Solutions

A search led to indications that some problems of this type could be due to the program, as I had feared in the case of IrfanView.  For instance, one webpage indicated that a faulty Skype extension could produce the foregoing Firefox error.  Presumably an attempt to open the JPG in some other program, as above, would help to clarify whether it was a program issue rather than a JPG issue.  A discussion thread raised the prospect that this kind of thing could result from various kinds of file system or drive problems.  Other webpage said that USB flash drives (especially improperly removed) or Picasa could be an issue.

In some cases, a backup could be a solution, possibly beginning with a DoubleKiller search for other files having the same filename (just in case there might be another copy of the same file somewhere on the computer).  That discussion thread also suggested that it could be a machine-specific problem, but it wasn't in my case:  same problem when trying to open on another computer.

As noted in a previous post, building on an earlier effort and leading to some additional refinements, it was possible to use IrfanView to detect corrupted JPGs scattered around the computer.  The basic idea was to do a search for *.jpg (in e.g., a command window, or using a file finding program like Everything) and then run IrfanView (using either File > Batch or command line methods) to see which JPGs would fail to convert to another format (e.g., PDF).

I had previously reviewed commercial software for fixing JPGs. The prices were generally high and my confidence in them was not great. I only had a few dozen corrupted JPGs and wasn't eager to spend much money on fixing them.  One commenter, responding to my post, said that she'd had generally good results with JPEG Recovery Pro ($50), but she still seemed to be looking for a better solution.  Voters on CNET had given it less than two stars.

One reviewer on CNET (like others) said the demo version of Corel's Paint Shop Pro had been useful.  On CNET, it got 3.5 stars from 466 users, though it didn't look like the dozen users who had rated the current version were quite as pleased with it.  I downloaded the latest version from CNET.  It was large (366MB) and it took a while.  CNET said it would be a 30-day trial version, $60 purchase price after that.  (I could also have downloaded from Corel's website.  Oddly, when I started to do that, the download dialog said their version was only 282MB.)  Unfortunately, when I tried to open a few of my corrupted files, Paint Shop said, "An error occurred while trying to read from the file."  Same outcome with a half-dozen different corrupted files.

Digging into the Files

My efforts (above) suggested that I might or might not be able to find a program that would help to automate the repair of corrupted JPGs.  Assuming I did find such a program, the next question would be whether it would work on all corrupted JPGs.  I was not finding a clear, obvious solution.

In other words, it seemed that, sooner or later, I was going to find myself among those who were talking of manually editing JPGs to fix them.  I hadn't done that before.  I had no idea whether that sort of process could be even partially automated.  But the next step seemed to be that I should see if I could fix at least some simple problems in JPGs.

In a thread cited above, someone said that I could open a JPG in Notepad and could tell, from its first few characters, what kind of file it was.  A GIF would tend to begin with "GIF89," a JPG would begin with "ÿØÿà," and a PNG would start with "‰PNG."  I looked at a couple of my JPGs.  Sure enough, the uncorrupted ones did begin with with "ÿØÿà."  But the corrupted ones I examined didn't have anything like any of these three options.  I knew, anyway, that it wasn't a case of the file being saved with the wrong extension.  If that had been the problem, IrfanView would have caught it (given the program options I had selected) and would have offered to change it to whatever the correct extension should be.

Another site offered a step-by-step guide to the process of editing the JPG.  The editing required a hex editor.  That site recommended Frhed, whose interface was far friendlier than that of HexEdit.  I opened one of my corrupted JPGs in Frhed.  It looked a lot less crazy than it had looked in Notepad.  According to a user-friendly version of the step-by-step guide, the JPG consisted of two sections:  header and image.  Corruption resulted from having a bad header.  Or so we hoped.  The solution was to replace the bad header with a good one.  So I made a backup of the files I would be working on, and set to work.

The user-friendly guide recommended xvi32 instead of Frhed.  It was rated 4.2 out of 5 stars (Very Good) by 47 users (76,213 downloads) at Softpedia -- vastly more than Frhed -- so I downloaded and ran that instead.  I had to bump up its font a bit -- 8-point type seemed unduly ascetic.

Now that I was getting organized, I looked at my first corrupt JPG.  Unlike the other one I had just glanced at, it did not have anything except zeroes.  This file was completely toast.  The next one had data.  The guide said I should look for "ff da" in the hexadecimal data, so I did a Ctrl-F and searched for FF DA.  (For some reason, xvi32 insisted on entering capital letters.)  My search seemed to think that it had succeeded:  it stopped at something that read 9F FA.  Call me crazy, but that did not look exactly like FF DA to me.  I tried searching the same file in Frhed.  A search for ff da found nothing, and a search for just ff didn't find much.  Going for a trifecta, I tried in HexEdit -- and there, I did find ff da. 

These hex editors, sounding an uncertain trumpet, inspired me to search for that funky string noted above -- ÿØÿà -- in Notepad.  It wasn't there.  I tried five other JPGs in Notepad.  No ÿØÿà in any of them.  I felt lost.

According to the user-friendly guide, a hex search for ff da in xvi32 should have had some luck.  I searched a good JPG in Notepad for ÿØÿà and, sure enough, there it was, right at the start of the file.  I took a look at that same good JPG in xvi32 and it, too, found ff da in a hex search -- and this time it really was ff da.  So were we saying that my bad JPGs did not have the ÿØÿà or the ff da that would be necessary for a manual repair job?  Was that why neither Paint Shop Pro nor IrfanView nor anything else had been able to do anything with these files -- were they all completely fubar?  Were people using these hex editors, searching for ff da, when instead they could have achieved the same thing with IrfanView or PSP?

Diagnosis

I had a folder full of JPGs.  Some of them worked; some didn't.  Having made a backup, I started at the beginning and viewed each of these files in IrfanView.  I had set my IrfanView properties so that it would go immediately to the next JPG upon hitting an arrow key or upon deleting a file.  In other words, I could just hold down the Del key until it came to an error, and then hit Enter and right-arrow to get to the next one.  This would delete each good JPG, which was fine, since I didn't need to be editing this copy of it.

That left me with 159 apparently bad PDFs.  Wow.  More than I expected.  This could take a lot of manual editing.  IrfanView couldn't open them, which probably meant nothing else could either.  Just to be sure, I tried opening several dozen of them in Paint Shop Pro.  No dice.

I wondered if there was a fast way of searching all 159 of these files for ÿØÿà.  Copernic Desktop Search didn't find anything like that.  Well, what if I glued a bunch of them together with the COPY command and then viewed the one huge file in Notepad?  To test this approach, I went into a command window and typed "COPY File1.jpg + File2.jpg Newfile.txt," where File1 and File2 were good JPGs.  Notepad told me that File1.jpg and File2.jpg each had exactly one occurrence of ÿØÿà.  How about Newfile.txt?  Yes, indeed, it had two occurrences of ÿØÿà.  So this approach seemed to work, at least for purposes of preserving occurrences of ÿØÿà in a concatenated file.

To run this little test, I needed a more powerful concatenator, unless I was willing to type out 159 filenames:  COPY wouldn't use wildcards (e.g., COPY *.jpg Combined.txt) -- but I had forgotten that /b would fix that:  COPY /b *.jpg Combined.txt.  (I chose a .txt extension so that (a) Notepad would open it automatically and (b) Combined.jpg would not get mixed up in copying itself into itself with the *.jpg wildcard.)  I ran that and then opened Combined.txt.  It was a large file, of course, so Notepad took a while to open it.  Once it was opened, I did a search for ÿØÿà.  I found hundreds of occurrences -- far more than 159.  Did this mean that my concatenation messed things up, or did it mean that some files had multiple occurrences of ÿØÿà?  I tried concatenating just 10 bad JPGs:  I had named them using sequential numbers, starting with ZZZ_0001.jpg, so I was able to use a wildcard to select ten:  COPY /b ZZZ_003*.jpg Thirties.txt.  The results were confusing.  Eventually I wrote a batch file to open each file individually in Notepad.  The batch file contained lines like these:

start notepad.exe ZZZ_0013.jpg
start notepad.exe ZZZ_0027.jpg

I thought I might crash the system if I opened 159 sessions of Notepad at once, so I broke it into four parts of about 40 lines each.  I ran the first one and did a Ctrl-F and then Ctrl-V in each one to paste ÿØÿà on the search line.  Now I had my answer.  Some of these files (e.g., ZZZ_0074.jpg) contained many iterations of ÿØÿà, while others contained none.  I went through them all.  After a while, it wasn't hard to guess that the ones that seemed to be filled with Chinese characters (and there were quite a few of them) would have no occurrences of ÿØÿà, while some but not all of the files containing more familiar if gibberishy characters (e.g., 1 Òhhh) would have at least one such occurrence.  There might also have been a way of speeding up the process by doing spot checks, since it seemed that files near to one another (probably originating from the same folder) (e.g., ZZZ_0058.jpg and ZZZ_0059.jpg) tended to follow the same pattern of having or not having occurrences of ÿØÿà.  In the end, 27 of the 159 had at least one occurrence of ÿØÿà, and 132 did not.

Fun with Hex Editors

These facts seemed to call for two separate approaches.  For the 132 jpgs that had no occurrences of ÿØÿà, maybe the situation was that the front ends had gotten lopped off, and that's why Paint Shop Pro et al. couldn't make anything of them.  What would happen if I just arbitrarily rammed a header onto each of these files, down to the ÿØÿà point?  The answer to that might give me some clues for the minority of files that did have multiple occurrences of ÿØÿà.

In a survey of good JPGs, I noticed that most began with this:

ÿØÿà JFIF         ÿ

There seemed to be at least one invisible character in there, so the best approach (outside of a hex editor) seemed to be be to copy it from the start of a working JPG in Notepad (i.e., not from this webpage), and save it as header.txt.  Then, starting with ZZZ_0013.jpg, the first of my bad JPGs, I typed this:

COPY /b header.txt + ZZZ_0013.jpg new0013.jpg

and then I tried to open new0013.jpg.  IrfanView gave me an error:  "Decode error!  JPEG datastream contains no image."  Tried it with a couple other files; same result.  A brief search suggested that this "decode error" problem could be just as bad as the original one.  So it appeared that this COPY approach was not the answer.

Back at the user-friendly guide, I confirmed that, in their view, the new header approach required me to locate the hex string FF DA, which I had not been able to do in many files.  Given the uncertainties I had encountered in the several hex editors (above), I wondered if there was a way to output the hex contents of a JPG in text form, so that I could do ordinary searches in Notepad (or whatever) to confirm that there was no FF DA in these files.  The solution was easy enough:  Frhed (but not HexEdit or xvi32) had an option, File > Export as hexdump, that gave me a text file displaying the hex data.  So if I wanted to see a file's hex in a text file, I could use that; and if I wanted to see a file's ASCII in a text file, I could use Notepad.  Xvi32 did have a File > Print option that gave me pretty printed pages (in e.g., PDF or hard copy), displaying both hex and ASCII, so I could have searched for either text or hex values in its PDF output.

But now, this was odd.  When I searched ZZZ_0030.jpg for "ff da" in Frhed, it found nothing; but when I searched Frhed's hex dump text file for "ff da" in Notepad, it found multiple occurrences.  Ah, but the problem seemed to be that Frhed was searching the ASCII side, not the hex.  Frhed's search box told me to consult the online help for guidance, but the program contained no link to any webpage as far as I could tell.  Xvi32's Find option let me search for either text or hex, but as noted above, it often led to 9F instead of FF.  Back in Frhed's hex dump file, I searched for the first occurrence of ff da.  It was on row 0050b8 (which the hex dump displayed as 0050b835, mistakenly running together the row number, it seemed, with the first column of data).  I tried to locate row 50b8 in xvi32, but there was no such row.  I guessed that 0050b8 must refer to a specific location (such as the "ff" in "ff da"), so what I was calling "rows" would actually have different numbers within the same file, according to how many data points were being displayed on a single row onscreen in the hex editor.  Armed with that theory, I did now see an occurrence of FF DA in xvi32, near where location 0050b8 should be; and when I clicked on that occurrence, the status bars in both Frhed and xvi32 displayed indications that I was at "hex address" or "offset" 50CD.

OK, so what could I do with this information?  There seemed to be some confusion here.  Those webpages had said that I was looking for ÿØÿà.  An ASCII code list told me that the ASCII code for ÿ was 152.  (This meant that I could type ÿ by holding down Alt and hitting 152 on the keyboard's numeric keypad.  It turned out that not all ASCII code lists agreed:  some versions of the extended ASCII table (most, it seemed) would say that 152 would give me a tilde (~), but that wasn't the tale told by my keyboard.)  According to my preferred ASCII list, the ÿ had a decimal value of 152 and a hex value of 0x98.  A converter told me that 152 in decimal = 98 in hexadecimal, so that added up.

So, ahh, now I thought maybe I was figuring this out.  Looking at ZZZ_0030.jpg in xvi32, I noticed that selecting FF in the hex area would highlight ÿ in the ASCII area.  As far as xvi32 was concerned, ÿ = FF, not 152.  I tried FF in my hex converter.  It said FF in hex = 255.  Well, and of course it did:  FF was as high as hex would go.  In hex, you don't count from 0 to 9; you count 0 to 9, and after 9 you continue on with a, b,c, d, e, and then f.  What we call 16 in decimal is called F in hex.  After F, you start over back at 0 in hex, just as you start back over at 0 after 9 in base-10 (decimal) counting.  So FF in hex actually meant 0FF:  it was similar to 099 in decimal.  After 099 comes 100 in decimal; after FF comes 100 in hex.  So never mind my keyboard:  these hex programs were interpreting FF as 255, and that was the last possible number in the 256-character extended ASCII set (beginning with zero).  In these ASCII code lists, 255 was not represented by ÿ or any other character.  Apparently it had some special meaning.  To clean up a loose end, I found an occurrence of hex 98 in xvi32, clicked on it, and saw that, sure enough, it was linked with the ASCII tilde.

I looked at a good JPG file in Frhed.  It began, as shown above, with "ÿØÿà JFIF ÿ" -- that is, with this set of hex values:

ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 01 2c 01 2c 00 00 ff

After that ending ff, the contents of the good JPGs seemed to diverge.  That 17th character seemed to be where the image content began.  Would I get a working file if, instead of pasting in that header.txt file (which, being represented in ASCII, was apparently not able to capture all of the hex nuances), I pasted these 17 (or maybe 16) codes at the start of a bad JPG?  Or, wait.  After the hex dump search experience (above), was I sure that this sequence, or part of it, was not already in those files?  The Notepad search for ÿØÿà had produced mixed results, but maybe that wasn't the right way to go about it.

(Note that the second character, Ø, was represented by d8.  There were apparently several different Ø-like characters in use in different languages.  The hex calculator indicated that d8 in hex meant 216 in decimal.  But when I typed Alt-216, I got ╪, not Ø.  That seemed to be an error, according to indications of what I should have gotten in the Latin-1 (ISO-8859-1) character set.  The answer was that I should have been typing Alt-0216, not Alt-216.)

In Frhed, I opened one of those JPGs in which, using Notepad, I had found no occurrences of ÿØÿà.  I did a search (Ctrl-F) for ÿØ, which I could either have pasted into the search box or entered via Alt-152, Alt-0216.  I did find an occurrence of ÿØ.  I noticed that it came shortly after a big section full of zeroes, which made me think that much of that particular file might be gone forever.  I tried again, this time searching for the full ÿØÿà (Alt-152, Alt-0216, Alt-152, Alt-133).  Nothing found.  So Frhed seemed consistent with Notepad in that particular search, at least in this file.

By this point, I was a bit lost.  It did occur to me that I might be able to automate the triage of potentially salvageable JPGs by doing a hex dump, counting the occurrences of 00 (nothing, empty space) as a percentage of the total number of hex values in the file, doing some sampling of partially zeroed but still readable JPGs, and identifying a threshold (10%?  20%) beyond which a JPG would not be worth saving.  But I wasn't there yet, because my JPGs weren't readable at all, and they did have non-zero data.  Most of them, that is; I had found a total of two that were completely empty.

A Bit of Clarity

I went back to the first of my bad JPGs that had no occurrences of ÿØÿà.  My hex editors all seemed to have some way to insert characters or a whole file.  The latter, offered by xvi32 and Frhed, seemed easier, so in xvi32 I went into File > Insert.  I had not yet created the file that I wanted to insert, so now I went into Notepad, pasted my string of 17 hex characters (above), saved it as Header.txt, and proceeded to insert that into my bad JPG in xvi32.  Oops:  that pasted the hex codes (ff d8 ff e0 ...) as text, not as hex.  Xvi32 also gave me an Edit > Insert > Hex string option, so I tried that.  That worked.  I saved the bad JPG and tried opening it in IrfanView.  This gave me a new error:  "Decode error!  Bogus marker length."  Unfortunately, a search led nowhere from that.  Another search produced more, including a FileFormat.info webpage that said this:

The first two bytes of every JPEG stream are the Start Of Image (SOI) marker values FFh D8h. In a JFIF-compliant file there is a JFIF APP0 (Application) marker, immediately following the SOI, which consists of the marker code values FFh E0h and the characters JFIF in the marker data, as described in the next section. In addition to the JFIF marker segment, there may be one or more optional JFIF extension marker segments, followed by the actual image data.

This was helpful.  It seemed that all I really needed, after all, was the FF d8 bytes.  The rest of "ÿØÿà JFIF" was perhaps related to JFIF compliance, but I didn't know if I needed that to produce a file that a program like IrfanView could read.  So I went back into xvi32, went to the 17th byte (i.e., the last one I had just entered) and used the Edit > Delete to Cursor option to delete what I had just added, and then used Edit > Insert String to add back ff d8.  I saved and tried again.  Now we were back to the "Can't read file header" error in Irfanview.  Another FileFormat webpage seemed to say that, to have a JPG file, all you needed was the first four bytes (ff d8 ff e0).  In xvi32, I went into File > New, inserted those four bytes, and saved that file as Test.jpg.  IrfanView gave me an error:  "JPEG datastream contains no image."  I inserted four empty bytes (i.e., eight zeroes) after those four header bytes but still got that error.  I replaced those with bytes 5 through 8 from a good JPG and tried again.  Still the same error.

Another search led to a webpage that seemed to explain something I had noticed in another FileInfo webpage:  it seemed that a JPG file was (or at least could be) defined as one beginning with FF D8 and ending with FF D9.  This page also explained that JFIF was an alternative to EXIF, but I wasn't sure whether I needed either of them.  It seemed that I hadn't really added four data bytes, when I added bytes 5 through 8:  I was just adding the JFIF part of the header.  Assuming I had to have either EXIF or JFIF, in xvi32 I now modified Test.jpg so that it contained what appeared to be the standard JFIF header (ff d8 ff e0 00 10 4a 46), then added four bytes (90 60 1B 88), then added FF D9 at the end, saved, and tried again.  Still "contains no image."  I looked at Test.jpg in Notepad.  Interestingly, it looked like I had made a start on one of those Chinese-looking files.

A more focused search tended to confirm the thought that I was getting in over my head and/or that there just might not be a solution.  People were talking about serious programming, and they were also giving me the impression that, of course, the image data for a JPG would state, or be influenced by, its size, color, compression, and other factors.  This was probably why the user-friendly webpage advised finding a header from a JPG of similar size, if possible, taken by the same camera and edited with the same software.  I didn't have that kind of knowledge about these particular JPGs, so my chances of adding a good header were limited.
The user-friendly webpage hadn't actually been very clear, to me, so I returned to the original advice page that it was trying to present in more user-friendly terms.  That page confirmed that there would typically be "several" occurrences of ff da in a JPG.  This suggested that a file without any such occurrences, searched in a hex editor or dump rather than in Notepad, could be beyond saving.  I had decided I wasn't going to make that determination today, though.  If I couldn't save something now, I was going to zip it up and save it until maybe some better tool came along.

The advice here was to look for the *second* occurrence of ff da, occurring somewhere around 2000 to 4000 bytes into the file.  I hadn't understood that from the other page.  Everything up to that second occurrence was supposedly part of the header; everything after it was image data.  So I would be replacing all of that header section with a similar section from a good JPG.  If this was right, then my attempt to create Test.jpg (above) appaerently needed a second occurrence of ff d8, followed by something resembling image data, in order to work.

At about this point, I discovered that I might have been confusing ff d8 and ff da.  Both appear in the foregoing paragraphs, and I was no longer sure which one I was supposed to be interested in.  A look at a working JPG called Good.jpg indicated that the first two ASCII characters (ÿØ -- which, pronounced yo!, could be a great way to start a JPG) were represented by hex FF D8.  But the original advice page was saying that the boundary between header and image data was marked by FF DA, not FF D8.  So apparently I confused that.  In Good.jpg, FF DA -- that is, ASCII characters ÿÚ, produced (as I now realized) by Alt-0255, Alt-0218 -- first appeared at location (would it be called "offset" or perhaps "byte number"?) 261. This was not nearly as far into the file as the advisor had suggested.  Perhaps it varied with the contents of the file.  But, no, again, this was the first occurrence of FF DA, not the second.  In xvi32, I hit F3 to repeat the search.  But xvi32 again took me to an instance of 9F DA, not FF DA.  I tried the same thing in Fhred, finding once again that it searched for ASCII, not hex; so I searched Good.jpg, in Fhred, for ÿÚ.  It said the offset or address of the first hit was at 609 or, in hex terms, 0x261.  But it could not find any more occurrences.  Was that why Fhred had taken me to the irrelevant 9F DA -- because there was only one FF DA?  (Probably not, I decided later; probably it did that because it was accepting Ÿ as equivalent to ÿ, since I had not specified a case sensitive search.)  HexEdit, too, appeared to be finding only one occurrence of FF DA in Good.jpg.

A Glimpse of Light

It seemed that I would have to try to make this work using the first rather than the second occurrence of FF DA in Good.jpg.  In Fhred, with the cursor blinking on DA at offset 609, I went into Edit > Select Block.  There, I typed x0 as the Start of Selection, and left the End of Selection at x262.  I clicked OK.  This selected everything from the start of the file to DA at offset 609.  Then I realized I was doing this in the wrong file.  But it was OK.  There seemed to be another way to proceed.  In xvi32, I went into File > Export as Hexdump.  I left the same range (x0 to x262) and selected Export to File and Just Hex Digits on a Line.  I saved it as Header.txt.  Then I closed Good.jpg.

Hopefully that gave me a working header.  In Fhred, I went into one of my bad JPGs and searched for FF DA.  It found nothing.  I double-checked, doing the same search in xvi32.  It found an occurrence of FF DA.  Plainly, I had still not quite gotten the hang of using these hex editors, or else maybe Fhred really was buggy, as the writer of the user-friendly webpage believed.

Viewing Bad.jpg in xvi32, the first occurrence of FF DA that xvi32 found was near the end of the file.  That couldn't be an end-of-header marker, could it?  The hex address was 4E6D4, indicating that there was a lot of data before this point.  My guess was that this was one of the later occurrences of FF DA, not the early occurrence that would mark the header.  Could I fix this bad JPG by just attaching Header.txt at the start of the file?  In Fhred, I used Ctrl-Home to go to the start of Bad.jpg.  There, I saw nothing that looked like ÿØ, which presumably would have appeared at the start of any good JPG.  I went into File > Import from Hexdump.  I named the newly created Header.txt as my source and clicked OK.  I got a question:  "Does this data have the same format as the Fhred display?  This data contains only whitespace and hexdigits. (unlike Fhred display)."  I assumed that a header would naturally get a question like this, so I clicked Yes to proceed.  It said, "Unexpected end of data found.  Cannot continue!  Do you want to keep what has been found so far?"  I clicked Yes.  It gave me a blank screen.  As you might have predicted, this did't fix Bad.jpg.

I tried again, this time following the instructions more closely, in case that made a difference.  Specifically, in xvi32, I saved Good.jpg as Header.txt, searched it for FF DA (case sensitive), went to the byte immediately after DA, and then went into Edit > Delete from Cursor.  Finally:  Header.txt really did contain only the information up through FF DA.  I saved it and opened Bad.jpg.  Still in xvi32, and with the cursor located at the start of Bad.jpg, I went into Edit > Insert > Header.txt.  This, I hoped, would prepend a good header to the image body of Bad.jpg and heal it.  I saved Bad.jpg and tried opening it in IrfanView.  Sadly, I got "Bogus marker length."  Paint Shop Pro couldn't open it either.

I tried again with a different Bad.jpg.  This was one of the files that I had identified (above) as having many instances of ÿØÿà, whereas the previous Bad.jpg (i.e., the one I had just been experimenting with) had none.  I searched for the first instance of FF DA, used Ctrl-Shift-PgUp to mark everything up and through FF DA, pressed Del to delete it, and then inserted Header.txt at the start of the file.  I saved it and tried opening it in IrfanView.  Again, "Bogus marker length."  I tried again, this time going to the second instance of FF DA.  "Bogus marker length" once again.

Wrap-Up

I was out of time for this project.  Perhaps some ideas would come to me later, or I would become aware of some new program or technique.  As always, comments and suggestions were welcome.  In the meantime, all I could do at this point was to archive these bad JPGs in a zip file and put them aside.

Batch Merging Many Scattered JPGs into Many Multipage PDFs - Second Try

I had previously looked for a way to combine multiple JPG files into a single PDF.  As explained in more detail in that previous post, the specific problem was that I might have sets of several JPGs in a single folder that should be merged into several different PDFs, and there might be multiple folders at various places that would have such PDF sets.  Hence, if I wanted to automate this project across dozens of folders containing hundreds of JPGs, it seemed that I would need a command-line solution rather than a GUI.  This post updates that previous attempt.

There were commercial programs that seemed to offer the necessary command-line capabilities, such as PDF Merger Deluxe ($30) and ParmisPDF Enterprise Edition ($300).  A search suggested, however, that PDFsam might offer a freeware alternative.

Assembling the List of JPGs to Be Converted

Before investigating PDFsam, I decided to get a more specific sense of what I needed to accomplish.  In a command window (Start > Run > Cmd), I navigated to the root of the drive I wanted to search (using commands like D: and "cd \").  (The root was the folder whose command prompt looked like C:\ or D:\, as distinct from e.g., C:\FolderZ.)  Being at the root folder meant that my command would apply to all subfolders on that drive.  Once I was there, I ran this command:

DIR *.jpg /s /b /a-d > jpgslist.txt

That gave me a list of files (but not directories, /a-d) in all subdirectories (/s), listed in bare (i.e., simple, /b) format, saved in a new file called jpgslist.txt.  (For more information on DIR or any other DOS-like command, type DIR /? at the command prompt, or search for informational webpages.)  If I'd had files with a .jpeg (as distinct from .jpg) extension, I would have added a second line, referring to *.jpeg and using a double-arrow (>>) to tell the program to add these to the existing jpgslist.txt, rather than creating a new jpgslist.txt (which was what the single arrow (>) would do).

Now I wanted to see which folders had more than one JPG.  I would use Microsoft Excel to help me with this.  I could either open jpgslist.txt and copy its contents into an Excel spreadsheet, or import it into Excel.  In Excel, I did a reverse text search to find the last backslash in each text line, so as to distinguish file names from the directories holding them.  I sorted the spreadsheet by folder name and file name.  I set up a column to detect folders containing more than one JPG, and deleted the rows of folders not containing more than one JPG.  I might still want to do another search and conversion for isolated JPGs at some point, but that would be a different project.

Next, I wanted to see if I could eliminate some folders.  For instance, I might not want to PDF and combine JPGs that were awaiting image editing, or important photos whose quality might get degraded in the PDF conversion.  In other words, I decided that this particular project was just for those JPGs that I was going to combine into a single PDF and then delete.  To get a concise list of folders containing multiple JPGs, I went into Data > Filter > Advanced Filter.  (That's Excel 2003.)  I moved the output into another worksheet.  I could then do a VLOOKUP to automatically mark rows to be deleted.  So that gave me the folders to work on.

Now it was time to decide which files to combine, and in what order.  In some cases, I had named files very similarly -- usually with just an ending digit change (e.g., Photo 01, Photo 02 ...).  So I set up a couple of columns to find the filename's length, subtract a half-dozen characters, and decide whether those first X characters were the same as in the preceding row.  If so, and if both were in the same folder, we had a match.  I discovered, at this point, that one or two folders contained large numbers of files.  I decided to combine those manually.  With those out of the way, it seemed that the next step was to decide the names of the resulting multi-image PDFs (e.g., Medieval Churches.pdf), and to put those names on the spreadsheet rows, next to the individual JPGs that would go into them.

At this point, as described in another post, I learned how to use PDFsam to combine several PDFs into one PDF.  So I had a rough idea of the start of my project (i.e., identify the JPGs that I would want to merge into a single output PDF), and I also had a basic sense of the end of my project (i.e., use PDFsam to merge multiple PDFs into that single output PDF).  I was missing the middle part, where I would convert the original JPGs into single-page PDFs and would get them into a form where PDFsam could work on them.

Converting Individual JPGs to Individual PDFs

I had originally assumed that I would start by converting the JPGs to PDFs within the various folders where they were originally located.  So if I had File1.jpg in E:\Folder1, and if I had File2.jpg in E:\Folder2, my conversion would result in two files in each of those folders:  File1.jpg and File1.pdf in Folder1, and File2.jpg and File2.pdf in Folder2.  Then I would use PDFsam to merge the PDFs (i.e., File1.pdf and File2.pdf) from those locations; delete the original JPGs and PDFs; and move Output.pdf to an appropriate location.

I didn't entirely like that scenario.  It seemed like it could make a mess.  As I reviewed another post in which I had worked through similar issues, I decided that a better approach might (1) make a list of original JPG file locations, (2) move those JPGs to a single folder where I could convert them to individual PDFs, (3) merge the individual PDFs into concatenated PDFs, (4) delete the individual JPGs and PDFs, and (5) move the concatenated PDFs to the desired locations.  I decided to try this approach.

One problem with moving files from many folders to one folder was that there might be two files with the same name.  They would coexist peacefully as long as they were in separate folders; but when they converged into one target folder, something would get overwritten or left behind.  It seemed that a first step, then, was to rename the source JPGs, so that each one would have a unique name -- preferably a short name without spaces, potentially making it easier to write commands for them as needed.  In this step, as in others, it would be important to keep a list indicating how various files were changed.  To rename the files where they were, I returned to my spreadsheet and used various formulas to produce a bunch of rename commands of this type:

ren "D:\Folder Name\File Name.jpg" "D:\Folder Name\ZZZ_00001.jpg"

after doing a search to make sure that my computer did not already have any files with names resembling ZZZ_?????.jpg.  The spreadsheet gave me one REN command for each JPG.  I copied those commands into a Notepad file, named it Renamer.bat, and double-clicked to run that batch file.  (A slower and more cautious approach would have been to run it in a command window, perhaps with Pause commands among its renaming lines, so that I could monitor what it was doing.)  A search in a file-finding program like Everything now confirmed that the number of files on my computer with ZZZ_?????.jpg names was equal to the number of files listed in my spreadsheet.  I cut and pasted all those ZZZ_?????.jpg files from Everything to a single folder, D:\Workspace.  (I could also have used the spreadsheet to generate Move commands to run in a batch file for the same purpose.)

Now I had a spreadsheet telling me what the original names of these ZZZ_?????.jpg files were, and I had all those ZZZ files together in D:\Workspace.  My spreadsheet also told me which of them were supposed to be put together into which merged output PDFs.  But they weren't ready to be merged by PDFsam, because they were still JPGs, not PDFs.

To convert the JPGs to PDFs, I could have prepared another batch file, using IrfanView commands to do the conversion, like those that I had previously played with in another project.  But I figured it would be easier to use IrfanView's File > Batch Conversion/Rename.  There, I told IrfanView to Add All of the ZZZ files to its list.  I specified PDF as the Batch Conversion Settings output format, and set its Options > General tab to indicate that Preview was not needed (and adjusted other settings as desired).  I told it to Use Current ("Look In") Directory as the Output Directory for Result Files (adding "Output" as the name of the output subfolder to be created).  Then I clicked Start Batch.

That produced one PDF, in the Output subfolder, for each original JPG.  I hadn't done anything to change their filenames, so ZZZ_00001.jpg had been converted to ZZZ_00001.pdf.  Spot checks indicated that the resulting single-page PDFs were good.  I deleted the original ZZZ*.jpg files, moved the output PDFs up into D:\Workspace, made a backup, and turned to the project of merging those single-page PDFs into multipage PDFs.

Preparing XML Files to Concatenate PDFs

In my spreadsheet, I had already decided which ZZZ files would be merged together, and what the resulting multipage PDFs would be called.  Now -- referring, again, to the other post in which I worked through the process of using PDFsam -- I needed that information to create File Value lines for a set of ConcatList.xml files that PDFsam would then merge into a set of output PDFs.

In other words, I would have a batch file that would run PDFsam, and I would have a data file, in XML format, to specify the single-page PDFs that PDFsam would combine into the multipage output PDF.  I would have a pair of such files (i.e., a batch file and an XML data file) for each resulting multipage PDF.  In my particular project, there were 65 single-page PDFs, and they would be combined into a total of eight multipage PDFs.  So I would have eight pairs of .bat + .xml files, and the eight XML files would contain a total of 65 File Value lines.

To the extent possible, I would want to automate the creation of these batch and data files.  Sorting 65 data lines into eight different XMLs would be tedious and easily confused.  Things would get much worse if I wanted, in some later project, to use these procedures for hundreds or thousands of JPGs or other files.

I began by adding a column to my spreadsheet that contained the exact text of the appropriate File Value line.  Example:  for ZZZ_00001.pdf, the line would read like this:

<file value="D:\Workspace\ZZZ_00001.pdf"/>

To produce that result, if the Excel spreadsheet's cell D2 contained ZZZ_00001.pdf, its cell E2 would contain this formula:

="<file value="&CHAR(34)&"D:\Workspace\"&D2&CHAR(34)&"/>"

(Note the use of CHAR(34) to add quotation marks where they would otherwise be misunderstood.)  Next, I wanted to assign those File Value lines to the appropriate batch files.  A search confirmed that I didn't have any files on my data drives with YYY_????? names, so I decided that my first multipage output PDF would be called YYY_00001.pdf, and that the pair of files used to produce it would be YYY_00001.bat and YYY_00001.xml.  In other words, the File Value line for ZZZ_00001.pdf (above) would have to be one of the File Value lines appearing in YYY_00001.xml.  But the next File Value line in YYY_00001.xml could be a ZZZ file out of sequence (e.g., ZZZ_00027.pdf), if that happened to be the next original file that I wanted to put into YYY_00001.pdf.

Since YYY_00001.pdf was going to be the temporary working name of the multipage PDF that I would ultimately be calling "Short Letters to Mother," I sorted the spreadsheet (making sure to first use Edit > Copy, Edit > Paste Special to convert formulas to values) by the column containing my those ultimate desired filenames, and worked up a column indicating the corresponding YYY filename.  In other words, each cell in that column contained one of eight different labels, from YYY_00001.pdf to YYY_00008.pdf.

With that in place, I was ready to generate some batch commands.  Each batch command would use the ECHO command to send the contents of spreadsheet cells to YYY*.xml files.  My first attempt looked like this:

echo <file value="D:\Workspace\ZZZ_00001.pdf"/> >> YYY_00001.xml

The double greater-than signs (">>") indicated that YYY_00001.xml would be created, if it didn't already exist, and that the File Value line (above) would be added to it.  This first try produced an error, as I feared it might:  ">> was unexpected at this time."  The less-than and greater-than symbols were confusing Windows.  I had to modify the formula in my spreadsheet (or use Ctrl-H) to add carets (^) before them, like this:

^<file value="D:\Workspace\ZZZ_00001.pdf"/^>

That worked.  Now YYY_00001.xml contained that line.  With commands like that for each of the 65 single-page PDFs, my spreadsheet now had cells like these:

echo ^<file value="D:\Workspace\ZZZ_00051.pdf"/^> >> YYY_00004.xml
echo ^<file value="D:\Workspace\ZZZ_00025.pdf"/^> >> YYY_00006.xml

I sorted the rows in my spreadsheet by the appropriate column to make sure the single-page PDFs would get added to their multipage PDFs in the proper order.  (If necessary, I would have added another column containing numbers that I could manipulate to insure the desired order.)  Then I copied all those cells over to Notepad and saved it as a new batch file that I called Sorter.bat.  I ran Sorter.bat and got eight XMLs, as hoped.  Spot checks seemed to indicate that the process had worked correctly.

My eight XML files were not complete for purposes of PDFsam.  Each of them would need lines of code preceding and following the File Value lines.  As described in the other post, those files would begin with

<?xml version="1.0" encoding="UTF-8"?>
<filelist>

and would end with

</filelist>

I saved those two beginning lines into a text file called Header.txt, and I saved that ending line into another text file called Tailer.txt.  Now I needed to combine them with the XML files that Sorter.bat had just created.  For that purpose, it seemed that my spreadsheet could benefit from a separate page dedicated to XML file manipulation.  I added that page, filtered my existing page for unique values in the YYY*.pdf column, and placed the results on that new page. 

I could now see that I was too early in adding .xml extensions to the eight files.  I went back into the spreadsheet and changed it to produce files without extensions (e.g., YYY_00001 instead of YYY_00001.xml), and then I deleted the XML files and re-ran Sorter.bat (as modified) to verify that it was all still working.

With that change, I returned to the spreadsheet's XML Files page.  Next to each of the eight XML filenames, I added columns to produce commands of this type:

copy Header.txt+YYY_00001+Tailer.txt YYY_00001.xml

I put those eight commands into a batch file and ran it.  It worked:  I had eight XML files with everything that PDFsam needed.  There was just one small glitch:  at the end of each resulting XML file, there was a little arrow, pointing to the right.  Searches didn't yield any obvious explanations.  I wasn't sure if it would make a difference; I thought it might just be a symbol representing end-of-file or line feed.  I decided to forge ahead and see what happened.  Except for that little character, I had exactly what I needed for my XML files.

Preparing Batch Files to Use the XMLs

The next step was to create a matching YYY_?????.bat file for each YYY_?????.xml file.  This batch file would run the commands necessary to merge the single-page PDFs listed in the XML file.  I would use the same techniques as in the XML files.  There would be no Tailer.txt file this time; the line that would need to change, in each BAT file, was the very last line.  So my COPY command (above) would just have Header.txt plus the variable line to produce the YYY_?????.bat file.  The variable (last) line of the batch file had to look like this:

%JAVA% %JAVA_OPTS% -jar %CONSOLE_JAR% -l D:\Workspace\YYY_00001.xml -o D:\Workspace\Merged\YYY_00001.pdf concat

In other words, it would have two variables:  the name of the XML file providing the input, and the name of the PDF file containing the output, saved in a Merged subfolder.  It was pretty straightforward, by now, to use the spreadsheet to generate the necessary commands and to run them in another Sorter.bat file (see above).  I just had to remember to delete my previous YYY_????? files (without extension), so that their contents would not get thrown into the mix.  I did wish that PDFsam's -l option were expressed as -L, so that nobody would think it was the number one, but I wasn't yet ready to experiment and find out whether -L would work just as well.  Anyway, to produce a line like the one shown immediately above, my Excel formula looked like this:

="echo %%JAVA%% %%JAVA_OPTS%% -jar %%CONSOLE_JAR%% -l D:\Workspace\"&A2&".xml -o D:\Workspace\Merged\"&A2&".pdf concat >> "&A2 

where cell A2 contained the filename without extension (e.g., YYY_00001).  I had to use double percentage symbols to get a single one to come through.  I put the resulting eight lines into Sorter.bat, and it produced eight YYY_????? files, as before.  I ran another batch file to combine Header.txt plus the YYY_????? files to produce YYY_?????.bat -- again, same as above, but without Tailer.bat and making sure to produce .bat files, not .xml files. 

These steps gave me eight pairs of .bat and .xml files.  The batch files looked good, except for the little arrow at the end (above).  Now, if all went well, the batch files would run, would consult the XML files for the lists of single-page PDFs to merge, and would produce eight YYY_?????.pdf output files in the Merged subfolder.  I would not want to run the batch files manually, if I were producing a large number of merged PDFs, so I wrote a batch file to run the batch files.  The commands in this file looked like this:

@echo off
call YYY_00001.bat
call YYY_00002.bat

and so forth.  I ran this batch file.  It gave me error messages.  I opened a command window and typed just its first action line:  call YYY_00001.bat.  The error was:

FATAL  Error executing ConsoleClient
java.lang.Exception: org.pdfsam.console.exceptions.console.ParseException: PRS001 - Parse error. Invalid name (D:\Workspace\YYY_00001.xml) specified for <l>, must be an existing, readable, file.

Oh.  Dumb mistake.  The XML files weren't in D:\Workspace.  I moved them and tried again on the command line.  Another error:

Error on line 5 of document file:///D:/Workspace/YYY_00001.xml : Content is not allowed in trailing section.

That little arrow was on line 5.  I moved it to line 6 and tried again.  Same error, except now it said the problem was on line 6.  I deleted the little arrow and tried again.  That solved that problem.  Now a different error:  "The system cannot find the path specified."  That was probably because I had not yet created the Merged subfolder.  Apparently PDFsam was not going to create a folder that did not already exist.  I created it and tried again.  Success!  YYY_00001.pdf was created in the Merged folder with the desired single-page PDFs in it.

Now I just had to figure out how to prevent that little arrow from appearing in the XML files. It came at the end of both the XML and the BAT files, and it got there when I used the COPY command to combine the header, the command, and the tailer text files.  The solution was to add the /b switch:

copy /b Header.txt+YYY_00001+Tailer.txt YYY_00001.xml

With that change, I went back through the process of creating the XML files.  Then I tried running YYY_00001.bat again.  I got an error:  "Cannot overwrite output file (overwrite is false)."  Oops.  I had forgotten to get rid of the previously produced YYY_00001.pdf in the Merged folder.  This time I was successful without having to manually remove the little arrow -- it wasn't there anymore in the new YYY_00001.xml.  I ran the batch file that called the eight YYY_?????.bat files.  It ran and produced eight multipage PDFs.  Those eight contained a total of 65 pages.  I combined them all in Acrobat, just to take a quick look.  They were all good.

Putting the Multipage PDFs Back Where They Belong

Now I needed to decide where to put the multipage PDFs.  In this case, the individual PDFs that went into each of the multipage PDFs had all come from the same folder.  That is, I did not have Folder1\PDF1 plus Folder73\PDF2 going into BigPDF-A.  So I could use the spreadsheet to determine semi-automatically the path and filename for a rename command.  I wound up with Rename commands like this:

ren YYY_00004.pdf "Medieval Craft Workers.pdf"

followed by Move commands like this:

move /y "Medieval Craft Workers.pdf" "E:\IMAGES\Medieval Craftsmanship\"

I verified that the multipage PDFs had returned to the folders whence the single-page PDFs had originated.  This project was finished.