In 2004, I bought a Palm VIIx personal digital assistant (PDA). A year later, I replaced that with a Palm Zire 21. The battery on the Zire died about 3.5 years later. I thought about replacing it, but these weren't cheap batteries. It also looked like it would be a hassle: the battery was built into the unit and was not designed for easy replacement. Besides, the screen on the Zire was tiny and was not backlit, and there had been quite a few times when it was hard to read. There were cellphones that would do PDA functions, and their prices were dropping. So I decided to get by without the PDA for a while. Instead, I used the Palm Desktop software as my personal information manager (PIM).
For some reason, a few months later, the Palm Desktop 4.1.4 software that I had been using started to exhibit some funky behavior. For example, the screen would go blank sometimes when I would look at the calendar. The first time this happened, I thought my data was gone. I restored a backup and upgraded to Palm 6.2. Unfortunately, the same thing was happening there. After fiddling around for a while (rebooting, perhaps - I'm not sure), I found that the data was still there; the problem was just in the Desktop software.
From what I could see of hardware developments in PDAs and cellphones, I felt I probably wouldn't be going back to a Palm product. At any rate, this situation needed to be fixed, and I wasn't finding attractive solutions within Palm's hardware and software offerings. So I began to look around for software to replace the Palm Desktop PIM software.
I had been transitioning, for the past two years, from Windows XP to Ubuntu Linux. We were now up to 64-bit Ubuntu 9.04 (Jaunty Jackalope). Ubuntu had come a long ways. To help the transition along, I had decided to use virtualization, so that I could run Windows programs in a Windows virtual machine (VM) on Ubuntu. This way, I could do things in Ubuntu whenever that was convenient; but if I needed access to Windows, I would just go into the virtual machine and work there. I did have the system set up as a dual-boot (WinXP and Ubuntu), but this VM approach did not require me to reboot. In effect, I was running Windows and Ubuntu at the same time.
The virtualization tool I had decided on was VMware Workstation. VMware and I had now graduated to Workstation 6.5.2. This software had cost me $189. It was possible to use a combination of VMware Converter and VMware Player to create and run VMs for free, but I had found some limitations in that approach, so I had decided to go ahead and buy Workstation. (More recently, in another post that I am now writing, that will probably be posted in August 2009, I also note some flaky behavior in Player.) There were also some free virtualization alternatives, and in fact I had recently been experimenting with VirtualBox; but at this point I was still finding that VMware Workstation was my best solution.
My previous efforts to connect the Palm Zire with the Palm Desktop software running in a WinXP VM inside VMware Workstation had encountered repeated problems. There did appear to be ways to connect Palm PDAs to Ubuntu PDA software, but I had not gotten around to trying that before the PDA died. So for purposes of backups and syncing with desktop software, I had been using only the Windows-compatible Palm software to sync with and back up the PDA's data.
At this point, then, with the Zire gone and the Palm Desktop software no longer behaving reliably, it seemed to be time to seach for a non-Palm alternative, starting with the software and transitioning later, as needed, to some new piece of PDA or cellphone hardware. Like others, I was about to find out that this was an impossibly convoluted and dissatisfying quest.
One possibility would have been to revert to Outlook, which I had been using before I bought my first Palm, but I wanted to continue to transition away from dependence on Windows when possible. Outlook 2003 was, to me, big and somewhat unreliable itself, and I didn't want to have to keep buying new versions of Microsoft Office to fix those problems. I also preferred, if possible, a more open-source solution, such that it would ideally be easy to transition from one PIM to another if better alternatives emerged down the road. [Note: until much later in the game, I assumed I was using Outlook 2003, and some of the following comments reflect that. I was, in fact, using Outlook 2007. See below.]
When I looked into the subject the previous year, I became aware of the Thunderbird and Evolution e-mail programs in Ubuntu. Now, Thunderbird was accompanied by the Lightning calendar program, or you could use the standalone Sunbird calendar. Google Calendar and Yahoo! Calendar offered cloud computing alternatives, each with their own advantages, that I had not considered previously. I was not ready at this point to move my contact list and other personal PIM data to someone else's server, however - what happens to my schedule when I don't have an Internet connection? - and in any event these various offerings did not yet seem to provide all of the functions that I had been using, to this point, in Outlook and Palm Desktop.
Evolution was the exception. It came included with Ubuntu 9.04 and was now being described as a complete replacement for Outlook. As I now researched the matter, I discovered that another PIM program, Zimbra Desktop (acquired by Yahoo!), was likewise presenting itself as a free and open-source Outlook replacement. (Zimbra had been acquired by Yahoo, and the Yahoo Calendar was now supposedly running on a Zimbra backend.) It certainly appeared to have a good and growing set of features. For my purposes, Zimbra had the great advantage of being available in both Windows and Ubuntu versions. I hoped, in other words, that I could do the transition from Palm and Outlook to Zimbra in Windows, and then access that same Zimbra PIM database from Zimbra in Ubuntu.
With that general game plan in mind, the ultimate question was, how do I get my PIM data into Zimbra? I had e-mail set up in Outlook, and calendar, to-do list, and contact list in Palm Desktop. According to Zimbra's End User Guide, I would be able to set up the same kinds of e-mail accounts (e.g., Hotmail, IMAP, Gmail) in Zimbra as I had set up in Outlook. The Guide did not seem to have specific information on how to import Outlook e-mail folders. A potentially outdated wiki-type page seemed to say that there was an Outlook PST import wizard. Evidently I would see a menu option along those lines once I installed the Zimbra program. According to that page, I would have to have Outlook 2003 installed on the machine (or else install a trial version of Outlook 2003). This seemed to imply that this wizard would only run in Windows. My version of Outlook was 2003, so I was in luck there. But it did seem, from these sources, that my easiest data migration path would be through Outlook 2003.
In that case, the next question was, how can I get my data out of the Palm Desktop software and into Outlook? After some digging around, I became aware of several Chapura products, of which PocketMirror Standard initially looked most likely to provide what I needed. It required Outlook 2007, which I was not then using. Also, as I read on, I came to understand that it was designed to sync a Palm PDA with Outlook, and would not work with Palm Desktop. Poking around further in Chapura's website, I came across PocketCopy, which offered to transfer all of my Palm data to Outlook for $25. Although Chapura offered trial downloads of some of their other products, they apparently realized that many people would use PocketCopy just one time. It was mildly irritating that they would try to collect $25 for what might be a two-minute conversion. I figured that I could probably find an alternate solution pretty quickly without spending the money, since that usually turned out to be the case. Possibly what I should have done, at this point, was to Google PocketCopy and see what alternatives came up.
As a first possible solution, somebody said that, if I'd still had my Palm PDA, I might have been able to sync it to Yahoo! (or at least the calendar part) and then get it into Outlook from there. Working from within Palm Desktop, there was no File > Save As option, and my only File > Export option would save the contents of individual data tables to Palm's proprietary format only. So while I was looking at that possibility, I went ahead and exported my Date (i.e., Calendar) items from Palm Desktop to a file called Palm Calendar.dba (.dba being the only file extension they would let me use); I exported my addresses to Palm Address.aba; and I exported my to-do list to Palm To-Do.tda. As an alternative, Palm Desktop naturally saved the data to files in its program directory called address.dat, datebook.dat, and todo.dat. (Of course, there would have been other exports and .dat files if I had been using the Expense, Memo, Notepad, Photo, or Voice Memo folders.)
Now, what could I do with these .dba, .aba, .tda, or .dat files? I toyed with the idea of finding an old copy of XyWrite to edit them since, as I dimly recalled, XyWrite was a pure ASCII editor and, as such, would not freak out when it encountered strange characters: I could perhaps do search-and-replace operations to pare the files down to the desired text data. Fortunately, that exercise in antiquity proved unnecessary. An article that was almost a year old at this point suggested that Yahoo Calendar would offer a solution in this case too. Following the recommended steps, I went into my Yahoo! Calendar and went into the Options > Import/Export choice. This gave me a page with options to import and export between Yahoo! Calendar and either Palm Desktop or Outlook. Of course, the only one of the Palm files that this page was willing to convert was the .dba (calendar) file. There were separate tabs, beside the Calendar tab there in Yahoo!, for Mail, Contacts, and Notepad, and I hoped those would handle my import/export needs for those other kinds of data.
I tried going ahead with the calendar conversion, but I got an error message: "The file was not imported or was the wrong format, please check your file." I guessed this was because my upgrade from Palm Desktop 4.1.4 to 6.2 was counterproductive and/or my Palm data file was corrupted. I had tried running Palm Desktop option for Tools > Options > Tools > Check and repair any corrupted records, but the software had frozen. So that was a bad sign, and I was trying to pretend it didn't happen, mostly because I wanted to see if I could make this conversion happen without getting diverted into yet another problem-within-a-problem.
Yahoo! Calendar did offer another possibility. Instead of selection Options, I tried Sync. Here, it seemed that installing their Autosync service would enable me to convert, not only the calendar data, but also the address book and the notepad (which, in my case, was empty) - and that the conversion options included not only Palm and Outlook but also Outlook Express. I clicked the Install button and got the information that I would have to uninstall synchronization software other than Palm HotSync. I didn't have any such software installed, so that was not a problem. They also advised making a backup, which I had done. When I clicked on the button to accept their End User License Agreement, it downloaded the Autosync program (autosync_1049.exe) to my computer. I installed that. When installation was complete, Autosync gave me only the option to synchronize contacts between Yahoo! and Outlook Express (where I had no contacts). I unchecked that option and clicked Next. I opted out of its automatic sync feature and it said, "Well done, your settings have been saved. I left a checkmark in the box that said, "Start first sync" and clicked Finish. It gave me a little notice: "Sync error. There is no section enabled for sync." And that was the end of that.
I decided to try the Contacts tab, there in Yahoo!. It said that their "Import contacts from another application" option included Palm Desktop. But it sure didn't look that way. The drop-down box provided in "Step 1: Select Program or File: Import contacts from the following program or file" listed only Outlook (.csv), Outlook Express (.csv), Netscape (.LDIF file), vCard (.vcf file), and Text File (comma separated values). I wondered if maybe I could get the vCard option to work. I went into Palm Desktop > Tools > Options > General and observed the statement, "Palm Desktop by ACCESS is not the default" handler of "vCard/vCal file associations." I clicked on the button that said "Make Default." I wasn't sure if this would make any difference, but I went to File > Export and saved the calendar data as a different .dba file. Then, back in Yahoo!, I tried the vCard option. It didn't work, not with the .dba file I had saved originally nor with this more recent one.
I looked into a few other possibilities. First, somebody suggested using TealScan, which to my eye did not look relevant. I found some knowledgebase pages at the Palm website. One of these drew my attention to something that I had overlooked. With the Address book selected in Palm Desktop, there was a File > Export vCard option. I tried that, saving the output as a file named Palm Address.vcf, and then went over to Yahoo! and tried that "Import contacts from another application" option again. I selected the vCard file export option and, voila! it imported one contact. Apparently I was supposed to hit Ctrl-A to select all of my contacts before doing this. I repeated the steps, with that change, and this time they all came over. I wondered whether the distinct fields had been preserved, so I tried Yahoo's Printable Address Book option. Well, it really wasn't anything to write home about, but at least the data was there. It wasn't entirely clear whether it had preserved the separate field names.
Now it occurred to me that I could probably just import those contacts directly into Outlook, without using Yahoo! at all. So I tried that. In Outlook 2003, I went to File > Import and Export > Import a VCARD file (*.vcf). It said "Initializing Places" for a minute. Then it asked me for the file. It, too, imported only one address; but unlike Yahoo, it preserved the fields clearly. I deleted the .vcf file and tried again. Still only one record imported. So for me, for now, the direct route from Palm's address book to Outlook did not work, but the route via Yahoo did. Or at least I hoped it did. I hadn't tested the final leg of it yet. Outlook did not seem to have an option to import from Yahoo, but Yahoo did have an option to export to Outlook. So, OK, back in Yahoo!, I went to Contacts > Import/Export and clicked on the Export Now button next to Microsoft Outlook. I got a pop-up giving me the option to open or save a "Microsoft Office Excel Comma Separated Values File" named yahoo_ab.csv. I saved it. Then, over in Outlook, I went to File > Import and Export > Import from another program or file > Comma separated values (Windows). I indicated that I wanted to import yahoo_ab.csv (where .csv is short, of course, for comma-separated values). This caused Outlook to crash, reminding me of a frequently encountered reason to get away from Outlook. As I looked closer at the Yahoo! export webpage, I saw this: "One .VCF file per contact." Apparently the .vcf output from Palm Desktop, containing many addresses, was beyond Outlook's import capabilities.
A short time earlier, it had occurred to me that I had the vCard .vcf file and that Zimbra would probably import it directly, without even bothering with Outlook. I downloaded and installed the Windows version of Zimbra. When installation finished, I took the option to start the program. It went right into the process of adding a new account. I designated my Yahoo! mail account as the first one to be added. There were options to synchronize all calendars and contacts. I selected those options. I got a notice, "In order to use calendar sync, you must upgrade your Yahoo! Calendar Classic to the All-New Yahoo! Calendar." I clicked on that provided link and went ahead and installed that. It looked nice, even if it was beta software.
Anyway, I went back to Zimbra and clicked on the Validate & Save button again. It said the account was added. It also said, "Note that the first synchronization may take up to several hours to complete depending on how much data is in the account. However, you may start using this account immediately" and "Sync has not started." For a minute, I thought I should have set up the All-New Yahoo! Calendar before starting Zimbra, in case it would import the .vcf file directly without this delay. But Zimbra gave me an option to Launch Desktop, so I went ahead with that. I went to its Options > Import/Export, and there I saw that it would only import in .csv format. In other words, there did not seem to be a direct route from Palm to Zimbra for contact data.
Zimbra said, there in its Import/Export page, that it would import calendar data in .ics format. This was not an option I had seen thus far, but given the corporate ties between Zimbra and Yahoo, I had to assume that this would be something that I could get through Yahoo as well. So, again, the question was, how can I get my Palm calendar data into Yahoo? Reflecting on the solution for the contact data (above), I went back to Palm Desktop and noticed, this time, the greyed-out option for File > Export vCal. This was not entirely a solution, as the Yahoo! Calendar (like Zimbra) required .ics format for imports, and I had no idea how to create an .ics data file. But the vCal export option, there in Palm Desktop, might at least get me to first base. The thing was, as I say, it was greyed out. I found I could select one calendar item and then it wouldn't be grayed out anymore. Likewise, the Edit > Forward as vCal option was not greyed out when a calendar item was selected. But the software did not allow selecting more than one calendar item at a time. So the vCal approach looked like a dead end.
At this point, I found a post by cygnusX1 that pointed me toward a program called CompanionLink. Apparently this program came in different flavors, each of which cost $39.95 after a 14-day trial, and it seemed the flavor I needed was called CompanionLink for Google. The webpage for that particular variety of CompanionLink said this:
CompanionLink for Google will sync Palm Desktop contacts, calendar, tasks, and memos with your Google account. Once your data is in your Google account, Google's free over-the-air sync service (called Google Sync) will sync this data with your phone.
My sense at this point was that Google, like Yahoo!, would export data in Outlook-compatible form. So I downloaded CompanionLink for Google. This trial download, they said, would be fully functional, so I didn't have to worry about it being restricted to a certain number of data entries. The download page might not have made sense without the foregoing information; but now that I understood the concept, I was pleased to see that they actually offered versions of CompanionLink that would sync Palm Desktop, not only with Blackberry, Google, Palm, and Windows Mobile devices, but also with a Google account, Outlook, and "Other." I went with the Outlook option. When it was downloaded, I installed it. The various options just described didn't seem to matter, because even though I had specified Outlook, the installer told me I was installing CompanionLink for Google.
Anyway, I installed it. It gave me desktop (but not Start Menu) icons for two programs: Setup and Synchronize. I ran Setup first. When it asked what I wanted to synchronize, I noticed that it gave me multiple options, so I wasn't sure why they had asked me (above) to indicate precisely which device or program I would be using it with. I also noticed that the Palm Desktop option that they specified was "Palm Desktop by ACCESS." Palm Desktop 4.1.4 had not referred to itself that way, so perhaps upgrading to 6.2 was an essential step for purposes of being able to use CompanionLink. As we were going through setup, in the Transfer Method dialog, on the Transfer Method tab, I indicated that I wanted to "Add Palm Desktop by ACCESS info to Google." I made sure I was happy with the value on the Date Range tab, and with other tabs, and then I told it to go ahead. But then - what's this? - I got "Palm Desktop 6.2 files not found." I followed their instructions there, but they didn't help.
I referred to CompanionLink's Setup and Installation Guides. They pointed me toward a tutorial video. As I watched the tutorial, I noticed that their example used a different version of Palm software. So apparently I was seeing "Palm Desktop by ACCESS" listed in my version of the program because CompanionLink's installer had hunted around and figured out that this was the version I had installed. But the tutorial didn't explain the solution to my problem. It occurred to me that maybe the problem was that I had customized my Palm Desktop installation by moving my data files to a separate drive from the program files, so that problems, crashes, and restorations on the program drive would not affect my data. I played around with fixing this and finally decided to uninstall Palm Desktop 6.2 and reinstall my old Palm Desktop 4.1.4. The reason was that the ACCESS version seemed to use Microsoft Access data tables, and this represented a variation that I didn't know how to play with. With 4.1.4 reinstalled, I went back into the setup process. This time, I noticed that it said that it would be syncing the Palm Desktop To-Do List into the Calendar, and the Memo Pad (which I didn't use) into the Contacts list. I decided this was OK. This time, I did not have any problems. So apparently that ACCESS version's use of Access data tables was, not surprisingly, screwing up the conversion operation. In other words, CompanionLink had been programmed to work with regular old Palm Desktop software, not with this weird 6.2 ACCESS version.
So CompanionLink did its thing, and after five or ten minutes it said, "Completed Synchronization." Now what did I have? I had - whoa - a fully populated Google calendar. Tasks appeared at the top of each day, in the calendar's timeless zone, in this format: "Task: This is a task." There in Google Calendar, I went to Settings > Calendars > Export Calendars. It offered to download a file called [my Gmail account].ical.zip. At first, I didn't think that would be useful, because I knew Zimbra was looking for an .ics file, not an .ical file. But I went ahead anyway, and when I unzipped it, it turned out that I did have an .ics file after all.
Now, what to do with it? I went back into Zimbra > Account Setup. I saw that it had now completed its synchronization. I went into its Desktop and saw that it had a large number of error reports. I looked at a few of those errors and saw this summary repeated: "Yahoo address book synchronization failed." So apparently the approach of importing all my contacts from a single Yahoo .vcf file was not going to work any better in Zimbra than it had worked in Outlook. I had one successful contact and a couple hundred failures; and although Zimbra's entry page had seemed to say that the last sync was done, I now understood that it was still underway, and it was going to end with one successful contact imported and all the rest failed. There wasn't a Stop button, so apparently I had to let this wild horse run. I said to hell with it and went to bed. When I got up the next day, the list of failures was complete. I deleted them - often a good strategy with failures - and started over. Ah, but no, the list of failures continued. Life was having its way with me! I killed Zimbra - another strategy, I suppose - and after restarting it, I saw that the nightmare was destined to continue. Apparently the tale could not be fully told even in six or seven straight hours of gradual error reporting, one slow, painful failure at a time. I ignored the error reporting (sometimes the only strategy that works) and went to Zimbra's Options > Import/Export > Import > Calendar. There, I imported the .ics file that Google Calendar had produced. Import Succeeded! Alright. Now we were getting somewhere. One thing I didn't like was that I couldn't actually read my Calendar entries in Zimbra Calendar, though. It showed a colored spot for the entry; but to read the text I had to resize the colored spot. Otherwise, it was just a colored blank. Not impressive.
In the Import screen, I saw that Zimbra would require a .csv file, not the .ics file, to import my Contacts. Apparently the .ics file was good only for the Calendar. But I tried importing it into Contacts anyway. I got "Import Started." Zimbra turned my cursor into an hourglass and would not let me do anything else inside the program until the import completed. This continued for five minutes or so, and then I got "Import succeeded." I clicked OK and went to Contacts. But no, I clicked the wrong tab and went to the Contacts Options instead of the actual Contacts folder. I tried again. It asked me if I wanted to save my changes. I said yes. I got "Configuration error: incorrect mailbox class: LocalMailbox." Apparently, while fooling around, I had changed something that I should not have changed. I tried again, and this time said I didn't want to save my changes. Now I was able to go to the Contacts area, but I still had no contacts. So what was it importing, or what import had succeeded? I was afraid it meant that I had imported a duplicate set of Calendar entries, but that didn't appear to be the case either. So that was a mystery.
Trying again to get the Contacts, I went back to Zimbra's Import option. This time, I tried importing the yahoo_ab.csv file that I had exported from Yahoo Calendar (above). I got "Import Started" and then, a moment later, "Warning: Unresponsive script. A script on this page may be busy, or it may have stopped responding." I opted to continue. I got another one. After several repeats, I chose the Stop Script button. A minute later, it said "Import succeeded." I went to look at the Contacts list. I didn't look to see if it had all of my contacts. It did have a lot of them. But they were all duplicated, and information was missing. I opened the yahoo_ab.csv file in Excel and saw that the records were duplicated there too. As I recalled, they weren't duplicated in my contacts list within Yahoo. I went back to Yahoo Calendar to check. It, like Google Calendar, was a bit irritating at this point, because neither of them had a link to my contacts list from the calendar webpage I was looking at, and I couldn't figure out how to get there from here. It turned out that I had to get there via the Mail link - in Yahoo, at least - so I did that. Now I saw that, yes, everything was duplicated here too. I wasn't sure whether this was their fault or mine - probably mine - but it did seem that neither Yahoo! nor Zimbra had checked for duplicates when I was importing contacts into them. In Yahoo > Contacts > Options, I found a Clean Up Duplicates (beta) option. I tried that, and selected "Merge EXACT Matches Automatically" option. I didn't find a Clean Up Duplicates option in Zimbra; and once again, when I viewed Zimbra's Options and then tried to leave the options area, it asked me if I wanted to save my changes even though I had made none. I decided to delete the contacts in Zimbra and start over, but there didn't seem to be a way to select and delete all contacts: Ctrl-click would work to select multiple entries (if I wished to manually select hundreds of them), but Shift-click didn't work. Or I could just hit the Delete key several hundred times. Holding it down worked only as long as I held it down; it didn't keep on deleting after I let up. Ctrl-A selected the whole program screen, but not the entire list of contacts. When I clicked the All button to see all of my contacts, I saw hundreds of blanks - or, more precisely, 100 blanks. It wasn't really showing "All"; it was just showing 100 at a time. Finally, regarding the missing data, I found, for example, a Contact record that showed a phone number and e-mail address for one of my associates in the yahoo_ab.csv file, but nothing other than the name in the duplicative Zimbra contact entries.
Altogether, I was not finding Zimbra too impressive at this point. I decided to regroup. I went to Windows Control Panel > Add/Remove Programs and uninstalled Yahoo Zimbra, including its entire dataset. (I was surprised to see that Add or Remove Programs reported Zimbra as occupying 242MB of disk space.) I went back to the address book in Yahoo Calendar, now that it had finished deleting duplicate contacts. It now reported having a bit more than half as many records as appeared in the yahoo_ab.csv file in Excel. So the deletion of duplicates in Yahoo appeared to have worked, more or less. I exported a new yahoo_ab.csv file. I tried editing that file, searching for occurrences of a semicolon (;), since these appeared with various other characters (e.g., & and #) in place of some text characters. When saving, I got a warning that some of my changes might not be compatible with the .csv format. I realized that the special codes might be there for a reason, so I deleted this one and exported another fresh new yahoo_ab.csv file from Yahoo Calendar. This time, I did not try editing it.
So, to regroup: I now had a yahoo_ab.csv file containing my contacts and a Calendar.ics file containing my calendar and to-do list entries exported from Palm Desktop. I decided to try importing these into Outlook 2003, so that at least the outlook.pst file would contain everything - tasks, contacts, calendar, and e-mails. This was somewhat ironic reasoning, since the goal was to get away from Outlook, not to make it even more of a fortress in my PIM world. I was just hoping, based on assorted posts I had noticed along the way, that it might be easier to import the whole .pst in one swell foop, rather than wrestle with all these parts of it separately. In Outlook, I went to File > Import and Export > Import an iCalendar (.ics) or vCalendar file (.vcs). The import proceeded and finished almost instantly. The resulting calendar looked good. It looked more or less like what I had seen in Google Calendar. Then, to import my contacts from the yahoo_ab.csv file, I went back to Outlook's File > Import and Export > Import from another program or file > Comma Separated Values (Windows). Outlook crashed again, as it had done when I had made a similar move while trying to import .vcf entries (above). Oh, Outlook, you sweet thing. I wondered if I should have closed Outlook and reopened it after doing the one import, before starting the other, so as to give it a chance to organize itself - because now, when I restarted it, it was really confused. It opened two separate sessions of itself, one focused on Contacts and one on Calendar, and then it crashed again. On the following restart, it did it again, but this time I killed one of those sessions immediately. In the other one, the Calendar still looked good, but I still had no Contacts. I tried again, this time selecting Comma Separated Values (DOS). Outlook crashed again. I tried once more. This time, I opened the yahoo_ab.csv file in Excel, did the cleanups mentioned above (e.g., searching for occurrences of semicolons and replacing the #;&! codes with the desired characters), and saved the .csv file as an Excel .xls file; and then I imported that Excel file (yahoo_ab.xls) into Outlook instead of trying to import the Comma Separated Values (yahoo_ab.csv) file. Or at least I tried to. Outlook was not interested in restarting right now, unfortunately, and would I please just check back later? I rebooted the system - it had been on overnight, and of course that would tend to consternate WinXP - and tried again. Outlook was not happy - it failed to start through safe mode and data repair cycles - but eventually I did coax it back to life; and when I then tried to import the yahoo_ab.xls file into Contacts, I got this:
Translation Error
An error has occurred in the Microsoft Excel 97-2003 translator while getting
the contents of a file system.
The Microsoft Excel 97-2003 file "D:\Current\yahoo_ab.xls" has no named
ranges. Use Microsoft Excel 97-203 to name the range of data you want to
import.
I responded to this message, first, by editing a copy of the .xls file. I called that copy yab1.xls. In it, I used a COUNTA() function at the top of each column to determine which columns had nothing in them. I deleted those, so as to not waste effort on them and to make the file smaller and easier to work with. Next, I adjusted line heights and column widths for the same purpose - to make it easier to view the data. Acting on a tip from slurpee55, I checked the phone number and zip code columns and, sure enough, Excel had removed the leading zeroes (e.g., 02072 was now 2072). Instead of his suggested procedure involving Notepad, I did this: (1) format the whole spreadsheet as text, (2) highlight the top row of the spreadsheet, (3) select Data > Filter > AutoFilter, (4) filter the Zip Code column (or whatever) for those that don't fit the mold (e.g., 2072), and (5) change them manually. While I was at it, I filtered entries for errors (e.g., those that had city names in the wrong columns). I wasn't sure Outlook was going to swallow all these edits, so after making some, I saved yab1.xls, saved it again as yab2.xls, and tried importing yab1.xls. This gave me the "no named ranges" error again - I forgot to fix that part - so I went back to yab1.xls and named ranges. This required simply highlighting the whole spreadsheet and then selecting Insert > Name > Create > Top Row. Yahoo! or Palm Desktop had already used that top row to assign names to each column; this command made clear that the labels in those top columns would be treated as the names of the columns (i.e., as the names of the range of cells under each column heading). I saved yab1.xls and also saved it again as yab2.xls, and made sure I was out of yab1.xls so that Outlook would have access to it. I was now able, in Outlook, to progress to the place where I had a dialog box, Import a File, that allowed me to check which fields I wanted to import from yab1.xls. I checked one, and this took me to another dialog, Map Custom Fields. For instance, Outlook did not automatically recognize the field called "Business_City." But Outlook did have a field called "Business City" (without the underscore), underneath its Business Address heading. In the Map Custom Fields dialog, I dragged F1 (representing column F in yab1.xls) from the left side to the Business City item on the right side. But I wanted this process to be automatic. I went back to yab1.xls and put quotation marks around each column heading that contained spaces. For example, the Business City heading became "Business City" (with quotation marks). (This could be automated by adding another temporary row and putting a formula like this in it: =char(34)&a1&char(34), and then highlighting and using Edit > Copy and Edit > Paste Special to convert the formulas to values.) I tried importing again. This time, Outlook didn't take me automatically to the Map Custom Fields dialog as soon as I clicked Business City. When it did take me there, after I had clicked several fields, I clicked Cancel and resumed checking the rest of the items shown in the Import a File dialog. Then I clicked Finish. This yielded an interesting result: Outlook crashed. Again. It was probably my fault, but whatever. I tried the same exact approach again and got the same exact result. So, OK, it was my fault. This time around, I clicked only the first three items under Import a File. These did not take me automatically to the Map Custom Fields dialog. Then I clicked Finish. Outlook still crashed. So it seemed that the problem was not that I was trying to import nonconforming data.
I decided to take a different approach. In Ubuntu, I went to Applications > Internet > Evolution Mail. I went through the Evolution Setup Assistant, using my Hotmail account as my default e-mail address. Using pretty much the same settings as shown for a Gmail account, in the Receiving Email dialog I indicated a server type of POP, server = pop.hotmail.com, username = my e-mail account, SSL encryption, and Remember password. For Sending Email, I wasn't sure what to enter, so I tried the Server Configuration information provided by Host45.com for my ISP. The only other thing I changed on Sending Email was to tell it to Remember password. Evolution then opened up. At the bottom, I saw this: "Error while Fetching Mail." When I clicked on it, it said "Name or service not known." What I really wished, at this point, was that Evolution had offered to import my settings from Outlook somehow, because it now occurred to me that Outlook probably already contained all this information. I went into Outlook's Tools > Account Settings and double-clicked on the Hotmail entry. It didn't actually tell me the Hotmail settings that worked there; somehow it just knew. According to Paresh, what I needed as my Hotmail receiving server was pop3.live.com and the sending server was smtp.live.com with TLS encryption. I saved that and got a dialog asking for my Hotmail password. I entered that and got another dialog that said this:
Unlock Keyring
Enter password for default keyring to unlock
The application 'evolution' (/usr/bin/evolution) wants access to the default
keyring, but it is locked.
I didn't know what this was about. I tried my Ubuntu login password. Now I got another request for the Hotmail password, this time with this message: "Unable to connect to POP server pop3.live.com. Please enter the POP password for [my Hotmail account]." I tried the Hotmail password again. It didn't work. The error message this time was, "Unable to connect to POP server pop3.live.com. Error sending username: -ERR unrecognized command." I tried a couple more times; I think I may have changed something minor; and then, suddenly, it worked, and I was looking at my Hotmail e-mail account in Evolution. It automatically removed messages from Hotmail when it downloaded them to Evolution, and I wasn't ready for that, so I went back into Preferences and changed the option to leave messages on the server, and also unchecked the box for "Checking for New Mail." If I wanted to check, I'd do it manually, by clicking Send/Receive. To get them back in my Hotmail Inbox for the moment, I went into Hotmail's Deleted folder and marked and moved them to the Inbox.
Next, I tried sending a message from myself (in Evolution) to myself (in a non-Hotmail account viewed on Outlook). I got a message at the bottom of the Evolution screen: "Sending message (100% complete)." But evidently it wasn't 100% complete, because the circular time-lapsed hourglass equivalent kept going slowly around. I tried the advice to set the sending server to smtp.live.com:587, but when I then clicked on Send/Receive, I got a dialog stating, "Unable to authenticate to SMTP server. Bad authentication response from server." So I changed the sending server back to smtp.live.com. While I was there, I typed my complete username: not just the username, that is, but also the @hotmail.com part. The Send & Receive Mail dialog again indicated that my receive was complete, but the send was stalled. I went back to the Send options again, and this time I followed some Microsoft advice and changed the encryption to SSL rather than TLS, but that wasn't the answer either.
It seemed likely that I could get past that problem somehow: that a solution would appear, or that I could use web-based Hotmail to send messages, or would continue to run Outlook for some purposes. If I could get everything else working on Evolution, perhaps I could live, for a while, with this problem of sending e-mail. On that basis, I wanted to know if I could get my Outlook data into Evolution. There seemed to be several approaches to that problem. An apparently time-tested kludge for using outlook.pst in various Linux e-mail programs (including Thunderbird as well as Evolution), as described by Elvis, was to install Thunderbird for Windows, import into Thunderbird, copy the relevant files as needed to your Ubuntu partition, and then start up Thunderbird on Ubuntu. Once this was done, apparently Evolution would be able to import the stuff from Thunderbird. There was also another approach, recommended by Deepspring for those who do not have Windows on their computers. This approach was comparable to that recommended in the GNOME Documentation Library for migrating Outlook mail folders.
I had played with Thunderbird previously and had decided not to use it. The official Thunderbird FAQ said that Thunderbird saved e-mails in plain text, which meant that HTML formatting and such would disappear. I was also interested in avoiding extra steps if possible. So I decided to start by exploring what seemed like the easiest and most direct routes from Outlook to Evolution. One possibility was to try importing outlook.pst into Evolution for Windows. Another approach was to use Outport to convert the Outlook data to Evolution format, and then copy the resulting .db files into your linux .evolution folders to replace what’s there and reboot. In this approach, the focus was on what was available in Outlook - that is, Outport was not a .pst file converter.
It seemed like the most direct route would be to install Evolution for Windows and see if I could import all of my PDA data into that. I also realized that, if the Windows version was better for sending and receiving e-mail, I might just stop the transition process there and wait for a solution to the e-mail sending problem in the Ubuntu version. I started into the setup process and almost immediately received this message: "evolution-exchange-storage.exe has encountered a problem and needs to close." Wow. The thing was crashing as soon as I looked at it. A quick search suggested that hardly anybody else had experienced this problem. I wondered what would happen if I ignored that message. What happened was, it went away. Evidently what crashed was not the main program. I continued the setup process, using the same settings as I had used in the Ubuntu version of Evolution (above). I tried sending messages between Hotmail and my Yahoo! e-mail account. As in Ubuntu, the message was received without a problem, but Evolution was not able to send the test message to Yahoo! e-mail. As above, I got this message:
Error while Sending message.
Could not connect to smtp.live.com: Unknown error
Changing between SSL and TLS encryption did not solve the problem but, as above, it did change the error message slightly: with TLS, the second line read, "Welcome response error: Input/output error." Moving on to the matter of data importation, I opened the Evolution Import Assistant by going to File > Import. I indicated that I wanted to "Import data and settings from older programs," but the Assistant said it had checked several programs (not including Outlook) and concluded, "No importable settings found." So I tried the "Import a single file" option. Sadly, the Assistant was again uninterested in what I had to offer. Among the preferred file types discussed above, the only ones it would import were .csv and .ics - not .pst.
Well, if I couldn't import a .pst, how about Edit > Synchronization Options? That gave me this error: "The GNOME Pilot tools do not appear to be installed on this system." According to Novell (creator or owner of Evolution), the software in question could also be called the "GNOME Pilot configuration tool." Searching for such terms did not turn up a Windows version of that Linux software. I gathered, in any case, that "synchronization" in this context meant synchronizing Evolution to a handheld device, which was not what I wanted. I tried checking the Help file within Evolution for Windows, but I got an error message stating, "Could not display help for this application."
Next, I tried installing Outport. Clicking on the outport.exe file produced nothing for a minute or so, although the program did appear to start up Outlook. Then a simple dialog box opened up. In that box, I could check what I wanted to export and the format in which I wanted it. Outport seemed to detect all of my folders in Outlook (e.g., Inbox, Outbox, Sent Items). I chose to export all of the files in my Personal Folders, and I left everything in its default format. So, for example, my Outlook contacts were going to be exported in Evolution Contacts format, but my mail messages were going to be exported in Native Format. When I went ahead with this, Outport asked me to designate a server location from a list. I chose the city where I lived. This resulted in an error message: "The RPC server is unavailable." I clicked OK on that. The Exporting dialog box remained open. Nothing seemed to be happening there, but the cursor remained in a pointer/hourglass shape. After a couple of minutes, I clicked Cancel, intending to try a different server location. But though the dialog disappeared, the Export button on Outport remained greyed out, suggesting that something was indeed underway and that maybe I should have just let it go. Yet there was nothing in the output folder I had designated. I clicked on the X in the upper-right corner to kill Outport and try again, but it did not want to die. I used Windows Task Manager (Ctrl-Alt-Del) to kill it. This time, I started Outlook first, and then Outport. In Outport, I didn't designate everything for export, as I had done previously; I checked only those folders that actually had stuff I wanted to keep. Outport took my instructions and ran with them. The Exporting dialog box showed progress almost immediately. In about 15 minutes, it had created subfolders (in the target folder I had created for its output in Windows Explorer) for each of the Outlook subfolders I had asked it to export (e.g., Inbox), and had filled those subfolders with .doc files, one for each message I was trying to export. Or almost each message. When the export was 87% done, it stopped and eventually said "Not Responding." This seemed to be related to a "Server Busy" message that said, "This action cannot be completed because the "E-mail Archive - Microsoft Outlook" program is not responding." I may have caused this: while Outport was still running, I opened up one of the output subfolders to see what was in it. At any rate, the "Switch To" and "Retry" buttons on that "Server Busy" message did not seem to do anything. I finally killed that message, and that killed Outport. I ran it again. Now the whole computer locked up. I did a hard reset and ran Outport again. I got the "Server Busy" message again, but this time it seemed to be because Outlook was not yet up and running. As soon as Outlook got underway, the "Server Busy" message disappeared. This suggested that maybe I had gotten that message previously because of a problem with Outlook, not Outport: maybe Outlook had ceased to function as Outport needed it to function. On a system where Outlook was now sometimes taking five minutes to get itself running, this would not be surprising.
I let Outport run overnight, and in the morning I had what looked like a complete set of e-mail messages in .doc format (e.g., "inbox/message8.doc"), with correspondingly named subfolders that apparently contained the attachments (e.g., "inbox/message8/mypicture.jpg") that accompanied those e-mail messages in outlook.pst. The numbers of .doc files in each subfolder matched the number of e-mail messages in the correspondingly named folder in Outlook (e.g., the inbox folder contained "message1.doc" through "message6.doc" and there were six messages in the Inbox on Outlook). Outport also gave me files called "local/Calendar/calendar.ics" and "local/Contacts/addressbook.db."
Now, what to do with them? I started Evolution. That, or something else happening at the same time, caused Internet Explorer to close immediately. Evolution took about a minute to start up; and before it did, once again I got that error message mentioned above, "evolution-exchange-storage.exe has encountered a problem and needs to close," and then Evolution ran as usual. According to some old instructions, I could import the Outlook e-mails into Evolution by simply going to Evolution's File > Import option. I did that and got the Evolution Import Assistant, as above, and once again that did not have an option to import .doc files. I thought I had already read the Outport documentation, but somehow I had missed the plain statement, in its FAQs, that "Unfortunately, Outport doesn't support moving your email from Outlook into Evolution." The FAQs recommended using the route via Thunderbird instead. So while it did appear that Evolution had given me importable Calendar and Contacts data, the e-mail question remained open. Just in case, I looked to see where Evolution for Windows was saving the few e-mail messages that it already contained. The actual data seemed to be in the C:\Documents and Settings\[username]\.evolution\mail\local\Inbox file. But I did not know how to add .doc file contents to that Inbox file, the Evolution for Windows route seemed closed for purposes of importing e-mails.
Before using the Outport output to fill the Calendar and Contacts in Evolution for Windows, I thought I should try Thunderbird for Windows for the e-mails, just in case its new 3.0 beta 3 version would now take care of the Calendar and Contacts too. I installed it and went to its Tools > Import. I selected "Import Everything" and went through the rest of the steps. It said, "The following items were successfully imported: Account Settings, Address Book, Mail Folders." Not to disagree, but it didn't look like anything had actually happened. I tried again, this time designating only the address book. Now we were closer to the truth of the matter: Thunderbird said, "Addresses successfully imported from Outlook" but also "No address books were found to import." So, OK, apparently it didn't know where my .pst file was (and wasn't interested in asking). I had relocated outlook.pst to another drive for backup purposes. Now I copied it to C:\Documents and Settings\[username]\Local Settings\Application Data\Microsoft\Outlook, which was its default location, and tried again. Still no luck. So now, as I researched the matter, it seemed that other people were having similar problems with this supposedly straightforward path away from Outlook. I killed and restarted Thunderbird and tried again. Same thing. Someone suggested that it was necessary to let Thunderbird be the default e-mail program, so I went into Thunderbird's Tools > Options > General and set that. But importing still didn't work. I wondered if maybe the situation with Thunderbird was the same as with Outport (above) - if, that is, it needed Outlook to be up and running in order to proceed with its import/conversion functions. I started Outlook, which had been shut down, and tried again. But no, that wasn't it: when I tried to import e-mail from Outlook, Thunderbird said, "Unable to find mail to import."
Samuel suggested - you're not going to believe this - that I import the outlook.pst into Outlook Express (OE) and import it from there into Thunderbird (from which I would then proceed on to Evolution). Whew! The OE route, he said, "solves most Outlook import problems." So, OK, I did it. I hadn't used OE on this machine, though (if I recall correctly) it came pre-installed with Windows, so now I went through the OE setup process, including importing settings from Outlook. The OE setup process next asked if I wanted to import e-mails and addresses, so I went with that. Unfortunately, I got this error: "The MAPI Spooler could not be started. Close and then restart all mail-enabled applications." As instructed, I closed Outlook and Thunderbird (both were open) and then restarted them. This, I thought, might be a way of clearing the pipes, but perhaps I had better shut them down before trying again. So I did. I still got the same error, so I restarted OE. This time, it didn't give me that auto-import wizard, so I went to File > Import > Address Book. It was looking for a .wab file, and AvaFind told me that my .wab was at C:\Documents and Settings\[username]\Application Data\Microsoft\Address Book\Ray Woodcock.wab, so I navigated there. OE instantly said, "The import process completed successfully." So, OK, that was a futile gesture: there was nothing there, because I hadn't successfully imported my Contacts into Outlook. Trying again with File > Import > Messages, I got the same error message as above when I tried to import from Outlook.
To solve this error message, Tom Koch provided a series of OE repair steps, beginning with rebooting the computer, closing all programs other than OE and Internet Explorer, and cleaning out the Temporary Internet Files folder. After that, I noticed that he said, "Many bizarre behaviors in Outlook Express are caused by small errors in the file Folders.dbx." I wondered if any of my Outlook (not OE) folders were corrupt. It seemed to me that there was some kind of utility that would check and repair them. There was an Inbox Repair Tool, but this seemed to be called for if you couldn't open your e-mail folders, which I could. Likewise, there was the option of buying something like Recovery Toolbox for Outlook. But Outlook didn't seem to have the "compaction" option that Tom described for OE. I decided that I wasn't actually having bizarre behaviors; I was just having difficulty exporting from Outlook. So I skipped that step. But I did restart the computer - actually, I knew that a cold restart could sometimes make a difference, so I completely shut down the computer for a minute or two and then restarted it
It was at this point I discovered I had been operating under a misimpression. I had been assuming that I was using Outlook 2003, because I was using Microsoft Office 2003. I forgot that, quite a while back, I had switched to using Outlook 2007. I'd had an opportunity to upgrade, and I'd done it. This would presumably not make much difference for the present attempt to migrate from Outlook 2007 to OE. But it might have made a difference for importing from Outlook to Thunderbird. Still, at this point, shouldn't the latest version of Outlook Express be able to import from the latest version of Outlook 2007? I decided not to revisit the Thunderbird question (by e.g., seeking to convert from Outlook 2007 back to 2003) at the moment. I hoped, instead, that I could make the OE approach work.
But how? OE still gave me the error when I tried importing from Outlook. Continuing with Tom Koch's advice, then, I re-registered OE by going to Start > Run > msimn /reg, followed by going into Internet Explorer and choosing Tools > Internet Options > Programs and then selecting Outlook Express as the default e-mail and newsgroups client. Tom then had me do a Start > Run for five different DLLs, as follows:
regsvr32 urlmon.dll
regsvr32 mshtml.dll
regsvr32 shdocvw.dll
regsvr32 browseui.dll
regsvr32 msjava.dll
But I didn't get too far on that list. For the second one, mshtml.dll, I got an error message: "mshtml.dll was loaded, but the DllRegisterServer entry point was not found. This file can not be registered." It can be funny to hear that someone or something was loaded, and so I had a mild chuckle about that, and then my customary frown returned as I realized I was now going to have to research a problem within a problem (within a problem, within ...). Moreover, it quickly appeared that this could be a complex problem within itself. LyonHaert suggested that perhaps regsvr32 was not appropriate for this task. Since Tom Koch's article dated from 2005, I thought it might not be providing the best guidance at this point. I did wonder, though, whether I should try his penultimate suggestion of repairing or reinstalling OE. I went into Windows and selected Control Panel > Add or Remove Programs. At the left side, I selected Add/Remove Windows Components. I unchecked Outlook Express and proceeded with the uninstall; and then I returned to that same place and reinstalled ... and got the same result as before. [It later occurred to me that I may not have rebooted after taking this step, and that doing so might have shown that this fixed the problem.]
Trying another randomly offered solution, I went to Start > Run and typed in "C:\Windows\System32\fixmapi.exe." This did not appear to accomplish anything. Nevertheless, following the instructions, I rebooted. Well, I was wrong. This step did achieve something. When I tried importing from Outlook into OE this time, I did not immediately get the error message. Instead, OE paused; Outlook started up; and then OE moved on to the next step and asked me which folders I would like to import. It had obviously consulted Outlook on the matter: the list of folders included custom folders that I had set up in Outlook. I told it to import all folders, and it proceeded. Then it said, "Congratulations! Your data from 'Microsoft Outlook' was successfully imported into Outlook Express." Congratulations indeed. I was looking for the champagne. I took a look at what had been accomplished and, sure enough, it looked like everything was there - the e-mails, the HTML (in some incoming e-mails), and their attachments. Woo hoo!
Now what? As I dimly recalled, I had tried so desperately to get this stuff from Outlook into OE so that I could then move it from OE into Thunderbird. Was T-bird going to swallow it? I opened Thunderbird. I went to Tools > Import > Import Everything > from Outlook Express. And it worked. Just like that. Well, not just like that. As with OE, T-bird took a minute or two to import all of those contents. I inspected the results. A more careful audit revealed that all was not perfect. For one thing, I had been excited about the e-mails and had neglected to check the Calendar in OE. It was empty. Also, a precise count of e-mails found that one message (from more than 1,000) had failed to make it from Outlook to OE, and two more had failed to make it from OE to Thunderbird. I wasn't going to try to figure out which ones. Possibly they had been duplicates.
The Calendar in OE was empty because, as I now noticed specifically, OE did not have a Calendar feature, and neither did Thunderbird. OE had copied my Calendar folder from Outlook, and Thunderbird had duly copied it from OE, but that folder was empty. I was definitely going to have to continue on down the road to Evolution if I wanted e-mails, Contacts, and Calendar all in one place. And, reviewing earlier findings, I had not even been able to get my Contacts into Outlook. It looked like I was done with Outlook Express, then, except to delete its contents so that I did not wind up with unwanted duplicates sometime down the line. It also seemed that I was done with Outlook, unless I wanted to try downgrading from Outlook 2007 to Outlook 2003, on the hunch that Thunderbird might find it easier to import the Calendar from the latter. It looked like people had had some problems with that downgrade, so that wouldn't be my first choice.
Reviewing my notes, I recalled that I had begun by moving my Contacts from Palm Desktop into Yahoo, and had then exported the contact information to yahoo_ab.csv. I had not been able to import my Palm Calendar and To-Do list into Yahoo Calendar, but by using CompanionLink I had been able to import these data into Google Calendar. Then, from Google, I had exported the calendar data to a Calendar.ics file. In short, I had my contacts in yahoo_ab.csv, my calendar in Calendar.ics, and my e-mails in Outlook. I had successfully moved the e-mails from Outlook via OE to Thunderbird. Now I wanted to see if I could get the contacts and calendar into Thunderbird. I could not get them there from Outlook, but that was fine; it might make more sense to import them directly from yahoo_ab.csv and from Calendar.ics.
Or, backing up a step, I wondered whether my tinkering around had now opened up any other ways to get the contacts from Yahoo to Thunderbird without the additional step involving the exported yahoo_ab.csv. In Yahoo, I saw that Contacts > Import/Export > Export did give me the option to export to Netscape/Thunderbird. I took that option and got a file called yahoo_ab.ldif. I wondered if Evolution could also import this. In Evolution's File > Import > Import data and settings from older programs, I saw that it was willing to import settings (which, I hoped, included e-mail account setup information, which I hoped had made their way, by some miracle, from Outlook through OE to Thunderbird) from Netscape, but not from Thunderbird. It was still possible to download a copy of Netscape Navigator, but I couldn't remember whether Navigator included an e-mail program, or precisely what they were talking about here. I hadn't heard much about Netscape for years, and I had to assume that it would have its own difficulties in importing from these newer programs. Anyway, from a look into OE, I surmised that no account settings had actually survived even the first leg of the journey. So I didn't bother installing Netscape, to see whether it would somehow aid in this process.
So this left me with the other Evolution File > Import option, which was to import a single file. It was willing to import three different kinds of .csv files as well as .ldif files. I now happened to have an .ldif file lying around, so I navigated to that. (At first, Evolution didn't see the folder it was in, but screwing around for a while cures all.) Out of curiosity, I backed up and selected the yahoo_ab.csv file instead. This, it said, was an Evolution-type .csv file. Fine. It seemed I could use either a .csv or an .ldif to get the data from Yahoo into Evolution. I went with the .ldif. After importing, I went to Evolution's View > Window > Contacts. There, I observed that my contacts had indeed been imported successfully into Evolution. One relatively minor problem: those codes mentioned above (e.g., "&") had not been converted back into the original characters (e.g., "&"). I did a search for ";" and that seemed to turn up all of the address book records containing those semicolon-type codes. I thought about editing them manually and then wondered what would happen if I edited the .ldif or .csv file before importing. I had already done that before importing into Outlook, but in that case I had also made other changes (e.g., deleting empty columns) that, I feared, might confuse Evolution. Then again, I thought it might be worth seeing what would happen. So I opened that yab1.xls file I had created to import a modified address book into Outlook, and I saved it in .csv format.
Before trying to import that .csv into Evolution, I tried to select and delete all of the address records that I had already created in Evolution, so that I would not have duplicates. Unfortunately, this ran into a new problem. A moment earlier, while tinkering in Evolution, I got an error message: "evolution-data-server-2.26.exe has encountered a problem and needs to close." Unlike the other Evolution error mentioned above, this one seemed to paralyze the program. I killed Evolution and restarted it. It was very reluctant to restart; it probably took five minutes to do so. Not a good sign. When it did come back up, it said it was searching the contacts. I had to enter a new search for nothing (i.e., an empty search field) to get all of the contacts listed again. I selected all addresses (Ctrl-A) and deleted them. But even though they were all highlighted, this did not delete all of them; I had to repeat the operation several times to delete them all. When I gave the command to import the edited .csv file, Evolution said "Importing" but then froze; apparently it did not have a filter for incompatible imports. Ultimately, I clicked on the X to kill the "Importing" dialog, and that killed Evolution. I tried again: this time, I went back to the original yahoo_ab.csv, edited it to remove the semicolon codes, and saved it as yab1.csv, without deleting any fields. This, too, caused the Evolution importer to freeze, and after this crash Evolution failed to come back when I tried to restart it. So apparently importing the .ldif was the best approach. I went into Windows Task Manager (Ctrl-Alt-Del > Processes) and saw that a half-dozen Evolution processes were running, but I wasn't seeing any Evolution program in operation. I killed all those processes and tried to start it again, but it really didn't want to run.
This made me wonder whether I should perhaps try importing the .ldif and other data into the Ubuntu version of Evolution. I had seen one or two critical comments on the Windows version; perhaps it was not very well developed yet. In Ubuntu, on another computer, I took a look at the list of file types that Evolution was willing to import. It looked more or less identical to the list in the Windows version of Evolution: .csv, .ldif, .ics, etc. So, OK, what would I have to make available to Evolution in Ubuntu? The .ldif, for Contacts; the .ics, for Calendar, and the e-mails, in some form from OE or Thunderbird. I decided to focus on that last part.
When I tried exporting e-mails from OE, it said that it would export to Outlook or Microsoft Exchange, neither of which seemed very helpful; and when I went through its export process with a sample folder, it gave me no options, and I couldn't tell where the output had gone. In Thunderbird, meanwhile, there did not seem to be an export option. The FAQ said that, since T-bird saved e-mails in plain text, the process involved simply copying or moving the e-mail files to another program's mail folder. But this didn't seem right: the e-mails I was looking at in Thunderbird sure looked like they contained HTML.
I wondered whether I could use Yahoo or Google to convert my e-mails more directly to a form that Evolution could import. Could I get the e-mails from Outlook, or at least from OE, into one of those web-based PIMs? Surely I could, but their web interfaces did not make the process obvious. As another possibility, I had resisted the idea of exporting e-mails from Outlook in .csv format, assuming that they would be plain text and, moreover, would not have their attachments. But now I thought I had better test those assumptions. In Outlook, I exported a small folder of e-mails in Windows- (not DOS-) style .csv format. I copied that .csv file over to the Ubuntu machine and imported it into Evolution. Evolution recognized it as an "Evolution CSV or Tab (.csv, .tab)" type of file. Weird thing, though - Evolution did import those e-mails, but it treated them as Contacts, not as e-mail messages, and it didn't seem willing to let me move them to the Mail folder.
I felt I was beginning to run out of good options for PIM software. I had reached a similar point in the past, at various times when I had looked at alternatives to Outlook and Palm. Certainly I was not confident that Ubuntu, Evolution, Thunderbird, and my understanding of the Ubuntu world were at a point where I could quickly and intuitively use reliable Ubuntu-based software to handle PIM needs. We - the programmers and I - had made some progress in that direction, but we were not there yet. Yet I did need to get past Outlook and Palm somehow.
It seemed like the next step was to try getting everything - contacts, calendar, and e-mails - into Yahoo! or Google. As I had been working through the approaches described in this post, I had slowly come to appreciate the possibilities in a PIM that I could access from any system, anywhere. Ideally, I would find ways to back up the online data, to use it offline, and otherwise to address the concerns that had prevented me from seriously considering a cloud PIM. By this time, though, I had come across some materials that suggested that I could indeed address those concerns. It seemed that I might look especially for Ubuntu tools that would meet these needs. There would still be the security concern. To get into Gmail, someone needs only an account name and a password. To get into archives that I save on my computer, they need my computer. That substantially narrows down my exposure. So probably I would not tend to keep everything online.
My transition from the Palm PDA and Outlook 2007 to Ubuntu was therefore going to stop, for now, at either Yahoo! or Google online PIM software. Between the two, I did not have a strong inclination one way or the other. At first, I found Google's interface slightly more intuitive. It also seemed like there were more guides and utilities available for Google than for Yahoo. So I assumed that I would work toward getting the rest of my PIM data into Google.
To get everything into Google, there were still a few more things I needed to do. Google already had my calendar. Somehow, in this process or previously, it had also acquired my contacts. To be on the safe side, I imported my yab1.csv file into Google as well. It automatically detected and merged some of the duplicates, but there were quite a few left over. It occurred to me that, to make a clean sweep of it, I should also bring in my contacts from Hotmail. That process was nearly instantaneous: export a .csv from Hotmail and import it into Gmail.
Now, how to get e-mails from Outlook into Gmail? According to Digital Inspiration, the first step was to follow the advice of the How-To Geek to use Gmail IMAP in Outlook. I was partway through this process when I discovered that Gmail did not allow me to create folders. Gmail instead used labels. This was great for some people, no doubt. For me, it meant that I couldn't put e-mails into different files. They were quite right in acknowledging (if only belatedly) that not everybody was going to be happy with this innovation. I was also told that Gmail groups emails in a thread into a single line in the inbox. I decided to back up and switch directions: I was going to make Yahoo, not Google, my online PIM.
So here, it was the same basic question: how do I get all this data from these various sources into Yahoo? Starting again with the contacts, now that I had them all combined (with quite a few redundancies) into Google, how to export from there to Yahoo? I went to Gmail > Contacts and indicated that I wanted to export all contacts to Outlook CSV format. That produced a CSV file. I went to Yahoo! Mail > Contacts > Import/Export > Import Microsoft Outlook (CSV) and imported that file. It worked, but Yahoo did not filter out any duplicates. From Yahoo, I exported a new yahoo_ab.csv file and opened it in Excel. I then deleted all my contacts from Yahoo (Contacts > Address Book > Erase Your Address Book). In Excel, I sorted all rows by first name. I inserted three columns before the existing column A. In the new column A, I concatenated the first 20 or so columns of the spreadsheet. For example, in row 2, I used a formula that began with "=c2&d2&e2&f2" and so forth. In the new column B, I counted the number of cells in the row that contained data, using "=counta(c2:be2)." In the new column C, I asked whether the entries in columns A and B were the same in this row as they had been in the previous row. (I used "=IF(A1<>A2,"",IF(B1<>B2,"","x"))".) I copied these formulas down to all rows in the spreadsheet. Basically, column C told me if this row contained the same text in its first 20 columns and, moreover, that the total number of columns containing data in this row was the same as the total number of columns containing data in the previous row. If both of those conditions were met, I got an "x" in column C. I converted the calculations in these rows into values (i.e., working from cell A1, select everything and then do Edit > Copy and Edit > Paste Special). I sorted all rows by column C, and deleted all rows containing an x in column C. That got rid of many duplicates. To check for more, I resorted by first names, cleared out the contents of columns A through C, and marked rows in which the first and last names were the same as in the previous row. These required more manual editing. When such steps had deleted all duplicates as far as I could see, I cleaned up other things (e.g., the semicolon codes, above). Then I saved the file as no_dups.csv and re-imported it into Yahoo.
Next, the Calendar. I had my Calendar.ics file from Google, courtesy of CompanionLink, but wasn't sure what to do with it. Yahoo's instructions didn't say anything about an .ics file. Instead, they told me to click on their Share Calendar link and, in effect, send an e-mail to myself at another address. I used my Hotmail account, which then received an e-mail letting me know that I had shared my Yahoo Calendar with myself. As another instruction page promised me, at the bottom of that e-mail was a line that began, "Add to Outlook, Apple iCal, or another calendar," followed by a long URL. I copied and pasted that URL into the Firefox address bar in another tab in my browser. This caused my Calendar.ics folder to be downloaded from Yahoo - which was not too helpful because, among other things, my Yahoo! Calendar was obviously empty. OK. Apparently I had found the wrong instruction page. I removed the Hotmail share. Eventually I found instructions by which I went to the Calendars heading on the left side of the new Yahoo! Calendar beta version. There, I clicked the + sign and selected Create New Calendar. I noticed that the "Import Events - Import from an .ics file" option at the right side was grayed out; but as I eventually figured out, first I had to create a calendar in order to get that import option to be available. I typed Personal as the name of my new calendar and clicked the Save button at the top of the webpage. The Calendar section at the left side now showed that I had created a Personal calendar, though it looked like they were spelling it as "Persona." Anyway, I could now browse to the Calendar.ics file in the Import Events box at the right side. I clicked "Upload" and it worked: I had a Personal calendar full of stuff.
Yahoo! had already created a Ray_Woodcock calendar for me, and I wanted to delete that; but as others had found, the button available on the calendar page only allowed me to delete events, not the calendar itself. I found some Yahoo instructions that said I should just click on a Delete button, but there wasn't one. Their instructions told me to get there by clicking on the calendar's name, but when I did that, all I got was "Edit Calendar" and "Share With Contacts" options, neither of which applied. But anyway, I did have what they called a calendar "layer," my Personal layer. I wanted to redistribute items from that layer, in two ways: I wanted to get the items that had been imported as "Tasks" (see above) out of the Calendar and back into Yahoo's To-Do list, and I also wanted to move some items from the Personal calendar into a not-yet-created Business or Professional calendar. But there appeared to be no mass-moving option, where I could check boxes next to a number of tasks or scheduled items and move them to another calendar or to the to-do list. When I edited an individual item, there was no "move" option. There was a drop-down box where I could choose between the existing calendars, so apparently I would be able to move an item to the Professional calendar once I created it. I did so, using grey rather than blue for its color, and that was indeed how it worked.
So anyway, I did have the Calendar in Yahoo. Now, how about the e-mails from Outlook? There was no button on the Yahoo! e-mail site, as far as I could tell, to permit me to import my e-mail, and some users had concluded that it was not possible to import e-mail into Yahoo. It seemed that I might try using Outlook as a clearinghouse: bring e-mail into there from Hotmail or wherever, and then archive it all into Yahoo. The solution was supposedly to download and install YPOPs, and then configure Outlook for the Yahoo account. In Outlook 2007, this involved going to Tools > Account Settings > E-mail > New > Microsoft Exchange. There, I entered the information for my Yahoo! e-mail account. Then I clicked on the "Manually configure" box and went to Next. I may have missed something here - I had some distractions at this point - but it seemed like Outlook tried to set things up automatically anyway, despite my "manually configure" indication. At any rate, after entering my password several times without success, I got an error message, "Cannot Log on to Server" and another opportunity to manually configure server settings. For the incoming mail server, in place of pop.mail.yahoo.com (which had not worked), the instructions had me enter "localhost" (or, if that failed, 127.0.0.1), and likewise for outgoing. Then I selected "Remember password" and More settings > Outgoing server > My outgoing server (SMTP) requires authentication. And that was supposed to do it.
And it did, but in an odd way. In Outlook, I had been able to click on a separate folder for my Hotmail and Gmail accounts. But in Yahoo, under this arrangement, there was no separate folder. Whatever was in my Yahoo Inbox was immediately brought into my Outlook Inbox. It looked like I could change the settings in YPOPs to control that, but the two would still apparently function as one. Or - no, more accurately, it was a one-way street. Outlook could see what was in Yahoo!, but not vice versa. So it still did not look like I could transfer my e-mails directly from Outlook to Yahoo.
I played with the Calendar in Yahoo for a while. I set up another category of calendar entries; I added and changed some items. A little bit of work with Yahoo's web-based PIM made me crave desktop PIM software. The web interface was slow, and in a number of little ways I found Yahoo's implementation frustrating. I couldn't click on a date and go directly there, for example; and when I was working through a calendar of upcoming dates, adding them to my schedule, Yahoo insisted on reverting to today after each saved entry, even though I wanted to be going forward, not backward. I had left Evolution open on the Ubuntu machine, and had also played with it a bit, and decided I was not entirely ready to give up on it.
There were some other Yahoo-related tools I could have explored more at this point. One worth mentioning was TrueSwitch, which was apparently prepared to transfer my previous e-mail account (e.g., Hotmail or Gmail) to Yahoo. But it looked like it might make it appear that I was no longer using the previous account, and I didn't want that.
My working conclusions, at this point, were (a) Google had its act together, more than Yahoo at least, in the PIM area, and (b) Evolution on Ubuntu (not Windows) was the most likely alternative to Outlook. I really wanted to get away from Outlook, so I decided to investigate the Outlook to Gmail to Evolution route. Because Gmail lacked categories for sorting e-mails, and because I expected its calendar interface to be similarly slow, I might use it primarily as an intermediary, with Evolution as the preferred working area. Evolution had not been able to send messages in Hotmail, but how about in Gmail?
I started by moving everything to Google. For the calendar, I began by deleting the one that was already in Google, since the one in Yahoo now represented the state of the art. To delete the Google Calendar, the instructions told me to go, in Google Calendar, to Settings > Calendars > Unsubscribe or Delete. Or, since what I was seeking to do was to delete all events from my primary calendar, the instructions were, instead, to go to Settings and then select the Delete link by my primary calendar. After doing the latter, sure enough, the Google Calendar was empty. Now, to import from Yahoo Calendar, the instructions said to export a .csv file, break it into separate parts if it covered more than one year (to avoid errors), and then import the .csv. Nasty little problem: "The exported file doesn't support recurring Yahoo! events. Individual items will be created for recurring events that fall between the dates you've selected." If I did go this route, it appeared that I could try using a package someone had prepared to sync Evolution with Gmail Calendar. Or maybe that would no longer be necessary: it looked like it was now possibly to simply integrate Google Calendar into the GNOME (Ubuntu) desktop.
I checked Yahoo again. After some digging around, I eventually found Yahoo help pages on exporting both to a .csv file and to an .ics file. The steps for the latter were easy enough: click on a Yahoo calendar layer, select Edit Calendar, and choose Export. So I exported just one Yahoo calendar layer to an .ics file - but what could I do with it? In Ubuntu, I went into Evolution and selected File > Import and chose the .ics file I had just exported from Yahoo Calendar. It didn't work. It looked like it was going to start, but then the importer disappeared and I was left with the same old empty calendar. Others had had the same experience. I had recently downloaded some Ubuntu updates and hadn't yet rebooted the system, so I tried that, just in case some of those updates would conveniently solve this issue. Unfortunately, no, they didn't.
I thought about trying Zimbra for Ubuntu. I might have tried it before; by this time, I had been at this project for quite a while, and was not remembering all of the various permutations I had taken for a spin. It sounded, though, like Zimbra was not being updated very actively; in particular, a Zimbra installation guide indicated that it did not support Ubuntu 9.04. A search for information specific to importing an .ics file into Zimbra for Ubuntu turned up practically nothing. This was looking like one blind alley that I should not be entering. Finally, a general search for ways to import .ics files into anything in Ubuntu 9.04 turned up very little.
Well, all right. How about Thunderbird for Ubuntu? I went into Synaptic and installed the package called "thunderbird" (just plain old thunderbird, not "thunderbird-traybiff" or thunderbird-locale-tr" or any of the other hyphenated thunderbirds. I went into Applications > Internet > Mozilla Thunderbird Mail/News. This gave me an Import Wizard, with the options to "Import Preferences, Account Settings, Addressbook and other data from" followed immediately by "Don't import anything." So I guess I didn't have that option, at least not right away. I finished that wizard. Next, Thunderbird gave me an option to set up different kinds of e-mail, RSS, and newsgroup accounts. The e-mail options were a plain vanilla "Email account," a Unix Mailspool (Movemail), or Gmail. I wanted integration with Firefox, so that I would get a pre-addressed Thunderbird e-mail, ready to fill out and send, when I clicked on a mailto link in a webpage, so I thought I probably wanted an Email account rather than just one of the other two. "Email account" was the default anyway. So I went with that. Even though I didn't expect success in linking with Hotmail, I named my Hotmail account (which was the e-mail account I used most frequently) as my e-mail address. Next, it asked what kind of incoming and outgoing server I was using. According to kerry_s, as above, this required pop3.live.com and smtp.live.com. After entering the other settings, Thunderbird started up. Its Inbox was empty. I went into File > Get New Messages for > Get All New Messages. Its status bar said, "Connecting to pop3.live.com." And that's as far as it got. Eventually, I got an Alert that said, "Connection to server pop3.live.com timed out." Using the same approach as with Evolution, I used the rest of the advice, regarding servers: I went into Edit > Account Settings > Hotmail > Server Settings and changed the port to 995 and the secure connection to SSL. I also edited the outgoing server. Then I tried again. This time, I got as far as a dialog asking me to enter my password. I got an "authentication failed" message, probably because I didn't use my complete Hotmail account name in all of the boxes. This time it worked. I was seeing, in T-bird, all of the e-mails in my Hotmail Inbox. Now I tried clicking on a mailto link in a webpage. It gave me an e-mail. I typed what I wanted and clicked "Send." It disappeared, so I didn't know whether it went or not. It occurred to me that Evolution might still be my default e-mail client, in which case my message was probably sitting in its outbox. I started Evolution and took a look. Sure enough! I went into Thunderbird's Edit > Preferences and made some adjustments, including designating Thunderbird as my default. I tried clicking on that webpage's mailto link again. There wasn't a Help > About menu option, but it looked like I got the same blank e-mail form again. I went into Synaptic and uninstalled Evolution. I tried clicking on that mailto link again. This time, nothing happened. I shut down and restarted Thunderbird. That didn't make any difference either. I tried uninstalling and reinstalling Thunderbird. This didn't seem to change anything at all, not even my settings.
Following the advice of Jesus_Valdez, I went to Ubuntu's System > Preferences > Preferred Applications. Under Internet > Mail Reader, it said "Custom" with a command of "evolution %s." (Incidentally, according to Jesus, a very similar setup (using "firefox %s") would cause mailto links to open a separate tab in Firefox for web-based outgoing e-mails.) Instead of "Custom," I had a choice between Thunderbird and Mozilla Thunderbird. This may have meant that the latter was already installed on my system when I used Symantec to install the former. So I selected Thunderbird. This changed the command line to "thunderbird %s." I closed that dialog and clicked again on that mailto link in that webpage. I got what was apparently a Thunderbird e-mail form. I typed some information into an e-mail message whose subject line said "Test," and clicked on Send. I got a Send Message Error:
Sending of message failed. The message could not be sent because
connecting to SMTP server smtp.live.com failed. The server may be
unavailable or is refusing SMTP connections. Please verify that your SMTP
server setting is correct and try again, or else contact your network
administrator.
What was different about this failure was that it wasn't a matter of an e-mail sitting in the Outbox and not being sent, as had been the case with Evolution. This one wasn't even being saved and placed in the Outbox. In that message, I clicked on File > Save. Then I closed that message and took a look at Thunderbird. Instead of an Outbox, T-bird had an Unsent folder under its Local Folders heading. There was nothing in the Unsent folder. So I didn't know what had happened to that message I had supposedly saved. I went back into Thunderbird's Edit > Account Settings and switched it from TLS to SSL. Again, clicking on the webpage's mailto link produced an e-mail form; I filled it out (with a change to the e-mail address, in case all these junk messages were actually being delivered to someone without seeming to be); and when I clicked Send, I got a dialog that said, "Sending Messages - Test. Status: Connecting to smtp.live.com." Then I got that same "Send Message Error" message. I tried again, this time using Edit > Account Settings > Outgoing Server (SMTP) from inside the Test e-mail to change to "TLS, if available." When that failed, I tried the "No" option in response to the "Use secure connection" question. It did not seem that the secure connection setting was the issue. I changed it back to TLS. I tried changing the port to 995 (see above). This at least gave me the delayed "Sending Messages" dialog (above) before giving me the "Send Message Error" dialog. To see if I could learn more about the problem, I followed Jesus's advice and tried the "firefox %s" alternative. It didn't work. I just got a blank tab in Firefox - one that kept replicating itself until I changed it back to "thunderbird %s."
I researched the "Sending of message failed" error message. Jeffry D suggested changing the outgoing port from 25 to 587 because sometimes routers, ISPs, or firewalls block port 25. When I tried it with 587, I got the "Sending Messages" dialog and then "Mail Server Password Required." This looked like progress. I entered my password. Unfortunately, I still got the same "Sending of message failed" error message. Another post suggested trying this new port number with SSL instead of TLS. That didn't help. And then, after letting everything sit overnight (without a reboot), I was able to send, using these settings: smtp.live.com, port 587, TLS. Success! (It occurred to me, about this same time, that possibly my problem was due to the fact that I had done an upgrade rather than a fresh install of Ubuntu 9.04.)
Retracing my steps, I wondered if those settings would now work for Evolution as well - the advantage of the latter being that, of course, it was a complete PIM, not just an e-mail program. I had uninstalled Evolution, so now I reinstalled it via Synaptic and named it as my default e-mail program. I clicked on the mailto link in that same webpage. I changed the e-mail address to the same one I had used for the message (above) that I had just sent from Thunderbird (so as not to bury some innocent and unwilling recipient), and sent it. This put it in Evolution's outbox. I then clicked Evolution's Send/Receive button. This gave me the "Unlock Keyring - Enter password for default keyring to unlock" message that I had gotten above. This time, I researched that message. People proposed various solutions for this. Before I could try them, I had to close the dialog box. If I didn't enter a password, it would only go away when I clicked "Deny." When I did that, I got a new dialog, asking for my Hotmail password. I entered that, but now I was back at the Unlock Keyring dialog. I tried Deny again. Now it just kept coming back. I entered my basic Ubuntu login password. It asked for my Hotmail password again and again. Finally I canceled. Following the advice of psyke83, I typed "seahorse" at the Terminal prompt. This gave me "Passwords and Encryption Keys." Under Passwords (the only tab with any contents), what I saw seemed to differ from what some others were seeing. The advice seemed to be to delete this entry, but I was hesitant to delete what looked like my login password. A couple of people advised typing "cd ~/.gnome2/keyrings" and then "rm default.keyring," but this gave me "rm: cannot remove `default.keyring': No such file or directory." All I had in .gnome2/keyrings was login.keyring. But Matt Cutts said he deleted this too, and it worked, so I tried it. I went back to Evolution and tried Send/Receive again. It asked me for my Hotmail password again. I entered that. Then it asked me to "Choose password for default keyring." I did. It still wouldn't send messages, though. It wasn't asking me for the keyring password anymore; now it gave me "Evolution Error: Error while performing operation. Welcome response error: Connection reset by peer." I tried again; same thing. I tried sending again in Thunderbird; success again. So, OK, I went into System > Preferences > Preferred Applications and indicated that I wanted Thunderbird as my default e-mail application. Then it occurred to me to send myself a test message, to make sure that when T-bird said it was sending successfully, it was actually doing so. It did work.
Now I wanted a way to get my e-mails from Outlook to Thunderbird. I could move them from Outlook to Hotmail to Thunderbird, but I had noticed that Hotmail seemed to insert extra blank lines in e-mails that I brought in from Outlook, and I didn't see a setting in Thunderbird to remove extra blank lines. But could I sync Thunderbird with my Microsoft Exchange account? In T-bird, I went to Edit > Account Settings > Add Account and filled in the blanks. Following Peter's advice, I chose IMAP and, for an incoming server, I just copied over what was shown as the Microsoft Exchange server in the Microsoft Exchange Settings box in Outlook's Tools > Account Settings > (click the Microsoft Exchange line) > Change. Thunderbird informed me, "Your existing outgoing (SMTP) username, [hotmail account name] will be used. You can modify outoing server settings by choosing Account Settings from the Tools menu." This put me back at Thunderbird's Account Settings dialog, with a new folder on the left side for my new account. I clicked OK and then, in Thunderbird, I clicked on the new Inbox under my new account. The connection did not succeed. I went back into T-bird's Account Settings and changed the server to the one shown in Outlook's More Settings (inside the dialog mentioned above) > Connection > Exchange Proxy Settings. I used the complete URL, beginning with https, but this resulted in a "Failed to connect to server https." I tried again without the https part (i.e., beginning with www), but this too timed out. It seemed that I had the server right the first time, so I went back to that. I still couldn't connect to the Inbox. I killed and restarted T-bird. Now I got an Alert: "The STAT command did not succeed. Error getting message number and sizes. Mail server pop3.live.com responded: login allowed only every 15 minutes." I didn't think I had logged into Hotmail within the past 15 minutes, but whatever. One source told me to use SSL encryption in Thunderbird and right-click > Subscribe to "all required folders." My right-click brought up a Subscribe option, but that just said "Please wait ..." and then timed out. No clue what to do next. Next, I found some Ubuntu community documentation that I couldn't understand. DownloadSquad said I needed to choose "Never" under Thunderbird's "Use secure connection" option. So far, no difference. DownloadSquad gave me more instructions on using Thunderbird to access a workplace Microsoft Exchange account, but these didn't seem to be written for my version of Outlook. As far as I could tell, their gist was that I should use the same settings for the outgoing SMTP server as I had already set for the incoming server. This didn't work, though. Lubeck recommended Outpod for moving Outlook data to Evolution, but someone said it didn't work on 64-bit Ubuntu. DjmUK outlined an approach that began with installing Thunderbird for Windows on the same computer as Outlook. This was for transferring e-mails, not for viewing Outlook's Inbox in Ubuntu Thunderbird (which, by now, I had decided I wanted), so I held this one in reserve. Besides, I was not eager to have yet another dependency on Windows.
It started to sound like Gmail, once again, might provide the smoothest solution. The scenario, in this case, was that I would link Gmail and Outlook 2007, so that I could use Gmail to view e-mails incoming to my work account (which I had previously viewed in Outlook); and then I would archive those e-mails off from Gmail to Ubuntu Thunderbird. In Thunderbird, I could sort e-mails into folders, and I wouldn't worry about doing that in Gmail. So that took care of the concern that had stopped me from trying Gmail previously (above). Working on the first step, I went to Gmail Help, which wasn't very clear. I switched to the How-To Geek (again) and went into Gmail's Settings > Forwarding > Enable IMAP. Then, in Outlook, I went to Tools > Account Settings. There, I saw that I had already enabled IMAP for Gmail, presumably while starting to follow the How-To Geek's instructions previously (above). I checked to make sure I had done all of the recommended steps. Then I clicked on Test Account Settings, and it reported success. I saw that I had the Gmail folders visible in Outlook (again, a deja vu). In Outlook, I dragged one of my e-mail messages from my Outlook Inbox over to the Gmail Inbox. I checked in Gmail, and there it was. Very easy. I could do this, I assumed, with all of my Outlook e-mails.
Now, how to get the e-mail from Gmail to Thunderbird? At this point, I began to slip into the Thunderbird vortex. Thunderbird, I was starting to see, was a world unto itself. There was Saman Sadeghi's Top Eight Thunderbird Plugins and some unknown individual's Top 100 Thunderbird Themes and Extensions and . . . It was like Firefox all over again. Which I liked. But first things first. Following instructions provided by es developed, I began by making a backup of my Thunderbird profile. In the search for instructions on how to do this, I ran across MozBackup, which offered a Windows-based backup solution for Firefox, Thunderbird, etc. Sounded good. I downloaded a copy on the Windows XP machine. But anyway, how to back up Thunderbird in Ubuntu? The advice of nixCraft as modified by some others led me to create this command:
tar zcvf /media/BACKROOM/Tbird/Tbird01.tar.gz
/home/[username]/.mozilla-thunderbird
The basic idea there was that my .mozilla-thunderbird folder would be archived into a single .tar.gz backup file called Tbird01, in a folder called Tbird, on a separate partition (named BACKROOM) on my computer. The .mozilla-thunderbird folder was said to contain all of my e-mails and all of my Tbird settings. I got an error message: "Cannot open: No such file or directory." It seemed that tar would not create the Tbird folder for me. No problem. That done, I tried again. It did it. I right-clicked on the newly created target file (Tbird01.tar.gz) and viewed it in Archive Manager. It looked fine. Now, continuing with the instructions of es developed (above), the first step was to enable IMAP following Google's instructions (see How-To Geek, above). Next, following other Gmail instructions, I had to set up IMAP access to Gmail in Thunderbird. Gmail's instructions didn't exactly match my situation: in the Ubuntu version, this involved Edit > Account Settings > Add Account > Gmail. There were some additional instructions, some of which seemed to have been built into this Gmail option in Tbird already. Once I took these steps, I had my Gmail account in the left-side "All Folders" pane of Tbird, but the e-mail message that I had successfully transferred from Outlook to Gmail wasn't appearing in the Inbox under that Gmail account. So I continued with the instructions provided by es developed. When that was done, I still had nothing in the Gmail Inbox on Tbird. I clicked Get Mail. And that did it. Presto! I was reading Gmail in Tbird. I created an Archive folder in Local Folders on Tbird, highlighted the incoming e-mail in Gmail, and moved it to the Archive. This seemed to be ready for prime-time. I went into Outlook and moved a handful of e-mails from the archive there to Gmail's Inbox. I went into Gmail to make sure they had arrived OK, and saw that they had been promptly placed in . . . the Trash? Along with a bunch of other things! Why was Gmail trashing stuff from my Inbox? Well, it seemed I had told Gmail to forward my incoming e-mail to Hotmail, and then trash the copy on Gmail. I fixed that and began moving my e-mails from Outlook through Gmail to Thunderbird. (For what it's worth, Webmail seemed to offer another approach that I noticed later.)
While that was going on, I took a look at that list of someone's Top 100 Thunderbird Themes and Extensions. I chose the Modern Modoki theme, which I was familiar with, and skipped down to the extensions and add-ons. From these, I decided to skip the Greek-to-English translator, along with many others. I did select quite a few, though, and then I followed a link to the Top 60 Tools for Gmail. I selected a bunch of those, and then followed yet another link to the Top 40 Tools For Google Calendar.
Thunderbird did not include a calendar. I would apparently be adding Lightning for that. I had briefly considered it (above), but now I was seeing that these lists of top extensions kept referring to it. So now that the e-mail situation seemed to be under control, I went into Synaptic and installed lightning-extension. I closed that and went into Ubuntu's System > Administration > Update Manager > Check. This gave me an error:
Could not download all repository indexes.
The repository may no longer be available or could not be contacted because
of network problems. If available an older version of the failed index
will be used. Otherwise the repository will be ignored. Check your
network connection and ensure the repository address in the preferences is
correct.
GPG error: http://ppa.launchpad.net/
jaunty Release: The following signatures couldn't be verified because the public
key is not available: NO_PUBKEY 3B22AB97AF1CDFA9Failed to fetch http://ppa.launchpad.net/ubunut-x-swat/x-updates/ubuntu/dists/jaunty/main/source/Sources
404 Not Found
Some index files failed to download, they have been ignored, or old ones used
instead.
When I closed that, Update Manager ran and found an update to another program. I went ahead with that. I suspected, though, that the error just shown came from the misspelling of Ubuntu in the line beginning with "GPG error." I went into System > Administration > Software Sources > Third-Party Software. There, I looked for the one named in the error message. I clicked on Edit and changed the spelling. I ran Update Manager again. This time, I got a different error:
An error occurred
The following details are provided:
W: GPG error: http://ppa.launchpad.net
jaunty Release: The following signatures couldn't be verified because the public
key is not available:
NO_PUBKEY 3B22AB97AF1CDFA9
That was the same NO_PUBKEY line as in the previous error. I Googled that and found that (maybe) it had to do with automatically importing Launchpad PPA keys when updating. I wasn't too sure what that meant, but I was glad to cooperate with whatever they wanted me to do. The process involved, first, downloading a script that someone had created to do the automatic update or importing or whatever. I saved it, typed "sudo nautilus" in Terminal, found where I had saved the download (which was called launchpad-update.tar.gz), and extracted its contents to /usr/local/bin. I exited that Nautilus session and navigated there in Terminal (i.e., "cd /usr/local/bin"). I typed "sudo chmod 775 launchpad-update." Then I navigated to /etc/apt/apt.conf.d and typed "sudo gedit 05ppa." That opened gedit (i.e., Text Editor) in superuser mode with a new file called 05ppa. In that empty file, I pasted this:
APT::Update::Pre-Invoke { "if [ -x /usr/local/bin/launchpad-update ]; then
/usr/local/bin/launchpad-update; fi"; };
I saved and exited gedit and tried running Update Manager again. It seemed my changes had had one undesired side effect, insofar as now Update Manager would not run at all. I closed down all programs and tried again. Still no luck. I restarted the computer. Still wouldn't run. The fix seemed to be designed to work with "apt-get update" so I typed that in Terminal. This gave me an error message: "Extra junk at end of file." I looked in the 05ppa file again and saw that I had pasted it wrong, or for some other reason the contents got garbled. I copied and repasted the line shown above and, what do you know, the full thing wasn't being pasted properly. When I fixed it, Update Manager ran. While I was in the process of fixing that, I noticed a "Do Not Enter" kind of sign on my Ubuntu panel:
An error occurred, please run Package Manager from the right-click menu or
apt-get in a terminal to see what is wrong. The error message was: 'Unkown
Error:' . . .
and so forth. ("Unkown" was the spelling in the message.) Anyway, when I restarted Thunderbird, I got an indication that Lightning was ready to import data from Evolution. I said no, because at this point I wasn't sure what was in Evolution. So, OK, I I had Lightning, and I could see calendar and task icons in Thunderbird; now how was I going to get my calendar here? I still had a Calendar.ics file from Yahoo (above), containing the export of one Yahoo Calendar layer. In Thunderbird Mail (not Lightning), I went to Tools > Import, but it was willing to import only Address Books, Mail, and Settings. I clicked on the Calendar icon and tried Tools, but there was no import option. Martin led me to double-click on that icon (so that the screen would actually change) and then go to File > Import Calendar. This worked; and unlike the .csv situation described above, repeating items imported via .ics were preserved as repeating items. This particular Yahoo layer contained relatively major events, occurring only once a year. I wanted a Lightning category for this type of item. I called this category "Milestones." So I went into Lightning's Edit > Preferences > Lightning > Categories option. There, I deleted the categories they offered, inserted one for Milestone events, and chose its color. I clicked on Lightning's "Find Events" button and selected "All Events." I highlighted all events, but right-clicking didn't give me an option of changing their category (or anything else) en masse. On the left side, under Calendar, the only one listed was "Home," which apparently couldn't be deleted. I renamed it to Milestones. Now all of these items I had imported were in the Milestones calendar. I didn't really need a Milestones category for them after all, so I left that alone. Now I wanted to create a new calendar for Appointment items, so that when I imported my Appointments.ics file, maybe all of the items from the Appointments layer in my Yahoo calendar would go into that calendar. But I wasn't sure how, so I just imported them. Sure enough, the appointments went into the Milestones calendar. Now, belatedly, I found that I should have gone to Calendar > New Calendar (duh). I went to the Find Events button, chose All Events, selected them all (Shift-Click), and clicked on the Delete button. I repeated this several times. It still didn't delete all events. I right-clicked on the Milestones calendar and deleted it. But this deleted everything *except* the Milestones calendar. Which was good, in a way: all of these Appointment items I had just added seemed to be gone. There were three bad parts to this: (1) I now had two Appointments calendars, (2) I couldn't delete either of them, and (3) I also couldn't delete the individual Milestones items that now were apparently being seen as Appointments. I killed Thunderbird and restarted it. That solved the problem. I now had just the Milestones calendar, and it was empty. I redid the Milestones.ics calendar import process. But this gave me an error message:
Error getting calendar
20 item(s) were ignored since they exist in both the destination calendar and
[source filename].
Accompanying this was another:
An error has occurred
An error occurred when writing to the calendar Milestones!
I recreated the Appointments calendar. Now I was able to delete the Milestones calendar. I imported Appointments.ics again. That seemed to work. I deselected the Appointments calendar, recreated the Milestones calendar, and re-imported Milestones.ics. Now I got a dialog that said this: "Select Calendar. Which calendar do you want to import these items into?" Not sure why I didn't get that before. I chose Milestones and clicked OK. Now things seemed to be working just right: if I selected the Milestones calendar, I got Milestones items in orange; if I selected the Appointments calendar, I got Appointments in red; and if I selected both, I got both. I recreated this process with the remaining Yahoo calendar layers, and wound up with a calendar that seemed to look and function well. Somehow, during these various importings and exportings, some items on my calendar had acquired a prefix: the word "Tasks:" appeared in front of them. Unfortunately, there didn't seem to be a global search and replace option, so apparently I would have to deal with them manually.
Now seemed like a good time to explore the Thunderbird add-ons that I had encountered previously, to see whether any of them might enhance my Tbird experience. I decided, for the moment, that I probably didn't have enough e-mail volume to justify (or to learn) the fast e-mail handling shortcuts provided by Nostalgy and GmailUI. It looked like the Buttons extension might be more my speed. To install it, I downloaded it and then went, in Thunderbird, to Tools > Add-ons > Install and navigated to the download. I also installed Adblock Plus, ImageZoom, and QuickNote. I was going to install Enigmail, but at this time there didn't seem to be a 64-bit version that was compatible with Firefox 3.0 and Thunderbird 2.0. Same problem with some other recommended add-ons.
Now, how could I sync this Lightning calendar with Google Calendar? I hoped to have it accessible when I was away from my desktop computer. David Pierce pointed me toward Provider for Google Calendar, which would let me sync it with Lightning. But it looked like a potentially complex program, with lots of bug reports. I was hoping for something simple, primarily because I had used up my time for this project and needed to be moving on. This concern also deterred me from trying GCALDaemon. I didn't actually need the Google connection right away, and it was not farfetched to hope that things would become significantly more streamlined in the coming year (since I mostly had time for this sort of pursuit in the summer). In the meantime, since Thunderbird's mailbox (and presumably Lightning's data files as well) were apparently capable of being backed up or copied with simple rsync or tar commands, it seemed that I might be able to get by, for the time being, with perhaps keeping the calendar primarily on my laptop, transferring calendar files via jump drive, etc. This, anyway, was the state in which I had to leave the topic of calendar synchronization for now.
All right, then, how about bringing my Contacts into Thunderbird? I still had my .csv file output from Yahoo. In Thunderbird, I went to Mail > Address Book > Tools > Import > Address Books and navigated to where that .csv file was. But the dialog box didn't see it. Then I saw that, in the lower right-hand corner of the dialog, I needed to specify that I was looking for a comma-separated file. When it imported the contacts, I went into the upper-right corner of the address list and clicked on the unlabeled corner box to indicate which fields I wanted to display. Oddly, although it had looked like Tbird knew the difference between a first and a last name, there were not separate name fields in the actual list of addresses; there was the last name only. But then I saw there was an option in View > Show Name As for first name first, or last name first, or display name. But when I switch to last-first, I saw that maybe 100 of my contact list entries (the status bar didn't say how many I had selected) had neither first nor last names, and those that did have names in the name fields were now showing only the first name. Both first and last names had been imported - I could see them in the "card" shown at the bottom of the screen - but they did not display in the list. Also, if I displayed all of the fields for each name horizontally, there was not space; there was no horizontal scrollbar. I deleted the imported address book and tried again. I now saw my mistake. I had not read the Import Address Book dialog box carefully. It now appeared that I needed to conform my .csv file's columns to those of the Thunderbird Address Book template. The order could vary, but not the content. For example, apparently I had to have a Display Name field. The instructions said I could check or uncheck fields, but that was not so; the boxes did not accept checkmarks.
This was OK - I could do this - but I wondered if it might be easier to import the .csv file into Gmail instead. I intended to sync Thunderbird and Gmail in any case, so why not try? In Gmail, I went to Contacts. I had not yet cleared out the many redundant addresses that I had imported there, early in the game. So now I selected them all and deleted them. Wow: once again, that was easy. I went to Import, browsed to the .csv file, and imported the contacts. They seemed to be displayed properly. So now, upon Adam Myers's referral, I thought I'd try using Zindus, a Thunderbird extension that would supposedly allow me to manage my Gmail contacts from within Thunderbird. Once installed, I went into Thunderbird's Tools > Zindus > Add. It defaulted to "sync with Google." I entered my Google e-mail address and password. It said, "Login succesful" [sic]. I went ahead with the sync. It worked; Tbird's Address Book now contained the names and data that I had imported into Gmail from that .csv file. The names only displayed properly if I chose View > Show Name As > Display Name. The display of names in Tbird's Address Book was still poor; the other approach (of cleaning up the .csv file and matching its field names with those used in Thunderbird) would still probably yield a better result. But for now, this would do; it did seem that the contacts were syncing between Gmail and Tbird.
This marked the end of this exploration for now. I had gotten my data out of Outlook and the Palm PDA. That had been a bit of an ordeal in itself. In doing so, I had severed one significant long-term reason for dependence upon Microsoft. I had also experimented with a variety of potential PIM solutions, and had gained an introduction to their capabilities and drawbacks. I was pleased to have found, in Thunderbird, an Ubuntu-based solution that appeared capable of doing what I needed. I had also achieved partial synchronization of e-mail, contacts, and calendar among various computers, operating systems, and programs; and by playing around with these possibilities at length, I had gained some experience with these solutions and some degree of confidence in their reliability. I expected that, by the time another ten months passed, I would have much more experience in what was currently feasible in the PIM area, and would be more attuned to new developments. And I expected that there would be significant developments in that time - with, among other things, the release of two Ubuntu upgrades and the approach of Google's anticipated Chrome operating system.
With all that said, plainly I had invested much more time in this investigation than would ideally have been required. Ultimately, while I found the hobbyist aspect interesting, and while I had kept my computer skills somewhat up to date through just this sort of occasional exploration over the years, I did approach this project because I was searching for solutions that would work. I was not entirely happy with the investment and the payoff from this project, notwithstanding all the rationalizations just offered. It was ironic, in this sense, that I wound up investing so much time in order to find a PIM solution whose purpose was, in significant part, to facilitate efficient time management. What tipped the balance, in this particular case, was that I had repeatedly sought to move away from Outlook, and had undertaken the whole Ubuntu project in order to reduce my dependence on Windows. This time around, I decided to spend the time necessary to actually make the switch, and at this point it appeared that I had succeeded in doing so.