Thursday, September 30, 2010

Connecting Network Attached Storage (NAS) to a WinXP Guest in VMware: FAIL

I had two desktop computers running Ubuntu 10.04.  On one of them, I was running VMware Workstation 7.1, with Windows XP SP3 as a guest operating system in a virtual machine (VM).  I had just figured out how to network these two computers using Samba shares within a home network, where the two computers were connected via ethernet cables to a shared router.

Now there was a new question.  Could I add a Synology DS109 network-attached storage (NAS) device (essentially an external hard drive enclosure designed for network backup and file serving) to this network?  Of course I could, in the sense of running an ethernet cable from the Synology to the router; but what I was wondering was whether I could make this work despite the fact that the software for the Synology was available only for Windows and Mac, and not Linux.

It was a question, in other words, of whether I could run the Synology software in Windows XP in a guest VM.  I gave it a whirl.  I ran the Synology installation CD and went through the steps to set up the Synology Server.  This opened the Synology Assistant, a setup wizard; and after a moment, it gave me an error message:

No Synology Server was found on the local network.  Please make sure:

1.  Synology Assistant is not blocked by the firewall of your computer OS or anti-virus applications.

2.  Synology Server and your computer are both connected to the network.

3.  You have switched on the power of Synology Server.
Option 1 was the only one that seemed to explain the situation.  I decided to back up and make sure that I could see a shared folder on the other computer from within Windows.  In my first try, I set up that shared folder on an NTFS partition, and that led to a separate investigation of the difficulties of sharing an NTFS partition in Ubuntu.

That wound up taking longer than expected, so in the meantime I just focused on the link between the Synology and the computer in which I had VMware running.  I noticed that, in Ubuntu's Places > Network, it listed three items:  ANTEC (the name of this computer), Windows Network, and WINXP8 (the name of the computer running in the WinXP VM).  Plainly, Ubuntu was seeing Windows.  Was Windows seeing Ubuntu?  Or did it need to?  A first answer was that, of course, you could go into Windows Explorer > Tools > Map Network Drive and (assuming you had VM > Settings > Options tab > Shared Folders set up) you could gain access to NTFS and ext3 partitions outside of the drive C that existed inside the virtual machine.  These drives would be visible in Windows Explorer > My Network Places > Entire Network > VMware Shared Folders.

I tried running the Synology setup wizard again.  It gave me the same error as before.  I did a search and found webpages describing how to use NAS freeware to use another computer as an NAS device.  This raised two thoughts.  First, possibly I could use some software other than Synology's CD to make contact with the NAS device.  Second, perhaps I should consider using another computer myself, in lieu of the Synology unit.  I decided to go ahead with the Synology project for now; I could return or sell the device if it really wasn't what I wanted.  I probably could have assembled another computer at equal or lower cost, with far greater potential storage capacity, with more RAID options, with a more powerful processor (for e.g., checksum calculations) if needed, with what might prove to be more options in the choice of software packages and commands to manage and adjust it, and with more flexible hardware troubleshooting options (i.e., more than just fix it or replace it) in the event of malfunction.  Its drawbacks would include time and expense for software and hardware selection, learning, installation, maintenance, and troubleshooting; physical space requirements; power consumption; and noise and heat generation.

For the time being, I searched Synology's website and found a post raising the thought that perhaps a Windows connection was crucial only for the initial setup of the Synology device.  So I rebooted the computer into Windows XP instead of Ubuntu and ran the Synology setup CD from there.  This time, the wizard found the DiskStation right away.  So, really, I probably could have set the thing up using my laptop.  It seemed to be just a matter of connecting a Windows-based computer to configure the hard drive that I had inserted into the NAS unit.

Following the Quick Installation Guide, I looked for a Browse option in the Synology Assistant, but didn't see one.  Instead, in the Management tab of the Assistant, I double-clicked on the DiskStation entry, and that seemed to be the correct thing to do:  it opened a different Setup Wizard, or maybe a continuation of the same one.  The wizard said, "Please input the path of installation file."  Maybe this was where I was supposed to browse to the .pat file?  Sure enough. Browse brought up four different .pat files.  I chose the one for the 109 and opted for One-Click Setup.  It warned me that all data in the hard drive would be deleted.  I hoped it meant the hard drive that I had inserted into the NAS unit.  Lights began flashing on the unit.  It went through several steps:  Apply network settings, Format hard drive, Install DSM (DiskStation Manager) to hard drive, and Write configurations.  For my 2GB drive, the whole process took about 20 minutes.

When it was done, it said, "System has been installed successfully."  Then it just sat there.  Now what?  The other programs on the CD's Installation Menu were Data Replicator, in case I wanted to use the unit for backup rather than as a file server, and Download Redirector, for some purpose I didn't fully understand.  For lack of any better ideas, I rebooted into Ubuntu > Places > Network.  The list of places was the same as before.  I tried another search of the Synology website.  The product page for the DS109 definitely said that the unit was "designed for data storage and sharing among Windows, Mac, and Linux."  But how?

I knew I was desperate when I thought that perhaps I should consult the User's Guide.  But then -- what's this?  When I went to the downloads website, I saw that Synology Assistant was also available for Linux!  I had no idea.  I downloaded that and, while I was at it, also snagged what appeared to be a more recent DSM patch (.pat) file.  The User's Guide on the CD was for DSM 2.3, but the one online was for DSM 3.0, so I copied that too.  Apparently DSM was the firmware updater.  The included instructions were incorrect, as I eventually figured out.  All I had to do was to navigate to the folder where I had put the downloaded .tar.gz file ("cd /LOCAL/Synology") and the accompanying file, type "," designate /usr/local as the target directory, watch a bunch of error messages roll by, accept its offer to try again by sudo, copy and paste the command it offered to create a symbolic link, and then type "SynologyAssistant."

With that, Synology Assistant was up and running, and it found the DiskStation.  I double-clicked on it.  It opened a webpage in Firefox.  Having used the One-Click installation previously, I knew there was no administrator password, so I just clicked right on in.  Now I was looking at Management and Online Resources icons.  Management gave me all kinds of options.  I noticed I was in DiskStation Manager 2.3; did this mean that there was no DSM 3.0 for Linux?  On the left side, under System, I clicked on DSM Update.  Ah, of course.  This was the part where I got to Browse to the new .pat file I had downloaded.  It said, "Transferring data to the server.  Please wait."  This time, it was done in under 10 minutes.  It then confronted me with a signin screen.  I could not just click on through; it demanded that I enter something.  I tried Administrator without a password.  No go.  I tried my normal Ubuntu login.  Aha! . . . er, no.  That wasn't it either.  The hell.  I was locked out of my own NAS.  I wasn't alone.  Several other people had experienced this just within the last few days.  I suspected it was due to some quirk in newly released software.  I posted a "me too" note on it in Synology's moderated forum and waited.

But then -- reverting again, desperately, to the manual -- I noticed I was supposed to log in as "admin" with no password.  That worked, and now I was in DiskStation Manager 3.0.  I clicked on "Set up a volume and create a shared folder."  That opened Storage Manager.  I selected Storage > Create and that put me in Volume Creation Wizard.  The only option that wasn't greyed out was ISCSI LUN.  The manual didn't define that term, but Wikipedia said it was short for Internet SCSI, where SCSI is short for Small Computer System Interface.  The idea seemed to be that you were using the Internet instead of cables to create a SCSI setup.  LUN was short for "logical unit number."  An ISCSI LUN was apparently just any one of a set of drives in a SCSI array.  In other words, I was creating a logical drive.  So I went with that.

That gave me a choice of some more properties.  One was Thin Provisioning (default = yes), which was said to increase efficiency.  I was supposed to say how much of my 2TB (actually, 1829GB available, according to the dialog) I wanted to allocate to this first volume (default name:  LUN-1).  I was going to be backing up this file server to a 2TB drive, so I didn't worry about splitting the volume to a size that would match the external drive.  I thought it might be a good idea to have more than one volume, in case one went bad or needed maintenance.  The manual said that, on my unit, I could have up to ten.  I looked at my data and decided to go with three volumes of 600GB each.  (This would be changing later.)  Finally, there was an iSCSI Target Mapping option.  Again, the manual didn't explain this.  I found a website that sort of halfway did.  Eventually I just decided to go with the default, which was no, thank you.  I clicked Next > Apply and, in a few seconds, it was done.  I repeated for the other volumes -- or, I guess, LUNs, not volumes.  Then I clicked on the icons this process had created.  Each indicated that it had a 600GB capacity, but none of them actually seemed to have taken a bite out of the 1.8TB total.  Apparently that was how Thin Provisioning worked.  Then, to finish up with Storage Manager, I went to the HDD Management tab > Cache Management > Enable Write Cache.  I also ran a quick S.M.A.R.T. test.

This was all very nice, but I wasn't sure what it was actually accomplishing.  There weren't any new partitions appearing in Nautilus.  I wasn't sure if there were supposed to be.  I bailed out of Storage Manager.  I was looking again at Quick Start.  It said that now I needed to create a shared folder in the Synology.  I followed its link.  It put me into Control Panel - Shared Folder.  I clicked on Create.  In Create New Shared Folder, I set up a folder for LUNDATA, the first of my three LUNs.  It wouldn't let me select "Mount automatically on startup."  I gave both admin and guest read/write privileges for now.  I did the same with the other two LUNs.  I was confused, though:  after completing that step, I still didn't have anything to show for it.

It seemed that Chapter 7 of the User's Guide was where I wanted to be.  It told me to go to Main Menu (i.e., the down-arrow icon) > Control Panel > Win/Mac/NFS if I wanted to enable file sharing.  But that gave me an error:  "You are not authorized to use this service."  So, oops, that meant I had gotten logged out for dillydallying.  (First of many times!)  After re-login, the Quick Start reminded me that next on the list was "Create a User and assign privileges."  It had admin as the system default user already.  I selected that one and clicked edit.  Spooky thing here:  admin did have a password.  I wasn't sure why I didn't have to enter it when logging in.  I wasn't allowed to change the name of admin or disable that account.  I decided to change the password to something that I would actually know.  Admin already had full read/write privileges to my three LUNs.  The guest account was disabled.  I left it that way.  The manual (p. 66) said that each user could also have his/her/its own "home" folder.  It was something I had to enable if I wanted it.  I didn't need it, so I skipped that.

So now I went back to Win/Mac/NFS.  The User's Guide (p. 59) said that the unit supported file sharing in Linux in SMB, FTP, NFS, and WebDAV.  I unclicked the boxes so that the Synology would not offer Windows or Mac file service, which I did not need (and did not intend to provide to anyone else).  Instead, I clicked the Enable NFS box which, the manual (p. 61) said, was for Linux clients.  I figured that, in my Windows XP virtual machine, I would access the folders or LUNs on the Synology as network drives, just as if they had been ext3 drives inside the computer.

The remaining tab in this part of Control Panel had to do with Domain/Workgroup.  I didn't know if I wanted or needed to have the Synology be part of a domain, a workgroup, or both.  But then I found that the Domain/Workgroup tab was greyed out.  As I might have assumed, "workgroup" and "domain" appeared to be Microsoft-specific.  If I went back and enabled Windows file service, the Domain/Workgroup tab became ungreyed.  So that explained that:  it wasn't something I needed in Ubuntu.

In the Control Panel > Groups section of the Synology DSM, I saw that the default "users" group had read/write privileges only to the public folder, which I had disabled.  It was just me, so I didn't need a group.  So I left that all as it was.  Next, in Control Panel > Application Privileges, it appeared I could give users access to specific Synology applications (FTP, WebDAV, File Station, Audio Station, Download Station, or Surveillance Station).  Admin wasn't listed.  I assumed it didn't need to be.  I had no other users, so I skipped that part too.

Chapter 3 in the User's Guide, "Modify System Settings," told me that in Control Panel > Network, I could choose among several types of networks.  In my version of the Network dialog, those options were LAN, PPPoE, Wireless Network, and Tunnel.  The choice for my purposes seemed to be between LAN and PPPoE.  The manual said that I should use PPPoE if I used a cable or DSL modem and if my ISP used PPPoE.  I didn't know how to check that.  It didn't sound familiar, so I decided to start with LAN, the default (first) tab.  It gave me an option of manual or automatic configuration; I chose automatic (which was, again, the default).  That seemed to be about all I could do there.  While I was in the neighborhood, I went to Control Panel > Time and set it to synchronize itself with an NTP server.  

Now it was time to set up shared folders (User's Guide, p. 69).  In Control Panel > Shared Folder, I saw the three LUNs I had set up.  So apparently a LUN was a shared folder.  I had already taken care of this.  But that raised some questions.  If it was shared, what more did I need to do so that the computer would see it?  Should I have set up a "target" when I was creating the LUNs?  And did I want to encrypt them?

If I clicked on the Encrypt box, the "Mount automatically on startup" option became ungrayed.  I would want to enable that option.  But I had to think about that for a minute.  It seemed that encryption would protect the contents of the Synology in case of theft or loss of the physical device.  But apparently it would not protect those contents while the computer was turned on.  Anyone who could get into my computer, either physically or via the Internet, would have access to those contents.  I wasn't presently requiring myself to enter a login ID when I turned on the computer, so anyone sitting in my position would still have access, despite encryption.  I hadn't yet reviewed the part of the manual having to do with Internet access to the Synology, but evidently I would also have the option of logging in to it from elsewhere.  On the other hand, I had once had the experience of not being able to get into a backup that I had encrypted.  I wasn't sure if I had mis-recorded the password or if the encryption system on that backup had somehow gotten corrupted.  On balance, I decided that it would probably be a good idea to password the Internet-accessible data on the Synology, and to start requiring myself to enter a password to log in on the computer (System > Administration > Users and Groups).  But then, when I entered the password for the Synology and clicked OK, I got a warning telling me, "The performance of the encrypted shared folder will be decreased" and "The encrypted shared folder will not be available via NFS."  That would have defeated the purpose of having the Synology.  So I backed out of that.  No hard drive encryption in the Synology.

Well, the Synology was still not showing up in Nautilus.  I searched the manual for "target," in case that was the missing ingredient.  The User's Guide (p. 41) explained, "An iSCSI Target is like a connection interface . . . . [A]ll the LUNs mapped to the iSCSI Target are virtually attached to the client's operation [sic] system."  So apparently I would map my three LUNs to a target, and Ubuntu would see the target.  As the manual advised, I went into Synology's Storage Manager > iSCSI Target > Create.  There was an option to enable CHAP authentication, where the server would verify the client's identity.  I went with that.  I didn't go further and enable two-way authentication; I didn't need the computer to verify that it was contacting the right NAS unit.  I mapped all three LUNs to a single target.

In Edit > Advanced, I had an option to have it calculate CRC checksums for header and data digests.  The purpose would be to reduce or prevent data corruption.  The calculation would burden the CPU in the NAS, but I suspected the cabling would be more of a bottleneck than the processor nonetheless.  One post said that CRC might be a good idea for data traveling through a router, as would be the case here.  A year-old VMware webpage pertaining to a different VMware product (ESX) said that data digest for iSCSI was not supported on Windows VMs.  I decided to start out with these checksum items turned on, and see what the performance was like.  I also had options pertaining to maximum receive and send segment bytes.  The manual didn't seem to have anything on that, and nothing popped out in several different Google searches.  I decided to leave those at their default values of 262144 and 4096, respectively.

I still didn't see the Synology in Nautilus, but now (as I viewed p. 72 of the manual) I believed that was probably because I had not enabled my own username (ray) to have access.  In Synology's Control Panel > User, I added that username and gave myself full read/write access to the LUNs.  But then, whoa, on the next page, the User's Guide said that, to allow a Linux client to access a shared folder, I would have to go into Control Panel > Shared Folders > select the folder > NFS Privileges > Create and set up an NFS rule.  The first box there called for Hostname or IP.  It looked like the best way to identify the client would be by its IP address.  What was the IP address of my Ubuntu computer?  Zetsumei said I should type "/sbin/ifconfig" in Terminal.  I did that and got a bunch of information regarding eth0, lo, vmnet1, and vmnet8.  Same thing if I just typed "ifconfig -a."  A search didn't shed any light.  The number for eth0 came first and looked most familiar, so I tried that, with no mapping and asynchronous enabled.  This still didn't produce anything in Nautilus, so I thought probably I should have mapped.  But to what?  The only options were "Map to admin" or "Map to guest."  How about "Map to ray"?

A search of the Synology website led to a thread that yielded more questions than answers.  For the first time, the thought crossed my mind that the quality of the Synology organization was possibly not as gold-plated as I had hoped or imagined.  Surely the manual could have been clearer; surely, at these prices, the people posting these questions deserved some enlightenment.  At any rate, links in that thread led to one of those multiyear Ubuntu discussions, this one dealing particularly with NFS.  It seemed I should focus on learning about NFS; among other things, some posters felt that it was far better than Samba for sharing files and folders.

So I did a search and found a recent webpage promising to show me how to set up NFS.  I guessed that the real problem might be on the client side, so I started with that part of the webpage.  First off, they wanted me to install some packages:  portmap, nfs-common, and autofs.  A check of Synaptic told me that Synology had not installed these.  After installing them, I looked in the manual for the Synology IP address.  On page 161 (after many references to the IP address), the manual said that I could find it in Main Menu > System Information -- not, that is, in Control Panel.  The IP address it gave was, however, the same as the default entry it showed in Control Panel > Network > Use manual configuration; it was not the number shown in the DNS Server box.  So in the client, following the instructions on that webpage about NFS, I typed "sudo gedit /etc/hosts.deny" and added a line that said "portmap : ALL."  Then I typed "sudo gedit /etc/hosts.allow" and added a line that said "portmap : [Synology IP address]," using the address I had just found in Main Menu > System Information.  Next, I typed "sudo gedit /etc/hosts" and added a line near the top that said "[Synology IP address] [Synology Server Name]," in the same format as the other lines there.  (The server name was shown in Main Menu > System Information.)

Continuing with the NFS webpage's instructions, I was supposed to type something along the lines of "sudo mount [Synology Folder] [Local Folder]."  For that purpose, I understood that Synology Folder = [Synology IP address]:[Synology Shared Folder].  But I was not sure what the Shared Folder part was supposed to be.  Was I supposed to refer to the LUN or the iSCSI Target on the Synology unit?  Since the User's Guide (p. 41) said that an iSCSI Target was "like a connection interface," and that all the LUNs attached to it would be attached to the operating system, it seemed that I would need only one target, as I had set it up.  But now that I had learned more about security on the Synology, I had changed my mind about the number of shared folders I wanted.  I just wanted two, each 900GB in size:  one to contain stuff that shouldn't be changing very often, and that only the administrator should have write privileges for, and one for everything else, i.e., for the stuff that I would want to be able to mess with on a daily basis.  So after changing the LUNs and target in Storage Manager, I guessed that I would be creating two folders using the pattern of "/home/[username]/[foldername]" (where "username" would be "ray" in my case) -- one for each of the two LUNs on the Synology.  One of them was called SYNDATA.  On that basis, I typed "sudo mount [Synology IP address]:[Synology 900GB folder name] /home/ray/SYNDATA."  This gave me "access denied by server while mounting [Synology Folder]."  Not a desirable answer, but at least it was a reply of some kind!

By now, I was completely confused, and more than a little irritated at how very long this was taking.  The NAS was supposed to be simplifying my situation, not making it more complex.

It did seem, at this point, that it might have been easier to troubleshoot this if I had been using a computer as my NAS:  I could have gone into it and typed various commands to maybe get a bit more insight on what was happening in there.  A search for that error message led to the suggestion that I type "/usr/sbin/rpcinfo -p" to see what ports the server was using, but that gave me a "No such file or directory" error.

I decided to put in a support request at Synology.  The form required me to enter the Firmware Version -- but, of course, this was not provided in the System Information dialog.  I just entered something that seemed approximately right.  It also asked for the serial number -- and that, they helpfully indicated, was located on the bottom or perhaps the rear of the unit.  After turning it around and risking unplugging it, doing gymnastics to hold it while typing, I realized that, well, they might have mentioned that that bit of information actually *was* in the System Information dialog.  But when I got down to the part where they were ready and listening to what I had to say, I was not sure what to type.  There wasn't an option of talking to (or even chatting with) a live person.  I had to type something.  But what?  How could I possibly explain all this in a few words?

What I needed, somewhere in the Synology software, was a tool that would tell me what was happening.  "You have connected to a computer" or "You have not connected to a computer," etc.  I wasn't sure -- I hadn't done much networking before -- but I suspected that I could get that kind of information by using regular Linux commands on a computer in a network.

I decided that what I would tell the Synology people was just that they should look at this post.  I had identified a number of areas they could improve; and if they really got on the stick, they might even be able to respond in time to help me, before I returned the unit to the vendor or resold it.  The unit had more than a dozen positive remarks from other purchasers at Newegg, so I was hopeful.  But meanwhile, I started a post on the alternative of using a separate computer to create my own NAS.