Sound Device Problem in VMware Workstation 7

I was running Windows XP as a guest in a virtual machine (VM) in VMware Workstation 7 on an Ubuntu 9.10 (Karmic Koala) host.  When I used Workstation's VM > Removable Devices > Sound Card > Connect option, I got this error message:

Failed to open sound device /dev/audio:  Device or resource busy
Failed to connect virtual device sound.

I had also gotten this message:

Failed to open sound device /dev/dsp: Device or resource busy
Sound will not be available

I had been having this problem ever since I started using VMware.  I had previously changed the sound device (using VM > Settings > Hardware tab > Sound Card > Use physical sound card) from /dev/dsp to /dev/audio, and had also changed the sound card so that it would not connect at power on.  These and other efforts (e.g., rebooting) had fixed things temporarily, but then stopped working.

Some time back, I had found that I got this problem when Second Life was running in Ubuntu (that is, not in the WinXP guest).  More recently, I had had the same problem when Firefox was running in Ubuntu.  What I worked out that time was as follows:

For the time being, the solution seems to be either (a) to watch videos and other webpages that use Flash, do it in a browser session that is running inside your VM, not in a browser running in Ubuntu, or (b) after watching a video or otherwise using Flash in Ubuntu, kill the program that used it (e.g., Firefox) and manually reconnect with your sound card inside the VM.

So now I powered down my VM (i.e., not just suspended) and went into Workstation's VM > Settings > Sound Card option.  There, I put my settings at "Connect at power on" and Use physical sound card:  Auto detect.  I saved that and killed all browsers in Ubuntu, including Firefox and also Google Chrome.  I then powered on the VM.  WinXP had a hard time booting itself -- it always seemed to reboot a couple of times before it finally got itself together -- so while I was waiting, I started another session of Workstation and, in that one, I set the sound card to "Use physical sound card:  ALSA: Default sound card," and powered up that VM as well.  That one booted.  The one set to Auto Detect seemed frozen, so I changed it to "Use physical sound card:  OSS:  /dev/dsp" and tried again.  It gave me a notice that VMware Tools was not installed (it had been installed previously), but it booted, and Tools did appear to be running OK.

At this point, sound was working OK in the one that used ALSA, but not in the one using OSS.  That is, when I tried the virtual machine that was using VM > Removable Devices > Sound Card > Connect in the OSS, I got an error message.  I changed it to ALSA, like the other one, and then the sound worked in both.  The solution reached previously, of killing competing programs in Ubuntu, seemed to be the answer again.

Later, though -- after rebooting, I think, and anyway after running Firefox in Ubuntu again -- I found the audio was stuttering.  It stuttered at different speeds if I used ALSA or OSS, but either way it stuttered.

I played a sound file in the VM, and while it was stuttering along, I went to Ubuntu's System > Preferences > Sound > Applications tab.  There, I saw "ALSA plug-in [vmware-vmx]" listed, and underneath it I saw "No application is currently playing or recording audio."  That message was flickering on and off with each stutter in the sound.  A search along these lines led to a thread in which someone suggested reinstalling the ALSA packages.  In System > Administration > Synaptic Package Manager I did a quick search for ALSA.  I saw that I had a number of ALSA-related programs installed.  I started to mark one of them, alsa-base, for removal, but when I did so, Synaptic indicated that it would also need to remove a package called ubuntu-desktop.  That sounded pretty major, so I bailed out of the reinstall-ALSA approach.

I did a different search and came up with a thread in which someone said that VMware Workstation 7 supports ALSA perfectly, and if you are getting stuttering it may be because of a setting in the WinXP guest, not in Ubuntu or Workstation.  Following the advice there, I went into XP's Control Panel > Sounds and Audio Devices > Volume tab > Speaker settings > Advanced > Performance tab.  There, as advised, I set both the Hardware Acceleration and the Sample Rate Conversion Quality sliders to the lowest possible settings.  I clicked Apply and OK and then rebooted the XP guest.  With ALSA as the chosen sound card setting, and with the sound card connected in VM > Removable Devices > Sound Card, I found on reboot that I had no sound at all.  I tried changing from ALSA to OSS and auto detect and back to ALSA.  Now I could connect the sound card, and now I had sound.  But I still had stuttering.  I tried typing “/dev/audio” into the box in place of ALSA, and (after reconnecting the sound card) that sped up the stuttering but did not eliminate it.

I wondered if the stuttering was due to a larger problem with the VM.  I wondered this because its performance was generally slow at this point, and had been slow before I started this latest round of tinkering.  (In other words, I hadn’t caused that slowness just by playing with the sound problem.)  I suspended the VM and shut down the system.  Next day, I started the machine and tried again.  There was still a bit of stuttering, but the situation was much better.  (See also this other writeup on audio problems in Ubuntu and VMware.)