VMware Server Disk IO Performance

By
Lode
Published: October 24, 2008

After adding some guests to our VMware Server installation (more information about the installation of VMware Server in Ubuntu) I noticed it becoming realy slow. Even when the guests weren’t realy doing something, it was slow.I found that rather strange since the machine running VMware Server isn’t running anything else and has the following configuration :

  • Intel(R) Core(TM)2 Quad  Q9300  @ 2.50GHz
  • 8GB ram
  • 500GB SATA disk

Not that that’s a real monster, but it should do the trick.

 

The Problem 

In my search for a solution I had to look for a possible cause first. So I tried looking at some stats. Top showed a constant high number for CPU IO Wait. This wasn’t normal since the guests weren’t doing anything, nor was the host. The effect can be seen on this Munin graph: CPU Usage on FulliautomatixNo real activity, only IO wait. 

The Cause 

After many many Google searches, I finally found a forum thread called The struggle for disk IO scalability in Server 1.x and RC2. The guy (since his nick is davidyon, I suppose it’s a guy) complains about roughly the same symptoms and lists a truckload of things he tried to solve the problem. None of them realy made a great difference, except the one in the end of his post. Starting with “Finally, FINALLY, I hit the magic bullet” I knew he found wat I was looking for, the cause :

The problem seems to lie in the memory-mapped file that VMWare Server insists on creating and keeping up to date. The pauses were a result of iowaits while the disk backed memory file (.vmem files) was updated.   

The Solution

To put the .vmem file in /tmp instead of the directory where your guest lives, change your .vmx file to state: 

mainMem.useNamedFile = "FALSE"

Now you can optimize the IO to that directory. And the fastest way to read and write from it is getting it into memory. You can do this with tmpfs. Add the following into /etc/fstab: 
tmpfs    /tmp    tmpfs    rw,size=12G    0    0

And then mount it with: 

sudo mount /tmp 

The Result

The result is stunning! IO Wait number in top dropped to almost 0.

I also compared disk read performance before and after the change with hdparm. The numbers before the change :
  • Host : 138 MB in  3.01 seconds =  45.83 MB/sec
  • Guest : 18 MB in  3.35 seconds =  5.38 MB/sec

And after the change :

  • Host : 260 MB in  3.00 seconds =  86.62 MB/sec
  • Guest : 182 MB in  3.02 seconds =  60.31 MB/sec

About the Author

Lode Blomme works as a Software Engineer at RouteYou, a Belgian startup company active in digital outdoor navigation.