Pankaj Kumar's Weblog

lmbench results for Amazon EC2

I recently downloaded and ran lmbench-2.5 on an Amazon Elastic Computing Cloud (EC2) instance. As you might know, lmbench is a fairly old microbenchmark for low level OS functions such as context switch, process creation, networking operations and so on. Amazon EC2 is an innovative service offering from the famous online retailer that allows you to run an OS within a XEN virtualization environment, guaranteeing the equivalent of a system with a 1.7Ghz x86 processor, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth.

Based on the the above specification, I had expected its benchmark results to be in line with the ones reported in xen-devel mailing list post. However, lmbench results for Amazon EC2 I recorded were very different. You could study the two results yourself and draw your own conclusions. Or just look at my summary of the similarities and differences:

1. Results reported in xen-devel mailing list used a dual processor 2.4GHz Xeon box with SMP Xen. It used lmbench-3.0 alpha with Linux 2.6.8.1/xen kernel on Xen version 1.1301. The underlying h/w for Amazon EC2 is not known, but a listing of /proc/cpuinfo shows it to be 2405.450 MHz AMD Opteron(tm) Processor 250. I used lmbench-2.5 for Amazon EC2. Both machines seem to have same clock speed. Other differences, including presence of 2 CPUs and differing versions of Linux kernel, lmbench and Xen, may or may not be material to the lmbench benchmark numbers.


2. xen-devel mailing list post reports TCP connect time to be 53 micro secs, the same metric for Amazon EC2 is 345 micro sec. Amazon EC2 numbers are 3 to 6 times slower for most other operations as well.
   

How can this be explained? I would offer multiple possibilities:

1. The underlying hardware reserved by Amazon EC2 is much less powerful than a 2.4 Xeon CPU (Assuming that lmbench benchamrks utilize only one CPU, even when 2 are present). Unlikely, IMHO.
2. Amazon EC2 has not optimized Xen for its hardware. Most likely this is the case.
3. Version differences are main culprit. Unlikely.

What would you say?

Note: I have comments disabled due to heavy comment spam. If you do have something interesting to say then send me an email at pankaj.kumar@gmail.com and I will post your message.