How to benchmark a VM?

[Peroxyde]

Hi,

After our company has consolidated a group of physical servers into VM (using VMWare ESX). I have noticed some changes, some servers are slower than the physical server it replaced. The opposites is also true.

Recently we have had some performance issue on a critical SQL Server. We would like to rule out the hardware factors. My idea is to have some benchmark values to compare a VM against a baseline machine. Our servers are mostly Web Servers and SQL Servers running on Windows 2003. I am interested on performance measure related to CPU, Memory and Disk I/O.

Is it an acceptable approach to evaluate the performance of a VM? If so, can you please suggest a way to carry out these benchmarks?

Thanks in advance for any help.

 

[mpilchfamily]

Bench it like you would any PC. Run the various benchmarks in each VM.

 

[Peroxyde]

Bench it like you would any PC. Run the various benchmarks in each VM.

Can you please suggest which software to use to run these benchmarks?

 

[mooseracing]

HDTune, HD Tach, Microsoft has bench/sim tools as well. Such as SQL I/O, they also have and exchange load generator.

The biggest thing for me was to make small OS IDE drives and all data activity for SQL/Exchange and whatnot to be on SCSI fixed size drives, in Hyper V that is.

Depending on the I/O you may need direct access drives

 

[mpilchfamily]

Try PC Mark.
http://www.futuremark.com/products/pcmark05/

Also some of the number crunchers like:
Super Pi
http://files.extremeoverclocking.com/file.php?f=36
wPrime
http://www.wprime.net/about.html

 

[Syran]

Just out of question, what hardware are you running your VMs on?

 

[lxskllr]

Try PC Mark.
http://www.futuremark.com/products/pcmark05/

I had issues with PCMark when benching VMs. I used GeekBench, wprime, CrystalMark, and SysSoftSandra successfully.

Edit:
Everest also works

 

[Peroxyde]

Just out of question, what hardware are you running your VMs on?

I don't own the hardware. For now I can read through Ctrl Panel / System shows "Xeon E5420 @ 2.50 Ghz, 3.75 GB RAM", running Win 2003 R2 Enterprise x32. Task Manager shows 2 CPUs.

I suppose that the VM relays correctly the hardware specs of the physical host running VM Ware ESX.

Thank you for all the advices. Seems like there is anough for some investigations. I rarely see benchmarks running against VM. I wonder if there is any special precaution or even if the idea is applicable.

 

[Syran]

I don't own the hardware. For now I can read through Ctrl Panel / System shows "Xeon E5420 @ 2.50 Ghz, 3.75 GB RAM", running Win 2003 R2 Enterprise x32. Task Manager shows 2 CPUs.

I suppose that the VM relays correctly the hardware specs of the physical host running VM Ware ESX.

Thank you for all the advices. Seems like there is anough for some investigations. I rarely see benchmarks running against VM. I wonder if there is any special precaution or even if the idea is applicable.

Is this a hosted environment of some sort?

Just wondering, as the compression ratio's (# of VM / Physical Host) as well as types of disks used, arrays, fiber, etc, can all factor into that.

 

[Chiropteran]

Bench it like you would any PC. Run the various benchmarks in each VM.

That really isn't the correct way to do it. The issue is that VM performance can be impacted by usage of the shared resources by all the other VMs on the same host. If you bench when the other hosted VMs are "busy" you will get lower numbers than you would get if you do so when other hosted VMs are idle.

After giving the issue some thought, I am thinking the best way to benchmark a worst case scenario is to run the benchmark program simultaneously on every VM that is running on the host.

I don't own the hardware. For now I can read through Ctrl Panel / System shows "Xeon E5420 @ 2.50 Ghz, 3.75 GB RAM", running Win 2003 R2 Enterprise x32. Task Manager shows 2 CPUs.

Typically it will show the correct CPU type and speed, but the number of cores and RAM is determined by the settings used when creating the VM. For example the host of the VM you listed is using a quad core CPU, but the guest OS only sees 2 cores because that is what ESXi was configured to give it. It could even be a dual-quad (8) core server.

In my experience hard drive speeds can be the biggest limiting factor on VM performance, memory and CPU is cheap in comparison but most servers just don't have enough drives to give responsive access to all the VMs you are running at the same time.