Windows IT Pro is the authoritative and independent resource for windows nt, windows 2000, windows 2003, windows xp. Features a collection of resources and magazines for windows IT professionals.
  
  
  Advanced Search 


May 27, 2008

Virtualization Shootout, Part 1

Microsoft Hyper-V vs. VMware ESX Server 3.5
RSS
Subscribe to Windows IT Pro | See More Products / Software Articles Here | Reprints | Or get the Monthly Online Pass—only $5.95 a month!
SideBar    Commonsense Virtualization

Executive Summary:

With the release of Windows Server 2008 and its all-new Hyper-V virtualization support, Microsoft has finally mounted a serious challenge to VMware’s mature and robust ESX Server, the established leader in the enterprise virtualization market.

With the release of Windows Server 2008 and its all-new Hyper-V virtualization support, Microsoft has finally mounted a serious challenge to VMware’s mature and robust ESX Server, the established leader in the enterprise virtualization market. In this twopart article I’ll compare Microsoft’s Hyper-V to VMware’s ESX Server 3.5. In part 1 of this two-part comparative review, I’ll compare the different architecture and feature sets of each product. In part 2, I’ll do some basic performance testing of the two products to see if Hyper-V can deliver comparable performance to ESX Server.

A Tale of Two Architectures
Both VMware’s ESX Server 3.5 and Microsoft’s Hyper-V are built using a hypervisorbased architecture. This architecture gives both platforms bare-metal performance that significantly outperforms older hosted virtualization products such as Microsoft Virtual Server 2005 and VMware’s Virtual Server 2.0. Hosted virtualization products run the virtualization software on top of the host OS, which introduces additional overhead and a longer code execution path for the virtual machines (VMs) that run in the hosted virtualization environment. In contrast, hypervisor-based products such as ESX Server and Hyper-V are designed to run the hypervisor directly on the system hardware. Although ESX Server and Hyper-V both share a similar hypervisor-based architecture, there are significant differences in the way the products are designed, as you can see in Figure 1.

In both cases, the hypervisor runs directly on the system hardware. However, with ESX Server the hardware drivers are all part of the hypervisor, which significantly increases the size of the hypervisor. In addition, the device drivers are created by the hardware vendors, which introduces third-party code into the hypervisor and limits the hardware that ESX Server supports. Even so, ESX Server is supported on most of the server systems made by all the tier-one vendors, such as HP, Dell, and IBM. Many of these vendors also sell systems configurations with VMware ESX Server preloaded.

In contrast, Hyper-V uses a microkernel hypervisor in which the hypervisor contains the minimal amount of code required to schedule and share hardware resources between the active VMs. The Hyper-V hypervisor has no device drivers and no third-party code, which ensures the best possible performance and reduces security exposure. Hyper-V leverages the native Windows device driver model, utilizing the device drivers in the guest VMs. For more information about the Hyper-V architecture, see “A First Look at Windows Server 2008 Hyper-V” (February 2008, InstantDoc ID 97857).

Both products are managed from the first VM partition. In ESX Server this VM partition, typically called the service console, is based on the Linux shell and is managed via the command line. However, you can download an easier-to-use Windowsbased management client, called the Virtual Infrastructure Client, from ESX Server’s Web console. Hyper-V is also managed using the VM running in the first partition. In Hyper-V this partition is called the parent partition. In addition to VM management, the parent partition is also used to run VMs with legacy OSs such as Windows NT and Windows 2000 that can’t utilize Hyper-V’s new VMBus architecture and must use the older emulated hardware model.

Pound for Pound
Unlike the earlier version of Virtual Server 2005 R2, Hyper-V’s new architecture and 64-bit foundation bring its feature set into parity with the features that are present in VMware’s ESX Server. Table 1 shows a feature-by-feature comparison of VMware’s ESX Server 3.5 and Microsoft’s Server 2008 Hyper-V.

The primary differences begin with the hypervisor itself. As I explained previously, the ESX Server hypervisor is a heavyweight hypervisor that contains device drivers. In contrast, the Hyper-V hypervisor is a thin hypervisor that contains no drivers and no thirdparty code. Hyper-V’s device drivers are in the guest OSs, which makes the Hyper-V hypervisor smaller and more secure. Both platforms provide support for 32-bit x86 and 64-bit x64 guest OSs and large VMs with up to 64GB of RAM per VM. For more efficient memory usage, ESX Server provides a shared memory feature that lets VMs share common memory blocks. Although this feature can enable more simultaneously active VMs, it generates additional performance overhead. Hyper-V doesn’t support shared memory between VMs. Both platforms support booting VMs from either an iSCSI or Fibre Channel SAN. One area where VMware excels is in support for live migration (i.e., moving running VMs from one host to another). However, this feature requires the VMware Virtual- Center Server product. Hyper-V doesn’t support live migration, but when coupled with Windows Server 2008 Enterprise Edition and Microsoft System Center Virtual Machine Manager, it does provide support for what Microsoft calls quick migration— quickly saving the state of a running VM and then moving that VM and saved state to another host. Quick migration requires the use of failover clustering. ESX Server is limited to 128 active VMs (probably enough for anyone), whereas Hyper-V is limited only by the available system resources. Unlike the desktop virtualization products, neither product provides support for guest audio or USB. ESX Server supports guest VM backup using the integrated Consolidated Backup feature, which takes a snapshot image of the VM and writes it to a backup server. Hyper-V supports live backup of VMs using Volume Shadow Copy Service (VSS).

Are You Experienced?
Setting up both systems was relatively easy. The basic setup for ESX Server was actually easier than the Hyper-V installation. Although the ESX Server installation was character based, the screens were easy to follow and I had a completely functional server in about 20 minutes.

For Hyper-V the actual installation process was easy but the subsequent system setup on Windows Server Core was a manual piecemeal process that required a good deal of Windows command-line knowledge to complete. The Hyper-V virtualization role can be installed on either a full Server 2008 installation or on a minimal Server Core installation. Server Core is the better choice for a virtualization server host because it has all the extraneous Windows components stripped out (e.g., the graphical shell, Internet Explorer—IE, Outlook). This bare-metal approach gives Server Core less overhead and makes it more efficient. Server Core is also more secure because of the reduced attack surface area, as well as more reliable because of the smaller number of components that might need patching. Installing the Server Core OS took about 15 minutes; running the subsequent system configuration commands took about 20 more minutes and a couple of reboots. For more information about the commands to configure a Server Core system and add the Hyper-V virtualization role, see Top 10, “Essential Server Core Setup Commands.”

To manage the system, I needed to attach to it remotely using the Hyper-V management console, which Figure 2 shows. You can run this console from a Server 2008 system with Hyper-V installed or from a Windows Vista system with the update installed that’s discussed in the Microsoft article “Availability of the Windows Vista Service Pack 1 management tools for the Hyper-V release candidate” (support.microsoft.com/?kbid=949758). Any Windows administrator will feel right at home with this Microsoft Management Console (MMC) 3.0–based interface.

You can manage multiple Hyper-V server instances in the console’s left pane. Selecting a server instance displays that server’s VMs in the center Virtual Machines pane. You can then manage the VMs by right-clicking them and selecting options from the context menu. All Hyper-V management tasks can be performed using the Hyper-V Management Console.

Managing ESX Server is another story. ESX Server uses a Linux-based command shell—this command line might be comfortable to a Linux administrator, but I’m not one. (In fact, I rarely need to deal with Linux.) Fortunately, the VMware Virtual Infrastructure Client is a graphical tool that you can download by pointing your browser to the server’s URL. Figure 3 shows the VMware Virtual Infrastructure Client.

The VMware Virtual Infrastructure Client lets you create and manage VMs. However, it doesn’t let you perform server management functions such as adding and removing network cards. You need to use the command line to perform those types of functions.

Using built-in management consoles to manage a few servers is just one aspect of virtual server management. But if you have more than just a few virtual server host platforms to manage, you’ll need more powerful management tools. Both VMware and Microsoft offer such tools. To find out more about the virtualization management tools offered by each company, see “Virtualization Management.”

Continue to page 2

   Previous  [1]  2  Next 


Reader Comments
II registered specifically to comment on this, I must say it was an interesting read. Comments such as "The VMware Virtual Infrastructure Client [...] doesn’t let you perform server management functions such as adding and removing network cards." are just _wrong_, there's no other way of putting it.

I'm also not entirely sure of your driver arguments - as I understand from looking at the ESX boot procedure, it only loads the ones it needs, and you're also relying on thirdparty drivers for hyper-v. In fact, I would agree with the infoworld post (http://weblog.infoworld.com/enterprisedesktop/archives/2008/05/hypervs_achille.html), the drivers are just not up to snuff for me to trust running multiple servers in a production environment on it.

jutasuk May 29, 2008 (Article Rating: )


Yo brother, if you ain't familiar with the cli either nix or Mickeysoft how do you survive in the real world ? All S&P 500 enterprizes have nix and MS on board. But I hope you will have a good look at DRS and HA along with storage virtualisation which are available with VirtualCenter, I did not hear you mention this sofar.

rohad May 29, 2008 (Article Rating: )


> The VMware Virtual Infrastructure Client lets you create and manage VMs. However, it doesn’t let you perform server management functions such as adding and removing network cards. You need to use the command line to perform those types of functions.
The above statement and several others in the article are false. One can manage ESX without knowing any Linux. The article insinuates that one can manage (i.e. create VM, configure the host, etc) from the console using a command line. That is not true and really casts doubt on the article an “unbiased” comparison. The VI client can manage the host fully including adding hardware. The install process does have a GUI version and with ESXi you essentially don’t have a service console anymore.
> The Hyper-V hypervisor has no device drivers
This of course is a complex topic but in this article it is written to insinuate that Hyper-V will be a much more stable product. That remains to be seen, but it is misleading to suggest that it will be based on what is still release candidate software. The hypervisor level for ESX (the vmkernel) is a few MB in size – about 200K lines of code. The Hyper-V hypervisor is smaller (600 kb), but it has a significant dependence on the parent partition (as well as for device drivers). If you lose the parent partition with Hyper-V you lose your VM (see 1st link). With ESX if the service console goes down, your VMs continue to run. You might not be able to manage your host, but at least the VMs run until you can bring the host down at a more convenient time. Which is the better architecture remains to be seen, but to say Hyper-V is better today is poor journalism.
http://blogs.technet.com/mattmcspirit/archive/2008/05/16/getting-my-head-around-the-hyper-v-architecture.aspx
http://it20.info/blogs/main/archive/2007/06/17/25.aspx

Dave Mishchenko May 29, 2008 (Article Rating: )


continuing from above...
>> which introduces third-party code into the hypervisor and limits the hardware that ESX Server supports. <<

Various statements in the article suggest that ESX will be less stable due to 3rd party drivers. It’s somewhat of a misconception that one can add any Linux driver to ESX to have it support a given piece of hardware. This is not the case. Driver are created in co-operation with VMware and based on past experience, I have no concerns about drivers that have been developed for ESX. All software, including drivers, will have potential bugs, but this is the case with drivers for both ESX and Hyper-V and in both cases VMware and Microsoft are taking the steps necessary to ensure that only quality drivers are provided.
A few other shorter points on this http://windowsitpro.com/files/01/98879/table_01.gif
- ESX is 32 bit / Hyper-V is 64 bit therefore it is better. Your performance testing will show that is not the case.
- “Hyper-V can run an unlimited number of active VMs” - but practically how many will a host be able to run. The 128 VM limit for ESX in a practical limit.
- Hyper-V host memory supported is 2TB – ESX is actually 256 GB and that’s the tested figure.

Dave Mishchenko May 29, 2008 (Article Rating: )


And lastly ... :)

Pricing – a) if one is going to include 3 years of 24 x 7 support on the VMware side, then one should also include Software Assurance on the Microsoft side as well as support cost. It would be more accurate to have the cost of ESX Foundation at $1545 and that would include one year’s worth of upgrades and support.
b) It would have been better to have used Datacenter processor licensing in this case. In both cases, 2 DC processor licenses would be roughly $6000 lowering the overall cost of both systems.
c) Pricing comparisons such as this are disappointing. The cost of a system is much more than just adding up the licensing components. Plus in the situation given, ESXi would be a fairer product to use with a price of only $500 (actually $99 from Dell). But in reality, one must add up the cost of hardware, training, VM density, etc to really do a fair cost comparison.

Dave Mishchenko May 29, 2008 (Article Rating: )


Another thing that is not pretty correct is the data in Figure 1.

Support for 32 socket in ESX and 64 socket in Hyper-V is wrong.

ESX supports 32 logical processors that in actual tecnology would mean 32 cores (example: 8-socket quad-core). Hyper-V only supports 16 logical processor or simply put, with the latest technology, 4-socket quad-core.

When Intel is coming out with their new 6-core cpu's (sooner than later) this will make 24 cores for a standard 4-socket systems .... Hyper-V won't likely be able to handle that.

I wanted to mention this because this is usually a big source of confusion.

Massimo.

MassimoReFerre May 30, 2008 (Article Rating: )


Just dont understand the many comments in the article, such as the one I've copied in below, that refer to ESX as the "heavyweight" hypervisor - inferring that Hyper-V is a smaller footprint, with therefore a smaller attack surface. That is rediculous - has the author ever heard of ESX Server 3i - or whatever VMware is calling it these days?

Our testing shows ESX Server 3i - is a fully functional hypervisor in a 32 mg footprint. Our current evaluations of the pre-release of Hyper-V, which requires at least Windows Server Core to function, show that it has at minimum a 3.9GB footprint. That's a full 120X larger than ESX 3i. Even ESX Server 3.5 with the service console is arund 2GB - half the size of Hyper-V. With obvious faults like this inthe aurthors analysis, it makes me wonder how much influence MSFT had in this article - I can't view the rest of the analysis as unbiased - the article even uses MSFT graphics.



"the ESX Server hypervisor is a heavyweight hypervisor that contains device drivers. In contrast, the Hyper-V hypervisor is a thin hypervisor that contains no drivers and no thirdparty code."

BretStephens May 30, 2008 (Article Rating: )


Here are some major corrections for your article. It will be in multiple parts:

Note: I do work for VMware but the items I'm listing are public fact and just corrections to the author's research.

1) The "real" drivers for the Hyper-V solution are housed in the parent partition. They use regular Windows drivers developed by MS and 3rd parties like the OEMs and others. The VMs have generic drivers that must push traffic through the real drivers in the parent partition. Both VMware and MS use 3rd party drivers. The big disadvantage of the Hyper-V solution is scaling of I/O since all I/O must go through the parent. MS delayed the release of Hyper-V previously because of "scalability of I/O" issues. It's also been well blogged and documented on the web. A little google search will turn it up in a flash. What you stated in the article is completely inaccurate about the drivers and shows a complete lack of knowledge of the architecture. You also might want to read Randall Kennedy's article about 3rd party drivers in Hyper-V and the security risk there: http://weblog.infoworld.com/enterprisedesktop/archives/2008/05/hypervs_achille.html.

mikedipetrillo June 02, 2008 (Article Rating: )


2) Reliance on the Linux CLI:

Very few people out there use the CLI in our Linux Service Console. Furthermore this has been removed in the ESXi product line. Everything in the product is and can be done from the Virtual Center Client. The client is free with every version of the product so there's no extra cost for using it. In addition, you were incorrect about needing the CLI in order to add network or storage devices. Both of those are in the GUI and have been since VMware ESX 1.5 (2002). Here are links to the manuals with pictures in case you need a refresher before posting a more accurate article: http://www.vmware.com/support/pubs/vi_pages/vi_pubs_35.html.

Here's the part on networking: http://pubs.vmware.com/vi35/server_config/sc_networking.5.6.html.

Here's the part on storage: http://pubs.vmware.com/vi35/server_config/sc_storage.10.1.html.

Lastly, we have tens of thousands of installations in Microsoft only shops and those customers have had no issues installing or managing VMware for the past 9 years.

mikedipetrillo June 02, 2008 (Article Rating: )


“The Hyper-V virtualization role can be installed on either a full Server 2008 installation or on a minimal Server Core installation. Server Core is the better choice for a virtualization server host because it has all the extraneous Windows components stripped out (e.g., the graphical shell, Internet Explorer—IE, Outlook). This bare-metal approach gives Server Core less overhead and makes it more efficient. Server Core is also more secure because of the reduced attack surface area, as well as more reliable because of the smaller number of components that might need patching.”

Server Core may be less than a full installation but at the current RC1 release of Hyper-V an install of Hyper-V plus the required Server Core installation runs 3.99 GB. That’s not a typo. That’s still a HUGE trusted computing base when you compare it to 32 MB for ESXi or even 1.8 GB for a full ESX install. Microsoft may claim that VMware is a large, monolithic hypervisor because of where we put the drivers but 32 MB still seems pretty compact compared to the 3.99 GB of a minimal Hyper-V install.

mikedipetrillo June 02, 2008 (Article Rating: )


 See More Comments  1   2   3 

You must log on before posting a comment.

If you don't have a username & password, please register now.




Top Viewed ArticlesView all articles
VMware and the Future of Virtualization

What's next for virtualization and business IT? Windows IT Pro senior editor Jeff James speaks with VMware President and CEO Diane Greene on the future of virtualization technology. ...

Friday at PASS Europe 2006

Kevin talks about the closing day of the event and shares a funny Microsoft film. ...

The Memory-Optimization Hoax

Don't believe the hype. At best, RAM optimizers have no effect. At worst, they seriously degrade performance. ...


Related Articles Essential Server Core Setup Commands

The Soul of Windows Server 2008: Server Core and Hyper-V

A First Look at Windows Server 2008 Hyper-V

Windows Server Virtualization Features

News and Analysis Whitepapers MIMO Wireless LAN PHY Layer (RF) Operation & Measurement Application Note 1509

IT Industry Trends 2005 Study

The Who, What, Why, and Where of IT Events and Trade Shows

Related Events Check out our list of Free Email Newsletters!

Related News and Analysis Resources Become a VIP member of the Windows IT Pro community!
Get it all with the VIP CD and VIP access. A $500+ value for only $279!

Subscribe to Windows IT Pro!
Solve your toughest technical problems with our experts and access 10,000 + articles online. 30% off

Monthly Online Pass - Only $5.95!
Get instant access to 10,000+ articles from Windows IT Pro Magazine!

TechNet Virtual Labs
Evaluate and test Microsoft's newest products.

Job Openings in IT


ADS BY GOOGLE SPONSORED LINKS FEATURED LINKS

IT Connections
Dive into the new Microsoft platforms and products you implement and support with the experts from Microsoft, TechNet Magazine, Windows ITPro and industry gurus. There are 70+ sessions and interactive panels with networking opportunities.

Attention User Group Leaders...
Announcing the eNews Generator—a FREE HTML e-newsletter builder for user group leaders. Build your HTML and text e-newsletters in minutes and add Windows IT Pro & SQL Server Mag articles alongside your own message!.

Master SharePoint with 3 eLearning Seminars
Learn how to build a better SharePoint infrastructure and enable powerful collaboration with MVPs Dan Holme and Michael Noel. Register today!

Get SQL Server 2008 at WinConnections
Don’t miss Microsoft Exchange and Windows Connections conferences, the premier events for Microsoft IT Professionals in Las Vegas, November 10-13. Every attendee will receive a copy of SQL Server 2008 Standard Edition with one CAL.



Interested in Email Encryption?
Read about the advantages of identity-based encryption in this free report.

Order Your SQL Fundamentals CD Today!
Learn how to use SQL Server, understand Office integration techniques and dive into the essentials of SQL Express and Visual Basic with this free SQL Fundamentals CD.

Virtualization Congress Oct. 14-16 in London
Don't miss Virtualization Congress, the premiere EMEA conference dedicated to hardware, OS and application virtualization. Oct. 14-16.
Windows IT Pro Home Register FAQ for Windows WinInfo News
Europe Edition About Us Contact Us/Customer Service Media Kit Affiliates / Licensing  
SQL Server Magazine Office & SharePoint Pro Windows Dev Pro IT Job Hound ITTV
IT Library Technical Resources Directory Connected Home Windows Excavator Windows SuperSite 
 
 Windows IT Pro is a Division of Penton Media Inc.
 Copyright © 2008 Penton Media, Inc., All rights reserved. Terms and Use | Privacy Statement | Reprints and Licensing