Hyper-V R2 Upgrade Problems When Using Broadcom NetXtreme II 5708 NICs Bound to Virtual Switches

I have been very busy over the last couple of months upgrading Hyper-V hosts to R2. For the most part, these upgrades have been uneventful, without any issues at all, which is always a good thing. However, I said "for the most part", so lets talk about what happened?

Early in the upgrades, I was seeing some random issues with Virtual Switches losing their bindings with the physical NIC. This caught my attention, but after poking around and with no issues being evident, I just recreated the virtual switch and moved on. However, as I started working more and more larger scale upgrades I started to see a concerning trend. I noticed any Virtual Switch that was bound to a Broadcom NetXtreme II 5708 was losing its bindings. I decided to see if I could reproduce this in a lab and found that I could reproduce this easily.

I am sure there are those of you out there wondering why I would even be using Broadcom NIC's for my virtual networks instead of the good old dependable Intel that we all know and love. The answer is simple. I run all Dell in my data center and all Dell PowerEdge servers come with Broadcom NIC's on-board. I am running some pretty dense Hyper-V configurations with one cluster running over 12 virtual networks, so I need all the NIC's I can get, so not using the four on-board Broadcom's in this configuration wasn't an option. Well, at least it wasn't before, but that has changed.

I decided to reach out to my friends over at the Microsoft Product group and see if they had heard of this problem internally or from feedback for other customers. Apparently, this is a recent issue that has been discovered and is now officially documented as a known issue. Currently, Microsoft doesn't have a work around for this issue and nor is there an ETA for resolution. After all, it may not be Microsoft, but could very well be Broadcom.

So, what do we do? Well, in my configurations I can't afford these little gotchas and I will be working only with my trusted Intel NIC's for my virtual networks. And for my Broadcom NIC's? Well, they can still be used, but in my opinion are well suited for your management and iSCSI connections only.

Improving Linux Performance on Hyper-V

This was a very busy week for my team and I as we were working with Microsoft to develop a video documenting our work and successes with Microsoft Virtualization. I had the pleasure of working with some very hardworking, dedicated folks and you really have to see them in action to appreciate all they give day after day. Part of this process involved a number of interviews outlining our work and experience with Hyper-V, Management Tools, Etc.

Later that evening, the team and I headed to dinner and found ourselves discussing the performance of Linux on Hyper-V. It appeared that most of the team wasn't aware that there was a recent update to the Linux Integration Components. At the end of last month, Microsoft released updated components known as the "Linux Integration Components Version 2" for use with all versions of Hyper-V. After learning my team wasn't aware, I started reaching out to others that I network with and realized that there wasn't as much awareness as expected and felt it was appropriate to post about it in detail.

The updated components are supported on all x64 versions of Windows Server 2008, Hyper-V Server 2008, Windows Server 2008 R2 and Hyper-V Server 2008 R2. Supported Linux Guest are SUSE Linux Enterprise Server 10 SP2 (x86 and x64), SUSE Linux Enterprise Server 11 (x86 and x64) and Red Hat Enterprise Linux 5.2, 5.3 and 5.4 (x86 and x64), but these guest are only supported when configured with 1 virtual CPU. The updated components provide support for synthetic storage and network drivers allowing them to take advantage of the very fast VMBus. The components also allow the guest to leverage Fastpath boot support, allowing boot devices to leverage the storage VSC (Virtualization Service Client) for enhanced performance. However, the update still doesn't provide mouse support directly and you must leverage a driver made available as part of the Citrix Project Satori.

The certification efforts leading to the development of these components has allowed me to change my thinking from if, to when I will start migrating my production Linux environments to Hyper-V. Every test scenario I have implemented has resulted in a stable well performed virtualized Linux guest which is what every admin strives for.

For more information or to download the Linux Integration Components v2, then please visit the link below.

http://www.microsoft.com/downloads/details.aspx?FamilyID=c299d675-bb9f-41cf-b5eb-74d0595ccc5c&displaylang=en

Destination.......Microsoft Tech-Ed North America 2010

One of the things that I look forward to every year is attending Microsoft's annual meeting of Tech-Ed North America. Every year I wait in anticipation of the announcement of location and open registration like an eight year old boy on Christmas morning waiting to see if he got the Red Ryder BB gun he wanted. I love this conference because its the only venue like it where I can get information on all of the latest Microsoft and partner solutions, speak with engineers and meet with other folks like myself and see how others are implementing Microsoft solutions in their environments.

This year is a going to be a little different for me as not only will I be attending, but I have been invited to participate as a speaker as well. I will be presenting on vSphere vs. Hyper-V, giving a real world approach to migrating a data center from VMWare to Microsoft. Using my own data center as an example, I will show how we completed this transition successfully and achieved greater VM density then VMWare states is possible and few data centers are achieving. I always look forward to meeting my readers and fellow Hyper-V users, so please reach out to me if you are attending.

I mentioned before that Tech-Ed is also a great forum for meeting the excellent Microsoft partners out there. Some would say that Microsoft make the products, but partners make them work. Now, that is up for interpretation, but you get the point. I posted a blog a little over a week ago about a new Microsoft partner, Virsto Software, that was developing a product to compliment Hyper-V virtualization by significantly improving server provisioning time, I/O performance and managing VM sprawl. Well they have arrived and today they officially announced the launch of their new solution Virsto One. Starting today you can obtain lots of great product information on the Virsto website and even request a 30-day evaluation copy of their software, so you can test it for yourself.

Virsto will be attending and demonstrating their product at many of this years technical shows to include Microsoft Tech-Ed, Microsoft Management Summit and EMC World, so stop by and check these guys out. I would love hear about others experience with the new Virsto One solution, so please let me know what you think.

If you are interested in an eval copy of the Virsto One solution or want more information on any of the conferences I mentioned, then visit the links below.

http://www.virsto.com/

http://www.microsoft.com/events/techednorthamerica/

http://www.mms-2010.com/public/home.aspx

http://www.emcworld.com

Hyper-V Denial of Service Vulnerability Discovered

For those of us managing IT infrastructure, we have adapted to the constant surprises that Murphy's Law imposes on us. But, the one thing that we can always count on is a security announcement from Microsoft on the second Tuesday of every month also known as "Patch Tuesday".

However, this month we were informed of a new vulnerability (977894) discovered in Hyper-V that could cause a denial of service if an authenticated user were to run a sequence of malformed machine instruction from within a virtual guest. This vulnerability can only be exploited if a user has valid logon credentials and logs on locally to the virtual guest and can not be exploited remotely or by anonymous users. This security update applies to all x64-based editions of Windows Server 2008, Windows Server 2008 R2, Hyper-V Server 2008 and Hyper-V Server 2008 R2. The security bulletin doesn't make it clear that this update applies to versions of Hyper-V Server, but is buried within the FAQ. This update is to be installed on the host only and not on the virtual guest. Environments that are configured with Automatic Updates enabled will apply this update automatically.

For more information on this security bulletin or to download the security update, then please visit the link below.

http://www.microsoft.com/technet/security/bulletin/MS10-010.mspx