Friday, March 26, 2010

Configure Antivirus Exclusions for Improved Hyper-V Performance

This is a topic that has been discussed in plenty of detail across the web for some time now. But, recently I have noticed an increased number of inquiries regarding the proper configurations of antivirus solutions installed on Hyper-V host and general performance problems tied to improperly configured antivirus agents. I expect this recent surge is contributed to the increased adoption rate of Hyper-V R2 and the incorporation of Live Migration with Cluster Shared Volumes.

I am sure we all understand the importance of running antivirus on any server in the enterprise. It protects us from all the bad stuff that is crawling the web and from time to time creeps its way into our networks. Any good AV solution intercepts calls to local memory and disk and in some cases can even intercept running processes. This interception is by design and is what keeps our systems safe, but can lead to poor performance, especially on a Hyper-V host.

Proper configuration of your AV solution on a Hyper-V host includes exclusions of both processes, directories and file types and not doing so correctly can not only lead to poor performance, but can even lead to your VM’s going offline. So let’s look at what the exclusion configuration should look like.

On any Hyper-V host you will find a couple of core processes that is crucial to host and VM performance. Prevent the following processes from AV scans by excluding the following as part of you Hyper-V AV policy.

VMMS.exe
VMWP.exe

You also want to exclude the root directories where VM configurations and Virtual Hard Disks are stored. Exclude the following directories.

C:\ProgramData\Microsoft\Windows\Hyper-V
C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks
Custom VM configuration, Virtual Hard Disk and Snapshot directories

Next, you want to create AV exclusions for the following file extensions.

*.XML
*.VHD
*.AVHD
*.VFD
*. VSV
*.ISO

Finally, if you are using Hyper-V R2’s Live Migration feature with Cluster Shared Volumes, then you will need to exclude the CSV path and any sub-directories. The CSV path is as follows.

C:\Clusterstorage

Failure to create this exclusion on hosts using CSV, can not only result in poor performance, but can also result in a missing or corrupt VM configuration. Refer to Microsoft Knowledge Base Article 961804 found at the link below for more information on this issue.

http://support.microsoft.com/kb/961804

Sunday, March 7, 2010

Hyper-V R2 Upgrade Problems When Using Broadcom NetXtreme II 5708 NICs Bound to Virtual Switches

I have been very busy over the last couple of months upgrading Hyper-V hosts to R2. For the most part, these upgrades have been uneventful, without any issues at all, which is always a good thing. However, I said "for the most part", so lets talk about what happened?

Early in the upgrades, I was seeing some random issues with Virtual Switches losing their bindings with the physical NIC. This caught my attention, but after poking around and with no issues being evident, I just recreated the virtual switch and moved on. However, as I started working more and more larger scale upgrades I started to see a concerning trend. I noticed any Virtual Switch that was bound to a Broadcom NetXtreme II 5708 was losing its bindings. I decided to see if I could reproduce this in a lab and found that I could reproduce this easily.

I am sure there are those of you out there wondering why I would even be using Broadcom NIC's for my virtual networks instead of the good old dependable Intel that we all know and love. The answer is simple. I run all Dell in my data center and all Dell PowerEdge servers come with Broadcom NIC's on-board. I am running some pretty dense Hyper-V configurations with one cluster running over 12 virtual networks, so I need all the NIC's I can get, so not using the four on-board Broadcom's in this configuration wasn't an option. Well, at least it wasn't before, but that has changed.

I decided to reach out to my friends over at the Microsoft Product group and see if they had heard of this problem internally or from feedback for other customers. Apparently, this is a recent issue that has been discovered and is now officially documented as a known issue. Currently, Microsoft doesn't have a work around for this issue and nor is there an ETA for resolution. After all, it may not be Microsoft, but could very well be Broadcom.

So, what do we do? Well, in my configurations I can't afford these little gotchas and I will be working only with my trusted Intel NIC's for my virtual networks. And for my Broadcom NIC's? Well, they can still be used, but in my opinion are well suited for your management and iSCSI connections only.