Managing VMware DRS rules using PowerCLI

One of the core features of VMware vSphere is the Distributed Resource Scheduler (DRS). VMware DRS is vSphere’s workload load balancer and relies on VMware vMotion technology to live-migrate workloads from one ESX host to another.

You can constrain the VMware DRS decisions by defining DRS Rules. As of vSphere 4.1 there are 2 type of DRS rules:

  • VM-to-VM rules
  • VM-to-Host rules (new in vSphere 4.1)

VM-to-VM rules

Until vSphere 4.1 you could only create VM-to-VM rules. This type of rule specifies the affinity between virtual machines (VMs). You can define both affinity- and anti-affinity rules.

Affinity rules

Affinity rules keep workloads together on the same ESX host. This is beneficial if for instance an application server needs fast access to its database that resides on another VM. Keeping both VMs together on the same host boosts performance as network traffic between the VMs won’t leave the ESX host. Note that if both servers are in different VLANs or subnets this benefit could be undone, as network traffic might need to be routed externally.

Anti-affinity rules

Anti-affinity rules separate workloads on different ESX hosts. This is beneficial if you have services running on multiple servers for redundancy, like domain controllers or Network Load Balancing (NLB) clusters. In these situations you want to make sure that the VMs are distributed over different ESX hosts. If one ESX host fails, the service would still be available through other VM(s) residing on other ESX hosts.

Using PowerCLI

You can manage the DRS rules through the properties of the Cluster (Remember that DRS is a vCenter cluster feature) using the vSphere Client, but you can also use PowerCLI. Using PowerCLI, DRS rules are retrieved with the Get-DrsRule cmdlet:

[vSphere PowerCLI] C:\Scripts> Get-DrsRule –Cluster CL01
Name            Enabled KeepTogether VMIDs
----            ------- ------------ -----
App01_Affinity  True    True         {VirtualMachine-vm-35672, VirtualM...
DC_Separate     True    False        {VirtualMachine-vm-35697, VirtualM...

Notice that only the VM IDs are returned for the configured VMs and not their names. You probably want the associated VM’s names to appear in your report instead of the VM IDs.

This can be accomplished using the New-VIProperty cmdlet:

New-VIProperty -Name VMs -ObjectType DrsVMAffinityRule -Value {
param($rule)
foreach($vm in $rule.VMIDs) {(Get-VM -Id $vm).Name}
}
[vSphere PowerCLI] C:\Scripts> Get-DrsRule –Cluster CL01 | Select Name,
>> Enabled, KeepTogether, VMs
Name           Enabled KeepTogether VMs
----           ------- ------------ ---
App01_Affinity    True         True {Arnim1, Arnim2}
DC_Separate       True        False {Arnim3, Arnim4}

You can now easily export your DRS rules for the sake of documentation or backup using the Export-CSV cmdlet.

[vSphere PowerCLI] C:\Scripts> Get-DrsRule –Cluster CL01 | Select Name,
>> Enabled, KeepTogether, VMs | Export-Csv CL01_Rules.csv

If you want to add a new rule, you can use the New-DRSRule cmdlet. To create a new DRS rule you need to specify the following mandatory parameters:

  • Name: The name for the new DRS rule.
  • Cluster: The cluster for which the new DRS rule applies.
  • KeepTogether: Specify $true to create a new affinity rule, or $false to create a new anti-affinity rule.
  • VM: The VMs to include in the new DRS rule

Let’s create a new anti-affinity rule for our proxy NLB cluster:

[vSphere PowerCLI] C:\Scripts> New-DrsRule –Name ProxyNLB_separate `
>> -Cluster CL01 –KeepTogether:$false –VM Proxy01,Proxy02

To verify the new rule, simply retrieve the DRS rules using the Get-DrsRule cmdlet again:

[vSphere PowerCLI] C:\Scripts> Get-DrsRule –Cluster CL01 | Select Name,
>> Enabled, KeepTogether, VMs
Name                       Enabled        KeepTogether VMs
----                       -------        ------------ ---
App01_Affinity                True                True {Arnim1, Arnim2}
DC_Seperate                   True               False {Arnim3, Arnim4}
ProxyNLB_Separate             True               False {Proxy01, Proxy02}

Removing a DRS rule is accomplished using the Delete-DrsRule cmdlet. This cmdlet requires you to specify a DrsVMAffinityRule object. This means that you can’t specify the rule by name, but you need to retrieve the rule object using the Get-DrsRule cmdlet first. Let’s remove the App01_Affinity rule:

[vSphere PowerCLI] C:\Scripts> Get-DrsRule –Cluster CL01 `
>> –Name App01_Affinity | Remove-DrsRule –Confirm:$false

VM-to-Host rules

vSphere 4.1 extended DRS functionality with DRS groups and the VM-to-Host rule type. With this type of rule you can specify the affinity between a group of VMs and a group of ESX hosts (inside a cluster). There are four rule types:

  • Should
  • Should not
  • Must
  • Must not

Mandatory rules (“Must” rules)

Mandatory rules are used in specific cases, such as licensing requirements. As an example let’s have a look to Oracle’s licensing requirements. Oracle states that in a virtual environment every socket in every ESX host in the cluster must be licensed, because the Oracle VM can potentially run on every host. Using a mandatory rule you can limit the Oracle VM to only run on the licensed ESX hosts.

Preferential rules (“Should” rules)

Preferential rules can be used in availability scenario’s, where a group of VMs is split across blade enclosures or racks. In the event of a loss of an enclosure or rack, the availability of the service or application is not jeopardized and HA is allowed to “violate” the preferential rule and restart the failed VMs on the surviving enclosures or racks. VMware HA isn’t aware of preferential rules because they aren’t communicated to HA by DRS.

Creating the new VM-to-Host type of rules is only available through the vSphere client and not using the standard VMware PowerCLI cmdlets at this time. But do not fear. I’ve created a couple of functions to allow you to automate this. The creation of a VM-to-Host rule is a three-step process:

  1. Define a group of VMs (VMGroup)
  2. Define a group of ESX hosts (HostGroup)
  3. Create the VM-to-Host rule

Creating VMGroups

A VMGroup can be created using the New-DrsVmGroup function from Listing 1. To create a new VMGroup that contains virtual machines VM001 and VM002:

Get-VM VM001,VM002 | New-DrsVmGroup -Name "VmGroup01" -Cluster CL01

Listing 1

Function New-DrsVmGroup {
<#
.SYNOPSIS
  Creates a new DRS VM group
.DESCRIPTION
  This function creates a new DRS VM group in the DRS Group Manager
.NOTES
  Author: Arnim van Lieshout
.PARAMETER VM
  The VMs to add to the group. Supports objects from the pipeline.
.PARAMETER Cluster
  The cluster to create the new group on.
.PARAMETER Name
  The name for the new group.
.EXAMPLE
  PS> Get-VM VM001,VM002 | New-DrsVmGroup -Name "VmGroup01" -Cluster CL01
.EXAMPLE
  PS> New-DrsVmGroup -VM VM001,VM002 -Name "VmGroup01" -Cluster (Get-CLuster CL01)
#>

	Param(
		[parameter(valuefrompipeline = $true, mandatory = $true,
		HelpMessage = "Enter a vm entity")]
			[PSObject]$VM,
		[parameter(mandatory = $true,
		HelpMessage = "Enter a cluster entity")]
			[PSObject]$Cluster,
		[parameter(mandatory = $true,
		HelpMessage = "Enter a name for the group")]
			[String]$Name)

	begin {
	    switch ($Cluster.gettype().name) {
      		"String" {$cluster = Get-Cluster $cluster | Get-View}
      		"ClusterImpl" {$cluster = $cluster | Get-View}
      		"Cluster" {}
      		default {throw "No valid type for parameter -Cluster specified"}
		}
		$spec = New-Object VMware.Vim.ClusterConfigSpecEx
		$group = New-Object VMware.Vim.ClusterGroupSpec
		$group.operation = "add"
		$group.Info = New-Object VMware.Vim.ClusterVmGroup
		$group.Info.Name = $Name
	}

	Process {
		switch ($VM.gettype().name) {
      		"String" {Get-VM -Name $VM | %{$group.Info.VM += $_.Extensiondata.MoRef}}
      		"VirtualMachineImpl" {$group.Info.VM += $VM.Extensiondata.MoRef}
      		"VirtualMachine" {$group.Info.VM += $VM.MoRef}
      		default {throw "No valid type for parameter -VM specified"}
	    }
	}

	End {
		if ($group.Info.VM) {
			$spec.GroupSpec += $group
			$cluster.ReconfigureComputeResource_Task($spec,$true)
		}
		else {
      		throw "No valid VMs specified"
		}
	}
}

Creating HostGroups

Now that a VMGroup is created, you need to define a HostGroup. This can be accomplished using the New-DrsHostGroup function from Listing 2. To create a new HostGroup that contains ESX hosts ESX001 and ESX002:

Get-VMHost ESX001,ESX002 | New-DrsHostGroup -Name "HostGroup01" `
    -Cluster CL01

Listing 2

Function New-DrsHostGroup {
<#
.SYNOPSIS
  Creates a new DRS host group
.DESCRIPTION
  This function creates a new DRS host group in the DRS Group Manager
.NOTES
  Author: Arnim van Lieshout
.PARAMETER VMHost
  The hosts to add to the group. Supports objects from the pipeline.
.PARAMETER Cluster
  The cluster to create the new group on.
.PARAMETER Name
  The name for the new group.
.EXAMPLE
  PS> Get-VMHost ESX001,ESX002 | New-DrsHostGroup -Name "HostGroup01" -Cluster CL01
.EXAMPLE
  PS> New-DrsHostGroup -Host ESX001,ESX002 -Name "HostGroup01" -Cluster (Get-CLuster CL01)
#>

	Param(
		[parameter(valuefrompipeline = $true, mandatory = $true,
		HelpMessage = "Enter a host entity")]
			[PSObject]$VMHost,
		[parameter(mandatory = $true,
		HelpMessage = "Enter a cluster entity")]
			[PSObject]$Cluster,
		[parameter(mandatory = $true,
		HelpMessage = "Enter a name for the group")]
			[String]$Name)

	begin {
	    switch ($Cluster.gettype().name) {
      		"String" {$cluster = Get-Cluster $cluster | Get-View}
      		"ClusterImpl" {$cluster = $cluster | Get-View}
      		"Cluster" {}
      		default {throw "No valid type for parameter -Cluster specified"}
		}
		$spec = New-Object VMware.Vim.ClusterConfigSpecEx
		$group = New-Object VMware.Vim.ClusterGroupSpec
		$group.operation = "add"
		$group.Info = New-Object VMware.Vim.ClusterHostGroup
		$group.Info.Name = $Name
	}

	Process {
		switch ($VMHost.gettype().name) {
      		"String" {Get-VMHost -Name $VMHost | %{$group.Info.Host += $_.Extensiondata.MoRef}}
      		"VMHostImpl" {$group.Info.Host += $VMHost.Extensiondata.MoRef}
      		"HostSystem" {$group.Info.Host += $VMHost.MoRef}
      		default {throw "No valid type for parameter -VMHost specified"}
	    }
	}

	End {
		if ($group.Info.Host) {
			$spec.GroupSpec += $group
			$cluster.ReconfigureComputeResource_Task($spec,$true)
		}
		else {
      		throw "No valid hosts specified"
		}
	}
}

Creating VM-to-Host rules

Now that both VM- and HostGroup are created, you can start creating a VM-to-Host DRS rule. As mentioned before, there are rules to keep VMs on the specified set of Host or to keep them off these hosts. Using the New-DrsVmToHostRule function from Listing 3 you can create your VM-to-Host rule. By default the function will create an affine type of rule, unless you specify the opposite using the -AntiAffine switch parameter. The rule type is a preferential rule by default (“Should” rule) unless you specify the -Mandatoy parameter, which creates a mandatory rule (“Must” rule).

To create a mandatory anti-affinity rule (“Must Not” rule) between VMGroup01 and HostGroup01 to prevent the VMs from VMGroup01 to be executed on the ESX hosts in HostGroup01:

New-DrsVMToHostRule -VMGroup "VMGroup01" -HostGroup "HostGroup01" –Name `
    "VMToHostRule01" -Cluster CL01 -AntiAffine –Mandatory

Listing 3

Function New-DRSVMToHostRule{
<#
.SYNOPSIS
  Creates a new DRS VM to host rule
.DESCRIPTION
  This function creates a new DRS vm to host rule
.NOTES
  Author: Arnim van Lieshout
.PARAMETER VMGroup
  The VMGroup name to include in the rule.
.PARAMETER HostGroup
  The VMHostGroup name to include in the rule.
.PARAMETER Cluster
  The cluster to create the new rule on.
.PARAMETER Name
  The name for the new rule.
.PARAMETER AntiAffine
  Switch to make the rule an AntiAffine rule. Default rule type is Affine.
.PARAMETER Mandatory
  Switch to make the rule mandatory (Must run rule). Default rule is not mandatory (Should run rule)
.EXAMPLE
  PS> New-DrsVMToHostRule -VMGroup "VMGroup01" -HostGroup "HostGroup01" -Name "VMToHostRule01" -Cluster CL01 -AntiAffine -Mandatory
#>

	Param(
		[parameter(mandatory = $true,
		HelpMessage = "Enter a VM DRS group name")]
			[String]$VMGroup,
		[parameter(mandatory = $true,
		HelpMessage = "Enter a host DRS group name")]
			[String]$HostGroup,
		[parameter(mandatory = $true,
		HelpMessage = "Enter a cluster entity")]
			[PSObject]$Cluster,
		[parameter(mandatory = $true,
		HelpMessage = "Enter a name for the group")]
			[String]$Name,
			[Switch]$AntiAffine,
			[Switch]$Mandatory)

    switch ($Cluster.gettype().name) {
   		"String" {$cluster = Get-Cluster $cluster | Get-View}
   		"ClusterImpl" {$cluster = $cluster | Get-View}
   		"Cluster" {}
   		default {throw "No valid type for parameter -Cluster specified"}
	}

	$spec = New-Object VMware.Vim.ClusterConfigSpecEx
	$rule = New-Object VMware.Vim.ClusterRuleSpec
	$rule.operation = "add"
	$rule.info = New-Object VMware.Vim.ClusterVmHostRuleInfo
	$rule.info.enabled = $true
	$rule.info.name = $Name
	$rule.info.mandatory = $Mandatory
	$rule.info.vmGroupName = $VMGroup
	if ($AntiAffine) {
		$rule.info.antiAffineHostGroupName = $HostGroup
	}
	else {
		$rule.info.affineHostGroupName = $HostGroup
	}
	$spec.RulesSpec += $rule
	$cluster.ReconfigureComputeResource_Task($spec,$true)
}

As a general rule of recommendation: Use DRS rules, both VM-to-VM and VM-to-Host, sparingly, as every rule you create limits the choices DRS has to move VMs across ESX hosts. Also remember that mandatory rules stay in effect even when DRS is disabled! To disable mandatory rules, you need to disable them manually.

If you want to learn more about VMware DRS, I highly recommend reading the VMware vSphere 4.1 HA and DRS technical deepdive.

Related posts:

  1. Rescan VMware vSphere Hypervisor (ESXi) using PowerCLI simplified Tweet While preparing some disaster recovery (DR) tests, I had to add and remove a couple of LUNS from several ESX hosts in different clusters. Doing so,  I had to...
  2. PowerCLI: Disable/Enable HA and DRS Tweet Before upgrading my Virtual Center 2.5 server to vCenter Server 4.0, I decided to temporarily disable HA and DRS. This is just a precaution taken to avoid waiting for...
  3. ESXCLI the PowerCLI way Tweet When trying to perform a Round Robin (RR) test on my vSphere 4.1 environment, I needed to know the number of paths available to the luns of the test...
  4. ESXi Tech Support Mode Tweet As a security recommendation you should always disable Tech Support Mode (TSM) on your ESXi servers, but sometimes it’s helpful if you’re able to connect to your ESXi server...
  5. PowerCLI 4.1 namespace changes Tweet Output type changes in PowerCLI 4.1In PowerCLI 4.1 VMware changed the namespaces of the output types. According to VMware, this was done to improve the internal structure and enable...

6 Comments on “Managing VMware DRS rules using PowerCLI”

  1. #1 Christian
    on Jun 23rd, 2011 at 2:00 pm

    Hi there
    Great writing
    Could you add a script to easily list VMs and Hosts being a part of the “Virtual Machines DRS Groups” and “Host DRS Groups”
    Purpose if for me to get an overview – and the possibility to export the current DRS groups as evidence for our MS SQL licensing – we had Microsoft verify that we can use DRS host affinity to handle the licensing issues…
    All the Best
    Christian

  2. #2 Hugo Phan
    on Jun 28th, 2011 at 10:26 pm

    Cool use of PowerCLI for DRS management Arnim :) . Top 5 in this week’s Planetv12n for good measure.

  3. #3 Arnim van Lieshout
    on Jun 30th, 2011 at 1:20 pm

    Thanks.
    When I find some spare time I will look into that. Thanks for sharing the idea. Didn’t thought of that one.

  4. #4 Arnim van Lieshout
    on Jun 30th, 2011 at 1:21 pm

    Thanks Hugo.
    Didn’t noticed the Planetv12n Top5 notation. C00l 8-)

  5. #5 Ed
    on Nov 25th, 2011 at 11:14 pm

    Awesome! Thanks Arnim!

    How I’ve used it to save me a TON of money:

    1. Create DRS host groups named linuxHosts and windowsHosts and put the right number of hosts in each. I did it manually but the code is included to do it programatically.

    2. Make sure that VM groups and linuxGuests and windowsGuests do not exist (they’ll be created shortly). I’d like to delete them if they exist but I don’t how to do that yet :(

    3. Run the following one-liners against your favorite cluster to create the linuxGuests and windowsGuests VM groups.

    get-cluster $cluster | get-vm | get-view | where-object { ($_.Guest.GuestFamily -eq “windowsGuest”) } | New-DrsVmGroup -name “windowsGuests” -cluster $cluster
    get-cluster $cluster | get-vm | get-view | where-object { ($_.Guest.GuestFamily -eq “linuxGuest”) } | New-DrsVmGroup -name “linuxGuests” -cluster $cluster

    If either of these tasks fail with “A specific parameter was not correct” then the VM group may already exist.

    4. Now create a hostVM affinity rule for the guests on the hosts. I’ve done it manually but the script is listed above if you want to call it.

    I’ve managed to split my 16-node cluster (each with 2 sockets) into exactly half for Windows and half for Linux. I’ve now saved myself 16 sockets of Windows licensing and 16 sockets of RHEL subscriptions, roughly a $50K first-year savings.

    I can very easily move a blade from one logical “cluster” to another simply by updating the host groups. If I want to provide capacity for both Windows and Linux, I just license the sockets for both and then add the host to both groups.

    To refresh the rules after guests are added, I just delete the 2 VM groups manually and re-run the script.

  6. #6 Arnim van Lieshout
    on Dec 8th, 2011 at 9:41 am

    Great to hear your success story Ed.
    Thanks for sharing!

Leave a Comment