Identifying storage intensive VMs in Hyper-V 2016 Clusters

We’ve all had the case where there was a volume running hot on your cluster and you spend ages wrestling with perf counters to try to find that VM that’s causing your storage to burn. Well let me introduce you to a magical new command in Windows Server 2016

Get-StorageQoSFlow

This miracle command can give you insights on all the VHD(x)s running on your cluster, revealing IOPS, Latency and Bandwidth stats for them all without the need for large-scale monitoring solutions.

Let’s look at a few examples of how we can use this to our advantage, starting with the top 5 busiest VHD(x)s

Get-StorageQoSFlow -CimSession ClusterName | Sort-Object InitiatorIOPS -Descending | select -First 5
Get-StorageQoSFlow Top 5

In my example above, we can see straight away both the busiest machine, and exactly which of its disks is creating the load.

Next let’s move on to checking the performance of a specific VM, which would look like this

Get-StorageQoSFlow -InitiatorName VMName -CimSession ClusterName
Get-StorageQoSFlow Specific VM

Now these two uses of the command alone have already helped a fellow admin find several VMs that were thought to be idle, actually consuming around about 15,000 IOPS due to a few rough processes. These were precious IOPS that were better served being available to more critical services.

For bonus points, here are a few more commands to look at when looking for storage stats

# Get stats for any CSV
Get-StorageQoSVolume -CimSession ClusterName

# Get stats for a Storage Spaces Direct Cluster
Get-StorageSubSystem clu*  | Get-StorageHealthReport

# Get stats from the health service for all CSVs
Get-Volume -CimSession ClusterName | ?{$_.FileSystem -eq 'CSVFS'} | Get-StorageHealthReport -CimSession ClusterName

Until next time!