CMG05 trip comments and "utilization is useless..."

I have a good time at the Computer Measurement Group meeting in Orlando recently. Mario Jauvin and I put together a tutorial on "Capacity Planning and Performance Monitoring with Free Tools" that was well attended, although relatively few attendees seem to be using free tools, mostly due to policy and support issues. We also attended James Holtman's workshop on using the free statistics package 'R' for performance data analysis and plotting.

The main new theme at this year's conference seemed to be CPU virtualization. Many people using VMware, Zen, Solaris containers and other virtualization facilities are finding that their measurements of CPU utilization don't make sense any more. Both BMC and Teamquest are working on building some support for virtualization concepts into their tools.

My observation is that utilization is useless as a metric and should be abandoned. It has been useless in virtualized disk subsystems for some time, and is now useless for CPU measurement as well. There used to be a clear relationship between response time and utilization, but systems are now so complex that those relationships no longer hold. Instead, you need to directly measure response time and relate it to throughput. Utilization is properly defined as busy time as a proportion of elapsed time. The replacement for utilization is headroom which is defined as the unused proportion of the maximum possible throughput. Dave Fisk calls this Capability Utilization.

I'm thinking of writing a paper, maybe for next year's CMG on this topic....

Happy holidays everyone
Cheers Adrian