I have a good time at the Computer Measurement Group meeting in Orlando recently. Mario Jauvin and I put together a tutorial on "Capacity Planning and Performance Monitoring with Free Tools" that was well attended, although relatively few attendees seem to be using free tools, mostly due to policy and support issues. We also attended James Holtman's workshop on using the free statistics package 'R' for performance data analysis and plotting.
The main new theme at this year's conference seemed to be CPU virtualization. Many people using VMware, Zen, Solaris containers and other virtualization facilities are finding that their measurements of CPU utilization don't make sense any more. Both BMC and Teamquest are working on building some support for virtualization concepts into their tools.
My observation is that utilization is useless as a metric and should be abandoned. It has been useless in virtualized disk subsystems for some time, and is now useless for CPU measurement as well. There used to be a clear relationship between response time and utilization, but systems are now so complex that those relationships no longer hold. Instead, you need to directly measure response time and relate it to throughput. Utilization is properly defined as busy time as a proportion of elapsed time. The replacement for utilization is headroom which is defined as the unused proportion of the maximum possible throughput. Dave Fisk calls this Capability Utilization.
I'm thinking of writing a paper, maybe for next year's CMG on this topic....
Happy holidays everyone
Cheers Adrian
Subscribe to:
Post Comments (Atom)
I enjoyed your workshop on free tools. I didn't take me long to implement Orca once I got back to the office and we're already using Nagios.
ReplyDeleteI'm on the lookout for other uses for the tools you mentiond.
hi adrian,
ReplyDeletethanks for your post.
I am downloading ORCA!
Would be nice to know about the other tools your commentator mentioned. Since the tools are free/open source, it will be useful to others as well :-) I know that you would not want to duplicate your conference presentation here but would appreciate if there's something that could be useful.
BTW, I've discovered tools like Groundwork that build on top of Nagios --- any opinions on this space ?
Thank you,
BR,
~A
Hi Stephen, one of the few tools that can model Hyperthreading correctly is the Hyperformix IPS modelling tool.
ReplyDeleteI have submitted a paper abstract for CMG06 called "Utilization is virtually useless as a metric" which I now have to finish writing up into an actual paper....
One new development is the use of power managed CPUs in servers. That nice 2.6GHz Opteron machine will notice that you aren't using it much and decide to save power by running at 1GHz. The CPU utilization is not normalized to the clock rate, so the system looks busier than it is, and the utilization metric is polluted by a large hidden step function as the clock rate changes.
Adrian