Showing posts with label solaris. Show all posts
Showing posts with label solaris. Show all posts

Saturday, April 21, 2007

Load Average Differences Between Solaris and Linux

A lot of people monitor their servers using load average as the primary metric. Tools such as Ganglia colorize all the nodes in a cluster view using load average. However there are a few things that aren't well understood about the calculation and how it varies between Solaris and Linux.

For a detailed explanation of the algorithm behind the metric, Neil Gunther has posted a series of articles that show how Load Average is a time-decayed metric that reports the number of active processes on the system with a one, five and fifteen minute decay period.

The source of the number of active processes can be seen in vmstat as the first few columns, and this is where Solaris and Linux differ. For example, some Linux vmstat from a busy file server is shown below.
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 4 43      0  32384 2993312 3157696  0    0  6662  3579 11345 7445  7 65  0 27


The first two columns show the number of processes that are in the run queue waiting for CPU time and in the blocked queue waiting for disk I/O to complete. These metrics are calculated in a similar manner in both Linux and Solaris, but the difference is that the load average calculation is fed by just the "r" column for Solaris, and by the "r" plus the "b" column for Linux. This means that a Linux based file server that has many disks could be running quite happily from a CPU perspective but show a large load average.

The logic behind the load average metric is that it should be a kind of proxy for responsiveness on a system. To get a more scalable measure of responsiveness, it is common to divide the load average by the number of CPUs in the system, since more CPUs will take jobs off the run queue faster. For disk intensive workloads on Linux, it may also make sense to divide the load average by the number of active disks, but this is an awkward calculation to make.

It would be best to take r/CPU count and b/active disk count then average this combination with a time decay and give it a new name, maybe the "slowed average" would be good?

Monday, February 12, 2007

AMD Enhanced Power Now - Variable Cores and Clock Rates

There is an interesting article in The Register about the latest variant of AMD's enterprise power management system.

As I've mentioned before, in the interests of saving power, some enterprise server systems are varying their clock rates so that they end up showing a higher utilization at low load levels that you would expect. This non-linear relationship of load to utilization is one of the things I highlighted in my CMG06 paper called "Utilization is virtually useless as a metric".

The latest twist: in AMD's upcoming four core systems, individual cores will be stopped completely if there isn't enough work for them to do.

The effect on utilization metrics will depend upon how each operating system interacts with the power management capabilities...

For Solaris the so-called "idle loop" is actually quite busy. An idle CPU watches its neighbors to see if they have too many jobs on their run queues, and gets work by migrating processes that won't get to run soon to itself. Interrupts are also bound to individual CPUs, so that data structures don't have to migrate between caches at high interrupt rates.

It will be interesting to see how these technologies interact.

Adrian

Sunday, November 19, 2006

Skype on Solaris

http://blogs.sun.com/darren/entry/skype_1.3.0.53_on_solaris_via

Solaris has a Linux compatible subsystem called BrandZ for running Linux binaries that don't have Solaris builds (like Skype). Darren figured out how to get the Linux build of Skype to run on Opensolaris.

Thanks to Alec for pointing this out.