VC thoughts via Peter Cranstone
From Vinod Knosla: The mobile phone will be a mainstream personal computer.
From Roger McNamee: Betting on smart phones: The mobile device migration to smart phones from features phones will produce even greater disruption than PC industry moving from character mode to graphical interface.
Pretty much what I've been saying for the last year or two...
Friday, May 16, 2008
Friday, May 09, 2008
Speaking at UK CMG TEC Conference - May 19-21st 2008
UKCMG TEC 2008 is near Northampton in the UK. I'm giving two of the same presentations as I gave at the US CMG, the enterprise version of my Millicomputing talk, and a half day workshop on Unix/Linux Performance.
My old friend Phil Harman is also giving a talk on DTrace, and I'm staying in the UK for a few days after the conference to catch up with friends and family.
My old friend Phil Harman is also giving a talk on DTrace, and I'm staying in the UK for a few days after the conference to catch up with friends and family.
Saturday, May 03, 2008
Usenix 08 - Tutorials and Invited Paper - Boston June 23-27
I'm giving the two tutorials that I have been presenting at CMG the last few years, but at Usenix in Boston. The tutorials are the latest update of my Solaris/Linux performance metrics and tuning class, and Capacity Planning with Free Tools.
I'm also giving an invited speaker talk on Millicomputing, and have a nice big time slot, so will cover both the mobile and enterprise aspects in some depth, with time for discussion.
I hope to see you there...
Monday, April 28, 2008
XE Toolkit 1.2 - new portable performance tools from Rich Pettit
Rich just updated the portable Java based XE tookit, it even runs on mainframes now :-)
Rich (with some contributions from me) wrote many versions of the SE toolkit for Solaris. This is his latest project, please try it out, give him feedback and buy a support contract to encourage him...
Cheers Adrian
Subject: [Xetoolkit-interest] XE Version 1.2 Available
To: xetoolkit-interest@xetoolkit.com
XE Version 1.2 is available in pre-packaged form through the Captive
Metrics online store, http://store.captivemetrics.com.
New platforms for 1.2:
AIX (5.3 base)
Linux on Power (IBM pSeries & iSeries)
Linux on s390 (IBM zSeries)
Please report bugs to support@captivemetrics.com.
Changes since Version 1.1
Bug fixes
Solaris locking problems
Performance improvements in mdr
Locale problems in non-US-English Windows installs
Support for AIX (5.3 base)
Support for Linux on Power (IBM pSeries & iSeries)
Support for Linux on s390 (IBM zSeries)
Code restructuring and use of new "snap" utility
and command-line build for Windows
VirtualGuru rotates log files when the day changes
bbClient groovy script for monitoring BigBrother clients
The source code is available at
http://sourceforge.net/projects/xe-toolkit as an SVN tree.
There will be a web site with "tarballs" eventually. It's not
high on the priority list.
The build process uses "snap", a /bin/sh utility written by me that
generates Makefiles on the fly.
Please support future development by purchasing the package.
Thank you for your continued interest.
Rich
--
Richard Pettit
Captive Metrics Software Corporation
Em: richp@captivemetrics.com
Ph: 8-MONITOR-01
Rich (with some contributions from me) wrote many versions of the SE toolkit for Solaris. This is his latest project, please try it out, give him feedback and buy a support contract to encourage him...
Cheers Adrian
Subject: [Xetoolkit-interest] XE Version 1.2 Available
To: xetoolkit-interest@xetoolkit.com
XE Version 1.2 is available in pre-packaged form through the Captive
Metrics online store, http://store.captivemetrics.com.
New platforms for 1.2:
AIX (5.3 base)
Linux on Power (IBM pSeries & iSeries)
Linux on s390 (IBM zSeries)
Please report bugs to support@captivemetrics.com.
Changes since Version 1.1
Bug fixes
Solaris locking problems
Performance improvements in mdr
Locale problems in non-US-English Windows installs
Support for AIX (5.3 base)
Support for Linux on Power (IBM pSeries & iSeries)
Support for Linux on s390 (IBM zSeries)
Code restructuring and use of new "snap" utility
and command-line build for Windows
VirtualGuru rotates log files when the day changes
bbClient groovy script for monitoring BigBrother clients
The source code is available at
http://sourceforge.net/projects/xe-toolkit as an SVN tree.
There will be a web site with "tarballs" eventually. It's not
high on the priority list.
The build process uses "snap", a /bin/sh utility written by me that
generates Makefiles on the fly.
Please support future development by purchasing the package.
Thank you for your continued interest.
Rich
--
Richard Pettit
Captive Metrics Software Corporation
Em: richp@captivemetrics.com
Ph: 8-MONITOR-01
Thursday, March 27, 2008
Patent on Performance and Capacity Planning
Here is another patent that I filed while working at Sun.
System and method for generating a probability distribution of computer performance ratios
System and method for generating a probability distribution of computer performance ratios
A system and method for generating a probability distribution indicating the probability of a range of performance ratios between computer systems. Benchmark and/or other testing results are obtained for both computer systems. For each test, a ratio of one system's performance to the other is calculated. The ratios and/or testing results may be weighted. From the performance ratios, a histogram is produced to indicate the probabilities of achieving the various ratios. Based on the distribution of probabilities, a particular performance ratio can be predicted with a corresponding level of confidence, and used for capacity planning or configuring a computing environment.
Patents on Peer to Peer Trading Platform
While working at eBay Research Labs I did some work using Skype that resulted in four patents in the area of P2P trading and reputation systems. I just noticed that they are now public, as of late 2007, and here they are:
Peer-to-peer trading platform
Peer-to-peer trading platform with relative reputation-based item search and buddy rating
Peer-to-peer trading platform with search caching
Peer-to-peer trading platform with roles-based transactions
The seed of the idea came from my good friend Josep Ferrandiz, who still works at PayPal. The ideas were developed with help from Zahid Ahmed who is now working for EMC, and with a lot of help from Neel Sundaresan who manages eBay Research labs. The patents are in all of our names.
I think we came up with some cool ideas, we also stress tested the Skype API's in some novel ways and got some things fixed. The lasting legacy of this work is more robust application to application messaging API's and improvements in Skype4Java.
Peer-to-peer trading platform
Peer-to-peer trading platform with relative reputation-based item search and buddy rating
Peer-to-peer trading platform with search caching
Peer-to-peer trading platform with roles-based transactions
The seed of the idea came from my good friend Josep Ferrandiz, who still works at PayPal. The ideas were developed with help from Zahid Ahmed who is now working for EMC, and with a lot of help from Neel Sundaresan who manages eBay Research labs. The patents are in all of our names.
I think we came up with some cool ideas, we also stress tested the Skype API's in some novel ways and got some things fixed. The lasting legacy of this work is more robust application to application messaging API's and improvements in Skype4Java.
Sunday, March 16, 2008
BIL Podcast: Garrett Lisi - Exceptionally Simple Theory of Everything
I published a BIL podcast from the recordings I made, first talk is my favorite, particle physics explained by Garrett Lisi
http://web.mac.com/adriancockcroft/BILcast/BILcast/BILcast.html
Subscribe to the podcast to pick up new talks as they are added.
http://web.mac.com/adriancockcroft/BILcast/BILcast/BILcast.html
Subscribe to the podcast to pick up new talks as they are added.
Monday, March 03, 2008
BIL - recording an excellent adventure
The closing slide of BIL by Garrett Lisi

Two days of intense expansion of my horizons, a self-organizing conference, and I am left holding a jewel. My contribution to BIL was that I basically did tech-support and audio for all the speakers in the main hall for two days. I brought my audio recorder a good mic and stand, a guitar amp (that was used as a PA speaker) and bags of cables and adaptors. Other BILsters provided a video projector and a karaoke machine for PA. I hooked up the mic and a mini-jack cable for computer audio out, and kept it all running as best I could. I recorded almost 800 minutes of content on my Korg D16 hard disk recorder, which is almost all of every talk. There were other people recording on Saturday, including video, but they had technical problems on Sunday, so we have little or no video of the Sunday talks.
Next I'm going to make a backup copy, then start working on extracting the raw data for other people to process. I'll make mp3's of my favorite talks as I get time. Several TED speakers talked at BIL, including Aubrey De Grey on Saturday and Garrett Lisi, who closed out the conference with a fantastic talk. I chatted to Garrett a bit, very cool...
For the stream of consciousness with pictures that I uploaded during the talk, look at my twitxr page. I will update my twitter account with progress on the audio files. I don't expect to use twitter and twitxr on a daily basis, but I think they are useful for conferences. I'll track progress at Ecomm08 on March 12-14th the same way.
Two days of intense expansion of my horizons, a self-organizing conference, and I am left holding a jewel. My contribution to BIL was that I basically did tech-support and audio for all the speakers in the main hall for two days. I brought my audio recorder a good mic and stand, a guitar amp (that was used as a PA speaker) and bags of cables and adaptors. Other BILsters provided a video projector and a karaoke machine for PA. I hooked up the mic and a mini-jack cable for computer audio out, and kept it all running as best I could. I recorded almost 800 minutes of content on my Korg D16 hard disk recorder, which is almost all of every talk. There were other people recording on Saturday, including video, but they had technical problems on Sunday, so we have little or no video of the Sunday talks.
Next I'm going to make a backup copy, then start working on extracting the raw data for other people to process. I'll make mp3's of my favorite talks as I get time. Several TED speakers talked at BIL, including Aubrey De Grey on Saturday and Garrett Lisi, who closed out the conference with a fantastic talk. I chatted to Garrett a bit, very cool...
For the stream of consciousness with pictures that I uploaded during the talk, look at my twitxr page. I will update my twitter account with progress on the audio files. I don't expect to use twitter and twitxr on a daily basis, but I think they are useful for conferences. I'll track progress at Ecomm08 on March 12-14th the same way.
Friday, February 29, 2008
Millicomputing at BIL this weekend
I'm presenting (and generally helping out) at the BIL un-conference this weekend in Monterey. I have posted some slides on my Millicomputing blog, on the subject of "The Future in Your Pocket", that speculates on what could happen over the next few years in the mobile space, and starts a discussion of what we might be able to use it for.
Thursday, February 28, 2008
Hiring a researcher at Netflix
Here is the job specification for a research oriented position working for me at Netflix.
We are looking for a very experienced engineer who can take the lead, researching, modeling, presenting, designing and building algorithms that run in a service oriented architecture to support millions of customers. The successful candidate will be a self-motivated, intellectually curious individual. A Doctorate or experience working in a research environment, work with personalization algorithms, machine learning, adaptive systems, and statistical analysis would be a bonus.
I manage a sister group to the one that does the Cinematch star ratings predictions and hosts the Netflix Prize, we use that data as an input along with everything else we know about movies and customers to come up with the actual lists of movies that are shown all over the site. The research position in my group works on algorithms and mining for new data sources.
Also, Netflix is a great place to work, big enough to be interesting and small enough to be fun.
We are looking for a very experienced engineer who can take the lead, researching, modeling, presenting, designing and building algorithms that run in a service oriented architecture to support millions of customers. The successful candidate will be a self-motivated, intellectually curious individual. A Doctorate or experience working in a research environment, work with personalization algorithms, machine learning, adaptive systems, and statistical analysis would be a bonus.
I manage a sister group to the one that does the Cinematch star ratings predictions and hosts the Netflix Prize, we use that data as an input along with everything else we know about movies and customers to come up with the actual lists of movies that are shown all over the site. The research position in my group works on algorithms and mining for new data sources.
Also, Netflix is a great place to work, big enough to be interesting and small enough to be fun.
Tuesday, February 12, 2008
Emerging Communications Conference - EComm March 12-14
I'm presenting at the EComm conference on the subject of Millicomputing on the morning of March 14th. There is a Homebrew Mobile Club meeting at the conference (at the Computer History Museum in Mountainview) the evening of March 12th.
Tuesday, January 15, 2008
MacWorld report: iZap, Apple MacBook Air, iPhone and AppleTV
I tracked the MacWorld keynote via the Macrumors site on my iPhone this morning. It worked very well, with updates being pushed every few minutes. Over-all there was some good stuff released, but I think expectations were higher.
Later in the afternoon I dropped in on MacWorld for an hour or so, which was plenty of time to check out the interesting stuff.
The MacBook Air is another hit product, destined to become a geek status symbol. It looks good, feels good to use, and has the option of a solid state disk. I had some hands-on time, and I want one.... They are expensive, but the format is very well designed, and I expect it will come down in price and get new options over the coming years. To make the case strong enough despite being thin, there are more curves in the case (curved metal is much stronger than flat sheet) and it has no large holes in it. So there is no removable battery pack, and each key has its own small hole in the casing. It actually looks more mechanically robust than my MacBook pro.
I hoped to find some interesting third party iPhone applications but didn't see anything worth mentioning. I played with a demo iPhone with 1.1.3, it had a bunch of safari based games bookmarked on the second page of icons. Once I got home I upgraded my iPhone, its running fine and the new features work well. They were well leaked a few weeks ago, so there is nothing new to report. The iPod touch picked up the missing set of applications as I expected, but Apple want $20 for the privilege. This is the first example of optional applications, and charging for applications, and I expect the mechanism will get well tested by the Touch then it will be introduced for the iPhone and third party applications.
I have had an AppleTV for six months or so, and the upgraded software looks much better. I don't think I will be renting or buying many movies, but the user interface is better, and the ability to browse both audio and video based podcasts could be interesting. I mostly use my AppleTV for YouTube, playing music and sharing photo albums. One new feature is the ability to pull up photo albums over the Internet from the .mac galleries generated by iPhoto. I've been using .mac for years, and the latest (Leopard) version of iPhoto galleries is very slick and well integrated.
Apple also introduced Time Capsule, which is a network backup server combining a 500GB or 1TB of disk with an Airport. The hardware was not on show as far as I could tell, and it looks similar to the AppleTV package, but with more Ethernet ports and no video port. Personally, I don't want two separate products in this space, I want an AppleTV that operates as a Time Capsule and Airport. I want to backup my laptop and be able to play the music and video directly from the backup copy.
I'm disappointed that there was no new iPhone hardware or Leopard upgrade, and no business oriented iPhone software. I guess that's going to be featured in the next show....
Monday, December 31, 2007
Some New Performance Monitoring Tools
There is a simple and extensible open source C based daemon called collectd that writes to RRD files, an alternative to Orca/procallator for people who don't want the Perl based memory footprint of procallator. I'll check it out on my Gumstix millicomputer sometime.
There is yet another open source full-function monitoring tool called Zabbix that looks similar to Cacti in scope, possibly with more features, and with a SQL database backend. It has a commercial company backing it with support contracts etc, somewhat like the XE Toolkit.
The most interesting commercial tool I saw at CMG earlier this month is a capacity monitoring tool called PAWZ from Perfcap Corporation. The key thing they have worked on is taking the human out of the loop as much as possible with sophisticated capacity modelling algorithms and a simple and scalable operational model. It is very similar in concept to the capacity planning research I was working on and publishing in 2002-2004. The core idea is that you care about "headroom" in a service, and anything that limits that headroom is taken into account. Running out of CPU power, network bandwidth, memory, threads etc. will increase response time of the service, so monitor them all, track trends in headroom and calculate the point in time where lack of headroom will impact service response time. At eBay we used to call this the "time to live" for a service. You can easily focus on the services that have the shortest time to live, and proactively make sure that you have a low probability of poor response time. I'm going to take a closer look at this one...
Sunday, December 30, 2007
Thoughts on iPhone 1.1.3, Macworld, 3G and business users
Reports indicate the the 1.1.3 update is likely to ship before MacWorld in January, it includes an update to Google Maps that adds the same location feature as recently shipped on other non-GPS platforms (basically locating to the nearest cell-tower). It also provides mechanisms to select and arrange the applications installed on the iPhone. This is a key new feature, since the iPhone has been updated by Apple as a monolithic all-or-nothing set of applications so far. It greatly reduces the need to hack into the iPhone to customize it.
The updates themselves are fairly small, and to me it makes sense to get them installed before MacWorld. We will then see announcements at MacWorld of a set of optional applications that users can pick and choose to install on their iPhones. This could include the ability to add the Mail application and other "missing" iPhone apps to the iPod Touch. It could include brand new applications (I'm going to nominate iChat as a candidate yet again...). The public announcement of the SDK is due at MacWorld, with shipping in February, which will open up anyone to build officially sanctioned applications. However key vendors will have been testing the SDK over the last few months so I expect a bunch of third party applications to be announced or ready to ship at MacWorld.
The other leaks and rumors indicate that there is likely to be a second generation iPhone with 3G support shipping in the spring, and announced at MacWorld. This would also support launching the iPhone in Asian markets like Japan, where there is no GSM support.
I also expect that Apple will start to make moves towards business use of the iPhone, with some tools and upgrades provided by Apple, and others by key third parties.
I currently carry a Verizon Blackberry 8703e for work use, and my iPhone for personal use and iPod functionality. In order to use the iPhone as my work phone I need a few key features.
- Firewall support - the BB is inside the corporate firewall, the iPhone can't access it. We use Juniper Network Connect which is a Java based VPN solution on MacOS/XP.
- WiFi support - we use LEAP to login to WiFi at work, need support for LEAP on iPhone, it works fine on my MacOS X laptop, should be a simple feature to add.
- Exchange support - I can't use the IMAP workaround due to firewall issues, properly integrated Exchange email and calendar support is what everyone is asking for.
- Ideally RIM will port the Blackberry application suite to the iPhone, like they did for the Treo...
The two other biggest missing features are Flash and Java support. I know there are lots of issues with CPU/memory/battery life. Perhaps the next generation iPhone will be based on a more advanced ARM CPU (e.g. the ARM Cortex based Qualcomm Scorpion) with more performance and more memory so it can run Flash and Java apps alongside the existing apps?
We'll find out in a few weeks...
Happy new year.
Friday, December 28, 2007
MacOS X Leopard, iPhone and Stereo Bluetooth Headphones - A2DP
Over a year ago I bought stereo bluetooth headphones on eBay, its a multifunction unit the OMIZ OMS600, and it includes an MP3 player (with MicroSD slot), FM radio, Stereo A2DP headphone and Headset with Mono audio/Microphone. When I tried to configure it on MacOS X Tiger it didn't work because there was no A2DP headphone support.
MacOS X Leopard now supports A2DP and "just works" with this headset. The Mac sees both the Headset (Mono audio/mic for Skype etc) and Headphone (A2DP Stereo) as separate devices. After the usual Bluetooth device wizard setup, simply put the OMS600 into the headphone mode and pick "Use Headphone" from the Bluetooth dropdown menu on the Mac, the Mac's internal speakers mute, and the Headphones play. It worked over a 10 foot range walking around a room, crackled a bit at the limit and dropped the connection if I went too far away.
I don't see the OMS600 for sale any more, but there are plenty of A2DP headphones out there now, and its nice to see that Apple finally got the devices to work, and made it "Just Work" as usual.
The headset mode also works on my iPhone, but the A2DP mode doesn't. I paired the headset with the iPhone and was able to make calls and receive them. When using the FM radio or the MP3 player built-into the headset it paused and resumed for incoming calls. However when the Headphone mode was paired with my Leopard machine the iPhone didn't route calls to the Headset. The OMS600 headset has the microphone built into the left side earpiece with no voice tube or boom down nearer my mouth, and it doesn't pick up very clearly. The noise cancelling Jawbone headset works far better.
I listened to some music on iTunes (Radiohead In Rainbows and Gorillaz D-Sides are my current albums for serial listening) and waited forever for XP to start up in Parallels so I could fire up IE7 and see a Netflix Watch Instantly show about the Pixies reunion tour (called LoudQuietLoud). [Yes I know it would be nice if it worked natively on the Mac, but the studios only approve Windows DRM, and the alternatives all have issues that are taking way too much time to sort out].
Happy new year...
Sunday, December 09, 2007
A. A. Michelson Award Acceptance Speech
Last week I was selected as this year's winner of the A. A. Michelson Award for Lifetime Achievement and Contribution by the Computer Measurement Group. Here is a transcript of my acceptance speech.

Thanks to CMG, the Michelson Award winners who voted to add me to their ranks, and everyone who I have met here since I first attended CMG in 1994. I treat the CMG conference as my annual training class, a chance to mingle with my peers, learn and share the things I have learned.
There are some people who I have met at CMG or while working in this field who have become special friends, not just people I see at the conference each year. Cathy Nolan is not only national CMG president this year, she is president of my local CMG group in Northern California and was also instrumental in the process of getting me nominated for this award. We have all learned a lot over the years from Neil Gunther. Neil and I have jointly presented many training classes so I have had plenty of opportunity to learn his material. He has made queuing theory accessible and useful for very many people. Mario Jauvin has been my CMG conference buddy for many years, and for the last three years we have jointly presented the Capacity Planning with Free Tools workshop. Yefim (Fima) Somin encouraged me to attend CMG in the first place, we met in 1994 while working to port BGS Best/1 to Solaris, and we have kept in touch over the years, even though he doesn't get to come to CMG nowadays. Fima and Henry Newman were part of the Universal Measurement Architecture standards body that I joined in 1995, and I have also kept in touch with Henry. We worked to get some of his ideas (like extended system accounting) to be implemented in Solaris. Bob Sneed worked with me at Sun and is now the main contact point and conscience for CMG at Sun.
Outside CMG there are some very significant people I would also like to thank. Brian Wong is one of my closest friends, responsible for getting me to move to the USA in 1993, and is also author of the Capacity Planning and Configuration for Solaris Servers book. Brian introduced me to Rich Pettit, and we worked together for many years to build and extend the SE Toolkit, the vehicle for most of my performance measurement ideas. Allan Packer was the co-author of my first CMG paper on Database Sizing. He knows everything about database performance and wrote the book Configuring and Tuning Databases on the Solaris Platform. We borrowed Allan from Sun's office in Adelaide Australia for a month or two, and he introduced me to Richard McDougall, who we also borrowed from Adelaide. Allan and Richard both ended up working for Sun's Performance group in California. Richard built the first tools that could measure memory usage, fixed the Solaris 8 memory system, and co-authored the Solaris Internals books. He became a Sun Distinguished Engineer and recently joined VMware as Chief Performance Architect. Jim Mauro is the other author of Solaris Internals, we borrowed him from Sun's New Jersey office a few times before he also joined the performance group full time. Finally Dave Fisk worked with Brian Wong and I at Sun, and became one of my closest friends. He knows more about Unix storage performance than anyone else I've met, and I've called on him many times (he's now a consultant) to help figure out what's really going on.
I would also like to thank my family. My father is a retired Statistics Professor from the University of Hertfordshire (a.k.a. Hatfield Polytechnic) in the UK. He was programming himself in the 1960's and gently encouraged me to tinker with computers from an early age. My son recently graduated with his BSc in Computer Science from the same University, and now works as a project manager at eBay in California. My daughter has just started a degree course in European Literature at Royal Holloway College, London, so I'm no help at all with her assignments :-) I'm very proud of them. I moved to the USA on my own a few years ago and met my lovely wife Laurel in 2003. I'm very happy that she has also been able to attend CMG with me and meet many of you.
I have always been glad that I studied Physics, the work we do in performance and the diagnosis and measurement of performance leans heavily on my training as a Physicist and I think that pure Computer Scientists are lacking some of the tools they need to understand measurement and performance issues. Since A. A. Michelson was a Physicist this award is especially gratifying for me.
In 1972 my High School had a link to the computer that my Father used at Hatfield Polytechnic. I taught myself BASIC and Algol during recess....
This is what most people know me for, its getting a little out of date now, since the second edition was published in 1998, but a big thank-you to my co-author Rich Pettit and everyone that bought a copy!
The book documented metrics and performance rules, and they were implemented as an SE Toolkit script called Virtual Adrian. I spent a lot of time with all the tools vendors helping them to manage Solaris better, and with Solaris engineering getting them to make Solaris more manageable. This is what justified my involvement with CMG during my time at Sun, and its the legacy that I am most proud of.
To finish, some of the things I have learned over the years that you may find useful...
If you ever try to write a book, you need to pay attention to Cockcroft's Law of Book Writing. It seems counter-intuitive to people who have never attempted to write a book, but it is the most important thing to know if you ever intend to finish a book. For example, you scope a 200 page book and start writing. After you have written 50 pages you re-scope and find your book is now a 300 page book. You write another 50 pages and find you are now writing a 400 page book. The number of pages you have left to write to finish the book is increasing! You have to reduce scope continuously and plot a trend line that slopes downwards if you ever hope to be published.
If you collect information and share it freely, you will become an expert. You become a magnet for questions, issues and information. Some people tightly control their expertise, this is a huge mistake as it gives others incentives to look for alternative experts or to become experts themselves. If you give out your expertise freely, you become the go-to person in your field, and gather many more recommendations from the people you help. Try it!
Some observations that may help you deal with executive level management. The first one is a big problem that Sun and other manufacturers have, because the things they build take many years to go from conception to delivery. If the managers in charge of these projects don't stick around then the project is likely to be tinkered with and delayed further or dropped. Its nice being at Netflix, where the executives have been there for years and the projects take months to deliver. The second observation is that Capacity Planners have a vertical role in the organization, they gather the lowest level metrics from operations and have to present findings to upper management. We love our detailed metrics, but the only metric that upper management should have to deal with is the dollar. If you can present everything in terms of return on investment (spend $100K to save or grow by $1M) then your executive presentations will be much more effective.
What am I working on now? I'm managing a team that develops the Netflix web site. Its a personalized web service and I think that is the future of the computer industry. The scalability and performance challenges are very interesting. I'm also pushing an idea I call "Millicomputing" which could have a disruptive effect on the industry as storage moves from disks (spinning rust) to flash, and the computers move to ultra low power system-on-a-chip devices. I gave a paper on Millicomputing at CMG07 and have a separate blog on the subject.
My paper at last year's CMG was called "Utilization is Virtually Useless as a Metric", and this answer sums it up. Those who ask questions about utilization don't understand that their questions have no meaning so the answers are irrelevant :-)
Thanks again!
Adrian
CMG07 and the A. A. Michelson Award
From the Computer Measurement Group website:
Thanks to everyone who has supported me over the years, reading the blogs, papers and books; coming to training classes; the engineers at Sun who fixed my bugs and added the features I lobbied for to make Solaris more measurable; and the friends, mentors and mentees that I have worked with for many years.
Cheers!
Adrian
Each year, CMG considers exceptional individuals who have made significant contributions to our profession over their entire professional careers as possible recipients of the A.A. Michelson Award.The recipient is nominated and chosen by the past winners, and it has been granted to one person every year since 1974. This year I was very honored to be the recipient! It was announced on Monday 3rd December, at the start of the CMG07 conference. I gave an acceptance speech, that I have summarized into a blog post.Albert Abraham Michelson, for whom the award is named, was known for his outstanding technical accomplishments in measuring the speed of light as well as for his role as a teacher and inspirer of others. CMG presents this lifetime achievement award to a single individual to recognize and encourage the same combination of technical excellence and professional contributions found in only an exceptional few.
Thanks to everyone who has supported me over the years, reading the blogs, papers and books; coming to training classes; the engineers at Sun who fixed my bugs and added the features I lobbied for to make Solaris more measurable; and the friends, mentors and mentees that I have worked with for many years.
Cheers!
Adrian
Saturday, December 01, 2007
Jobs at Netflix - One down, one to go...
I have been working on hiring for the last month or so. We found and hired one good match and had a bunch of near misses, so I have revised a reissued job description that is clearer about what we are looking for. I need one more senior Java/SQL developer with experience of large scale consumer web site development. We are looking for someone who has a lot of experience in product development, may have spent some time in management or team lead roles, but still "has their head in the code". We run small agile teams, you get to architect and code new product features yourself and deliver to the site every few weeks.
Development is very rapid, agile and iterative. Features go through rigorous statistical A/B testing, and have to show a significant benefit before all the users get to see them. There is a great deal of freedom to try out ideas and an extremely analytical approach to picking the winners.
My group develops core personalization algorithms that are used to generate lists of candidate movies and filter them down to the top few to be shown in any particular block on the site. We work on the pages grouped under the main "Browse DVDs" tab at www.netflix.com. We collaborate with a sister group that collects and predicts star ratings and runs the Netflix Prize, they work on pages under the "Movie's You'll Love" tab.
The business logic is written in Java, pulling its data out of Oracle and via middle tier services. The front end presentation layer (javascript/CSS etc.) work is done by engineers working for Bill Scott, Director of UI Engineering, who used to work at Yahoo! where he was chief AJAX evangelist. He is also hiring....
Development is very rapid, agile and iterative. Features go through rigorous statistical A/B testing, and have to show a significant benefit before all the users get to see them. There is a great deal of freedom to try out ideas and an extremely analytical approach to picking the winners.
Netflix is also something of a social experiment, the company culture is unique and very employee oriented. There is minimal process, a huge reliance on personal judgement, and zero tolerance for antisocial behaviors. What would normally be a fat binder of HR policies has been summarized into a few lines: "Act in Netflix's best interest" and the vacation policy is "take some". If you don't have the personal judgement to do the right thing, we say goodbye...
Web Engineering consists of a relatively small number of senior people working very efficiently and productively. We like to hire the best "stars" we can find and build up "bench strength" like a championship winning sports team. We are looking for some more star talent...
Here is the opening: http://jobs.netflix.com/DetailFlix.asp?flix1916
I'm hiring one engineer at this time - apply at the above link or find me in LinkedIn or FaceBook to discuss.
Wednesday, November 21, 2007
Parallel Application Performance and Tuning with OpenMP
I just had dinner with my old friend Ruud van der Pas, who works for Sun's developer tools organization. He is a world class expert in making parallelized applications scale, and recently gave a talk with a very amusing performance conversation.
Ruud is an advocate and expert in OpenMP, and has just published an excellent book on the subject. "Using OpenMP". Published by MIT Press. The work was done together with Barbara Chapman and Gabriele Jost. The ISBN numbers are ISBN-10: 0-262-53302-2 and ISBN-13: 978-0-262-53302-7.
Even if you aren't using OpenMP, the concepts for performance and scaling are the same for everyone...
Ruud is an advocate and expert in OpenMP, and has just published an excellent book on the subject. "Using OpenMP". Published by MIT Press. The work was done together with Barbara Chapman and Gabriele Jost. The ISBN numbers are ISBN-10: 0-262-53302-2 and ISBN-13: 978-0-262-53302-7.
Even if you aren't using OpenMP, the concepts for performance and scaling are the same for everyone...
Friday, November 16, 2007
Computer Measurement Group - CMG07 - San Diego - December 2-7
As usual I will be presenting at CMG07. I have attended every year since 1994, and its the place where I keep track of new developments in the performance management tools industry, as well as learning and sharing with my peers.
CMG moves around the US, last year it was in Reno NV, before that in Orlando FL.
On Sunday 2nd December there is a workshop day. I'm co-presenting (with Mario Jauvin) a half day tutorial on Capacity Planning and Performance Monitoring with Free Tools (for the third year). We updated our slides a bit more this year to track the latest versions of the free tools.
During the week I'm giving a new paper on Millicomputing, that introduces the concepts I have been discussing on my Millicomputing blog, and talks about some of the management challenges of managing large numbers of small computers.
I'm also giving a half day training class on Unix/Linux Performance Management, which is based on the materials I developed over many years in my books and talks. It is still somewhat biased towards Solaris since I know that better, but I have been adding more on Linux and AIX as that is the world I currently live in at work (Netflix runs RedHat Linux and AIX on IBM P-series) and in my Millicomputing sideline (I run Ubuntu on my development host and linux 2.6 on my gumstix).
The conference contains a wide range of content, there is:
CMG moves around the US, last year it was in Reno NV, before that in Orlando FL.
On Sunday 2nd December there is a workshop day. I'm co-presenting (with Mario Jauvin) a half day tutorial on Capacity Planning and Performance Monitoring with Free Tools (for the third year). We updated our slides a bit more this year to track the latest versions of the free tools.
During the week I'm giving a new paper on Millicomputing, that introduces the concepts I have been discussing on my Millicomputing blog, and talks about some of the management challenges of managing large numbers of small computers.
I'm also giving a half day training class on Unix/Linux Performance Management, which is based on the materials I developed over many years in my books and talks. It is still somewhat biased towards Solaris since I know that better, but I have been adding more on Linux and AIX as that is the world I currently live in at work (Netflix runs RedHat Linux and AIX on IBM P-series) and in my Millicomputing sideline (I run Ubuntu on my development host and linux 2.6 on my gumstix).
The conference contains a wide range of content, there is:
- a large vendor exhibition showing the latest tools, vendor presentations and user group meetings to talk about their next steps.
- everything from beginner level introductions to deep mathematical theory.
- coverage of networking, storage, CPUs, application level performance.
- discussion of Linux, Solaris, AIX, Windows, Mainframe and all combinations
- panel discussions on virtualization, service oriented architectures, scaling large sites
I hope to see some of my readers there. I used to give training classes for Sun on a regular basis, and this is now the main venue for me to present updates versions of those classes.
Subscribe to:
Posts (Atom)