tag:blogger.com,1999:blog-74340082024-03-12T23:43:01.439-07:00Adrian Cockcroft's BlogStarted in 2004. Covers anything I find interesting, like clouds, cars and strange complex music. Views expressed are my own and not those of my employers (currently AWS). See also @adriancoAdrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.comBlogger296125tag:blogger.com,1999:blog-7434008.post-76232213997070236122014-12-12T23:11:00.000-08:002014-12-13T00:42:35.261-08:00Interesting Videos from AWS Re:Invent 2014There are too many, but here are the ones I've watched that I found particularly interesting. I'll add to this list as I find time to watch more of them.<br />
<br />
To start with here's my own talk. I'm looking for feedback on it, as it's the first time I've tried this topic and I'm looking for more data to support the concepts I'm discussing.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/LZwlkqERv2g?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
The "must see" talk for Re:Invent is James Hamilton on Innovation at Scale.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="https://ytimg.googleusercontent.com/vi/JIQETrFC_SQ/0.jpg" height="266" width="320"><param name="movie" value="https://youtube.googleapis.com/v/JIQETrFC_SQ&source=uds" /><param name="bgcolor" value="#FFFFFF" /><param name="allowFullScreen" value="true" /><embed width="320" height="266" src="https://youtube.googleapis.com/v/JIQETrFC_SQ&source=uds" type="application/x-shockwave-flash" allowfullscreen="true"></embed></object></div>
<br />
<br />
The new EC2 Container Service that supports Docker, described by Deepak Singh<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/2vJLS8qfhI0?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
An excellent and entertaining talk on networking optimization by Becky Weiss - learn what SR-IOV means.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="https://ytimg.googleusercontent.com/vi/JUw8y_pqD_Y/0.jpg" height="266" width="320"><param name="movie" value="https://youtube.googleapis.com/v/JUw8y_pqD_Y&source=uds" /><param name="bgcolor" value="#FFFFFF" /><param name="allowFullScreen" value="true" /><embed width="320" height="266" src="https://youtube.googleapis.com/v/JUw8y_pqD_Y&source=uds" type="application/x-shockwave-flash" allowfullscreen="true"></embed></object></div>
<br />
<br />
EC2 Instance Deep Dive by John Phillips explains HVM and storage path optimization.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/ujGx0tiI1L4?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
The VPC Deep Dive by Kevin Miller talks about some long awaited features such as migration tools from EC2 Classic to VPC.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/HexrVfuIY1k?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
There were lots of Netflix talks, but from Asgard to Zuul by Ruslan included the announcement of Docker images to make trying out NetflixOSS tools trivial.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/p7ysHhs5hl0?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
One of the most popular talks at the event was Brendan Gregg of Netflix on EC2 Performance Tuning<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/7Cyd22kOqWc?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
PFC305 Embracing Failure Fault Injection and Service Reliability - Monkeys and the new FIT failure injection system that replaces Latency Monkey.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/wrY7XoOnysg?feature=player_embedded' frameborder='0'></iframe></div>
<br />
More Netflix Talks<br />
BDT403 Next Generation Big Data Platform at Netflix<br />
ARC317 Maintaining a Highly Available Front Door at Massive Scale<br />
PFC304 Effective IPC for Microservices in the Cloud<br />
ENT209 Cloud Migration, DevOps and Distributed Systems<br />
APP310 Using Apache Mesos in the Cloud<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-41158426557813174762014-04-03T12:57:00.000-07:002014-04-03T13:59:41.647-07:00Public Cloud Instance Pricing Wars - Detailed Context and AnalysisAs part of my opening keynote at <a href="http://www.interop.com/lasvegas/conference/cloud-connect.php?_mc=we_cc_le_intlv14_cportalbtn" target="_blank">Cloud Connect in Las Vegas</a> I summarized the latest moves in cloud, the slides are available via the new <a href="http://www.battery.com/powered/" target="_blank">Powered by Battery</a> site as "<a href="http://www.battery.com/powered/the-whole-stack/2014/03/18/you-need-to-act-now-tapping-the-cloud-to-more-quickly-deliver-it-services-and-lessons-from-netflix/" target="_blank">The Good the Bad and the Ugly: Critical Decisions for the Cloud Enabled Enterprise</a>". This blog post is a detailed analysis of just part of what happened.<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Summary points<o:p></o:p></div>
<div class="MsoNormal">
</div>
<ul>
<li>AWS users should migrate from obsolete m1, m2, c1, c2 to the
new m3, r3, c3 instances to get better performance at lower prices with the
latest Intel CPUs.</li>
<li>Any cloud benchmark or cost comparison that uses the AWS m1 family
as a basis should be called out as bogus benchmarketing.</li>
<li>AWS and Google instance prices are essentially the same for
similar specs.</li>
<li>Microsoft doesn’t appear to have the latest Intel CPUs generally available
and only matches prices for obsolete AWS instances.</li>
<li>IBM Softlayer pricing is still higher, especially on small
instance types</li>
<li>Google's statement that prices should follow <a href="http://en.wikipedia.org/wiki/Moore%27s_law" target="_blank">Moore’s law</a> implies that we should expect prices to halve every 18-24 months</li>
<li>Pricing pages by <a href="http://aws.amazon.com/ec2/pricing/" target="_blank">AWS</a>, <a href="https://cloud.google.com/products/compute-engine/" target="_blank">Google Compute Engine</a>, <a href="http://blogs.msdn.com/b/windowsazure/archive/2014/03/31/microsoft-azure-innovation-quality-and-price.aspx" target="_blank">Microsoft Azure</a>, <a href="http://www.softlayer.com/virtual-servers" target="_blank">IBM Softlayer</a></li>
<li>Adrian’s spreadsheet summary of instances from the above vendors at <a href="http://bit.ly/cloudinstances">http://bit.ly/cloudinstances</a></li>
<li>Analysis of the prices by <a href="http://www.rightscale.com/blog/cloud-cost-analysis/aws-responds-price-cuts-google-vs-aws-pricing-round-2" target="_blank">Rightscale</a></li>
</ul>
<br />
<div class="MsoNormal">
On Tuesday 25<sup>th</sup> March 2014 Google announced some
new features and steep price cuts, the next day Amazon Web Services also announced new features and matching price cuts. On Monday 31<sup>st</sup> March Microsoft Azure also reduced prices. Many
pundits repeated talking points from press releases in blog posts but unfortunately
there was little attempt to understand what really happened, and explain the context and outcome. When I wrote up a summary for my opening keynote at Cloud
Connect on 31<sup>st</sup> March I looked at the actual end result and came up
with a different perspective and a list of gaps.</div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
I’m only going to discuss instance types and on-demand
prices here. There was a lot more in the announcements that other people have
done a good job of summarizing. The <a href="http://www.rightscale.com/blog/cloud-cost-analysis/aws-responds-price-cuts-google-vs-aws-pricing-round-2" target="_blank">Rightscale blog</a> linked above also gives an
accurate and broader view on what was announced. I will discuss other pricing
models beyond on-demand in future blog posts.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
There are some things you need to know to get the right
background context for the instance price cuts. The most important is to
understand that AWS has two generations of instance types, and is in a
transition from Intel CPU technology they introduced five or more years ago to
a new generation introduced in the last year. The new generation CPUs are based
on an architecture known as <a href="http://en.wikipedia.org/wiki/Sandy_Bridge" target="_blank">Sandybridge</a>. The latest tweak is called Ivybridge
and has incremental improvements that give more cores per chip and slightly
higher performance. Since Google is a recent entrant to the public cloud
market, all their instances types are based on Sandybridge. To correctly
compare AWS prices and features with Google, there is a like-for-like
comparison that can be made. AWS is encouraging the transition by pricing its
newer faster instances at a lower cost than the older slower ones. In the
recent announcement, AWS cut the prices by obsolete instance type families by a
smaller percentage than the newer instance type families, so the gap has just widened.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Old AWS instance types have names starting with m1, m2 and
c1, c2. They all have newer replacements known as m3, r3 and c3 except the
smallest one – the m1.small. The newer instances have a similar amount of RAM
and CPU threads, but the CPU performance is significantly higher. The new
equivalents also replace small slow local disks with smaller but far faster and
more reliable solid-state disks, and the underlying networks move from 1Gbit/s
to 10Gbit/s. The newer instance families should also have lower failure rates.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Most people are much more familiar with the old generation
instance types, and competitors write their press releases they are able to get
away with claiming that they are both faster and cheaper than AWS, by comparing
against the old generation products. This is an old “benchmarketing” trick –
compare your new product against the competitions older and more recognizable
product.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
For the most commonly used instance types there is a close specification
match between the AWS m3 and the Google n1-standard. They are also exactly the
same price per hour. Since AWS released its changes after Google, this implies
that AWS deliberately matched Google’s price. The big architectural difference
between the vendors is that Google instances are diskless, all their storage is
network attached, while AWS have various amounts of SSD included. The AWS
hypervisor also makes slightly more memory available per instance, and ratings
for the c3 imply that AWS is supplying a slightly higher CPU clock rate for that
instance type. I think that this is because AWS has based its compute intensive
c3 instance types on a higher clock rate Ivybridge CPU rather than the earlier
Sandybridge specification. For the high memory capacity instance types it is a
little different. The Google n1-himem instances have less memory available than
the AWS r3 equivalents, and cost a bit less. This makes intuitive sense as this
instance type is normally bought for its memory capacity.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Microsoft previously committed to match AWS prices, and in
their announcement their comparisons matched the m1 range exactly at it’s new
price, and they compared their memory oriented A5 instance as cheaper than an
old m2.xlarge, but the A5 is an older slower CPU type, more expensive ($0.22 vs
$0.18) and has less memory (14GB vs. 15GB) than the AWS r3.large. The common
CPU options on Azure are aligned with the older AWS instance types. Azure does have Intel Sandybridge CPUs for compute use cases as the A8 and A9 models, but I couldn't find list pricing for them and they appear to be a low volume special option. The
Azure pricing strategy ignores the current generation AWS product, so the price
match guarantee doesn’t deliver. In addition the Google and AWS price changes were effective from April 1<sup>st</sup>, but Azure takes effect May 1<sup>st</sup>.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
IBM Softlayer has a choose-what-you-want model rather than a
specific set of instance types. The smaller instances are $0.10/hr where AWS
and Google n1-standard-1 are $0.07/hr. As you pick a bigger instance type on
Softlayer the cost doesn’t scale up linearly, while Google and AWS double the
price each time the configuration doubles. The Softlayer equivalent of the
n1-standard-16 is actually slightly lower cost than Google. Softlayer pricing
on most instances is in the same ballpark as AWS and Azure were before the cuts,
so I expect they will eventually have to cut prices to match the new level.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Gaps and Missing Features<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
The remaining anomaly in AWS pricing is the low-end
m1.small. There is no newer technology equivalent at present, so I wouldn’t be
surprised to see AWS do something interesting in this space soon. Generally AWS
has a much wider range of instances than Google, but AWS is missing an
m3.4xlarge to match Google's n1-standard-16, and the Google hicpu range has
double the CPU to RAM ratio of the AWS c3 range so they aren’t directly
comparable.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Google has no equivalent to the highest memory and CPU AWS
instances, and has no local disk or SSD options. Instead they have
better attached disk performance than AWS Elastic Block Store, but attached
disk adds to the instance cost, and can never be as fast as local SSD inside the instance.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Microsoft Azure needs to refresh its instance type options,
it has a much smaller range, older slower CPUs, and no SSD options. It doesn’t
look particularly competitive.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Conclusion<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
If you buy hardware and capitalize it over three years, and later on there is a price cut; you don’t get to reduce your monthly costs. Towards the
end your CPUs are getting old, leading to less competitive response times and
higher failure rates. With public cloud vendors driving the costs down several
times a year and upgrading their instances, your model of public vs. private
costs needs to factor in something like Moore’s law for cost reductions and a
technology refresh more often than every three years. Google actually said we
should expect Moore’s law to apply in their announcement, which I interpret to
mean that we can expect costs to halve about every 18-24 months. This isn’t a race
to zero; it’s a proportional reduction every year. Over a three-year period the
cost at the end is a third to a quarter of the cost at the start.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
I still hear CIOs worry that cloud vendor lock-in would let
them raise prices. This ruse is used to justify private cloud investments. Even
without switching vendors, you will see repeated price reductions for the public
cloud systems you are already using. This was the 42<sup>nd</sup> price cut for
AWS, the argument is ridiculous.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<!--[if gte mso 9]><xml>
<o:DocumentProperties>
<o:Revision>0</o:Revision>
<o:TotalTime>0</o:TotalTime>
<o:Pages>1</o:Pages>
<o:Words>1298</o:Words>
<o:Characters>7403</o:Characters>
<o:Company>Battery</o:Company>
<o:Lines>61</o:Lines>
<o:Paragraphs>17</o:Paragraphs>
<o:CharactersWithSpaces>8684</o:CharactersWithSpaces>
<o:Version>14.0</o:Version>
</o:DocumentProperties>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]-->
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>JA</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
<w:UseFELayout/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="276">
<w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" Priority="39" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" Name="toc 9"/>
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]-->
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:Cambria;
mso-ascii-font-family:Cambria;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Cambria;
mso-hansi-theme-font:minor-latin;}
</style>
<![endif]-->
<!--StartFragment-->
<!--EndFragment--><br />
<div class="MsoNormal">
I’ve <a href="http://www.slideshare.net/adrianco/yowworkshop-131203193626phpapp01-1" target="_blank">previously published presentation materials on costoptimization with AWS</a>. I’m researching this area and over the coming months
will publish a series of posts on all aspects of cloud optimization. <o:p></o:p></div>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com3tag:blogger.com,1999:blog-7434008.post-13101620786600596552014-03-03T01:16:00.001-08:002014-03-03T01:16:03.936-08:00Speeding Up Innovation - Cloud Expo Europe and QCon London 2014There is a huge transformation going on in enterprise IT at the moment. Decades old companies are discovering that IT which used to be a small side operation keeping track of things like employees and inventory, is now perhaps a thousand times more demanding and is central to their ability to compete.<br />
<br />
<ul>
<li>From tracking inventory levels at sales outlets to handling web based sales directly.</li>
<li>From managing a few employees to managing huge numbers of connected devices and mobile applications.</li>
<li>From placing print adverts in newspapers and trade journals each month to using big data analytics to do real time bidding on internet adverts and sales force optimization.</li>
</ul>
<br />
Technologies and practices that have been developed in the leading web scale companies are finding their way into cloud based services that address the broadening of demand for these technologies, while packaging them to make them easier to implement. In addition, the pace of product development has greatly increased, and the ability to get faster feedback from customers and act on it conveys a significant competitive advantage.<br />
<br />
The opening keynote talk I gave at Cloud Expo Europe on Feb 26th is aimed at setting the context and explaining why management needs to act. It ends with a few pointers towards next steps. The slideshare below doesn't include the builds that unfold with the explanation, and there isn't a video of the event, but I will be giving similar talks at other events in the coming months.<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="356" marginheight="0" marginwidth="0" scrolling="no" src="http://www.slideshare.net/slideshow/embed_code/31799721?rel=0" style="border-width: 1px 1px 0; border: 1px solid #CCC; margin-bottom: 5px; max-width: 100%;" width="427"> </iframe> <br />
<div style="margin-bottom: 5px;">
<strong> <a href="https://www.slideshare.net/adriancockcroft/speeding-up-31799721" target="_blank" title="Speeding Up Innovation">Speeding Up Innovation</a> </strong> from <strong><a href="http://www.slideshare.net/adriancockcroft" target="_blank">Adrian Cockcroft</a></strong> </div>
<br />
On Thursday next week I will be giving a developer oriented talk at <a href="http://qconlondon.com/" target="_blank">QCon London</a> "Migrating to Microservices". The talk goes into details on transitional techniques, architectures and tools that can be used to move a product development organization from the old world to a highly agile and innovative cloud based operation. This is an excellent conference and registration is still open, so I hope to see you there.<br />
<br />
In my new role at Battery Ventures, we are looking for opportunities to fund companies in the enterprise IT space that can support the transition to this new world.<br />
<br />
<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-19979311196183969162014-02-12T08:52:00.001-08:002014-02-12T08:52:20.591-08:00Battery Ventures at Strata, Santa Clara<br />
<div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="color: black;"><span style="background-color: rgba(255, 255, 255, 0);"><a href="http://strataconf.com/strata2014">Strata</a>’s theme this year is “Making Data Work”. The conference is sold out, but some of the sessions will be available <a href="http://strataconf.com/strata2014/public/content/video">as a live stream</a> so you can follow along at home.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="background-color: rgba(255, 255, 255, 0);">Battery Ventures kicked off the conference the night before, with friends and portfolio companies getting together nearby. For me attending Strata is a great way to keep track of what is going on in the rapidly developing Big Data space.<o:p></o:p></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="color: black;"><span style="background-color: rgba(255, 255, 255, 0);"><a href="x-apple-data-detectors://2" x-apple-data-detectors-result="2" x-apple-data-detectors-type="calendar-event" x-apple-data-detectors="true">On Wednesday at 1:30pm</a> Mike Dauber of Battery Ventures is moderating <a href="mailto:http://strataconf.com/strata2014/public/schedule/detail/33146">“Not your typical VC panel”</a>. (Follow Mike <a href="http://www.twitter.com/dauber">@dauber</a><span class="MsoHyperlink" style="text-decoration: underline;">)</span><o:p></o:p></span></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt 0.5in;">
<span style="background-color: rgba(255, 255, 255, 0);"><i>A group of VCs who invest from very early, through later stage investments talk about all things Big Data. There will be no “3 Vs” discussion here. The Panelists are committed to making this a lively discussion about topics ranging from the typical (what sectors do they want to invest in) to the atypical (what’s out there that they don’t like? Which Hadoop Distros are in trouble? Should any entrepreneur care about Data Scientists?).</i><i><o:p></o:p></i></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="color: black;"><span style="background-color: rgba(255, 255, 255, 0);"><a href="x-apple-data-detectors://3" x-apple-data-detectors-result="3" x-apple-data-detectors-type="calendar-event" x-apple-data-detectors="true">On Thursday at 4:00pm</a> I’m a panelist in<a href="http://strataconf.com/strata2014/public/schedule/detail/33544">“The Great Debate: Technology Creates More Jobs than it Destroys”</a>. (Follow me<a href="http://www.twitter.com/adrianco">@adrianco</a><span class="MsoHyperlink" style="text-decoration: underline;">)</span><o:p></o:p></span></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt 0.5in;">
<span style="background-color: rgba(255, 255, 255, 0);"><i>Is technology creating new jobs, and ridding us of drudgery? Or is it spawning an era of rampant unemployment and class divides? That’s what we’ll be debating.</i><i><o:p></o:p></i></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="background-color: rgba(255, 255, 255, 0);">Strata also features Battery Ventures portfolio companies as sponsors:<o:p></o:p></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt 0.5in;">
<span style="color: black;"><span style="background-color: rgba(255, 255, 255, 0);"><a href="http://www.platfora.com/">Platfora</a> masks the complexity of Hadoop, making it easy for customers to understand all the facts in their business across events, actions, behaviors and time.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt 0.5in;">
<span style="color: black;"><span style="background-color: rgba(255, 255, 255, 0);"><a href="http://www.continuuity.com/">Continuuity</a> makes it easy for any Java developer to build, deploy, scale and manage Apache Hadoop and HBase applications in the cloud or on-premise.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="background-color: rgba(255, 255, 255, 0);">We look forward to catching up with everyone and finding new and interesting people and products to discuss.<o:p></o:p></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="background-color: rgba(255, 255, 255, 0);"><br /></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
Coming up next for me is a trip to the UK. I'm presenting at Cloud Expo Europe and QCon London. </div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="background-color: rgba(255, 255, 255, 0);"><br /></span></div>
<div class="MsoNormal" style="margin: 0in 0in 0.0001pt;">
<span style="background-color: rgba(255, 255, 255, 0);"><br /></span></div>
</div>
<span id="OLK_SRC_BODY_SECTION" style="background-color: rgba(255, 255, 255, 0);"></span><br />
<div style="-webkit-composition-fill-color: rgba(175, 192, 227, 0.235294); -webkit-tap-highlight-color: rgba(26, 26, 26, 0.301961); -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; color: black; font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: 300; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">
<br /></div>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-74557090108200304212014-01-30T10:02:00.001-08:002014-01-30T17:59:42.045-08:00Latest Local Music - Fractal, Beautiful Machines, Cash Pony, Moetar and Atomic Ape... and Mirthkon!There is a huge amount of good original new music that is looking for an audience. Most of what I listen to is from bands you've probably never heard of, so I'm going to share some updates and some local gigs that are happening over the next few days in the SF Bay Area. I encourage you to go to the gigs and buy a CD, or buy the album online to support local musicians. There is a <a href="https://www.facebook.com/groups/203618604214" target="_blank">Facebook group for Bay Area Prog rock</a> that's worth tracking.<br />
<br />
Earlier this week <a href="https://www.facebook.com/fractalcontinuum" target="_blank">Fractal released the first track of a free album of improvisations</a>. They plan to release the <a href="https://soundcloud.com/nicroozeboom/sets/fractiles" target="_blank">tracks on soundcloud around once a month</a>. It's hard to categorize what they sound like other than excellent musicians composing instrumentals in real time. They tend to have interesting sound effects, looped phrases and are known for using odd time signatures. I've been a non-playing member of this band since they started, helping in various ways including naming this album "Fractiles".<br />
<br />
I've been meaning to write this for a while, and my deadline is that <a href="https://www.facebook.com/events/468529613251183/" target="_blank">tonight (Thursday Jan 30th) there is a gig</a> including <a href="https://www.facebook.com/BeautifulMachines" target="_blank">Beautiful Machines</a> in San Francisco. I can't attend, but I do have their first album and have been listening to it a lot over the last few months. They write songs with influences from 80's synth-pop bands, and remind me of Japan or Duran Duran perhaps. Their bass player Van is one of my favorite local musicians, he used to play in <a href="http://www.youtube.com/watch?v=v4xqehWGgKc" target="_blank">Headshear and has a unique and interesting style on fretless bass</a>. Here's a trailer for a video of Animammal from their album "Disconnect :: Reconnect".<br />
<br />
<iframe allowfullscreen="" height="281" mozallowfullscreen="" src="//player.vimeo.com/video/85324703" webkitallowfullscreen="" width="500"></iframe>
<br />
<br />
On <a href="https://www.facebook.com/events/493240727463599/" target="_blank">Saturday I will be in Oakland for a gig with three of my favorite local bands</a>. Please come along!<br />
<br />
<a href="https://www.facebook.com/MoeTar" target="_blank">Moetar are a pop band</a> with a unique sound that blends jazzy female vocals and prog rock instrumentation into some very commercial sounding songs. They don't sound like Muse, but if you like Muse I think you will like Moetar. They have a unique mix, and I've played their first album regularly over the last year or two. Their second album has been recorded and is due for release soon. I've seen them live several times, and try not to miss a gig, they are awesome and deserve to be a big hit. Here's a video they made for <i>Butchers of Baghdad</i> from their first album <i>From These Small Seeds</i>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/a5bYz78ZpWk?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
<a href="https://www.facebook.com/cashponyband" target="_blank">Cash Pony</a> have played with Moetar several times, and I bought their CD at a gig, then just kept playing it. It's an interesting mix of instruments, with a bass sax and an electric sitar on some songs. It's interesting uptempo pop songs, and they have a very smooth sound on the CD, but I can't figure out what to compare it to. Take a listen to a live version of <i>1000 Layer Curry Dawg</i> and get their <i>Carpal Tunnel Vision Quest</i> album.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/JrEcVIhgkdM?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
Years ago, there was an amazing local band called Estradasphere. They released several albums that I still listen to regularly. They have dispersed, and the guitarist Jason Schimmel has a <a href="http://www.atomicapeband.com/" target="_blank">new band called Atomic Ape</a> with a new album, <i>Swarm</i>. I haven't heard them before, but am looking forward to seeing them live for the first time on Saturday. <a href="http://www.youtube.com/watch?v=A5sIXUUMQGw" target="_blank">Here's a rather dark video of them</a> playing for a radio show.<br />
<br />
Update - I just found out that on Feb 15th MirthKon and Secret Chiefs 3 are <a href="https://www.songkick.com/events/18750234" target="_blank">playing a gig in San Francisco</a>. Not yet sure if I can make it to attend. Yet another of my favorite local bands, intensely complex virtuoso playing. Matt the bass player in Mirthkon is also the keyboard player in Moetar. If you like Frank Zappa you'll like MirthKon, and you should give the video below a listen.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/NK-oo-tbFvM?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
<br />
<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-68231249208019080832014-01-07T07:32:00.001-08:002014-01-14T12:05:07.697-08:00Starting a new role at Battery VenturesCareful what you wish for. Take some side interests and activities that
aren't central to your real job, have a bit too much fun with them,
mention to <a href="http://www.battery.com/our-team/member/mike-dauber/" target="_blank">a friend</a> that it would be cool to do them for real, and have
him convince his company to create a completely new position tailored for
you that they've never had before, and haven't heard of anyone else doing. In a week.<br />
<br />
Over the last few years I've given a lot of talks, and the industry reaction to Netflix's all-in public cloud strategy has evolved from <i>"It won't work"</i> through <i>"It only works for unicorns like Netflix"</i> to <i>"How do we get there?"</i> My answer to that question is the Netflix Open Source Software (NetflixOSS) platform, consisting of around forty distinct projects shared at <a href="http://netflix.github.com/" target="_blank">netflix.github.com</a> that provides an on-ramp for applications to move to a cloud native architecture.<br />
<br />
I have presented aspects of this work to different audiences at public conferences, private events and directly within companies. For developers, I concentrated on explaining the NetflixOSS components, DevOps and continuous delivery patterns. For managers I concentrated on how an open source and cloud native strategy delivers rapid product evolution, scalability and high availability. As interest from individual companies grew it became clear that there was an opportunity to take what I had learned and apply it directly to a wider audience.<br />
<div>
<br /></div>
<div>
In my new role at Battery Ventures I will be continuing my conversations with large enterprise companies planning a move to cloud native, and SaaS vendors who are re-architecting for scale, to understand the gaps and demands of this market. With that context I will work in the Battery Ventures team that is looking for opportunities to fund companies who are enabling the transformation of enterprise IT, and I will also provide advice and mentoring for portfolio companies.<br />
<br />
The role is called <i>Technology Fellow</i>. It's different to the short term Entrepreneur in Residence (EIR) position, and is similar to full time investing staff. Battery just raised their tenth fund, and have a total of $900M to invest. My job is to help spend that wisely and get the best return. That also means even more conference presentations, more international coverage, and a broader range of subjects to discuss.<br />
<br />
I will no longer explain how Netflix works, so the starting point for my new presentation materials is going to be something like <a href="http://perfcap.blogspot.com/2013/12/velocity-and-volume-speed-wins-keynote.html" target="_blank">my Flowcon talk</a> with the Netflix specific parts removed. If you still want a Netflix Architecture overview talk, start with <a href="https://twitter.com/jedberg" target="_blank">@jedberg</a>, he's a very highly rated speaker, and he can also connect you to the many excellent speakers at Netflix that cover their own specific areas.<br />
<br />
It¹s going to take me a little while to figure out the new role, but a big part of the job is meeting lots of people. I'm taking bookings for February...<br />
<br />
<h4>
Press Coverage</h4>
<a href="http://www.marketwired.com/press-release/Adrian-Cockcroft-Joins-Battery-Ventures-as-Technology-Fellow-1866835.htm" target="_blank">Press Release</a> by Battery Ventures<br />
<a href="http://thewholestack.com/2014/01/07/cold-pizza-and-aws-adrian-cockcroft-joins-battery-from-netflix/" target="_blank">The Full Stack blog post</a> by Mike Dauber of Battery Ventures<br />
<a href="http://www.businessweek.com/articles/2014-01-07/netflix-loses-its-cloud-guru-to-a-vc-firm" target="_blank">Business Week</a> story by Ashlee Vance<br />
<a href="http://gigaom.com/2014/01/07/netflixs-cloud-architect-adrian-cockcroft-is-leaving-to-join-battery-ventures/" target="_blank">GigaOm story</a> by Stacy Higginbotham<br />
<a href="http://uk.news.yahoo.com/battery-ventures-lands-netflix-executive-cockcroft-200627603--sector.html" target="_blank">Reuters story on Yahoo UK</a><br />
<a href="http://www.boston.com/business/innovation/blogs/inside-the-hive/2014/01/07/battery-ventures-hires-netflix-executive/X83gBm94A1MNsQtTyGUsMK/blog.html" target="_blank">Boston Globe story</a> by Michael Farrell<br />
<a href="http://blogs.wsj.com/venturecapital/2014/01/07/netflix-cloud-expert-joins-battery-ventures-to-help-startups-tablets-expert-departs/" target="_blank">Wall Street Journal story</a> by Deborah Gage<br />
<a href="http://www.theregister.co.uk/2014/01/08/netflix_cockcroft_battery_ventures/" target="_blank">The Register story</a> by Jack Clark<br />
<a href="http://venturebeat.com/2014/01/07/netflix-cloud-architect-to-share-his-wisdom-from-new-post-at-venture-firm/" target="_blank">Venture Beat story</a> by Jordan Novet<br />
<h4>
About Battery Ventures</h4>
Battery invests in cutting-edge, category-defining businesses in markets including software and services, Web infrastructure, e-commerce, digital media and industrial technologies. Founded in 1983, the firm backs companies at stages ranging from seed to buyout and invests globally from offices in Boston, Silicon Valley and Israel. To learn more, visit <a href="http://ctt.marketwire.com/?release=1062754&id=3586642&type=1&url=http%3a%2f%2fwww.battery.com%2f" target="_blank">www.battery.com</a> and follow the firm on Twitter <a href="https://twitter.com/BatteryVentures" target="_blank">@BatteryVentures</a>.</div>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com14tag:blogger.com,1999:blog-7434008.post-85509537435522472532013-12-26T13:44:00.001-08:002013-12-26T13:45:34.344-08:00Looking back at 2013, with pointers to 2014<br />
I wrote a prediction post a year ago so I'm going to review and update it. Here's last year in full "<a href="http://perfcap.blogspot.com/2013/01/looking-back-at-2012-with-pointers-to.html" target="_blank">Looking back at 2012, with pointers to 2013</a>" .<br />
<br />
The headlines from last year, with comments and updates:<br />
<br />
<i>Mobile Bandwidth Greater than Fixed Bandwidth</i><br />
This trend continues. LTE can get congested in cities, but the latest news is <a href="http://gigaom.com/2013/12/05/verizon-quietly-unleashes-its-lte-monster-tripling-4g-capacity-in-major-cities/" target="_blank">Verizon upgrading it's network</a> to have more capacity and speeds up to 100Mbit/s. I'm about to get an LTE MiFi for our house so that when we want higher speed or our local DSL gets congested, it's available, and when we go away we can take it with us. I'll keep the DSL for background connectivity, and to avoid hitting the MiFi bandwidth cap too much.<br />
<br />
<br />
<i>Cutting The Cable/Satellite TV Feed</i><br />
<br />
We still don't have cable/satellite. I got a Google Chromecast, but it's slow and fiddly to use compared to the AppleTV, and buggy for streaming Pandora, keeps dropping the stream. The picture quality is good though.<br />
<br />
<br />
<i>The Netflix Open Source Cloud Platform Got Traction</i><br />
<br />
We close out 2013 with 39 distinct projects at <a href="http://github.netflix.com/">github.netflix.com</a>, a successful Netflix Cloud Prize contest, endorsements from many more companies including IBM, and growing acceptance that Cloud Native is an important concept that supports highly agile continuous delivery, and NetflixOSS is an onramp that accelerates transitioning to Cloud Native.<br />
<br />
<br />
<i>Netflix Cloud Architecture Presentations</i><br />
<br />
I presented even more than in 2012, see the slides I posted at <a href="http://www.slideshare.net/adrianco">http://www.slideshare.net/adrianco</a>. I've also got a permanent link to a full set of workshop slides at <a href="http://bit.ly/netflix-workshop">bit.ly/netflix-workshop</a> which is easier to remember, and lets me update the workshop slides from time to time.<br />
<br />
<br />
<i>The Concept of Anti-Fragility Took Off</i><br />
<br />
<a href="http://www.amazon.com/Antifragile-Things-That-Disorder-ebook/dp/B0083DJWGO/ref=tmm_kin_title_0">Taleb's book</a> and concept became more accepted. Ariel Tseitlin wrote an ACM paper for Netflix on <a href="http://queue.acm.org/detail.cfm?id=2499552" target="_blank">The Antifragile Organization</a>.<br />
<br />
<br />
<i>Cloud, Open Source, SaaS and the End of Enterprise Computing</i><br />
<br />
<div>
"<i>During 2013 we will see if Google manages to invest heavily and execute well enough to build up a big user base.</i>"</div>
<div>
Google came out of beta and closed some gaps, but it's not clear that they are building up a big user base. They have their fans, and in some areas have some technical advantages, but still have a lot to prove. Other public cloud vendors didn't make much headway. Microsoft Azure remains strong in it's own ecosystem, but hasn't broken out into general use, and others are getting further behind or being bought.</div>
<div>
<br /></div>
<div>
"<i>I personally think in 2014 we will likely see [...] the scale, features and price point of AWS and Google clouds make everyone else irrelevant.</i>" I still think this is true. The 2013 Garner Magic Quadrant for IaaS didn't include Google as they were in beta, but showed AWS as dominant. It also included an estimate that AWS delivered capacity was five times bigger than the next 14 vendors combined. i.e. AWS was 85% of the market by delivered capacity (not by $ revenue). My tracking of AWS size by looking at their reserved IP address space <a href="http://bit.ly/awsiprange">bit.ly/awsiprange</a> continues to show that AWS is doubling in size every year, and has grown 10x over the last three years, reaching 5.1 million IP addresses in September 2013.<br />
<br />
Most big enterprise companies are actively working on their AWS rollout now. Most of them are also trying to get an in-house cloud to work, with varying amounts of success, but even the best private clouds are still years behind the feature set of public clouds, which is has a big impact on the agility and speed of product development.<br />
<br />
<br />
<i>Solar Powered Electric Cars Are For Real Now</i><br />
<br />
Our Nissan Leaf is getting towards the end of it's three year lease, and we're replacing it with a Fiat 500e. The Fiat is smaller and lighter, but has the same size battery, so gets a bit more range. It's also cheaper and more fun to drive. During 2013 a lot of people bought Tesla Model S, including people who traded in Tesla Roadsters. We picked up a second hand 2010 Tesla Roadster with full factory warranty, and although the technology is a bit older, it's a great fun car with over 200 miles real world range for longer trips. Even with two electric cars, we still generated more electricity than we used this year, so the marginal cost of energy is still zero for our household and our gasoline spend is way down.<br />
<br />
<br />
<i>Global Warming Arrived in the USA in 2012</i><br />
<br />
"<i>I'm going to try and re-balance my 401K retirement accounts to divest from oil companies. Many students are now<a href="http://gofossilfree.org/">pressuring their colleges</a> to divest from oil as well.</i>" I spent a few hours on Fidelity Investments web site and reduced my investments in the energy sector to a minimum. The divestment movement is also gathering momentum. The public conversation continues to shift, more extreme weather in the US and worldwide is helping, and the <a href="http://ipcc.ch/" target="_blank">IPCC</a> released an updated report.<br />
<br />
<br />
<i>Twitter and Snapchat</i><br />
<br />
I had 6,500 followers on Twitter at the end of 2012, and I have 10,500 at the end of 2013. I correctly predicted that Snapchat would continue to grow in 2013, and it was reported that more photos are uploaded per day into Snapchat than into Facebook. Twitter had it's IPO, and is becoming part of the news and entertainment infrastructure with it's own ecosystem. I think they will figure out how to continue to grow and make money, so I bought a few shares to have skin in the game.</div>
<div>
<br /></div>
<div>
New for 2014:</div>
<h4>
Google Glass will have a successful public launch</h4>
<div>
I got Glass last summer, and have been using it a bit and letting other people try it a lot. I just got a hardware update that makes my developer set compatible with the final consumer version, and it's clear that Google is getting much closer to having a real product to launch. No-one knows the price, and that will determine how widely people get Glass, but the feature set and support is now quite interesting. There is a generational divide, in that many younger people like and want Glass, and older people are more wary or bothered by it. Trying it out in person lets people understand what it does and doesn't do, and greatly increases acceptance.</div>
<div>
<br /></div>
<div>
The Glass features that I was waiting for have mostly been addressed. The MyGlass app now supports iPhone, there is support for corporate Gmail accounts and multiple Gmail accounts on a single Glass. There is a headphone to supplement the built in speaker that was too quiet, and there will soon be prescription lens support. The last of those is the main reason I don't wear Glass every day, as I have to put in contact lenses to use it currently, and my contacts don't work as well as my glasses.</div>
<div>
<br /></div>
<div>
The ability to get personal GPS directions while walking (or cycling) is one of my favorite features. "First person" hangout support has huge potential although it's still too fiddly to setup, and needs good network bandwidth. Video use drains the battery quickly, although in normal use it lasts long enough to be useful all day. The add-on applications I have installed include Twitter, so I get notified immediately if someone mentions or DMs me; Evernote for keeping track of shared to-do lists; and Translate where you look at a sign and it makes an English version of the sign for you.</div>
<div>
<br /></div>
<div>
Voice control works better than most people expect, directions to local places is remarkably good, but voice input dictation is very random. It needs quite a lot of practice to get messages recorded that contain more than "on my way" or other simple phrases. But then I don't use Siri on my iPhone either.</div>
<div>
<br /></div>
<div>
Best wishes to everyone for 2014.</div>
<div>
<br /></div>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com2tag:blogger.com,1999:blog-7434008.post-51301198488565238652013-12-19T12:42:00.000-08:002013-12-19T12:43:48.628-08:00Velocity and Volume - Speed Wins - Keynote at FlowconNovember 1st was the first ever Flowcon, held in San Francisco, a new event to focus on continuous delivery related development topics.<br />
<br />
I was honored to be asked to present as the opening keynote speaker, and I owe a big thank you to Jez Humble and Gene Kim for the invitation.<br />
<br />
The material I presented was less focused on Netflix related technologies than most of my talks, and instead looks at the challenges and motivations of speeding up the page of development. I also tried to provide a historical perspective of how the state of the art for software delivery has changed over the last few decades.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/wyWI3gLpB8o?feature=player_embedded' frameborder='0'></iframe></div>
<br />
There was a second chance to present these slides with a bit more time a few days later, and the slides for that are linked below. There's a bit more of an introduction, and a more discussion of tools on the end, but it's basically the same message.<br />
<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="356" marginheight="0" marginwidth="0" scrolling="no" src="http://www.slideshare.net/slideshow/embed_code/28103959?rel=0" style="border-width: 1px 1px 0; border: 1px solid #CCC; margin-bottom: 5px;" width="427"> </iframe> <br />
<div style="margin-bottom: 5px;">
<strong> <a href="https://www.slideshare.net/adrianco/flowcon-added-to-for-cmg-keynote-talk-on-how-speed-wins-and-how-netflix-is-doing-continuous-delivery" target="_blank" title="Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is doing Continuous Delivery">Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is doing Continuous Delivery</a> </strong> from <strong><a href="http://www.slideshare.net/adrianco" target="_blank">Adrian Cockcroft</a></strong> </div>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-77637932203498766982013-11-24T12:24:00.000-08:002013-11-24T12:24:50.390-08:00Free Communication is Disrupting HierarchiesJeff Jarvis <a href="http://buzzmachine.com/2013/08/08/give-up-on-the-net/" target="_blank">recently blogged</a>:<br />
<blockquote class="tr_bq">
<span style="background-color: #fdfdfd; font-family: 'Open Sans', 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 25px;"><i>I doubt the net’s creators realized how subversive it was to connect anyone to anyone, bypassing the institutions that mediated those connections: from media to government, universities to retailers. These institutions are now circling wagons to protect their prerogatives: copyright for media, secrecy for government.</i></span></blockquote>
<br />
Somewhat related discussions are taking place at the <a href="http://defragcon.com/2013/bigger-networks-of-smaller-things-part-2/" target="_blank">Defrag conference</a> and on <a href="http://switchandshift.com/the-death-of-workplace-hierarchy" target="_blank">blogs talking about</a> corporate organizations and <a href="http://www.theguardian.com/commentisfree/2013/sep/05/government-betrayed-internet-nsa-spying" target="_blank">government surveillance</a>.<br />
<br />
However this is nothing new, over centuries there has been a gradual reduction in the cost of communication between people, which has caused social changes, disrupted old industries, created new industries and led to political revolutions. The end point that we have reached in the last few years is that the cost of communication is now zero. There is no incremental cost for me to send a tweet that is delivered to thousands of people a few seconds later. If the tweet contains novel ideas or information that people want to share, then in a few seconds, millions of people could see it. With about 10,000 direct followers my own twitter mention reach was in the millions last week.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPpHk-fbNvwKpQDzh-NAJLozT2AWOlirXSKUUtuGCLY38PWQVGriUaV0gXjyn7lajXgXDkyxo5fidf8u4p-ETaBHB2J7BuR_Jt_m3e_veT2MvFqGMHJFBCMxAs3hv8eUg3qzwy/s1600/Screen+Shot+2013-11-24+at+11.56.23+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="80" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPpHk-fbNvwKpQDzh-NAJLozT2AWOlirXSKUUtuGCLY38PWQVGriUaV0gXjyn7lajXgXDkyxo5fidf8u4p-ETaBHB2J7BuR_Jt_m3e_veT2MvFqGMHJFBCMxAs3hv8eUg3qzwy/s320/Screen+Shot+2013-11-24+at+11.56.23+AM.png" width="320" /></a></div>
<br />
<br />
When I see an industry or organization that was built on assumptions that are no longer true, I expect to see disruption follow, and opportunities to create new industries and organizations. <b>The assumption that communication is expensive is baked into our world. The new industries and organizations that take advantage of this change are emerging now.</b><br />
<br />
To see how subversive this is, we need to look at some history. Every step in the reduction of communication cost has enabled changes that have been resisted by the hierarchy, led to revolutions, and in most cases have been co-opted and controlled by the elite.<br />
<br />
The ability to control, coordinate and influence a large group of people depends upon effective communication. Early examples include the creation of sea-going ships and roads to sustain empires, from the Romans to the British Empire as examples. The leaders of these empires could gather information via messengers and send out decrees to manage at a distance, by using a coordinated hierarchical communication network. The cost of communication for the workers was prohibitive, so all information trickled down through the local authorities, whether civil, military or religious. The only low cost communication took place within the local village.<br />
<br />
Most workers were illiterate, and taking England in the middle ages as an example, a series of disruptions were caused by the invention of the printing press. The loss of control that resulted caused major problems for authorities, and allowed ideas to spread in new ways.<br />
<br />
As cities got bigger, local communication got faster, newsletters and newspapers started to spread ideas more quickly and the creation of postal services brought the cost of peer to peer communication down far enough that everyone could communicate over larger distances. Governments co-opted these new mechanisms, regulating the news, running the postal services. The industrial revolution created commercial entities with their own hierarchies of control and communication. The ability of ideas to spread rapidly and reach many people allowed groups to self-organize, leading to political upheaval. Revolution was enabled by the ability to spread ideas outside the incumbent ruler's hierarchy of communication and control.<br />
<br />
Today we live in societies where effectively everyone is literate, connected and mobile. However we still have the same hierarchical institutions that were developed hundreds of years ago. Governments to manage the hierarchies of society, Companies to manage the hierarchies of commerce, Religions to manage the hierarchies of belief, Universities to manage the hierarchies of education.<br />
<br />
These hierarchies still have value, they continue to represent ownership and have responsibilities but they no longer have control. They were sustained in the past by imposing their will on people, and carefully managed what information people are exposed to. The hierarchies are now very large and deeply nested, so rather than a small ruling elite and a large peasant population, there are very many levels in the middle, which makes many of us have a vested interest in the continuation of parts of the hierarchies.<br />
<br />
We are now all connected by a series of relatively flat peer to peer networks. Facebook for a curated network of family, friends and interest groups, twitter for real-time news and a much more dense ad-hoc interconnection of interests, email for direct communication, and many other services. The "Arab Spring", the "Occupy Movement" and other recent political revolutions were able to find and coordinate large numbers of people extremely rapidly (hours or days) without forming a hierarchy first.<br />
<br />
The new world is a blend of hierarchies with many peer to peer links that reach across and between the old hierarchies. The problem for the elite leaders is that they are used to having control, owning the flow of information and managing deeply nested organizations. In this new world, the leaders have ownership, responsibility, influence but much less control, they control and manage a small part of the flow of information, and their organizations are flatter and less well defined. At the extreme there are <a href="https://github.com/rtomayko/distributed-management-talk" target="_blank">organizations like Github,</a> which are figuring out how to make an emergent self organizing structure work. To scale a flat organization there needs to be a very strong sense of culture and unity of purpose. I think Netflix has found a productive balance between a hierarchy of responsibility for aspects of the business and freedom at each level to self organize the delivery of what the business needs. Broad and open communication across the business, and many skip-level one-on-one meetings flattens the organization and minimizes "politics".<br />
<br />
Looking beyond business, Religion only really works if you don't think too hard about any other variant of religion. Religions all claim to be true, but claim different incompatible and incoherent things, so that exposure to ideas such as the <a href="http://debunkingchristianity.blogspot.com/2007/03/outsider-test-for-faith.html" target="_blank">Outsider Test for Faith</a> causes deep problems for the faithful. Religion in general is being disrupted by science and free communication, and polarization into atheism and reactionary fundamentalism is one of the outcomes in society.<br />
<br />
Education is based on the communication of skills and information from teachers to students, and the big institutions of education are being disrupted. They no longer control the flow of education, and the costs are increasing beyond the ability of the students to repay them, so companies like Coursera and the Khan Academy are building alternatives. For software developers, the body of code you own and contributed to on github matters far more than where (or even if) you studied. The state of the art isn't being created and taught in University research departments any more, the commercial ecosystems that form around technologies are where the bleeding edge of innovation is being disseminated to new recruits.<br />
<br />
These disruptions are on the way, but change tends to occur rapidly when a tipping point is reached rather than gradually. A point made by Nicholas Taleb in his discussion of <a href="http://en.wikipedia.org/wiki/Black_swan_theory" target="_blank">Black Swan events</a>, is that its easier to point out that a change could happen and plan for what to do next, than to predict when exactly it will come. So like a cartoon Wily Coyote who has run off a cliff, we ignore the change until we look down and realize that nothing is holding us up.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVPEENiVLjPdKaSxac49Hcr-4617KBKh8UGctCzylT3XS2toe-I8g5CcAMunuzgRoj9rZrfSyatYYt3hFLU-hZbfQUfmLsCElKKn3zDbJylt-eDqUpru9kGP6FOlx3urRakGEe/s1600/Unknown.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVPEENiVLjPdKaSxac49Hcr-4617KBKh8UGctCzylT3XS2toe-I8g5CcAMunuzgRoj9rZrfSyatYYt3hFLU-hZbfQUfmLsCElKKn3zDbJylt-eDqUpru9kGP6FOlx3urRakGEe/s1600/Unknown.jpeg" /></a></div>
<br />
<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-1136745457854346242013-09-05T12:40:00.000-07:002013-09-05T14:21:09.622-07:00Performance Testing of NetflixOSS for the Cloud Prize<br />
Where is everyone? I've been writing blogging and tweeting about performance and capacity planning for many years, so I thought I was connected to one or two performance folks....<br />
<br />
When we setup the <a href="https://github.com/Netflix/Cloud-Prize" target="_blank">NetflixOSS cloud prize</a> I created a category just for you:<br />
<br />
<table cellpadding="0" cellspacing="0" dir="ltr" style="font-family: arial,sans,sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="362"></col></colgroup><tbody>
<tr style="height: 17px;"><td style="border-color: rgb(0, 0, 0) rgb(204, 204, 204) rgb(204, 204, 204); border-style: solid; border-width: 1px; direction: ltr; padding: 0px 3px; text-align: left; vertical-align: bottom;">Best Contribution to Performance Improvements</td></tr>
<tr style="height: 17px;"><td style="border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="border-bottom-color: rgb(0, 0, 0); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; text-align: left; vertical-align: bottom;">New performance testing tools, results of performance tests and performance oriented bugs. Since this may be aggregated across many pull requests, they should be listed and documented in the Submission.</td></tr>
</tbody></table>
<br />
While there are some interesting contestants in other categories, I've seen very little activity in this one.<br />
<br />
There's $10K cash and $5K of AWS credits waiting for someone to do enough work to claim them!<br />
<br />
It takes a bit of resource to run tests, so I also have a $100 AWS gift card that I will pass on to the first person to make an entry in this category, regardless of whether they win the prize or have generated any useful data at that point. Make a valid submission to the Cloud Prize by forking the github repo, editing the Submission.md file to describe your entry, entering your details (email/address etc) in the Mailchimp form, and then post a comment here claiming the $100 AWS card, and I'll mail it to you.<br />
<br />
After the first one, if you make a case for having generated lots of good data, I'll get some more $100 AWS cards.<br />
<br />
Here's some performance project ideas.<br />
<br />
<ul>
<li>Run throughput and latency stress tests on the Zuul API gateway on various instance types</li>
<li>Run stress tests for sizing data against other service level projects - Eureka, Edda, Ice, etc.</li>
<li>Make the cass_jmeter project better in some way</li>
<li>Analyze the code paths in Astyanax and find ways to make it run faster and use less memory</li>
</ul>
<div>
Go for it, only a few days left. Contest ends at midnight Pacific time Sunday Sept 15th.</div>
<div>
<br /></div>
<div>
Best wishes</div>
<div>
Adrian</div>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com3tag:blogger.com,1999:blog-7434008.post-82317346795851667162013-04-07T11:41:00.000-07:002013-04-07T11:41:08.971-07:00Tutorials and Training on Cloud Architecture and NetflixOSSThere are two places that I'm giving in-depth training on cloud architecture and NetflixOSS this year.<br />
<br />
On May 22nd-23rd at <a href="http://www.gluecon.com/2013/" target="_blank">Gluecon 2013</a> in Broomfield Colorado I'm giving one of the opening keynote talks that introduces the concepts of a Cloud Native Architecture, then spending all afternoon on a tutorial to go into the details of how to get there and how to use NetflixOSS tools as an on-ramp to accelerate the process.<br />
<br />
For people in Europe, I'm teaching at a summer school on "<a href="http://laser.inf.ethz.ch/2013/" target="_blank">Software for the Cloud and Big Data</a>" September 8th-14th in Italy. It's organized by ETH Zurich. I'll be giving six one-hour talks over a week, and there are seven other speakers.<br />
<br />
<a href="http://laser.inf.ethz.ch/2013/" target="_blank" title="LASER 2013: Software for the Cloud and Big Data"><img alt="LASER 2013 banner" src="http://laser.inf.ethz.ch/2013/img/banner.png" /></a><br />
<br />
When I get back from LASER, it will be time to start evaluating nominations for the NetflixOSS Cloud Prize, and these are both opportunities to figure out how to build a strong entry.<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com1tag:blogger.com,1999:blog-7434008.post-87814957343940021002013-03-26T12:53:00.003-07:002013-03-26T12:53:33.391-07:00Comment on How Netflix Is Ruining Cloud Computing<br />
I wrote a long comment response to <a href="http://www.informationweek.com/cloud-computing/platform/how-netflix-is-ruining-cloud-computing/240151650" target="_blank">how-netflix-is-ruining-cloud-computing</a> on Information Week, but they don't seem in a hurry to post it. Luckily I saved a copy so here it is:<br />
<br />
<br />
<br />
<div style="font-family: Helvetica; font-size: 12px;">
There should be a http://techblog.netflix.com post in the next day or so that will give more context to the Cloud Prize and clarify most of the points above. However I will address some of the specific issues here.</div>
<div style="font-family: Helvetica; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: Helvetica; font-size: 12px;">
Cloud 1.0 vs. 2.0?</div>
<div style="font-family: Helvetica; font-size: 12px;">
I would argue that the way most people are doing cloud today is to forklift part of their existing architecture into a cloud and run a hybrid setup. That's what I would call Cloud 1.0. What Netflix has done is show how to build much more agile green field native cloud applications, which might justify being called Cloud 2.0. The specific IaaS provider used underneath, and whether you do this with public or private clouds is irrelevant to the architectural constructs we've explained.</div>
<div style="font-family: Helvetica; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: Helvetica; font-size: 12px;">
Outages</div>
<div style="font-family: Helvetica; font-size: 12px;">
The outages that have been mentioned were regional, they didn't apply to Netflix operations in Europe for example. Our current work is to build tooling for multi-regional support on AWS (East cosat/West coast), including the DNS management that was mentioned. This removes the failure mode with the least effort and disruption to our existing operations.</div>
<div style="font-family: Helvetica; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: Helvetica; font-size: 12px;">
Portability</div>
<div style="font-family: Helvetica; font-size: 12px;">
Other cloud vendors have a feature set and scale comparable to AWS in 2008-2009. We're still waiting for them to catch up. There are many promises but nothing usable for Netflix itself. However there is demand to use NetflixOSS for other smaller and simpler applications, in both public and private clouds, and Eucalyptus have demonstrated Asgard, Edda and Chaos Monkey running, and will ship soon in Eucalyptus 3.3. There are signs of interest from people to add the missing features to OpenStack, CloudStack and Google Compute so that NetflixOSS can also run on them.</div>
<div style="font-family: Helvetica; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: Helvetica; font-size: 12px;">
Edda</div>
<div style="font-family: Helvetica; font-size: 12px;">
You've completely missed the point of Edda. It does three important things. 1) if you run at large scale your automation will overload the cloud API endpoint, Edda buffers this information and provides a query capability for efficient lookups. 2) Edda stores a history of your config, it's a CMDB that can be used to query for what changed. 3) Edda cross integrates multiple data sources, the cloud API, our own service registry Eureka, Appdynamics call flow information and can be extended to include other data sources.</div>
<div style="font-family: Helvetica; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: Helvetica; font-size: 12px;">
AMInator</div>
<div style="font-family: Helvetica; font-size: 12px;">
If you want to spin up 500 identical instances, having them each run Chef or Puppet after they boot creates a failure mode dependency on the Chef/Puppet service, wastes startup time, and if anything can go wrong with the install you end up with an inconsistent set of instances. By using AMInator to run Chef once at build time, there is less to go wrong at run time. It also makes red/black pushes and roll-backs trivial and reliable.</div>
<div style="font-family: Helvetica; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: Helvetica; font-size: 12px;">
Cloud Prize</div>
<div style="font-family: Helvetica; font-size: 12px;">
The prize includes a portability category. It's a broad category and might be won by someone who adds new language support to NetflixOSS (Erlang, Go, Ruby?) or someone who makes parts of NetflixOSS run on a broader range of IaaS options. The reality is that AWS is actually dominating cloud deployments today, so contributions that run on AWS will have the greatest utility by the largest number of people. The alternatives to AWS are being hyped by everyone else, and are showing some promise, but have some way to go.</div>
<div style="font-family: Helvetica; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: Helvetica; font-size: 12px;">
We hope that NetflixOSS provides a useful driver for higher baseline functionality that more IaaS APIs can converge on, and move from 2008-era EC2 functionality to 2010-era EC2 functionality across more vendors. Meanwhile Netflix itself will be enjoying the benefits of 2013 AWS functionality like RedShift.</div>
<div style="font-family: Helvetica; font-size: 12px;">
<br /></div>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com12tag:blogger.com,1999:blog-7434008.post-2398180043283282882013-01-16T17:17:00.000-08:002013-01-16T17:17:50.797-08:00The IT swamp draining manual for anyone who is neck deep in alligatorsI've spent the last year or so reviewing Gene Kim's <a href="http://www.amazon.com/gp/product/B00AZRBLHO/" target="_blank">new book - the Phoenix Project </a>and encouraging him to get it finished. It came out this week, is the top business book on Amazon as I write this, and I got a nice back-cover quote shown below with Gene's actual finger in the photo.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv2LGOtFyUVu26pbETWfqEV18ygMT0XtSVdOFfjEPWghasfUi3TT5p7gaT40QestNfqNlD0tJY3wGm0icZFCe83Lqd5kYPOHoTV9AT2cvTIyW-7eel6-PaMfN13wcszjF5yJ_D/s1600/PastedGraphic-4.tiff" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv2LGOtFyUVu26pbETWfqEV18ygMT0XtSVdOFfjEPWghasfUi3TT5p7gaT40QestNfqNlD0tJY3wGm0icZFCe83Lqd5kYPOHoTV9AT2cvTIyW-7eel6-PaMfN13wcszjF5yJ_D/s320/PastedGraphic-4.tiff" width="240" /></a></div>
<br />
Many years ago <a href="http://www.amazon.com/The-Goal-Process-Improvement-ebook/dp/B002LHRM2O/" target="_blank">someone gave me a copy of The Goal</a>, which is the inspiration for The Phoenix Project. In both cases the book is a novel about a company that is dysfunctional and on the verge of going out of business. The lead character is dropped into the job of figuring out how to dig their way out of the problem, and in the case of The Phoenix Project, the company fumbles its way from legacy enterprise where IT isn't regarded as central to their success, into the modern world of agile development practices and DevOps deployments where IT becomes a competitive advantage.<br />
<br />
Don't just read it, give copies of it to your friends in management. It should be on every CxO's bookshelf.<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com1tag:blogger.com,1999:blog-7434008.post-43146843582687207432013-01-01T13:59:00.000-08:002013-01-01T13:59:25.489-08:00Looking back at 2012, with pointers to 2013A collection of things that seem to have pivoted in 2012.<br />
<h4>
Mobile Bandwidth Greater than Fixed Bandwidth</h4>
I've been talking about <a href="http://perfcap.blogspot.com/2010/02/bil2010-millicomputing-future-in-your.html" target="_blank">LTE and the growth in mobile since 2008</a>, but I started 2012 with a Verizon iPhone 4 which maxed out at 2Mbit/s over 3G and at home in the mountains I would get less than 1Mbit/s. I ended 2012 with a Verizon iPhone 5 which is about ten times faster at home, I regularly see 8-9Mbits/s, and the best speed I have seen anywhere so far was in downtown Los Gatos at over 50Mbit/s. My home fixed wire Internet is a 3Mbit/s DSL that has neighborhood congestion at peak times. I now find it works better to have WiFi turned off on my iPhone at home.<br />
<br />
This is one of those pivotal changes, similar to the change from having predominantly fixed wire telephone service at home, to having many people use mobile phones exclusively. It costs more, but if you already have a high bandwidth connection to your phone with a high data cap because you use it a lot, why pay to also have a low bandwidth connection to your house? Bandwidth caps and data usage plans will slow the switchover, but the writing is on the wall.<br />
<br />
<h4>
Cutting The Cable/Satellite TV Feed</h4>
In 2013 we finally turned off our TiVo and shut down our DirecTV account. We weren't using it enough to make it worth while. For some of the sports events (Laurel follows the Stanford Cardinals), we go to a sports bar to watch, which is more fun anyway. Everything else that we have time to watch, we can watch online, and we get all our news updates from Twitter, RSS feeds and Facebook posts. By the time it's on TV or in a newspaper, it's already old news.<br />
<br />
The TV has an AppleTV connected to it, which gets almost all the usage. We watch a few things on laptops, and sometimes I connect a laptop to the TV. I also stream music from my iPhone to the AppleTV because I can't get Pandora or Spotify on it. Come on Apple, where's the AppleTV App Store? Maybe that's a 2013 thing.<br />
<br />
<h4>
The Netflix Open Source Cloud Platform Got Traction</h4>
We started the year with a handful of disconnected projects, and ended it with a large chunk of the platform on Github, and some high profile users. Most people are still picking it up piecemeal but in 2013 we plan to get the whole thing put together as an installable bundle. This is the Alan Kay approach, "The best way to predict the future is to invent it". Netflix has been out in front of the industry in terms of cloud adoption, inventing the future. Next we make it easier for others to join us in that future, and have some ideas for how to drive adoption to new heights.<br />
<br />
<h4>
Netflix Cloud Architecture Presentations</h4>
I was going to list all the talks I gave, but there are too many, so go see the slides I posted at <a href="http://www.slideshare.net/adrianco" target="_blank">http://www.slideshare.net/adrianco</a>. Highlights were QConSF, QConLondon, Gluecon in Colorado, GOTO in Aarhus and of course <a href="http://techblog.netflix.com/2012/12/videos-of-netflix-talks-at-aws-reinvent.html" target="_blank">AWS Re:Invent in Las Vegas</a>. The impact of these talks grew through the year, reaching a peak at Re:Invent, where we had lots of speakers and attention to the way the Netflix cloud and open source story was bringing value to the company and reaching out into the technical community. A big thanks to everyone who came to my talks, and all the other Netflix speakers who have been out there broadening the story. It's almost impossible to write an article or do a presentation about cloud without mentioning Netflix. In 2013 there will be even more talks, I focus on local and US based events that are strongly developer oriented like <a href="http://www.infoq.com/" target="_blank">QCon</a>, <a href="http://gluecon.com/" target="_blank">Gluecon</a>, and <a href="http://gotocon.com/" target="_blank">GOTO</a>. We will definitely be back at AWS Re:Invent next November.<br />
<br />
<h4>
The Concept of Anti-Fragility Took Off</h4>
<a href="http://www.amazon.com/Antifragile-Things-That-Disorder-ebook/dp/B0083DJWGO/ref=tmm_kin_title_0" target="_blank">Nassim Taleb's latest book</a> crystallizes the way I tend to approach things and gives it a name. The Netflix cloud architecture is anti-fragile, we run "Chaos Monkey's" continuously to try and break it, and that makes it stronger. The Netflix culture is anti-fragile, it's decentralized with as little process and rules as possible and a lot of local autonomy. Netflix management is not afraid of change or of being first to do something and tends to navigate disruptive transitions well. From the outside this can look chaotic or confusing, but it works, and recovers well from missteps, which are always going to happen. If you're not failing occasionally you aren't trying hard enough, and you are missing opportunities. Getting stronger through failure is the basis of anti-fragility. Avoiding failure at all costs (as many people try to do) makes you brittle and vulnerable to unexpected Black Swan events that will have a much bigger impact.<br />
<br />
<h4>
Cloud, Open Source, SaaS and the End of Enterprise Computing</h4>
Taleb makes the point that big companies become increasingly fragile as they lose agility and the ability to move with the markets, and we are seeing that play out in the Enterprise Computing space. There is still money to be made from the late adopter customers, but the trend is clearly towards development using exclusively open source tools, with applications and infrastructure delivered as a service. There is zero revenue for traditional Enterprise Computing vendors in this model. The current interest in building out private cloud infrastructure is real and will continue to support traditional vendors into 2013, but it's a short term investment. At best you end up with a much better automated datacenter, but it isn't elastic and it has far fewer features than AWS, so it's going to be marginalized over time. At worst, you discover just how hard it is to run a reliable private cloud based on immature software, with incompatible upgrade paths, and it turns out to be much more expensive to run.<br />
<br />
The Enterprise Computing vendors haven't been able to build a public cloud that competes with AWS on scale, price or features, and AWS is now focused on building everything their customers need to take the next generation of application investment out of the datacenter, so the high margin revenue is going to gradually go away for the traditional vendors.<br />
<br />
The most interesting development in 2012 was the re-launch of Google as a public cloud infrastructure vendor, and the mini-price-war between AWS and Google over instance and storage costs makes it clear where the real action is. During 2013 we will see if Google manages to invest heavily and execute well enough to build up a big user base. In mobile, as I predicted years ago, we are now in an iPhone vs. Android battle that is wiping out everyone else. I personally think in 2014 we will likely see a similar effect as the scale, features and price point of AWS and Google clouds make everyone else irrelevant. The only question in my mind is whether AWS runs away with this on their own, or Google manages to get some traction as the alternative.<br />
<br />
Note to sales reps (who won't listen), I'm not interested in anything to do with datacenters, private cloud, or other public clouds in 2013. I'm only interested in SaaS apps, things that run on AWS, and interesting open source projects.<br />
<br />
<h4>
Solar Powered Electrics Cars Are For Real Now</h4>
We drive our Nissan Leaf all the time, it's fun to drive and the first car we pick for most trips, adding up to almost 1000 miles a month. The marginal cost of running the car is near zero. New tires and a cabin air filter at 15K miles is all the maintenance it needs. We have an excess of solar power generation that added up to $500 in unused electricity over the year. At 10c/KWh and 3.5KWh/mile that's plenty for us run a second electric car before we start paying for the power, and there are a lot more choices coming in 2013. There are many charging stations around the Bay Area, lots of other people running Leafs, and the Tesla Model S got car of the year awards. It takes a test drive to realize what fun it is to have instant torque and no gear shifts. This is a case of the future being unevenly distributed. If you don't live in California, it's a bit further out, but it's coming.<br />
<br />
A friend recently got a quote for Solar Power installation which was about half what we paid two years ago, and we got a good deal then. Prices have dropped fast and are much lower than most people think. If you don't already have solar panels on your roof, you should get them. If you don't use enough electricity to justify solar panels, get an electric car as well, and save at the gas pump.<br />
<br />
<h4>
Global Warming Arrived in the USA in 2012</h4>
The well funded <a href="http://www.amazon.com/Merchants-Doubt-Scientists-Obscured-ebook/dp/B003RRXXO8/ref=tmm_kin_title_0?ie=UTF8&qid=1357067866&sr=8-1" target="_blank">Merchants of Doubt</a> (read the book) managed to confuse and suppress public discussion of global warming in the USA for the last few years, but the effects just became too obvious this year and it broke through, creating the scenarios that James Hansen warned of in <a href="http://www.amazon.com/Storms-My-Grandchildren-Catastrophe-ebook/dp/B002Z8IWLO/ref=sr_1_1?s=digital-text&ie=UTF8&qid=1357068138&sr=1-1&keywords=storms+of+my+grandchildren" target="_blank">Storms of My Grandchildren</a>. The arctic ice cap melt continues to accelerate, seas are warming and rising, drought and record heat hit most of the USA, and everything wrapped up with Hurricane Sandy, pushing the topic onto the front page. The dice are all loaded now, and 2013 is already rolling those dice as the drought continues and the Mississippi river is empty. I've been saying for the last few years that if you own property at sea level, you should find someone who doesn't believe in global warming to sell it to, because it's going to become increasingly uninsurable and end up as worthless as the houses along the New Jersey shoreline that were swept away.<br />
<br />
The Republican party is still in denial, a combination of funding from big oil companies and an inability to accept or admit that their demonized Al Gore could have been right all along. In 2013 it will be interesting to see how they deal with losing the election, and perhaps there will be a split into a group of Republicans that see the path to re-election in 2014 as needing to accept reality by voting for some Global Warming related legislation, versus the hard core that are trying to pray their way out. The current battle is over stopping the Keystone XL pipeline that would move the dirtiest kind of tar oil from Alberta Canada to Texas. It may be symbolic, but if KXL is stopped, the tide will have turned. <a href="http://350.org/" target="_blank">Carbon needs to be left in the ground</a>. For 2013, I'm going to try and re-balance my 401K retirement accounts to divest from oil companies. Many students are now <a href="http://gofossilfree.org/" target="_blank">pressuring their colleges</a> to divest from oil as well.<br />
<br />
<h4>
Twitter and Snapchat</h4>
Personally, 2012 was an excellent year for me, I've made lots of new friends and learned a lot by being active on <a href="http://www.twitter.com/adrianco" target="_blank">twitter</a>, ending the year with about 6500 followers. I joked on twitter that I posted my new years resolutions for 2013 to <a href="http://www.snapchat.com/" target="_blank">Snapchat</a>, but you missed them. If you don't know what Snapchat is for, ask a teenager. You'll probably hear a lot more about it in 2013, then, when their parents figure it out and join too, the teens will be onto the next thing....<br />
<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com4tag:blogger.com,1999:blog-7434008.post-14411209467166745152012-11-26T10:18:00.000-08:002012-12-28T11:13:44.288-08:00Lots of Netflix talks at AWS Re:Invent<!--StartFragment-->
<br />
<br />
[Update: here's video's of these talks <a href="http://techblog.netflix.com/2012/12/videos-of-netflix-talks-at-aws-reinvent.html">http://techblog.netflix.com/2012/12/videos-of-netflix-talks-at-aws-reinvent.html</a> along with slides]<br />
<br />
There is a Netflix booth in the expo center, we will be talking about our open source tools from <a href="http://netflix.github.com/">http://netflix.github.com</a> and collecting resumes from anyone interested in joining us.<br />
<br />
<table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse; mso-yfti-tbllook: 1056; width: 684px;">
<colgroup><col style="mso-width-source: userset; width: 137pt;" width="137"></col>
<col style="mso-width-source: userset; width: 158pt;" width="158"></col>
<col style="mso-width-source: userset; width: 389pt;" width="389"></col>
</colgroup><tbody>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small; font-weight: bold;">Date/Time</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small; font-weight: bold;">Presenter</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small; font-weight: bold;">Topic</span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Wed 8:30-10:00</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Reed Hastings</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Keynote with Andy</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;">
Jassy</span></span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Wed 1:00-1:45</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Coburn Watson</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Optimizing</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;">
Costs with AWS</span></span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Wed 2:05-2:55</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Kevin McEntee</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small; font-style: italic;">Netflix’s Transcoding
Transformation</span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Wed 3:25-4:15</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial;">Neil Hunt /</span><span style="font-family: Arial; vertical-align: baseline;"> Yury I.</span></span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Netflix:</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;">
Embracing the Cloud</span></span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Wed 4:30-5:20</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Adrian Cockcroft</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small; font-style: italic;">High Availability Architecture at
Netflix</span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Thu 10:30-11:20</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Jeremy Edberg</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small; font-style: italic;">Rainmakers – Operating Clouds</span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Thu 11:35-12:25</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Kurt Brown</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Data Science</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;">
with Elastic Map Reduce (EMR)</span></span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Thu 11:35-12:25</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial;">Jason</span><span style="font-family: Arial; vertical-align: baseline;"> Chan</span></span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Security Panel: Learn from</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;">
CISOs working with AWS</span></span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Thu 3:00-3:50</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Adrian Cockcroft</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Compute & Networking</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;">
Masters Customer Panel</span></span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Thu 3:00-3:50</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial;">Ruslan</span><span style="font-family: Arial; vertical-align: baseline;"> M./Gregg U.</span></span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Optimizing Your Cassandra
Database</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;"> on
AWS</span></span></div>
</td>
</tr>
<tr height="27" style="height: 26.86pt; mso-height-source: userset;">
<td class="oa1" height="27" style="height: 26.86pt; width: 137pt;" width="137"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Thu 4:05-4:55</span></div>
</td>
<td class="oa1" style="width: 158pt;" width="158"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-family: Arial; font-size: x-small;">Ariel Tseitlin</span></div>
</td>
<td class="oa1" style="width: 389pt;" width="389"><div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; mso-line-break-override: none; punctuation-wrap: hanging; text-align: left; unicode-bidi: embed; word-break: normal;">
<span style="font-size: x-small;"><span style="font-family: Arial; font-style: italic;">Intro</span><span style="font-family: Arial; font-style: italic; vertical-align: baseline;"> to
Chaos Monkey and the Simian Army</span></span></div>
</td>
</tr>
</tbody></table>
<!--EndFragment-->Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-54604475585793994162012-11-16T09:39:00.002-08:002014-12-20T09:08:21.620-08:00Cloud Outage ReportsThe detailed summaries of outages from cloud vendors are comprehensive and the response to each highlights many lessons in how to build robust distributed systems. For outages that significantly affected Netflix, the Netflix techblog report gives insight into how to effectively build reliable services on top of AWS. I've included some Google and Azure outages here because they illustrate different failure modes that should be taken into account. Recent AWS and Azure outage reports have far more detail than Google outage reports.<br />
<br />
I plan to collect reports here over time, and welcome links to other write-ups of outages and how to survive them. My naming convention is {vendor} {primary scope} {cause}. The scope may be global, a specific region, or a zone in the region. In some cases there are secondary impacts with a wider scope but shorter duration such as regional control planes becoming unavailable for a short time during a zone outage.<br />
<br />
This post was written while researching my AWS Re:Invent talk.<br />
Slides: <a href="http://www.slideshare.net/AmazonWebServices/arc203-netflixha" target="_blank">http://www.slideshare.net/AmazonWebServices/arc203-netflixha</a><br />
Video: <a href="http://www.youtube.com/watch?v=dekV3Oq7pH8" target="_blank">http://www.youtube.com/watch?v=dekV3Oq7pH8</a><br />
<br />
<hr />
<h2>
November 18th, 2014 - Azure Global Storage Outage</h2>
<h4>
Microsoft Reports</h4>
<div>
<a href="http://azure.microsoft.com/blog/2014/11/19/update-on-azure-storage-service-interruption/">http://azure.microsoft.com/blog/2014/11/19/update-on-azure-storage-service-interruption/</a><br />
<br />
<a href="http://azure.microsoft.com/blog/2014/12/17/final-root-cause-analysis-and-improvement-areas-nov-18-azure-storage-service-interruption/">http://azure.microsoft.com/blog/2014/12/17/final-root-cause-analysis-and-improvement-areas-nov-18-azure-storage-service-interruption/</a></div>
<h3>
</h3>
<hr />
<h2>
January 10th, 2014 - Dropbox Global Outage</h2>
<h4>
Dropbox Report</h4>
<div>
<a href="https://tech.dropbox.com/2014/01/outage-post-mortem/">https://tech.dropbox.com/2014/01/outage-post-mortem/</a></div>
<h3>
</h3>
<hr />
<h2>
April 20th, 2013 - Google Global API Outage</h2>
<h4>
Google Report</h4>
<div>
<a href="http://googledevelopers.blogspot.com/2013/05/google-api-infrastructure-outage.html">http://googledevelopers.blogspot.com/2013/05/google-api-infrastructure-outage.html</a></div>
<h3>
</h3>
<hr />
<h2>
February 22nd, 2013 - Azure Global Outage Cert Expiry</h2>
<h4>
Azure Report</h4>
<div>
<a href="http://blogs.msdn.com/b/windowsazure/archive/2013/03/01/details-of-the-february-22nd-2013-windows-azure-storage-disruption.aspx">http://blogs.msdn.com/b/windowsazure/archive/2013/03/01/details-of-the-february-22nd-2013-windows-azure-storage-disruption.aspx</a></div>
<h3>
</h3>
<hr />
<h2>
December 24th, 2012 - AWS US-East Partial Regional ELB State Overwritten</h2>
<h4>
AWS Service Event Report</h4>
<a href="http://aws.amazon.com/message/680587/" target="_blank">http://aws.amazon.com/message/680587/</a><br />
<h4>
Netflix Techblog Report</h4>
<a href="http://techblog.netflix.com/2012/12/a-closer-look-at-christmas-eve-outage.html" target="_blank">http://techblog.netflix.com/2012/12/a-closer-look-at-christmas-eve-outage.html</a><br />
<h3>
</h3>
<hr />
<h2>
October 26th, 2012 - Google AppEngine Network Router Overload</h2>
<h4>
Google Outage Report</h4>
<div>
<a href="http://googleappengine.blogspot.com/2012/10/about-todays-app-engine-outage.html">http://googleappengine.blogspot.com/2012/10/about-todays-app-engine-outage.html</a></div>
<h3>
</h3>
<hr />
<h2>
October 22, 2012 - AWS US-East Zone EBS Data Collector Bug</h2>
<h4>
AWS Outage Report</h4>
<div>
<a href="http://aws.amazon.com/message/680342/">http://aws.amazon.com/message/680342/</a></div>
<div>
<h4>
Netflix Techblog Report</h4>
</div>
<div>
<a href="http://techblog.netflix.com/2012/10/post-mortem-of-october-222012-aws.html">http://techblog.netflix.com/2012/10/post-mortem-of-october-222012-aws.html</a></div>
<h3>
</h3>
<hr />
<h2>
June 29th 2012 - AWS US-East Zone Power Outage During Storm </h2>
<h4>
AWS Outage Report</h4>
<div>
<a href="http://aws.amazon.com/message/67457/">http://aws.amazon.com/message/67457/</a></div>
<h4>
Netflix Techblog Report</h4>
<div>
<a href="http://techblog.netflix.com/2012/07/lessons-netflix-learned-from-aws-storm.html">http://techblog.netflix.com/2012/07/lessons-netflix-learned-from-aws-storm.html</a></div>
<h3>
</h3>
<hr />
<h2>
June 13th, 2012 - AWS US-East SimpleDB Region Outage</h2>
<h4>
AWS Outage Report</h4>
<div>
<a href="http://aws.amazon.com/message/65649/">http://aws.amazon.com/message/65649/</a></div>
<h3>
</h3>
<hr />
<h2>
February 29th, 2012 - Microsoft Azure Global Leap-Year Outage</h2>
<h4>
Azure Outage Report</h4>
<div>
<a href="http://blogs.msdn.com/b/windowsazure/archive/2012/03/09/summary-of-windows-azure-service-disruption-on-feb-29th-2012.aspx">http://blogs.msdn.com/b/windowsazure/archive/2012/03/09/summary-of-windows-azure-service-disruption-on-feb-29th-2012.aspx</a></div>
<h3>
</h3>
<hr />
<h2>
August 17th, 2011 - AWS EU-West Zone Power Outage</h2>
<h4>
AWS Outage Report</h4>
<div>
<a href="http://aws.amazon.com/message/2329B7/">http://aws.amazon.com/message/2329B7/</a></div>
<h3>
</h3>
<hr />
<h2>
April 2011 - AWS US-East Zone EBS Outage</h2>
<div>
<h4>
AWS Outage Report</h4>
</div>
<div>
<a href="http://aws.amazon.com/message/65648/">http://aws.amazon.com/message/65648/</a></div>
<div>
<h4>
Netflix Techblog Report</h4>
</div>
<div>
<a href="http://techblog.netflix.com/2011/04/lessons-netflix-learned-from-aws-outage.html">http://techblog.netflix.com/2011/04/lessons-netflix-learned-from-aws-outage.html</a><br />
<h3>
</h3>
<hr />
<h2>
February 24th, 2010 - Google App Engine Power Outage</h2>
</div>
<h4>
Google Forum Report</h4>
<div>
<a href="https://groups.google.com/forum/#!topic/google-appengine/p2QKJ0OSLc8">https://groups.google.com/forum/#!topic/google-appengine/p2QKJ0OSLc8</a></div>
<h3>
</h3>
<hr />
<h2>
July 20th, 2008 - AWS Global S3 Gossip Protocol Corruption</h2>
<h4>
AWS Outage Report</h4>
<div style="text-align: justify;">
<span style="color: #29303b; font-family: Georgia, Times New Roman, sans-serif; font-size: x-small;"><a href="http://status.aws.amazon.com/s3-20080720.html">http://status.aws.amazon.com/s3-20080720.html</a></span></div>
<div style="text-align: justify;">
<span style="color: #29303b; font-family: Georgia, Times New Roman, sans-serif; font-size: x-small;"><br /></span></div>
<span style="background-color: #fff3db; color: #29303b; font-family: Georgia, 'Times New Roman', sans-serif; font-size: 13px; text-align: justify;"><br /></span>
Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com6tag:blogger.com,1999:blog-7434008.post-84859412195226413332012-10-26T13:53:00.001-07:002012-10-26T15:18:11.583-07:00What's a Distinguished Engineer?A <a href="http://www.kitchensoap.com/2012/10/25/on-being-a-senior-engineer/" target="_blank">recent post by John Allspaw on what it means to be a senior engineer</a> reminded me of something I put together years ago while I was a Distinguished Engineer at Sun. One question from senior engineers looking at their career path was what did it take to become a Distinguished Engineer?<br />
<br />
Although Sun is no more, across the industry, there are engineers who are "distinguished" and the title is used in a few places. At Sun, there were between 50 and 100 people in the role, who were mostly director level individual contributors, although there were also Sun Fellows who were VP level, and some were also line managers.<br />
<br />
I boiled it down into a few questions.<br />
<br />
First I made a list of the names of all the Sun Distinguished Engineers and Fellows, and the first question was "how many of these names do you recognize, and know what they did". The intent is to get a baseline level of understanding of what might be expected. The list included people who invented software languages and frameworks that lots of people use, microprocessor architects, and fundamental researchers in security and networking. There were also CTOs of companies that Sun had acquired, and a few like me who mostly got in through writing books that everyone else had read.<br />
<br />
The next question is "how many of these people know who you are?". If you think you did do something special, we would expect that the existing Distinguished Engineers would have heard of it. Since at Sun the way to become a DE involved having the existing DE and Fellows vote for you, this was critical.<br />
<br />
The final question was "how many DE and Fellows are hanging around your cube on a regular basis waiting to talk to you?". This shows that you are the go-to person for something that matters.<br />
<br />
Translating this into a broader context, more current questions for being distinguished might be "Do the top conferences invite you to speak?", "How many of the other invited speakers and conference organizers do you know?" and "how many know you?". The other dimension of what you did to deserve it is nowadays a mixture of open source projects that lots of people use, or key ideas shared through books or blogs.<br />
<br />
Here's the original slide from 2002, how many of these names do you know, what did they do then and where are they now?<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikKv3AE85jCqyeY9RvZYteLwpzpDwIVVxgYhP-eDAE5QjezHxeRMT1dhXm1v12cbYmpd9bwBJlUSKno-mPrWwMpCa5r2VC3Za0kM142eoSai4lhC0Ih3s_4udyBUKUXEP9wDm9/s1600/SunDEs.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikKv3AE85jCqyeY9RvZYteLwpzpDwIVVxgYhP-eDAE5QjezHxeRMT1dhXm1v12cbYmpd9bwBJlUSKno-mPrWwMpCa5r2VC3Za0kM142eoSai4lhC0Ih3s_4udyBUKUXEP9wDm9/s320/SunDEs.png" width="320" /></a></div>
<br />
<br />
<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-63965333358281501502012-09-02T01:22:00.000-07:002012-12-23T10:58:21.180-08:00Solar Power Update - Annual Costs<br />
I've blogged before about our solar power installation and we just completed our first full year where we didn't change anything, so we have a clear baseline of how much we generated and used, which I will describe in this post.<br />
<br />
Our system is a grid tied net metering setup. This means that we generate more electricity than we use during the day and run the meter backwards in the summer, then use (cheaper) electricity at night and in the winter from the grid. Once a year our PG&E bill is netted out and we pay the difference. Last year our total electric bill was about $500 but during the year we changed everything by adding a second solar array, a Nissan Leaf electric car and a heat pump for heating and cooling. This year, with a consistent setup for the entire period, our bill was negative, so we didn't owe PG&E anything. However, this didn't mean that PG&E paid us the difference, because the payback rule is based on how many KWh you generate rather than how much you would have paid for it. In our case, although our net bill was -$495, we used 610 KWh more than we generated for the whole year. The details of our PG&E bill are shown at the end of this post.<br />
<br />
In effect, we could have used another $495 worth of electricity for free. Going into next year, we will set the heat pump to work harder at lower temperatures before it switches over to propane, which will reduce our carbon footprint and save us some propane costs. For some of last winter we used propane when the outside temperature fell below 45F, and at some point we reduced that to 40F. Unfortunately the controller setting we have is in increments of 5F, and I'm doubtful that we can pump heat out of air at 35F, but it's worth a try. We already converted our other appliances from propane to electric, and other than the heat pump our major consumers are a large tank electric water heater, induction range, clothes dryer, hot tub and well pump.<br />
<br />
We did around 11,000 miles in the Nissan Leaf in the last year, but estimate that about a third of our charging was at work, as Laurel's commute is far enough to be an easy one way run, but requires a charge to get home again up the hill (we live at 2400ft, and she works at sea level!). She can take the car pool lane with the "white sticker" that pure electric cars get, which saves a lot of time, but running at high speed on the freeway uses a lot more power than around town. When I get to use the Leaf my commute is much closer and I don't bother charging it at work. Our long term average consumption is 3.7 miles/KWh, which is worse than most Leaf owners because of the freeway miles, hill climbing and "having fun". We don't pay for charging at work, and the marginal cost of electricity at home is zero because we are generating a negative net bill for the year, so we only use the "ECO" driving style if we are pushing the limits of its range on an unusual trip. The Leaf is the first car we pick for drives in its range, it's entertaining to drive as well as having extremely low running costs. All it needs for maintenance is tires, there is no engine oil to change, and the brakes wear very slowly due to electric regeneration. Here's the year so far as recorded by the Leaf's "Carwings" system.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWg_Ggex7kSEv8WoeHjNE4XQSEqdaqEnJ2WictWEi4hEKRfeUY7YuW_mFPMUenQbpwhqw780aDs6rT9e_ZbRJlJUhWomcJczalVbDL7oHxaeIgzzN-Tgsa5X9-uRvknSmlyw9i/s1600/Screen+Shot+2012-09-02+at+12.57.19+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="386" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWg_Ggex7kSEv8WoeHjNE4XQSEqdaqEnJ2WictWEi4hEKRfeUY7YuW_mFPMUenQbpwhqw780aDs6rT9e_ZbRJlJUhWomcJczalVbDL7oHxaeIgzzN-Tgsa5X9-uRvknSmlyw9i/s640/Screen+Shot+2012-09-02+at+12.57.19+AM.png" width="640" /></a></div>
<br />
<br />
The PG&E tariff we are on is called E6 Net Energy Metering, it has three rates for the summer, and two rates in the winter. The rules and rates are described in these documents at the PG&E web site: Rules <a href="http://www.pge.com/tariffs/tm2/pdf/ELEC_SCHEDS_NEM.pdf">http://www.pge.com/tariffs/tm2/pdf/ELEC_SCHEDS_NEM.pdf</a> and Rates <a href="http://www.pge.com/tariffs/tm2/pdf/ELEC_SCHEDS_E-6.pdf">http://www.pge.com/tariffs/tm2/pdf/ELEC_SCHEDS_E-6.pdf</a><br />
The rate that PG&E will pay us for any extra KWh we generate is about 3 cents/KWh as described in this document <a href="http://www.pge.com/includes/docs/pdfs/shared/solar/AB920_RateTable.pdf">http://www.pge.com/includes/docs/pdfs/shared/solar/AB920_RateTable.pdf</a><br />
Our base overnight rate that we pay is around 10 cents/KWh, the afternoon rate is around 28 cents/KWh, this is why we can generate hundreds of dollars in a negative bill while using hundreds of KWh.<br />
<br />
Our <a href="http://perfcap.blogspot.com/search?q=solar&max-results=20&by-date=true" target="_blank">solar setup is in two arrays that I described on my blog before</a>, the total summer peak output is about 10KW, and we get up to 70KWh per day. In the winter this drops to about 40KWh per day, and clouds and rainy days greatly reduce the output. We're in a sunny spot on top of the mountains, and the monthly output from the 6.5KW array is shown in the plots below since we turned it on in April 2011. The other array adds about half this amount. Our annual true-up period runs from the beginning of September to the end of August.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnvjnWcKWceXk1GW1-OfBa4rVHXU6MMIgulfichlUdO0GTi5rEPQpNDeEmdGoSRl2o-u61NAZtCjLwRY921IEoruxeY45onAZJ_adciIIJ8fxjaEsTGmEIvYKO0xpaTFG61U0Z/s1600/Screen+Shot+2012-09-01+at+11.32.36+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="235" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnvjnWcKWceXk1GW1-OfBa4rVHXU6MMIgulfichlUdO0GTi5rEPQpNDeEmdGoSRl2o-u61NAZtCjLwRY921IEoruxeY45onAZJ_adciIIJ8fxjaEsTGmEIvYKO0xpaTFG61U0Z/s640/Screen+Shot+2012-09-01+at+11.32.36+PM.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5NR41gd0NPqlWGtm2tgKzlSJ92x44eTY-6f0S5QN6HyqSBaSLTutWnhxjtwk7lpNsgMnkbKtMOjvl-XcBxk7ag7hivz9UFt-d4K8T2Zje_STsYJnKT_j3Q4uy1xtiIz8i5GEt/s1600/Screen+Shot+2012-09-01+at+11.32.09+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="235" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5NR41gd0NPqlWGtm2tgKzlSJ92x44eTY-6f0S5QN6HyqSBaSLTutWnhxjtwk7lpNsgMnkbKtMOjvl-XcBxk7ag7hivz9UFt-d4K8T2Zje_STsYJnKT_j3Q4uy1xtiIz8i5GEt/s640/Screen+Shot+2012-09-01+at+11.32.09+PM.png" width="640" /></a></div>
<br />
PG&E send us a bill every month for about $11 of connection fees and taxes, and the current running total. Our final bill for the last year is shown below. <a href="http://perfcap.blogspot.com/2011/02/solar-power-more-panels-on-garage-roof.html" target="_blank">Solar costs were described in a previous post</a>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTl1tQkXnpDYkRN5DnhEhJ5r3l1MFl0apUgyMg1qM67BOQBFdUFgLsY4Zewey3Tv0NfyCy_vcQwUE9HNaOPZBDUqz7p_dq_98p9LWyiii7yB-xxRC_cLCWIRxOY4wKQRqXdEjp/s1600/pge2012-1small.JPEG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTl1tQkXnpDYkRN5DnhEhJ5r3l1MFl0apUgyMg1qM67BOQBFdUFgLsY4Zewey3Tv0NfyCy_vcQwUE9HNaOPZBDUqz7p_dq_98p9LWyiii7yB-xxRC_cLCWIRxOY4wKQRqXdEjp/s640/pge2012-1small.JPEG" width="600" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgehqxKQk2ITSpjh9BGwk7KV_dUQ1IfU4qbDthp06yleTpARB-Tkct5HNuO6sU6EaRdpFgWEeHg2iAvR3xIvZOjU-XI2qh1b3sbXtXmG0AJdluobgczPhWAJ2JdDqrLV7wMGITr/s1600/pge2012-2small.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgehqxKQk2ITSpjh9BGwk7KV_dUQ1IfU4qbDthp06yleTpARB-Tkct5HNuO6sU6EaRdpFgWEeHg2iAvR3xIvZOjU-XI2qh1b3sbXtXmG0AJdluobgczPhWAJ2JdDqrLV7wMGITr/s640/pge2012-2small.jpg" width="469" /></a></div>
<br />
<br />
<br />
<br />Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com0tag:blogger.com,1999:blog-7434008.post-17994594808896712762012-06-16T13:16:00.001-07:002012-09-04T17:35:27.825-07:00Cloud Application Architectures: GOTO Aarhus, Denmark, OctoberThanks to an invite via <a href="http://www.michaelnygard.com/" target="_blank">Michael Nygard</a>, I ended up as a track chair for the <a href="http://gotocon.com/aarhus-2012/" target="_blank">GOTO Aarhus conference</a> in Denmark in early October. The track subject is <a href="http://gotocon.com/aarhus-2012/schedule/tuesday.jsp" target="_blank">Cloud Application Architectures</a>, and we have three speakers lined up. You can <a href="https://secure.trifork.com/aarhus-2012/registration/" target="_blank">register with a discount</a> using the code <b>crof1000</b>.<br />
<br />
We are starting out with a talk from the Citrix Cloudstack folks about how to architect applications on open portable clouds. These implement a subset of functionality but have more implementation flexibility than public cloud services. [Randy Bias of Cloudscaling was going to give this talk but had to pull out of the trip due to other commitments].<br />
<br />
To broaden our perspective somewhat, and get our hands dirty with real code, the next talk is a live demonstration by Ido Flatow,
<a href="http://gotocon.com/aarhus-2012/presentation/Building%20secured,%20scalable,%20low-latency%20web%20applications%20with%20the%20Windows%20Azure%20Platform" target="_blank">Building secured, scalable, low-latency web applications with the Windows Azure Platform</a>.<br />
<blockquote class="tr_bq">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif; font-size: 11px; line-height: 17px;">In this session we will construct a secured, durable, scalable, low-latency web application with Windows Azure - Compute, Storage, CDN, ACS, Cache, SQL Azure, Full IIS, and more. This is a no-slides presentation!</span></blockquote>
Finally I will be giving my latest update on <a href="http://gotocon.com/aarhus-2012/presentation/Globally%20Distributed%20Cloud%20Applications%20at%20Netflix" target="_blank">Globally Distributed Cloud Applications at Netflix</a>.<br />
<blockquote class="tr_bq">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif; font-size: 11px; line-height: 17px;">Netflix grew rapidly and moved its streaming video service to the AWS cloud between 2009 and 2010. In 2011 the architecture was extended to use Apache Cassandra as a backend, and the service was internationalized to support Latin America. Early in 2012 Netflix launched in the UK and Ireland, using the the combination of AWS capacity in Ireland and Cassandra to create a truly global backend service. Since then the code that manages and operates the global Netflix platform is being released as a series of open source projects at netflix.github.com (Asgard, Priam etc.). The platform is structured as a large scale PaaS, strongly leveraging advanced features of AWS to deploy many thousands of instances. The platform has primary language support for Java/Tomcat with most management tools built using Groovy/Grails and operations tooling in Python. Continuous integration and deployment tooling leverages Jenkins, Ivy/Gradle, Artifactory. This talk will explain how to build your own custom PaaS on AWS using these components.</span></blockquote>
<br />
There are many other excellent speakers at this event, which is run by the same team as the global series of QCon conferences, unfortunately, the cloud track runs at at the same time as Michael Nygard and Jez Humble on Continuous Delivery and Continuous Integration, however I'm doing another talk in the NoSQL track, (along with Martin Fowler and Coda Hale). <a href="http://gotocon.com/aarhus-2012/presentation/Running%20Netflix%20on%20Cassandra%20in%20the%20Cloud" target="_blank">Running Netflix on Cassandra in the Cloud</a>.<br />
<blockquote class="tr_bq">
<span class="Apple-style-span" style="font-family: Verdana, sans-serif; font-size: 11px; line-height: 17px;">Netflix used to be a traditional Datacenter based architecture using a few large Oracle database backends. Now it is one of the largest cloud based architectures, with master copies of all data living in Cassandra. This talk will discuss how we made the transition, how we automated and open sourced Cassandra management for tens of clusters and hundreds of nodes using Priam and Astyanax, backups, archiving and performance and scalability benchmarks.</span></blockquote>
<br />
I'm looking forward to meeting old friends, getting to know some new people, and visiting Denmark for the first time. See you there!Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com1tag:blogger.com,1999:blog-7434008.post-49981336240023325762012-04-29T17:33:00.000-07:002012-04-29T17:33:40.151-07:00It's not obvious how to be insanely simplethree books that I read recently resonated with me and fitted together so I'm going to try to make sense of them in a blog post rather than in a series of cryptic tweets.<br />
<br />
My son (who is a <a href="http://touch.ebay.com/" target="_blank">product manager at eBay</a>) told me about the most recent publication:<br />
<a href="http://www.amazon.com/Insanely-Simple-Obsession-Drives-Success/dp/1591844835/ref=sr_1_1?s=books&ie=UTF8&qid=1335735223&sr=1-1" target="_blank">Insanely Simple: The Obsession That Drives Apple's Success by Ken Segall</a><br />
<br />
At the <a href="http://www.defragcon.com/2012/" target="_blank">Defrag conference</a> last year I saw Duncan Watts present and recently finished reading his book <a href="http://www.amazon.com/Everything-Is-Obvious-Answer-ebook/dp/B004DEPHGQ/" target="_blank">Everything Is Obvious: Once You Know The Answer</a><br />
<br />
I also recently <a href="http://www.youtube.com/watch?v=pCofmZlC72g" target="_blank">watched Sam Harris give his talk</a> on Free Will, and then <a href="http://www.amazon.com/Free-Will-ebook/dp/B006IDG2T6" target="_blank">read the book</a>.<br />
<br />
The connection starts with Free Will, which explains what is really going in our heads, along with Everything is Obvious which explains how our minds work collectively and interact with the real world these two books are the "Missing Manuals" for our brains. It's hard enough to figure out what is going on in the world and how best to navigate it, but it's doubly hard when you don't realize how your subconscious is pulling the strings and how common sense is confusing everyone around you.<br />
<br />
Inside your head, the conscious thread of thoughts that you hear are post rationalizing decisions that your subconscious mind has already made. Feeding yourself a broad range of information with an open mind, connecting to your intuition and letting the power of your subconscious find the right patterns and responses lets you make faster and better decisions.<br />
<br />
In society, we are surrounded by common sense explanations that we use to post rationalize the events around us and which are fed to us by the media, historians, politicians and our friends. Duncan deconstructs common sense to show that these explanations are mirages driven by our inner need to find a narrative and cause for effects that are essentially random co-incidences with far less significance than we assume. He then explains what un-common sense looks like and how to question the received wisdom and have better strategies for getting things done successfully.<br />
<br />
I'm not going to summarize the whole book but there is a very useful section that should be read by anyone doing "big data analytics" that sets out the kind of things that are know-able and what (and why) other things will always remain un-knowable and impossible to predict. The advice I distilled from the discussion of strategy is that there is so much randomness in the outcome of business decisions that you cannot reliably evaluate the difference between a good strategy and a poor strategy. If you are able to get ever more detailed data about what happened you become more convinced in the value of your analysis, but the predictions you make about what to do next don't get any better. This is a counter-intuitive outcome (i.e. it violates common sense), so please read the book, which explains why you shouldn't be trusting your common sense in the first place.<br />
<br />
The positive things we can do to overcome random outcomes really resonated with me, as they put into words several of the things I've been doing for many years, which have in some sense given me a better way to understand what's going on and get stuff done. They also describe many of the ways that Netflix figures out how to build it's products.<br />
<br />
The first thing I do when I hear something like A caused B is a reflex reaction, I flip it around in my head, take the devils advocate position, look at the situation from a few different angles. This can be quite annoying in "polite company" as I tend to question received wisdom and common sense assertions, however I usually find a missing piece of information that could falsify the assertion, and ask the question. It could be as simple as asking exactly what time A and B happened, since if B turned out to happen before A then the assertion is clearly false. In statistics and physics this is codified as asserting the null hypothesis. (I'm the son of a statistician and I have a Physics degree...).<br />
<br />
At Netflix we always try to construct parallel "A/B" tests of our hypotheses, like the double blind tests used in clinical trials of new medicines. We take a large number of new customers and give them a range of different experiences for long enough to measure a difference in their responses. This is the only way to reliably tell whether a new feature works, and it often goes against the common sense of what we expect and what many customers and industry analysts helpfully suggest we should be doing. As Duncan explains we can usually figure out what factors will affect an outcome, but we are extremely poor judges of how to weight those factors, even with post rationalization of what we saw happen, and all we can do is bias the statistics in a preferred direction. A recurring example is the suggestion that Netflix should allow half-stars in its movie ratings, but it turns out that given more fine grain choice fewer people rate movies, and the reduction in the number of data points out-weighs the increased accuracy. We can post rationalize why this occurs as an example of giving people too much choice, but we don't have to rationalize it, we just measured it.<br />
<br />
In the discussion of strategy Duncan talks about creating a set of strategies that cover many scenarios, and using scenario planning to build more flexible and fuzzy strategies which are more likely to work under a range of random external influences. By putting yourself in the path of possible good randomness and avoiding bad outcomes, you can "make your own luck". By detecting problems early and having the flexibility to adapt your strategy you can run around the problems that will randomly come your way. If instead you concentrate on coming up with the best possible strategy or assuming that previous success was due to strategy rather than random outcomes you are building a brittle future that is likely to disappoint you.<br />
<br />
The final point I will lift from Duncan's discussion of uncommon sense is that speed of execution and iteration is another fine way to cheat the chance events that will derail your plans. Long term detailed plans are a waste of time. This is one of the foundations of agile development, where rapid iteration of product features lets you discover what your users actually do with your product, as opposed to what you thought they would do or what they say they will do.<br />
<br />
This leads to the Insanely Simple book, which talks about Apple's approach to product design, with particular emphasis on branding and marketing since Ken Segall was the guy who came up with the i in iMac and has many other fascinating stories. One reason I like working at Netflix is that for agile web services, product ideas can be built and tested in a week or a month, and fixed in minutes. For Apple they work on products for years and need to have them work perfectly when they are released. This gives them two big problems, since its hard to iterate and hard to test ideas and products in advance. Their solution seems to be that they allow better ideas to form and develop, take bigger risks and make decisions faster than their competition, which helps stay ahead of the market. The Insanely Simple design philosophy is based on the idea that its easy to listen to all those great common sense ideas about features your product has to have, but if you learn to ignore the common sense and give the customers a simple and distilled experience you will reach beyond the people who want a complicated product and find a much bigger market of people who were waiting for a simple way to get something done. Apple's competitors are so bogged down in committees and approval processes, and helpful common sense advice from customers that they are unable to release simple products.<br />
<br />
A key example from the book is that Apple has had many award winning advertising campaigns, "Think Different", "PC and Mac" and the iPod silhouette, and none of them were test marketed in advance. Their competitors make less risky adverts after getting broad internal consensus, take much longer to get them to market and fail to understand that the success of an advert is a randomized event (with lots of useless common sense post rationalizations) so the test market response is a very poor predictor of success. It's more important to be bold, different and go big. For example Apple only advertised Think Different on the back cover of magazines, which costs far more but has a much bigger impact than inside pages.<br />
<br />
From these three books I've found some useful focus on how to approach things, but they also give me some backup to explain to others why I think some things are important. A key part of what I have been doing for Netflix is looking out into the future of cloud and related technologies and developing a portfolio of fuzzy strategies and options. They don't all work out, but by having a well instrumented but loosely coordinated architecture that doesn't have central control and strict processes we can iterate rapidly, adopt (and discard) interesting new technologies as they come along. We can all have more fun and less frustration making Netflix Insanely Simple, and ignore all the bad common sense advice and analyst opinions that swirl around everything we do.<br />
<br />
I'm planning a complete re-write of my cloud architecture tutorial for <a href="http://gluecon.com/2012/" target="_blank">Gluecon in May</a>, that will be a great opportunity to discuss these things in person over a few beers, and now is a good time to sign up to attend - you can get a 10% discount with code spkr12.Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com2tag:blogger.com,1999:blog-7434008.post-25763940411450044432012-03-19T22:45:00.006-07:002012-03-27T12:04:03.008-07:00Ops, DevOps and PaaS (NoOps) at NetflixThere has been a sometimes heated discussion on twitter about the term NoOps recently, and I've been quoted extensively as saying that NoOps is the way developers work at Netflix. However, there are teams at Netflix that do traditional Operations, and teams that do DevOps as well. To try and clarify things I need to explain the history and current practices at Netflix in chunks of more than 140 characters at a time.<br /><br />When I joined Netflix about five years ago, I managed a development team, building parts of the web site. We also had an operations team who ran the systems in the single datacenter that we deployed our code to. The systems were high end IBM P-series virtualized machines with storage on a virtualized Storage Area Network. The idea was that this was reliable hardware with great operational flexibility so that developers could assume low failure rates and concentrate on building features. In reality we had the usual complaints about how long it took to get new capacity, the lack of consistency across supposedly identical systems, and failures in Oracle, in the SAN and the networks, that took the site down too often for too long.<br /><br />At that time we had just launched the streaming service, and it was still an experiment, with little content and no TV device support. As we grew streaming over the next few years, we saw that we needed higher availability and more capacity, so we added a second datacenter. This project took far longer than initial estimates, and it was clear that deploying capacity at the scale and rates we were going to need as streaming took off was a skill set that we didn't have in-house. We tried bringing in new ops managers, and new engineers, but they were always overwhelmed by the fire fighting needed to keep the current systems running.<br /><br />Netflix is a developer oriented culture, from the top down. I sometimes have to remind people that our CEO Reed Hastings was the founder and initial developer of Purify, which anyone developing serious C++ code in the 1990's would have used to find memory leaks and optimize their code. Pure Software merged with Atria and Rational before being swallowed up by IBM. Reed left IBM and formed Netflix. Reed hired a team of very strong software engineers who are now the VPs who run developer engineering for our products. When we were deciding what to do next Reed was directly involved in deciding that we should move to cloud, and even pushing us to build an aggressively cloud optimized architecture based on NoSQL. Part of that decision was to outsource the problems of running large scale infrastructure and building new datacenters to AWS. AWS has far more resources to commit to getting cloud to work and scale, and to building huge datacenters. We could leverage this rather than try to duplicate it at a far smaller scale, with greater certainty of success. So the budget and responsibility for managing AWS and figuring out cloud was given directly to the developer organization, and the ITops organization was left to run its datacenters. In addition, the goal was to keep datacenter capacity flat, while growing the business rapidly by leveraging additional capacity on AWS.<br /><br />Over the next three years, most of the ITops staff have left and been replaced by a smaller team. Netflix has never had a CIO, but we now have an excellent VP of ITops Mike Kail (@mdkail), who now runs the datacenters. These still support the DVD shipping functions of Netflix USA, and he also runs corporate IT, which is increasingly moving to SaaS applications like Workday. Mike runs a fairly conventional ops team and is usually hiring, so there are sysadmin, database,, storage and network admin positions. The datacenter footprint hasn't increased since 2009, although there have been technology updates, and the over-all size is order-of-magnitude a thousand systems.<br /><br />As the developer organization started to figure out cloud technologies and build a platform to support running Netflix on AWS, we transferred a few ITops staff into a developer team that formed the core of our DevOps function. They build the Linux based base AMI (Amazon Machine Image) and after a long discussion we decided to leverage developer oriented tools such as Perforce for version control, Ivy for dependencies, Jenkins to automate the build process, Artifactory as the binary repository and to construct a "bakery" that produces complete AMIs that contain all the code for a service. Along with AWS Autoscale Groups this ensured that every instance of a service would be totally identical. Notice that we didn't use the typical DevOps tools Puppet or Chef to create builds at runtime. This is largely because the people making decisions are development managers, who have been burned repeatedly by configuration bugs in systems that were supposed to be identical.<br /><br />By 2012 the cloud capacity has grown to be order-of-magnitude 10,000 instances, ten times the capacity of the datacenter, running in nine AWS Availability zones (effectively separate datacenters) on the US East and West coast, and in Europe. A handful of DevOps engineers working for Carl Quinn (@cquinn - well known from the Java Posse podcast) are coding and running the build tools and bakery, and updating the base AMI from time to time. Several hundred development engineers use these tools to build code, run it in a test account in AWS, then deploy it to production themselves. They never have to have a meeting with ITops, or file a ticket asking someone from ITops to make a change to a production system, or request extra capacity in advance. They use a web based portal to deploy hundreds of new instances running their new code alongside the old code, put one "canary" instance into traffic, if it looks good the developer flips all the traffic to the new code. If there are any problems they flip the traffic back to the previous version (in seconds) and if it's all running fine, some time later the old instances are automatically removed. This is part of what we call NoOps. The developers used to spend hours a week in meetings with Ops discussing what they needed, figuring out capacity forecasts and writing tickets to request changes for the datacenter. Now they spend seconds doing it themselves in the cloud. Code pushes to the datacenter are rigidly scheduled every two weeks, with emergency pushes in between to fix bugs. Pushes to the cloud are as frequent as each team of developers needs them to be, incremental agile updates several times a week is common, and some teams are working towards several updates a day. Other teams and more mature services update every few weeks or months. There is no central control, the teams are responsible for figuring out their own dependencies and managing AWS security groups that restrict who can talk to who.<br /><br />Automated deployment is part of the normal process of running in the cloud. The other big issue is what happens if something breaks. Netflix ITops always ran a Network Operations Center (NOC) which was staffed 24x7 with system administrators. They were familiar with the datacenter systems, but had no experience with cloud. If there was a problem, they would start and run a conference call, and get the right people on the call to diagnose and fix the issue. As the Netflix web site and streaming functionality moved to the cloud it became clear that we needed a cloud operations reliability engineering (CORE) team, and that it would be part of the development organization. The CORE team was lucky enough to get Jeremy Edberg (@jedberg - well know from running Reddit) as its initial lead engineer, and also picked up some of the 24x7 shift sysadmins from the original NOC. The CORE team is still staffing up, looking for Site Reliability Engineer skill set, and is the second group of DevOps engineers within Netflix. There is a strong emphasis on building tools too make as much of their processes go away as possible, for example they have no run-books, they develop code instead,<br /><br />To get themselves out of the loop, the CORE team has built an alert processing gateway. It collects alerts from several different systems, does filtering, has quenching and routing controls (that developers can configure), and automatically routes alerts either to the PagerDuty system (a SaaS application service that manages on call calendars, escalation and alert life cycles) or to a developer team email address. Every developer is responsible for running what they wrote, and the team members take turns to be on call in the PagerDuty rota. Some teams never seem to get calls, and others are more often on the critical path. During a major production outage con call, the CORE team never make changes to production applications, they always call a developer to make the change. The alerts mostly refer to business transaction flows (rather than typical operations oriented Linux level issues) and contain deep links to dashboards and developer oriented Application Performance Management tools like AppDynamics which let developers quickly see where the problem is at the Java method level and what to fix,<br /><br />The transition from datacenter to cloud also invoked a transition from Oracle, initially to SimpleDB (which AWS runs) and now to Apache Cassandra, which has its own dedicated team. We moved a few Oracle DBAs over from the ITops team and they have become experts in helping developers figure out how to translate their previous experience in relational schemas into Cassandra key spaces and column families. We have a few key development engineers who are working on the Cassandra code itself (an open source Java distributed systems toolkit), adding features that we need, tuning performance and testing new versions. We have three key open source projects from this team available on github.com/Netflix. Astyanax is a client library for Java applications to talk to Cassandra, CassJmeter is a Jmeter plugin for automated benchmarking and regression testing of Cassandra, and Priam provides automated operation of Cassandra including creating, growing and shrinking Cassandra clusters, and performing full and incremental backups and restores. Priam is also written in Java. Finally we have three DevOps engineers maintaining about 55 Cassandra clusters (including many that span the US and Europe), a total of 600 or so instances. They have developed automation for rolling upgrades to new versions, and sequencing compaction and repair operations. We are still developing our Cassandra tools and skill sets, and are looking for a manager to lead this critical technology, as well as additional engineers. Individual Cassandra clusters are automatically created by Priam, and it's trivial for a developer to create their own cluster of any size without assistance (NoOps again). We have found that the first attempts to produce schemas for Cassandra use cases tend to cause problems for engineers who are new to the technology, but with some familiarity and assistance from the Cloud Database Engineering team, we are starting to develop better common patterns to work to, and are extending the Astyanax client to avoid common problems.<br /><br />In summary, Netflix stil does Ops to run its datacenter DVD business. we have a small number of DevOps engineers embedded in the development organization who are building and extending automation for our PaaS, and we have hundreds of developers using NoOps to get their code and datastores deployed in our PaaS and to get notified directly when something goes wrong. We have built tooling that removes many of the operations tasks completely from the developer, and which makes the remaining tasks quick and self service. There is no ops organization involved in running our cloud, no need for the developers to interact with ops people to get things done, and less time spent actually doing ops tasks than developers would spend explaining what needed to be done to someone else. I think that's different to the way most DevOps places run, but its similar to other PaaS enviroments, so it needs it's own name, NoOps. [Update: the DevOps community argues that although it's different, it's really just a more advanced end state for DevOps, so lets just call it PaaS for now, and work on a better definition of DevOps].Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com23tag:blogger.com,1999:blog-7434008.post-45526522884564988052012-03-16T22:19:00.003-07:002012-03-16T22:38:45.111-07:00Cloud Architecture TutorialI presented a whole day tutorial at <a href="http://qconlondon.com/">QCon London</a> on March 5th, and presented subsets at an AWS Meetup, a Big Data / Cassandra Meetup and a Java Meetup the same week. I updated the slides and split them into three sections which are hosted on <a href="http://slideshare.net/adrianco">http://slideshare.net/adrianco</a> along with all my other presentations. You can find many more related slide decks at <a href="http://slideshare.net/Netflix">http://slideshare.net/Netflix</a> and other information at the <a href="http://techblog.netflix.com">Netflix Tech Blog</a>.<br /><br />The first section tells the story of why Netflix migrated to cloud, how we think about choosing AWS as our cloud supplier and what features of the Netflix site were moved to the cloud over the last three years.<br /><br /><div style="width:425px" id="__ss_11995595"> <strong style="display:block;margin:12px 0 4px"><a href="http://www.slideshare.net/adrianco/cloud-architecture-tutorial" title="Cloud Architecture Tutorial - Why and What (1of 3) " target="_blank">Cloud Architecture Tutorial - Why and What (1of 3) </a></strong> <iframe src="http://www.slideshare.net/slideshow/embed_code/11995595?rel=0" width="425" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no"></iframe> <div style="padding:5px 0 12px"> View more presentations from <a href="http://www.slideshare.net/adrianco" target="_blank">Adrian Cockcroft</a> </div> </div><br /><br />The second section is a detailed explanation of the globally distributed Java based Platform as a Service (PaaS) we built, the open source components that we depend on, and the open source projects that we have started to share at <a href="http://netflix.github.com">http://netflix.github.com</a>.<br /><br /><div style="width:425px" id="__ss_11994976"> <strong style="display:block;margin:12px 0 4px"><a href="http://www.slideshare.net/adrianco/arch-tutoriallo2of3" title="Cloud Architecture Tutorial - Platform Component Architecture (2of3)" target="_blank">Cloud Architecture Tutorial - Platform Component Architecture (2of3)</a></strong> <iframe src="http://www.slideshare.net/slideshow/embed_code/11994976?rel=0" width="425" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no"></iframe> <div style="padding:5px 0 12px"> View more presentations from <a href="http://www.slideshare.net/adrianco" target="_blank">Adrian Cockcroft</a> </div> </div><br /><br />The final section talks about how we run these PaaS services in the cloud, and includes details of our million writes per second scalability benchmark.<br /><br /><div style="width:425px" id="__ss_11994912"> <strong style="display:block;margin:12px 0 4px"><a href="http://www.slideshare.net/adrianco/cloud-architecture-tutorial-part-1of3" title="Cloud Architecture Tutorial - Running in the Cloud (3of3)" target="_blank">Cloud Architecture Tutorial - Running in the Cloud (3of3)</a></strong> <iframe src="http://www.slideshare.net/slideshow/embed_code/11994912?rel=0" width="425" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no"></iframe> <div style="padding:5px 0 12px"> View more presentations from <a href="http://www.slideshare.net/adrianco" target="_blank">Adrian Cockcroft</a> </div> </div><br /><br />If you would like to see these slides presented in person, I'm teaching a half day cloud architecture tutorial at <a href="http://gluecon.com/2012/">Gluecon in Broomfield Colorado</a> in May 23-24th. I hope to see you there...Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com2tag:blogger.com,1999:blog-7434008.post-67177550890977371722012-01-19T14:25:00.000-08:002012-01-20T13:20:10.677-08:00Thoughts on SimpleDB, DynamoDB and CassandraI've been getting a lot of questions about DynamoDB, and these are my personal thoughts on the product, and how it fits into cloud architectures.<br /><br />I'm excited to see the release of DynamoDB, it's a very impressive product with great performance characteristics. It should be the first option for startups and new cloud projects on AWS. I also think it marks the turning point on solid state disks, they will be the default for new database products and benchmarks going forward.<br /><br />There are a few use cases where SimpleDB may still be useful, but DynamoDB replaces it in almost all cases. I've talked about the history of Netflix use of SimpleDB before, but it's relevant to the discussion on DynamoDB, so here goes.<br /><br />When Netflix was looking at moving to cloud about three years ago we had an internal debate about how to handle storage on AWS. There were strong proponents for using MySQL, SimpleDB was fairly new, and other alternatives were nascent NoSQL projects or expensive enterprise software. We started some pathfinder projects to explore the two alternatives and decided to port an existing MySQL app to AWS, while building a replication pipeline that copied data out of our Oracle datacenter systems into SimpleDB to be consumed in the cloud. The MySQL experience showed that we would have trouble scaling, and SimpleDB seemed reliable, so we went ahead and kept building more data sources on SimpleDB, with large blobs of data on S3.<br /><br />Along the way we put memcached in front of SimpleDB and S3 to improve read latency. The durability of SimpleDB is its strongest point, we have had Oracle and SAN data corruption bugs in the Datacenter over the last few years, but never lost or corrupted any SimpleDB data. The limitations of SimpleDB are its biggest problem. We worked around limits on table size, row and attribute size, and per-request overhead caused by http requests needing to be authenticated for every call.<br /><br />So the lesson here is that for a first step into NoSQL, we went with a hosted solution so that we didn't have to build a team of experts to run it, and we didn't have to decide in advance how much scale we needed. Starting again from scratch today, I would probably go with DynamoDB. It's a low "friction" and developer friendly solution.<br /><br />Late in 2010 we were planning the next step, turning off Oracle and making the cloud the master copy of the data. One big problem is that our backups and archives were based on Oracle, and there was no way to take a snapshot or incremental backup of SimpleDB. The only way to get data out in bulk is to run "SELECT * FROM table" over HTTP and page the requests. This adds load, takes too long, and costs a lot because SimpleDB charges for the time taken in SELECT calls.<br /><br />We looked at about twenty NoSQL options during 2010, trying to understand what the differences were, and eventually settled on Cassandra as our candidate for prototyping. In a week or so, we had ported a major data source from SimpleDB to Cassandra and were getting used to the new architecture, running benchmarks etc. We evaluated some other options, but decided to take Cassandra to the next stage and develop a production data source using it.<br /><br />The things we liked about Cassandra were that it is written in Java (we have a building full of Java engineers), it is packed full of state of the art distributed systems algorithms, we liked the code quality, we could get commercial support from Datastax, it is scalable and as an extra bonus it had multi-region support. What we didn't like so much was that we had to staff a team to own running Cassandra for us, but we retrained some DBAs and hired some good engineers including Vijay Parthasarathy, who had worked on the multi-region Cassandra development at Cisco Webex and who recently became a core committer on the Apache Cassandra project. We also struggled with the Hector client library, and have written our own (which we plan to release soon). The blessing and a curse of Cassandra is that it is an amazingly fast moving project. New versions come fast and furiously, which makes it hard to pick a version to stabilize on, however the changes we make turn up in the mainstream releases after a few weeks. Saying "Cassandra doesn't do X" is more of a challenge than a statement. If we need "X" we work with the rest of the Apache project to add it.<br /><br />Throughout 2010 the product teams at Netflix gradually moved their backend data sources to Cassandra, we worked out the automation we needed to do easy self service deployments and ended up with a large number of clusters. In preparation for the UK launch we also made changes to Cassandra to better support multi-region deployment on EC2, and we are currently running several Cassandra clusters that span the US and European AWS regions.<br /><br />Now that DynamoDB has been released, the obvious question is whether Netflix has any plans to use it. The short answer is no, because it's a subset of the Cassandra functionality that we depend on. However that doesn't detract from the major step forward from SimpleDB in performance, scalability and latency. For new customers, or people who have outgrown the scalability of MySQL or MongoDB, DynamoDB is an excellent starting point for data sources on AWS. The advantages of zero administration combined with the performance and scalability of a solid state disk backend are compelling.<br /><br />Personally my main disappointment with DynamoDB is that it doesn't have any snapshot or incremental backup capability. The AWS answer is that you can extract data into EMR then store it in S3. This is basically the same answer as SimpleDB, it's a full table scan data extraction (which takes too long and costs too much and isn't incremental). The mechanism we built for Cassandra leverages the way that Cassandra writes immutable files to get a callback and compress/copy them to S3 as they are written, it's extremely low overhead. If we corrupt our data with a code bug and need to roll back, or take a backup in production and restore in test, we have all the files archived in S3.<br /><br />One argument against DynamoDB is that DynamoDB is on AWS only, so customers could get locked in, however it's easy to upgrade applications from SimpleDB, to DynamoDB and to Cassandra. They have similar schema models, consistency options and availability models. It's harder to go backwards, because Cassandra has more features and fewer restrictions. Porting between NoSQL data stores is trivial compared to porting between relational databases, due to the complexity of the SQL language dialects and features and the simplicity of the NoSQL offerings. Starting out on DynamoDB then switching to Cassandra when you need more direct control over the installation or Cassandra specific features like multi-region support is a very viable strategy.<br /><br />As early adopters we have had to do a lot more pioneering engineering work than more recent cloud converts. Along the way we have leveraged AWS heavily to accelerate our own development, and built a lot of automation around Cassandra. While SimpleDB has been a minor player in the NoSQL world DynamoDB is going to have a much bigger impact. Cassandra has matured and got easier to use and deploy over the last year but it doesn't scale down as far. By that I mean a single developer in a startup can start coding against DynamoDB without needing any support and with low and incremental costs. The smallest Cassandra cluster we run is six m1.xlarge instances spread over three zones with triple replication.<br /><br />I've been saying for a while that 2012 is the year that NoSQL goes mainstream, DynamoDB is another major step in validating that move. The canonical CEO to CIO conversation is moving from 2010: "what's our cloud strategy?", 2011: "what's our big data strategy?" to 2012: "what's our NoSQL strategy?".Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com9tag:blogger.com,1999:blog-7434008.post-4823303557016111322011-12-30T18:25:00.000-08:002011-12-31T00:12:44.832-08:00How Netflix gets out of the way of innovation#defrag 2011 presentation script.<br /><br />I'm the cloud architect for Netflix, but rather than tell you about why we moved Netflix to a cloud architecture or how we built our cloud architecture, I'm going to tell you what we do differently at Netflix to create a culture that supports innovation.<br /><br />What is it that lets us get things done very quickly. Sometimes a bit too qwikly…. but how did we keep making big strategic moves, from DVD to streaming, from Datacenter to Public Cloud, from USA only to International, all in very short timescales with a fairly small team of engineers.<br /><br />My presentation slides are just box-shots of movies and TV shows that are available on Netflix streaming. This script is based on the notes I made to figure out what I was going to say for each box shot. If some of you see a show you didn't know we had and want to watch that would make me happy, you can click on the box shot to visit that movie at Netflix, they were all available for streaming in the USA at the time of writing.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/How_the_Universe_Works/70211549"><img src="http://cdn-9.nflximg.com/en_us/boxshots/tv_sdp_s/70211549.jpg" border="0" width="420"></a></center><br /><br />I've attempted to match the box shots loosely as cues to what I'm saying, but I've also used a musical theme in places since this is for Defrag and Defrag rocks!<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Amazon_IMAX/70000637"><img src="http://cdn-7.nflximg.com/en_us/boxshots/ghd/70000637.jpg" border="0"></a></center><br /><br />Netflix is now one of the largest sites that runs almost entirely on public cloud infrastructure. We have become a poster child for how to build an architecture that takes full advantage of the Amazon Web Services cloud. But when I talk to other large companies about what we have done, they seem to have a lot of reasons why they couldn't or didn't do what we did, even if they wanted to.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/The_Walking_Dead/70177057"><img src="http://cdn-7.nflximg.com/en_us/boxshots/tv_sdp_s/70177057.jpg" border="0" width="420"></a></center><br /><br />Why is that? Why are we heading in one direction while everyone else is going the other way? Are we crazy or are they zombies? Well, I've worked at other large companies so I have some perspective on the issues.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/The_Story_of_1/70046548"><img src="http://cdn-8.nflximg.com/en_us/boxshots/ghd/70046548.jpg" border="0"></a></center><br /><br />Before I joined Netflix I worked at eBay for a few years, and helped found eBay Research Labs. This was setup because eBay felt it wasn't innovating fast enough, and they were looking for the one missing ingredient that would drive more innovation into the company.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Beer_Pong_Saved_My_Life/70183286"><img src="http://cdn-6.nflximg.com/en_us/boxshots/ghd/70183286.jpg" border="0"></a></center><br /><br />This is a fairly common approach. <span style="font-style:italic;">"You guys go and be innovative, then hopefully we will find ways to spread it around a bit."</span> Unfortunately the end result of setting up a separate group to add innovation to a big company is more comical than useful.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Airplane/60002777"><img src="http://cdn-7.nflximg.com/en_us/boxshots/ghd/60002777.jpg" border="0"></a></center><br /><br />The most interesting projects got tied in knots, they trod on too many toes or were scary. We visited Xerox Parc and IBM Santa Teresa Labs to discuss how they were setup, to try and learn what might work., and we went to an Innovation Forum in New York. That was weird, some of the primary examples they were talking about emulating were eBay and Paypal! <br /><br /><center><a href="http://movies.netflix.com/WiMovie/Gnomeo_and_Juliet/70140917"><img src="http://cdn-7.nflximg.com/en_us/boxshots/ghd/70140917.jpg" border="0"></a></center><br /><br />The projects that did get out were minor tweaks to existing ideas, they could be fun, but ultimately not very interesting.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Breaking_Bad/70143836"><img src="http://cdn-6.nflximg.com/en_us/boxshots/tv_sdp_s/70143836.jpg" border="0" width="420"></a></center><br /><br />So I had to break out of there and find something new to do, and in 2007 I joined Netflix just as they first launched streaming.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Being_John_Malkovich/28363298"><img src="http://cdn-8.nflximg.com/en_us/boxshots/ghd/28363298.jpg" border="0"></a></center><br /><br />One of the key attractions for me was the Netflix culture I heard about in the interviews, I wanted to get inside their heads and figure out if what they were describing was real, and if so, was it sustainable as the company grew.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Strange_Culture/70066357"><img src="http://cdn-7.nflximg.com/en_us/boxshots/ghd/70066357.jpg" border="0"></a></center><br /><br />What I found out over the next few years is that the culture is what enables innovation, so that Netflix can get things done quickly that other companies are too scared or too slow to try. The rest of this talk is about the key things that we do differently at Netflix.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Dilbert/70202747"><img src="http://cdn-7.nflximg.com/en_us/boxshots/tv_sdp_s/70202747.jpg" border="0" width="420"></a></center><br /><br />Before I get into them I want to warn you that even with a roadmap and a guide, you probably won't be able to follow this path if you are in a large established company. Your existing culture won't let you. However if you are creating a new company from scratch, I hope you can join me in what I hope is the future of cool places to work.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Red_Dwarf/70140428"><img src="http://cdn-8.nflximg.com/en_us/boxshots/tv_sdp_s/70140428.jpg " border="0" width="420"></a></center><br /><br />Here's the key insight. It's the things you don't do that make the difference. You don't add innovation to a company culture, you get out of its way.<br /><br />I'm mostly going for SciFi at this point, because it's going to sound like I was beamed in from the future to some of you.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Futurama/70153380"><img src="http://cdn-0.nflximg.com/en_us/boxshots/tv_sdp_s/70153380.jpg" width="420" " border="0"></a></center><br /><br />Let me repeat that. You have to setup a company that doesn't do many of the things you would consider business as usual. That's why it's so hard to retrofit.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Madonna_Truth_or_Dare/1068501"><img src="http://cdn-1.nflximg.com/en_us/boxshots/ghd/1068501.jpg" border="0"></a></center><br /><br />How about some audience participation? Hands up everyone who hates answering questions by putting their hands up..<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Mega_Shark_Versus_Giant_Octopus/70117672"><img src="http://cdn-2.nflximg.com/en_us/boxshots/ghd/70117672.jpg" border="0"></a></center><br /><br />Who works at a company that has more than one product line? Do you get along? The problem is that the company loses focus and has trouble allocating resources where they are needed so there are big fights. Pick one big thing and do it well. For Netflix, our addressable market is everyone in the universe who likes watching movies and TV shows, that should keep us busy for a while.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Star_Trek_The_Next_Generation/70158329"><img src="http://cdn-9.nflximg.com/en_us/boxshots/tv_sdp_s/70158329.jpg" border="0" width="420"></a></center><br /><br />Who has teams spread over multiple sites and countries? We don't. It adds communication and synchronization overhead that slows your organization down. For the geeks, think of Amdahl's law applied to people. We have as many people as possible in the same building on the same site. We are planning a new bigger building to make sure we can keep everyone close together. High bandwidth, low latency communication.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Take_Me_Home_Tonight/70117577"><img src="http://cdn-7.nflximg.com/en_us/boxshots/ghd/70117577.jpg" border="0"></a></center><br /><br />Who's worked for a place that bought another company, then run it into the ground, laid everyone off and wrote down the value. Over and over again. It's crazy. I don't think Netflix has ever bought another company. It's a huge disruption to the culture, if you see something you like just hire away their best people and out execute them in the market.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Iron_Man_2/70111470"><img src="http://cdn-0.nflximg.com/en_us/boxshots/ghd/70111470.jpg" border="0"></a></center><br /><br />Who has junior engineers, graduate hires and interns writing code? We don't. We find that engineers who cost twice as much are far more than twice as productive, and need much less management overhead. <span style="font-weight:bold;">Reducing management overhead</span> is a key enabler for an innovative culture. Engineers who don't need to be managed are worth paying extra for. We are tiny compared to companies like Google, they take on raw talent and develop it, we sometimes take a chance on someone with only five years experience.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Tron_Legacy/70124805"><img src="http://cdn-5.nflximg.com/en_us/boxshots/ghd/70124805.jpg" border="0"></a></center><br /><br />Who has an architecture review board and centralized coding standards? We don't have that either. What we do have is tooling that creates a path of least resistance, which combined with peer pressure keeps quality high. The engineers are free and responsible for figuring it out for themselves.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/The_IT_Crowd/70140450"><img src="http://cdn-0.nflximg.com/en_us/boxshots/tv_sdp_s/70140450.jpg" border="0" width="420"></a></center><br /><br />Who has an ITops team that owns keeping code running in production? We don't. The developers run what they wrote. Everyone's cell phone is in the pagerduty rota, the trick is making sure you don't need to get called. All the ops people here have horror stories of stupid developers, and vice versa, but it doesn't have to be that way. We have one dev organization that does everything and no IT ops org involvement in our AWS cloud deployment.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Star_Trek_Deep_Space_Nine/70158330"><img src="http://cdn-0.nflximg.com/en_us/boxshots/tv_sdp_s/70158330.jpg" border="0" width="420"></a></center><br /><br />Who has to ask permission before deploying 100's or 1000's of servers? We don't. The developers use our portal directly, they have to file a Change Management ticket to record what they did if it's in production, that's all. We've trained our developers to operate their own code. We create and destroy up to 1000 servers a day, just pushing new code. AWS takes about 5 minutes to allocate 100 servers, it takes longer than that just to boot Linux on them.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/30-Second_Bunnies_Theatre/70142341"><img src="http://cdn-1.nflximg.com/en_us/boxshots/tv_sdp_s/70142341.jpg" border="0" width="420"></a></center><br /><br />Who has a centralized push cycle and has to wait for the next "train" before they can ship their code? We don't. Every team manages their own release schedule. New code updates frequently, and the pace slows for mature services. Teams are responsible for managing interface evolution and dependencies themselves. Freedom and responsibility again.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Battlestar_Galactica/70136119"><img src="http://cdn-9.nflximg.com/en_us/boxshots/tv_sdp_s/70136119.jpg" border="0" width="420"></a></center><br /><br />Who has project managers tracking deliverables? We don't. The line managers do it themselves. They own the resources and set the context for their teams. They have time to do this because we took the BS out of their role.<br />Managers have to be great at hiring, technical and hands on enough to architect what their team does, and project manage to deliver results. Don't split this into three people. Reduce management overhead, minimize BS and time wasted. Teams are typically 3-7 people. Have a weekly team meeting and 1on1 with each engineer to maintain context.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Metropolis_Restored/70132372"><img src="http://cdn-2.nflximg.com/en_us/boxshots/ghd/70132372.jpg" border="0"></a></center><br /><br />Who has a single standard for development tools? We don't. We assume developers already know how to make themselves productive. We provide some common patterns to get new hires started, like Eclipse, IntelliJ, on Mac, Windows. Some people use Emacs on Linux. Hire experienced engineers who care, and they will take care of code quality and standards without being told how to.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Dumb_and_Dumber_Unrated/70043038"><img src="http://cdn-8.nflximg.com/en_us/boxshots/ghd/70043038.jpg" border="0"></a></center><br /><br />Who has to work with people they don't respect? It's much too disruptive. The only way to get high talent density is to get rid of the people who are out of their depth or coasting.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Ugly_Americans/70195797"><img src="http://cdn-7.nflximg.com/en_us/boxshots/tv_sdp_s/70195797.jpg" border="0" width="420"></a></center><br /><br />That also applies to what you might call brilliant jerks. Even if they do great work, the culture can't tolerate prima donna anti-social behavior, so people who don't trust others or share what they know don't fit in.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Shaun_the_Sheep/70155567"><img src="http://cdn-7.nflximg.com/en_us/boxshots/tv_sdp_s/70155567.jpg" border="0" width="420"></a></center><br /><br />So does that mean we value conformity? No but it's really important to be comfortable as part of a high performance team, so we look for people who naturally over-communicate and have a secure confident personality type.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Top_Gear_U.K./70140457"><img src="http://cdn-7.nflximg.com/en_us/boxshots/tv_sdp_s/70140457.jpg" border="0" width="420"></a></center><br /><br />If you haven't experienced a high performance culture, think about what it's like to drive flat out at a race track. Some people will be too scared to deal with it and drive around white knuckled at 40 mph, some will be overconfident and crash on the first corner, but for people who fit into the high performance culture it's exhilarating to push yourself to go faster each lap, and learn from your peers without a speed limit. When you take out the BS and friction, everyone gets so much more done that productivity, innovation and rapid agile development just happen. This is the key message, removing obstacles to a high performance culture is how innovation happens throughout an organization. Doing less to get more.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/A_Hard_Day_s_Night_Collector_s_Series/60023947"><img src="http://cdn-7.nflximg.com/en_us/boxshots/ghd/60023947.jpg" border="0"></a></center><br /><br />We don't pay bonuses. We don't have grades other than senior engineer, manager, director, VP. We don't count the hours or the vacation days, we say "take some". Once a year we revise everyones salary to their peers and current market rate - based on what we are paying now to hire the best people we can find.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Mad_Men/70136135"><img src="http://cdn-5.nflximg.com/en_us/boxshots/tv_sdp_s/70136135.jpg" border="0" width="420"></a></center><br /><br />We also have what sounds like a crazy stock option plan that grants options every month, vests the same day, and they last 10 years even if you leave Netflix. The net of this is less work for managers, they can concentrate on hiring top people, and almost everyone that leaves takes a pay cut. The test we make is "would you fight to keep your engineers if they tried to leave". If not, let them go now and get someone better. We don't make it hard to let people go.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Capitalism_A_Love_Story/70122701"><img src="http://cdn-1.nflximg.com/en_us/boxshots/ghd/70122701.jpg" border="0"></a></center><br /><br />Some of you may be thinking this sounds expensive, but what is the value of being incredibly productive and able to move faster than your competition? You can get out ahead and establish a leading position before anyone else realizes you are even in the game. Remember how a few years ago the "Analysts" said that Netflix the DVD company was going to get killed by other companies streaming, then all of a sudden people realized that we were streaming more bandwidth than anyone else?<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Meteor_Apocalypse/70130480"><img src="http://cdn-0.nflximg.com/en_us/boxshots/ghd/70130480.jpg" border="0"></a></center><br /><br />So what could possibly go wrong? We had a near miss recently, we went too fast, partly because we could, got unlucky and screwed up. The good thing is that Netflix could re-plan and execute on the fixes we need very quickly as well, with no internal angst and finger-pointing. Also there was an Asteroid nearby earlier this week. By the way, my stepdaughter @raedioactive was the art director for this movie.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Eric_Clapton_Crossroads_Guitar_Festival_2010/70213186"><img src="http://cdn-6.nflximg.com/en_us/boxshots/tv_sdp_s/70213186.jpg" border="0" width="420"></a></center><br /><br />So, you are at a crossroads, you could be on stage with Eric Clapton, or in the audience watching and wondering why you can't do what they are doing. It's a radically different way to construct a corporate culture, it doesn't work for everyone, and we can't all be up on stage with Eric, but the talent is out there if you start by building a culture focused on talent density to find it and keep it.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/The_Men_Who_Stare_at_Goats/70117293"><img src="http://cdn-3.nflximg.com/en_us/boxshots/ghd/70117293.jpg" border="0"></a></center><br /><br />Is it going to be the goats or the glory? I just told you all to stop doing things, what could be easier than that? It takes less process, fewer rules and simpler principles. Give people freedom, hold them responsible, replace the ones that can't or won't perform in that environment. Focus on talent density and conserving management attention span by removing the BS from their jobs.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/The_Blues_Brothers/60036233"><img src="http://cdn-3.nflximg.com/en_us/boxshots/ghd/60036233.jpg" border="0"></a></center><br /><br />This is your challenge, can you get a band together and go on a mission to save your company? Stop doing all the things that are slowing you down, and get rid of the unproductive BS that clogs up your management and engineers.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Roast_of_Charlie_Sheen/70211794"><img src="http://cdn-4.nflximg.com/en_us/boxshots/ghd/70211794.jpg" border="0"></a></center><br /><br />I will take questions in the comments or on twitter to <a href=http://www.twitter.com/adrianco>@adrianco</a>. Thank you.<br /><br /><center><a href="http://movies.netflix.com/WiMovie/Manswers/70215420"><img src="http://cdn-0.nflximg.com/en_us/boxshots/tv_sdp_s/70215420.jpg" border="0" width="420"></a></center><br /><br />Each question got a new box shot, but all the answers were musical.Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com20tag:blogger.com,1999:blog-7434008.post-17050285702638457072011-09-22T00:33:00.000-07:002012-12-23T10:58:27.627-08:00The economics of Nissan Leaf ownership[Updated December 2012] We got our @NissanLeaf at the end of June 2011, and are very happy with it. It's fun to drive and is our first choice for any journey within range. Since the car is nearly silent and has over 200 lb-ft of torque, I often use full power for acceleration away from the lights, which quietly leaves the other cars behind. The low noise level is also great for listening to speech or music, and the steering wheel controls and bluetooth integration with my iPhone works well for phone calls and iPod or Pandora.<br />
<br />
Laurel usually takes it for her 66 mile round trip commute, and if not, I take it on my 20 miles round trip commute, and any shopping trips at the weekend. So far we have averaged about 1000 miles per month, mostly mountain roads and freeways, which is the worst case for electrical consumption. The Leaf collects its milage and power use, and we can go back to look at the activity.<br />
<br />
The record shows 1003 miles at 3.8 miles/KWh and a total of 264.6 KWh. We pay 10c/KWh, so if we had charged at home that's $26.46. Since we charge at work for free it's more like $15. We have grid-tied solar power and time of use metering for a cheap overnight rate. The meter runs backwards during the day at a higher price, and the Leaf has a charging delay timer so that we can plug it in when we get home, then it starts charging when the cheapest rate starts at 9pm.<br />
<br />
1003 miles in the cars we would normally drive, which get about 20mpg, uses 50 gallons of premium gas at about $4.10 and about $206. So we actually saved $180 in August 2011.<br />
<br />
If Laurel drives every day, 22 work days a month at 66 miles is 1452 miles, she tops up the charge at work each day and gets about 4 miles/KWh on that route so that's under a dollar a day. Our gas cost would be $298, so she could be saving about $280/month. That takes a big chunk out of the cost of buying the car in the first place. On top of that, the servicing costs are minimal, no engine oil changes, no gearbox, and the brakes last longer because the regenerative braking system takes a lot of the load. We have had to buy a new tire after popping it on a pothole, that was about $150, installed, but could happen to any car. We could have saved on gas costs by buying a hybrid, but they are less fun to drive and you are still paying to maintain a gas engine and a very complex transmission.<br />
<br />
The icing on the cake is our white (for pure electric) car pool lane stickers, so Laurel can take the freeway in rush hour and zip silently past all the Prius drivers whose yellow (for hybrid) stickers no longer get them in the car pool lane. It took a total of ten weeks to get the license plates, then apply and get the white stickers.<br />
<br />
So the value proposition for the Leaf is that it is much more fun to drive than a high mpg economy car like a Prius, gets you in the car pool lane (if you live in California), and the purchase cost is offset by ultra-low running costs if you use it regularly.<br />
<br />
We aren't alone in figuring out that this is a good deal. At last count (end of 2012) there are more than ten Nissan Leaf owners at Netflix, along with several Volts (the latest version gets a green carpool lane sticker) and several Teslas. At Informatica, Laurel is among several Leafs and Volts sharing the chargers.<br />
<br />
We leased our car on a three year 36,000 mile plan. We included the home charger installation in the payment (about $2K), put down a $2K initial payment, got a $7500 federal rebate bundled into the deal. Actual payment including taxes as one of the first Leaf owners was over $500/month, the current deals are much lower than this and Nissan is about to release the cost reduced US built Leaf in January 2013. We got a $2500 state rebate paid directly to us after signing up for it which covers the initial payment. We leased because we think that in three years time there may be big advances in electric car technology, we could decide to keep the Leaf, or give it back and get a the pick of the 2014 models.<br />
<br />
For much more discussion about the car, the <a href="http://www.mynissanleaf.com/">My Nissan Leaf forum</a> is quite active. One thing I found there is that upgraded springs and dampers are available, since we do a lot of mountain driving, we upgraded the suspension to be stiffer and better damped than stock.<br />
<br />
The first question everyone asks is how far will the Leaf go, and the answer is between 60 and 100 miles per charge, but it depends on where you live and how hard you drive. The usable capacity of the battery pack is about 21KWh, the actual spec is 24KWh, so there is a little bit of extra capacity beyond it's "I'm empty" point. if you drive a lot of freeways at speed and climb mountains like we do, 3-4 miles/KWh gets you 60-80 miles. In a flat urban environment 4-5 miles/KWh is quite possible to get 80-100 miles.<br />
<br />
Since we live at the top of a mountain (2400ft) and work near sea level, it's a good idea to charge the car to 80% full at home, and 100% full at work. This way there is regenerative braking for the initial downhill run, which is free power and also saves the brake pads.<br />
<br />
Our "carwings" summary page for August 2011 is shown below.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip8DRL2xEkDSuPbQcgThI6JdawZ9gXPQ1a2jncQ0qz7g60PIGT5HITujDdlT13aVU1jBfWE_kwcaQEa7L8HFRPFIntH4AvDbv9_y18W5LL0Sq7czZx-wB3I4P2_dvd7jNJRAMW/s1600/Screen+shot+2011-09-22+at+12.54.35+AM.png" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5655102158133447218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip8DRL2xEkDSuPbQcgThI6JdawZ9gXPQ1a2jncQ0qz7g60PIGT5HITujDdlT13aVU1jBfWE_kwcaQEa7L8HFRPFIntH4AvDbv9_y18W5LL0Sq7czZx-wB3I4P2_dvd7jNJRAMW/s320/Screen+shot+2011-09-22+at+12.54.35+AM.png" style="cursor: hand; cursor: pointer; height: 218px; width: 320px;" /></a>Adrian Cockcrofthttp://www.blogger.com/profile/14695336135416848505noreply@blogger.com5