Interesting Videos from AWS Re:Invent 2014

There are too many, but here are the ones I've watched that I found particularly interesting. I'll add to this list as I find time to watch more of them.

To start with here's my own talk. I'm looking for feedback on it, as it's the first time I've tried this topic and I'm looking for more data to support the concepts I'm discussing.

The "must see" talk for Re:Invent is James Hamilton on Innovation at Scale.

The new EC2 Container Service that supports Docker, described by Deepak Singh

An excellent and entertaining talk on networking optimization by Becky Weiss - learn what SR-IOV means.

EC2 Instance Deep Dive by John Phillips explains HVM and storage path optimization.

The VPC Deep Dive by Kevin Miller talks about some long awaited features such as migration tools from EC2 Classic to VPC.

There were lots of Netflix talks, but from Asgard to Zuul by Ruslan included the announcement of Docker images to make trying out NetflixOSS tools trivial.

One of the most popular talks at the event was Brendan Gregg of Netflix on EC2 Performance Tuning

PFC305 Embracing Failure Fault Injection and Service Reliability - Monkeys and the new FIT failure injection system that replaces Latency Monkey.

More Netflix Talks
BDT403 Next Generation Big Data Platform at Netflix
ARC317 Maintaining a Highly Available Front Door at Massive Scale
PFC304 Effective IPC for Microservices in the Cloud
ENT209 Cloud Migration, DevOps and Distributed Systems
APP310 Using Apache Mesos in the Cloud

Public Cloud Instance Pricing Wars - Detailed Context and Analysis

As part of my opening keynote at Cloud Connect in Las Vegas I summarized the latest moves in cloud, the slides are available via the new Powered by Battery site as "The Good the Bad and the Ugly: Critical Decisions for the Cloud Enabled Enterprise". This blog post is a detailed analysis of just part of what happened.

Summary points

• AWS users should migrate from obsolete m1, m2, c1, c2 to the new m3, r3, c3 instances to get better performance at lower prices with the latest Intel CPUs.
• Any cloud benchmark or cost comparison that uses the AWS m1 family as a basis should be called out as bogus benchmarketing.
• AWS and Google instance prices are essentially the same for similar specs.
• Microsoft doesn’t appear to have the latest Intel CPUs generally available and only matches prices for obsolete AWS instances.
• IBM Softlayer pricing is still higher, especially on small instance types
• Google's statement that prices should follow Moore’s law implies that we should expect prices to halve every 18-24 months
• Pricing pages by AWS, Google Compute Engine, Microsoft Azure, IBM Softlayer
• Adrian’s spreadsheet summary of instances from the above vendors at http://bit.ly/cloudinstances
• Analysis of the prices by Rightscale

On Tuesday 25^th March 2014 Google announced some new features and steep price cuts, the next day Amazon Web Services also announced new features and matching price cuts. On Monday 31^st March Microsoft Azure also reduced prices. Many pundits repeated talking points from press releases in blog posts but unfortunately there was little attempt to understand what really happened, and explain the context and outcome. When I wrote up a summary for my opening keynote at Cloud Connect on 31^st March I looked at the actual end result and came up with a different perspective and a list of gaps.

I’m only going to discuss instance types and on-demand prices here. There was a lot more in the announcements that other people have done a good job of summarizing. The Rightscale blog linked above also gives an accurate and broader view on what was announced. I will discuss other pricing models beyond on-demand in future blog posts.

There are some things you need to know to get the right background context for the instance price cuts. The most important is to understand that AWS has two generations of instance types, and is in a transition from Intel CPU technology they introduced five or more years ago to a new generation introduced in the last year. The new generation CPUs are based on an architecture known as Sandybridge. The latest tweak is called Ivybridge and has incremental improvements that give more cores per chip and slightly higher performance. Since Google is a recent entrant to the public cloud market, all their instances types are based on Sandybridge. To correctly compare AWS prices and features with Google, there is a like-for-like comparison that can be made. AWS is encouraging the transition by pricing its newer faster instances at a lower cost than the older slower ones. In the recent announcement, AWS cut the prices by obsolete instance type families by a smaller percentage than the newer instance type families, so the gap has just widened.

Old AWS instance types have names starting with m1, m2 and c1, c2. They all have newer replacements known as m3, r3 and c3 except the smallest one – the m1.small. The newer instances have a similar amount of RAM and CPU threads, but the CPU performance is significantly higher. The new equivalents also replace small slow local disks with smaller but far faster and more reliable solid-state disks, and the underlying networks move from 1Gbit/s to 10Gbit/s. The newer instance families should also have lower failure rates.

Most people are much more familiar with the old generation instance types, and competitors write their press releases they are able to get away with claiming that they are both faster and cheaper than AWS, by comparing against the old generation products. This is an old “benchmarketing” trick – compare your new product against the competitions older and more recognizable product.

For the most commonly used instance types there is a close specification match between the AWS m3 and the Google n1-standard. They are also exactly the same price per hour. Since AWS released its changes after Google, this implies that AWS deliberately matched Google’s price. The big architectural difference between the vendors is that Google instances are diskless, all their storage is network attached, while AWS have various amounts of SSD included. The AWS hypervisor also makes slightly more memory available per instance, and ratings for the c3 imply that AWS is supplying a slightly higher CPU clock rate for that instance type. I think that this is because AWS has based its compute intensive c3 instance types on a higher clock rate Ivybridge CPU rather than the earlier Sandybridge specification. For the high memory capacity instance types it is a little different. The Google n1-himem instances have less memory available than the AWS r3 equivalents, and cost a bit less. This makes intuitive sense as this instance type is normally bought for its memory capacity.

Microsoft previously committed to match AWS prices, and in their announcement their comparisons matched the m1 range exactly at it’s new price, and they compared their memory oriented A5 instance as cheaper than an old m2.xlarge, but the A5 is an older slower CPU type, more expensive ($0.22 vs $0.18) and has less memory (14GB vs. 15GB) than the AWS r3.large. The common CPU options on Azure are aligned with the older AWS instance types. Azure does have Intel Sandybridge CPUs for compute use cases as the A8 and A9 models, but I couldn't find list pricing for them and they appear to be a low volume special option. The Azure pricing strategy ignores the current generation AWS product, so the price match guarantee doesn’t deliver. In addition the Google and AWS price changes were effective from April 1^st, but Azure takes effect May 1^st.

IBM Softlayer has a choose-what-you-want model rather than a specific set of instance types. The smaller instances are $0.10/hr where AWS and Google n1-standard-1 are $0.07/hr. As you pick a bigger instance type on Softlayer the cost doesn’t scale up linearly, while Google and AWS double the price each time the configuration doubles. The Softlayer equivalent of the n1-standard-16 is actually slightly lower cost than Google. Softlayer pricing on most instances is in the same ballpark as AWS and Azure were before the cuts, so I expect they will eventually have to cut prices to match the new level.

Gaps and Missing Features

The remaining anomaly in AWS pricing is the low-end m1.small. There is no newer technology equivalent at present, so I wouldn’t be surprised to see AWS do something interesting in this space soon. Generally AWS has a much wider range of instances than Google, but AWS is missing an m3.4xlarge to match Google's n1-standard-16, and the Google hicpu range has double the CPU to RAM ratio of the AWS c3 range so they aren’t directly comparable.

Google has no equivalent to the highest memory and CPU AWS instances, and has no local disk or SSD options. Instead they have better attached disk performance than AWS Elastic Block Store, but attached disk adds to the instance cost, and can never be as fast as local SSD inside the instance.

Microsoft Azure needs to refresh its instance type options, it has a much smaller range, older slower CPUs, and no SSD options. It doesn’t look particularly competitive.

Conclusion

If you buy hardware and capitalize it over three years, and later on there is a price cut; you don’t get to reduce your monthly costs. Towards the end your CPUs are getting old, leading to less competitive response times and higher failure rates. With public cloud vendors driving the costs down several times a year and upgrading their instances, your model of public vs. private costs needs to factor in something like Moore’s law for cost reductions and a technology refresh more often than every three years. Google actually said we should expect Moore’s law to apply in their announcement, which I interpret to mean that we can expect costs to halve about every 18-24 months. This isn’t a race to zero; it’s a proportional reduction every year. Over a three-year period the cost at the end is a third to a quarter of the cost at the start.

I still hear CIOs worry that cloud vendor lock-in would let them raise prices. This ruse is used to justify private cloud investments. Even without switching vendors, you will see repeated price reductions for the public cloud systems you are already using. This was the 42^nd price cut for AWS, the argument is ridiculous.

I’ve previously published presentation materials on costoptimization with AWS. I’m researching this area and over the coming months will publish a series of posts on all aspects of cloud optimization.

Speeding Up Innovation - Cloud Expo Europe and QCon London 2014

There is a huge transformation going on in enterprise IT at the moment. Decades old companies are discovering that IT which used to be a small side operation keeping track of things like employees and inventory, is now perhaps a thousand times more demanding and is central to their ability to compete.

• From tracking inventory levels at sales outlets to handling web based sales directly.
• From managing a few employees to managing huge numbers of connected devices and mobile applications.
• From placing print adverts in newspapers and trade journals each month to using big data analytics to do real time bidding on internet adverts and sales force optimization.

Technologies and practices that have been developed in the leading web scale companies are finding their way into cloud based services that address the broadening of demand for these technologies, while packaging them to make them easier to implement. In addition, the pace of product development has greatly increased, and the ability to get faster feedback from customers and act on it conveys a significant competitive advantage.

The opening keynote talk I gave at Cloud Expo Europe on Feb 26th is aimed at setting the context and explaining why management needs to act. It ends with a few pointers towards next steps. The slideshare below doesn't include the builds that unfold with the explanation, and there isn't a video of the event, but I will be giving similar talks at other events in the coming months.

Speeding Up Innovation from Adrian Cockcroft

On Thursday next week I will be giving a developer oriented talk at QCon London "Migrating to Microservices". The talk goes into details on transitional techniques, architectures and tools that can be used to move a product development organization from the old world to a highly agile and innovative cloud based operation. This is an excellent conference and registration is still open, so I hope to see you there.

In my new role at Battery Ventures, we are looking for opportunities to fund companies in the enterprise IT space that can support the transition to this new world.

Battery Ventures at Strata, Santa Clara

Strata’s theme this year is “Making Data Work”. The conference is sold out, but some of the sessions will be available as a live stream so you can follow along at home.

Battery Ventures kicked off the conference the night before, with friends and portfolio companies getting together nearby.  For me attending Strata is a great way to keep track of what is going on in the rapidly developing Big Data space.

On Wednesday at 1:30pm Mike Dauber of Battery Ventures is moderating “Not your typical VC panel”. (Follow Mike @dauber)

A group of VCs who invest from very early, through later stage investments talk about all things Big Data. There will be no “3 Vs” discussion here. The Panelists are committed to making this a lively discussion about topics ranging from the typical (what sectors do they want to invest in) to the atypical (what’s out there that they don’t like? Which Hadoop Distros are in trouble? Should any entrepreneur care about Data Scientists?).

On Thursday at 4:00pm I’m a panelist in“The Great Debate: Technology Creates More Jobs than it Destroys”.  (Follow me@adrianco)

Is technology creating new jobs, and ridding us of drudgery? Or is it spawning an era of rampant unemployment and class divides? That’s what we’ll be debating.

Strata also features Battery Ventures portfolio companies as sponsors:

Platfora masks the complexity of Hadoop, making it easy for customers to understand all the facts in their business across events, actions, behaviors and time.

Continuuity makes it easy for any Java developer to build, deploy, scale and manage Apache Hadoop and HBase applications in the cloud or on-premise.

We look forward to catching up with everyone and finding new and interesting people and products to discuss.

Coming up next for me is a trip to the UK. I'm presenting at Cloud Expo Europe and QCon London. 

Latest Local Music - Fractal, Beautiful Machines, Cash Pony, Moetar and Atomic Ape... and Mirthkon!

There is a huge amount of good original new music that is looking for an audience. Most of what I listen to is from bands you've probably never heard of, so I'm going to share some updates and some local gigs that are happening over the next few days in the SF Bay Area. I encourage you to go to the gigs and buy a CD, or buy the album online to support local musicians. There is a Facebook group for Bay Area Prog rock that's worth tracking.

Earlier this week Fractal released the first track of a free album of improvisations. They plan to release the tracks on soundcloud around once a month. It's hard to categorize what they sound like other than excellent musicians composing instrumentals in real time. They tend to have interesting sound effects, looped phrases and are known for using odd time signatures. I've been a non-playing member of this band since they started, helping in various ways including naming this album "Fractiles".

I've been meaning to write this for a while, and my deadline is that tonight (Thursday Jan 30th) there is a gig including Beautiful Machines in San Francisco. I can't attend, but I do have their first album and have been listening to it a lot over the last few months. They write songs with influences from 80's synth-pop bands, and remind me of Japan or Duran Duran perhaps. Their bass player Van is one of my favorite local musicians, he used to play in Headshear and has a unique and interesting style on fretless bass. Here's a trailer for a video of Animammal from their album "Disconnect :: Reconnect".



On Saturday I will be in Oakland for a gig with three of my favorite local bands. Please come along!

Moetar are a pop band with a unique sound that blends jazzy female vocals and prog rock instrumentation into some very commercial sounding songs. They don't sound like Muse, but if you like Muse I think you will like Moetar. They have a unique mix, and I've played their first album regularly over the last year or two. Their second album has been recorded and is due for release soon. I've seen them live several times, and try not to miss a gig, they are awesome and deserve to be a big hit. Here's a video they made for Butchers of Baghdad from their first album From These Small Seeds.

Cash Pony have played with Moetar several times, and I bought their CD at a gig, then just kept playing it. It's an interesting mix of instruments, with a bass sax and an electric sitar on some songs. It's interesting uptempo pop songs, and they have a very smooth sound on the CD, but I can't figure out what to compare it to. Take a listen to a live version of 1000 Layer Curry Dawg and get their Carpal Tunnel Vision Quest album.

Years ago, there was an amazing local band called Estradasphere. They released several albums that I still listen to regularly. They have dispersed, and the guitarist Jason Schimmel has a new band called Atomic Ape with a new album, Swarm. I haven't heard them before, but am looking forward to seeing them live for the first time on Saturday. Here's a rather dark video of them playing for a radio show.

Update - I just found out that on Feb 15th MirthKon and Secret Chiefs 3 are playing a gig in San Francisco. Not yet sure if I can make it to attend. Yet another of my favorite local bands, intensely complex virtuoso playing. Matt the bass player in Mirthkon is also the keyboard player in Moetar. If you like Frank Zappa you'll like MirthKon, and you should give the video below a listen.

Starting a new role at Battery Ventures

Careful what you wish for. Take some side interests and activities that aren't central to your real job, have a bit too much fun with them, mention to a friend that it would be cool to do them for real, and have him convince his company to create a completely new position tailored for you that they've never had before, and haven't heard of anyone else doing. In a week.

Over the last few years I've given a lot of talks, and the industry reaction to Netflix's all-in public cloud strategy has evolved from "It won't work" through "It only works for unicorns like Netflix" to "How do we get there?" My answer to that question is the Netflix Open Source Software (NetflixOSS) platform, consisting of around forty distinct projects shared at netflix.github.com that provides an on-ramp for applications to move to a cloud native architecture.

I have presented aspects of this work to different audiences at public conferences, private events and directly within companies. For developers, I concentrated on explaining the NetflixOSS components, DevOps and continuous delivery patterns. For managers I concentrated on how an open source and cloud native strategy delivers rapid product evolution, scalability and high availability. As interest from individual companies grew it became clear that there was an opportunity to take what I had learned and apply it directly to a wider audience.

In my new role at Battery Ventures I will be continuing my conversations with large enterprise companies planning a move to cloud native, and SaaS vendors who are re-architecting for scale, to understand the gaps and demands of this market. With that context I will work in the Battery Ventures team that is looking for opportunities to fund companies who are enabling the transformation of enterprise IT, and I will also provide advice and mentoring for portfolio companies.

The role is called Technology Fellow. It's different to the short term Entrepreneur in Residence (EIR) position, and is similar to full time investing staff. Battery just raised their tenth fund, and have a total of $900M to invest. My job is to help spend that wisely and get the best return. That also means even more conference presentations, more international coverage, and a broader range of subjects to discuss.

I will no longer explain how Netflix works, so the starting point for my new presentation materials is going to be something like my Flowcon talk with the Netflix specific parts removed. If you still want a Netflix Architecture overview talk, start with @jedberg, he's a very highly rated speaker, and he can also connect you to the many excellent speakers at Netflix that cover their own specific areas.

It¹s going to take me a little while to figure out the new role, but a big part of the job is meeting lots of people. I'm taking bookings for February...

Press Coverage

Press Release by Battery Ventures
The Full Stack blog post by Mike Dauber of Battery Ventures
Business Week story by Ashlee Vance
GigaOm story by Stacy Higginbotham
Reuters story on Yahoo UK
Boston Globe story by Michael Farrell
Wall Street Journal story by Deborah Gage
The Register story by Jack Clark
Venture Beat story by Jordan Novet

About Battery Ventures

Battery invests in cutting-edge, category-defining businesses in markets including software and services, Web infrastructure, e-commerce, digital media and industrial technologies. Founded in 1983, the firm backs companies at stages ranging from seed to buyout and invests globally from offices in Boston, Silicon Valley and Israel. To learn more, visit www.battery.com and follow the firm on Twitter @BatteryVentures.

Looking back at 2013, with pointers to 2014

I wrote a prediction post a year ago so I'm going to review and update it. Here's last year in full "Looking back at 2012, with pointers to 2013" .

The headlines from last year, with comments and updates:

Mobile Bandwidth Greater than Fixed Bandwidth
This trend continues. LTE can get congested in cities, but the latest news is Verizon upgrading it's network to have more capacity and speeds up to 100Mbit/s. I'm about to get an LTE MiFi for our house so that when we want higher speed or our local DSL gets congested, it's available, and when we go away we can take it with us. I'll keep the DSL for background connectivity, and to avoid hitting the MiFi bandwidth cap too much.


Cutting The Cable/Satellite TV Feed

We still don't have cable/satellite. I got a Google Chromecast, but it's slow and fiddly to use compared to the AppleTV, and buggy for streaming Pandora, keeps dropping the stream. The picture quality is good though.


The Netflix Open Source Cloud Platform Got Traction

We close out 2013 with 39 distinct projects at github.netflix.com, a successful Netflix Cloud Prize contest, endorsements from many more companies including IBM, and growing acceptance that Cloud Native is an important concept that supports highly agile continuous delivery, and NetflixOSS is an onramp that accelerates transitioning to Cloud Native.


Netflix Cloud Architecture Presentations

I presented even more than in 2012, see the slides I posted at http://www.slideshare.net/adrianco. I've also got a permanent link to a full set of workshop slides at bit.ly/netflix-workshop which is easier to remember, and lets me update the workshop slides from time to time.


The Concept of Anti-Fragility Took Off

Taleb's book and concept became more accepted. Ariel Tseitlin wrote an ACM paper for Netflix on The Antifragile Organization.


Cloud, Open Source, SaaS and the End of Enterprise Computing

"During 2013 we will see if Google manages to invest heavily and execute well enough to build up a big user base."

Google came out of beta and closed some gaps, but it's not clear that they are building up a big user base. They have their fans, and in some areas have some technical advantages, but still have a lot to prove. Other public cloud vendors didn't make much headway. Microsoft Azure remains strong in it's own ecosystem, but hasn't broken out into general use, and others are getting further behind or being bought.

"I personally think in 2014 we will likely see [...] the scale, features and price point of AWS and Google clouds make everyone else irrelevant." I still think this is true. The 2013 Garner Magic Quadrant for IaaS didn't include Google as they were in beta, but showed AWS as dominant. It also included an estimate that AWS delivered capacity was five times bigger than the next 14 vendors combined. i.e. AWS was 85% of the market by delivered capacity (not by $ revenue). My tracking of AWS size by looking at their reserved IP address space bit.ly/awsiprange continues to show that AWS is doubling in size every year, and has grown 10x over the last three years, reaching 5.1 million IP addresses in September 2013.

Most big enterprise companies are actively working on their AWS rollout now. Most of them are also trying to get an in-house cloud to work, with varying amounts of success, but even the best private clouds are still years behind the feature set of public clouds, which is has a big impact on the agility and speed of product development.


Solar Powered Electric Cars Are For Real Now

Our Nissan Leaf is getting towards the end of it's three year lease, and we're replacing it with a Fiat 500e. The Fiat is smaller and lighter, but has the same size battery, so gets a bit more range. It's also cheaper and more fun to drive. During 2013 a lot of people bought Tesla Model S, including people who traded in Tesla Roadsters. We picked up a second hand 2010 Tesla Roadster with full factory warranty, and although the technology is a bit older, it's a great fun car with over 200 miles real world range for longer trips. Even with two electric cars, we still generated more electricity than we used this year, so the marginal cost of energy is still zero for our household and our gasoline spend is way down.


Global Warming Arrived in the USA in 2012

"I'm going to try and re-balance my 401K retirement accounts to divest from oil companies. Many students are nowpressuring their colleges to divest from oil as well." I spent a few hours on Fidelity Investments web site and reduced my investments in the energy sector to a minimum. The divestment movement is also gathering momentum. The public conversation continues to shift, more extreme weather in the US and worldwide is helping, and the IPCC released an updated report.


Twitter and Snapchat

I had 6,500 followers on Twitter at the end of 2012, and I have 10,500 at the end of 2013. I correctly predicted that Snapchat would continue to grow in 2013, and it was reported that more photos are uploaded per day into Snapchat than into Facebook. Twitter had it's IPO, and is becoming part of the news and entertainment infrastructure with it's own ecosystem. I think they will figure out how to continue to grow and make money, so I bought a few shares to have skin in the game.

New for 2014:

Google Glass will have a successful public launch

I got Glass last summer, and have been using it a bit and letting other people try it a lot. I just got a hardware update that makes my developer set compatible with the final consumer version, and it's clear that Google is getting much closer to having a real product to launch. No-one knows the price, and that will determine how widely people get Glass, but the feature set and support is now quite interesting. There is a generational divide, in that many younger people like and want Glass, and older people are more wary or bothered by it. Trying it out in person lets people understand what it does and doesn't do, and greatly increases acceptance.

The Glass features that I was waiting for have mostly been addressed. The MyGlass app now supports iPhone, there is support for corporate Gmail accounts and multiple Gmail accounts on a single Glass. There is a headphone to supplement the built in speaker that was too quiet, and there will soon be prescription lens support. The last of those is the main reason I don't wear Glass every day, as I have to put in contact lenses to use it currently, and my contacts don't work as well as my glasses.

The ability to get personal GPS directions while walking (or cycling) is one of my favorite features. "First person" hangout support has huge potential although it's still too fiddly to setup, and needs good network bandwidth. Video use drains the battery quickly, although in normal use it lasts long enough to be useful all day. The add-on applications I have installed include Twitter, so I get notified immediately if someone mentions or DMs me; Evernote for keeping track of shared to-do lists; and Translate where you look at a sign and it makes an English version of the sign for you.

Voice control works better than most people expect, directions to local places is remarkably good, but voice input dictation is very random. It needs quite a lot of practice to get messages recorded that contain more than "on my way" or other simple phrases. But then I don't use Siri on my iPhone either.

Best wishes to everyone for 2014.

Velocity and Volume - Speed Wins - Keynote at Flowcon

November 1st was the first ever Flowcon, held in San Francisco, a new event to focus on continuous delivery related development topics.

I was honored to be asked to present as the opening keynote speaker, and I owe a big thank you to Jez Humble and Gene Kim for the invitation.

The material I presented was less focused on Netflix related technologies than most of my talks, and instead looks at the challenges and motivations of speeding up the page of development. I also tried to provide a historical perspective of how the state of the art for software delivery has changed over the last few decades.

There was a second chance to present these slides with a bit more time a few days later, and the slides for that are linked below. There's a bit more of an introduction, and a more discussion of tools on the end, but it's basically the same message.

Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is doing Continuous Delivery from Adrian Cockcroft

Free Communication is Disrupting Hierarchies

Jeff Jarvis recently blogged:

I doubt the net’s creators realized how subversive it was to connect anyone to anyone, bypassing the institutions that mediated those connections: from media to government, universities to retailers. These institutions are now circling wagons to protect their prerogatives: copyright for media, secrecy for government.

Somewhat related discussions are taking place at the Defrag conference and on blogs talking about corporate organizations and government surveillance.

However this is nothing new, over centuries there has been a gradual reduction in the cost of communication between people, which has caused social changes, disrupted old industries, created new industries and led to political revolutions. The end point that we have reached in the last few years is that the cost of communication is now zero. There is no incremental cost for me to send a tweet that is delivered to thousands of people a few seconds later. If the tweet contains novel ideas or information that people want to share, then in a few seconds, millions of people could see it. With about 10,000 direct followers my own twitter mention reach was in the millions last week.

When I see an industry or organization that was built on assumptions that are no longer true, I expect to see disruption follow, and opportunities to create new industries and organizations. The assumption that communication is expensive is baked into our world. The new industries and organizations that take advantage of this change are emerging now.

To see how subversive this is, we need to look at some history. Every step in the reduction of communication cost has enabled changes that have been resisted by the hierarchy, led to revolutions, and in most cases have been co-opted and controlled by the elite.

The ability to control, coordinate and influence a large group of people depends upon effective communication. Early examples include the creation of sea-going ships and roads to sustain empires, from the Romans to the British Empire as examples. The leaders of these empires could gather information via messengers and send out decrees to manage at a distance, by using a coordinated hierarchical communication network. The cost of communication for the workers was prohibitive, so all information trickled down through the local authorities, whether civil, military or religious. The only low cost communication took place within the local village.

Most workers were illiterate, and taking England in the middle ages as an example, a series of disruptions were caused by the invention of the printing press. The loss of control that resulted caused major problems for authorities, and allowed ideas to spread in new ways.

As cities got bigger, local communication got faster, newsletters and newspapers started to spread ideas more quickly and the creation of postal services brought the cost of peer to peer communication down far enough that everyone could communicate over larger distances. Governments co-opted these new mechanisms, regulating the news, running the postal services. The industrial revolution created commercial entities with their own hierarchies of control and communication. The ability of ideas to spread rapidly and reach many people allowed groups to self-organize, leading to political upheaval. Revolution was enabled by the ability to spread ideas outside the incumbent ruler's hierarchy of communication and control.

Today we live in societies where effectively everyone is literate, connected and mobile. However we still have the same hierarchical institutions that were developed hundreds of years ago. Governments to manage the hierarchies of society, Companies to manage the hierarchies of commerce, Religions to manage the hierarchies of belief, Universities to manage the hierarchies of education.

These hierarchies still have value, they continue to represent ownership and have responsibilities but they no longer have control. They were sustained in the past by imposing their will on people, and carefully managed what information people are exposed to. The hierarchies are now very large and deeply nested, so rather than a small ruling elite and a large peasant population, there are very many levels in the middle, which makes many of us have a vested interest in the continuation of parts of the hierarchies.

We are now all connected by a series of relatively flat peer to peer networks. Facebook for a curated network of family, friends and interest groups, twitter for real-time news and a much more dense ad-hoc interconnection of interests, email for direct communication, and many other services. The "Arab Spring", the "Occupy Movement" and other recent political revolutions were able to find and coordinate large numbers of people extremely rapidly (hours or days) without forming a hierarchy first.

The new world is a blend of hierarchies with many peer to peer links that reach across and between the old hierarchies. The problem for the elite leaders is that they are used to having control, owning the flow of information and managing deeply nested organizations. In this new world, the leaders have ownership, responsibility, influence but much less control, they control and manage a small part of the flow of information, and their organizations are flatter and less well defined. At the extreme there are organizations like Github, which are figuring out how to make an emergent self organizing structure work. To scale a flat organization there needs to be a very strong sense of culture and unity of purpose. I think Netflix has found a productive balance between a hierarchy of responsibility for aspects of the business and freedom at each level to self organize the delivery of what the business needs. Broad and open communication across the business, and many skip-level one-on-one meetings flattens the organization and minimizes "politics".

Looking beyond business, Religion only really works if you don't think too hard about any other variant of religion. Religions all claim to be true, but claim different incompatible and incoherent things, so that exposure to ideas such as the Outsider Test for Faith causes deep problems for the faithful. Religion in general is being disrupted by science and free communication, and polarization into atheism and reactionary fundamentalism is one of the outcomes in society.

Education is based on the communication of skills and information from teachers to students, and the big institutions of education are being disrupted. They no longer control the flow of education, and the costs are increasing beyond the ability of the students to repay them, so companies like Coursera and the Khan Academy are building alternatives. For software developers, the body of code you own and contributed to on github matters far more than where (or even if) you studied. The state of the art isn't being created and taught in University research departments any more, the commercial ecosystems that form around technologies are where the bleeding edge of innovation is being disseminated to new recruits.

These disruptions are on the way, but change tends to occur rapidly when a tipping point is reached rather than gradually. A point made by Nicholas Taleb in his discussion of Black Swan events, is that its easier to point out that a change could happen and plan for what to do next, than to predict when exactly it will come. So like a cartoon Wily Coyote who has run off a cliff, we ignore the change until we look down and realize that nothing is holding us up.

Performance Testing of NetflixOSS for the Cloud Prize

Where is everyone? I've been writing blogging and tweeting about performance and capacity planning for many years, so I thought I was connected to one or two performance folks....

When we setup the NetflixOSS cloud prize I created a category just for you:

Best Contribution to Performance Improvements

New performance testing tools, results of performance tests and performance oriented bugs. Since this may be aggregated across many pull requests, they should be listed and documented in the Submission.

While there are some interesting contestants in other categories, I've seen very little activity in this one.

There's $10K cash and $5K of AWS credits waiting for someone to do enough work to claim them!

It takes a bit of resource to run tests, so I also have a $100 AWS gift card that I will pass on to the first person to make an entry in this category, regardless of whether they win the prize or have generated any useful data at that point. Make a valid submission to the Cloud Prize by forking the github repo, editing the Submission.md file to describe your entry, entering your details (email/address etc) in the Mailchimp form, and then post a comment here claiming the $100 AWS card, and I'll mail it to you.

After the first one, if you make a case for having generated lots of good data, I'll get some more $100 AWS cards.

Here's some performance project ideas.

• Run throughput and latency stress tests on the Zuul API gateway on various instance types
• Run stress tests for sizing data against other service level projects - Eureka, Edda, Ice, etc.
• Make the cass_jmeter project better in some way
• Analyze the code paths in Astyanax and find ways to make it run faster and use less memory

Go for it, only a few days left. Contest ends at midnight Pacific time Sunday Sept 15th.

Best wishes

Adrian

Tutorials and Training on Cloud Architecture and NetflixOSS

There are two places that I'm giving in-depth training on cloud architecture and NetflixOSS this year.

On May 22nd-23rd at Gluecon 2013 in Broomfield Colorado I'm giving one of the opening keynote talks that introduces the concepts of a Cloud Native Architecture, then spending all afternoon on a tutorial to go into the details of how to get there and how to use NetflixOSS tools as an on-ramp to accelerate the process.

For people in Europe, I'm teaching at a summer school on "Software for the Cloud and Big Data" September 8th-14th in Italy. It's organized by ETH Zurich. I'll be giving six one-hour talks over a week, and there are seven other speakers.



When I get back from LASER, it will be time to start evaluating nominations for the NetflixOSS Cloud Prize, and these are both opportunities to figure out how to build a strong entry.

Comment on How Netflix Is Ruining Cloud Computing

I wrote a long comment response to how-netflix-is-ruining-cloud-computing on Information Week, but they don't seem in a hurry to post it. Luckily I saved a copy so here it is:

There should be a http://techblog.netflix.com post in the next day or so that will give more context to the Cloud Prize and clarify most of the points above. However I will address some of the specific issues here.

Cloud 1.0 vs. 2.0?

I would argue that the way most people are doing cloud today is to forklift part of their existing architecture into a cloud and run a hybrid setup. That's what I would call Cloud 1.0. What Netflix has done is show how to build much more agile green field native cloud applications, which might justify being called Cloud 2.0. The specific IaaS provider used underneath, and whether you do this with public or private clouds is irrelevant to the architectural constructs we've explained.

Outages

The outages that have been mentioned were regional, they didn't apply to Netflix operations in Europe for example. Our current work is to build tooling for multi-regional support on AWS (East cosat/West coast), including the DNS management that was mentioned. This removes the failure mode with the least effort and disruption to our existing operations.

Portability

Other cloud vendors have a feature set and scale comparable to AWS in 2008-2009. We're still waiting for them to catch up. There are many promises but nothing usable for Netflix itself. However there is demand to use NetflixOSS for other smaller and simpler applications, in both public and private clouds, and Eucalyptus have demonstrated Asgard, Edda and Chaos Monkey running, and will ship soon in Eucalyptus 3.3. There are signs of interest from people to add the missing features to OpenStack, CloudStack and Google Compute so that NetflixOSS can also run on them.

Edda

You've completely missed the point of Edda. It does three important things. 1) if you run at large scale your automation will overload the cloud API endpoint, Edda buffers this information and provides a query capability for efficient lookups. 2) Edda stores a history of your config, it's a CMDB that can be used to query for what changed. 3) Edda cross integrates multiple data sources, the cloud API, our own service registry Eureka, Appdynamics call flow information and can be extended to include other data sources.

AMInator

If you want to spin up 500 identical instances, having them each run Chef or Puppet after they boot creates a failure mode dependency on the Chef/Puppet service, wastes startup time, and if anything can go wrong with the install you end up with an inconsistent set of instances. By using AMInator to run Chef once at build time, there is less to go wrong at run time. It also makes red/black pushes and roll-backs trivial and reliable.

Cloud Prize

The prize includes a portability category. It's a broad category and might be won by someone who adds new language support to NetflixOSS (Erlang, Go, Ruby?) or someone who makes parts of NetflixOSS run on a broader range of IaaS options. The reality is that AWS is actually dominating cloud deployments today, so contributions that run on AWS will have the greatest utility by the largest number of people. The alternatives to AWS are being hyped by everyone else, and are showing some promise, but have some way to go.

We hope that NetflixOSS provides a useful driver for higher baseline functionality that more IaaS APIs can converge on, and move from 2008-era EC2 functionality to 2010-era EC2 functionality across more vendors. Meanwhile Netflix itself will be enjoying the benefits of 2013 AWS functionality like RedShift.

The IT swamp draining manual for anyone who is neck deep in alligators

I've spent the last year or so reviewing Gene Kim's new book - the Phoenix Project and encouraging him to get it finished. It came out this week, is the top business book on Amazon as I write this, and I got a nice back-cover quote shown below with Gene's actual finger in the photo.

Many years ago someone gave me a copy of The Goal, which is the inspiration for The Phoenix Project. In both cases the book is a novel about a company that is dysfunctional and on the verge of going out of business. The lead character is dropped into the job of figuring out how to dig their way out of the problem, and in the case of The Phoenix Project, the company fumbles its way from legacy enterprise where IT isn't regarded as central to their success, into the modern world of agile development practices and DevOps deployments where IT becomes a competitive advantage.

Don't just read it, give copies of it to your friends in management. It should be on every CxO's bookshelf.

Looking back at 2012, with pointers to 2013

A collection of things that seem to have pivoted in 2012.

Mobile Bandwidth Greater than Fixed Bandwidth

I've been talking about LTE and the growth in mobile since 2008, but I started 2012 with a Verizon iPhone 4 which maxed out at 2Mbit/s over 3G and at home in the mountains I would get less than 1Mbit/s. I ended 2012 with a Verizon iPhone 5 which is about ten times faster at home, I regularly see 8-9Mbits/s, and the best speed I have seen anywhere so far was in downtown Los Gatos at over 50Mbit/s. My home fixed wire Internet is a 3Mbit/s DSL that has neighborhood congestion at peak times. I now find it works better to have WiFi turned off on my iPhone at home.

This is one of those pivotal changes, similar to the change from having predominantly fixed wire telephone service at home, to having many people use mobile phones exclusively. It costs more, but if you already have a high bandwidth connection to your phone with a high data cap because you use it a lot, why pay to also have a low bandwidth connection to your house? Bandwidth caps and data usage plans will slow the switchover, but the writing is on the wall.

Cutting The Cable/Satellite TV Feed

In 2013 we finally turned off our TiVo and shut down our DirecTV account. We weren't using it enough to make it worth while. For some of the sports events (Laurel follows the Stanford Cardinals), we go to a sports bar to watch, which is more fun anyway. Everything else that we have time to watch, we can watch online, and we get all our news updates from Twitter, RSS feeds and Facebook posts. By the time it's on TV or in a newspaper, it's already old news.

The TV has an AppleTV connected to it, which gets almost all the usage. We watch a few things on laptops, and sometimes I connect a laptop to the TV. I also stream music from my iPhone to the AppleTV because I can't get Pandora or Spotify on it. Come on Apple, where's the AppleTV App Store? Maybe that's a 2013 thing.

The Netflix Open Source Cloud Platform Got Traction

We started the year with a handful of disconnected projects, and ended it with a large chunk of the platform on Github, and some high profile users. Most people are still picking it up piecemeal but in 2013 we plan to get the whole thing put together as an installable bundle. This is the Alan Kay approach, "The best way to predict the future is to invent it".  Netflix has been out in front of the industry in terms of cloud adoption, inventing the future. Next we make it easier for others to join us in that future, and have some ideas for how to drive adoption to new heights.

Netflix Cloud Architecture Presentations

I was going to list all the talks I gave, but there are too many, so go see the slides I posted at http://www.slideshare.net/adrianco. Highlights were QConSF, QConLondon, Gluecon in Colorado, GOTO in Aarhus and of course AWS Re:Invent in Las Vegas. The impact of these talks grew through the year, reaching a peak at Re:Invent, where we had lots of speakers and attention to the way the Netflix cloud and open source story was bringing value to the company and reaching out into the technical community. A big thanks to everyone who came to my talks, and all the other Netflix speakers who have been out there broadening the story. It's almost impossible to write an article or do a presentation about cloud without mentioning Netflix. In 2013 there will be even more talks, I focus on local and US based events that are strongly developer oriented like QCon, Gluecon, and GOTO. We will definitely be back at AWS Re:Invent next November.

The Concept of Anti-Fragility Took Off

Nassim Taleb's latest book crystallizes the way I tend to approach things and gives it a name. The Netflix cloud architecture is anti-fragile, we run "Chaos Monkey's" continuously to try and break it, and that makes it stronger. The Netflix culture is anti-fragile, it's decentralized with as little process and rules as possible and a lot of local autonomy. Netflix management is not afraid of change or of being first to do something and tends to navigate disruptive transitions well. From the outside this can look chaotic or confusing, but it works, and recovers well from missteps, which are always going to happen. If you're not failing occasionally you aren't trying hard enough, and you are missing opportunities. Getting stronger through failure is the basis of anti-fragility. Avoiding failure at all costs (as many people try to do) makes you brittle and vulnerable to unexpected Black Swan events that will have a much bigger impact.

Cloud, Open Source, SaaS and the End of Enterprise Computing

Taleb makes the point that big companies become increasingly fragile as they lose agility and the ability to move with the markets, and we are seeing that play out in the Enterprise Computing space. There is still money to be made from the late adopter customers, but the trend is clearly towards development using exclusively open source tools, with applications and infrastructure delivered as a service. There is zero revenue for traditional Enterprise Computing vendors in this model. The current interest in building out private cloud infrastructure is real and will continue to support traditional vendors into 2013, but it's a short term investment. At best you end up with a much better automated datacenter, but it isn't elastic and it has far fewer features than AWS, so it's going to be marginalized over time. At worst, you discover just how hard it is to run a reliable private cloud based on immature software, with incompatible upgrade paths, and it turns out to be much more expensive to run.

The Enterprise Computing vendors haven't been able to build a public cloud  that competes with AWS on scale, price or features, and AWS is now focused on building everything their customers need to take the next generation of application investment out of the datacenter, so the high margin revenue is going to gradually go away for the traditional vendors.

The most interesting development in 2012 was the re-launch of Google as a public cloud infrastructure vendor, and the mini-price-war between AWS and Google over instance and storage costs makes it clear where the real action is. During 2013 we will see if Google manages to invest heavily and execute well enough to build up a big user base.  In mobile, as I predicted years ago, we are now in an iPhone vs. Android battle that is wiping out everyone else. I personally think in 2014 we will likely see a similar effect as the scale, features and price point of AWS and Google clouds make everyone else irrelevant. The only question in my mind is whether AWS runs away with this on their own, or Google manages to get some traction as the alternative.

Note to sales reps (who won't listen), I'm not interested in anything to do with datacenters, private cloud, or other public clouds in 2013. I'm only interested in SaaS apps, things that run on AWS, and interesting open source projects.

Solar Powered Electrics Cars Are For Real Now

We drive our Nissan Leaf all the time, it's fun to drive and the first car we pick for most trips, adding up to almost 1000 miles a month. The marginal cost of running the car is near zero. New tires and a cabin air filter at 15K miles is all the maintenance it needs. We have an excess of solar power generation that added up to $500 in unused electricity over the year. At 10c/KWh and 3.5KWh/mile that's plenty for us run a second electric car before we start paying for the power, and there are a lot more choices coming in 2013. There are many charging stations around the Bay Area, lots of other people running Leafs, and the Tesla Model S got car of the year awards. It takes a test drive to realize what fun it is to have instant torque and no gear shifts. This is a case of the future being unevenly distributed. If you don't live in California, it's a bit further out, but it's coming.

A friend recently got a quote for Solar Power installation which was about half what we paid two years ago, and we got a good deal then. Prices have dropped fast and are much lower than most people think. If you don't already have solar panels on your roof, you should get them. If you don't use enough electricity to justify solar panels, get an electric car as well, and save at the gas pump.

Global Warming Arrived in the USA in 2012

The well funded Merchants of Doubt (read the book) managed to confuse and suppress public discussion of global warming in the USA for the last few years, but the effects just became too obvious this year and it broke through, creating the scenarios that James Hansen warned of in Storms of My Grandchildren. The arctic ice cap melt continues to accelerate, seas are warming and rising, drought and record heat hit most of the USA, and everything wrapped up with Hurricane Sandy, pushing the topic onto the front page. The dice are all loaded now, and 2013 is already rolling those dice as the drought continues and the Mississippi river is empty. I've been saying for the last few years that if you own property at sea level, you should find someone who doesn't believe in global warming to sell it to, because it's going to become increasingly uninsurable and end up as worthless as the houses along the New Jersey shoreline that were swept away.

The Republican party is still in denial, a combination of funding from big oil companies and an inability to accept or admit that their demonized Al Gore could have been right all along. In 2013 it will be interesting to see how they deal with losing the election, and perhaps there will be a split into a group of Republicans that see the path to re-election in 2014 as needing to accept reality by voting for some Global Warming related legislation, versus the hard core that are trying to pray their way out. The current battle is over stopping the Keystone XL pipeline that would move the dirtiest kind of tar oil from Alberta Canada to Texas. It may be symbolic, but if KXL is stopped, the tide will have turned. Carbon needs to be left in the ground. For 2013, I'm going to try and re-balance my 401K retirement accounts to divest from oil companies. Many students are now pressuring their colleges to divest from oil as well.

Twitter and Snapchat

Personally, 2012 was an excellent year for me, I've made lots of new friends and learned a lot by being active on twitter, ending the year with about 6500 followers. I joked on twitter that I posted my new years resolutions for 2013 to Snapchat, but you missed them. If you don't know what Snapchat is for, ask a teenager. You'll probably hear a lot more about it in 2013, then, when their parents figure it out and join too, the teens will be onto the next thing....