Can OpenStack catch up with AWS? Looks unlikely to me.

There has been a lot of chatter about the new OpenStack standard in the last few days, and how it could become a third option in the cloud infrastructure market alongside Amazon AWS and VMWare vCloud. For example Randy Bias/Cloudscaling has a good overview of the players.

The real question is whether OpenStack can catch up and become viable. Amazon is always described as being years ahead of the competition and having the lions share of the market. Estimates of the size of their lead depends on who you talk to, but I don't see anyone disputing their lead. On top of that, Amazon is investing heavily, has big customers like Zynga and Netflix stretching and hardening their systems (along with a huge number of small customers), and has already added many features beyond the basic compute (EC2) and storage (S3) that have been copied by others including OpenStack.

So for OpenStack to catch up, means that they have to move faster than Amazon, and leverage the slipstream effect where Amazon has had to educate the market and figure out what works, so competitors can copy their successes. However when I look at the details of the OpenStack specification they appear to be copying some of Amazon's problems as well. In particular the account and authentication model, which does not scale for enterprise use.

At the most basic level, it is infeasible to change the security model of a platform architecture, it's one of the most fundamental starting points that conditions the layers above. Changes to account and authentication management break all the layers of applications and tools that are built on the platform. One of the first problems that Netflix had with Amazon, was the lack of sub-accounts and role based access control (RBAC), and while beating up Amazon on this point for the last two years, we have built our own platform layers and tools to compensate. As a result, we find it impossible to use any of the web consoles or tools produced by Amazon or the many cloud vendors, which assume that there is a single account owner who can do anything. We hope that Amazon will eventually support sub-accounts, and when they do, I expect it will break everyone's tools.

At this point OpenStack is just the base level of the platform, it doesn't really have layers of tools on top yet, but its account and authentication model appears to be exactly the same as Amazon, so they will end up with layers of tooling that don't meet the needs of enterprise customers.

What's the difference between a startup and an enterprise? In a startup, everyone in IT knows the root password to every machine in their infrastructure. In an enterprise root passwords are carefully controlled, they change when someone in-the-know leaves, and specialist groups manage different parts of the system (Network ops can only mess with the switches, DBA's can only mess with the database etc.). The problem with a single account for the cloud is that everyone who needs to do anything to that account can do everything to it, and the common tools are oriented to a single user, managing several accounts, rather than a hierarchy of users managing parts of one account. All the systems in the account need an authentication key to access cloud services, and changing the password and key means you have to re-key every system. Get this wrong and your cloud will evaporate in an instant.

So in my opinion, a necessary but not sufficient condition for OpenStack to eventually catch up with AWS is that they need to build sub-accounts and RBAC into their spec from the start. However it seems much more likely that Amazon will just disappear into the distance from what I've seen so far.