I recently heard about Illumos via a tweet from Alec Muffett, and responded with my own tweet "I predict that #illumos will be just as irrelevant as Solaris has been for the last few years. Legacy." - personally I haven't logged into a Solaris or SPARC machine for about four years now. There are none at Netflix.
I have also been talking to a few friends who stayed at Sun and are now at Oracle, and there is a common thread that I decided to put out there in this blog post.
This week I presented at a local Computer Measurement Group meeting, talking about how easy it is to use the Amazon cloud to run Hadoop jobs to process terabytes of data for a few bucks [slideshare]. I followed a talk on optimizing your Mainframe software licensing costs by tweaking workload manager limits. There are still a lot of people working away on IBM Mainframes, but it's not where interesting new business models go to take over the world.
The way I see the Oracle/Sun merger is that Oracle wanted to compete more directly with IBM, and they will invest in the bits of Sun that help them do that. Oracle has a very strong focus on high margin sales, so they will most likely succeed in making good money with help from Solaris and SPARC to compete with AIX, z/OS and P-series, selling to late-adopter industries like Banking, Insurance etc. Just look where the Mainframes are still being used. Sun could never focus on just the profitable business on its own, because it had a long history of leading edge innovation that is disruptive and low margin. However, what was innovative once is now a legacy technology base of Solaris and SPARC, and it's not even a topic of discussion in the leading edge of disruptive innovators, who are running on x64 in the cloud on Linux and a free open source stack. There is no prospect of revenue for Oracle in this space, so they are right to ignore it.
That is what I meant when I tweeted that Illumos is as irrelevant as Solaris, and it is legacy computing. I don't mean Solaris will go away, I'm sure it will be the basis of a profitable business for a long time, but the interesting things are happening elsewhere, specifically in public cloud and "infrastructure as code".
You might point to Joyent, who use Solaris, and now have Bryan Cantrill on board, but they are a tiny bit-player in cloud computing and Amazon are running away with the cloud market, and creating a set of de-facto standard APIs that make it hard to differentiate and compete. You might point to enterprise or private clouds, but as @scottsanchez tweeted: "Define: Private Cloud ... 1/2 the features of a public cloud, for 4x the cost", that's not where the interesting things are happening.
So to my Sun friends at Oracle, if you want to work for a profitable company and build up your retirement fund Oracle is an excellent place to be. However, there are a lot of people who joined Sun when it was re-defining the computer industry, changing the rules, disrupting the competition. If you want some of that you need to re-tool your skill set a bit and look for stepping stones that can take you there.
When Sun shut down our HPC team in 2004 I deliberately left the Enterprise Computing market, I didn't want to work for a company that sold technology to other companies, I wanted to sell web services to end consumers, and I had contacts at eBay who took me on. In 2007 I joined Netflix, and it's the best place I've ever worked, but I needed that time at eBay to orient myself to a consumer driven business model and re-tool my skill set, I couldn't have joined Netflix directly.
There are two slideshare presentations on the Netflix web site, one is on the company culture, the other on the business model. It is expected that anyone who is looking for a job has read and inwardly digested them both (its basically an interview fail if you haven't). These aren't aspirational puff pieces written by HR, along with everyone else in Netflix management (literally, at a series of large offsites), I was part of the discussion that helped our CEO Reed Hastings write and edit them both.
What can you do to "escape"? The tools are right there, you don't need to invest significant money, you just need to carve out some spare time to use them. Everything is either free open source, or available for a few cents or dollars on the Amazon cloud. The best two things you can have on your resume are hands on experience with the Amazon Web Services tool set, and links to open source projects that you have contributed to. There isn't much demand for C or C++ programmers, but ObjectiveC is an obvious next step, it's quite fun to code in and you can develop user interfaces for iPhone/iPad in a few lines of code, that back-end into cloud services. Java code (for app servers like Tomcat) on Android phones, Ruby-on-Rails, and Python are the core languages that are being used to build innovative new businesses nowadays. If you are into data or algorithms, then you need to figure out how to use Hadoop, which as I describe in one of my slideshare decks is trivially available from Amazon. You can even get an HPC cluster on a 10Gbit ethernet interconnect from Amazon now. There is hadoop based open source algorithm project called Mahout that is always looking for contributors.
To find the jobs themselves, spend time on LinkedIn. I use it to link to anyone I think might be interesting to hire or work with. Your connections have value since it is always good to hire people that know other good people. Keep your own listing current and join groups that you find interesting, like Java Architecture or Cloud Computing, and Sun Alumni. At this point LinkedIn is the main tool used by recruiters and managers to find people.
Good luck, and keep in touch (you can find me on LinkedIn or twitter @adrianco :-)
Subscribe to:
Post Comments (Atom)
This is a blog, a manifesto, an historical document, a guide, a hypnotic, yet rational "Augmented Reality" that you called in one slide "Assisted Telepathy". I noticed that rarely people comment your blogs and slides-share with many words. They just say "Fantastic", "I agree", "Very Good" and nothing else. After reading this blog, people are in a state of shock, unable or fearful to utter thoughts. It challenges our innate political correctness, our inertia.
ReplyDeleteIt is also a personal retrospective hard to imitate. I wish we can all join Ebay and then go to Netflix and go through the learning process you describe. Obviously, all of us have our own path, yet few of us have your gift to see through every day what is going on.
This blog is not about Solaris, or #illumos as de-facto becoming legacy computing - although this statement will shock. Ironically, such a strong opinion may lead to changes to make these project relevant and defeat the prediction. A classic psychoanalyst (Rollo May) said: "If I have discovered the science to tell the future. I will be right only if I do not tell you!"
This blog is about defeating our innate , stubborn resistance to have the life we really want, instead of the life we have now. This has to do more with with tolerance to risk, fear of unknown and sheer luck, which, even if we put all humanely possible efforts, it must step in to seal our destiny.
Did I say "destiny"? We live in a rational Silicon Valley, yet I remember how Netflix started as a snail mail DVD lending business, a video rental store with a web presence. For years it struggled, until it attracted talent and the on-demand content could delivered commercially via Ethernet home cable. Like eBay, Facebook, Amazon and so on, the technology created was out of necessity to stay in business and thrive, and not a goal by itself.
Their success, - and your personal success - should pay tribute to the correct definition of business model and to the fabulous product management work. This made the superb technology - yet invisible to Netflix users - wanted by people, and not the other way around.
Miha Ahronovitz
http://ahrono.com
Thanks Miha, I really appreciate the conversational reply.
ReplyDeleteThe challenge that Solaris can't overcome is that we don't spend a significant time worrying about the OS any more. It's a sedimented commodity, doesn't cost anything (we use CentOS on AWS), doesn't get in the way, just works, and is supported by all the layered tools we might want. There is no incremental value in looking at change in this layer.
Back in the 1980's instruction sets mattered (remember the debate about SPARC register windows?). In the 1990's the system call interface was what mattered (SVR4 vs. OSF/1 etc), then the window system library mattered (OpenLook vs. Motif), and Solaris vs. Linux mattered, and Oracle vs. MySQL mattered but that is all in the past, none of those matter any more.
Nowadays, with the web, and the failure of Solaris and Linux on the desktop, Windows vs. MacOS matters, and which web browser you use matters, and whether you use iPhone or Android matters. For web services, the cloud vendor matters - AWS, Azure, Rackspace, Google Apps etc, and there is intense innovation and competition happening in the cloud API space. There are a lot of free or service based "NoSQL" competitors competing to be the persistent storage tier of the future (Cassandra, Voldemort, SimpleDB etc.) and evaluating them to figure out which is good for what is the next big interesting thing to work on.
Your point is compelling and clearly worded:
ReplyDelete"The challenge that Solaris can't overcome is that we don't spend a significant time worrying about the OS any more. It's a sedimented commodity, doesn't cost anything (we use CentOS on AWS), doesn't get in the way, just works, and is supported by all the layered tools we might want. There is no incremental value in looking at change in this layer."
So instead of asking, whether application A runs on Linux or Mac, we ask whether it runs on AWS or CloudSigma.
The set of skills developed in Solaris are extremely useful for cloud computing. Joyent has the ambition to produce what they call a new cloud operating system and storage:
"Joyent, named Bryan Cantrill as VP of Engineering. Cantrill will be responsible for overseeing worldwide development focused on operating systems, including building additional innovation into Joyent’s SmartOS and SmartDataCenter."
http://bit.ly/bzxNmw
Bryan, like you, has a solid contribution to make. If Solaris can not completely transform itself within the boundaries of a giant corporation who owns the "legacy" product, someone outside Oracle may be able to pull it out.
But, what is the motivation to do it? Does it fit a business model? Does it serve some needs of the customers in an elegant way?
Is there a magic button any end-user could press on Windows, Linux or Solaris desktops that takes any application and delivers it as SaaS? Scaling over thousands of nodes someone else has?
If the answer is Yes - and this answer is based on faith plus reason - then we have - for illustration sake - a business model for the OSs of tomorrow.
If the answer is No, too complex, there are simpler ways than using an OS - then your answer for Solaris makes sense.
2 cents,
Miha
There is no "magic button" for anything involving moving apps between architectures and never will be. However, there are two sides to cloud architecture, one is the developer operations (devops) side that is about replacing most of your IT department with some APIs and automating all your deployment and scaling processes. The other side is coding robust, secure and scalable applications that work well in the cloud. It is a best practice to make data center apps robust, secure and scalable, but it is an essential practice in the cloud. The people who are already running at scale (Google, eBay, Yahoo, Amazon etc.) already build their apps that way, the public cloud is a way to leverage their investment in solving the hard problems in this space, that is one of the big reasons Netflix is using Amazon.
ReplyDeleteRunning Netflix on Amazon is cost effective. Your slides always have examples of a few dollars or cents in fees. But pricing the HPC configuration Amazon claimed is #146 on Top500 costs between $8M to $12M per year!!! It is only available in North East US, so if Netfilx or other customers want to use it on West Coast, they need to wait... http://bit.ly/aRjnD2 and http://bit.ly/cWySC0
ReplyDeleteThis is not something Netflix needs now, and your points are valid and lucidly explained. For others in high end computing ,the Netflix model on AWS can not be automatically used: at least not without doing a study of costs first and negotiating with Amazon, & comparing to a private or hybrid cloud solution.
My last 1 cent! We are preaching to the choir, because essentially what you say is what I think, and you added some concrete technical points to the debate
What wonderful insights from both of you - Adrian and Miha. I especially like Miha's comment, "This blog is about defeating our innate , stubborn resistance to have the life we really want, instead of the life we have now. This has to do more with with tolerance to risk, fear of unknown and sheer luck, which, even if we put all humanely possible efforts, it must step in to seal our destiny."
ReplyDeleteBreaking our longtime pattern is often a very good thing to do. "Come up for air", look around, look inside/meditate, re-assess your skills, desires (for life - it is short!) and determine the best next direction to take for fulfillment. Money is not fulfilling... a certain amount can give us some security, but nothing is guaranteed, especially in present times. Life seems to speed up as we continue through it - and, as a passage in the Bible that I like says, "There is a time for every season". If we don't "seize the moment" for what we really want in life, it will pass us by.
By the way, I'm not "Ray"... I'm just using his computer today. I'm Natalie.
ReplyDelete:)
It's sad that Netflix uses CentOS to power its systems and I still can't watch Netflix on my Linux desktop.
ReplyDeleteHi Miha, I just found your last comment got caught by the spam filter. The point about the cost of an HPC cluster assumes that you need it all the time, but there is a lot of downtime on real clusters, and there are times when there is no work for them, and times when they are over or under sized. Running an HPC cluster on AWS means that you get a cluster that is the size you want, when you want it. The same applies to AWS EMR for running hadoop jobs. For the same money, you can get the result in half the time by using twice as many machines.
ReplyDeleteAs a researcher it would make sense to use AWS HPC to get results more quickly, because there is less waiting and less contention for the resource. The cost of AWS systems is also fully burdened with overheads, and the cost of most systems has hidden costs like the staff needed to run them, and the unproductive downtime for maintenance isn't subtracted out.
Someone just sent me a link to your blog. Great stuff! As a former Sun type, I thought you might find this post a bit ironic. http://consultingadultblog.blogspot.com/
ReplyDeleteHi Nancy, I've been reading your blog already, nice to get some Sun history and some background on Patty, she hasn't changed :-)
ReplyDeleteIt has been a disappointing last 8 years or so across the industry as the OS layer has become less and less important to the service layer. However, on the dawn of 10GBe, FcoE, and infiniband becoming mainstream or specialized file systems offering more tricks like de-dup, snapshots, etc or the sedimentation of hardware hyper-threading being around long enough now that some people know how to code to leverage it or a focused usage of persistent and cached storage using more efficient hardware...I feel we are coming back into to a new OS focused era. Intersect how different types of virtualization behave well or not on top of those topics in terms of mobility and performance and you'll find even more reasons to pay attention to the OS layer. Now I do believe we will package the OS interface or better said expose the OS to the end user differently. People will spend less time in the traditional OS, but that's really a manageability conversation. None the less the OS will be there and whose you choose to use will impact your bottom line.
ReplyDeleteWe just need someone to knock the cover off the ball again to remind us of what an OS can bring to a business service conversation. There are still some people with good batting averages out there with the fortitude to stand up and take a swing.
Hey, I'm going to dump Netflix if you don't start streaming new movie releases soon :)