It's not obvious how to be insanely simple

three books that I read recently resonated with me and fitted together so I'm going to try to make sense of them in a blog post rather than in a series of cryptic tweets.

My son (who is a product manager at eBay) told me about the most recent publication:
Insanely Simple: The Obsession That Drives Apple's Success by Ken Segall

At the Defrag conference last year I saw Duncan Watts present and recently finished reading his book Everything Is Obvious: Once You Know The Answer

I also recently watched Sam Harris give his talk on Free Will, and then read the book.

The connection starts with Free Will, which explains what is really going in our heads, along with Everything is Obvious which explains how our minds work collectively and interact with the real world these two books are the "Missing Manuals" for our brains. It's hard enough to figure out what is going on in the world and how best to navigate it, but it's doubly hard when you don't realize how your subconscious is pulling the strings and how common sense is confusing everyone around you.

Inside your head, the conscious thread of thoughts that you hear are post rationalizing decisions that your subconscious mind has already made. Feeding yourself a broad range of information with an open mind, connecting to your intuition and letting the power of your subconscious find the right patterns and responses lets you make faster and better decisions.

In society, we are surrounded by common sense explanations that we use to post rationalize the events around us and which are fed to us by the media, historians, politicians and our friends. Duncan deconstructs common sense to show that these explanations are mirages driven by our inner need to find a narrative and cause for effects that are essentially random co-incidences with far less significance than we assume. He then explains what un-common sense looks like and how to question the received wisdom and have better strategies for getting things done successfully.

I'm not going to summarize the whole book but there is a very useful section that should be read by anyone doing "big data analytics" that sets out the kind of things that are know-able and what (and why) other things will always remain un-knowable and impossible to predict. The advice I distilled from the discussion of strategy is that there is so much randomness in the outcome of business decisions that you cannot reliably evaluate the difference between a good strategy and a poor strategy. If you are able to get ever more detailed data about what happened you become more convinced in the value of your analysis, but the predictions you make about what to do next don't get any better. This is a counter-intuitive outcome (i.e. it violates common sense), so please read the book, which explains why you shouldn't be trusting your common sense in the first place.

The positive things we can do to overcome random outcomes really resonated with me, as they put into words several of the things I've been doing for many years, which have in some sense given me a better way to understand what's going on and get stuff done. They also describe many of the ways that Netflix figures out how to build it's products.

The first thing I do when I hear something like A caused B is a reflex reaction, I flip it around in my head, take the devils advocate position, look at the situation from a few different angles. This can be quite annoying in "polite company" as I tend to question received wisdom and common sense assertions, however I usually find a missing piece of information that could falsify the assertion, and ask the question. It could be as simple as asking exactly what time A and B happened, since if B turned out to happen before A then the assertion is clearly false. In statistics and physics this is codified as asserting the null hypothesis. (I'm the son of a statistician and I have a Physics degree...).

At Netflix we always try to construct parallel "A/B" tests of our hypotheses, like the double blind tests used in clinical trials of new medicines. We take a large number of new customers and give them a range of different experiences for long enough to measure a difference in their responses. This is the only way to reliably tell whether a new feature works, and it often goes against the common sense of what we expect and what many customers and industry analysts helpfully suggest we should be doing. As Duncan explains we can usually figure out what factors will affect an outcome, but we are extremely poor judges of how to weight those factors, even with post rationalization of what we saw happen, and all we can do is bias the statistics in a preferred direction. A recurring example is the suggestion that Netflix should allow half-stars in its movie ratings, but it turns out that given more fine grain choice fewer people rate movies, and the reduction in the number of data points out-weighs the increased accuracy. We can post rationalize why this occurs as an example of giving people too much choice, but we don't have to rationalize it, we just measured it.

In the discussion of strategy Duncan talks about creating a set of strategies that cover many scenarios, and using scenario planning to build more flexible and fuzzy strategies which are more likely to work under a range of random external influences. By putting yourself in the path of possible good randomness and avoiding bad outcomes, you can "make your own luck". By detecting problems early and having the flexibility to adapt your strategy you can run around the problems that will randomly come your way. If instead you concentrate on coming up with the best possible strategy or assuming that previous success was due to strategy rather than random outcomes you are building a brittle future that is likely to disappoint you.

The final point I will lift from Duncan's discussion of uncommon sense is that speed of execution and iteration is another fine way to cheat the chance events that will derail your plans. Long term detailed plans are a waste of time. This is one of the foundations of agile development, where rapid iteration of product features lets you discover what your users actually do with your product, as opposed to what you thought they would do or what they say they will do.

This leads to the Insanely Simple book, which talks about Apple's approach to product design, with particular emphasis on branding and marketing since Ken Segall was the guy who came up with the i in iMac and has many other fascinating stories. One reason I like working at Netflix is that for agile web services, product ideas can be built and tested in a week or a month, and fixed in minutes. For Apple they work on products for years and need to have them work perfectly when they are released. This gives them two big problems, since its hard to iterate and hard to test ideas and products in advance. Their solution seems to be that they allow better ideas to form and develop, take bigger risks and make decisions faster than their competition, which helps stay ahead of the market. The Insanely Simple design philosophy is based on the idea that its easy to listen to all those great common sense ideas about features your product has to have, but if you learn to ignore the common sense and give the customers a simple and distilled experience you will reach beyond the people who want a complicated product and find a much bigger market of people who were waiting for a simple way to get something done. Apple's competitors are so bogged down in committees and approval processes, and helpful common sense advice from customers that they are unable to release simple products.

A key example from the book is that Apple has had many award winning advertising campaigns, "Think Different", "PC and Mac" and the iPod silhouette, and none of them were test marketed in advance. Their competitors make less risky adverts after getting broad internal consensus, take much longer to get them to market and fail to understand that the success of an advert is a randomized event (with lots of useless common sense post rationalizations) so the test market response is a very poor predictor of success. It's more important to be bold, different and go big. For example Apple only advertised Think Different on the back cover of magazines, which costs far more but has a much bigger impact than inside pages.

From these three books I've found some useful focus on how to approach things, but they also give me some backup to explain to others why I think some things are important. A key part of what I have been doing for Netflix is looking out into the future of cloud and related technologies and developing a portfolio of fuzzy strategies and options. They don't all work out, but by having a well instrumented but loosely coordinated architecture that doesn't have central control and strict processes we can iterate rapidly, adopt (and discard) interesting new technologies as they come along. We can all have more fun and less frustration making Netflix Insanely Simple, and ignore all the bad common sense advice and analyst opinions that swirl around everything we do.

I'm planning a complete re-write of my cloud architecture tutorial for Gluecon in May, that will be a great opportunity to discuss these things in person over a few beers, and now is a good time to sign up to attend - you can get a 10% discount with code spkr12.