Tuesday 14 July 2015

Career Advice and That Jason Post

Last week Jason Calacanis wrote this post, and I sent a kind of expansion of it to my team.  A few of them came back and told me I should post it, so here goes...

This one stood out from the general noise in ‘career advice tweetstorms’ because, when I read it, I thought to myself holy shit that is exactly how I think of people and exactly how I know my boss thinks about me. That is pretty important intel for anyone hoping to achieve meteoric growth in my kind of company, so I think it is worth you each internalizing what it means for you. Let me break it down a little more first, into some more grounded practicalities:

1 and 2 are items I consider pretty immutable, so just do it. Besides - this isn’t work. If you’re in the right career then this is fun, you’re passionate about it, and you eat it up 24x7 whether you’re paid for it or not. It doesn’t feel like work. Achieving mastery in your craft is its own reward. However I acknowledge that not everyone can be in a job they find personally fulfilling and enlightening and, if you are in a product or technology role just because it pays well, then I’m not going to get all preachy about your motives. Just keep in mind that if you want to stay in that role and keep growing the rewards over time, then you need to do this just as much as (and maybe more than) those who are pursuing their passions. Because they will leave you behind, and they will out-compete you for the best roles.

3 is where Jason and I are going to disagree or – at best – there is a subtlety we agree on for which ‘startup’ is useful shorthand. Where I think we might philosophically agree is that you need to get somewhere where you can be individually visible (not buried in a huge team of homogenous ‘resources’) and your ability to step up and take on more, to exceed the normal boundaries of your primary responsibility, are not structurally constrained. Big, mature organizations tend to be set up such that the system of production (roles/structure, process, inputs and outputs) is defined in the abstract, and pursued ahead of unique or especially talented individuals who may not fit easily into any one predefined box. And, because these things tend to become more rigid over time, it is difficult to exceed one’s personal remit in a constructive way. Pick a business in a growth industry, and pick a team which is big enough to do cool stuff but not so big as to require Vogon-like bureaucracy, and pick a boss who values utilizing (and stretching) individuals where their passions and aptitude converge over having everyone nicely fit into a tidy box with the ‘right’ label on it. Startups are like this out of necessity – that’s why I can agree that ‘startup’ is a compact way to communicate this kind of sentiment – but they don’t have a monopoly on this. It can be a sustainable lifestyle choice in any phase of a business. 

Item 4 is one of the characteristics I have seen in almost every high potential high performer I have had the pleasure of looking after. To what Jason already has there I would add two things; first working hard and taking on more doesn’t mean being in the office 24x7. It just means intensity, urgency, focus, and prioritization. I won’t personally give you any points for being in the office any longer than me – in fact I am slightly more likely to wonder if you need a little extra support. Second thing here is to know your cake from your icing, as a really fabulous CEO I worked for a long time ago told me. Go after more, take on more, over-deliver unexpected surprises, but never at the expense of your core responsibilities. The reason your primary role exists at all is because a lot of customers and colleagues rely on you delivering on time and to a high quality. That’s your cake. And if your cake starts to suffer for more icing – all the extras etc – that marks the difference between a high performing high capacity individual who is obviously in need of a promotion and an irresponsible slacker who doesn’t understand the business needs and is letting the team down. 

I work with a few companies at different stages of growth, and I wish I saw a little more of item 5 everywhere I go. You should actively look for chances to do this, not wait until you’re asked to do a brown bag session or something. The fastest path to true mastery of anything is to have to teach it to another. Or, in career advice terms if you prefer, unless there is at least someone around who is as good as you at what you currently do then this will eventually become a blocker to your personal advancement. Giving you a bigger role is important but creates a difficult rubix cube-esque puzzle, but giving you a bigger role when you have a solid, practiced succession plan ready and waiting becomes a no-brainer. 

6 yup another JFDI. This I would enhance with Postel’s Law. If you practice a kind of ‘human equivalent’ of the robustness principle then you will shut down negativity instead of amplify it and deescalate potential conflict. ‘Assuming positive intent’ is such a powerful tool for keeping everything constructive and feeling awesome about yourself and others. 

I’d read 7 as never be reluctant to ask for things. And not just in reward; also resources, opportunities, mentoring, training, chances to join in senior forums, a place in special projects. Whatever. And I know this town is equity-crazed, and equity is certainly nice, but it isn’t the only way to be rewarded and it isn’t the only path to wealth. Regardless, you should be fairly compensated for the value you create. Amid all today’s rhetoric about leaning in and whatever, just keep in mind that the prerequisite to this is to actually demonstrate your value though real results, consistently over time first. Similar to my addendum on number 4, this distinguishes the merited from the ‘participation award’ crowd who want it all just handed to them for showing up. You own your career. 

8 is table stakes in product. You’re here to define an alternate future, where your company is better tomorrow than it is today. Startups are just one killer feature around which the rest of a business and a complete product emerges, so all the same instincts and behaviors will carry you to success in any product-led business whether or not you might call it a startup. 

So that’s some career advice I think everyone can take something away from. I believe that people achieve more and feel more passionate about the product (and stick around longer!) when they can see how doing so is helping them grow and taking them closer to their idea of success for themselves. That’s why this shit matters to me and it matters that my leadership team take it equally seriously for all my people.

Friday 16 January 2015

The power and the pitfalls of test and learn

My core philosophy on product is that product/market fit is a journey; it is a function of discovery and learning over time, and it is impractical to expect to be able to make a set of ideal decisions up front (i.e. in advance of any product development and real feedback from real customers).

We now live in the world where the fast eat the slow, and I am going to argue that learning is speed:

At Hotwire we’ve dramatically improved our business performance in the last year by focussing on learning as a primary goal.  It’s a first order metric, along with the ‘hard’ KPIs most businesses are familiar with (revenue, room nights, activations, etc) but - as any good teacher will tell you - measuring learning is extremely hard to do.  Fortunately for us, there’s another feature which is both easy to observe and highly correlated with learning: experimentation.

Whenever you see a high rate of improvement (in nature and in science) you will almost always see a high rate of experimentation.  There’s a great Thomas Edison quote which goes something like; “None of my inventions came by accident. I see a worthwhile need to be met and I make trial after trial until it comes.”  Most of our early learning as human beings is heavily experiment driven; it is interactions with the world that teach us how to behave effectively within it.  But my favorite story about the value of pursuing learning as a primary goal is the Kremer Prize:

In 1959 Henry Kremer, an industrialist and patron of early aviation, established a prize of £50,000 for the first human-powered aircraft to achieve a controlled flight.  Hundreds of attempts were made over 20 years with no success.  And most of these were well-informed experts; aviation companies and universities etc.  The prize went unclaimed until 1977 when Paul MacCready won it with his Gossamer Condor.  The secret to his success wasn’t being smarter than any of those who previously attempted flights or knowing something they didn’t, it was his approach to the problem being fundamentally different.

Most teams spent months designing and building their craft, then they took it out to a field or an airstrip to try it out, then they crashed, then they swept all the pieces up and returned to the hanger for another few months to rebuild.  MacCready focused not on the airframe that would work, but on a cheap construction which was easy to assemble and disassemble by hand in the field.  Using this he was able to try more designs out in one single day than every preceding team added together for the entire previous year.

His formula for success had 2 main features; variation and repetition.  You can see this in nature too; natural selection tries out variations of organisms over and over again (optimizing to the ultimate KPI - life itself!) as those organisms improve their suitability to their environments.  While not strictly mathematical, Fisher’s theorem is a useful formalization of this (and the other examples we’ve discussed here).  It goes something like this; “The rate of increase in fitness of any organism at any time is equal to its genetic variance in fitness at that time.”  Or, more simply:

The capability to try out more hypothesis at any given time is highly correlated with faster improvement.

So learning is speed, and number of experiments is a useful approximation of learning.  But before you just count all a/b tests etc, there are some subtleties to success here:

The first thing I like to watch out for is confirmation bias.  As you embrace test and learn and make experiments cheaper to run you lower the bar for organizational participation.  This is unquestionably another benefit - harnessing the innovation of a larger slice of your org - but not everyone is as academically disciplined as you’d hope, and there is an underlying human tendency to like our own ideas and see them in a less-critical light than the ideas of others.  A while ago I was fortunate enough to be able to spend some time with Alan Kay, and he told me that science is the process that stops people from falling in love with their own ideas.  This is more than just a principle; if you come up with a hypothesis which you’re trying to prove instead of disprove, you will tend to discount contradictory evidence (i.e. proof points that suggest customers do not like it) and waste a lot of time trying marginally different manifestations of the same core idea in the desperate hope that you can somehow make them love it.  You lose speed, not gain speed, this way.

A fun way to look at this is to examine the difference between scientific thinking and religious thinking.  Jack Cohen is another wonderful human being, with a vast amount to teach the world, with whom I had the pleasure of spending a little time.  He argues that; in religious thinking all that matters is how hard you believe, especially in the presence of contradictory evidence (it’s there to test your belief).  In scientific thinking, all that matters is how hard you doubt, especially in the presence of confirmatory evidence (easy answers there to trick you into overgeneralizing your observations).

People will naturally come up with ideas that they like, but good product hygiene is about coming up with ideas that your customer likes.  Rigor in testing is the ultimate arbiter which can make that distinction clear to you, if you let it.  If you’re not open to being wrong - in fact expecting it and actively seeking to make it so - then you cannot, by definition, learn.

Another big-picture mistake is confusing iteration with learning.  It’s a common problem in the whole ‘transition to agile’ world; break up a big-up-front-plan into a number of predetermined phases, label them ‘iterations’ and then cash in your huge cheque as a profound agile coach.  Starting with the same inflexible ideas and delivering them across a higher number of software releases has some benefits in terms of quality and system risk, but it does nothing to improve your product/market fit.  You had determined the level of product/market fit at the beginning of the project and you have not improved that fit over the whole, say, year you were working on it whether it’s one single release at end of the 12 months on a whole series of incremental drops every two weeks.  Learning is about regularly shipping something customers can touch, then watching their interactions with that thing.  Looking for where it enriches and where it detracts, and only then deciding on the final scope for that next iteration.  The point is that each iteration should reflect the learnings from the previous iteration and improve upon it with respect to the customer experience, and therefore cannot be rigidly determined in advance of that feedback.

To be inclusive and stimulate the organizational pathways for innovating, a pretty broad filter is useful in the beginning.  All ideas are valid, and anything can be reduced to a testable hypothesis.  But, as you execute on a test and learn backlog, selecting ideas which are both coherent with the existing product and carry a higher probability of resulting in an improved experience for the customer becomes important.  Think back to our Kremer Prize example; while MacCready focused on the ability to explore a large number of variants, he was not randomly experimenting with geometry in the hope that he got lucky and found something that would fly.  He was an engineer who ran an aerospace company and he had a detailed understanding of all the mechanics involved in lift and control etc.  In user experience terms that means figuring out what customers are sensitive to, and using those things as inspiration points for ideas.  On the internet these sensitivities are rooted in behavioral science (decision theory, nudge theory etc) and are often things like social signals (how many others have bought the same item or currently viewing this page), urgency messaging (this is a popular item or this deal expires soon), and recommendation (if you like x you might like y) etc.  Discovering what these sensitivities are for your particular product can get you more ideas aligned to the physics of your particular business.

When talking about ways to increase the likelihood of ‘winning’ tests I always like to reiterate the value of ‘losers’ too.  A losing test is essentially an idea someone had for a new product or interaction which, without the experiment disproving the hypothesis, would have resulted in a project that would have taken away valuable product development resources for no (or negative) return.  What did you save right there?  How much customer attrition did you avoid by staying away from that unpopular option?

There is much more to doing this well - a whole series of posts wouldn’t do it justice - but I like to focus on the quality of the thinking first.

So does learning == speed?  At least in the special case of how rapidly you can get to the right product/market fit and grow, it certainly does here.  Is the number of experiments a good heuristic for organizational learning?  If you chart the improvement in our real business performance and the number of concurrent tests over time, you see very similar growth curves, just as Fisher’s theorem predicted...