Nobody cares about your tickets

As engineers we like to speak in tickets and stories and bugs IDs etc - those units of currency we find easy to grok; they're usually finite, easy to define, narrowly scoped, and uniquely identifiable.

Our customers - for the most part - like to speak in outcomes and business plans and commercial objectives and projects.  The longer term, more vague, abstracted-from-the-work things that we like to digest by breaking them down into those stories etc that we're so comfortable with.

This can lead to a weird situation where you have all your iterations scoped out and your releases forecast (with painstaking detail in every story) yet the business still lacks clarity about what's being delivered by who and when.  Hmmm.

I think a bit of generalisation can help us to understand this.  Lets appreciate that engineers, for the most part, are deeply practical people who are comfortable handling actionable tasks.  Since we're also geeks - again for the most part - this descends quickly into technical details.  You'd be surprised how unintelligible this can all be when you're not a developer.

What I've learned over the years is that stakeholders are interested in the meta problem, the commercial plans and outcomes, not the tangible problem, the technical tasks and code which need to come together to solve it.  As technical leaders we need to talk about the meta problems with the business, and the technical problems with the engineers.

This comes with two obvious challenges - whenever we're talking about the same things but using different currency we rely on a degree of interpretation and some things can be lost in translation.  This most commonly surfaces as that old "unclear requirements" thing.  The other challenge is making sure that, as we solve our technical problems, we're solving our meta problems.  This isn't easy and this is why great product managers are so crazy hard to find!

This kind of interpretation of work - the aggregation of agile units into bigger picture project plans - is usually an early casualty of agile adoptions.  It is reminiscent of waterfall so we label it bad.  But it is a helpful mechanism which bridges the gap between agile execution and regular business planning.  It gives you both currencies.

Get awesome product people - they are the only other ones who live in this gap with you and have to speak both languages.

The secret of managing software projects

As a technology manager there is almost unlimited discourse you can read about how to successfully manage delivery and control projects.  The real secret here is you don't manage projects, you manage the circumstances around a project...

This assumes that you have a team of talented engineers, a project manager as strong of character as he is on process, and an architecturally sound plan with some slack in your estimates.  If you don't already have this then don't worry, you're not about to fail, you've already failed.  What you're about to experience is you doing your best to mitigate the consequences of poor leadership.  Good luck with that.

But let's just say that you've got the structure right - now what?  If that's true, then the best contribution you can make as a leader is managing the environment around the project.  Provide a vision and simple insight into what moves the dial for the business and then move the obstacles, keep the distractions away, make sure the priorities don't change, and protect the productivity of the team members.  Keep the project fed and watered; get it the budget and skillsets it needs, get it the right visibility and attention from external teams who may be dependencies.  You're usually on the hook for the whole thing, so the temptation to dive into the details is often overwhelming.  Do it if you must but know that you probably aren't helping.  Your guys won't feel trusted and you won't be encouraging them to take ownership of their own space.  If you have a good plan on a sensible horizon, don't fiddle with it.  Every time you do you increase the chances of things not working out.

That doesn't mean you should be disinterested or uninvolved - quite the opposite.  Be there 100% for every engineer every day and give them everything they need to give you what you need.  Be easily accessible and take on any potential problem, consequence-free, that your team feels like raising.  They don't want to waste your time any more than you want to waste theirs, so if they come to you with something, it will usually be because there is a genuine risk that you're not going to get what you planned and there is something you can do to put things back on track.

I love technology.  We all do.  That's why we got into this business.  I love what my guys do, and I love to talk to them about it, but I am always aware of when prudent interest in the organization's deliverables crosses the line and becomes micromanagement and interference.

The most important thing you can give to any project is certainty.  Make a good plan based around people better than you are and then defend them.

Great Quote and Greater Source

This article has a great take on developer-husbandry, and I particularly like the last paragraph:

"I’ve gone back and forth on whether managers should code and my opinion is: don’t stop coding. Each week that passes where you don’t share the joy, despair, and discovery of software development is a week when you slowly forget what it means to be a software developer. Over time it means you’ll have a harder time talking to engineers because you’ll forget how they think and how they become bored."

Even better is who sent it over to me - one of my commercial stakeholders.  Being in a tech savvy business makes my job a lot easier in a million little ways.

Why don't we invent more?

I sat in on a conference call today (actually I'm still on it!) where we talked a lot about innovation - specifically why we haven't done more of it.  Lots of different stories but the common theme seemed to be governance.

When I say governance I mean things like roadmaps, product councils, architectural oversight, PMO, etc.  Basically the constructs we've set up to steer our technology investments and spend our resources wisely.  Properly applied, they are critical to the success of any significant engineering endeavour.  Improperly applied they are critical to the failure of any significant engineering endeavour!  So are they being properly applied to invention and experimentation?

I don't think it is a case of proper application as much as it is a case of application at all.  I think any engineer has two jobs; serve the roadmap - build the applications and systems defined and managed by our governance processes - and serve the technology - discover new ways of solving our business problems, advance your own knowledge by experimenting, come up with new ideas and see if they fly.  If you're only doing the former then you're only doing half your job.

These are two very distinct types of activity and they shouldn't be governed by the same controls.  It just makes no sense to apply ROI and risk controls and rigid scheduling to a journey of discovery with a totally unknown destination.  The difference between teams who innovate and teams who just talk about it is recognising the immense value in the journey alone...

Besides, you don't need all that bureaucracy.  All you need to be great at this is to leave a space for it to grow and - if you have the right people - it will expand to fill it.  That is how I roll and that is what I want to see.

Big Roadmap seeks Innovative Thinkers for long term relationship

I haven't posted much recently - my new gig has been keeping me quite busy - but life is returning to a more sustainable pace now so I'll be able to start sharing more often again.  In the meantime I once again find myself drawing up battle plans in the war for talent.  So what are my 3 top weapons I can bring to the fight?

Step 1 - oops almost out of desks; let's get a really kick ass place to house all these smart engineers.  So we're moving to a sweet new spot in Angel.  We're growing fast and it's already nearly standing room only at the current office.  Covent Garden has its charms, but our new spot is being custom fitted - just for us - to be one of the best work spaces in the UK.  Check out the progress:

The list of features includes eco-geekery such as biomass boilers and heating and lighting driven by room occupancy sensors, high ceilings with 360 degree sunlight, huge storage space for bikes with plenty of showers and lockers, a variety of breakout areas for informal meetings, and a nifty cafe and Jamie's Italian restaurant on the ground floor.  All 80,000 sq ft of it (a huge uplift to accommodate our growth) is coming along nicely and is 30 seconds walk from Angel tube station.

Cool.  Now for step 2 - set up the right environment and culture.  So what is it actually like to work here?  Well, you could take our word for it, or I could give you the inside scoop...

When you get right down to it technology is core to what we do - our whole business is built on it and it drives our partner's businesses too.  Therefore we have to have a culture which supports great engineering in order to succeed.  That means bringing in smart developers, letting them own the problems, and then giving them the space to find the best ways of solving them.  It means creating time for innovation, it means allowing technologists to make technical decisions, and it means permission to work in the implementations which make sense (and maybe even a little pushing to keep you expanding your horizons) instead of being dogmatic about any particular stack.  Add to this a sustainable business model with plenty of big wins left to achieve and some clear priorities and shared steering of the business, and you have yourself the right ingredients for putting together some really successful software.

In the macro, Expedia is a very large place but we operate in a kind of group structure.  So yes, it is a very big company and that [fortunately] comes with some big company benefits, but the work experience is more similar to working for a well funded startup.  We work as small, independent, cross-functional teams, with each team having a good mixture of autonomy and support.  We believe that our guys should have end-to-end ownership of the systems they build - everyone is part of a small elite team and feels like they're personally making a difference.  If you've read a few of my posts before then you probably have a fair idea about how I like to run things.

Step 3 - challenges that really make you think.  This is an exciting time to be part of creating something pretty unique; I've said recently that we're about to change the rules in web travel and it's going to be fun.  Obviously I won't share my roadmap here, but I will talk about the sort of technology we'll be working with.  How about creating an eventually consistent distributed datastore latency tolerant?  What about peer-to-peer cluster management in the cloud?  Making an n+1 node in-memory cache topology aware?  Automated provisioning based on realtime system utilisation feedback?  Inventing new sorting algorithms and queuing strategies?  Or your own anti-entropy protocol?  Collecting obscene amounts of performance data and rebuilding your tools to watch those numbers move?...

You'll notice that I didn't say 'Java' or 'C#' or 'SQL' or 'PHP' at all.  That's because, to a certain extent, those things aren't what's important.  What's important are the patterns, the fundamentals of how we logically solve business problems using computer science, and that is what great engineers do.  Then they write some code as a result.

If that sounds awesome to you, and you can show me some aptitude in this kind of space, then we should talk (especially if you're Cassandra/MapReduce curious).  Get hold of me or my recruitment ninja, Roopesh Panchasra, right now.

PS - I am also hiring in the US, so drop me a line if you're stateside and want to work with smart folks on some sharp stuff.

a geographic sense of urgency

A friend of mine recently visited the Boeing factory just outside of Seattle and came back with a story I thought was worth sharing; something they called 'a geographic sense of urgency' over there.  It's a nifty story about motivation and incentive.  I'll do my best to retell it 2nd hand.

A few years ago Boeing was struggling with efficiency on their assembly lines.  They needed a way to build airplanes faster but were approaching limits on what could be done in parallel, how long a working day could be extended, or how many more resources could be applied to each assembly (outside of the economic ceiling I guess there must be some limitation to the number of people who can fit around the airframe).

To grossly oversimplify - with apologies to aircraft mechanics everywhere - a plane started at one end of the production line and moved through a series of stages; stopping at each 'station' to fit whatever parts that station fitted until eventually arriving at the other end of the hanger complete.  Another way to describe this might be to say that a plane moved through a series of bottlenecks, with all pressure focused on the current station until its work was complete.

The answer borrowed heavily from lean but was all anchored around a single, creative idea; the plane never stops from one end of the hanger to the other.  Instead of rolling forward a few dozen meters and then stopping to have work done (loop until plane=true) the plane is constantly moving forward by a few centimeters an hour and only stops when it reaches the end of the hanger - hopefully as a complete aircraft!  Flow.  Work happens more fluidly and consistently all the way along the plane's terrestrial journey.

This simple change to constant motion had a remarkable impact on the teams.  I wish I had more data (and if I can find any I will post it) but the improvements in throughout were astounding.  The physiological effect was a sense of pace - this is what they call a geographic sense of urgency.  The results were significant increases in efficiency and total throughput without extending crews or hours or resorting to the traditional but seldom as effective as expected measures such as pay increases.

Smart idea Boeing.

Intellectual Property in 3 posts – episode I

I originally called this ‘Intellectual Property Done Quick’ but, when I was finished and looked at what I had wrought, I realized that ‘done quick’ was likely to be in breach of the trade descriptions act.

So I’ve broken it up into 3 tasty morsels; today I’m doing an introduction to different types of intellectual property and I’ll follow up with applying for a patent and then finally some tips for managing IP in the enterprise.

There are 4 main types of intellectual property defined in UK law and, along with their relatively dry legal descriptions, they are:

Trade marks

Indefinitely protects signs which distinguish the goods or services of one undertaking from those of another.

A trade mark is a distinguishing badge of origin which can be a valuable asset when it imbues a product with the reputation and perception of a certain maker.

It can become a trademark either by initial registration or it can acquire trademark status over time through use.

McDonald’s golden arches, the UPS shield, and Nike’s swoosh are all good examples of a trademark – instantly recognizable and clearly identifying a specific brand.

Registering a trademark gives an organization or an individual the right to prevent others from using the same mark it in relation to similar goods or services (listed in the application). Even if you have no appetite for hunting down and suing IP trespassers, registration is still a good idea because it prevents that happening to you – priority (‘I was already using that before they registered it’) is a tough argument in trade marks. And, in the longer term, if you’re ever likely to want to license the use of your trademark to others then registration establishes an official article against which a license can be granted.

Patents

Protects the technical aspects of products or processes for 20 years.

A patent gives the patent holder exclusive rights to prevent anyone else making, using, or selling their invention for a fixed period of time (usually 20 years) before it becomes part of the public domain. Patents cannot be extended beyond their initial term – they arose as a way of governments encouraging innovation; essentially we agree to keep inventing stuff for the good of mankind in exchange for a period of state sanctioned selfishness in how that invention is manufactured and sold.

Patents are territorial – only good for the countries they’re granted in – but a number of international agreements exist which allow. In the EU we have the European Patent Office where a single application covers 36 nations, and for the wider world the Patent Cooperation Treaty covers 130 countries. In both cases you have 12 months from your ‘home’ filing date to extend internationally before it is considered a new application – this can be important as your initial filing date is considered to be the date from which protection becomes effective.

Check out James Cameron’s platform for stereoscopic image acquisition (also known as a camera mount) for example and improvement in velocipedes (also known as bicycles) for something a little more old school.

Registered designs

Protects the appearance of products for 25 years.

For a long time it has been recognized that the appearance of a product can be the key to it’s commercial success – may I present, as evidence, anyone who buys anything from Apple.

A registered design gives its holder the right to prevent anyone else making or selling products which look and feel the same as the registered design. It covers tangible, physical, crafted things (like the famous coke bottle) as well as conceptual, aesthetic things (like the famous coke logo).

Registering a design is quite straightforward – compared to a patent application – as it does not involve any detailed scrutiny by an examiner. Apply to the Intellectual Property Office and, within 3 months, you could be the proud new owner of a registered design. Also unlike patents, you have 12 months from when your design first becomes public to register it; with a patent when it’s out it’s out!

Design rights

Protection for appearance of products without any special application being made (for example copyright).

Design rights are a collection of automatic protections which apply to various original works. In practice they work very similarly to registered designs – and apply to similar works and materials – but do not need to be applied for in order to take effect.

The subclasses (if you want to be geeky about it) are:

• UK design right – protects shape and appearance, except for surface decoration, for either 15 years from the creation of the design or 10 years from the first time the design was marketed.
• Community design right – similar protection to the UK design right and also covers surface decoration, however it is only valid for a maximum of 3 years from the date the design was made public.
• UK copyright – can be thought of almost as the opposite of UK design rights because copyright covers surface decoration (not form and function) and ‘artistic’ qualities. In force for 25 years from the end of the year in which the design was first marketed.

If design rights apply automatically why would you pay to register a design? If you rely solely on automatic design rights then you must prove that a similar product was copied from your protected material – with a registered design you can prevent another party from making or selling something similar whether they copied you intentionally or by coincidence. It’s pretty easy to determine that two items are similar, but a lot more difficult to prove why.

So that’s our 4 identified types of IP.

In the technology and software business you’ll mostly deal with copyright and patents. In my next post I’m going to walk though patent application process since copyrights kind of happen on their own and the granting of a patent looks simple from the outside but turns into quite an arcane art.

And finally, now that we’ve covered formally establishing and reserving rights using the various offensive and defensive legal tools available to manage IP, the thought I want to leave you with is that there is nothing quite like good citizenship. Treat the property and creations of others in the same way as you’d like them to handle yours – patents, copyrights, or not.

Steriods for Signups

Anyone who runs a business on the web should be intimately familiar with the registration funnel (sometimes called the recruitment or conversion funnel) so - except for the briefest review to frame the problem - I'm not going to rehash that here. Instead I'm going to cover some common and some not-so-common ways to boost registration numbers.

So, for the quick review, the registration funnel is basically the set of steps a potential customer traverses on their way to becoming an actual customer (converted) and typically starts with a site hit and ends with an active user account. What these steps consist of varies with the nature of each business and might include things like a shipping address or a credit check along with the basics such as username and password.

The problem every web business faces is that the registration funnel is essentially a very lossy process. For example; of every 100 site visitors, 50 will start the registration process, of that 50 25 will complete the process, and of that 25 10 might go on to place their first order. Even with the best husbandry many web businesses have single digit conversion rates. With marketing on the web becoming more sophisticated and SEO/semantics driving smarter access to content getting the numbers in the top of the funnel usually isn't the problem and - just like any other lossy process - there can be a big impact on the bottom line by improving the rate by just a couple of points.

So how can you juice this up?

The very first thing you need is data. Just like everything else you need to understand the flow and then you need to instrument the heck out of it before you should be fiddling with it. If you haven't already got this in place then stop reading now and get on with it - the rest of this is useless without insight.

Separate your registration process out in your head; this will help you understand and instrument it. Think through all the steps in your funnel make it as granular as possible - this gives you much more insight into where potential customers drop out and much more flexibility in how you modify the process in response. Crudely; if you're thinking visitor -> credentials -> personal details -> checkout then you should be thinking visitor -> credentials -> name and email address -> shipping address -> checkout. This doesn't necessarily mean more pages or more data for customers to enter (in fact the goal is the opposite) but it does mean than you can count these things individually to see exactly what bit of information your potential customers get reluctant about handing over and it allows you to stage things a little more (see below).

The general principle is to put the fewest barriers between your potential customers and your product as possible. This means looking at what information you're asking for, when you're asking for it, and how it's captured. In some businesses there can be constraints which have to be acknowledged; with a product which is regulated or controlled in some way there may be some external rules which can reduce your options - such as the requirement for identity verification or credit checking - but the keyword here is reduce. The registration pipeline in these types of business is often poorly managed for this reason but it doesn't always need to be; take the time to understand the details of your governance and you will usually find that you have more flexibility than you assumed. For example you might need to verify your customers identity somehow before allowing them to make a withdrawal but does that have to stop you from granting them access to the rest of your product entirely?

Nice segue into staging. One of my favorite techniques in businesses with more complex registration requirements is to focus on capturing the absolute bare minimum during an initial signup (usernames, email addresses, passwords, etc) and capturing the rest 'just in time' in line with access to features. For example; why not delay asking for a physical address until the first time a user reaches the checkout? Why not ask for payment information just before their first purchase? You don't need that data until those events and having to enter it all up front can be prohibitive especially when potential customers might not be 100% sure that you're the service they want. Also think carefully about when you want to trigger the 1st step of the registration process - there can be benefit to letting potential customers play with a little bit more of your product before hitting them up for some credentials (but you have to balance this with less complete early contact information and the risk of taking it too far and having fewer accounts because the majority of the value is available anonymously).

Usability is an important aspect here. Considering the number of screens in your signup process, the number of fields, and even the phraseology of the questions can improve your conversion rates.

Also consider the order that you're capturing information in; if you start with email addresses and you're automatically storing fields as they're completed - a highly recommended Ajax trick - then you have some actionable data to go with your telemetry. You might want to use it for a follow-up contact to see if you can claw a customer back from the edge of registration oblivion.

Another consideration when you're designing your forms is to keep in mind the sort of information users are likely to have close at hand when signing up for your site. The more esoteric (in day-to-day terms) the information you ask for is the higher the chances that a potential customer will need to go elsewhere to retrieve it during the signup process. Data always shows that anytime a customer leaves the page(s) for any reason the changes that they'll return and complete them drop through the floor. What might you not even need users to enter at all? You can pick up things like language and locale from the browser and tools like geo-IP; adjusting a probably-correct field from a list of common choices is far preferable to data entry.

While we're on forms; short and sweet is the way to go. Whenever you have to go 'below the fold' then you're better off splitting the form into multiple pages (otherwise it appears too daunting) and, when you're going across multiple pages, then label the progress. A simple 'page X of Y' can work wonders setting expectations.

And finally, if there is a relevant infrastructure for your line of business (meaning that you can trust it and that a portion of your users are likely to be members) you can consider using an identity service, such as OpenID, to implement a kind of 1-click signup and then just capture the additional information you need to operate the account.

Change it and see what happens - after all you're collecting all the data you need, right? If you're crafty enough then you can do some A/B testing with alternate pages or different text. As long as you're measuring the process you will really quickly find out what's better and what's worse.

Adding a feature dimension to planning boards

Planning boards – sometimes called task boards – are a great way to make work visible, facility teamwork, and surface impediments.  The work required for a given iteration of software is broken down into a collection of individually owned tasks and then, as work progresses, we watch those tasks migrate towards the right hand side with a sense of satisfaction.

This is unprecedented transparency and one of the reasons why I love it, but it does have one small shortcoming.  If you’re not intimately familiar with the work (say, for example, a non-technical stakeholder) then it can be a bit hard to determine meaning.  Yes things are moving along, but what really are those things?  Of the 4 stories we’re doing this sprint, which business feature does this progress (or impediment) relate to?  It looks almost all done but are we saying that we’ve done 3 of our features and we’re working on the 4th or that we have we done 80% of each of them?

We do want the business to be engaged and to participate in the process so we ought to try and make it interpretable for them.

Some of my guys recently started adding a 2nd dimension to their planning boards; features.  By adding a horizontal view which groups tasks by the requirement they relate to as well as the usual vertical view which groups tasks by their status (not started, done, etc) you get an at-a-glance view of the progress of each piece of business-relevant functionality, rather than just a broad sense of things happening.

Even I find this easier to consume!  Rock on you little geniuses (you know who you are).

MVP in the Enterprise

Minimum Viable Product (MVP) is a well understood concept in startups.  It is all about distilling your features down to the barest of essentials – those things which really make your product your product - in order to get something out rapidly and be able to quickly iterate guided by feedback.  From working at both ends of the startup-to-mature-business spectrum, it occurs to me that the enterprise could learn a thing or two from its lighter-weight cousins.

To broadly generalise the circumstances I’ve seen:

We’re pretty good at this MVP thing in early stage startups.  Mostly because, as we plan an iteration, we are fully aware that we might not be around for another iteration so whatever goes into this one has to count (in fact it might count for everything).  We think lean.  We’re typically good at serving our customer’s most immediate and pressing needs.

In the enterprise we have the luxury of working with the confidence that, even if the immediate next set of features don’t quite hit the sweet spot, we’ll be around for many more iterations yet before times start to get tough.  We think deep and wide.  We’re typically good at serving a big strategic master plan.

Being able to take a longer term view of a roadmap is definitely an advantage and having to spend time worrying about things like scalability is a nice problem to have, but this doesn’t mean that we can’t also think lean.  In fact some of the most value I’ve added has been in the amount of work I haven’t done...

It can work in big business.

Recently I looked at a system to capture and analyse certain customer activity in order to get an earlier prediction of lifetime value and make data-driven cross selling decisions (which ought to push up average revenue a notch) and, on the surface, that seems like it’s worth the pretty hefty licensing, integration project, and infrastructure costs.  We started off modelling it and running the numbers; at first synthetically and later with real data.  What we learned through this early prototyping is that those features which were initially so attractive made very little difference to our business, however our little robot was uncannily good at identifying fraudulent activity.  So we switched the direction of the project, chose an alternative platform, saved a bunch of money and got some functionality which genuinely benefitted us.  Sometimes you know what the real benefits are going to be up front, sometimes you surprise yourself along the way.

We’re also planning to introduce a pay-as-you-go commercial model for our feed products.  When you add up a metering system to watch client consumption, a dynamic pricing system to set tariffs, a billing engine to produce the required invoices, and some reporting tools to keep an eye on its performance it starts to become a fairly significant undertaking.  Will customers like it?  Will it work like we hope (add top-up revenues to committed subscriptions) or like we fear (cannibalise commitment for shorter term hits)?  The best way forward in these circumstances is to ask what is the absolute minimum amount of work I can do to see if this works?  As it turns out we could do a some basic instrumentation (to get simplified metering), use some static pricing, and do the billing manually.  Net result?  A much smaller piece of work – useable in production with real customers – demonstrating how successful it’s going to be.  Now we can build on this with further iterations until we have the fully-featured and refined version we first dreamed up.

There are many more examples, some of which have worked out just fine – so we’ve kept building on them until they were feature complete, fully automated, and enterprise quality.  Others have failed – so we’ve cut our losses (fail fast fail cheap; writing off a few thousand not a few million) and moved onto something different which did turn out to be a winner.

Just because we have the budget and the appetite to do the whole shooting match right away doesn’t always mean we ought to.

The obvious risk here is becoming too short sighted with your plans.  The secret to making this work is to imagine big but plan small – keep a long term roadmap and have a clear vision of where you want all your products to get to, but take small, tightly scoped steps along that path.  Stop and evaluate frequently, adjust big picture where necessary, rinse and repeat.