Hi, I’m Eachan and I’m a fail-a-holic

I attended a Harvey Nash CIO leadership forum a few days ago, subtitled The Strategic Insight Survey. Combining words like “strategic” and “insight” as a standalone sentence usually sets off my spider sense, however I must say it was a most worthwhile evening. Other heavily overused and frequently misunderstood words that were thrown about with reckless abandon were innovation, transformation, and ROI etc – all against the backdrop of this current economic climate.

All this talk of innovation, and how you measure its ROI, was starting to tickle my aforementioned spider sense, but since a couple of the panel were from the public sector, I couldn’t help but want to explore the effect that the general public’s reaction to failure has on the culture within those organisations. This is why I’m either never invited back to these things or I’m on the panel next time.

Firstly, let’s just agree that the cost of innovation is not in salaries paid to folk you can point to and say, “he’s doing some innovating” or in capital expense on new widgets and bleeding edge technology. The price of innovation is failure. In fact, I would even suggest that the price of learning anything material is failure.

So then I look at the slating that any public service gets when things go wrong. We bay for blood in the most public of forums, newspapers, radio, and even (opportunistically) on the opposition benches. A major mistake was made, so someone has to go, right? There are countless examples of this; ranging from embarrassing (eg HMRC data loss), to genuinely horrific (eg Baby P tragedy), to that thin line between the rules and the spirit of the rules (eg MP’s expenses scandal).

It’s difficult to come to an agreement on what should have happened in any one incident, as we will all feel differently about each incident and will be affected differently by it, but that’s OK, because it is the pattern that matters. The pattern is what sets the culture.

If, every time things go wrong, we excise every last individual involved, how then do we ever hope for the lesson to be learnt? Who has felt the pain, and is carrying that forward as a driver for change? Where is, to quote a buddy of mine, the feedback loop?

Is it right for us to do this, to create such epic risk aversion, and then still expect these organisations to deliver even the most basic of improvements let alone giant innovative leaps forward? Are we just robbing them of the best ingredients needed to create the huge changes we’re demanding?

When I think more about it, I wonder if I really am a fail-a-holic at all. Perhaps I only embrace failure so much because I’m actually a success-a-holic…

Don't out-specify your knowledge

Combining some points from this post by Joe Armstrong and this post by Dan Creswell gives rise to some interesting implementation thoughts.

Standardisation is important, particularly in the context that Armstrong frames it in - if the interaction between a client and a server closely adheres to a predetermined protocol, then the systems overall behaviour becomes more predictable and easier to adapt and fault find.

But then this depends on the maturity of your understanding of your problem and your solution. As Creswell says at the beginning of his post, defining too much up front can be a trap. Coming up with a set of standards that are too rigid too early might start you down a road of building that system instead of building the right system.

Standards, patterns, and contracts are all important - but timing is key.

Choosing Recruitment Agents

Every now and again you'll go through that compulsory "we use too many agencies and we need to rationalize" process. As part of that, you'll be working out a few criteria for evaluating recruiters and putting them on some sort of preferred supplied list.

The typical approach to this pretty much involves coming up with some lower rates than you currently pay (cost control is usually the motivation behind the PSL exercise) and seeing who wants to sign up. Not a bad start, especially if you come up with some tiers which (for example) encourage recruiters to take a lower fee for a period of exclusivity, but you should probably add some metrics to ensure quality.

Firstly, think about why you use an agency. Hiring strangers is the worst recruitment strategy ever, so when you use an agency, you are basically using their network as a replacement for yours.

So how about finding out some other stuff too; do they meet candidates in person before they put them forward? How long do they typically have a relationship with a candidate for before they start to place them? How often do they place the same candidate in consecutive roles? Anything you can come up with to give you confidence that their network is going to be a passable proxy for yours is going to be a worthwhile comparison point.

No problem too small, no reward too big

Do you ever sit around with friends, talking over a few beers, and solve all the worlds problems? As I get older this seems to be becoming my chief pastime, and we recently got ever so excited about power generation and distribution. We figured the whole thing could be completely re-architected in a P2P model, where everyone in the grid both consumed and generated/supplied power.

Rushing to the nearest browser in case a patent application was necessary, we became a slightly despondent upon seeing a bunch of articles like this one on microgrids and what the EPRI are getting up to. That said, it's great to see this kind of stuff getting mindshare.

So what, if anything, is still missing? To see this come true, it seems like we'd have to work our how the power companies would still make money in our community-spirited utopian electricity supplying future. So perhaps we can still contribute by bringing some economics into the solution.

Someone has to provide that infrastructure - the grid we'd all have to be connected to in order to exchange electrical capacity. That sounds like something someone should be paid to provide. So perhaps there is a subscription fee? Or maybe we can do one better - how about a variable charge based on each household's net balance at the end of every month/quarter/year? The closer you are to netting out (producing the same number of kilowatt hours you consume) the closer to zero the bill gets.

The beauty of that model is it incentivizes consumers to be as self-sufficient as possible and disincents (is that even a word?) waste. The expected impact of such a price control is for participating individuals to want to align their production with their consumption - probably by boosting the capabilities of the former while working out efficiencies in the latter.

Reducing overconsumption is probably a no-brainer, but you could look at reducing overproduction both ways. On the one hand, overproducers supply overconsumers and we'd need that, but on the other hand we wouldn't want participating individuals to profit from overproduction. Ultimately all overproduction is wasteful (by definition if nothing else) so there should still be a price for it, a lower price than overconsumption, but not as low as netting out.

Well that was all kind of off-topic, but I hope you enjoyed it nevertheless. There are some intriguing engineering challenges in the basic problems inherent in sustaining our society, and if we gave this kind of stuff a little more attention, we'd all be living in a much better world before you know it.

Gartner Economic Downturn Briefing - Part 2

Continuing on from yesterdays post; the sessions reinforced to me how little of the core Gartner content is aimed at web companies. Almost all yesterday's stuff was only really relevant to 10,000 seat corporate enterprises. I've worked in several big companies before, and the internal "business process support" technical challenges they face are very different to our external product-led technical challenges. We also don't tend to have such epic workforces and supply chains, we tend to be pretty lightweight and entrepreneurial by comparison.

Back on track - the economic downturn. It's kind of uncomfortable to think about the upside of something that is affecting so many so negatively, however we're well positioned to take advantage of the opportunities that are coming to the surface.

With a battening-down-the-hatches mentality, people are preparing for the worst. Sites like Fool and Lovemoney pick up as we try to get as many money-saving tips as we can. The first preventative action we're likely to take is reviewing utilities and other basic household expenses, and sites like Gocompare and Moneysupermarket see a lot more visitors.

On top of this, there are a whole slew of creative new ideas starting to come through, all predicated on our current need to squeeze a little more value out of things we otherwise tolerated waste in.

At the worst end of the scale, people are being made redundant. Again this is a terrible tragedy affecting people I know, and it feels unfair to be thinking about the opportunity at a time like this. Nonetheless, the internet is now the de facto tool of choice for finding your next role, and this means more actives on Jobserve and Monster. Whether you've been affected yet or not, it is now more important than ever to get an up-to-date profile out there and build up strength in your network (before you need it). Off we go to the professional social networking sites like Plaxo and Linkedin.

Lining my own pockets for a moment, online gaming and gambling is fairly recession-resilient, but even simple ad-supported business models can do OK. You're still turning page views and actives into revenue, and if you're providing the service or content that's currently in demand, then that's not insurmountable in any times.

Gartner Economic Downturn Briefing - Part 1

It's been ages since I went to a Gartner briefing, and I figured this would be a topical one to pick the habit back up with - perhaps get a look at the ways other organizations are tackling this credit crunch thing. You guys often tell me that my posts can be a little too long, so today I'll summarize the key content and tomorrow (or maybe Monday - hey I have a day job too you know!) I'll put my thoughts up.

The sessions were focussed on how CIOs and CTOs are responding to the current economic circumstances, and the techniques IT organizations can use to keep cost under control and contribute to the businesses downturn survivability. So here goes - in no particular order...

• Focus on projects that are "shovel ready". Bang for buck is more critical than ever, and projects ready to be actually done are worth a lot more than potentially better activities that are still in the planning phase.

• Expect to deal with massively escalated regulation. In the post-credit crunch world, there will be a lot more regulation aimed at preventing repeat performances. This is likely to mean increased reporting and compliance overhead, as well as more constraints on how technology can be deployed and data used.

• In the worst of cases, there may well be a sharp rise in the number of legal actions. This is going to put increased demand on e-discovery and BI as organizations scrabble for the information they need for defence.

• A general loss of trust is going to lead to more diligence efforts. In a climate where any number of suppliers could go bust at any time, you'll need to spend extra time reviewing the financial integrity of key partners you depend on, and perhaps even making backup plans.

• The downsizing mentality could rob you of the top talent that you'll need on hand to steal a march on your competitors once times start looking up. Don't sack them, squirrel them away any way you can!

• Expensive oil means old assumptions on locations, topology, and transport need to be challenged. Get better at communications, and be prepared to accommodate supply chain changes aimed at closing the distance between steps.

• Check out your PR infrastructure. During these times the amount of media attention on companies is at a peak. Where is your corporate site hosted? Who edits the content? It is often an overlooked side-system, and right now you don't need the attention you'll get if it goes down.

• Take a look at your HR systems too. When companies get told to lay off 12,000 people over 12 months, their HR processes face a task they were never intended to handle. Pay special attention to data accuracy - mistakes in calculations on this scale will just kill you.

• Watch out for desperation moves. These are the times when you'll be the most likely to get stitched up by some crazy marketing plan, an over the top offer that gets madly oversubscribed, and so you'll want to keep your interdepartmental communication flowing and your eye on capacity.

There is some good sense in this, and some stuff I want to look at from a slightly different angle. My side of it - and what I think it means for us web guys - in the next post.

Scalability is not just a technical problem

There is so much content out there about how to scale out web sites, platforms, and databases – but it all focuses on the production system architecture.  Do a Google search for scalability – go on, I’ll wait for you...  See what I mean?

Now I know that’s the fun bit to talk about, but being practical for a minute, if you’re starting to scale out systems using techniques like Digg, Flickr, Xbox Live, or the usual suspects like Amazon, Google, and eBay, then chances are that you’ve got a whole lot more scalability challenges than just the product.

If you’re dealing with at least hundreds of thousands of daily actives, then we can probably deduce a bunch of other stuff about your circumstances.  We can guess that you’re after reasonably frequent feature drops, have a significant amount of horizontal distribution, a healthy sized engineering organisation, and a strong bias towards availability.  And if even half of that stuff is true, then what are the other scalability challenges will you be up against?

How about multiple concurrent projects?  If your estate is divided into more than one product then you will more than likely be working on more than one new feature in parallel.  This gives rise to all sorts of version control and regression test problems, which demand process and infrastructure quite different to a single effort.

What about disparate teams?  You might have people in a number of locations branching, or depending on, the same codebase.  That’s a communication barrier which can be tough to solve.  Large enough organisations also tend to sprout specialist disciplines, such as user experience and IA – this changes the nature of how teams engage and how work is specified, estimated, and delivered.

And how do you manage environments, tools, and documentation?  A complex production architecture begets a complex development infrastructure, as there is a lot more interoperability to test for.  Don't forget that with more teams working concurrently, managing contention for these expensive environments also becomes a tricky balancing act.  As your products increase in popularity (a good proxy for profitability on the web) NFRs like performance and capacity will become more important and will require specialised tools to measure.

I’d like to see us sharing a little more about our experiences with this side of highly scalable systems – it might not be as sexy as memcached, CAP, and Gossip, but the reality is it is just as important a part of the solution nonetheless.

The Good and the Bad of Google Product Management

I don't often pick up stuff from the news - I don't really consider what I write here as commentary - but this one I couldn't resist because it's such a good segue into something I wanted to talk about anyway.

In a way it's a little sad to think that these harsh economic times are starting to have an effect on even that most hallowed of engineer-driven organization. Still, there are those of us probably a little less melancholy - I can just imagine how much fun serial cloud hater, John C Dvorak, is having - "told you not to keep stuff in the cloud, one day they just switch it off and then what do you do?"

The decision to shelve a product is a difficult one, and one that inevitably leads to some degree of unpopularity both internally and externally. What I wanted to do here was take a look at what this article tells us about product management practices in general.

Good Stuff

• Willingness to take risk. Especially in these times, it can be tempting to stick to safe territory. That can be good advice, but what are you missing out on? Another way to look at it is when everybody else is playing it safe, it’s easier to stand out with something unique.
• Recognise the capability gain in products even if the product itself doesn’t work out in its current incarnation. Taking what they gained from Dodgeball and turning it into the Maps add-on Latitude is a great example of the sort of utility you can get from something that doesn’t work out in its initial form.
• Customer feedback. Accessing the community through techniques like product development blogs and using that knowledge to shape the features and user experience is a powerful way to make sure your system ends up meeting expectations as closely as possible.
• Attitude to failure. Viewing things that don’t work out as contributing to overall success rather than detracting from it helps you really internalise what you learn and encourages the innovation that fear of failure will crush.
• Internal testing. Eating your own dog food is always helpful, after all, if you find your product difficult to use then what are you asking of your customers?
• Opportunity cost. I liked the part where they consider the opportunity cost of a project as well as its real cost; i.e. what is the value of the things that the engineers are not working on because they’re working on this?

Bad Stuff

• Incentives. Paraphrasing, the article talks about difficulty attracting engineers to work on certain products. There are some reasonably well publicised shared reward schemes in place, and in this free market-like environment management needs to consider how it might need to provide external subsidies to make sure that less glamorous, but perhaps strategic, projects get attention too.
• Perpetual betas. I really couldn’t decide whether this one belonged in the good or the bad section, so I let the need for balanced paragraphs drive the decision. On one hand, I think there is no better way to refine user experience and focus the tail end of your development efforts on polishing the things users like the most, but on the other hand there are circumstances under which you need to show the commitment of a production-ready seal of approval.
• Communication. While not exclusively a product management issue, communication across geographical boundaries was a major issue here. This shows how important collocation is in a product development process.
• 20% time. I think we’ve all heard mixed reports about how this really works, but regardless of that, if you have a product that you’re committed to and have set targets for then you simply must formally assign it resources. Relying on product management to recruit people from their 20% time is setting it up for failure.

The article also mentions internal performance targets for new products, and in all businesses objectives and measures are the true meat of how product decisions are driven, so it's a shame we didn't learn more about that. Also some kudos due for how they ended Jaiku, the Twitter clone, it's good to see them pass that on to the community rather than just confine it to the shelf.

Giving References

I was recently asked to act as a reference for a couple of people, and that got me thinking about the policy most organisations seem to have on this kind of thing (i.e. don’t do it). I can understand where this came from – representing the opinion of your company is a significant responsibility which has all too often been wielded by the undeserving.

Obviously I’d never advise flying in the face of company policy, but I really like giving references. In fact, I consider being able to give a good reference to be a lagging indicator of great success with people.
You see, if I’m able to give a good reference for someone, it means two things:

1. We’ve had a successful, effective working relationship in which we’ve helped each other deliver tangible value to the business.

2. I’ve been able to help that person develop professionally, and I’m now helping them to continue to grow their career.

Is there any greater satisfaction to be had in management? It’s why I do the job.

Why are they never happy?

This is another one of those stories that my girlfriend will tell you explains why we don't get invited around for dinner more than once, but I'll tell you is dead essential to the world of technology becoming a better place - one manager at a time...

Anyway, I'm listening to this buddy complain about how busy he is, how hard he's working, how many hours his team are putting in, and how much stuff they're getting out - yet still his customers aren't happy. They're always finding something to complain about, pointing out the things that are missing rather than thanking him for the things that are there.

When I hear this, what I think most people are really saying is "my team and I are working really hard and really long hours doing the things we want to do for this customer". Whenever I've seen this issue on the floor, it has almost always been because the guys were dedicated and hardworking, but being dedicated to and working hard on the wrong stuff.

Think about it like this; if I really want A, and you give me B, C, D, E, F, and G, then I might be impressed by how busy you've been and how much work you've done, but I still won't thank you for it. Yes, B through G were relevant to my department and they were things I wanted done - no argument there - but if A was the thing that meant the difference between getting that big contract or not, or making this quarters numbers or not, then I would have happily traded it all to have just A.

I'm not saying that there aren't ungrateful people in the world, but I am saying that until you've adopted a partnership-based, customer-centric view of how you deliver technology, you haven't earned the right to complain about them.

Look frequently and carefully at your priorities, and compare them against what your business is being driven to achieve - you never know - you might even end up doing less work and having happier customers.