The Fletcher Project

Eachan's 5 Laws of Platform Architecture

2011-12-03T17:04:00.001Z

Architecture - the shape of the system, the patterns used - is by far the most meaningful thing to get right in any system. Architectural decisions have far more influence on the capabilities and limitations of any given system than the implementation decisions - frameworks and languages - ever can.

This effect is hugely magnified when you're building a platform, because you're creating tools and raw materials for other developers, and your decisions can enable them to easily build apps any way they can imagine or limit their options to a handful of pre-configured choices.

Side note here; the internet is becoming full of platforms. It has never been easier to programmatically consume services on the web, even through interfaces originally only intended for humans, and combine existing data in new ways to create whole new experiences. You might put an app out there for customers, which you consider to be a finished product, and then discover that someone else is using your app as a building block in their app. One man's front end becomes another man's back end. I think this is a good thing for momentum and creativity; allowing new takes on current ideas to be spun up and explored quickly and cheaply.

But if you intend to build a platform, if that's what you're explicitly setting out to do, then there are some design guidelines which will lend it greater commercial utility. Not patterns per se, more like principles, which I'm going to call the 5 laws of platform architecture:

1st Law of Platform Architecture - the value of a platform is directly proportionate to the amount of work it causes to be unnecessary.

Most web businesses are more similar than they are different. There is a set of web basics they all need (identity, profile, cart, payment, analytics...) and there are usually a set of line-of-business basics that most of them have in common (in my case that's things like inventory, price, geography, booking, weather...) and that's an awful lot of code that we're all writing. If a platform solves the 'common problems' via a handful of simple web service calls, then partners using said platform can put the majority of their time and passion and investment where it will make the biggest difference to their business; building the unique features which distinguish them from the rest of the marketplace. I like to ask "what don't my partners have to do because we already did it?" and I like to have a long list of things to put into that bucket.

2nd Law of Platform Architecture - ease of adoption maps directly to business momentum.

Most serious platforms are monetised in the same handful of ways, and in most cases the sooner you can have partners up and running the sooner you're counting revenue from them, ergo time to market for your partners is a lever in your own P&L and if you focus on ease of adoption you will bring on more partners, they will each 'go live' quicker, and it is cheaper than more traditional incentives such as offering improved commercial terms in exchange for faster rollouts etc. The kinds of things you ought to consider here go beyond the simplicity of the services you expose (which must be easy for the most pedestrian of developers to grok - in the platform game you are penalised for externally-observable complexity) and into how much you invest in documentation, SDK and client code, and developer community support. These things pay you back.

3rd Law of Platform Architecture - entrypoint depth multiplies the business flexibility of any given service.

Higher layer abstractions always infer business logic. That's what a composite service really is; someone decided that for use case alpha, you always do X then Y then Z, and therefore aggregating calls out to services A and B and C solves that problem in a single call. You have just created a simpler solution for alpha - which is an admirable achievement and it belongs in your platform exposed to everyone else who has use case alpha. But let's say I have use case alpha prime? A basically similar business process but with key differences that no longer require data from B? That's why your underlying services belong in your platform too; if a client has some business logic which maps quite closely to the higher layer abstractions then that's great orchestration for that app. If a partner want to do something a little different, then he might have to consume those composites and do a bunch of his own transformations on the data, perhaps even store the data himself, and then present it to the client apps through his own interfaces. With the option of consuming services from lower down the stack clients can build unique apps on top of the platform with far less 'back end' work (stripping out the inferred but irrelevant business logic) on the client side.

4th Law of Platform Architecture - compute work becomes exponentially cheaper and faster as it approaches the edge.

At last something more technical! If you're building a platform, and you're not planning on comprehensive global coverage, then I would question your commitment to your idea on a web where ubiquitousness has never been cheaper to claim. So let's assume that you are doing this globally. Without getting into a discourse on distributed systems, you're going to want to deliver straightforward and performant access to every consumer regardless of where they're based. That is a distributed system problem, and so your platform should run on a system with a diameter greater than 1, and you should try to service requests as close to the edge (and as high up in your stack) as you can. There are plenty of mature grid and edge computing infrastructures out there to provide the scaffolding for this. What does it mean for your partners? They're using a platform with better durability and lower latency than alternatives that they might choose, which translates into better SLAs and better service which also happens to be cheaper to deliver.

5th Law of Platform Architecture - as services become more atomic the addressable market segments become more broad.

My 3rd law deals with entrypoints, and this is nearly the same thing. Lower layer abstractions allow platform consumption without constraining the use cases, but another critical property of those granular services is their atomicity. Your low level granular services must perform generic, stateless (independent) operations, all be individually consumable, and each should be of business value individually. When I say individually consumable I think I really mean individually usable. If one has to consume a handful of other services to be able to make sense of the data returned from a single service, then it clearly isn't too useful in it's own right. Perhaps it needs slightly more data or metadata around it or maybe it is just noise in your stack - keeping the catalogue manageable is actually a harder problem than it may seem. When your lower layer abstractions are individually useful you find that more kinds of businesses can make use of them. To round this out with an example; exposing our geography data distinctly (i.e. outside of a composite which applies it to a booking flow) enables us to power navigation and mapping apps, instead of just travel apps.

These are guidelines for architecting a platform to maximise its utility to developers and its commercial success as a set of business building blocks for a range markets. But what about the shape of the system(s) behind each service? That dimension of architecture - how you manifest each of your APIs - is just as important.

We all have our favourite ways of doing things, and pushing one pattern over another without an understanding of the business problem and environment is just bad science, so the best I can do here is suggest establishing some engineering principles to give engineers a decision making framework within which they can own the problem and still consistently meet the higher standards expected from platforms. Scalability, Maintainability, Quality, Security, Availability, and User Experience have always worked for me.

Whatever you do keep in mind that if you operate a platform, then you're partly responsible for the availability and integrity and brand and revenue of n number of businesses (i.e. the partners building their apps on your stuff) so you have an even greater responsibility for quality than you do when your building your own customer-facing apps. I'm a pretty casual guy about most things, but that's a responsibility I take very seriously indeed...

The difference between taking a shortcut and cutting a corner

2011-11-05T20:31:00.000Z

There are two very different things to talk about here, with two very different implications for your software, but I'll admit that which way around you label them can simply be language pedantry. So for the purposes of this discourse, let's say that:

Taking a shortcut is using your knowledge and experience in place of studying things from first principles every time.
Cutting a corner is a compromise in quality or good practices in the place of the prudent actions otherwise required.

Both are used to speed progress through a given piece of work. Using knowledge and experience is a time to market competitive advantage gained through having talented people, compromising quality is a time to market advantage gained through process; albeit purchased at the price of later cost.

Taking Shortcuts

Let's talk about taking shortcuts first. When you can do this confidently it is a magical place to be. I like to hire really smart people and give them the freedom to use the experience they've gained throughout their careers to leap frog us straight over the other organisations who had to do it all from scratch, the sources of the experience I want to tap into.

Experienced people have been through years of doing everything by careful analysis and research, gradually proving out a larger and larger set of assumptions which they can build up from, and using life's great feedback loops to tune their thinking until their instinctive decisions are as a good as a noobie's 3 months of science. That's why we hire the kind of people we hire.

Having all that collective wisdom at hand and not using it is irresponsible leadership and suffocating to the innovators you should be growing. There are still times when the right thing to do is take things back to first principles and do that 3 months of science, but the beauty of experience is it also tells you when it's right to use your instincts and when it's right to figure it out.

Cutting Corners

And now cutting corners. Reducing time to market but doing something less well (sacrificing quality) or by skipping validation steps such as doing less testing (taking more risk) works in the sort term, but really just exchanges a short term temporal win for a lot more trouble later. Often more later trouble than you really think, which is why this accelerent is so frequently misused.

I say 'freqently misused' quite deliberately, because in a pragmatic real world it is not always the wrong thing to do. Occasionally it can even be the path to very big wins, when time is absolutely of the essence and every day counts materially. But it is always the wrong thing to do to unconsciously blunder into cutting corners, to not be actively managing your quality, and to incur this type of technical debt without having a credible plan to come back to fix it up before it festers too long.

If you are not grown up enough to manage the payback, then you are not grown up enough to incur the debt.

So...

Hire people smarter than you are. Empower them to use what they know and what they've done for your benefit - that's what they want anyway - and to use their time with you to further expand their horizons. Carefully manage your quality, incur technical debt strategically, and never do it without paying it back.

You'll do a good job and everyone will have a good time, most importantly your consumers.

Nobody cares about your tickets

2011-09-18T21:52:00.000+01:00

As engineers we like to speak in tickets and stories and bugs IDs etc - those units of currency we find easy to grok; they're usually finite, easy to define, narrowly scoped, and uniquely identifiable.

Our customers - for the most part - like to speak in outcomes and business plans and commercial objectives and projects. The longer term, more vague, abstracted-from-the-work things that we like to digest by breaking them down into those stories etc that we're so comfortable with.

This can lead to a weird situation where you have all your iterations scoped out and your releases forecast (with painstaking detail in every story) yet the business still lacks clarity about what's being delivered by who and when. Hmmm.

I think a bit of generalisation can help us to understand this. Lets appreciate that engineers, for the most part, are deeply practical people who are comfortable handling actionable tasks. Since we're also geeks - again for the most part - this descends quickly into technical details. You'd be surprised how unintelligible this can all be when you're not a developer.

What I've learned over the years is that stakeholders are interested in the meta problem, the commercial plans and outcomes, not the tangible problem, the technical tasks and code which need to come together to solve it. As technical leaders we need to talk about the meta problems with the business, and the technical problems with the engineers.

This comes with two obvious challenges - whenever we're talking about the same things but using different currency we rely on a degree of interpretation and some things can be lost in translation. This most commonly surfaces as that old "unclear requirements" thing. The other challenge is making sure that, as we solve our technical problems, we're solving our meta problems. This isn't easy and this is why great product managers are so crazy hard to find!

This kind of interpretation of work - the aggregation of agile units into bigger picture project plans - is usually an early casualty of agile adoptions. It is reminiscent of waterfall so we label it bad. But it is a helpful mechanism which bridges the gap between agile execution and regular business planning. It gives you both currencies.

Get awesome product people - they are the only other ones who live in this gap with you and have to speak both languages.

The secret of managing software projects

2011-09-12T15:13:00.001+01:00

As a technology manager there is almost unlimited discourse you can read about how to successfully manage delivery and control projects. The real secret here is you don't manage projects, you manage the circumstances around a project...

This assumes that you have a team of talented engineers, a project manager as strong of character as he is on process, and an architecturally sound plan with some slack in your estimates. If you don't already have this then don't worry, you're not about to fail, you've already failed. What you're about to experience is you doing your best to mitigate the consequences of poor leadership. Good luck with that.

But let's just say that you've got the structure right - now what? If that's true, then the best contribution you can make as a leader is managing the environment around the project. Provide a vision and simple insight into what moves the dial for the business and then move the obstacles, keep the distractions away, make sure the priorities don't change, and protect the productivity of the team members. Keep the project fed and watered; get it the budget and skillsets it needs, get it the right visibility and attention from external teams who may be dependencies. You're usually on the hook for the whole thing, so the temptation to dive into the details is often overwhelming. Do it if you must but know that you probably aren't helping. Your guys won't feel trusted and you won't be encouraging them to take ownership of their own space. If you have a good plan on a sensible horizon, don't fiddle with it. Every time you do you increase the chances of things not working out.

That doesn't mean you should be disinterested or uninvolved - quite the opposite. Be there 100% for every engineer every day and give them everything they need to give you what you need. Be easily accessible and take on any potential problem, consequence-free, that your team feels like raising. They don't want to waste your time any more than you want to waste theirs, so if they come to you with something, it will usually be because there is a genuine risk that you're not going to get what you planned and there is something you can do to put things back on track.

I love technology. We all do. That's why we got into this business. I love what my guys do, and I love to talk to them about it, but I am always aware of when prudent interest in the organization's deliverables crosses the line and becomes micromanagement and interference.

The most important thing you can give to any project is certainty. Make a good plan based around people better than you are and then defend them.

Great Quote and Greater Source

2011-08-22T21:17:00.000+01:00

This article has a great take on developer-husbandry, and I particularly like the last paragraph:

"I’ve gone back and forth on whether managers should code and my opinion is: don’t stop coding. Each week that passes where you don’t share the joy, despair, and discovery of software development is a week when you slowly forget what it means to be a software developer. Over time it means you’ll have a harder time talking to engineers because you’ll forget how they think and how they become bored."

Even better is who sent it over to me - one of my commercial stakeholders. Being in a tech savvy business makes my job a lot easier in a million little ways.

Why don't we invent more?

2011-07-22T18:15:00.000+01:00

I sat in on a conference call today (actually I'm still on it!) where we talked a lot about innovation - specifically why we haven't done more of it. Lots of different stories but the common theme seemed to be governance.

When I say governance I mean things like roadmaps, product councils, architectural oversight, PMO, etc. Basically the constructs we've set up to steer our technology investments and spend our resources wisely. Properly applied, they are critical to the success of any significant engineering endeavour. Improperly applied they are critical to the failure of any significant engineering endeavour! So are they being properly applied to invention and experimentation?

I don't think it is a case of proper application as much as it is a case of application at all. I think any engineer has two jobs; serve the roadmap - build the applications and systems defined and managed by our governance processes - and serve the technology - discover new ways of solving our business problems, advance your own knowledge by experimenting, come up with new ideas and see if they fly. If you're only doing the former then you're only doing half your job.

These are two very distinct types of activity and they shouldn't be governed by the same controls. It just makes no sense to apply ROI and risk controls and rigid scheduling to a journey of discovery with a totally unknown destination. The difference between teams who innovate and teams who just talk about it is recognising the immense value in the journey alone...

Besides, you don't need all that bureaucracy. All you need to be great at this is to leave a space for it to grow and - if you have the right people - it will expand to fill it. That is how I roll and that is what I want to see.

Big Roadmap seeks Innovative Thinkers for long term relationship

2011-07-13T22:20:00.002+01:00

I haven't posted much recently - my new gig has been keeping me quite busy - but life is returning to a more sustainable pace now so I'll be able to start sharing more often again. In the meantime I once again find myself drawing up battle plans in the war for talent. So what are my 3 top weapons I can bring to the fight?

Step 1 - oops almost out of desks; let's get a really kick ass place to house all these smart engineers. So we're moving to a sweet new spot in Angel. We're growing fast and it's already nearly standing room only at the current office. Covent Garden has its charms, but our new spot is being custom fitted - just for us - to be one of the best work spaces in the UK. Check out the progress:

The list of features includes eco-geekery such as biomass boilers and heating and lighting driven by room occupancy sensors, high ceilings with 360 degree sunlight, huge storage space for bikes with plenty of showers and lockers, a variety of breakout areas for informal meetings, and a nifty cafe and Jamie's Italian restaurant on the ground floor. All 80,000 sq ft of it (a huge uplift to accommodate our growth) is coming along nicely and is 30 seconds walk from Angel tube station.

Cool. Now for step 2 - set up the right environment and culture. So what is it actually like to work here? Well, you could take our word for it, or I could give you the inside scoop...

When you get right down to it technology is core to what we do - our whole business is built on it and it drives our partner's businesses too. Therefore we have to have a culture which supports great engineering in order to succeed. That means bringing in smart developers, letting them own the problems, and then giving them the space to find the best ways of solving them. It means creating time for innovation, it means allowing technologists to make technical decisions, and it means permission to work in the implementations which make sense (and maybe even a little pushing to keep you expanding your horizons) instead of being dogmatic about any particular stack. Add to this a sustainable business model with plenty of big wins left to achieve and some clear priorities and shared steering of the business, and you have yourself the right ingredients for putting together some really successful software.

In the macro, Expedia is a very large place but we operate in a kind of group structure. So yes, it is a very big company and that [fortunately] comes with some big company benefits, but the work experience is more similar to working for a well funded startup. We work as small, independent, cross-functional teams, with each team having a good mixture of autonomy and support. We believe that our guys should have end-to-end ownership of the systems they build - everyone is part of a small elite team and feels like they're personally making a difference. If you've read a few of my posts before then you probably have a fair idea about how I like to run things.

Step 3 - challenges that really make you think. This is an exciting time to be part of creating something pretty unique; I've said recently that we're about to change the rules in web travel and it's going to be fun. Obviously I won't share my roadmap here, but I will talk about the sort of technology we'll be working with. How about creating an eventually consistent distributed datastore latency tolerant? What about peer-to-peer cluster management in the cloud? Making an n+1 node in-memory cache topology aware? Automated provisioning based on realtime system utilisation feedback? Inventing new sorting algorithms and queuing strategies? Or your own anti-entropy protocol? Collecting obscene amounts of performance data and rebuilding your tools to watch those numbers move?...

You'll notice that I didn't say 'Java' or 'C#' or 'SQL' or 'PHP' at all. That's because, to a certain extent, those things aren't what's important. What's important are the patterns, the fundamentals of how we logically solve business problems using computer science, and that is what great engineers do. Then they write some code as a result.

If that sounds awesome to you, and you can show me some aptitude in this kind of space, then we should talk (especially if you're Cassandra/MapReduce curious). Get hold of me or my recruitment ninja, Roopesh Panchasra, right now.

PS - I am also hiring in the US, so drop me a line if you're stateside and want to work with smart folks on some sharp stuff.

a geographic sense of urgency

2011-04-28T10:32:00.000+01:00

A friend of mine recently visited the Boeing factory just outside of Seattle and came back with a story I thought was worth sharing; something they called 'a geographic sense of urgency' over there. It's a nifty story about motivation and incentive. I'll do my best to retell it 2nd hand.

A few years ago Boeing was struggling with efficiency on their assembly lines. They needed a way to build airplanes faster but were approaching limits on what could be done in parallel, how long a working day could be extended, or how many more resources could be applied to each assembly (outside of the economic ceiling I guess there must be some limitation to the number of people who can fit around the airframe).

To grossly oversimplify - with apologies to aircraft mechanics everywhere - a plane started at one end of the production line and moved through a series of stages; stopping at each 'station' to fit whatever parts that station fitted until eventually arriving at the other end of the hanger complete. Another way to describe this might be to say that a plane moved through a series of bottlenecks, with all pressure focused on the current station until its work was complete.

The answer borrowed heavily from lean but was all anchored around a single, creative idea; the plane never stops from one end of the hanger to the other. Instead of rolling forward a few dozen meters and then stopping to have work done (loop until plane=true) the plane is constantly moving forward by a few centimeters an hour and only stops when it reaches the end of the hanger - hopefully as a complete aircraft! Flow. Work happens more fluidly and consistently all the way along the plane's terrestrial journey.

This simple change to constant motion had a remarkable impact on the teams. I wish I had more data (and if I can find any I will post it) but the improvements in throughout were astounding. The physiological effect was a sense of pace - this is what they call a geographic sense of urgency. The results were significant increases in efficiency and total throughput without extending crews or hours or resorting to the traditional but seldom as effective as expected measures such as pay increases.

Smart idea Boeing.

Intellectual Property in 3 posts – episode I

2010-12-07T12:07:00.001Z

I originally called this ‘Intellectual Property Done Quick’ but, when I was finished and looked at what I had wrought, I realized that ‘done quick’ was likely to be in breach of the trade descriptions act.

So I’ve broken it up into 3 tasty morsels; today I’m doing an introduction to different types of intellectual property and I’ll follow up with applying for a patent and then finally some tips for managing IP in the enterprise.

There are 4 main types of intellectual property defined in UK law and, along with their relatively dry legal descriptions, they are:

Trade marks

Indefinitely protects signs which distinguish the goods or services of one undertaking from those of another.

A trade mark is a distinguishing badge of origin which can be a valuable asset when it imbues a product with the reputation and perception of a certain maker.

It can become a trademark either by initial registration or it can acquire trademark status over time through use.

McDonald’s golden arches, the UPS shield, and Nike’s swoosh are all good examples of a trademark – instantly recognizable and clearly identifying a specific brand.

Registering a trademark gives an organization or an individual the right to prevent others from using the same mark it in relation to similar goods or services (listed in the application). Even if you have no appetite for hunting down and suing IP trespassers, registration is still a good idea because it prevents that happening to you – priority (‘I was already using that before they registered it’) is a tough argument in trade marks. And, in the longer term, if you’re ever likely to want to license the use of your trademark to others then registration establishes an official article against which a license can be granted.

Patents

Protects the technical aspects of products or processes for 20 years.

A patent gives the patent holder exclusive rights to prevent anyone else making, using, or selling their invention for a fixed period of time (usually 20 years) before it becomes part of the public domain. Patents cannot be extended beyond their initial term – they arose as a way of governments encouraging innovation; essentially we agree to keep inventing stuff for the good of mankind in exchange for a period of state sanctioned selfishness in how that invention is manufactured and sold.

Patents are territorial – only good for the countries they’re granted in – but a number of international agreements exist which allow. In the EU we have the European Patent Office where a single application covers 36 nations, and for the wider world the Patent Cooperation Treaty covers 130 countries. In both cases you have 12 months from your ‘home’ filing date to extend internationally before it is considered a new application – this can be important as your initial filing date is considered to be the date from which protection becomes effective.

Check out James Cameron’s platform for stereoscopic image acquisition (also known as a camera mount) for example and improvement in velocipedes (also known as bicycles) for something a little more old school.

Registered designs

Protects the appearance of products for 25 years.

For a long time it has been recognized that the appearance of a product can be the key to it’s commercial success – may I present, as evidence, anyone who buys anything from Apple.

A registered design gives its holder the right to prevent anyone else making or selling products which look and feel the same as the registered design. It covers tangible, physical, crafted things (like the famous coke bottle) as well as conceptual, aesthetic things (like the famous coke logo).

Registering a design is quite straightforward – compared to a patent application – as it does not involve any detailed scrutiny by an examiner. Apply to the Intellectual Property Office and, within 3 months, you could be the proud new owner of a registered design. Also unlike patents, you have 12 months from when your design first becomes public to register it; with a patent when it’s out it’s out!

Design rights

Protection for appearance of products without any special application being made (for example copyright).

Design rights are a collection of automatic protections which apply to various original works. In practice they work very similarly to registered designs – and apply to similar works and materials – but do not need to be applied for in order to take effect.

The subclasses (if you want to be geeky about it) are:

UK design right – protects shape and appearance, except for surface decoration, for either 15 years from the creation of the design or 10 years from the first time the design was marketed.
Community design right – similar protection to the UK design right and also covers surface decoration, however it is only valid for a maximum of 3 years from the date the design was made public.
UK copyright – can be thought of almost as the opposite of UK design rights because copyright covers surface decoration (not form and function) and ‘artistic’ qualities. In force for 25 years from the end of the year in which the design was first marketed.

If design rights apply automatically why would you pay to register a design? If you rely solely on automatic design rights then you must prove that a similar product was copied from your protected material – with a registered design you can prevent another party from making or selling something similar whether they copied you intentionally or by coincidence. It’s pretty easy to determine that two items are similar, but a lot more difficult to prove why.

So that’s our 4 identified types of IP.

In the technology and software business you’ll mostly deal with copyright and patents. In my next post I’m going to walk though patent application process since copyrights kind of happen on their own and the granting of a patent looks simple from the outside but turns into quite an arcane art.

And finally, now that we’ve covered formally establishing and reserving rights using the various offensive and defensive legal tools available to manage IP, the thought I want to leave you with is that there is nothing quite like good citizenship. Treat the property and creations of others in the same way as you’d like them to handle yours – patents, copyrights, or not.

Steriods for Signups

2010-11-19T09:48:00.002Z

Anyone who runs a business on the web should be intimately familiar with the registration funnel (sometimes called the recruitment or conversion funnel) so - except for the briefest review to frame the problem - I'm not going to rehash that here. Instead I'm going to cover some common and some not-so-common ways to boost registration numbers.

So, for the quick review, the registration funnel is basically the set of steps a potential customer traverses on their way to becoming an actual customer (converted) and typically starts with a site hit and ends with an active user account. What these steps consist of varies with the nature of each business and might include things like a shipping address or a credit check along with the basics such as username and password.

The problem every web business faces is that the registration funnel is essentially a very lossy process. For example; of every 100 site visitors, 50 will start the registration process, of that 50 25 will complete the process, and of that 25 10 might go on to place their first order. Even with the best husbandry many web businesses have single digit conversion rates. With marketing on the web becoming more sophisticated and SEO/semantics driving smarter access to content getting the numbers in the top of the funnel usually isn't the problem and - just like any other lossy process - there can be a big impact on the bottom line by improving the rate by just a couple of points.

So how can you juice this up?

The very first thing you need is data. Just like everything else you need to understand the flow and then you need to instrument the heck out of it before you should be fiddling with it. If you haven't already got this in place then stop reading now and get on with it - the rest of this is useless without insight.

Separate your registration process out in your head; this will help you understand and instrument it. Think through all the steps in your funnel make it as granular as possible - this gives you much more insight into where potential customers drop out and much more flexibility in how you modify the process in response. Crudely; if you're thinking visitor -> credentials -> personal details -> checkout then you should be thinking visitor -> credentials -> name and email address -> shipping address -> checkout. This doesn't necessarily mean more pages or more data for customers to enter (in fact the goal is the opposite) but it does mean than you can count these things individually to see exactly what bit of information your potential customers get reluctant about handing over and it allows you to stage things a little more (see below).

The general principle is to put the fewest barriers between your potential customers and your product as possible. This means looking at what information you're asking for, when you're asking for it, and how it's captured. In some businesses there can be constraints which have to be acknowledged; with a product which is regulated or controlled in some way there may be some external rules which can reduce your options - such as the requirement for identity verification or credit checking - but the keyword here is reduce. The registration pipeline in these types of business is often poorly managed for this reason but it doesn't always need to be; take the time to understand the details of your governance and you will usually find that you have more flexibility than you assumed. For example you might need to verify your customers identity somehow before allowing them to make a withdrawal but does that have to stop you from granting them access to the rest of your product entirely?

Nice segue into staging. One of my favorite techniques in businesses with more complex registration requirements is to focus on capturing the absolute bare minimum during an initial signup (usernames, email addresses, passwords, etc) and capturing the rest 'just in time' in line with access to features. For example; why not delay asking for a physical address until the first time a user reaches the checkout? Why not ask for payment information just before their first purchase? You don't need that data until those events and having to enter it all up front can be prohibitive especially when potential customers might not be 100% sure that you're the service they want. Also think carefully about when you want to trigger the 1st step of the registration process - there can be benefit to letting potential customers play with a little bit more of your product before hitting them up for some credentials (but you have to balance this with less complete early contact information and the risk of taking it too far and having fewer accounts because the majority of the value is available anonymously).

Usability is an important aspect here. Considering the number of screens in your signup process, the number of fields, and even the phraseology of the questions can improve your conversion rates.

Also consider the order that you're capturing information in; if you start with email addresses and you're automatically storing fields as they're completed - a highly recommended Ajax trick - then you have some actionable data to go with your telemetry. You might want to use it for a follow-up contact to see if you can claw a customer back from the edge of registration oblivion.

Another consideration when you're designing your forms is to keep in mind the sort of information users are likely to have close at hand when signing up for your site. The more esoteric (in day-to-day terms) the information you ask for is the higher the chances that a potential customer will need to go elsewhere to retrieve it during the signup process. Data always shows that anytime a customer leaves the page(s) for any reason the changes that they'll return and complete them drop through the floor. What might you not even need users to enter at all? You can pick up things like language and locale from the browser and tools like geo-IP; adjusting a probably-correct field from a list of common choices is far preferable to data entry.

While we're on forms; short and sweet is the way to go. Whenever you have to go 'below the fold' then you're better off splitting the form into multiple pages (otherwise it appears too daunting) and, when you're going across multiple pages, then label the progress. A simple 'page X of Y' can work wonders setting expectations.

And finally, if there is a relevant infrastructure for your line of business (meaning that you can trust it and that a portion of your users are likely to be members) you can consider using an identity service, such as OpenID, to implement a kind of 1-click signup and then just capture the additional information you need to operate the account.

Change it and see what happens - after all you're collecting all the data you need, right? If you're crafty enough then you can do some A/B testing with alternate pages or different text. As long as you're measuring the process you will really quickly find out what's better and what's worse.

Adding a feature dimension to planning boards

2010-10-27T20:54:00.001+01:00

Planning boards – sometimes called task boards – are a great way to make work visible, facility teamwork, and surface impediments. The work required for a given iteration of software is broken down into a collection of individually owned tasks and then, as work progresses, we watch those tasks migrate towards the right hand side with a sense of satisfaction.

This is unprecedented transparency and one of the reasons why I love it, but it does have one small shortcoming. If you’re not intimately familiar with the work (say, for example, a non-technical stakeholder) then it can be a bit hard to determine meaning. Yes things are moving along, but what really are those things? Of the 4 stories we’re doing this sprint, which business feature does this progress (or impediment) relate to? It looks almost all done but are we saying that we’ve done 3 of our features and we’re working on the 4th or that we have we done 80% of each of them?

We do want the business to be engaged and to participate in the process so we ought to try and make it interpretable for them.

Some of my guys recently started adding a 2nd dimension to their planning boards; features. By adding a horizontal view which groups tasks by the requirement they relate to as well as the usual vertical view which groups tasks by their status (not started, done, etc) you get an at-a-glance view of the progress of each piece of business-relevant functionality, rather than just a broad sense of things happening.

Even I find this easier to consume! Rock on you little geniuses (you know who you are).

MVP in the Enterprise

2010-10-04T20:34:00.001+01:00

Minimum Viable Product (MVP) is a well understood concept in startups. It is all about distilling your features down to the barest of essentials – those things which really make your product your product - in order to get something out rapidly and be able to quickly iterate guided by feedback. From working at both ends of the startup-to-mature-business spectrum, it occurs to me that the enterprise could learn a thing or two from its lighter-weight cousins.

To broadly generalise the circumstances I’ve seen:

We’re pretty good at this MVP thing in early stage startups. Mostly because, as we plan an iteration, we are fully aware that we might not be around for another iteration so whatever goes into this one has to count (in fact it might count for everything). We think lean. We’re typically good at serving our customer’s most immediate and pressing needs.

In the enterprise we have the luxury of working with the confidence that, even if the immediate next set of features don’t quite hit the sweet spot, we’ll be around for many more iterations yet before times start to get tough. We think deep and wide. We’re typically good at serving a big strategic master plan.

Being able to take a longer term view of a roadmap is definitely an advantage and having to spend time worrying about things like scalability is a nice problem to have, but this doesn’t mean that we can’t also think lean. In fact some of the most value I’ve added has been in the amount of work I haven’t done...

It can work in big business.

Recently I looked at a system to capture and analyse certain customer activity in order to get an earlier prediction of lifetime value and make data-driven cross selling decisions (which ought to push up average revenue a notch) and, on the surface, that seems like it’s worth the pretty hefty licensing, integration project, and infrastructure costs. We started off modelling it and running the numbers; at first synthetically and later with real data. What we learned through this early prototyping is that those features which were initially so attractive made very little difference to our business, however our little robot was uncannily good at identifying fraudulent activity. So we switched the direction of the project, chose an alternative platform, saved a bunch of money and got some functionality which genuinely benefitted us. Sometimes you know what the real benefits are going to be up front, sometimes you surprise yourself along the way.

We’re also planning to introduce a pay-as-you-go commercial model for our feed products. When you add up a metering system to watch client consumption, a dynamic pricing system to set tariffs, a billing engine to produce the required invoices, and some reporting tools to keep an eye on its performance it starts to become a fairly significant undertaking. Will customers like it? Will it work like we hope (add top-up revenues to committed subscriptions) or like we fear (cannibalise commitment for shorter term hits)? The best way forward in these circumstances is to ask what is the absolute minimum amount of work I can do to see if this works? As it turns out we could do a some basic instrumentation (to get simplified metering), use some static pricing, and do the billing manually. Net result? A much smaller piece of work – useable in production with real customers – demonstrating how successful it’s going to be. Now we can build on this with further iterations until we have the fully-featured and refined version we first dreamed up.

There are many more examples, some of which have worked out just fine – so we’ve kept building on them until they were feature complete, fully automated, and enterprise quality. Others have failed – so we’ve cut our losses (fail fast fail cheap; writing off a few thousand not a few million) and moved onto something different which did turn out to be a winner.

Just because we have the budget and the appetite to do the whole shooting match right away doesn’t always mean we ought to.

The obvious risk here is becoming too short sighted with your plans. The secret to making this work is to imagine big but plan small – keep a long term roadmap and have a clear vision of where you want all your products to get to, but take small, tightly scoped steps along that path. Stop and evaluate frequently, adjust big picture where necessary, rinse and repeat.

Mechanical Sympathy

2010-09-20T09:54:00.002+01:00

In the world of motorsports ‘mechanical sympathy’ is the measure of a driver’s impact on their vehicle. As well as fuel economy it is probably well understood by most people that your driving habits do have some impact on how long your car is likely to last, how often it needs attention, and how quickly the parts will wear out. The world of professional drivers is no different, except that excess punishment of the car can mean that vital few more seconds spent in the pit lane – which can cost a racing team a lot more than just this year’s annual service coming up sooner.

Mechanical sympathy attempts to quantify that impact and allows teams to acknowledge it and work on balancing car wear against squeezing that last little bit extra out of it.

I did a motorsports driving course a while back (number of formula 1 events participated in since training: 0 – number of speeding tickets around town since training: 1) and was mercilessly rated in this category. I guess that explains the frequency of servicing that my long-suffering saloon seems to require…

As an outsider to the world of motorsport the importance of certain metrics is immediately obvious – lap times or time spent under power etc – but why measure drivers on something mechanical? These guys don’t fix their own engines, swap out their own gearboxes, or tune their own brakes; that’s someone else in the team. But, as drivers, they do have a direct impact on how often this needs done, how severe it’s going to be, and how expensive it will be to do – and that has a direct impact on winning and losing.

So why don’t we measure the non-IT people in our businesses on their impact on technology? I’d like to introduce the concept of technical sympathy.

I think that people in our businesses should have at least a small part of their KPIs relate to technology. We’re not expecting them to be developers or sysadmins – in the same way that drivers aren’t expected to be mechanics – but we have to acknowledge that they do have an impact on this stuff (probably more than they realise).

Good stakeholdership can make the difference between a well-run project hitting time and budget and a chaotic piece of work drifting on endlessly. Considerate use of systems can avoid the need for expensive environmental controls and the wasting of shared resources such as disks and tapes. The costs and turnaround times of support can be dramatically different with more personal responsibility and better use of self-service at the desktop.

In almost every company I have worked in we’ve measured engineers on business knowledge and company performance to some degree, yet we never seem to have thought of the reverse even though business behaviours can have a profound effect on the cost and quality of technology.

The Product Management Boundary

2010-09-15T16:39:00.001+01:00

Talking to a few of my industry peers (web CTOs and CIOs) about what they do, and to a few CEOs about their expectations, something that’s becoming clear is that we’re increasingly being expected to know what to do, not just how to do it.

Traditional IT uses that age old you-give-us-requirements–we-give-you-back-[mostly]-matching-systems paradigm or variations on that theme. Something we do would typically have a sponsor who dreams up a course of action and a handful of stakeholders who detail it out. What you call those people will depend on whether you think you’re doing agile, RUP, waterfall, etc but typically you’d place them in ‘the business’ rather than technology.

That seems to be changing.

Less and less are we being handed requirements documents or project specs and then going off and doing work to order. Now we’re being asked things like; how do we reach new users? What things can we do to increase wallet share? Should we be doing something with social media? How are we going to internationalize this product? Yeah, that’s right, we’re finally dealing in problems not in todo lists, and - in a trend which is gaining popularity - product management is increasingly being based inside the technical delivery teams.

The days of a business giving us defined work and us delivering projects against it are going to come to an end. A pessimist might suggest that this is ‘the business’ escaping responsibility for defining it’s future. I’d simply argue that technology is an integral part of any modern business, not something else extra and external, and since we’re all part of the same ‘business’ anyway then why shouldn’t we take our fair share of determining the strategy?

I think this is pretty awesome and, in some cases, overdue – after all, don’t we want a bigger influence over what the future of the business looks like and how we get there? Do it CxOs!

The Abstraction Trap

2010-08-31T14:36:00.001+01:00

Abstractions exist so that we can take shortcuts when building stuff. They readily implement some of the messy necessities behind things like, for example, network access in distributed systems – but too often they’re used as an alternative to in-depth understanding rather than a reusable way to render a system.

Staying with our distributed system example; understanding the latency between nodes, synchronisation and blocking (and their impact on concurrent work), transport overhead and reliability, and what the behaviour will be if you don’t hear back from the remote service are all critical to the success of a complex project. Without having to explicitly program these things it’s too easy to just drop something into a pipe or throw it at a web service and merrily continue on with life having never even considered those neatly masked complexities. Until go live.

It’s the difference between something that functions on the bench and something that will work reliably in real life’s volatile conditions.

On the infrastructure side I see the same pattern emerging with virtualisation – we’ve now got this nifty and easy to use platform which takes us another step further away from the metal. On the plus side we can stamp out more nodes really quickly and move instances of a server around from hardware to hardware to accommodate growth and failure, but on the minus side correlating what’s happening on pretend CPUs with a real CPU and oversubscribing hosts makes capacity planning a degree harder.

All these environments are designed to abstract things away from us so that we don’t have to worry about them or recreate them every time – but that’s not the same as saying we no longer need to understand the basics of what happens under the hood.

You’ll always have better product when your engineers have the low level knowledge (primitives and patterns) needed to design and build systems and understand how they will behave. That’s different to simply knowing how to drive the tools.

5 Less Obvious Leadership Qualities

2010-08-13T15:34:00.001+01:00

There is so much material out there on leadership and management – the basics of the topic are generally well covered and so I’d like to add some of the less obvious ingredients to the mixture:

1 - Do what needs done, not what you’re familiar with

When under pressure or dealing with unfamiliar situations people will often reach for the tools and techniques that they’re comfortable with – regardless of whether or not those things will have an impact. We have to avoid falling into this trap and look at what the circumstances and the data are telling us needs to be done. Those are the actions that will turn a situation around – even if they’re a little foreign and uncomfortable to begin with.

2 – Don’t get swept along with the tide

Change is often hard because most organisations (especially large ones) are great big machines that are good at assimilating new parts and reluctant to part with old habits. Making a difference can sometimes feel like the only person rowing south on a boat full of people rowing north – it’s hard work and you have to be strong and you have to expect it to take some time. Organisational culture pop quiz; what’s the street address of Microsoft’s headquarters? One Microsoft Way. Is there something there beyond the literal? Try changing the culture there and then you tell me…

3 - Create the right environment

Put a piece of paper with some questions on it face down on a table and, with no further rules of instructions, ask a group of people to turn over the pages and answer the questions. They’ll all do it silently and individually and studiously avoid even the slightest peek at each others papers (the honest ones anyway). Take that same sheet with the same questions and place it face down on the floor in front of chairs arranged in a circle and, again with no further rules or instructions, you’ll find that people will discuss the questions openly and compare answers and debate choices. I’ve tried this experiment myself and it goes down exactly like that; but why?

It’s because we’re all programmed to behave in certain ways in certain environments and those responses are sometimes so deeply wired in that they’re automatic. You can fight this all you like but it is just the human condition. As a leader you are better off acknowledging this universal constant and turning it to your advantage by creating the environment which produces the behaviours you are trying to bring out.

4 – Deal with opportunities first and problems second

Why are humans so naturally drawn to problems and so blind to opportunity? Perhaps it’s because problems are right here and now and in our faces demanding our attention and opportunities are off in the distance somewhere, shy and subtle and just waiting to for us to make the first move. It’s easy to get sucked down into the issues of the present – and in any case there are usually a multitude of people already doing this is most companies - but the most successful organisations I know didn’t get where they are today because their teams of execs sat around dealing with day to day issues. You can’t ignore problems – they won’t just go away – but you have to accept that your big leaps forward will come from your ability to see opportunity rather than fix something broken.

5 – Do what serves the outcome, not what serves yourself

Obvious on the surface however I mean it in a more intimate and everyday way. Do I get disappointed about stuff? You bet. Do I feel mad when some things go wrong? I’m only human. The difference between good leadership and average leadership isn’t what happens and how you feel about it, its how you respond to those events. Would I like to go nuts and shout at a few people from time to time? You bet. While that might make me feel better I have to ask myself if it will improve the situation and help the rest of those involved cope – whatever does that is the response I owe the team.

6 - Use role power sparingly

Reliance on role power is lazy management and breeds unengaged, dispassionate, dependent teams. People always perform better – and add more value beyond the basics of their role – when they are bought into what they’re doing, understand the value and the part they play, and believe in the vision and the goals. That’s harder work than simply telling people what to do (you have to invest in sharing a vision and collaboratively coming up with plans that the whole team can feel that they own and identify with) but then again we’re here to do what’s effective, not what’s easy.

7 - Make decisions not choices

When faced with options most people can choose between ones they favour more and ones they favour less. But making a decision is making a choice and then seeing to it being carried it out. Drucker’s definition of a decision is that a choice has been made and then:

someone has been made accountable for it’s implementation
a deadline has been established
people involved or affected have been communicated with

When making decisions what really matters is that you recognise that your role as an executive is to create the necessary action rather than just pick from a menu.

I said I would share 5 of the less obvious leadership lessons I’ve learned over the years and the more astute reader will note that I have given 7. Overdelivery – that’s the final lesson!

Architecture in Agile

2010-07-27T16:31:00.001+01:00

When a team first starts practicing agile a question that usually comes up fairly early is where architects fit in. In a set of processes which usually considers ‘big-up-front requirements’ it’s nemesis, it can be a startling realisation that we (the engineering side) might have our own version of that. Architecture tends to be associated with big-up-front specs and, if you’re not doing any of that big-up-front stuff, then where does the role of an architect fit in?

The first step towards finding your answer to that question is to understand what architecture actually is:

Architecture isn’t something separate and distinct from your projects that you can either choose to indulge in or choose to abstain from – architecture is simply the shape of what you produce. Everything piece of software in the world has architecture, the only question is whether it was done by design or by accident. The question is essentially whether you drive it explicitly and control your technical destiny or whether you leave it to chance and hope for the best. Driving it would be the job of the architect.

To make a slightly controversial statement, architecture by itself has no value. But architecture, as defined as ‘the correct technology platform meeting the current and future functional and non-functional needs of the business’ is worth everything that you can put into it. I believe in architecture.

Architecture must be explicitly handled – lest it get away from you and leave you with unsustainable systems – but that’s not the same as saying that you must have an architect. You simply need to have the responsibility for the shape of the system (the technical decisions) clearly understood. Why not devolve the responsibility? Something common to most agile approaches is the empowerment of the teams to self-organise and take control of the destiny of their own product. Architecture is no different and, where you can identify neat boundaries between different products owned by different teams, this type of ‘product architecture’ works very well with minimal enterprise oversight. Less distinct separation between systems generally = more enterprise oversight.

Somewhat counter to the freshman agilist’s intuition, you do need to make some up-front decisions in most projects. Agile is about making sure you know enough about something to be effective by when you need to know it – it’s not about knowing nothing. The trick to handling technical designs is knowing what those up-front decisions should be and what’s better handled as part of how the project evolves. Unfortunately this judgement takes some experience to develop, but a good guideline is to stick to things which can’t be easily changed later – such as schemas, APIs, and namespace.

If you’re a formally nominated architect in an agile environment my advice is stick to patterns and delegate implementation decisions. When you are dealing with a flow of work and you can’t initially nail down the complete set of requirements in buildable detail – the business case for agile – then you have to let your vision for the product, and the size and nature of the commercial opportunity it supports, help you pick out the right patterns. Patterns are the best currency for an architect to deal in if he has a team of smart developers keen to take some ownership; the ‘here is a design doc, build this’ approach becomes a constraint rather than a support.

In more larger or more complex agile environments – with multiple development teams and multiple products – an architects role moves into governance; the design of common elements and the way in which products integrate and interact. What we might call enterprise architecture I guess. The goal here is to put together a framework which promotes self-contained and distinct products, giving each team their own jurisdiction while still ensuring smooth overall performance of the whole estate. In my experience this maxes out the total delivery throughput of the whole organisation and gives the satisfaction of ownership to the doers.

If you’re new to architecture (giving or receiving!) and need to make sense of all this then I’d recommend joining IASA because a key part of our mission there is bringing clarity to what an architect really does for a team, a product, and a business.

Finding Talent Part 2

2010-07-08T16:48:00.002+01:00

Continuing on from my last post here is the balance of the talk:

Culture is critical to success

Culture is meaningful in 2 ways; it’s the environment that allows high performance people to contribute all they can, and it’s an essential ingredient in attracting them in the first place. Like most things I’ve touched on here, culture is a massive topic all by itself – how do you even define it? The freedom to work to the job and not to the clock, a casual office atmosphere, an appreciation of the value of their innovations, and the ability to try out new tools and processes are just a handful of the things I’d lump into this category, however if you want to distil it right down to it’s elemental for I’ve never heard better than Dan Pink’s:

AMP – Autonomy, Mastery, Purpose

Autonomy is the freedom to choose what to do, based on an overarching understanding of goals and objectives, rather than having work broken down and fed in as proscribed steps. It’s a ‘give me a problem and let me get on with it’ attitude.

Mastery is the desire to get better at something; to develop, improve, and perfect what you do for no other reason than that itself. It’s a strong motivator for high performance people; they constantly want to track progress and do better next time.

Purpose is the need to connect with a greater mission that they find inspiring. Traditionally you could say that companies are formed to extract profits from a market – and that’s always going to be true – but what will inspire high performers is changing the nature of friendship rather than building a social networking site or redefining how education is delivered instead of putting infrastructure in a school; and sure, everyone expects to, and intends to, make some money along the way.

Be available

Most high performance people I’ve worked with value time with their leadership – whether you’re their boss or their bosses boss etc, they want to tell you what they’re doing and they want to know what your thoughts, priorities, and plans are; the longer term the better. It’s how they feed their ‘autonomy’ part of AMP and it’s how you make sure it’s going in the right direction.

Proliferate

In most organizations I’ve worked the truly high performance teams are small pockets of greatness amongst a wider – shall we say more diverse – population. Out of all those places I could tell which ones were headed for success and which ones were headed for mediocrity (regardless of the industry or the product) because in the ones headed for success the wider organization picked up behaviours and practices from the higher performing – and more successful – teams and then everyone lifted their game. In those destined for mediocrity people resisted change regardless of the practical demonstration of it’s benefits being right under their noses. As managers of high performance people we can help this process in a number of ways; we have to recognize that most people (good or bad) don’t like being told they’re doing it wrong or that your guys know better, but what you can do is encourage external interest in what your guys are doing or how they’re doing it and, if one them is interested in spending time with (or moving into) another team, make sure you do everything you can to make space for that to happen. Everyone grows that way.

Know your role

It might seem a little bland or obvious but one thing high performance people will know is what they expect from you, as their manager, and if there is one thing that’s respected it is competence and solid leadership. These kind of guys don’t like working for weak or indecisive management, they take comfort from knowing that you know what you’re doing and you’re doing it like you mean it.

Expectations can be different internationally

This one is particularly true in the IT industry – given how portable our skills are, the way our technical vocabulary overcomes language barriers, and how much offshoring is being done. My experience setting up a develop centre in Romania taught me that, while most of this stuff is usually true, it should be used as a theme rather than a fixed set of rules to use in every circumstance. For example shoulder surfing with one of your developers while they step you through their code could be considered unwelcome micromanagement in one setting but essential interest and support in the other. There is no substitute for taking the time to appreciate what ever individual’s unique needs are. As leaders we’re in the business of making our guys as effective as they can be - whatever their interpretation of that is.

HR still wont help you!

I guess it’s starting to sound like I have a real downer on HR, but I really don’t. I do have a downer on bureaucracy (and even more of a downer on large scale process and paperwork when it doesn’t contribute to the performance of a team or individual) and I have a downer on efficiency at the expense of effectiveness. My point here is simply that no one else but you can or will take ultimate responsibility for the performance of your teams. HR is there to help you, but remember that they’ll never stand there with you and take a beating for things not working out in your team and I don’t know a single executive that would be satisfied with the excuse ‘I applied all the management steps HR specified’…

Set the bar high

It should be difficult to get a job in a high performance team because the rest of the people in the team will have a high standard and will not thank you for lowering the average. High performance people like to work with other high performance people – they’ll always be looking at what they can learn from those around them.

Mix it up

As much as high performance people are always looking at how their peers can help them develop they also enjoy helping others – with the potential but without the knowledge – to develop. For this to work you need to get good at spotting good quality high potential people rather than just those who have all the skills and knowledge right now today. These guys are usually good long term investments too because they’ll soak up knowledge like a sponge and there will be plenty of progression for them in your team as what they’re capable of doing for you expands.

Spend time on the best not the worst

There are a whole lot of nice sound bytes like ‘the squeaky door gets the oil’ etc to justify loading your time towards the people in your team which need the most support, and for some reason human beings are always more drawn towards problems than opportunities, however consider that the opposite bias might be better. How about spending the biggest share of your time with those in your team who will benefit most from it? Don’t allow your weakest performers to be a huge sink for your (highly limited) personal bandwidth and, as some pretty smart guys I know always say, ask yourself how much benefit the organization gets out of 10% more performance from your best performer vs. 10% more performance from your worst performer.

Don’t hoard

Weird point to close on given that this was the ‘retention’ section of the talk, but I think that too many managers focus on retention for retentions sake (which can be especially dangerous when it becomes a KPI). I think the focus has to be on establishing a win-win relationship with smart people which will result in a lot of value for both the organizations mission and the employees personal development and professional growth. Recognizing right from day 1 that this arrangement has a finite lifetime is simply being realistic. I always feel proud and happy when people that work with me move on to something else – in the same organisation or elsewhere - and I take some satisfaction in the knowledge that it’s a step forward for them and that I’ve helped them get there. Besides, I have always found that when people can see how the stuff they’re doing for you links in to the bigger plan they have for their lives they stick around longer because they see how working for you actually contributes to achieving their long term goals rather than preventing them from moving forward.

I originally agreed to do this talk because so many organisations have a very traditional approach to recruitment and management and these approaches are just no longer relevant when you’re dealing with complex technology challenges requiring talented engineers. It’d be a better profession to be in – for both the people on the ground and the companies who benefit from their expertise – if more of us did this stuff differently. You can find the original slides here.

Finding Talent Part 1

2010-07-08T16:44:00.001+01:00

Here’s a synopsis of my recent talk on recruiting high performance teams (you can find the slides here) which was made famous at this year’s IT Directors Conference.

The talk breaks down to 3 areas; recruiting, retaining, and running. The emphasis was on recruiting – because it was a talk about finding and hiring talent rather than general management instruction – so I’ll go over recruiting in detail in this post and combine retaining and running in the next:

Keep a bench – you are always recruiting

Any decent manager I know is constantly assessing everyone they meet – socially or professionally – and sorting them into ‘would hire’ and ‘wouldn’t hire’ buckets whether or not they are even currently hiring. This way they can start to build a relationship and get a real life feel for how the person performs outside of the artificial construct of an interview situation and then they have an extant ‘bench’ of talented people that they know of, and might even have been grooming, when positions come up. You see roles filled quicker and cheaper with better quality people.

Attracting talent – what YOU say doesn’t matter

One thing that attracts good people is what other good quality people have to say about working at a certain organization – and anything said by another employee or associate of the company automatically has weight and credibility that the official company line doesn’t carry. People expect YOU to say that your company is a great place to work, and therefore they take your recruitment sites/company spiel with a pinch of salt, but they’ll pay much more attention to what your guys say on social media platforms. So encourage your team to blog about work and the team and the projects.

HR is not going to help you

Now, let me just say that up front that I’m sure this isn’t always the case, but the fact is that the role HR departments take in modern organisations is changing. The focus is now much more frequently on compliance (employment law) and health and safety legislation and in putting in place cost reduction measures around recruitment (enter good old efficiency at the expense of effectiveness). Seldom is an HR department oriented toward courting talent. That said – if you do have HR guys that are all about the talent then you should hug them dearly and make them feel appreciated. They’re getting rarer.

Nobody is better than the wrong body

A common mistake in recruitment – especially under deadline pressure - is compromising on the person your looking for. This can be a tough one to stick to. In the worst of circumstances you’ll have no-one in a role, a business demanding quick progress (and maybe even viewing vacancies as one of the reasons things aren’t moving quicker), and you’ll have seen literally hundreds of CVs. But every single time I have taken on the best person I’ve seen so far – rather than holding out until I meet someone I considered to be the right person for the role – I have regretted it.

Salary is only part of the equation

There is no correlation between the best people I have ever had working for me and the people who have been on the highest salaries. Top talent are much more interested in taking a stake in something they believe in (read: equity), performance linked bonuses, the challenges of the projects, and the culture/working environment. Salary has to be enough to take pay off the table as an issue but the best people I know don’t chase salaries, they chase challenges.

Crazy about technology - not the industry

It’s easy to fall into the trap of hiring engineers that love the company subject matter over engineers that love engineering. Mistake. Your best technologists will understand the commercial forces that drive the business, the market, and the customer in their organisation – and it’s nice to have a passion for the product you’re building – but not at the expense of loving and living the bit you do. That’s code. I’m hiring guys who build world class highly scalable data distribution systems, not a football team.

PLU is nice – but I’d rather have talent

People Like Us (that syndrome of only ever recruiting a very narrow band of like-minded people) is a good way to make sure you end up with a group of people who seldom disagree, but no variety – no alternative points of view, no challenges to the status quo, no new ideas – doesn’t make a high performance team. Lumped into here I’d also underline how important it is to hire people smarter than you are. Not as clever, or less clever so that you can feel nice and superior, but more clever. That’s how things get pushed on and it shouldn’t be feared – it should be expected.

Agencies – surrogate networks

The best way to hire anyone is through your network – that is, people who are known quantities and proven performers who have the right personal qualities (which is why I say keep a bench) but eventually you will have to use agencies anyway because everyone goes to them (which I think is because, historically, everyone goes to them). The secret to dealing with agencies is to remember what you’re doing – buying the value of their network – so don’t accept ones that just keyword-match CVs or phone around on a per-engagement basis. You want to look at how long they know a candidate for before they put them forward, whether they’ve met them in person, and whether they’ve placed them anywhere before.

Paperwork is an artefact of good people

And finally remember that the goal is to get the right people in place, not to have completed the right forms and followed the right steps. Don’t establish an overreliance on artefacts - process is important but not at the expense of results; it exists to help you get the right outcomes and is not an outcome in itself. Not convinced? Test it yourself this way – imagine you’ve brought on board a series of poor performers who haven’t lived up to expectations. Your boss is going to be firmly asking you why (if they’re any good that is). You followed all the right steps and completed all the right forms. Do you think that absolves you of responsibility for those results?

Obviously this talk is aimed at the engineering end and, even then, only for those of us lucky enough to have tough technical challenges and interesting business problems to solve. I appreciate that isn’t everyone but I hope there’s something in there for you anyway. In a few days I’ll post a summary of the retaining and running sections.

Software vs. Infrastructure

2010-06-28T22:16:00.005+01:00

For some reason I’ve been running into a lot of other CIOs and CTOs lately and one topic that keeps coming up was whether we were development or infrastructure focused. The whole time I kept thinking to myself why is this even a question? Why do we feel like we should choose? Or, somewhat more provocatively, why do so many of us only feel like doing half the job?

I concede that there is a ‘background’ element to this. No one I know started off as an IT executive; we were all DBAs or developers or sysadmins first, and that gives us a nice comfortable area of personal expertise we can use as shortcuts but it shouldn’t set our agenda in a leadership role which [in most organisations] encompasses the whole shooting match.

One of the reasons why many organisations suffer the good old fashioned laundry list of regular technical woes (that we could probably all reel off by heart) is because of the interplay between development and infrastructure and a lack of end-to-end oversight over both. And, as the technology leaders, if we don’t understand both sides, keep strongly engaged with them, and create teamwork and cooperation where there has typically been borders and a lack of mutual interest, then who will? There are very few other roles with a remit in both areas and even fewer that should be taking responsibility for them failing to be an end-to-end unit.

There is a whole ‘devops’ kind of meme swelling up these days – and more power to it; I think it is dead on the money.

Devops is, by nature of the organic meme it’s emerging as, poorly defined but to me it is about the recognition that [in most cases] customers care about product – not software or systems – and product is that butter-smooth combination of software and infrastructure finely tuned to work together and operational know-how to keep it running.

It’s also about having the right feedback loops in place between engineers and the real world, so that as your products get used and abused, they become more fit for that very purpose.

What am I going to do today?

2010-06-22T21:12:00.002+01:00

Here I go again – with that ‘time’ spiel – anyone who works with me will be switching off about now; but it’s a good spiel and one that I think it isn’t common enough in the corporate consciousness, so now my readers (both of them) get to endure it. It’s really about scarcity, and recognising the resource which we should prize most dearly yet too often let slip through our fingers.

When asked what their most valuable resources is, most people will say money (the odd few might say people – true in another interpretation of the question). Cash is critical to a small business, as it’s often in sort supply, and critical to a large business, because of increased fiduciary responsibility and reporting requirements. But the thing that you can never replace is time and, no matter how big or cash rich an organisation gets, none of it’s individual members ever have any more of it available to them than anyone else anywhere else. It’s fixed. It’s also a unique type of resource to manage – you can’t save it up for a while so that you can spend more later and if you spend it unwisely, you can’t go back for a refund and try again.

Just how scarce is this time thing? Let’s do some quick maths and see how much of it we actually have available to us this year:

1 - Start with 365 days in 2010 (assuming that my time travel project doesn’t get off the ground)
2 - Most of us average out to 5 days per week, so let’s take away 104 leaving us with 261
3 - Most companies give an average of 22 days annual leave every year so now we’re down to 239
4 - On top of this Her Majesty bestows upon us 9 bank holidays in 2010 leaving us with 230
5 - Sick leave isn’t always all used up but let’s assume a worst case of 10 which puts us at 220

Starting off with 365 days makes a year sound like a long time, but we pretty quickly slashed a staggering 40% off that without really trying. And that’s without any contingency for things like sabbaticals, maternity/paternity leave, jury duty, family emergencies... and we can’t make any more of this ‘time’ stuff. It can’t be bought, borrowed, earned, duplicated, or recycled.

Most of us waste much more of this than we readily accept. Ever sat in a long meeting on a tangential topic with a large number of attendees – some of whom aren’t even really engaged (on blackberrys etc)? Ever wonder what the world might be like if those 10 people each got that 2 hours back? In the corporate world I think time is undervalued because it is a hidden cost (outside of professions which explicitly price their time such as legal advice and chartered accountants) and there are often few controls on how one spends their [the organisation’s] time compared to the controls and various levels of signoff on how one spends the organisations money.

Everyone has key projects to do and those projects only get done by people spending time on them. And, as we just worked out, a year sounds like plenty of time to do all this stuff but we usually end up with a fair bit less time on our hands than we started off with.

Getting what we want done means being aware of the priorities and valuing time enough to spend it deliberately – on the things that are most important to the organisation. I used to be surprised about the sort of things that really successful companies dropped on the floor – genuinely good ideas just not getting attention – and then I realised that they hadn’t been missing opportunities; they were just uncompromising about their priorities, staying focused on their most meaningful things, and fiercely guarding against distractions. That’s how they got really successful in the first place.

A different view on Job Satisfaction

2010-06-14T14:56:00.001+01:00

If you’ve been here before then you will know that I don’t just regurgitate content here unless I think I can add something significant to it, but I guess every rule has an exception. One of my guys sent this around a while ago, and again quite recently (OK I get the point), and I think it’s too good not to share:

Jobs are not meant to satisfy us. Jobs are not animate things that have knowledge of who we are, what we are seeking and what our special needs could be. You may say that I am just making a philosophical statement. To the contrary, I believe that it is the most practical and rewarding way of looking at many things in a professional career. When I see scores of successful people around me, I believe that their achievements are largely because of such a perspective. It also occurs to me that developing this perspective is eventually beneficial in every way possible.

Let me go back a century and tell you a story. My grandfather was a medical practitioner in the Bihar of 1920s. He had a brood of children who were orphaned due to his untimely death. Two of my uncles had just about finished high school when he moved on. Their older brothers could not afford to send them to college. The two had to be gainfully employed, somehow, as soon as possible. They were taken to Tata Steel, an hour away from where they lived. Tata Steel and the government of Bihar were the only two employers you could think of in a five-hundred mile radius of my uncles’ hometown. The possible work one could get at Tata Steel was that of a technologist- engineer or of a manual worker. So, what could be done with the two boys with their high school qualifications? They were neither fish nor fowl. “Take them to the lab,’’ someone said. A German technician who ran the place was looking for a few hands. The burly German took a hard look at the two. Then he showed them a broom standing at one corner of the lab and asked them to sweep the floor. By the end of day, one of the two just ran away. To him, it was too much to handle. The one who stayed back retired as a chief supervisor of Tata Steel. The difference between the two? The one that stayed on was not trying to seek ‘job satisfaction’. Instead, he focused on satisfying the job.

The more prosperous the industry, the higher the number of people looking for this elusive thing called ‘job satisfaction’. Similarly, the more qualified some people are, the higher is their need for ‘job satisfaction’. Sometimes, it is as elusive as seeking ‘true love’. There are times when we get lucky deservedly or otherwise. But we also get used to it and conclude that it is the responsibility of the organization to maintain a continuous supply of job satisfaction.
Whenever I think of job satisfaction, I remember all those who have to work at night—policemen, airline pilots, nurses and doctors, ambulance drivers and hotel staff, and of course the sentinel of the snow and the desert and the mountains. Do their jobs ‘satisfy’ these people or do these people satisfy the jobs with which they have been entrusted? Are jobs living things that can ever ‘satisfy’ us?

In the corporate world, like any other place, when we open the bonnet and look under it, we find a whole bunch of tough, dirty but strategic tasks that must get done for the bacon to come home. Sometimes, they are so tough and so dirty that they overshadow the strategic nature of the job. So, all such jobs have to be ‘sold’ to prospective incumbents. More they are sold, less buyers they attract. Often, the man who takes up the job is either a loser who has no other choice, or someone who just views it as a transit camp. For many potentially high-performance individuals, a false sense of survival, desire for glamour or just the need for creature comforts make these jobs undesirable. “I would rather be in Kolkata than be posted to Mungher.’’ “I rather have the corporate planning job than be collecting bad debts.’’ Or, consider this one here: “Give me a cerebral job, I do not enjoy handling transactions. ..’’

Few of us ever ask the boss to be rewarded with a tough and dirty job. We only look for the ‘plum’ ones. Yet, there are people, who given a tough and dirty job, make it strategic: they transform the job in unbelievable ways. In a typical career span, there must be at least four such solid stints in one’s life to make the person a solid professional. All the great people I know have been in the trenches for much of their lives, and their inventory of bruises outnumber the commendations they have received. The occasional commendations stay on the wall. It is the bruises that these people carry with pride.

Subroto Bagchi

That can come across a little old fashioned on a quick read, however I think there is a lot to it when you take the time to internalise it. When I think about the times that I’ve loved the most in my career, I can see a strong correlation with the times when I’ve been the most focussed on what needs done and making that happen. Weird how that seems to come back to you in the form of satisfaction…

Entropy

2010-06-09T14:18:00.001+01:00

A great note just came through on the Manager Tools mailing list the other day called ‘Everything Decays’ and I liked it because it explains something I’ve recently been wrestling with in my head much more eloquently than I could hope to put it.

Let me reproduce the first 2 paragraphs here so you can see what I mean (can’t find a link to the original work – will post when I do):

“One of the things we've noticed is that managers like having a solution which solves their problem forever. We suspect you've felt this way. In the rare instance you've found the solution you thought would solve the problem forever, you've probably also discovered it's not true. The problem has come back because the situation changes. The people change. The knowledge changes. The need changes.

The fact is, everything decays. For our technical readers, it's just entropy. At work, every problem, every meeting and every relationship is decaying, right now. Now matter how good a meeting is now, in six months, left to its own devices, it won't exist (the ultimate decay) or it will be a lot less effective. The process that you've been working so hard on, and finally got just right, in six months will either be OBE (Overcome By Events) or will need significant rework. You may not have understood, up till now, why good processes that worked before begin not to work. The answer is, the situation, systems or people changed and the process didn't. Entropy. Everything decays. It's not just YOUR stuff that decays, because you're not a good manager. EVERYBODY'S stuff decays. Always.”

The message goes on to talk about how professionals need to continually assess relevancy; keep looking at what they’re doing and making sure it makes sense as circumstances change and the business and the people in it move on. This resonates on a number of levels including – probably most acutely – the number of managers I still meet who believe there’s some kind of silver bullet out there for their troubles.

What isn’t touched on is the natural entropy you always get in any set of processes or practices even when the environment around them is stable – and the less ‘habitual’ the behaviours are (i.e. cemented through solid and continuous application), the quicker the decay kicks in and old habits creep back in.

tl;dr - there is no substitute for sustained attention and focus, especially during a period of change where some more dedicated coaching can make the difference between lasting change and a blip on the curve.

Technical Debt Credit Rating

2010-04-29T08:29:00.000+01:00

A while back I wrote about how closely technical debt paralleled financial debt and then a short follow-up post with the meta-message that, through the right reductionism, certain things will always cost you an inescapable minimum and the decision that you really have is about where you choose to make that payment (I argued that you either pay through the correct design and build initially, or you pay through unnecessarily increased maintenance costs later).

Outlawing technical debt is as foolhardy as always cutting corners - sometimes the right thing to do is to take a hit to get something out sooner or in a certain shape - the problem is in the repayments; in other words, they often don't exist. And if you don't go back and sort your hacks out early they tend to accumulate in the system and before you know it developing on it is like moving through treacle (and the treacle occasionally catches fire). Everything else you do takes that much longer and is that much more expensive (and those are business metrics) and can have unintended consequences - it's a crippling burden just like its financial counterpart.

I have a rule - no one is allowed to take technical shortcuts through a project [where that = incur some technical debt] without showing where, in the same plan, they go back to do some refactoring so that our long-term assets remain sustainable. We're pretty lucky at Sporting Index; most of our stakeholders value technology and recognize how critical it is to the business, so it's a conversation I rarely have to have - but I know many other companies are less fortunate.

So now I think I have a better idea. It came to me while I was thinking about why, if technical debt is so similar to real debt, it doesn't regulate itself in the same way - and I think that's becuase there is no concept of creditworthiness in technical debt, in other words no assessment of a team's ability to 'repay' a quality compromise (read: loan of functionality to the business ahead of when it otherwise would have been ready) alongside all the other outstanding quality compromises that they already have.

But we might be able to simulate that...

Imagine giving each product manager a 'credit rating' of, say, 3 - meaning they could call for unsustainable technical shortcuts 3 times as they worked with the delivery team to determine their solutions. Once those 3 debt points were used, that product manager would be unable to encourage the team to make further quality compromises until at least one of those was paid back. If you have product managers with a demonstrated track record of revisiting earlier deliveries to address technical debt then why not give them 5? A higher creditworthiness ought to enable an individual to take on more concurrent debt as we can be more confident that they'll come back to service it.

I pretty much guranttee that any product manager's cavalier attitude towards shortcuts would change and they'd suddenly treat technical debt as the instrument it really is and use it only where it mattered materially. That, ladies and gentlemen, is a feedback loop.

The downside is that you'd have to be better at tracking this sort of thing and, like any other method, it'll need some discipline when a product manager with maxed out technical debt points wants just one more shortcut. If you've ever had to maintain a spaghetti codebase (as most of us have at some point) then you'll know that these things need closely policed anyway, so perhaps being forced into formally tracking it isn't so bad.

Nobody Wants Computers

2010-04-28T12:28:00.001+01:00

Well they shouldn't - they should want what computers will do for them.

Many people turn up to conferences or SIG meetings and ask me how virtualisation or a centralised inventory database or VPNs (or pick something) will help their company. Misguided. My response is always the same - what is your business case and what benefits are you expecting? I do get quite upset when I don't get a good answer to that, because it means someone somewhere is cooking up some technology for technology's sake.

Sales functions are excellent at creating a need around a product they sell - great if you're in that business and I used to be - but it is the functionality your organization really needs that should matter. Talk to me about outcomes, goals, what you want to be able to do from a business capability perspective, and then the right technical solution will fall out of that.

Contorting business processes to fit a specific technology or vice versa - i.e. doing it backwards - only ever works when combined with a healthy dose of luck.

The ISO standard for business plans

2010-04-23T08:57:00.002+01:00

If I had a quarter for every time a budding entrepreneur has asked me about what should go into his business plan, then I'd have - let's see - three fifty? I don't know, but put it this way, it's in my top 10 FAQs. My two secrets behind this one are; a) there is no such thing as a standard format or expectation and b) it matters a lot less than you probably think.

Don't agonise over format and presentation - the vital thing is to intimately know your key facts and be able to provide a logical narrative for them. What are your costs? Where does the revenue come from? Who are your customers and how big is the market? How soon will you start bringing in revenue? How is your product different from any competitors (current or future)? What does it cost you to acquire new customers and what is their lifetime value? Have you done this before and who do you plan to work with? Even if you just talk through it you're always better off knowing those things inside and out than having a 200-page glossy document.

And when I say that a business plan matters less than you think it isn't because investors won't want one - they very much will - it's because in the early stages of a new idea what matters most is working product. The best, most detailed, most polished business plan in the world will always be worth much less to a potential backer than a pretty basic, hacked together but functional, beta product.

What kind of idiots are in charge here anyway?

2010-04-19T11:44:00.000+01:00

When I coach people about being effective in organizations something that regularly comes up is their ability to influence decisions made at the senior level. Because (with very few exceptions) most people in most organizations are rational and competent professionals with the organizations best interests at heart, any time they feel like their proposals or suggestions aren't being adopted, it can be easy for them to lose a little faith in senior ranks.

I never like hearing that because the truth is (again with very few exceptions) that most executives in most organizations are also rational and competent professionals with the organizations best interests at heart. So why don't your ideas stack up for them?

My advice is usually to look into these two things:

1 - What don't you know that they do - executives will usually have a much broader view of things and can therefore be privy to other information, plans or constraints elsewhere in the organization, that you may not be. Taking into account the things that might be invisible to you, not adopting your recommendations might well be the most beneficial course of action. In my experience, most executives are more than happy to share this information with reports who show an interest in the wider business, so just ask.

2 - What data aren't you being effective at conveying - sometimes the message just isn't getting through and therefore your decision makers don't fully understand what you're proposing and why. It happens to me; I see a lot of proposals for a lot of things (very few ill-conceived) and if the key facts aren't well summarised and the options clear, I am not able to make an effective decision. Again, the solution starts with communication. Ask why, use questioning techniques to check for understanding, find out how your audience wants things presented. I am yet to meet an executive who won't be willing to give that feedback - after all, it helps you both.

Answering these two questions usually gives people the insight they're missing to clear things up and, if it doesn't, then I guess it's always possible that you might be one of those few exceptions...

Things change because people change

2010-04-14T09:27:00.000+01:00

Occasionally I get a hard time for spending such a large part of my time with my teams, with the individuals, out on the floor in their work instead of in my office with 'my' work. Occasionally that's also fair - I have a different job than the people who report to me, and the the people who report to them, and that comes with different deliverables.

On the other hand a big part of my job is the performance of my department, the quality of their deliverables, and the continuous improvement of both of these.

New processes, technology changes, new policies, culture and structure are all part of it, but ultimately nothing is ever any different until individuals change their day-to-day actions and behaviours. You will never create lasting change by making rules and proclamations from behind a desk, or just by telling people to 'do better'. Lasting change only comes from sustained leadership focus; coaching and repetition and doing things differently together. When the man on the ground changes then overall results change.

Oh and don't forget my golden rule no. 22867: paperwork is an artifact of effective work - effective work is not an artifact of paperwork.

Let's Ban Agile

2010-04-05T15:04:00.001+01:00

Bet that made you look.

Whenever I'm dealing with the wider business and I come across confusion about (or resistance to) agile practices I typically find that, once the few genuine issues are swept up, much of what's left comes down to semantics. And fair enough too - the extensive metadata that has built up around engineering processes is intimidating.

If we put all that aside and just talk about the basic practicalities of doing projects, you are pretty much just looking at an inventory of requirements that must be turned into an inventory of working product. To do that, distinct individuals need to execute specific tasks within certain time constraints (or, more colloquially, who does what by when). From there, we build up some rules to make sure everyone can cooperate effectively around those basics, and it is these rules we'd tend to call our development process/project methodology.

It isn't any harder than that. But, for some reason, we like to make it sound difficult through a plethora of technical terminology.

Instead of iterations, why don't we just talk about tackling big projects a bit at a time? [In fact, I'm yet to discover a better way of solving huge problems than breaking them down into a collection of smaller ones] Instead of backlogs, let's just have a prioritised list of requirements. Instead of standups, let's just have regular and frequent meetings to keep everyone on the same page. Instead of sprint planning sessions why don't we all get together to iron out a detailed plan before we commit to our next chunk of work?

I guess what I really want to ban is unnecessary jargon - at least when dealing with the non-technical contingent of the business. I think we should use common everyday language to describe what are, ultimately, practical and common sense activities.

Just because we're solving complex and difficult problems doesn't mean we need to use complex and difficult processes - in fact, one could make a case for quite the opposite.

The Paradox of Freemium

2010-03-31T16:25:00.001+01:00

One of the businesses I advise recently flipped over to a freemium model and – as expected – saw a step change in paying subscribers. It does lead to the question; in a world where we typically make money by making things and then selling them to others, how did we decide that giving valuable things away made commercial sense?

Firstly, for those who may have spent the last few years in a cave, I broadly define freemium as giving away something of substantial value along with an associated upgrade/expansion path which returns benefit to the organisation.

This is already commonplace. So many of the everyday tools I use have an element of freemium to their business model; google apps, zen agile, bitbucket, linkedin, mockingbird, a number of podcasts (which lead to paid-for books or other products), and of course almost every iPhone app I see these days has a fairly comprehensive ‘lite’ version.

For a long time we’ve given away tightly limited trial versions, exposing a very small subset of the functionality (or usage time) in order to coax users into a purchase to unlock the rest, and freemium is almost a reversal of this. Give away quite a rich set of functionality with no expectations, and you’ll pick up a group of buyers for that additional stuff on top.

Making this happen comes down to product design; you have to make sure that you hold back something(s) compelling enough to encourage purchases while making sure that your base product is functional and complete enough to be of significant utility on its own. There are so many ways to slice this, and sometimes the differences don’t even need to be rendered – you might simply impose several distinct licensing terms (free for academic or home use for example).

In one of my all-time favourite Ted Talks, Dan Pink says:

"The mid 1990s, Microsoft started an encyclopedia called Encarta. They had deployed all the right incentives. All the right incentives. They paid professionals to write and edit thousands of articles. Well compensated managers oversaw the whole thing to make sure it came in on budget and on time. A few years later another encyclopedia got started. Different model, right? Do it for fun. No one gets paid a cent, or a Euro or a Yen. Do it because you like to do it.

Now if you had, just 10 years ago, if you had gone to an economist, anywhere, And said, "Hey, I've got these two different models for creating an encyclopedia. If they went head to head, who would win?" 10 years ago you could not have found a single sober economist anywhere on planet Earth, who would have predicted the Wikipedia model."

I think those same sober economists would probably have said much the same thing about freemium, and probably much more recently. Getting customers to pay for your product by first giving most of it away wasn’t common sense – and maybe it still isn’t – but you certainly can’t ignore the evidence that it works.

Encouraging Profitable Usage

2010-03-26T12:30:00.003Z

A couple of weeks ago a friend sent me this article from a mutual acquaintance at CCP. It's all about the continuous struggle that the MMO guys have keeping their in-game economies balanced amongst all the real-world trading of items and game currency.

I was particularly interested in the stats near the end - in a crackdown on gold farming they managed to drop CPU utilisation 30% through a 2% drop in players (banned accounts who were violating these terms). To paraphrase that; 2% of the player base [revenues] were creating 30% of the load [costs] and that my friends, is what I call unprofitable usage of a product.

When you read that last sentence it makes perfect sense, however so few of us on the web think this way. How often have you looked up your top workload creators (page impressions, transactions) and your top revenue creators (purchasers, subscribers) and compared the two? How sure are you that your costliest customers - that you are probably supporting a collection of CPUs to service - are not marginal or worse in terms of value to your organisation? With sites more data rich than ever (read: more back end work per view) and screen-scraping a more widespread way to acquire content it's worth working to discourage this behaviour or at least make conscious decisions to allow it.

This thinking should be taken all the way back to product design - on the web we tend to emphasise users/visitor/subscribers without really thinking through what constitutes profitable usage.

It's not just about nailing people who are out to exploit you either - as a long time data provider my biggest issues have usually been well-meaning customers with lazy integrations. If you're going to build a client on top of a web API it's quick and easy to poll the bejesus out of it round the clock and then react to data in the responses rather than, for example, get a schedule of 'interesting' data and dynamically set up threads pick up smaller changes to discreet items on separate and appropriate timers. Take trades for example; why hammer a Hang Seng Index Options feed after 4:15pm? Or how about our feed - do you need 5 prices a second for this weekend's football fixtures on Tuesday or would hourly pre-match odds and squad changes be more than sufficient that far out? A slightly trickier initial integration but a better and cheaper product (for both parties) thereafter.

And finally I've always found it more effective to encourage good behaviour than discourage bad behaviour. Sometimes this can be as simple as an SLA which guarantees a superb latency up to a certain number of requests and creeps out in bands beyond that. Other times this can be purely commercial - why not extend better pricing to customers who genuinely cost you less to service or a tiered revenue share to those who contribute more to the partnership?

Reporting & Agile

2010-03-22T09:39:00.000Z

When I talk to people about their experiences with agile development I frequently hear - from the business side - that once all the bureaucracy has disappeared (which no one tends to see as a bad thing), they struggle a bit with the visibility of projects and the reporting of progress.

This view is usually 100% in contradiction with what I hear from the same organisation's development practice - who will usually tell me that visibility has never been greater and that they have an unprecedented degree of transparency and openness with their business partners.

Can they both be right?

When we, as engineers, make statements like the above we're usually referring to the public nature of our SCRUM/XP/etc artifacts - product backlogs and burndown charts are open to anyone in real time, requirements, estimates, and plans tend to be kept in readily accessible web tools, and of course most events in the process (such as daily standups and each iteration's demo) are public forums. What else do you need to have the most detailed, accurate, up to date view of every piece of work anytime you want it?

This is all true, but where we tend to go wrong is in underestimating the underlying complexity of something that inherently makes sense to us. There is actually a significant degree of interpretation - based on technical knowledge built up over years - necessary to answer a simple question such as "how are things going?" through agile artifacts alone; we just tend to be pretty good at doing it in our heads as we go.

Here's a simple example to underline my point. O2 has a pretty nifty self-service portal, and I can use it at any time of the day or night to see what my current mobile phone bill is and how far through my monthly quota of data and text messages I am. It works well for me because they have taken some underlying data about my account, put it through some pre-processing to provide some plain-English conclusions, and presented it on the web conveniently formatted so that I can interpret its meaning. Instead, imagine if they did transparency and self-service as we sometimes do in software delivery and just told me that all the data is there - the same artifacts that they use internally are open to me - and I must draw my own conclusions from some raw material. I guess I would install toad, connect up to their DB, select my rows from the billing table, join that with my rows from the metering table, and dump them into a view that showed me a current bill value along with my unused quota of texts etc. A few of us out there would probably quite like this alternative but, nonetheless, we have to agree that it is unlikely to be a level of service that the majority of O2's customers are going to be happy with let alone know how to use.

So what can we do differently?

Try to aggregate and simplify a bit for your stakeholders; provide just a little more interpretation and presentation on top of the solid data you already keep and it will be much easier for non-techies to make sense of. As long as you are not misrepresenting the underlying facts (easy to do unintentionally whenever summarising data for consumption by others) you'll probably find that your business partners will refer to that as the 'transparency' you've been trying to achieve and they'll thank you dearly for it.

Cloud Congress Done Quick

2010-03-17T17:59:00.005Z

[well, my bit at least]

Yesterday I spoke at this year’s Cloud Computing Conference covering what cloud computing really is all about (once you peel away the hype) and how to break the back of the adoption problem – what do you put out there and how do you get started? It was the first time I’d ever done a talk that had absolutely no diagrams or pictures whatsoever in the presentation and, considering the plainness of the slide deck, it didn’t go too badly at all.

Excluding the mundane introduction, here’s the reader’s digest version of my key points:

What's in a cloud?

As a relativity new and trendy technology cloud computing is open to a lot of debate and even the 'correct' interpretation changes as the technology matures. We've had web applications for a long time now, so I'm not comfortable with SaaS being thrown into the cloud bucket. I like to define it in the context of how it changes how you deliver software (by abstracting away the complexities, layout, and connectivity of infrastructure from you developers) and how it impacts the way delivery costs are calculated (by exchanging metered billing on a usage basis for CAPEX-heavy up front acquisition).

My test (does your definition of cloud computing rely on the observations of your end users?) is simply to ask yourself "am I saying I am a cloud company just because my users access my product over the web?" If so, then perhaps you need to consider that maybe you might not be. Nothing wrong with that, but nothing new either.

What your FD sees

My next section expanded on the CAPEX vs OPEX shift that cloud platforms enable. If you have got your product design right then - on the web anyway - more usage should equate to more revenue and, with a cloud platform, more usage equates to more cost. See how that works? The cost base grows in line with revenues, therefore smoothing out that lumpy accounting and tricky budgeting activity that is characteristic of large hardware drops throughout a site's lifetime. You've also able to bring new things online relatively quickly (eliminating the ordering/delivery lead time/racking and stacking stages) and you can afford to try out slightly more speculative business cases - if I'm on the fence about a particular feature then I'm much more likely to give it a try if I know I can pack it up quickly and cheaply later if it's turns out for the worst.

Most cloud platforms have pretty good metering systems in place and that allows you a much more granular view of what part of your system is directly responsible for what parts of your cost base. From a business case perspective my advice here is to include all costs and not forget about time as a factor. Sounds a little obvious but when, for example, looking at Amazon's S3 as a storage platform for an analytical data set your total cost is going to include the transferring in and out of data as well as the cost per GB of storing it. Time matters too - I've seen the odd business case for a cloud platform fail to stack up because over a 3 year period more total cash is paid out than total cash spend once the large CAPEX bill is amortized over the same period.

Cloud capabilities

I also touched on some of the less obvious uses for cloud computing because so much emphasis is given to migrating existing systems into the cloud and I don't think enough time is given to considering what additional things that aren't being done now could be brought on because of the inherent properties of cloud platforms. These include private content delivery networks (because the larger cloud players tend to have a good reach), enabling offshore or outsourced development without opening your perimeters to external organizations who may have weaker security policies, neutral territory for integrations or joint ventures, and large scale load testing (because where else will you get hundreds of high spec load generators external to your network and connected over realistically-latent lines).

Development and testing environments are a good way to dip a toe in because, if you're doing it right, you will already have nicely sanitized data (which gets you clear of most of the oft-cited security concerns) and you won't be expecting production-sized load. It's also the best way to get a good feel for the cloud suitability of your production system without making any user impacting changes.

Architecting for the cloud

Many people - mostly those selling cloud integration tools or who charge by the hour - will tell you about how they can help you move your systems into the cloud. Don't kid yourself on. If you're using a half-way decent definition of a cloud platform then there is a lot more to it than is commonly appreciated. There are a number of good design patterns that I believe organizations should start adopting today and not just because they prepare systems for cloud runtime in the future, but also because they're pretty good ideas on their own merits.

It all starts with my favorite - decoupling and creating clear, distinct boundaries between functionality in a system and abstracting the specifics of the implementation which delivers said functionality behind a well defined interface. When you present specific uses of data to a network in this way then, subject to a bunch of common sense rules, you are able to host that individual part of your overall system another way - on different servers, in another data center, or hey, in the cloud! As long as it's reachable by it's consumers - which brings us nicely onto messaging and using state sparingly - you buy yourself the flexibility to move things quite dynamically. A service registry is also highly desirable if you a) have composites made up of many services and b) want to be able to move them and scale up/down dynamically.

All good practices for scalability and availability regardless of your stance on the cloud.

The crystal ball

You're not allowed to speak at an event and not give some predictions. I think there is an old charter or something somewhere. So mine were; barriers are coming down and this will continue with technologies such as private link and private clouds, as with all trendy concepts the waters will be muddy for a while as the word 'cloud' is appended to everything we've already got in order to sell it to us again, and within 5 years I expect to see hybrids (cloud-type platforms in use to some degree) in almost every organization.

Overall a good conference - some top panelists and speakers, and I met some great folks there. Thanks to all the guys and gals at SixDegrees for putting together a worthwhile and fun event.

You can find the slides here.

Just a little more on SLAs…

2010-03-08T23:03:00.001Z

Yesterday I posted a little something about SLAs and I’m always happier with things when I can wrap them up with a handful of guidelines. Not always possible in the complicated world we live in, but here goes anyway:

Discover the things that are meaningful for the business. I risk stating the obvious but there is always a temptation to approach this ‘backwards’ by starting off with what can be measured rather than what is significant (and then working out how to measure it). You don’t want to end up with a bunch of metrics that are easy to count but don’t describe desired system performance.
Strike a balance between persistence and change. Unless doing lots of projects isn’t important to you, be careful not to base all your KPIs on availability/stability metrics – or if you do, at least be aware of how that can drive reluctance to push changes through the system.
Make appropriate interpretations for each product or system. In most organisations different systems, or parts of each system, are subject to different uptime, capacity, latency etc demands. And assuming you pick some basics like performance they should be specific to each product; for a website that might be a number of page impressions, and for an analytic system that might be a time to render when a data set is updated.
Include time as a dimension. Most businesses – particularly on the web – have a number of 24x7 products, but there are also a lot of systems that only get used during business hours or at certain intervals (e.g. payroll is usually a monthly thing).
Disregard #1. Kind of. Now that you’ve gotten this far, you will need to consider some feasibility, because signing up to unachievable SLAs doesn’t help anyone. Have a look at what devices and services underpin the business functionality you are measuring. Trees of dependencies, composites in SOA for example, tend to live up to the least strict SLA rather than the aggregate of the set.

Rules of thumb – apply in conjunction with local knowledge!

More Meaningful SLAs

2010-03-07T09:09:00.000Z

Establishing internal service levels with the rest of the business is a difficult process - there are so many variables that can be measured and, as we all know, you change what you measure by measuring it. For example, if you express your SLA exclusively in terms of system uptime, then you improve all the activities around keeping your system available. The flipside of this is that you often discourage the activities around effecting change in the system - after all, any releases or upgrades or new features always carry some risk to availability and that's what they're measured on...

The place to start is to work out what's important to the organisation. Performance and availability are critical to us (a latency sensitive transactional platform with variable usage patterns) but so is change (a content driven web application correlated with events in the real world). We decided that performance, availability, change, and support response were the key metrics for us - nothing unique so far, and next we had to make an interpretation of each of these that was relevant to our various systems.

A basic principle here is recognising that it isn't just the raw numbers that should be appropriate to each individual product, but what is being measured too. Throwing an overall value at the problem (for example 99% availability across the board) makes the job of putting together your SLA easier, but is it a true reflection of your infrastructure? Whenever I've seen this coarse-grained approach used it has always led to less than acceptable uptime for the most critical applications and wasted investment propping up others that are realistically less important.

Another way to make sure your SLAs really closely matches business need is to introduce the dimension of time. In many systems and many organisations demand - and the cost of downtime - varies over time. For example, how many accounts and payroll systems are used around the clock? If you can trade off to 'best endeavors' over weekends and evenings then you shouldn't have too much trouble meeting a five nines commitment during business hours between Monday and Friday.

For our website we have a flat availability target (such is the nature of a 24x7 site) and performance we interpreted in a latency metric for price publishing and order placement. For reporting systems - which do not experience the same round-the-clock demands - we have different availability targets during business and after hours. Performance in the context of those systems is interpreted as a certain set of daily reports delivered by a fixed time each morning and a message delivery SLA on alerts on certain events. SLA's around change and product delivery are much more complicated and fraught with subjective measures. We've gone with measuring development projects iteration-by-iteration; what got delivered vs. what was committed during that sprint's planning. It's objective and encourages good estimation and strict control of scope creep during a sprint.

Making SLAs commensurate with what the business genuinely demands from a given piece of technology is important. Setting your sights high can seem like a good idea on the surface but, when you consider the frightening magnitude of difference in cost between 99.5% and 99.9% uptime, that couple of points can only ever be described as waste if they are not intimately linked to the organisations success.

Gates and Nonblocking Delivery Practices

2010-02-28T14:23:00.005Z

A few weeks ago one of my guys sent me this article from The Agile Executive which is - although not explicitly said - about the collision between ITIL infrastructure governance and agile development methodologies. I use the word 'collision' very deliberately because those guys deserve credit for how well they've managed to knit those (often opposing) worlds together. Which brings me nicely onto gates.

Gates, defined as the "You Shall Not Pass" checkpoints which must be navigated throughout a project, are not inherently bad. They just get misused in the same way that anything we do has the potential to be applied over zealously.

Gates have a place in delivery and fulfill an important function. They serve as a kind of final quality checklist to make sure that all those things you said mattered and had to be done actually have been done. As long as they are few, important, and near the end of the development cycle, you can make this work.

Where this tends to go wrong is when gates are established without clear and measurable clearance criteria defined up front. What you have then is an obstacle with a variable and subjective success criteria - and then everyone wonders why a project arrives at the gate and struggles to meet the requirements to move on. An understanding of exactly what's on the checklists a project will face as it goes live and exactly how that will be measured has to be front loaded, so that development teams know exactly how to make a piece of work transition through the lifecycle smoothly and organize resources in advance.

If this isn't established early and you get a project logjam downstream then you often end up having to compromise - and that means compromising on key quality or operational requirements that you believed are important enough to gate - in order to restart the flow. The good old fashioned 'defects fixed later cost more' curve still applies here.

Another common misuse of gates is introducing them into a process primarily for reporting purposes. It is true that a regular set of gates established throughout the end-to-end project process provides a set of convenient handholds against which to benchmark progress (we are at 'customer feedback on specification' or entering 'test run 3' etc) but there are a couple of downsides that go along with that.

Firstly, it tends to encourage over-reliance on artifacts such as documentation and reports rather than working software as a primary work output. Secondly, progress becomes measured by the clearance of gate after gate, step after step, and revisiting a previous step is seen as a step backwards. That lines up with an easy to describe liner view of the world, however with most complex projects in most organizations things are not that simple and progress often involves degrees of overlap and leapfrogging.

5 tips for more strategic budgeting

2010-01-27T22:18:00.008Z

With the end of the financial year looming as the next major fiscal date in many UK company's diaries, IT people everywhere will soon be embarking on the wishlist-to-business plan journey that is the budgeting process.

There is already a lot of good advice out there on how to estimate numbers and prioritise what makes the cut, and that can be a brutal process, so here's a few things I keep in mind to make sure I'm not being too short sighted:

1 - Know the cost of a customer. The rest of this list isn't in any particular order, but this is definitely number 1. Most good CIOs and CTOs I meet know what this disk array costs vs. that disk array and what these licenses cost vs. those licenses, but not that many know what a customer costs to acquire and convert or to reactivate if they drift away. The relevance? Knowing that means you can work out exactly how many frustrated customers you can lose because of that bug before its worth fixing, or that slow server before its worth upgrading. In this light, things that might otherwise not have made the cut become the no-brainers that they should be.

2 - Don't be influenced by trendy buzzwords. There is always a meme that vendors will be able to exploit because of our inability to apply common sense and logic to a pitch on whatever the currently fashionable thing is. SOA, virtualisation, and cloud computing come to mind whenever I recall ill thought through spending sprees. That's not the same as saying that these - or any of the other candidates in the same category - aren't good technologies with valid applications, just that any investment in them should be handled with the same pessimism and judgement of real business value as money spent anywhere else. Just adding 'cloud' onto the name of a product doesn't change it's fundamental ROI, so don't let these slip by you unchecked.

3 - Know the cost of scale. When sizing up a new project, give growth plans some weighting rather than just initial costs. Depending on the degree of speculation involved in your plan, taking cheaper entry options can sometimes bite you when you reach scalability limits.

4 - Wider consultation. A year is a long time and a budgeting exercise inevitably involves dusting off your crystal ball and trying to foresee everything that everyone is going to want in the coming 12 months. Good luck with that and, although it seems a little obvious, spending a bit more time with each area of the business trying to coax their plans and ideas out of them will at least expose the bigger ones up front.

5 - Increase priority on core things. Another one that sounds obvious but - through its limited practice - clearly isn't. In tougher economic times it is particularly important to give the most crucial things the most attention, and it can be mistake to favor too many might-lead-somewhere-might-not initiatives over those core basics upon which the business depends today. I think it is critical to set aside time and money for innovation and to take a punt on those less-certain ventures that our instincts tell us have that commercial je ne sais quoi but it needs to be proportionate to the main line. This is important when deciding what to drop in order to bring your wishlist and the available cash resources closer together - big but critical projects can be easy targets because they pull back more cost in a single swipe than a number of smaller but less meaningful items will.

The hidden wisdom of Kevin McCloud

2010-01-06T17:26:00.008Z

I was lucky enough to get a copy of the Grand Designs Handbook for Christmas - it's a compendium of building projects from the channel 4 show (the book, not Christmas). The first few chapters have a lot of lessons learned and good practices picked up from 9 seasons worth of experience with, well, variously successful building projects.

Reading some of the advice, there is a lot that can be applied directly to software projects. There is a section on 'how to behave with your team' which is aimed at helping people keep their building sites productive - but the advice translates well to technical projects. Here's a couple of my favorites along with my join-the-dots:

"Don't give verbal instructions on site unless they've been agreed with your consultants and they're backed up in writing. If you just issue verbal instructions, you risk confusion and possible extra costs."

This is a top one for me - especially in the agile world where many interpret 'agile' as 'make it up as you go along' and then end up disappointed. I think maximizing interaction between delivery team and customer team is a good thing especially when both sides recognise the boundary between clarification and change in in-flight work.

"Do take time to prepare detailed written briefs setting out all your requirements for all your consultants. A good brief is an essential part of the design process."

One of the simplest formulas the govern all project delivery (hardhat or mousepad) is that the quality of what you get out with never exceed the quality of what you put in - yet this seems to escape so many of us so often. What you invest in clear instruction and regular contact with developers is a key variable in how happy you'll be with the results.

"Do allow your experts the freedom to do their jobs. Stand back from the project and enjoy your role as client; making decisions and hanging around, generally being a good egg and imbuing the world with optimism and excitement."

I don't exactly advocate stakeholders 'standing back' from a project, and I don't think that's what Kevin means either. Don't micromanage your delivery team and, as a customer, be sure to perceive the difference between decisions you should make (and don't let the team wait for those) and decisions the team should make (and let them use their skills and experience to make them).

"Don't listen to other people outside the professional team. You'll only end up getting very confused."

External feedback and extra ideas are good, but remember that an engineer knows his business best and don't fall into the trap of listening to someone else simply because you like their answers better. I consider myself relatively smart but even I'll say all sorts of wacky things when I don't have any skin in the game...

There are a couple of other sections with close analogies to technical projects - how to be a perfect customer and how to put a good brief together - which I think are 2 other critical ingredients to any successful engineering project (construction or software) but they'll keep for another post.

Standups vs Team Meetings

2009-12-22T18:40:00.000Z

When teams first start to pick up SCRUM, there is a tendency to let stand ups replace regular team meetings. The risks here are that your stand up will elongate and get off topic because it is the only chance you get to talk about issues as a group, yet your team cohesion will still suffer because no matter how long you can string out a stand up in the morning it won't be long enough to table all the things you need to get through as a team.

A stand up is a project-oriented meeting, all about running the day to day work of the team, has a tightly fixed agenda, is time bound, and specifically focused on what's required for today.

A team meeting is a team-oriented meeting, all about maintaining the team, continuous improvement, future requirements, and focussed on bigger picture topics.

They are definitely two very different meetings with two very different purposes, and both of them are necessary for a team to run smooth projects and keep productivity and job satisfaction high.

Introducing Sporting Index Super Simple Integration

2009-12-16T11:06:00.001Z

This year we decided to give our B2B customers an early Christmas present - SSI.

Sporting Index (well, really Sporting Solutions, our B2B brand) provides pre-match and in-play pricing and content to fixed odds bookmakers for hundreds of markets across 16 sports with more events from more jurisdictions being added almost every month.

Traditionally we’ve done this by pushing XML documents around over reliable transports and leased lines. This works very well and there is nothing inherently wrong with this solution, but we weren’t happy that customers had to install expensive fixed lines and associated hardware, and our back end is capable of producing multiple-thousands of price changes a minute – well beyond the rate at which some of our customers can consume them.

On top of this, our customers wanted the low latency and reliable updates that come with a push solutions, but also the easy integration and flexible client-side controls that comes with a pull solution.

Well, you asked for it and you’re the customers, so let’s unwrap this bad boy.

Super Simple Integration is essentially a RESTful web services API available over the good-old-fashioned Internet. The feed is as easy to consume as you’d expect from any web API, and we’ve spent a lot of time on the data formats and doing some clever pub-sub tricks in the background – thus all the benefits of the heavyweight XML services but through an integration which is quick and easy to get to grips with.

There’s a lot to this, so let’s just cover two of the most common use cases:

Get Content

Our content products can be used to populate websites with upcoming fixtures and players, and to power pages or widgets showing points, corners, goals etc as they happen in running matches.

Content products are mostly stateless and work on a fairly simple request-response model, which can mean no integration work at all. Well that’s half true – you still need to consume the data from the API – but you don’t necessarily need to build a repository on the customer side or have to match up schemas.

Let’s say that wanted to build a little scoreboard for my site to show my visitors the progress of a live England v New Zealand rugby game;

You get the content you want, as often as you want, delivered back in name-value pairs. There are also a couple of methods to call to list out what sports and events are available, so you can pick out what to pull into your scoreboard app.

Pretty basic stuff so far, where it really gets sweet is how we implement real time price streaming.

Get Prices

Our price products can be used to price up sports betting markets, either as a risk advisory tool for human operators or a totally automated pricing service, and cover a wide range of sports in-play with high fidelity price output.

In price products - particularly once fast paced events turn in-play - state and reliable delivery become important, but we’ve enabled our customers to nicely sidestep the costs and technical complexity this usually involves. When your API client registers interest in pricing up an event we build you your very own data object, which we call a channel, and then it works like pub-sub system. Our back end pricing engine is event driven, and as events occur which trigger price changes, those changes are accumulated in the channel you’ve subscribed to and delivered out to your system in real time over secure HTTP;

This puts quite granular control of the product into the client-side’s hands, allowing you to programmatically build object(s) with any combination of markets and data that you’re interested in and then subscribe to your very own private feed over familiar transport.

By accumulating interesting data into user-defined objects this way we also buy back the robustness that dedicated WAN links gave us – if there is a network disconnection the client simply re-establishes an HTTP connection to the API and picks up where it left off – no lost updates or re-synching required.

That’s SSI in a nutshell. It’s the easiest integration in the industry, and from our perspective it scales out nicely - being just a collection of smart web services - and also enables us to accelerate the building of data-driven products for our own business too.

Development Environment SLAs

2009-12-14T08:53:00.001Z

When we use the word 'production' to label our customer-facing systems we create certain expectations around how we support and manage them - careful change control, detailed monitoring, rapid response to incidents, and administration that works around user requirements.

But when we label our software delivery lifecycle environments 'development' and 'test' systems, we likewise heap upon them certain expectations. Unfortunately for the development process, those expectations are usually secondary priorities and open season for any sort of experimentation or ad-hoc change.

Perhaps this is why so many projects come across environmental issues from time to time? Development environments are in production too, they just produce something a little different to what your customer facing systems produce - more customer facing systems.

Sure, when faced with a direct choice between doing something for a lifecycle environment and something for live products the decision is easy, however handling development and test systems with a little more production-like behaviors doesn't need to impact your live system activities and will make all your projects a whole lot smoother.

Welcome to Movember

2009-11-01T10:43:00.004Z

Movember is an annual event to raise awareness (and money) for men's health issues, and this year I'm joining in with a pretty sweet handlebar. Keep an eye on my Mo Space to check out the mo and back a good cause.

While we're talking about charity, remember this post from Christmas last year? We're now at £222.14 - and I guess we've found our good cause now, so this'll be our first donation.

* updated * money from the google-ads-for-charity stuff above has now been donated - check my Mo Space.

* updated again * grand total is in - we raised £1098.14 from family, colleagues, and friends. A massive thank you to all who supported men's health by donating or participating.

I think you might have it backwards

2009-10-03T10:46:00.000+01:00

The most common criticism I hear about agile is also the most common mistake teams make in their early adoption of iterative practices - not fixing any certainty.

One of the benefits of managing projects using, for example, SCRUM is that you're not burdened with having to define all the requirements up front and plan out all the tasks and times for all the resources for 6, 12, 18 months. Trying to precisely define everything a project needs up front and trying to plan for every dependency, holiday, resource issue, structural or strategic change over a long period of time proves again and again to be an impractical task.

Enter agile.

Suddenly we don't have to worry about the detail of every change and every movement over an entire year, and we're introducing a whole bunch of flexibility. Where it goes wrong is not fixing any certainty in the delivery process at all.

Agile is about certainty and detailed planning, just detailed planning over a period which is possible to predict. Who knows what'll happen over a year, but you at least have a fighting chance of planning out the next 2-4 weeks will some degree of accuracy. Beyond that you can keep plans and requirements fluid and high level, netting you the right balance of fixing doable deliverables and keeping flexibility.

Software delivery needs certainty to happen - you have to be able to rely on some requirements and some resources over some time span in order to make progress - and agile isn't about not having any, it's about having it on terms you can foresee.

Screen Scraping for Dummies

2009-09-27T23:15:00.000+01:00

If you run a website with any sort of valuable content, then you are almost guaranteed to run into scraping sooner or later. Screen Scraping is more or less an automated program taking an impression of a web page and then parsing it to pull out some specific bits of data that the scraper is interested in - which theoretically is then stored or used in some other way.

The piece of software that does this scraping is commonly called a robot, or bot, and it is really just an automated web client that accesses and uses sites in the exact same way as it's fleshy counterparts, just with machine precision and repetitiveness. A bot may be a large and complex program running on a server with it’s own database etc, or as simple as a script running in a browser on a desktop.

This is typically regarded as undesirable behavior by many sites because, in most cases, it’s a source of load and associated with unprofitable usage. Whenever we draw a page impression, which we’ll do for every bot hit just as we do for every human visitor, we consume web server time and, worse, back end time. When we structure highly functional pages loaded dynamic content we can create a very engaging user experience, and all that functionality is built on plenty of back end work and data. When used at regular human pace that's usually OK, but under the relentless rate of hammering robots are capable of, it starts to become expensive.

If you've got a case of bots, you have to start by identifying them. Their repetitiveness and flawless precision is, in this case, their downfall and we can usually spot them easily through proper analysis of web logs - no human user is as mechanically regular, millisecond quick, and consistent in journey as your average droid.

Spotting droids isn’t too hard, but then you've got to decide what you're going to do about it. Most scrapers aren't malicious and often don't realise the headache they're giving you. In the first instance it's best to try some detective work, see if you can find out who they are, and get in touch. Domain registrars can be a great resource for this, but don't overlook the obvious - maybe they even have an account with you if data they're using requires signup to view.

Beyond that it gets tricky, and can easily turn into an IP address blocking arms race. With some caching finesse or smart layer 7 rules you can throttle bot activity to more palatable levels, or persist them to their very own node that they can thrash all day without impacting the experience for the rest of your users.

If robots are a very big problem for you, then try and take it as a compliment on the value of your data and perhaps consider publishing it via a productised API or feed - if you make it simple enough to consume, then most scrapers will willingly change over to a more reliable integration mechanism and perhaps even pay you for it.

conferences - jolly good or just a jolly?

2009-09-20T09:17:00.000+01:00

It has always interested me that two different people can look at the exact same conference, and one declare it to be an unmissable event relevant to their work and the other consider it a waste of time; a free day out of the office at best.

It always leads to a discussion about which events are worthwhile and which aren't, but here's the secret - whether a conference is worthwhile or not is only 10% the event itself and 90% what you do while there and afterward.

You could carefully plan the sessions you attend, take the time to meet people with similar technical problems or who have done worked with relevant technology and follow up with them later and, when you return to the office, share your new knowledge around and adopt some better practices.

Or

You could have a few days out, enjoy some vendor's hospitality (free beer always tastes better), and check out some new bars and, when you return to the office, settle comfortably back into your old habits.

That, ladies and gentlemen, is the difference between a jolly good conference and just a jolly.

Delegation from the other side

2009-09-04T21:19:00.016+01:00

I really like Manager Tools - it's probably the best single source of personal development material that you're ever going to find bundled in easy to consume episodically content. Recently I thought I'd go back and listen to the basics series again, because you can never spend too much time on the fundamentals of good management and, like Mark and Mike say, if you don't do anything else except those basic practices, you'll probably do OK.

It was a really good refresher, and worthwhile doing from time to time. Then I got to the juggling koan cast, which was about how to handle that inevitable situation when your boss gives you another big task to work on when you're already maxed out with too much to do. It's worth listening to - so I won't spoil it entirely - except to say that the answer focuses on developing your delegation muscles, passing on some of your things to make space for the new action and helping your team to grow as an nice side effect.

The majority of the cast focused on how to handle events after you accept this new delegated responsibility, and I liked the point about deliverables gaining size as they flow down the delegation tree (i.e. your small tasks are bigger challenges for more junior members of your team), however I think there was a valuable lesson here about the delegation from the recipient's perspective.

The Manager Tools perspective on this was to accept the new delegation, and to view it as the expression of confidence and trust and the development opportunity that it really is. Many new managers would worry about whether they have the time to take it on or not (no thanks boss) or attempt to enter into a negotiation about what else to drop.* The reason I like this angle is that I don't think there is enough coverage out there on followership.

Leadership is glamorous and fashionable to write about and speak on, so we do a lot of it, but good followership is important to develop too. There is an inherent responsibility on all the individuals in an organisation to be rational participants, and I think a group who know and practice the right behaviors on both sides of that equation performs better. Anyway, isn't part of what we're doing when we're developing our people just cultivating better followership?

Something I'd suggest you ponder when you listen to the cast is clarifying the difference between being handed an action and given responsibility for an outcome. There was heavy emphasis on not entering into negotiations with your boss about whether or not you are going to take this thing on, however I think it's both valid and valuable to question the task and look for the underlying business value.

This isn't arguing about whether or not you or your department have the time, or forcing your boss to manage your priorities for you, it's simply clarifying what you're really being expected to achieve rather than what you're being asked to do. In my experience, particularly because I run technical departments, there is often a difference between what people want to achieve and what they initially ask for, as they think they have resolved the outcome they have in mind into a few actions and sometimes those actions don't add up.

If you can have that sort of open discussion, then you might agree to accept delegation of a slightly different task than was first put your way, but one that much more directly leads to the goal your boss really had in mind.

* Key caveat here about dropping stuff. Things will eventually be dropped, and when you're handling the most important things you can and have delegated the rest then so will your directs, and therefore [due to the hierarchical effect created when everyone delegates their least critical things] what gets dropped should eventually be the least important tasks overall.

A better way to manage equity?

2009-08-30T00:17:00.006+01:00

When we get together to start a new venture, the structure of the equity is always an early topic. The usual format is something along the lines of 'you do X for 20%, he does Y for 10%, and I do Z for 30%' and then we carve up the company up front.

The problem with this approach is that it doesn't easily cater for all the things that really happen during the course of a startup. For example, you can't always predict who's going to stick with it or how many people are ultimately going to be involved. As the idea evolves (because you almost never end up with the same plan you started out on) the bias of work can shift to a point where the equity split no longer reflects the effort split, and of course the value of effort to a business changes over time.

I don't think there is such a thing as a flawless mechanism for this, however I think I might be onto something a little more elegant. Here's how it works...

Break all the work - this same approach works for commercial and marketing tasks just as well as it does for the technical stuff - required to bring your idea to market into tasks or jobs which can be done individually by the members of your team. Decide on a number of points for each task, based on a sensible blend of value to the business and effort required to deliver. Points are the key to the whole thing; do the task -> earn the points -> exchange the points for shares.

Because you haven't fixed the equity allocations up front, it doesn't matter if you gain or lose co-founders and you minimise those 'who isn't pulling their weight' discussions, as you only get the shares you directly earn.

But why points when you could just do the same thing with the shares themselves?

Points give you a couple of extra controls you might not have if you immediately issued the shares; for example you can set an exchange rate between points and shares, allowing you to increase the relative value of tasks not being picked up, or offer individuals a better exchange rate once they have accumulated a large number of points (thus encouraging fewer greater contributors which makes for a more coherent project and a simpler shareholders register).

Setting specific milestones (for example beta launch or first partner signed up) at which points can be exchanged for equity ensures that everyone still focuses on bringing together an overall business delivery as well as their own contributions.

Forming your company with a healthy allocation of shares and some fairly accurate record keeping on tasks and points (I can recommend zen if you're an agile/lean thinker) is all that's required administratively.

There are a number of more subtle points to this, such as making sure you've got strong enough acceptance criteria on each of these tasks, but that's the meat of it. I'm running a project using this method now and - while we're not done yet - it's going pretty well and has already afforded us some flexibility we wouldn't otherwise have had.

Doing it on the side

2009-08-19T07:07:00.000+01:00

Something new managers often struggle with is what their response should be when members of their team are involved (or are considering being involved) in external, perhaps even related, business activities. I encourage it wholeheartedly, with a small number of key caveats, on personal development grounds.

Trying out "their own thing" will teach your guys about responsibilities well beyond those that you can reasonable expect to expose them to in their role, and give them some insight into the difficulties of running a business, taking care of finance, marketing to customers, and making product decisions. There is so much to the operational side of an organisation (even a tiny fledgling one) that they would otherwise not have many chances to experience first hand. This can help your team members develop their creative, entrepreneurial side and give them a much better appreciation of the challenges their colleagues elsewhere in the business face. This is so beneficial that I'm even found guilty of supporting such ventures with advice and coaching from time to time.

The caveats? There are only 2 key things that must be true to qualify what I’ve said above:

1 - You still have a job. Outside interests should not impinge on the quality, quantity, or timeliness of work, and if they do then one of these things must go. My policy is to provide a flexible working environment, but I'm not here to subsidise your startup either.

2 - You shouldn't compete with your employer. For most people in our industry, our roles come with a duty to innovate and develop new product to further the companies interests. Whenever something extracurricular can be even slightly interpreted as related, it needs to be raised with management and explicitly categorised into 'company IP' or 'fair game' for everyones benefit.

Regardless of which side of the employer/employee relationship you sit on it is always worth finding out what your company policies are on this kind of thing before getting underway. Being uninformed might mean you start out with the best intentions and end up having to make difficult choices later.

The nuances of Planning Poker

2009-06-16T16:37:00.001+01:00

I’m going to skip right over explaining what planning poker is and how it works - especially since there are so many simple and effective explanations out there - and assume that you’ve at least given it a bash.

What I want to address is the spirit in which group estimation happens.

Every potential task owner (read: qualified opinion) estimates each relevant task as it comes up, but the reason for this is commonly misinterpreted. The purpose is not to force the eventual task owner to reduce his or her estimate, or to adopt an estimate that others in the group ‘prefer’ and nor is it a criteria for selecting an owner for a task, based on the estimate the product owner or scrum master likes the most.

The purpose is to draw out knowledge from the outliers, so that the whole team can benefit from their experience. When someone in the group pulls a card miles away from the rest (assuming a consistent understanding of the task) that’s something to explore. You might have a much higher estimate – perhaps that person has been burnt before by that same inconspicuous looking requirement masking significant complexity. You might have a much lower estimate – perhaps that person knows a good work saving shortcut or pattern uncovered during a previous project.

If you take the time to ask those questions and consider that information, the group learns from one another and everyone gets better at estimating.

And whose estimate is used in the sprint? The person who is going to be doing the work...

Right First Time

2009-06-14T21:50:00.000+01:00

Imagine if the first wall you ever built had to form an integral part of your dream home, or if the first time you ever drew a picture it had to hang in the Louvre? Or if the first cake you ever baked had to be for your daughters wedding?

That’s clearly unreasonable.

So why does it seem reasonable that the first iteration of a new system you build should be the one that your business depends upon?

That’s why we prototype. That’s why we iterate. That’s why we beta. Don’t rob yourself, and your organisation, of the wisdom you’re owed from those lessons.

So who is in a team anyway?

2009-06-08T14:22:00.000+01:00

I caught up with a friend the other day; he’s in the system integrator business and is struggling a little with recent growth. Long story short, things are a bit a disorganized and efficiency is low. They’ve got some kind of consultant in, and we were talking over his advice.

So how does their guy propose they fight the first battle in the war for organizational effectiveness? Setting aside certain days when people in one team aren’t allowed to talk to another. No, you didn’t read that wrong, the ‘big plan’ is to actually create barriers to communication.

I just hope that wasn’t expensive.

In the reseller/systems integrator game it’s all down to the cooperation between sales and their technical teams, and so I really struggle to see how driving intentional wedges between them is going to help. So my theory? Perhaps the wrong people are teamed up together in the first place…

In any mission there are multiple people with different skillsets doing different jobs all coming together around a central project/customer/business problem. Isn’t that a team? The people who actually need to work together on a daily basis for the direct benefit of the customer? I don’t know why we keep insisting on grouping people by job or by skillset (sales team and technical team, or dev team and test team) because the cold, hard facts are that these teams can’t achieve anything end-to-end for a customer on their own. Maybe it just appeals to our human need to nicely categorize things.

When you put people together in a team, you create a tight loop of communication and cooperation between those people (it’s one of the reasons we do it in the first place) – but you do something else too – you also create a little organizational fence between them and other teams. A fence that we absolutely should work at keeping as low and friendly as possible, but a fence nonetheless. To me it just comes down to where you want to put your fences – does it make sense to create boundaries between the individuals whose effort must by synchronized daily to complete units of work?

In what I do, the unit of organizational delivery is the project (or, as a superset, the product). That’s that unique object around which we all gather and apply our individual skills and experience to create tangible value for the company. So by default, I have a nice, simple method for grouping my guys together in a way that creates the fewest barriers to the most common communication and coordination lines that they’ll be following each day. I suppose I could put all the developers in one team, all the testers in another team, all the project mangers in another team again etc – but if I did, then what’s the very first thing I’ll have to do as soon as I want to do any work? That’s right, pull them back together into virtual or temporary teams based on the projects they’re delivering against. Hmmm.

From my experience in the SI and reseller game, I’d suggest that the unit of organizational delivery is the customer. That’s the internally-ownable entity which performance and profit is measured against, margin is made against, and time is billed against. Yet almost every one of these companies is, at best, a matrix management organization. So you group people together into a sales team, a tech team, a presales team etc – and then wonder why, whenever they all have to get together around that common indivisible business problem (the customer) there are all sorts of communication issues, competing priorities, complexities planning time...

Perhaps a team in this type of organization should be something like a sales guy, some sales support, a presales guy, a couple of engineers, and a share of some administration – all grouped together around a set of similar customers that they own the end-to-end P&L for.

The inevitable argument against this kind of cellular organization is the same one I’m used to elsewhere – developers like hanging out with developers, infrastructure engineers like hanging out with infrastructure engineers and so on. My only response to this is that I don’t see how organizing your people into the teams which best reflects the true structure and purpose of the company precludes communities of interest, and again, just ask yourself where you want to put the barriers, and whether you consider the upside of keeping people grouped into teams of similar individuals worth the downside of trying like mad to make them work together efficiently and effectively whenever you have to get anything done.

Good ops guys are hard to find...

2009-06-01T09:45:00.002+01:00

Good operations is about staying one step ahead of the state of the system; taking proactive actions based on quality telemetry.

I’m reinventing how my sites are supported – if you’re an awesome one of these or a kick-ass one of these, then we should talk.

Hi, I’m Eachan and I’m a fail-a-holic

2009-05-29T17:11:00.004+01:00

I attended a Harvey Nash CIO leadership forum a few days ago, subtitled The Strategic Insight Survey. Combining words like “strategic” and “insight” as a standalone sentence usually sets off my spider sense, however I must say it was a most worthwhile evening. Other heavily overused and frequently misunderstood words that were thrown about with reckless abandon were innovation, transformation, and ROI etc – all against the backdrop of this current economic climate.

All this talk of innovation, and how you measure its ROI, was starting to tickle my aforementioned spider sense, but since a couple of the panel were from the public sector, I couldn’t help but want to explore the effect that the general public’s reaction to failure has on the culture within those organisations. This is why I’m either never invited back to these things or I’m on the panel next time.

Firstly, let’s just agree that the cost of innovation is not in salaries paid to folk you can point to and say, “he’s doing some innovating” or in capital expense on new widgets and bleeding edge technology. The price of innovation is failure. In fact, I would even suggest that the price of learning anything material is failure.

So then I look at the slating that any public service gets when things go wrong. We bay for blood in the most public of forums, newspapers, radio, and even (opportunistically) on the opposition benches. A major mistake was made, so someone has to go, right? There are countless examples of this; ranging from embarrassing (eg HMRC data loss), to genuinely horrific (eg Baby P tragedy), to that thin line between the rules and the spirit of the rules (eg MP’s expenses scandal).

It’s difficult to come to an agreement on what should have happened in any one incident, as we will all feel differently about each incident and will be affected differently by it, but that’s OK, because it is the pattern that matters. The pattern is what sets the culture.

If, every time things go wrong, we excise every last individual involved, how then do we ever hope for the lesson to be learnt? Who has felt the pain, and is carrying that forward as a driver for change? Where is, to quote a buddy of mine, the feedback loop?

Is it right for us to do this, to create such epic risk aversion, and then still expect these organisations to deliver even the most basic of improvements let alone giant innovative leaps forward? Are we just robbing them of the best ingredients needed to create the huge changes we’re demanding?

When I think more about it, I wonder if I really am a fail-a-holic at all. Perhaps I only embrace failure so much because I’m actually a success-a-holic…

Don't out-specify your knowledge

2009-03-10T10:11:00.002Z

Combining some points from this post by Joe Armstrong and this post by Dan Creswell gives rise to some interesting implementation thoughts.

Standardisation is important, particularly in the context that Armstrong frames it in - if the interaction between a client and a server closely adheres to a predetermined protocol, then the systems overall behaviour becomes more predictable and easier to adapt and fault find.

But then this depends on the maturity of your understanding of your problem and your solution. As Creswell says at the beginning of his post, defining too much up front can be a trap. Coming up with a set of standards that are too rigid too early might start you down a road of building that system instead of building the right system.

Standards, patterns, and contracts are all important - but timing is key.

Choosing Recruitment Agents

2009-03-08T19:21:00.002Z

Every now and again you'll go through that compulsory "we use too many agencies and we need to rationalize" process. As part of that, you'll be working out a few criteria for evaluating recruiters and putting them on some sort of preferred supplied list.

The typical approach to this pretty much involves coming up with some lower rates than you currently pay (cost control is usually the motivation behind the PSL exercise) and seeing who wants to sign up. Not a bad start, especially if you come up with some tiers which (for example) encourage recruiters to take a lower fee for a period of exclusivity, but you should probably add some metrics to ensure quality.

Firstly, think about why you use an agency. Hiring strangers is the worst recruitment strategy ever, so when you use an agency, you are basically using their network as a replacement for yours.

So how about finding out some other stuff too; do they meet candidates in person before they put them forward? How long do they typically have a relationship with a candidate for before they start to place them? How often do they place the same candidate in consecutive roles? Anything you can come up with to give you confidence that their network is going to be a passable proxy for yours is going to be a worthwhile comparison point.

No problem too small, no reward too big

2009-03-04T09:00:00.002Z

Do you ever sit around with friends, talking over a few beers, and solve all the worlds problems? As I get older this seems to be becoming my chief pastime, and we recently got ever so excited about power generation and distribution. We figured the whole thing could be completely re-architected in a P2P model, where everyone in the grid both consumed and generated/supplied power.

Rushing to the nearest browser in case a patent application was necessary, we became a slightly despondent upon seeing a bunch of articles like this one on microgrids and what the EPRI are getting up to. That said, it's great to see this kind of stuff getting mindshare.

So what, if anything, is still missing? To see this come true, it seems like we'd have to work our how the power companies would still make money in our community-spirited utopian electricity supplying future. So perhaps we can still contribute by bringing some economics into the solution.

Someone has to provide that infrastructure - the grid we'd all have to be connected to in order to exchange electrical capacity. That sounds like something someone should be paid to provide. So perhaps there is a subscription fee? Or maybe we can do one better - how about a variable charge based on each household's net balance at the end of every month/quarter/year? The closer you are to netting out (producing the same number of kilowatt hours you consume) the closer to zero the bill gets.

The beauty of that model is it incentivizes consumers to be as self-sufficient as possible and disincents (is that even a word?) waste. The expected impact of such a price control is for participating individuals to want to align their production with their consumption - probably by boosting the capabilities of the former while working out efficiencies in the latter.

Reducing overconsumption is probably a no-brainer, but you could look at reducing overproduction both ways. On the one hand, overproducers supply overconsumers and we'd need that, but on the other hand we wouldn't want participating individuals to profit from overproduction. Ultimately all overproduction is wasteful (by definition if nothing else) so there should still be a price for it, a lower price than overconsumption, but not as low as netting out.

Well that was all kind of off-topic, but I hope you enjoyed it nevertheless. There are some intriguing engineering challenges in the basic problems inherent in sustaining our society, and if we gave this kind of stuff a little more attention, we'd all be living in a much better world before you know it.

Gartner Economic Downturn Briefing - Part 2

2009-02-27T22:55:00.002Z

Continuing on from yesterdays post; the sessions reinforced to me how little of the core Gartner content is aimed at web companies. Almost all yesterday's stuff was only really relevant to 10,000 seat corporate enterprises. I've worked in several big companies before, and the internal "business process support" technical challenges they face are very different to our external product-led technical challenges. We also don't tend to have such epic workforces and supply chains, we tend to be pretty lightweight and entrepreneurial by comparison.

Back on track - the economic downturn. It's kind of uncomfortable to think about the upside of something that is affecting so many so negatively, however we're well positioned to take advantage of the opportunities that are coming to the surface.

With a battening-down-the-hatches mentality, people are preparing for the worst. Sites like Fool and Lovemoney pick up as we try to get as many money-saving tips as we can. The first preventative action we're likely to take is reviewing utilities and other basic household expenses, and sites like Gocompare and Moneysupermarket see a lot more visitors.

On top of this, there are a whole slew of creative new ideas starting to come through, all predicated on our current need to squeeze a little more value out of things we otherwise tolerated waste in.

At the worst end of the scale, people are being made redundant. Again this is a terrible tragedy affecting people I know, and it feels unfair to be thinking about the opportunity at a time like this. Nonetheless, the internet is now the de facto tool of choice for finding your next role, and this means more actives on Jobserve and Monster. Whether you've been affected yet or not, it is now more important than ever to get an up-to-date profile out there and build up strength in your network (before you need it). Off we go to the professional social networking sites like Plaxo and Linkedin.

Lining my own pockets for a moment, online gaming and gambling is fairly recession-resilient, but even simple ad-supported business models can do OK. You're still turning page views and actives into revenue, and if you're providing the service or content that's currently in demand, then that's not insurmountable in any times.

Gartner Economic Downturn Briefing - Part 1

2009-02-26T20:37:00.001Z

It's been ages since I went to a Gartner briefing, and I figured this would be a topical one to pick the habit back up with - perhaps get a look at the ways other organizations are tackling this credit crunch thing. You guys often tell me that my posts can be a little too long, so today I'll summarize the key content and tomorrow (or maybe Monday - hey I have a day job too you know!) I'll put my thoughts up.

The sessions were focussed on how CIOs and CTOs are responding to the current economic circumstances, and the techniques IT organizations can use to keep cost under control and contribute to the businesses downturn survivability. So here goes - in no particular order...

• Focus on projects that are "shovel ready". Bang for buck is more critical than ever, and projects ready to be actually done are worth a lot more than potentially better activities that are still in the planning phase.

• Expect to deal with massively escalated regulation. In the post-credit crunch world, there will be a lot more regulation aimed at preventing repeat performances. This is likely to mean increased reporting and compliance overhead, as well as more constraints on how technology can be deployed and data used.

• In the worst of cases, there may well be a sharp rise in the number of legal actions. This is going to put increased demand on e-discovery and BI as organizations scrabble for the information they need for defence.

• A general loss of trust is going to lead to more diligence efforts. In a climate where any number of suppliers could go bust at any time, you'll need to spend extra time reviewing the financial integrity of key partners you depend on, and perhaps even making backup plans.

• The downsizing mentality could rob you of the top talent that you'll need on hand to steal a march on your competitors once times start looking up. Don't sack them, squirrel them away any way you can!

• Expensive oil means old assumptions on locations, topology, and transport need to be challenged. Get better at communications, and be prepared to accommodate supply chain changes aimed at closing the distance between steps.

• Check out your PR infrastructure. During these times the amount of media attention on companies is at a peak. Where is your corporate site hosted? Who edits the content? It is often an overlooked side-system, and right now you don't need the attention you'll get if it goes down.

• Take a look at your HR systems too. When companies get told to lay off 12,000 people over 12 months, their HR processes face a task they were never intended to handle. Pay special attention to data accuracy - mistakes in calculations on this scale will just kill you.

• Watch out for desperation moves. These are the times when you'll be the most likely to get stitched up by some crazy marketing plan, an over the top offer that gets madly oversubscribed, and so you'll want to keep your interdepartmental communication flowing and your eye on capacity.

There is some good sense in this, and some stuff I want to look at from a slightly different angle. My side of it - and what I think it means for us web guys - in the next post.

Scalability is not just a technical problem

2009-02-21T00:49:00.003Z

There is so much content out there about how to scale out web sites, platforms, and databases – but it all focuses on the production system architecture. Do a Google search for scalability – go on, I’ll wait for you... See what I mean?

Now I know that’s the fun bit to talk about, but being practical for a minute, if you’re starting to scale out systems using techniques like Digg, Flickr, Xbox Live, or the usual suspects like Amazon, Google, and eBay, then chances are that you’ve got a whole lot more scalability challenges than just the product.

If you’re dealing with at least hundreds of thousands of daily actives, then we can probably deduce a bunch of other stuff about your circumstances. We can guess that you’re after reasonably frequent feature drops, have a significant amount of horizontal distribution, a healthy sized engineering organisation, and a strong bias towards availability. And if even half of that stuff is true, then what are the other scalability challenges will you be up against?

How about multiple concurrent projects? If your estate is divided into more than one product then you will more than likely be working on more than one new feature in parallel. This gives rise to all sorts of version control and regression test problems, which demand process and infrastructure quite different to a single effort.

What about disparate teams? You might have people in a number of locations branching, or depending on, the same codebase. That’s a communication barrier which can be tough to solve. Large enough organisations also tend to sprout specialist disciplines, such as user experience and IA – this changes the nature of how teams engage and how work is specified, estimated, and delivered.

And how do you manage environments, tools, and documentation? A complex production architecture begets a complex development infrastructure, as there is a lot more interoperability to test for. Don't forget that with more teams working concurrently, managing contention for these expensive environments also becomes a tricky balancing act. As your products increase in popularity (a good proxy for profitability on the web) NFRs like performance and capacity will become more important and will require specialised tools to measure.

I’d like to see us sharing a little more about our experiences with this side of highly scalable systems – it might not be as sexy as memcached, CAP, and Gossip, but the reality is it is just as important a part of the solution nonetheless.

The Good and the Bad of Google Product Management

2009-02-17T21:52:00.003Z

I don't often pick up stuff from the news - I don't really consider what I write here as commentary - but this one I couldn't resist because it's such a good segue into something I wanted to talk about anyway.

In a way it's a little sad to think that these harsh economic times are starting to have an effect on even that most hallowed of engineer-driven organization. Still, there are those of us probably a little less melancholy - I can just imagine how much fun serial cloud hater, John C Dvorak, is having - "told you not to keep stuff in the cloud, one day they just switch it off and then what do you do?"

The decision to shelve a product is a difficult one, and one that inevitably leads to some degree of unpopularity both internally and externally. What I wanted to do here was take a look at what this article tells us about product management practices in general.

Good Stuff

• Willingness to take risk. Especially in these times, it can be tempting to stick to safe territory. That can be good advice, but what are you missing out on? Another way to look at it is when everybody else is playing it safe, it’s easier to stand out with something unique.
• Recognise the capability gain in products even if the product itself doesn’t work out in its current incarnation. Taking what they gained from Dodgeball and turning it into the Maps add-on Latitude is a great example of the sort of utility you can get from something that doesn’t work out in its initial form.
• Customer feedback. Accessing the community through techniques like product development blogs and using that knowledge to shape the features and user experience is a powerful way to make sure your system ends up meeting expectations as closely as possible.
• Attitude to failure. Viewing things that don’t work out as contributing to overall success rather than detracting from it helps you really internalise what you learn and encourages the innovation that fear of failure will crush.
• Internal testing. Eating your own dog food is always helpful, after all, if you find your product difficult to use then what are you asking of your customers?
• Opportunity cost. I liked the part where they consider the opportunity cost of a project as well as its real cost; i.e. what is the value of the things that the engineers are not working on because they’re working on this?

Bad Stuff

• Incentives. Paraphrasing, the article talks about difficulty attracting engineers to work on certain products. There are some reasonably well publicised shared reward schemes in place, and in this free market-like environment management needs to consider how it might need to provide external subsidies to make sure that less glamorous, but perhaps strategic, projects get attention too.
• Perpetual betas. I really couldn’t decide whether this one belonged in the good or the bad section, so I let the need for balanced paragraphs drive the decision. On one hand, I think there is no better way to refine user experience and focus the tail end of your development efforts on polishing the things users like the most, but on the other hand there are circumstances under which you need to show the commitment of a production-ready seal of approval.
• Communication. While not exclusively a product management issue, communication across geographical boundaries was a major issue here. This shows how important collocation is in a product development process.
• 20% time. I think we’ve all heard mixed reports about how this really works, but regardless of that, if you have a product that you’re committed to and have set targets for then you simply must formally assign it resources. Relying on product management to recruit people from their 20% time is setting it up for failure.

The article also mentions internal performance targets for new products, and in all businesses objectives and measures are the true meat of how product decisions are driven, so it's a shame we didn't learn more about that. Also some kudos due for how they ended Jaiku, the Twitter clone, it's good to see them pass that on to the community rather than just confine it to the shelf.

Giving References

2009-02-04T09:35:00.003Z

I was recently asked to act as a reference for a couple of people, and that got me thinking about the policy most organisations seem to have on this kind of thing (i.e. don’t do it). I can understand where this came from – representing the opinion of your company is a significant responsibility which has all too often been wielded by the undeserving.

Obviously I’d never advise flying in the face of company policy, but I really like giving references. In fact, I consider being able to give a good reference to be a lagging indicator of great success with people.
You see, if I’m able to give a good reference for someone, it means two things:

1. We’ve had a successful, effective working relationship in which we’ve helped each other deliver tangible value to the business.

2. I’ve been able to help that person develop professionally, and I’m now helping them to continue to grow their career.

Is there any greater satisfaction to be had in management? It’s why I do the job.

Why are they never happy?

2009-01-29T18:58:00.000Z

This is another one of those stories that my girlfriend will tell you explains why we don't get invited around for dinner more than once, but I'll tell you is dead essential to the world of technology becoming a better place - one manager at a time...

Anyway, I'm listening to this buddy complain about how busy he is, how hard he's working, how many hours his team are putting in, and how much stuff they're getting out - yet still his customers aren't happy. They're always finding something to complain about, pointing out the things that are missing rather than thanking him for the things that are there.

When I hear this, what I think most people are really saying is "my team and I are working really hard and really long hours doing the things we want to do for this customer". Whenever I've seen this issue on the floor, it has almost always been because the guys were dedicated and hardworking, but being dedicated to and working hard on the wrong stuff.

Think about it like this; if I really want A, and you give me B, C, D, E, F, and G, then I might be impressed by how busy you've been and how much work you've done, but I still won't thank you for it. Yes, B through G were relevant to my department and they were things I wanted done - no argument there - but if A was the thing that meant the difference between getting that big contract or not, or making this quarters numbers or not, then I would have happily traded it all to have just A.

I'm not saying that there aren't ungrateful people in the world, but I am saying that until you've adopted a partnership-based, customer-centric view of how you deliver technology, you haven't earned the right to complain about them.

Look frequently and carefully at your priorities, and compare them against what your business is being driven to achieve - you never know - you might even end up doing less work and having happier customers.

You ALWAYS Pay for Quality

2009-01-25T19:11:00.001Z

What if I told you that every single organization pays the same for any given piece of functionality? You'd probably call me crazy, and that would probably be because you know that it's not true, right? You've negotiated different rates, done projects in different locations, used different vendors, and cut different corners - and consequently, it cost you a different amount of cash each time. So I can't possibly be right, can I?

But before you condemn my point entirely, consider this:

If you're buying in software, consultancy, per-project services etc, then the primary levers you can adjust are commercial terms. You might feel exceptionally good about reducing the cost of a project by £5K or shaving a couple of points off margin, but do you really think those suppliers are going to produce exactly the same output for you and just make £5K less on the deal? Good luck with that.

If you're building your own products, and assuming you've got a fixed size team, then the primary levers you can adjust are tradeoffs and shortcuts - i.e. where you spend your time. How you pay for quality is then divided up between how much you're prepared to pay for now up front, and how much you're going to pay later - typically through higher operational or technical support effort (less automation), increased cost of the next features (technical debt), and opportunity cost (revenue lost to product unavailability).

So I'd suggest that you always pay for quality, whether you intend to or not, and I'd add that it's usually less painful to pay for it overtly through a proper project plan, than inadvertently through reactive support, interest payments from future projects, and your customers patience.

I'm not saying that you should never try to sharpen a deal to save some budget or take a shortcut to get a product out quicker, simply that you should keep in mind the longer term costs as you congratulate yourself on the shorter term gains.

Presentation Skills

2009-01-22T15:50:00.002Z

A few days ago I posted about how important it is to present ideas well, rather than hoping some budget and resources will be conjured up by the sheer strength of the idea alone. I was going to write something about presentation skills in general, but given that last post, I'm going to tailor this slightly more towards doing proposals - and for most of us - proposals make up the majority of the presentations we do that really matter anyway.

The thing to keep in mind about your proposal is your goal; you're not there to do a really great presentation, you're there to convert someone into a mad raving fan of your idea, and how you do that is going to depend on who you're dealing with.

Step 1 for me is always start with the audience, as everything else flows from this. It drives your content (which benefits you accentuate, which risks you focus on) your delivery method (maybe powerpoint slides in the boardroom is best, maybe a casual discussion over a coffee is sufficient), and even the duration you're going to talk for.

Never simply have "your presentation on project X". Instead, have "your presentation on project X for the architecture steering group", and "your presentation on project X for the accountants", and "your presentation on project X for the board" etc. They're all going to want to see a different mixture of things before they get behind it. With some of them, you might want budget, so you'll want to spend a slide on the concept, then spend the balance of the time breaking down cost and benefit over the next few years. From others you might need their change control approval, so the commercials will be lightweight with much more detail about how your system will be deployed and how it will integrate with the rest of the enterprise. You might even be displacing the roles of operational people through a tool you're driving forward, so these guys will want to know how new processes will change their tasks and you'll need to spend a fair bit of time addressing concerns about the team, their jobs, and training.

That's all very good, but something people often ask me is how to handle mixed audiences - when you have to present your proposal to a collection of people in different roles. My answer is - don't. It might seem like a smart idea, get a large number of areas out of the way in one go, but it simply is not effective. You cannot adequately address each persons individual focal points, and you risk convincing nobody.

Granted, sometimes you don't have a choice, and if that's the situation you're in (and you can't get out of it) then keep it brief, simple, factual, and to the point. But do have all the detailed information right there with you - the same stuff you'd have if you were doing the customized versions for each individual - and that way you'll be knowledgeable and organized should anyone wish to drill down into any of the areas they represent. If you've been asked to do something for a multidisciplinary audience like this, then chances are it's at some kind of regular session the participants have anyway, which means they'll have other business to discuss, which means you can't expect a long slot, which means your brevity will be appropriate.

On the mechanics of presenting - and by mechanics I mean how to deliver slides - I will let David Rose tell the story for me. It's worth watching the whole clip, but at the very least watch the last few minutes in which he covers slide design, stance and tone, and using presenter mode in powerpoint etc. Second only to actually doing it yourself, watching great presenters in action is the best way to get better at it, so I'd recommend watching a few more TED talks and definitely subscribing to Presentation Zen.

When drawing up a slide deck to accompany a presentation (note that a deck of slides is not your presentation) I think it's important to resist incorporating all the whizzy features powerpoint has to offer. Animations, sounds effects, charts, and clever transitions are all entertaining, but they distract from your message. Also, avoid complex, cluttered slides loaded with detail and don't, ever, just write a speech onto slides and read it out along with your audience. Slides aren't cue cards, they're important material which backs up what you say, reinforces your message, and helps add impact to the story you're telling face to face.

A couple of final points - firstly, when you're "considering your audience" and thinking about who you'll need to get onside with your idea, don't forgot about thought leaders. It's usually easy to spot the people who will be directly affected by your plan, and decision makers tend to be obvious too, but so many neglect this final category. By thought leaders, I mean people who might not have any official hierarchical power or authority to help you, but nonetheless they set opinion and exert an influence. Getting these people onside with your plans will make adoption everywhere else much smoother.

Last of all, don't overdo it. You are kidding yourself if you don't believe you're selling when you're presenting a plan, but you must also be realistic and present a balanced view of whatever it is you're asking us for. As executives, we know that there is no such thing as an idea with no downsides, and pitching that way will just make us suspicious.

Don't Tell the plan, Sell the plan

2009-01-20T22:04:00.001Z

One of the top sources of frustration for engineers is not getting management buy-in for proposals that just obviously need to be done. And here's the kicker, if they really understood the business benefit of your ideas, then management would be frustrated about all the awesome stuff they've been missing out on (and in fact tend to be anyway, it's just that they don't realize that you could have already solved them). Well it doesn't have to be that way, and guess what, you are the ones that need to do something about it - not them.

Just remember the title of this post - don't tell the plan, sell the plan. As engineers, we come up with some pretty clever stuff. We're up against hard problems and we deliver smart solutions. Where we get unstuck is when we go about sharing these with the decision makers that we need to get behind them.

As engineers, we tend to think that "selling" an idea somehow devalues it - sacrifices it's integrity - perhaps we think it is sneaky, cunning, or politically motivated to properly pitch something rather than just retell the facts behind it. But believe me, for better or for worse, the things that actually end up getting done in organizations aren't the best ideas, they are the ideas that were sold the best.

Don't get me wrong - you still need that great idea, a solid business case, and a well thought through execution plan - it's just that those things alone rarely turn your proposal into a funded priority.

On a personal note, back when I was an engineer I always told myself that I'd be different, however as my career moved towards decision maker, I found that I ended up pretty much doing the exact same thing. It's not through any lack of technical knowledge or understanding of how a proposed system will work, it's simply that I do a different job now - and that job is to weigh up all the possible things we could do as a team, work out which ones will make us the most successful, and then not lose sleep over the rest.

I bet you anything your boss does exactly the same thing - he's not an idiot that doesn't seem to trust you - he just wants to quickly understand the core concept, then make a judgement based on the real cost you want today vs the potential future return against a background of all the other things he might choose to do instead. And you know what? We don't need to know exactly how the older cached inserts will be invalidated in the remote nodes once a newer update is detected locally (or whatever) in order to make that decision - but if we ask you then you'd better know!

Even if you look at it cynically - you have the best idea anyway right, so why not have the best presented idea too? If it really is the best proposal, then don't you owe it your organization to give it the best chance of becoming reality? The only danger you'll face by being better at presenting your plans is that they're more likely to get done...

So I hope I've convinced you that the presentation of your ideas is as critical to their success as the ideas themselves, so the next most obvious question is how do you do it effectively? I was planning to write something on doing presentations anyway, so come back in a few days and I'll tell you what I think.

No Pain No Gain?

2009-01-16T20:43:00.001Z

For the past couple of days I've been helping out a friend get a new team organized, and during our of discussions in their canteen, I overheard something that epitomizes the complexity-aholics that really drive me mental.

A couple of data architects were talking about a consolidation project they were kicking off. "Can you believe what they're doing in the Japanese office? Their schema is so basic" one complained to the other. "Yeah, they clearly don't know what they're doing" his colleague replied, "they should leave it up to us." A couple more minutes of the same and I couldn't really let it go.

Deep breath...

Gathering up my most diplomatic tone, I asked the resident geniuses: "OK lads, if you're saying that they have a basic, simple solution which meets the business requirements, and you have a complex, difficult solution which meets the business requirements - then isn't it you who have something to learn from them?"

You see this is why I don't have any friends.

But seriously, what's the attraction to building big, creaky, behemoth solutions where something easy and elegant would meet everybody's expectations? Why is it that we seem to derive satisfaction from making our lives hard? Unless you're very lucky (lazy?), you'll have to troubleshoot, maintain, and support whatever you put in place.

I've noticed before that good engineers can build vast systems with webs of dependencies and many moving parts, but it's the great engineers who realize that this is a bad idea.

Outside of the gym, there are very few places in the world where you'll be better rewarded for increasing the effort necessary to achieve a given outcome.

The Parable of the Big Hands

2009-01-13T21:28:00.002Z

This is an old story, retrieved from the dusty recesses of my memory, but it's as relevant today as it ever was. Names changed to protect the incompetent...

A number of years ago I was called in to help an integration company who had been struggling for a number of weeks with a manufacturing client. They had delivered a factory automation system, introducing computer-controlled fabrication and handling machines, replacing the manually operated mechanical kind. The project had been justified upon increased productivity and reduced material wastage - yet reduced productivity and increased metal wastage were being observed. Not really the results anyone was hoping for, customers blaming the vendor, vendor blaming the users etc.

My job was to "troubleshoot the new software" and "find the bugs" before the manufacturing company started firing live ammo.

Step 1 was to get a coherent problem statement. It seemed that in the vendor's lab everything went according to plan - sheet metal machines could be reconfigured in tens of seconds via a new software load - yet at the factory it was taking minutes and materials were frequently being cut to the wrong dimensions.

So what was different between the factory and the lab? I was assured that there was no differences at all - machines had been swapped, the exact same builds had been uploaded to the machines at both sites, error logs had been downloaded etc. But clearly there was a difference, otherwise it would be working (or broken) in both places.

I figured there must be something environmental - how else could the exact same machine, software, and configuration work perfectly in one location and not another. "So, tell me about this factory" I asked. "Never been there" was the answer (believe it or not). Ah-ha!

So off we go to the warehouse/factory thingy. We never even got through the tour of the floor before we made the critical observation - these metalworking dudes have really big hands! If your job consists of throwing around huge blocks of steel, cranking big heavy leavers, and turning giant bolts, then you soon develop a pair of mitts like shovels.

Why was this important? Quite simple. The keys on the small numpad-type keyboard that had been specially made for entering the program parameters on each machine were a standard keyboard size, and were therefore simply far too small for the bunch-of-bannans fingers of the guys that had to use it. These guys are hardcore welders and whatnot, they don't have our delicate little programmer hands!

So what's the lesson here? Know Your Users. It's as simple as that. You have no business even touching a keyboard (of any size) until you know who your users are, where they work, how they'll interact with the system, and what successful use is. Remember - you're not building the software for yourself or your business analysts, you're building the software for them. Get to know them.

Making Decisions

2009-01-11T09:27:00.001Z

I thought I'd start with this as my first 'proper' post of the year, mainly because decision making isn't usually considered a skill in it's own right, despite being deceptively simple and so commonly done poorly.

Making a decision isn't a stand alone task, when you make one, you really do a small collection of things together. You make choice from some options, you make a plan and delegate it, and then you follow up and review. Most people think that once they've picked their option they're done, but like Drucker says, you haven't made a decision until you have an action plan - you've only made a choice.

So let's talk a little more about effective choosing, as that's the first hurdle to trip over. The behaviors you want to avoid here are essentially procrastination. How many times have you been staring an obvious course of action in the face (or been presented with a fantastic opportunity) but instead of making progress, you're taking bets on which is growing faster - your number of choices or the list of data points people want to see for each one! You have to reduce your number of options to a manageable amount (3 is a magic number if you're stuck for inspiration), and while you should never ignore great last minute ideas just for the sake of rules, most business don't fully appreciate the cost of inaction. The point marked "doing anything is better than nothing" creeps up a lot quicker than you think.

The next thing you have to worry about is the ballooning amount of information on each of your choices. Too many and you risk it being treated like a smorgasbord (can we have this from option A, and that from option B) or being paralyzed by the sheer volume of statistics and technicalities. Try removing things that are the same for all choices - for example, if A, B, and C are all similarly expensive, why compare cost? And how about things you don't feel strongly about? I don't know how many times I've seen stakeholders agonize over details like the difference in reusability between 2 choices in a decision, and then when asked where else they plan to use the technology, answer "nowhere". If it's an attribute you don't feel strongly about, then it's unlikely to influence your decision, so why clutter the process with it?

We've talked a lot about paring things down, and now here's something to add in. Don't forget that for every choice you're comparing you need to know what you don't get if you don't pick it. Most people are generally pretty good at stacking up the costs and benefits, but opportunity cost is often neglected.

Hurdle number 2 is action plans and delegation - the behavior you want to avoid here are those deja vu meetings where you all sit around and 'review' the decisions you made, wonder at the lack of progress and then agree to 'update next week' all over again. Once you've made your choice, don't let yourself off the hook until you've agreed who is actually going to do the work, and by when. Follow the usual rules on delegation - make sure whoever you're going to hold accountable clearly understands their obligations, is empowered to do the work, and has the available resources.

Don't forget to follow up. You spend your time on the things that are important to you, and you're not making unimportant decisions, are you? The things that you focus on will get done - keep track of what's happening, make major milestones public, and use your sponsorship to see it through. And once whatever it is has been delivered (or has crashed and burned), you're still not quite finished. Now comes the feedback loop - what did you learn? How can the experience improve the rest of the organization? What pitfalls can you now help others avoid?

And finally, you shouldn't be afraid to revisit your decisions and change your mind. You should be keeping an eye on your customers, your portfolio, and your team, and continually assessing what any changing circumstances may mean for you. That said, you must resist being flighty as you need to show commitment, but keep in mind that "because we said we would" isn't a business benefit to completing a project - it's a sleepwalker's justification for not being in touch with the market.

Welcome to 2009

2009-01-10T09:22:00.000Z

It's been 2009 for a little under 2 weeks now, and so far I can confidently say it will continue to be for around another 50 weeks or so. OK - I accept that's not my A material - moving swiftly on...

At this time of year it's customary to make a few new year's resolutions - fresh commitments you somehow feel renewed fervor towards keeping just because we incremented the year counter by 1 - and they almost always involve gym memberships.

If I take a brief look back at last year's resolutions I think I can safely say I hit my top priorities, but just as predicted, the exercise part seems to have slipped through the cracks. No matter, that's what these annual counter-flips are for, getting invigorated about your health for 30 days or so.

I'm always interested in hearing from people out there in the technical community for any reason, and from time to time I am lucky enough to receive some feedback on my posts (thanks by the way!). Over the last 12 months the articles that have generated the most interest have been on management, dealing with people, and being effective in organizations. It seems like you want to hear more about getting things done around technology than about the technology itself. Well there's some resolution input right there...

This year I'm going to try changing the balance of the content I write, aiming for a higher ratio on project management and professional disciplines without completely dropping the things I like to bring up on architecture and computer science. After all, a hundred little things happen every day, and it's just a matter of being a bit more selective about what's captured and shared.

So yeah, welcome to 2009, you can make it whatever you what you want it to be.

Christmas, Ad Rev, and Charity

2008-12-22T10:36:00.001Z

It's nearly Christmas (again!) and I hope you're way too busy enjoying the best this season has to offer to be reading this! With any luck, the systems and people you look after will all be humming along just fine, leaving you able to spend plenty of time with family and friends.

At times like this, it can be easy to forget that there are a lot of people out there who don't have it as sweet as we do. Where running water is a luxury, we'd find it difficult getting sympathy for our lengthy outstanding bugs list or early morning release woes!

I'm no Bono, but I like to contribute somehow whenever I come across a cause I believe in, but doing something specifically at this time of year has the extra bonus of reminding you of exactly how lucky you are during what can be a time of excesses.

Now, this blog has always been about the free exchange of ideas - a place where I can capture my original thoughts and experiences from the work I do, in the hope that it will help others with the work that they do. Whatever else it is, it's never been a money making exercise - and that's why I've never before tried to monetize the content, despite a growing readership (hello to both of you).

However, as of today, keener-eyed observers will notice a Google Ads box lurking surreptitiously in the right hand panel, ready to corrupt my benevolent posts with it's raw, unbridled, marketing vivacity. But don't worry - I haven't sold out yet! Here is the plan...

I'll run Google AdSense for the coming year, and around this time next year, donate all the revenue to charity. Every cent, and a moderate top up from myself.

It is impractical to pick a charity 12 months ahead, but the money will be donated via Global Giving or Just Giving (or maybe both) so that you can keep me honest. I'm thinking about supporting something which helps to establish education in 3rd world countries - that appeals to my "help those who are prepared to help themselves" philosophy - but will take suggestions from the community.

As well as cheques being posted next December, you can expect a running total, say, quarterly - and heck, if things go really well maybe we'll have a mid-year checkpoint and make a donation then too.

So, Merry Christmas and all the best for 2009, and remember, if you see something that interests you on the right hand side then give it a click - it's for a good cause!

See you next year
Eachan

When is the right time to launch?

2008-12-21T14:40:00.000Z

Ignoring commercial concerns such as marketing campaigns, complimentary events, and market conditions, this essentially comes down to weighing the quality/completeness of the product against getting to market sooner.

The quality over time to market side has been advocated a few times in recent episodes of Paul Boag's podcast, so I figured I would speak up for the other side. But before I do, let me just say that I don't think there is a right answer to this and, as Paul also conceded, it depends on if your application's success requires a land grab or not.

I am, by nature, an "early and often" man - and that's kind of a vote for time to market over quality. I say kind of because I think it would be more accurate to say that it's a vote for time to market over perfection.

For me, the "often" part is inextricable linked to the "early" part. If you can show a pattern of frequent, regular improvement and feature releases, then you can afford to ship with less on day one. Users can often be more forgiving when see things turned around quickly, and new things regularly appearing can even become a reason in itself to return to the site more often.

Quality is still a factor, but it isn't as black and white as I've been hearing it presented. In the early days of a new product, I think where you spend your time is more important than overall quality. You should be very confident in anything that deals with accounts, real money, or the custody of user's data before shipping. I would argue that getting those areas right for launch at the expense of other parts of the system is better than a more even systemwide standard of quality. You always have limited resources. Spend the most where it counts the most.

And finally, openness. Be truthful and transparent with your users. Start a blog about development progress and the issues you've run into. Provide feedback mechanisms, and actually get back to people who've taken the time to share their thoughts - with something material too, not just an automated thankyou. Send out proactive notifications ahead of impactful changes and after unplanned events. Stick 'beta' labels on things you're shipping early - it'll keep user's blood pressure down, and you might be surprised by how much of the community is prepared to help.

I am aware that I haven't actually answered the question that lends this post its title. I don't know if there even is an off-the-shelf answer, but I hope that I've at least given you some more ideas on how to make the right decision for yourself.

Constraints and Creativity

2008-12-19T16:54:00.001Z

Does the greatest creativity come from unlimited choice; a totally blank canvas with every possible option and no restrictions, or from difficult circumstances; where pressure is high, the problems are acute and all the easy avenues are closed?

History has proven necessity to be the mother of invention time and time again, and I've personally seen a lot of otherwise-excellent people become paralyzed when faced with too much opportunity.

The current economic circumstances are bringing this into somewhat sharp relief. From time to time I get approached by other entrepreneurs with an idea they want to develop - usually only a couple of times a year, but in the last couple of quarters I've seen half a dozen. There is something about this credit crunch...

Conditions are right for the sorts of things I've seen; internet based, self-service, person to person applications that look for efficiency by cutting out middlemen and using smarter fulfillment. They're all great ideas and I really wish I had the time to help them all out.

Returning to the point, would you get this kind thinking in times of plenty? Who knows, maybe something good will come out of this credit crunch after all.

Tradeoffs - You're Doing It Wrong

2008-12-17T12:44:00.001Z

Our time is always our scarcest resource, and most of the web companies I know are perpetually bristling with more ideas than they could ever manage to ship. We always wish we could do them all, but at the same time we know deep down that it isn't realistic. Under these circumstances, tradeoffs are an inevitable and perfectly normal part of life.

The secret to doing this well is in what things you weigh against each other when deciding how to spend the resources you have.

It's so easy to start down the slippery slope of trading features off against the underlying technologies that support them. Fall into this trap and you'll get a small surge of customer-facing things out quickly, but stay with the pattern (easy to do - as it does reward you well in the short term) and you'll soon find yourself paralyzed by technical debt; struggling against continuously increasing time to market and building on top of a shaky, unreliable platform.

Instead, try trading off a feature and it's supporting technology against another feature and it's supporting technology. This isn't a carte blanche ticket for building hopelessly over-provisioned systems - it's simply about setting yourself up in the most appropriate way for future requirements and scalability, rather than foregoing whatever platform work that would have been diligent and starting off behind the curve.

Trading features off against their underlying technology is not just short sighted, it's also unrealistic. You see, the fact is that all customer-facing features have to be maintained and supported anyway, no matter how badly we wish we could spend the time on other things. And right up front, when we first build each feature, we have the most control we're ever going to have over how difficult and expensive that's going to be for us.

This is a philosophy that has to make it all the way back to the start of the decision making process. When considering options, knowing the true cost of ownership is as vital as the revenue opportunity and the launch cost. Remember - you build something once, but you maintain it for life.

Failure Modes

2008-12-13T13:24:00.001Z

Whenever we're building a product, we've got to keep in mind what might go wrong, rather than just catering mindlessly to the functional spec. That's because specifications are largely written for a parallel universe where everything goes as planned and nothing ever breaks; while we must write software that runs right here in this universe, with all it's unpredictability, unintended consequences, and poorly behaved users.

Oh and as I've said in the past, if you have a business owner that actually specs for failure modes, kiss them passionately now and never let them go. But for the rest of us, maybe it would help us keep failure modes in the forefront of our minds as we worked if we came up with some simplified categories to keep track of. How about internal, external, and human?

Internal
I'm not going to say too much about internal failure modes, because they are both the most commonly considered types and they have the most existing solutions out there.

You could sum up internal failures by imagining your code operating autonomously in a closed environment. What might go wrong? You are essentially catering for quality here, and we have all sorts of test environments and unit tests to combat defects we might accidentally introduce through our own artifacts.

External
The key difference between external and internal failure modes is precisely what I said above - you are imagining that your code is operating in perfect isolation. If you are reading this, then I sincerely hope you rolled your eyes at that thought.

Let's assume that integration is part of internal, and we only start talking external forces when our product is out there running online. What might go wrong?

Occasionally I meet teams that are pretty good detecting and reacting to external failures and it pleases me greatly. Let's consider some examples; what if an external price list that your system refers to goes down? How about if a service intended to validate addresses becomes a black hole? What if you lose your entire internet connection?

Those examples are all about blackouts - total and obvious removal of service - so things are conspicuous by their absence. For bonus points, how are you at spotting brownouts? That's when things are 'up' but still broken in a very critical way, and the results can sometimes cost you far more than a blackout, as they can go undetected for a while...

Easy example - you subscribe to a feed for up-to-the-minute foreign exchange rates. For performance reasons, you probably store the most recent values for each currency you use in a cache or database, and read it from there per transaction. What happens if you stop receiving the feed? You could keep transacting for a very long time before you notice, and you will have either disadvantaged yourself or your customers by using out of date rates - neither of which is desirable.

Perhaps the feed didn't even stop. Perhaps the schema changed, in which case you'd still see a regular drop of data if you were monitoring the consuming interface, but you'd have unusable data - or worse - be inserting the wrong values against each currency.

Human
Human failure modes are the least catered for in our profession, regardless of the fact they're just as inevitable and just as expensive. You could argue that 'human' is just another type of external failure, but I consider it fundamentally different due to one simple word - "oops".

To err is human and all that junk. We do stuff like set parameters incorrectly, turn off the wrong server, pull out the wrong disk, plug in the wrong cable, ignore system requirements etc - all with the best of intentions.

So what would happen if, say, a live application server is misconfigured to use a development database and then you unknowingly unleash real users upon it? You could spend a very long time troubleshooting it, or worse still it might actually work - and thinking about brownouts - how long will it be before you noticed? For users who'd attached to that node, where will all their changes be, and how will you merge that back into the 'real' live data?

Humans can also accidentally not do things which have consequences for our system too. Consider our feed example - perhaps we just forgot to renew the subscription, and so we're getting stale or no data even though the system has done everything it was designed to do. Hang on, who was in charge of updating those SSL certificates?

Perhaps we don't think about maintenance mistakes up front because whenever we build something, we always picture ourselves performing the operational tasks. And to us, the steps are obvious and we're performing them, in a simplified world in our heads, without any other distractions competing for our attention. Again - not real life.

And so...
All of these things can be monitored, tested for, and caught. In our forex example, you might check the age of the data every time you read the exchange rate value in preparation for a transaction, and fail it if it exceeds a certain threshold (or just watch the age in a separate process).

In our live server with test data example, you might mandate that systems and data sources state what mode they're in (test, live, demo, etc) in their connection string - better yet generate an alert if there is a state mismatch in the stack (or segment your network so communication is not possible).

The question isn't are there solutions; the question is how far is far enough?

As long as you think about failure modes in whatever way works for you, and make a pragmatic judgement on each risk using likelihood and impact to determine how many monitors and fail-safes it's worth building in, then you'll have done your job significantly better than the vast majority of engineers out there - and your customers will thank you for it with their business.

Speed is in the Eye of the Beholder

2008-12-02T21:36:00.001Z

Yesterday I was coming through Heathrow again, and while I was waiting for my luggage to surprise me by actually turning up, I was ruminating on performance, user experience, and the iris scanner (old BBC link). For those who haven't had the pleasure, we're talking about a biometric border security system based on the unique patterns in an individual's eye, implemented at certain UK borders as an optional alternative to traverse passport security in a self-service way.

Flying internationally at least once a month for the last couple of years, I've been a regular user of the iris scanner since it's early trials. Like all new technology, it went through a period of unreliability and rapid change in its early days, but its been pretty good for a while now. Step into booth. Have picture of eyes taken. Enter the UK. Awesome.

The only problem is that...
It...
Is...
Just...
Too...
Slow...

Don't get me wrong - you don't have to queue up for it (yet) and you do spend less time overall at the border - it's just that, for a high tech biometric solution, it seems to take an awful long time to look me up once it snaps what it considers to be a usable image of my retinas. You know, in real time it isn't even that expensive - probably an average of about 5 or 6 seconds - but it's just long enough for you to want to keep moving, hesitate, grumble to yourself, briefly wonder if it's broken, and then the gates open up.

It occurred to me that maybe my expectations as a user were perhaps unfair, but then quick as a flash I realized - hey, I'm the user here, so don't I set the standard?

So that's when I started to think about how the performance of this system could be improved, and caught myself falling straight into rookie trap number 1:

I started making assumptions about the implementation and thinking about what solutions could be put in place. I figured there must be some processing time incurred when the retinal image is captured as the machine picks out the patterns in my eyes. That's a local CPU power constraint. Once my retinal patterns are converted into a machine-readable representation, there must be some kind of DB lookup to find a match. Once I've been uniquely identified, I imagine there will be some other data sources to be consulted - checking against the various lists we use to keep terrorists and whatnot at bay.

Well this all sounds very read intensive, so it's a good case for having lots of local replicas (the iris machines are spread out over a number of terminals). Each unique 'eye print' is rarely going to come through more often than days or weeks apart, so most forms of request caching won't help us much with that. Of course there is also write load - we've got to keep track of who crossed the border when and what the result was - but we can delay those without much penalty, as it is the reads that keep us standing in that booth. Maybe we could even periodically import our border security lists to a local service if we observe a significant network cost in checking against them for each individual scanned through the gate (assuming they are currently maintained remotely by other agencies).

Ignoring the fact that I'm making all sorts of horrible guesses about how this system currently works, these seem like reasonably sensible patterns to me, so what's the rookie trap?

Simply that I didn't start by understanding the basic problem I am trying to solve, instead rolling up my sleeves and diving straight into the technical complexities. In doing so, I might have overlooked a plain, simple solution and that's usually bad - why solve a problem in a complex, expensive way when a simple, cheap way will do just as well?

In this example, the problem is not that 5 seconds is an unacceptable time to process the image, compare the data against the registered users, and then pass all the additional border security checks - the problem is that 5 seconds feels like a long time when you're standing in a glass box waiting for it all to happen!

So what else might we have done instead of a major re-architecture of the iris back end? How about just lengthening the booth to form slightly more of a corridor, with the scanner at one end and the gates at the other? With this simple trick, it still takes 5 seconds before the gates can open, but it doesn't feel like it - you haven't been standing still waiting, there is a sensation of progress.

Same level of system performance, very different user experience.

General engineering lesson 1 - customer experience is king. Monitoring and other data driven metrics are important, but how it really looks and feels to your users matters way more than whatever you can prove with data. They'll judge you by their own experiences of your system, not by your reports.

General engineering lesson 2 - you don't have to solve every problem. Sometimes it's better/cheaper/faster to neutralize or work around the issue instead. You might not be able to build a different floor layout, but you can do things like transfer data in the background (not locking out the UI) and show progress 'holding' pages during long searches etc.

Oh and just for the record, I really have no idea how the back end of the iris system works - you are supposed to be seeing the analogy here...

Change Control

2008-12-01T08:21:00.000Z

With another December rolling around already, we head into that risky territory that we must navigate once a year - seasonal trading for many companies is picking up, yet now is the time when most support staff are trying as hard as possible to be on holiday. A tricky predicament - and what better time to talk about change control?

A lot of people - particularly fellow agilists - regard change control as a pointless, work-creationist, bureaucratic impediment to doing actual work. If it's irresponsibly applied, then I'd have to agree with them, but there are ways to implement change control that will add value to what you do without progress grinding to a halt amid kilometers of red tape.

Firstly, let's talk about why we'd bother in the first place. What's in it for us, and what's in it for the organization, to have some form of change control in place? Talking about it from this perspective (i.e. what we want to get out of it) means that whatever you do for change control is much more likely to deliver the benefits - because you have a goal in mind.

Here's what I look for in a change control process:
• The discipline of documenting a plan, even in rough steps, forces people to think through what they're doing and can uncover gotchas before they bite.
• Making the proposed change visible to other teams exposes any dependencies and technical/resource conflicts with parallel work.
• Making the proposed changes visible to the business makes sure the true impact to customers is taken into consideration and appropriate communication planned.
• Keeping simple records (such as plan vs actual steps taken) can contribute significantly to knowledge bases about the system and how to own it.
• Capturing basic information about the proposed change and circulating it to stakeholders makes sure balanced risk assessments are made when we need to decide when and how to implement something, and how much to spend on mitigations.

Ultimately, this all adds up to confidence in the activities the team are undertaking, and over time, will lead to less late nights and less reactive work.

And here are my rules of thumb for how change control should be implemented:
• Never let any process get in the way of doing the bloody obvious. If someone's on fire, you don't go and get the first aid manual and look up 'F' for fire.
• Change control can be granular, with stricter controls on more critical elements (like settlements), and a more flexible approach on lower impact or easier to restore elements (like content and feeds).
• Don't just take a off the shelf or copy another organization verbatim - this is the kind of thing that got change control the reputation it has - think about what you need and do something appropriate.
• Start small and grow up - it's easy to add more diligence where it proves necessary, but much more difficult to relax controls on areas where progress is pointlessly restricted.

So what do you actually do? As I said above, start off lightweight and cheap - a spreadsheet should do it, there isn't always the need for a huge workflow management database. Make a simple template and make sure you circulate it the way information is best disseminated in your organization (email, intranet, pinned on the wall - whatever gets it seen). Borrow ideas from your industry peers, but keep in mind the outcomes that best serve your circumstances. Most of all, identify the right stakeholders for each area of the system, appreciate the different requirements the applications under your stewardship have, and get into the habit of weighting risk and thinking before you act.

Here's to peace of mind - let's spend December at christmas parties, not postmortems!

Root Cause Analysis

2008-11-21T09:40:00.001Z

To help me kill some time at an airport (which seems to be my second job these days), let me reach into my wardrobe of soap-box issues and pick something out. Ah, root cause analysis, here we go.

In my opinion, proper root cause analysis is the most important part of any operational support process.

Having a professional, predictable response and the skills to restore service quickly are critical - but you have to ensure that your support processes don't stop there. If they do, then you're simply doomed to let history repeat itself, and this means more downtime, more reactive firefighting, and less satisfied customers.

Good root cause analysis takes into account the entire event - systemwide conditions, the teams response, the available data, the policies applied - not just the technical issue which triggered the fault, and looks for ways to reduce the likelihood of recurrence.

Doing root cause analysis properly can be expensive, because you don't need to get to the bottom of why it happened this time, it's why it keeps happening, and why the system was susceptible to the issue in the first place that you need to uncover to really add future value. Think of the time spent on it as an investment in availability, freeing up your team to work more strategically (as well as enjoy their jobs more), and happier users (which oddly seems to make happier engineers).

But what you learn by doing this isn't really worth the time you spend on it without the organizational discipline to follow up with real changes. If you're truly tracing issues back to their root, you'd be surprised how many are the result of a chain of events that could stretch right back to the earliest phases in projects. This needs commitment.

If you make money out of responding to problems then you'll probably want to ignore my advice. There is a whole industry of IT suppliers whose core business lives here, and while it's an admirable pursuit, don't take the habit with you when you join an internal team!

The Confidence-o-Meter

2008-11-19T18:11:00.001Z

A while back we had a project that just seemed destined to face every type of adversity right from the outset (we've all had at least one). Being a new line of business for us, we didn't even have a customer team when work needed to begin! It was going downhill, and with strict regulatory deadlines to meet, we needed to get it back on track. Additional complications arose because the team was new, and as such were still gelling together as they tackled the work. Let's throw in an unusually high dosage of the usual incomplete specifications, changing requirements, and unclear ownership that regularly plague software projects and you have a recipe for a pretty epic train smash.

They say necessity is the mother of invention (Plato?) and there was certainly no shortage of necessity here, so we got on with some serious invention.

The Problem

We needed a way to bring the issues out in the open, in a way that a newly forming team can take ownership of them without accusation and defensiveness creeping in. More urgently still, we had conflicting views on the status of each of the workstreams.

It seemed sensible to start by simply getting in the same room and getting on the same page. To give the time some basic structure, we borrowed some concepts from the planning poker process. There were some basic ideas that made sense for us - getting the team together around a problem, gathering opinion in a way that prevents strong personalities dominating the outcome, and using outliers to zero in on hidden information. As an added bonus, the quasi-familiarity of the process gave the team a sense of purpose and went some way to dispel hostility in a high pressure environment.

The Solution

We started by scheduling a weekly session and sticking to it. Sounds simple, but when the world is coming down around your ears, it is too easy to get caught up in all the reactivity and not make the space to think through what you're doing.

We set aside some time at the end of each week, and our format for the session was fairly simple:
• All the members of the delivery team state their level of confidence that the project will hit its deadline by calling out a number between 1 and 10, 1 being definitely not, 10 being definitely so.
• We record the numbers, then give the lowest and highest numbers an opportunity to speak, uncovering what they know that makes them so much more or less confident than the rest of us. This way the whole group learned about (or dispelled) issues and opportunities they may have been unaware of.
• In light of the new information from the outliers, everyone has an opportunity to revise their confidence estimate. This is recorded in a spreadsheet which lends this post its title.
• Finally we took some time to talk over the most painful issues or obvious opportunities in the project and the team, then picked the most critical one and committed to changing it over the coming week. We also reviewed what we promised ourselves we'd change last week.

Through this discipline the team made, in real time, a bunch of very positive changes to themselves and how they work together without having to stop working and reorganize. We also had a very important index that we could use to gauge exactly how worried we should be - which is pretty important from a risk mitigation perspective. The trend was important too - we were able to observe confidence rise over time, and respond rapidly to the dips which indicated that the team had encountered something that they were worried would prevent them from delivering.

The Outcome

This exercise gained us a much more stable view of a chaotic process, and let us start picking off improvements as we worked. By making the time to do this and being transparent with the discussions and decisions, the team felt more confident and in control of their work - which always helps morale.

Because we were able to give the business a coherent, accurate assessment of where we were at, we were able to confidently ask for the right support - it was easy to show where the rest of the organization could help, and demonstrate the impact if they didn't meet their obligations to us.

In summary, we got our issues out in the open and got our project back on the rails. And by the time we got together for our post-implimentation retrospective, we were pleasantly surprised by how many of our most critical problems we'd already identified and fixed. If you're in a tough spot with a significant piece of work and a fixed deadline, consider giving something like this a try - I think it will work alongside any development methodology.

Cloud Computing Isn't...

2008-11-17T11:25:00.001Z

Thought for the day - when does a new idea gain enough structure to graduate from meme to tangible concept? Is there some quorum of 'experts' that need to agree on its shape? Perhaps we need a minimum number of books written on the topic, or a certain number of vendors packaging the idea up for sale? Or maybe it is as simple as all of us trying it our own way for long enough for observable patterns to emerge?

We might have already crossed this bridge with cloud computing thanks to the accelerated uptake of robust platforms such as EC2 and App Engine (and the adherence to theme of newer offerings like Azure), but there is still a lot of residual confusion that we might start to mop up if we were so inclined.

The first thing we might stop doing is retrospectively claiming any sort of non-local activity as cloud computing. What's that? You've been using Gmail or Hotmail for years? No. sorry. You are not an ahead-of-the-curve early adopter, you are just a guy who has been using a free web based email system for a while.

Before the inevitable torrent of angry emails rains down upon my inbox, let's pause to think about what we're trying to achieve here. Does classifying the likes of Hotmail and - well, insert your favorite SaaS here - as cloud computing help or hinder the adoption and development of cloud technology? I think we probably establish these analogies because we believe that the familiarity we create by associating a trusted old favorite with a radical new concept may add comfort to perceived risks. But what about the downside of such a broad classification?

These systems are typically associated with a very narrow band of functionality (for example, sending and receiving email or storing and displaying photos) and are freely available (supported by advertising or other 2nd order revenue). They tend to lack the flexibility, identity, and SLA that an enterprise demands. This analogy may well be restricting adoption in the popular mind. Besides, where do you draw the line? Reading this blog? Clicking 'submit' on a web form? Accessing a resource in another office on your corporate WAN? I'm not knocking anyone's SaaS, in fact the noble pursuits that are our traditional online hosted email and storage systems have been significant contributing forces in the development of the platforms that made the whole cloud computing idea possible.

So, if a lot of our common everyday garden variety SaaS != a good way to talk about cloud computing, then what is?

Let's consider cloud computing from the perspective of the paradigm shift we're trying to create. How about cloud computing as taking resources (compute power and data) typically executed and stored in the corporate-owned datacenter, and deploying them into a shared (but not necessarily public) platform which abstracts access to, and responsibility for, the lower layers of computing.

That may well be the winner of Most Cumbersome Sentence 2008, but I feel like it captures the essence to a certain degree. Let's test our monster sentence against some of the other attributes of cloud computing - again from the perspective of what you're actually doing in a commercial and operational sense when you build a system on a cloud platform:

• Outsourcing concern over cooling, power, bandwidth, and all the other computer room primitives.
• Outsourcing the basic maintenance of the underlying operating systems and hardware.
• Converting a fixed capital outlay into a variable operational expense.
• Moving to a designed ignorance of the infrastructure (from a software environment perspective).
• Leveraging someone else's existing investment in capacity, reach, availability, bandwidth, and CPU power.
• Running a system in which cost of ownership can grow and shrink inline with it's popularity.

I think talking about the cloud in this way not only tells us what it is, but also a little about what we can do with it and why we'd want to. If you read this far and still disagree, then enable your caps lock and fire away!

My agile construction yard

2008-11-11T11:37:00.001Z

I have been successfully practicing agile for quite a while now, and I've always believed that, given a pragmatic application of the principles behind it, it can be used to manage any process. Mind you, having only ever tried to deliver software with agile, this remained personally unproven. Gauntlet?

So I figured that if I am going to keep going on about it, I am going to have to put my money (and my house) where my mouth is. So I did, and this is the story...

The requirements

I wanted a bunch of work done on my house - extending a room, replacing the whole fence line, building a new retaining wall, laying a stonework patio, new roof drainage, building a new BBQ area, and some interior layout changes - and I thought it's now or never. I spoke to the builder and told him all about agile, lean thinking, project management practices like SCRUM and XP, and how it can benefit both of us in delivering House 2.0. He asked if he could speak to the responsible adult who looks after me. "Great, a waterfall builder" I say to myself as I try not to be offended by the 'responsible adult' quip.

But we strike a deal; he's going to do big up front drawings and a quote, and we'll proceed my way at my own risk and responsibility. The game is on.

The first thing we do is run through all the things I want done, which ones are most important to me, and roughly how I want them all to look. I guess you could call this the vision setting. Then my contractor asks me the few big questions; the materials I want to use, budget, and when I want it ready by. He makes some high level estimations on time and cost, based on which I rearrange my priorities to get a few quick wins. We have a backlog.

The project

Our first 'sprint' is the fence line. We come across our first unknown already - the uprights that hold the fence into the ground are concreted in and we either have to take twice as long to tear them out, or build the new fence on the old posts. Direct contact throughout the process and transparency of information ensures that we make the decision together, as customer and delivery team, so that neither of us is left with the consequences of unforeseen situations. I want the benefit of new posts so I'm quite happy to eat the costs put forward.

Next we do the retaining wall, and we have a quick standup to go over the details - we need to decide on a few details up exact height and the type of plants growing across the top. Since the fence has been done I go with some sandy bricks that match the uprights and the wall is constructed without incident. The next thing we're going to tackle is the BBQ area; however, beyond that the roadmap calls for the room extension and so we need to apply for planning consent in order to get the approval in time. Agile doesn't mean no paperwork and no planning, it means doing just enough just in time for when you need it.

Now we hit our first dependency - the patio must be laid first before we can build the BBQ area. That's cool, and through our brief daily catchups, we come up with an ideal layout and pick out some nifty blocks. A bit of bad weather slows down the cementing phase slightly, but we're expecting that - this is England. We use the opportunity to draft some drawings for the room extension and get the consent application lodged.

It's BBQ building time. I've been thinking about it since we started the project, and I decided I wanted to change it. The grill was originally going up against one wall, but wouldn't it be much more fun if it was right in the middle so everyone could stand 360 around it and grill their own meat? You bet it would. We built a couple of examples out of loose bricks (prototypes?) and then settled on a final design. It takes a bit more stone than the original idea, but it's way more awesome.

Then our project suffered it's first major setback - the planning consent process uncovers that a whole lot of structural reinforcement will be needed if they're going to approve the extension. That pretty much triples the cost of adding the extra space. Is it still worth it at triple price? Not to me. Lucky we didn't invest in a lot of architect's drawings and interior design ideas, they'd be wasted now (specifications are inventory and inventory is a liability). So we start talking about alternatives, and come up with a plan to create the new space as storage and wardrobes - not exactly what I had in mind up front, but at less than half the original cost it still delivers 'business value'.

The retrospective

So how did it all turn out? Well, as a customer, I felt involved and informed throughout the whole process, and the immediate future was usually quite predictable. Throughout the project I had the opportunity to adjust and refine my requirements as I saw the work progress, and I always made informed tradeoffs whenever issues arose. I am happy, the builder is happy, and I got exactly what I wanted - even though what I really wanted ended up quite different to what I thought I wanted when we started.

Oh and if anyone wants a good building contractor in Surrey...

Cost in the Cloud

2008-11-07T09:58:00.000Z

Cost is slated as benefit number 1 in most of the cloud fanboy buzz, and they're mostly right, usage-based and CPU-time billing models do mean you don't have tons of up front capital assets to buy - but that's not the same thing as saying all you cost problems are magically solved. You should still be concerned about cost - except now you're thinking about expensive operations and excess load.

Code efficiency sometimes isn't as acute a concern on a traditional hardware platform because you have to buy all the computers you'll need to meet peak load, and keep them running even when you're not at peak. This way you usually have an amount of free capacity floating around to absorb less-than-efficient code, and of course when you're at capacity there is a natural ceiling right there anyway.

Not so in the cloud. That runaway process is no longer hidden away inside a fixed cost, it is now directly costing you, for example, 40c an hour. If that doesn't scare you, then consider it as $3504 per year - that's for once instance, how about a bigger system of 10 or 15 instances? Now you're easily besting $35K and $52K for a process that isn't adding proportionate (or at worst, any) value to your business.

Yikes. So stay on guard against rogue process, think carefully about regularly scheduled jobs, and don't create expensive operations that are triggered by cheap events (like multiple reads from multiple databases for a simple page view) if you can avoid it. When you are designing a system to run on a cloud platform, your decisions will have a significant impact on the cost of running the software.

Eachan's Famous Interview Questions - Part II

2008-11-03T14:44:00.002Z

A while back I posted some engineering manager/leader interview questions that I frequently use - designed to test how someone think, what their priorities are, and how they'd approach the job - rather than whether or not they can do the job at all. As I said back then, if you're at a senior level and you're testing to see if someone is capable of the basic job all, then you're doing it wrong (rely on robust screening at the widest point - your time is valuable).

Like everything else, this is subject to continuous improvement (agile interviewing - we're getting better at it by doing it) and with more repetition you tend to develop more ways of sizing people up in a 1 hour meeting. So here is iteration 2:

1. What is the role of a project leader? Depending on your favorite SDLC, that might be a project manager, SCRUM master, or team leader - but what you're looking for is a distinction between line management (the maintenance of the team) and project management (the delivery of the work).

[You might not make such distinctions in your organization, it is important to note that all these questions are intended to highlight what an individuals natural style is, not to outline a 'right' way to do it.]

2. Walk through the key events in the SDLC and explain the importance of each step. It is unlikely any candidate (except maybe an internal applicant) is going to nail down every detail of your SDLC, but what you're hoping to see is a solid, basic understanding of how ideas are converted into working software. It seems overly simple, but you'd be surprised how many people, even those who have been in the industry many years, are really uneasy about this. Award extra credit for 'importances' that benefit the team as well as the business (for example - product demos are good for the team's morale etc).

3. Who is in a team? Another dead simple one, and what you are testing for is engagement, inclusion, and transparency. Everyone will tell you developers, and usually testers, but do they include NFRs like architects? Supporting trades like business analysts and project managers? How about the customer him/herself?

4. What is velocity, how would you calculate it, and why would you want to know? Their ability to judge what their team is capable of is the key factual basis to the promises they'll make and how they'll monitor teams performance and be able to help them improve over iterations.

5. Explain the software triangle. This is another one of my favorites - because the fundamental relationship between time, scope, and cost is as real a law as gravity yet so many engineering professionals still seem to live in some kind of weird denial. Perhaps afraid of falling off the edge of the earth? Nonetheless, someone who won't get swept along on a romanticized story of One Man's Heroic Triumph Over Project X will make sure you keep a sustainable team and not fall into the sarlacc pit of over-promising and under-delivering. You can also use this question as a springboard to explore how they'd negotiate tradeoffs with customers and how they'd make the costs of decisions visible.

6. How would you handle a team coming off a failed project? No one will ever preside over a flawless team that never drops anything, so being able to handle this effectively is a critical skill. For me, the ideal candidates have some answers to both 'what can we do to recover morale and re-motivate the team?' and 'what went wrong and how can we sidestep it next time?'

7. What's the definition of done? You need you're on definition of done, but I'm always looking for people who include testing, documentation, successful build and integration, failure scenarios, maintenance plans etc in their definitions. How about as far as commercial success? You can easily wander into estimation from here - protecting time to build sustainable software is a vital prerequisite to actually doing it.

8. Who are your stakeholders? Another one that varies terrifically from place to place. Don't let them get away with 'the business' because remember, you're testing for a depth of understanding of How It's Done. Do they include system administrators? How about operators? Customers themselves? Do they prefer to work in a close, personal way with these individuals, or to abstract them away behind business analysts and product managers? It is all valuable decision making data for you.

9. Imagine you could wave a magic wand and either make your products recover from failure 50% quicker, or make them 50% less likely to fail in the first place - which would you choose? A bit of a wily questions, but one that will expose their strategic vs operational bias. More interesting is the discussion around why they chose the way they chose.

10. Imagine you have a black box which fails regularly. You may chose to have basic observation in real time or vastly detailed statistics on a 24 hour delay - which would you choose? Alternatively, you can ask this one in a less course way by looking for examples of different types of system and the circumstances in which each choice might be appropriate. This type of question, along with number 9, can also demonstrate their ability to theorize and generalize (while appreciating that they're doing so) without studying the details of a specific example. This is usually indicative of experience.

There are no 'right' or 'wrong' answers to most of these questions (although I would argue there are 'better' answers to some), just answers that will suit you well, and answers that are less compatible. Ultimately, exploring people in this way will help you predict how they'll perform given autonomy - and why give yourself more governance than you need to do?

Fail More, Neo

2008-10-27T12:14:00.001Z

If you expect everything to work all the time, if you believe everything will be perfect first time around, if you think everything you try and every idea you have will always be brilliant - then you are living in some kind of delusional hyper-fantasy. Stay there, trust me, it's better than out here.

But if you decide to take the red pill, and join us here in the real world, there are a few things you should know that will make your integration easier.

Firstly, machines have not taken over the world, and human beings are not just a kind of great big box of Duracell batteries to them. But this is mostly because we can't make machines awesome enough yet, see below...

Out here in the real world, we mostly learn by doing things (until we can make those machines that beam kung-fu directly into our brains). We try something out, observe the results, and then we do it again with some slight variations. Those little tweaks we make as we try again and again are based on the pain we feel each time it doesn't work out. We call this a feedback loop, and we've learned this way for thousands of years - if there was a better way to do it, a way we could just "get it right first time", then trust me, we'd be all over that by now!

We'll be honest with you - you can, even in the real world, get by with very little of such trial-and-error experience if you want. A well-established pattern of mundane mediocrity leading directly to obsolescence is readily available. In fact, you'd be surprised how popular a choice this actually is! Let's call it the grey pill.

Assuming you don't fancy the drab, lackluster, second-rate existence the grey pill guarantees, what else can you do with the humans under your command?

Firstly - and most importantly - allow them to try. Don't expect every idea to be killer, or everything to work out first time around. Allow - no wait - require experimentation and iteration. It is the number 1 way your humans will expand their understanding, and believe it or not, a string of failures is the shortest path to that one thing that does work brilliantly. Tom Watson, an old golf-playing human we have out here, once said “if you want to succeed, double your failure rate”.

You might notice that some of your humans seems a little reluctant to embrace the idea - especially those recently defrosted from 'grey pill' institutions. How can you spruce them up into lean, mean, mistake-making machines?

People often fear the consequences of failure more than failure itself. So the best course of action is to make the consequences of failure something to look forward to - not something to hide from, and cover up. Why not try celebrating failure? If one of your humans has an idea, tries it out, and then brings back some knowledge and experience to the rest of the team, then you are much better off than you were before. Make a big deal out of it, demonstrate to the rest of the team that it's OK to try. Stretching yourself won't be punished.

That doesn't mean rewarding scattergun bravado - what you're trying to encourage is a culture of balanced risk and methodical approach.

I like the old saying "fail fast, fail cheap" because as a statement, it gives permission to try new things, yet it is also prescribes some basic guidelines. Take the shortest path you can to discovering your idea doesn't work, and invest the minimum you need to in order to reach that same point. After all, you'll need those resources for your next idea.

So, thanks for joining us out here in the real world. We really hope you'll make the most of it by embracing failure and trying new things out, because this is the only path to discovery and success - oh and we'll never build those dominant supercomputers to have a nifty war with if we don't believe in innovating!

Maybe this post will be a failure. Maybe my message won't get into the dreamworld (where we don't believe in failure) or grey pill land (where we don't try just in case). But do you know what? I won't mind if it doesn't - because I will have just eliminated one of the ways it doesn't work, and that's a step forward...

Availability or Control?

2008-10-24T19:39:00.002+01:00

In the web business, we usually consider availability to be paramount - and given that motivation, we're getting pretty good at things like graceful degradation and partial failure. But now that you've pulled your system apart and neatly isolated all the features, how do you cope with the situation where no service is preferable to partial service?

This can be true. Consider, if you will, a trading system operated by a team of risk managers. You have built the system to be fault tolerant and allow partial failures - and this usually works out great - but what happens if a failure in the infrastructure or application results in the risk managers no longer being able to administer the system? It's still running publicly (thanks to you awesome failure isolation) so customers are still buying and selling. You cant change your prices and respond to changing market conditions - uh oh - exposure. What do we do?

One answer is a word we don't like - especially if we just built a reasonably decoupled system - dependency. Yuck, but there is no shame in creating some intentional dependencies that support the business rules. If you never want to execute trades unless you can manage your position, then what is the advantage to running the trading system without the liability tool? Nothing - if anything it's an undesirable risk.

So draw up some service dependencies, or make the applications depend on their monitors at runtime. It might not appeal to how we'd like to run the system, but the truth is it accurately reflects how we'd like to run the business.

And the problem with agile is...

2008-10-20T21:12:00.000+01:00

I get drawn into a lot of debates about the pros and cons of agile (or maybe I just like a good fight?) and the standard attack pattern around quality is starting to become sooooo passe. So I'm going to tackle it here, in the hope that I can use a one-link defense in future. Kind of like TinyURL but for arguments.

Firstly, let me go on record by saying that I most certainly am an agile proponent, but at the same time I don't believe it is the single, solitary answer to how we should do everything in technology. You have to have more than one tool in your kit.

So let me go ahead and get this off my chest:

Agile != doing a poor job

Agile doesn't mean doing no testing, it means doing just enough testing to ensure appropriate quality.

Agile doesn't mean doing no documentation, it means doing just enough documentation to effectively convey meaning and understanding.

Agile doesn't mean doing no architecture, it means doing just enough design work to understand how to start iterating the solution.

Agile doesn't mean having no requirements, it means having just enough detail for the foreseeable future and embracing change beyond that.

It is also about appreciating that just enough can mean different things in different projects, different business problems, and different parts of the system.

Come on people, these aren't new ideas!

To be fair, I can see where some of this criticism comes from. There are cowboys out there who use agile as an excuse to dispense with necessary diligence or take ill-advised shortcuts. When it all comes crashing down, it does sound better to say "hey, we were doing agile" than "hey, we couldn't really be bothered planing this work properly" for the individuals concerned. The fact is that crappy engineers exist. There is no such thing as an SDLC that turns good engineers into poor engineers, we just started accepting that as an excuse.

If agile does have something to answer for here, it is that this kind of poor work is much more visible much earlier (if concealment is your game). The reality is the same team would make the same (or worse) errors in judgement regardless of the approach they used, you just wouldn't know about it for 12 months - and by then, who knows whose fault that was?

Don't accept output any crappier than you otherwise would just because it has agile stamped on it - if anything you should expect better, because you will have had more opportunities for course correction along the way.

What Your Network Guy Knows

2008-10-16T11:16:00.000+01:00

So you're getting into distributed systems; maybe you've got some real scalability issues on the horizon, or perhaps you want to better isolate failure, or be able to cope with more concurrent change in the system. So how do you do this webscale thing then?

Time for some homework. Listening to some vendor pitches, maybe reading some books, or getting an expensive consultant or two in for a while (I'll take your money from you if that's what you want) might possibly do it. But before all this gets out of hand, did you realize you're probably sitting right next to a distributed systems fountain of knowledge? You already have someone, right there in your team, who has spent their entire career working with the largest eventually consistent multi-master distributed systems in the world - the trick is they might not even know it themselves - and that someone is your network guy.

Let's test this assertion against a couple of technologies that network engineers deal with every day, and look at what we can take from them into our distributed systems thinking.

How about something fundament - routing protocols. Networking gurus have a small army of acronyms at their disposal here; OSPF, EIGRP, IS-IS, BGP, and the sinister sounding RIP. These are essentially applications that run on network devices (and sometimes hosts themselves), map out the network topology, and provide data for devices to make packet forwarding decisions.

So what can we import from this technology?
1. Partitioning - networks are broken down into manageable chucks (subnetworks) which scope load (broadcasts), ringfence groups of systems for security, and limit traffic across slow and expensive links.
2. Scalability - routing protocols allow massive global networks to be established by summarizing contiguous groups of networks again and again, and ensuring any node can establish end-to-end connectivity without having to understand every single path in the network (just a default route).
3. Failure isolation - subnets are bordered by routing protocols, which form a natural boundary to most forms of network malarky. In the event that a network becomes unpredictable (flapping), some routing protocols are able to mark them down for predetermined time, which aids in local stabilization and prevents issues spilling over into healthy networks.
4. Self healing - when a failure in a network or a link between networks occurs, routing protocols observe the problem (by missing hellos or interfaces going down) and take action to to reestablish reachability (work around the problem using alternate paths etc). Each node will recompute it's understanding of the networks it knows how to reach, learn who it's neighbors are and the networks they can reach, and then return to business as usual via a process called convergence (this is a really simple study in eventual consistency and variable consistency windows).
5. Management - for the most part, networks separate their control messages from the data they transport. A good practice, especially when combined with techniques like QoS, because it significantly reduces the risk of losing control of the infrastructure under exceptional load conditions.

Now let's look at something application layer - DNS. This should be a somewhat more familiar tool (or you're kind of reading the wrong blog) and we touch it quite regularly but probably don't appreciate what goes on in the background. At it's most basic level, DNS is a client/server system for providing a mapping between human-readable hostnames and machine-friendly IP addresses. Oh but it's so much more...

So what can we import from this technology?
1. Partitioning - DNS is hideously, frighteningly big, there are hundreds of thousands of nodes in this system, from the dozen or so root servers all the way down to the corporate internet access edge servers. It is a good example of dividing up a problem; to find us you'd work right to left through a fully qualified domain name, starting with the "." (root), we're in the "com" container (hosted by a registrar), then the "betfair" container (hosted by us), and finally you'd get back some data from a record matching "www" and arrive at our place.
2. Scalability - did I mention DNS is big? DNS uses a classic combination of master/slave nodes and caching on the client and server side to scale out. At the corporate edge, DNS proxies resolve addresses on behalf of internal clients and keep answers in a local cache, ISPs and those who run their own zones keep a number of slaves (many read only) and spread queries out amongst them, and finally an expiry timestamp (TTL) is set on query results permitting client side caching.
3. Resilience - clients can be configured with a list of servers, which they will cycle through should they receive no answer. Additionally, the DNS protocol is stateless, making it easy to move servers around hot and load balance using simple, lightweight algorithms.
4. CAP - DNS definitely prefers availability over consistency, the window for an updated record to be propagated around the internet being ~24hrs in most cases. It's also highly tolerant to network segmentation, individual servers being happy to live separated from the rest of the DNS infrastructure for long periods of time, answering queries, and then catch up with all the changes in the zones they host once connectivity is reestablished.
5. Operations - the hierarchical way the namespace is organized is perfectly matched to how authority is delegated. If you're going to have a massive system spread around the globe, you've got to think about how you're going to operate it, and the DNS model for this is based on allocating administration with ownership. This gives complete flexibility and control to namespace owners without risking the integrity of the system as a whole and let's us operate the biggest distributed system in the world without employing the biggest IT team in the world.

So buy your network guy a coffee. Ask him how his world works. If you can draw the philosophical parallels, it might be the most valuable couple of hours you've spent in ages.

Oh and by the way - distributed systems are all about the network, so you're going to need a friend here anyway...

Is Leadership a Noun or a Verb?

2008-10-12T17:22:00.000+01:00

Is technical leadership something you know or something you do? Is it a finite thing you can point to, and check off a list, or is it an ongoing process, more akin to a journey? Let's look at what it means in practice...

Firstly you have to have some answers - or better yet know the right questions. You need to have a vision, and preferably some ideas on how to break that down into Big Goals. You need to define what success looks like for your team, and know what sorts of things contribute to that success and what sorts of things distract you from it. Maybe this is leadership the noun?

All good work so far, but worth nothing to your organization if no one knows about it - who is it that's supposed to be working towards these goals? Who is it that you depend upon to realize your vision?

So the next thing you need is for everyone to proverbially Get On Board. You have to communicate that vision as simply as possible and help every individual understand exactly how they can contribute to it. The whole team has to get emotional about it, believe in it, and live it through how they do their jobs each day. Perhaps this is leadership the verb?

So perhaps the answer is both? I certainly think so. Share an empty vision or set the wrong goals and you won't get where you need to be. Keep quiet about your plans and how they can be achieved and you rob the organization of huge value.

Oh and don't forget to keep your eye on the horizon - things change and from time to time that means adapting your strategy or losing ground!

IASA Connections - Day 3

2008-10-09T00:30:00.001+01:00

Day 1, day 2, and now day 3:

Scott Ambler – Practice Leader in Agile Development, IBM Software Group
• Data modelling from an agile perspective.
• Don’t get too hung up on purist agile, good techniques are good techniques regardless of where they come from.
• Software development is more of a craft or an art than engineering – hence we tend to apply some practices that don’t make things the easiest for us.
• Agile is becoming mainstream – you cannot dodge it – and data is not going to change from being an asset, so how can these things coexist?
• Data and quality are 2 words that are hardly ever seen together, yet this is so important.
• In most “Big Requirements Up Front” projects, 45% of features never used, 19% of features rarely used, so this is effectively a waste of over half the development budget.
• 66% of teams work around internal data groups, most common reasons are difficulty in engagement, responses too slow, and development teams don’t understand the value. So data architects have work to do.
• Modelling is like anything else, it just needs to be good enough for the task at hand (repeatable results, not repeatable processes).
• If you can’t get active stakeholder participation, then cancel the project because all that will happen is disaster
• Clear box testing is just as important as black box testing, because you can test triggers, views, constraints, and referential integrity rather than just observable functionality.
• Do continuous integration of databases – there is nothing special about data so why shouldn’t it be treated the same?
• Make it easy for people to do the right thing – loads of verbose documentation and white papers are less likely to be effective than a small number of run-able assets and some 1:1 support to integrate them.

What I think
• Obviously Scott was wearing his data hat this time, but clearly the whole session was predicated upon the fact that you believe relational databases are themselves the right solution…
• Really like the “repeatable results not repeatable processes” phrase, it is such a powerful idea – I am always battling the ‘1 process to rule them all’ crowd.
• Probably best whiteboard walkthrough of the “karate school” example I’ve ever seen.
• My approach to modelling has always been to define things in order of ‘most difficult to change later’ to ‘least difficult to change later’ so when you run out of steam the unknowns you’re left with are the cheapest ones.
• Abstractions – a lot of the examples in the talk were based around the assumption that applications directly accessed data, we need to think more about how we can build systems that don’t need to know details about schemas etc (access via object etc).
• Totally agree with the active stakeholder participation thing, if it’s important enough for them to expect you to deliver it then its important enough for them to invest themselves in it.

Dr Neil Roodyn - independent consultant, trainer and author
• A session titled “software architecture for real time systems” was about patterns for performance and performance tuning.
• Hard vs. soft real-time distinctions important.
• Time is so often not considered in system design – we think about features and quality and failure but so little about latency.
• Automated failure recovery is so much more important in real time computing because you cannot stop to allow human intervention upon failure.
• There are some strong similarities between real time computing thinking and distributed systems thinking:
1. Consistency
2. Initialisation
3. Processor communication
4. Load distribution
5. Resource allocation
• Asynchronous is kind of “cheating” as it leads to the illusion actions have completed before responses are returned.
• The 3 most important considerations in real time computing are time, time, and time (haha very good Neil).
• Common software tricks and patterns for real time systems (obviously assuming real time performance is your overriding requirement):
1. Use lookup tables for decision making
2. Used fixed size arrays
3. Avoid dynamic memory allocation
4. Reduce number of tasks in the system
5. Avoid multithreaded design
6. Only optimise popular scenarios
7. Search can be more efficient than hash
8. Use state machines
9. Timestamps instead of timers
10. Avoid hooks for future enhancements
11. Avoid bit packed variable length messages
12. Reduce message handshakes
13. Avoid mixed platform support
14. Minimise configurable parameters
• Overall know your system and approach performance tuning scientifically, observe where time is lost and spend energy there, don’t just guess.

What I think
• When we think about SLAs for latency we have to make sure we consider time from the users perspective – if you have a very fast back end but it takes ages for results to render for users, then is it really high performance?
• Even if you have a few processes in your system that need to be real-time, chances are the majority of your system does not, so don’t be afraid to mix memes because if you make the whole thing real time end-to-end you might be unnecessarily sacrificing some scalability or maintainability.
• Totally agree with points on developers needing to know more about the tin their systems run on and how this will lead to better software overall.
• I cant’ help but think we’re getting lazy from our (relative) lack of constraints – back when you had 32K to play with you really thought hard about how you used that memory, when you had to load everything from tape you really planned that storage hit…

Neal Ford – Software Architect and Meme Wrangler, Thoughtworks
• Fairly by-the-book session on “SOA won’t save the world” but via the humorous and engaging analogy of Chuck Norris facts.
• “Peripherally technical manager” types are the key causes for SOA overpromising.
• Good definition of a service as a unit of course grained, self-contained business functionality.
• The tension will always be on between centralised planning and tactical action, so you need to learn how to plan ahead without the planning becoming a constraint on business agility.
• Beware of “cleaning up” the messy communications paths in the enterprise by hiding them inside a box (ESB) – you still have same underlying problems, but you suddenly have less visibility of them.
• Beware of the difference between ‘standards based’ and ‘standardised’ i.e. most vendor ESB solutions share some common basic standard foundations but the functionality has been extended in a proprietary way – so it can still mean lock in.
• Keep track of the number of exceptions you’re granting against the governance you have in place – too many and you might have excessive governance.
• Using ubiquitous language is a must; perhaps even have a formal dictionary for each project.
• The business must be in integration projects not just initiate them.
• We all like a bit of ESB-bashing but they can still be useful for connecting up things like mainframes and 3rd party systems to your nifty REST/WS fabric.
• Exchanging metadata is a great way to negative communication parameters (reliability, security, synchronicity etc) between services and consumers.
• SOA is undebugable – but it is testable - so a good testing discipline is essential.

What I think
• The insurance industry is the place I’ve worked with the most legacy out there (you cant retire certain mainframe systems until all the policyholders die!) and the ESB attachment is just the right milk for this cookie.
• We as engineers contributed to the overselling as much as the Neal’s peripherally technical manager did – I think we got carried away with the interest that a bit of technology was suddenly getting and we were happy to be on the bandwagon as we usually struggle so hard to get technical things on the agenda – how could we not ride this wave?
• There are more benefits to SOA than reuse yet that’s all we ever seem to talk about. How about what it can do for scalability? Failure isolation? Concurrent change? Hot maintenance?
• Yes. BPEL. Terrifying.

The Big Summary
Overall I think this was a very good flagship event, my thanks to the organisers. The turnout was a good size – small enough to feel personal and allow plenty of networking opportunities, yet big enough to ensure a variety of engaging discussions.

The IASA mission is a worthwhile one, and one that I think we don’t do enough about in any technology discipline. Whether we’re talking about architects, system administrators or developers, how much time do we spend investing in our community? In ourselves as a profession? When was the last time you looked for someone to mentor? Went out of your way to share your ideas beyond your organisational boundaries? Visited a university to see what the talent you’ll have to work with in 5 years is actually learning? Investing in ourselves as an industry is undervalued, and I’m happy to be part of a group trying to address this.

If there is 1 thing I would change for next year, it would be the format of the talks. There are already enough conferences you can go to and watch someone meander through a slide deck (this was great meandering though!). If we change the focus so that speakers only use 50% of the allotted time and treat their role as setting the scene, then we could use the other 50% to get a real group discussion going on the topic at hand. I would certainly find that more valuable, and I would suggest that promoting high intensity peer discussions on tough technology and strategy issues would probably better serve the mission of establishing architecture as a profession.

I have been assured that you will eventually be able to find the slides from the formal sessions and keynotes here.

So that was IASA Connections. Tomorrow I’m off to UC Berkeley for the day, so probably best ease back on the beer this evening (it makes it harder to maintain the illusion of cleverness I’ll need). Sayonara.

IASA Connections - Day 2

2008-10-08T02:21:00.001+01:00

Ceteris paribus, what comes after day 1 of a conference? Well, day 2 of course…

David Platt - teacher and author (and mad professor?), Rolling Thunder Computing
• Day 2’s keynote on why software sucks, which was adapted from his book by the same name.
• Lots of statistics on why users hate our software, mostly boiling down to the theory that software is built by geeks for geeks but actually used by regular folks.
• 2 million web users in 1994 has become 1200 million in 2006; and almost all are now laymen (in 1994 almost all were geeks themselves).
• Platt’s rule, “know thy user for he is not thee”
• Users don’t want to use your application, they want to have used it – so understanding that you provide a means to an end rather than an end in itself is a helpful thought pattern.
• Customers don’t buy software, they buy the outcome it’s intended to create, e.g. you don’t buy a drill from B&Q, you are really buying a hole.
• Transferring extra clicks into cash (salary costs of time spent clicking) is a useful way to demonstrate this to management.
• Suggested solutions:
1. Add a virgin to your design team, i.e. get someone never used your system before to try it and observe their challenges.
2. Break convention when needed – for example, why have a save button if you could just have the system save things as you go?
3. Don’t allow edge cases to reach mainstream – if something is really unlikely to be used by more than 1% of visitors, why include it in the main real estate?
4. Instrument carefully – watch user experience, track clicks, see what users actually click and how frequently, how do they arrive at certain pages/features?
5. Always question each design decision, “is it taking us closer to ‘just working’ or further away?”

What I think
• Some of the talk didn’t apply to Betfair per se, as online gaming/gambling etc joins the ranks of things like porn (can I say that?) in software that’s used for the experience, rather than to go through a process to arrive at a destination.
• Most of the examples are based on software that shipped with pointless features or usability issues. What wasn’t addressed was how architects and engineers relate to this – I agree that you can make terrible usability errors, but this is usually a decision making process by product managers and other stakeholders rather than the result of developers work. More material on how engineers can push back on these decisions would help us as a profession.
• Instrument carefully is good advice, and in my opinion is critical to effective A/B testing. Knowing what your customers like best coupled with the ability to try new layouts or versions in a lightweight way is a pretty important to keeping the best software out there.

Ted Neward - independent software development architect and mentor, Neward & Associates
• What are architects really about and what does architecture mean?
• We tend to get carried away with “what if’s” like what if I want to change logging system later so build in 96 logging APIs and pretty much use 1 or 2.
• Do we have architectural styles in the same way as building architecture has styles?
• Defining pragmatism – avoiding idealism in favor of something appropriate.
• There is no such thing as NFR, they are all just requirements.
• What do architects do:
1. Understand (what’s the problem, what’s the constraints)
2. Reassess (changing goals, changing constraints)
3. Explore (more potential solutions, new opportunities)
• Beware of the coolness trap – cool technology doesn’t mean appropriate technology.
• The 3 kinds of architects:
1. Infrastructure – datacenters, networks, storage
2. Enterprise – CTO high level leadership
3. Solutions– the “per project” answers to business questions
• Patterns are not architecture; they are simply a vocabulary we can use to have high-speed discussions about technical ideas.
• When designing systems, make big decisions first, then next most detailed, then next most detailed…
• Idea of architecture catalogue (the things you do as an architect)
1. Communication – transports, exchanges, and formats.
2. Presentation – UI, style, and delivery.
3. State management – durable vs. transient, context vs. process.
4. Processing – implementation, transactions, and shared data.
5. Resource management – location, registry, discovery.
6. Tools – languages, data formats.
• If you don’t explicitly define architecture it will be implicitly defined for you!
• Architecture is a bigger picture view than a developer would typically take.

What I think
• Not sure I agree with the bigger picture assertion, as I think this promotes the ivory tower perception of architects – and besides, successful software demands that everyone involved in building it understands the bigger picture.
• You need to consider future options, but take into account likelihood and difficulty to change and watch for entering diminishing returns!
• One of the reasons traditional architects don’t suffer the same difficulties we do is that the constraints and practicalities in real world construction are so much more visible. We need to put more effort into making these understood.
• I like the NFR idea because so few people talk about scalability and availability etc as business benefits, but they most certainly are!
• Agree with the patterns point, too many people get hung up on them but they do not describe a working solution.
• While I always like to talk about defining architecture as a trade, I am not sure about the catalogue idea – some of those decisions (like presentation, data formats, and languages) and surely best made by those closest to the problem; i.e. the delivery engineers.

Dr Neil Roodyn - independent consultant, trainer and author
• Gave a talk called “it’s all wrong, change it all” a rather provocative title that attracted me to the session…
• We know things are going to change, so how can we put ourselves in a position where this is as easy as possible?
• We’re in a change heavy industry, however, humans are naturally change averse so this is something we have to work hard at.
• Abstractions actually make it harder to change things, because they bind code to a particular abstraction – and people tend to feel emotional attachment to their implementations!
• Big visions are so important - always remember that’s what you’re delivering against, not the details.
• We get carried away with reusability – make sure the expected demand for it to be reusable justifies the cost of building it in a reusable way.
• How you design your system can make it more change-friendly or less change-friendly, this is how architecture can help.
• People are the biggest factor – there is a fine line between continuous improvement and gold plating.
• Don’t underestimate malleability of code.

What I think
• People typically associate change with risk or loss. Part of the trick to getting buy in to ideas and making people happy to adopt changes is to address good-old-fashioned “what’s in it for me?”
• Abstraction layers scares me because we’re not getting smarter at the same rate that we’re abstracting away from the primitives. I think we have a problem as an industry, which is convincing people to learn to be engineers (computer scientists) over quick and dirty coders (learn Java in 28 days and get a job).

Paul Preiss – President, IASA
• Dealing with how we establish architecture as a profession in it’s own right.
• To be a profession something must have:
1. An identifiable body of knowledge
2. Able to enter this trade without having to have been something else first
• Helping to realize architecture as a profession is a key purpose of the IASA group.
• America Institute of Architects formed in 1857 because (sound familiar?):
1. It is difficult to solve real architecture problems due to lack of targeted resources
2. It is difficult to find experienced architects
3. It is difficult to be of maximum value to our employers due to lack of stable job definition
4. It is difficult to tell a qualifies architect from an unqualified one
• Architects tend to be in one of 4 major groups:
1. Corporate
2. Vendors
3. Thought leaders
4. Service integrations
• Professional perspectives and skills taxonomy are resources we’re trying to build up – providing that ‘identifiable body of knowledge’
• The skills one must master to truly deserve the title of architect are:
1. The IT environment
2. Business technology strategy
3. Design
4. Human dynamics
5. Quality attributes
6. Software architecture
7. Infrastructure architecture

What I think
• Working towards formalizing architecture as a practice, a role, and a valuable and necessary part of a team is a massive undertaking – we have work to do here as an industry.
• Perhaps the key to this is working backwards from result? Maybe we need to ask ourselves “how do we test for architecture?” If we can work out what we’d be able to observe in an organization with this nebulous architecture thing in it vs. one that does not, then we’ll have an answer and some benefits to point at!
• In the short term we can’t test for architecture so how do we find them? Well you can test for thinking patterns, values, and philosophies so look for the ones that contribute the most to your organizations definition of success.
• Perhaps we need a charter? Some publically visible set of principles that anyone engaging a software architect should expect to be met?
• Good to see the ‘ities’ discussed as central to what we do.
• I still say it’s science Mr. Preiss – but pragmatic science! We solve real-world business problems by using our skill and experience to apply the right technology.

And with that I must once again apply my problem solving skills to the problem of unconsumed beer. See you tomorrow, same bat-time, same bat-channel...

IASA Connections - Day 1

2008-10-07T03:02:00.001+01:00

This week I skipped across the Atlantic to attend the International Association of Software Architects & Architect Connections conference in San Francisco. Between my failed attempts at getting a chapter off the ground in Romania and being a long-standing member of the London chapter, IASA has been a hugely valuable organization to be associated with. There are regular activities at most major IT hubs in the world, with a strong focus on sharing ideas and meeting other people with similar technology challenges.

For the next 3 days I have a fairly hefty schedule of people to meet and sessions to attend - so I'll try and throw up a brief summary post at the end of each day covering what's discussed. So let's dive straight into day 1:

David Chappell - serial technology writer and speaker, Chappell & Associates
• Gave the first days keynote on cloud computing, covered the basics (this is still a very misunderstood topic) and direction over the next few years.
• Cloud computing falls into 3 main categories; cloud delivered products like windows live, attached services such as spam filtering and email antivirus, and cloud platforms like EC2 and S3.
• There will always be a mix of 'onsite' software and cloud based services in every organization in the future - so clouds will not rules the world!
• Advantages of cloud computing include lower costs and less financial risk (as capacity grows with demand, rather than having to build for peak from day 1).
• Disadvantages of cloud computing include trust issues, data protection, and regulatory requirements.
• Important to distinguish between consumer (free and ad supported and usually less reliable) and business (usually paid directly and covered by an SLA) cloud solutions.
• Underlying message is really about change and how hard it is - and how this relates to making the paradigm shift we need to get the enterprise to accept cloud as a valid technology.

What I think
• The cynical side of me can’t help but think we’re throwing things we've been doing for ages (like hosted email virus and spam checking) into the cloud bucket now that we came up with a new word for it.
• I consider some of the constraints of building on a cloud platform to actually be benefits – forcing developers to abstract away from hardware means you’re building systems that are inherently more scalable, available, and hot maintainable (whether deployed in a cloud or onsite).
• Lack of enterprise strength SLAs are keeping a lot of organizations away from cloud computing. Perhaps we as engineers have an obligation to help the business start to look less at SLA and more at actual performance? Most of the time a cloud infrastructure will trash a fixed internal hardware deployment in overall availability yet he SLAs don’t yet demonstrate this.
• Cost is very much still a factor when moving to a cloud deployment, but you move to a marginal cost consideration – you don’t want to run an expensive piece of code for each hit, since that unnecessarily increases the cost of the hosting. In a traditional model it doesn’t matter so much if you have expensive code running most of the time if you’re using capacity normally idle except during peak – now that costs you every time you execute it.

Dr Neil Roodyn - independent consultant, trainer and author (and big fan of XP)
• The “capital A” Architect vs. the idea of “little a” architecture, talking about what the role really should be and how it works in agile.
• In software we sometimes borrow a little to heavily from industrial engineering.
• Architecture is a practice created to try and put predictability back into a process that is inherently unpredictable (software delivery).
• Traditionally we try to separate design from construction – i.e. have really smart architects design the system and then any idiots can code it – and how this is flawed thinking.
• Change is inevitable in software projects, most of our traditional processes require rigid and stable requirements in order to be successful but this is not real life.
• Do the best developers only end up architects because we romanticize the role too much? Or it is the only way they see they can get career progression? Wouldn’t we rather have the best coders coding?
• Maybe we need to start looking at developers more as craftsmen and less like engineers?
• Be careful how you incentivize people, because that’s what you’ll get!

What I think
• Something omitted here is that the quality of an engineering team’s output can never exceed the quality of it’s input, so there is a strong dependency on working well together with customers to truly understand requirements and make the right tradeoffs.
• Incentive is a very valid point, if you measure architects on creating verbose design documents or UML diagrams then that’s what you’ll get – try measuring them on quality, working software.
• The people that know how best to solve problems are the people closest to their effects, so architecture has to be about harnessing the individual developers skill, experience, and exposure to the domain – not trying to tell them what to do.
• I quite like the label “engineer” because it stands for solving real world problems with technology – but I do agree the process does need to be considered to be more creative than it currently is. I recently read something that likened software development to film making, which I think is a powerful analogy.
• Why does IT have such a bad reputation? I don’t think it is as simple as failed projects, I think it is how they’re ‘washed up’ afterwards too. Working closer with customers along the way makes sure they’re aware of the impact of their decisions (changing/adding requirements) so you can easily end up with projects over time/budget, which are not considered failures by the business – because they expected it and were involved along the way.

Randy Shoup - Distinguished Architect, eBay
• The architectural forces governing eBay are scalability, availability, latency, manageability, and cost.
• Their architectural strategies are partition everything, asynchronous everywhere, automate everything, and remember that everything fails!
• Data partitioning is dividing DBs by functional areas (user, items, feedback) then sharding so that each logical DB is horizontally partitioned (by a key such as itemID) which provides scalability for write heavy data sets.
• CAP - their system prefers availability and network partition tolerance and thus trades away consistency. Variable window for different features.
• Functional segmentation at application server layer follows the same pattern as the databases (selling, search, checkout).
• Most operations are stateless which means the most basic, off the shelf load balancers can be used to horizontally scale application server tier.
• When travelling though the system, what state is generated is accumulated in cookies, URLs, and a scratch (periodically flushed) DB.
• Patterns used:
1. Event dispatch (new item listed, item sold)
2. Periodic batch (auction close process, import 3rd party data)
3. Adaptive configuration (dynamically adjust consumer thread pools and polling frequency based on SLAs)
4. Machine learning (dynamically adapt experience by collecting data for the day on for example recommendations and then try different ones next day)
5. Failure detection (collect data on activity send on multicast message bus and listeners watch for certain error messages)
6. Rollback (entire site redeployed every 2 weeks and no changes cannot be undone, features rolled out in dependency tree order)
7. Graceful degradation (switch off non essentials, defer processing to more guaranteed async messages)
• Message reliability dealt with via framework.
• Code deployment decoupled from feature deployment, so new versions can be rolled out with dormant functionality (they call this wired off) that can then be turned on gradually.
• NOT agile and don’t claim to be!
• Completely redeploy entire site every 2 weeks – although this is more of a constant, rolling upgrade than the big bang it sounds like.

What I think
• A very agreeable talk, particularly around deployment, versioning, and A/B testing.
• The architectural forces are very similar to our 6 principles (availability, scalability, maintainability, quality, security, and user experience) and really are just different words to describe the same good practices needed to meet the challenges we have in common.
• State is the enemy of scalability, and it is refreshing to see someone tackling this well - instead of saying "how can we build a distributed state tracking system this big?" they're saying "how can we build our system so that the functionality does not depend upon a central state system?"

Rob Daigneau - Chief Architect, SynXis
• Anti-patterns - human behavior in software projects and how to deal with it.
• Overdependence on methodology as solution to all problems and smooth project execution – not enough focus on personalities and simple human nature.
• Individual anti-patterns:
1. Cyber addict – using only IM or email etc for communication is very efficient, but not necessarily effective as you can lose the meaning of your message.
2. Fortress – not sharing knowledge and being secretive leads to good old fashioned unsustainable codebases and key man dependencies.
3. Ugly baby – too little care taken with how and when criticism is issues can lead to valid advice and points being ignored to the detriment of the project.
4. Perfectionist – believing that masterpieces are the only way every time can lead to unnecessary gold plating, artificially inflating the cost and time to market beyond what’s beneficial to the organization.
5. Conquistador – being overzealous about "the one way to do it" can mean failure to make good tradeoffs.
6. Workaholic – not having the right work/life balance will only lead to burn out and mistakes made.
• Team anti-patterns:
1. Hazing – throwing people into the deep end with very little coaching or attention to integrating with the existing team often means they take a long time to become productive.
2. Fight club – the organization can become a casualty in pointless battles between architects and developers, developers and testers etc unless teams learn to work together.
3. Wishful thinking – working out some estimates that show a certain milestone cant be reached in a certain timeline means just that, no matter how much you’d like it not to.
4. Firing squad – if people are scared to disagree or the culture doesn’t encourage the introduction of new ideas; the organization can be robbed of great value that stays locked away in engineers’ heads.
5. Too many cooks – if there are no clear boundaries then consensus becomes a huge issue and teams become paralyzed, unable to make progress because of endless circular discussions.
• Leadership anti-patterns:
1. Deer in headlights – if a leader wont (cant?) make a decision then the company stands still and competitors gain ground.
2. Cliff jumpers – taking risks for the sake of risks or making decisions on dangerously incomplete data can waste time, money and patience.
3. Spin zone - talking talk not walking walk gets transparent pretty quick.
4. Drill sergeant – exercising authority in an aggressive way makes very short-term gains and never gets a team behind a goal.
5. False summits - artificial deadlines or objectives can be dangerous, if the activity doesn’t turn out to be committed to, a loss of morale and trust in management can result.

What I think
• Reasonably fundamental stuff for experienced management but good to see it getting airtime in this crowd, architects typically come up via technical tracks and probably don’t spend enough time on their influencing and consensus building skills given how important a part of the role this it.
• This would have been a good opportunity to talk more about incentive and how it drives behavior. A lot of anti-patterns can be traced back to KPIs that encourage/reward the wrong things.
• Preventing burnout and keeping sustainable development going are some key reasons I like SCRUM – working in sprints gives you a cyclic mix of high intensity work interspersed by more passive, thoughtful phases.
• I liked the Gerald Weinberg quite "no matter how it looks at first it is always a people problem".

Ken Spencer – Practice Manager, Solid Quality Mentors
How agile applies to traditional, formal technical projects using a current project for the US government as a case study.
Manufacturing background so most examples look back to this experience.
Looking at why most projects fail – this is because people fundamentally keep doing what they do.
Given that we almost automatically repeat what we know, the good news is that this means being successful once significantly increases the likelihood of being successful again.
Spend commensurate amount of time planning - you need plans but you need to know when to just start work and learn from doing.
Good enough – what are the testing criteria and when are you into diminishing returns?
The ‘love curve’ which is a cycle you go through when taking over a failed project; initially distrusted though bad experience, then looked upon as savior, then reality sets in, then they see sustainable success.
Getting back to basics – when things go off the rails look first at the simplest units of work the team ought to be doing and this is usually where you will find your answers.

What I think
• Largely based on the agile manifesto, so we have a pretty strong philosophical connection here.
• Slightly damming of waterfall, at the end of the day it isn’t all bad and isn’t evil, I think it’s quite suitable for very repeatable, commodity projects as long-horizon predictability is possible and the phase space of all possible features is finite.
• Not sure I like the label ‘formalized’ when used to describe waterfall vs. agile, as this kind of indicates less control, visibility, and organization around agile when in fact I think these are some of its strengths.

Whew! Time for beer now, see you tomorrow…