Software vs. Infrastructure

For some reason I’ve been running into a lot of other CIOs and CTOs lately and one topic that keeps coming up was whether we were development or infrastructure focused. The whole time I kept thinking to myself why is this even a question? Why do we feel like we should choose? Or, somewhat more provocatively, why do so many of us only feel like doing half the job?

I concede that there is a ‘background’ element to this. No one I know started off as an IT executive; we were all DBAs or developers or sysadmins first, and that gives us a nice comfortable area of personal expertise we can use as shortcuts but it shouldn’t set our agenda in a leadership role which [in most organisations] encompasses the whole shooting match.

One of the reasons why many organisations suffer the good old fashioned laundry list of regular technical woes (that we could probably all reel off by heart) is because of the interplay between development and infrastructure and a lack of end-to-end oversight over both. And, as the technology leaders, if we don’t understand both sides, keep strongly engaged with them, and create teamwork and cooperation where there has typically been borders and a lack of mutual interest, then who will? There are very few other roles with a remit in both areas and even fewer that should be taking responsibility for them failing to be an end-to-end unit.

There is a whole ‘devops’ kind of meme swelling up these days – and more power to it; I think it is dead on the money.

Devops is, by nature of the organic meme it’s emerging as, poorly defined but to me it is about the recognition that [in most cases] customers care about product – not software or systems – and product is that butter-smooth combination of software and infrastructure finely tuned to work together and operational know-how to keep it running.

It’s also about having the right feedback loops in place between engineers and the real world, so that as your products get used and abused, they become more fit for that very purpose.

What am I going to do today?

Here I go again – with that ‘time’ spiel – anyone who works with me will be switching off about now; but it’s a good spiel and one that I think it isn’t common enough in the corporate consciousness, so now my readers (both of them) get to endure it. It’s really about scarcity, and recognising the resource which we should prize most dearly yet too often let slip through our fingers.

When asked what their most valuable resources is, most people will say money (the odd few might say people – true in another interpretation of the question). Cash is critical to a small business, as it’s often in sort supply, and critical to a large business, because of increased fiduciary responsibility and reporting requirements. But the thing that you can never replace is time and, no matter how big or cash rich an organisation gets, none of it’s individual members ever have any more of it available to them than anyone else anywhere else. It’s fixed. It’s also a unique type of resource to manage – you can’t save it up for a while so that you can spend more later and if you spend it unwisely, you can’t go back for a refund and try again.

Just how scarce is this time thing? Let’s do some quick maths and see how much of it we actually have available to us this year:

1 - Start with 365 days in 2010 (assuming that my time travel project doesn’t get off the ground)
2 - Most of us average out to 5 days per week, so let’s take away 104 leaving us with 261
3 - Most companies give an average of 22 days annual leave every year so now we’re down to 239
4 - On top of this Her Majesty bestows upon us 9 bank holidays in 2010 leaving us with 230
5 - Sick leave isn’t always all used up but let’s assume a worst case of 10 which puts us at 220

Starting off with 365 days makes a year sound like a long time, but we pretty quickly slashed a staggering 40% off that without really trying. And that’s without any contingency for things like sabbaticals, maternity/paternity leave, jury duty, family emergencies... and we can’t make any more of this ‘time’ stuff. It can’t be bought, borrowed, earned, duplicated, or recycled.

Most of us waste much more of this than we readily accept. Ever sat in a long meeting on a tangential topic with a large number of attendees – some of whom aren’t even really engaged (on blackberrys etc)? Ever wonder what the world might be like if those 10 people each got that 2 hours back? In the corporate world I think time is undervalued because it is a hidden cost (outside of professions which explicitly price their time such as legal advice and chartered accountants) and there are often few controls on how one spends their [the organisation’s] time compared to the controls and various levels of signoff on how one spends the organisations money.

Everyone has key projects to do and those projects only get done by people spending time on them. And, as we just worked out, a year sounds like plenty of time to do all this stuff but we usually end up with a fair bit less time on our hands than we started off with.

Getting what we want done means being aware of the priorities and valuing time enough to spend it deliberately – on the things that are most important to the organisation. I used to be surprised about the sort of things that really successful companies dropped on the floor – genuinely good ideas just not getting attention – and then I realised that they hadn’t been missing opportunities; they were just uncompromising about their priorities, staying focused on their most meaningful things, and fiercely guarding against distractions. That’s how they got really successful in the first place.

A different view on Job Satisfaction

If you’ve been here before then you will know that I don’t just regurgitate content here unless I think I can add something significant to it, but I guess every rule has an exception.  One of my guys sent this around a while ago, and again quite recently (OK I get the point), and I think it’s too good not to share:

Jobs are not meant to satisfy us. Jobs are not animate things that have knowledge of who we are, what we are seeking and what our special needs could be. You may say that I am just making a philosophical statement. To the contrary, I believe that it is the most practical and rewarding way of looking at many things in a professional career. When I see scores of successful people around me, I believe that their achievements are largely because of such a perspective. It also occurs to me that developing this perspective is eventually beneficial in every way possible.

Let me go back a century and tell you a story. My grandfather was a medical practitioner in the Bihar of 1920s. He had a brood of children who were orphaned due to his untimely death. Two of my uncles had just about finished high school when he moved on. Their older brothers could not afford to send them to college. The two had to be gainfully employed, somehow, as soon as possible. They were taken to Tata Steel, an hour away from where they lived. Tata Steel and the government of Bihar were the only two employers you could think of in a five-hundred mile radius of my uncles’ hometown. The possible work one could get at Tata Steel was that of a technologist- engineer or of a manual worker. So, what could be done with the two boys with their high school qualifications? They were neither fish nor fowl. “Take them to the lab,’’ someone said. A German technician who ran the place was looking for a few hands. The burly German took a hard look at the two. Then he showed them a broom standing at one corner of the lab and asked them to sweep the floor. By the end of day, one of the two just ran away. To him, it was too much to handle. The one who stayed back retired as a chief supervisor of Tata Steel. The difference between the two? The one that stayed on was not trying to seek ‘job satisfaction’. Instead, he focused on satisfying the job.

The more prosperous the industry, the higher the number of people looking for this elusive thing called ‘job satisfaction’. Similarly, the more qualified some people are, the higher is their need for ‘job satisfaction’. Sometimes, it is as elusive as seeking ‘true love’. There are times when we get lucky deservedly or otherwise. But we also get used to it and conclude that it is the responsibility of the organization to maintain a continuous supply of job satisfaction.
Whenever I think of job satisfaction, I remember all those who have to work at night—policemen, airline pilots, nurses and doctors, ambulance drivers and hotel staff, and of course the sentinel of the snow and the desert and the mountains. Do their jobs ‘satisfy’ these people or do these people satisfy the jobs with which they have been entrusted? Are jobs living things that can ever ‘satisfy’ us?

In the corporate world, like any other place, when we open the bonnet and look under it, we find a whole bunch of tough, dirty but strategic tasks that must get done for the bacon to come home. Sometimes, they are so tough and so dirty that they overshadow the strategic nature of the job. So, all such jobs have to be ‘sold’ to prospective incumbents. More they are sold, less buyers they attract. Often, the man who takes up the job is either a loser who has no other choice, or someone who just views it as a transit camp. For many potentially high-performance individuals, a false sense of survival, desire for glamour or just the need for creature comforts make these jobs undesirable. “I would rather be in Kolkata than be posted to Mungher.’’ “I rather have the corporate planning job than be collecting bad debts.’’ Or, consider this one here: “Give me a cerebral job, I do not enjoy handling transactions. ..’’

Few of us ever ask the boss to be rewarded with a tough and dirty job. We only look for the ‘plum’ ones. Yet, there are people, who given a tough and dirty job, make it strategic: they transform the job in unbelievable ways. In a typical career span, there must be at least four such solid stints in one’s life to make the person a solid professional. All the great people I know have been in the trenches for much of their lives, and their inventory of bruises outnumber the commendations they have received. The occasional commendations stay on the wall. It is the bruises that these people carry with pride.

Subroto Bagchi

That can come across a little old fashioned on a quick read, however I think there is a lot to it when you take the time to internalise it.  When I think about the times that I’ve loved the most in my career, I can see a strong correlation with the times when I’ve been the most focussed on what needs done and making that happen.  Weird how that seems to come back to you in the form of satisfaction…

Entropy

A great note just came through on the Manager Tools mailing list the other day called ‘Everything Decays’ and I liked it because it explains something I’ve recently been wrestling with in my head much more eloquently than I could hope to put it.

Let me reproduce the first 2 paragraphs here so you can see what I mean (can’t find a link to the original work – will post when I do):

“One of the things we've noticed is that managers like having a solution which solves their problem forever. We suspect you've felt this way. In the rare instance you've found the solution you thought would solve the problem forever, you've probably also discovered it's not true. The problem has come back because the situation changes. The people change. The knowledge changes. The need changes.

The fact is, everything decays. For our technical readers, it's just entropy. At work, every problem, every meeting and every relationship is decaying, right now. Now matter how good a meeting is now, in six months, left to its own devices, it won't exist (the ultimate decay) or it will be a lot less effective. The process that you've been working so hard on, and finally got just right, in six months will either be OBE (Overcome By Events) or will need significant rework. You may not have understood, up till now, why good processes that worked before begin not to work. The answer is, the situation, systems or people changed and the process didn't. Entropy. Everything decays. It's not just YOUR stuff that decays, because you're not a good manager. EVERYBODY'S stuff decays. Always.”

The message goes on to talk about how professionals need to continually assess relevancy; keep looking at what they’re doing and making sure it makes sense as circumstances change and the business and the people in it move on.  This resonates on a number of levels including – probably most acutely – the number of managers I still meet who believe there’s some kind of silver bullet out there for their troubles.

What isn’t touched on is the natural entropy you always get in any set of processes or practices even when the environment around them is stable – and the less ‘habitual’ the behaviours are (i.e. cemented through solid and continuous application), the quicker the decay kicks in and old habits creep back in.

tl;dr - there is no substitute for sustained attention and focus, especially during a period of change where some more dedicated coaching can make the difference between lasting change and a blip on the curve.

Technical Debt Credit Rating

A while back I wrote about how closely technical debt paralleled financial debt and then a short follow-up post with the meta-message that, through the right reductionism, certain things will always cost you an inescapable minimum and the decision that you really have is about where you choose to make that payment (I argued that you either pay through the correct design and build initially, or you pay through unnecessarily increased maintenance costs later).

Outlawing technical debt is as foolhardy as always cutting corners - sometimes the right thing to do is to take a hit to get something out sooner or in a certain shape - the problem is in the repayments; in other words, they often don't exist. And if you don't go back and sort your hacks out early they tend to accumulate in the system and before you know it developing on it is like moving through treacle (and the treacle occasionally catches fire). Everything else you do takes that much longer and is that much more expensive (and those are business metrics) and can have unintended consequences - it's a crippling burden just like its financial counterpart.

I have a rule - no one is allowed to take technical shortcuts through a project [where that = incur some technical debt] without showing where, in the same plan, they go back to do some refactoring so that our long-term assets remain sustainable. We're pretty lucky at Sporting Index; most of our stakeholders value technology and recognize how critical it is to the business, so it's a conversation I rarely have to have - but I know many other companies are less fortunate.

So now I think I have a better idea. It came to me while I was thinking about why, if technical debt is so similar to real debt, it doesn't regulate itself in the same way - and I think that's becuase there is no concept of creditworthiness in technical debt, in other words no assessment of a team's ability to 'repay' a quality compromise (read: loan of functionality to the business ahead of when it otherwise would have been ready) alongside all the other outstanding quality compromises that they already have.

But we might be able to simulate that...

Imagine giving each product manager a 'credit rating' of, say, 3 - meaning they could call for unsustainable technical shortcuts 3 times as they worked with the delivery team to determine their solutions. Once those 3 debt points were used, that product manager would be unable to encourage the team to make further quality compromises until at least one of those was paid back. If you have product managers with a demonstrated track record of revisiting earlier deliveries to address technical debt then why not give them 5? A higher creditworthiness ought to enable an individual to take on more concurrent debt as we can be more confident that they'll come back to service it.

I pretty much guranttee that any product manager's cavalier attitude towards shortcuts would change and they'd suddenly treat technical debt as the instrument it really is and use it only where it mattered materially. That, ladies and gentlemen, is a feedback loop.

The downside is that you'd have to be better at tracking this sort of thing and, like any other method, it'll need some discipline when a product manager with maxed out technical debt points wants just one more shortcut. If you've ever had to maintain a spaghetti codebase (as most of us have at some point) then you'll know that these things need closely policed anyway, so perhaps being forced into formally tracking it isn't so bad.

Nobody Wants Computers

Well they shouldn't - they should want what computers will do for them.

Many people turn up to conferences or SIG meetings and ask me how virtualisation or a centralised inventory database or VPNs (or pick something) will help their company. Misguided. My response is always the same - what is your business case and what benefits are you expecting? I do get quite upset when I don't get a good answer to that, because it means someone somewhere is cooking up some technology for technology's sake.

Sales functions are excellent at creating a need around a product they sell - great if you're in that business and I used to be - but it is the functionality your organization really needs that should matter. Talk to me about outcomes, goals, what you want to be able to do from a business capability perspective, and then the right technical solution will fall out of that.

Contorting business processes to fit a specific technology or vice versa - i.e. doing it backwards - only ever works when combined with a healthy dose of luck.

The ISO standard for business plans

If I had a quarter for every time a budding entrepreneur has asked me about what should go into his business plan, then I'd have - let's see - three fifty? I don't know, but put it this way, it's in my top 10 FAQs. My two secrets behind this one are; a) there is no such thing as a standard format or expectation and b) it matters a lot less than you probably think.

Don't agonise over format and presentation - the vital thing is to intimately know your key facts and be able to provide a logical narrative for them. What are your costs? Where does the revenue come from? Who are your customers and how big is the market? How soon will you start bringing in revenue? How is your product different from any competitors (current or future)? What does it cost you to acquire new customers and what is their lifetime value? Have you done this before and who do you plan to work with? Even if you just talk through it you're always better off knowing those things inside and out than having a 200-page glossy document.

And when I say that a business plan matters less than you think it isn't because investors won't want one - they very much will - it's because in the early stages of a new idea what matters most is working product. The best, most detailed, most polished business plan in the world will always be worth much less to a potential backer than a pretty basic, hacked together but functional, beta product.

What kind of idiots are in charge here anyway?

When I coach people about being effective in organizations something that regularly comes up is their ability to influence decisions made at the senior level. Because (with very few exceptions) most people in most organizations are rational and competent professionals with the organizations best interests at heart, any time they feel like their proposals or suggestions aren't being adopted, it can be easy for them to lose a little faith in senior ranks.

I never like hearing that because the truth is (again with very few exceptions) that most executives in most organizations are also rational and competent professionals with the organizations best interests at heart. So why don't your ideas stack up for them?

My advice is usually to look into these two things:

1 - What don't you know that they do - executives will usually have a much broader view of things and can therefore be privy to other information, plans or constraints elsewhere in the organization, that you may not be. Taking into account the things that might be invisible to you, not adopting your recommendations might well be the most beneficial course of action. In my experience, most executives are more than happy to share this information with reports who show an interest in the wider business, so just ask.

2 - What data aren't you being effective at conveying - sometimes the message just isn't getting through and therefore your decision makers don't fully understand what you're proposing and why. It happens to me; I see a lot of proposals for a lot of things (very few ill-conceived) and if the key facts aren't well summarised and the options clear, I am not able to make an effective decision. Again, the solution starts with communication. Ask why, use questioning techniques to check for understanding, find out how your audience wants things presented. I am yet to meet an executive who won't be willing to give that feedback - after all, it helps you both.

Answering these two questions usually gives people the insight they're missing to clear things up and, if it doesn't, then I guess it's always possible that you might be one of those few exceptions...

Things change because people change

Occasionally I get a hard time for spending such a large part of my time with my teams, with the individuals, out on the floor in their work instead of in my office with 'my' work. Occasionally that's also fair - I have a different job than the people who report to me, and the the people who report to them, and that comes with different deliverables.

On the other hand a big part of my job is the performance of my department, the quality of their deliverables, and the continuous improvement of both of these.

New processes, technology changes, new policies, culture and structure are all part of it, but ultimately nothing is ever any different until individuals change their day-to-day actions and behaviours. You will never create lasting change by making rules and proclamations from behind a desk, or just by telling people to 'do better'. Lasting change only comes from sustained leadership focus; coaching and repetition and doing things differently together. When the man on the ground changes then overall results change.

Oh and don't forget my golden rule no. 22867: paperwork is an artifact of effective work - effective work is not an artifact of paperwork.

Let's Ban Agile

Bet that made you look.

Whenever I'm dealing with the wider business and I come across confusion about (or resistance to) agile practices I typically find that, once the few genuine issues are swept up, much of what's left comes down to semantics. And fair enough too - the extensive metadata that has built up around engineering processes is intimidating.

If we put all that aside and just talk about the basic practicalities of doing projects, you are pretty much just looking at an inventory of requirements that must be turned into an inventory of working product. To do that, distinct individuals need to execute specific tasks within certain time constraints (or, more colloquially, who does what by when). From there, we build up some rules to make sure everyone can cooperate effectively around those basics, and it is these rules we'd tend to call our development process/project methodology.

It isn't any harder than that. But, for some reason, we like to make it sound difficult through a plethora of technical terminology.

Instead of iterations, why don't we just talk about tackling big projects a bit at a time? [In fact, I'm yet to discover a better way of solving huge problems than breaking them down into a collection of smaller ones] Instead of backlogs, let's just have a prioritised list of requirements. Instead of standups, let's just have regular and frequent meetings to keep everyone on the same page. Instead of sprint planning sessions why don't we all get together to iron out a detailed plan before we commit to our next chunk of work?

I guess what I really want to ban is unnecessary jargon - at least when dealing with the non-technical contingent of the business. I think we should use common everyday language to describe what are, ultimately, practical and common sense activities.

Just because we're solving complex and difficult problems doesn't mean we need to use complex and difficult processes - in fact, one could make a case for quite the opposite.