Monday 25 August 2008

The Law of Conservation of Complexity

The law of conservation of complexity states that there is a minimum amount of trouble you need to go to in order to achieve a certain outcome. You can always make it harder, but you can never simplify beyond a certain minimum level without compromising the result. Imagine a system with 3 components all of which are at their least complex, by further simplifying one component of this system you do not reduce the minimum overall amount of complexity required, because another of the components must become more complex in compensation.

A couple of real life examples:

Something customer facing. Let's say we're selling car parts online and we need to check availability from manufacturers before we accept orders, and we'd quite like to do this in real time so we don't set our customers expectations wrongly. We could do this in the back end by making complimentary calls from our application to the manufacturers inventory system as we receive requests from our customers. This might be a smart solution because we'd be able to do some short-term caching of results for popularly looked-up items. We could do this in the front end by getting our users browsers to fetch the stock levels directly from the manufacturers website or (if we're lucky) API. This might be a smart solution because it would reduce the load our application takes.

So we can simplify the front end at the expense of the back end, or we can simplify the back end at the expense of the front end. Either way we can't escape the need to make that request somehow without changing the functionality of the system [being aware of stock levels].

Something platform. Let's say we've got a distributed application that we need to provide strongly consistent data storage for. Since consistency is one of our requirements, and assuming we don't want to centralize the system, we need a way to make sure that when we write to a piece of data in one place, a contradictory update is not being made in another. We could do this on the storage side, by using a distributed locking algorithm or electing certain partitions to be the designated 'writer' for certain data items. This might be a smart solution because it's more portable. We could do this in the application by making the storage client responsible for locking every copy of a datum, or sending a message to all it's peers advising them of updates it wants to make. This might be a smart solution because it simplifies our data administration.

So we can simplify the application at the expense of the storage system, or we can simplify the storage system at the expense of the application. Either way we can't escape the need to govern writes somehow without changing the functionality of the system [keeping consistent data].

Just as energy is never lost, the minimum amount of complexity an entire system must have to achieve its goals can never be reduced, it can only be moved around.

No comments: