Every open system develops towards the unusable

Kathy Sierra, on her excellent blog (which may have very sadly come to a close now, but that’s another long and sad story) has recently published a collection of her favourite graphics from the last few years.

Here is one that I really like:

Featuritis
(By Kathy Sierra, from http://headrush.typepad.com/,
shamelessly copied without permission – I hope she doesn’t mind!)

You should really look at the site yourself – there are many more great diagrams!

The Featuritis diagram above struck a nerve with me, since it’s something that’s been close to my heart for a long time. It is also related to what I think is one of the Natural Laws Of Software Development:

Every open system develops towards the unusable.

If you want to find out why I think that, read on.

The first part of this is the observation illustrated in Kathy’s diagram above: that the simple rule “more features is better” is just poppycock.

(Aside: a sad part of this is that even people who agree here in principle get sucked in to this regularly. Marketing people, of course, love features. Marketing people need arguments to sell stuff, and features give them their foot holds to stand on. Many people who buy products – professionally or privately – follow the tick list approach: if product X does all that product Y does, plus more, it must obviously be better.

Developers know about tick list purchasing too, and so they go into tick list development: Our product must do all that our competitor’s does, plus one more thing.

Even if you know about this, it’s sometimes hard to resist. Imagine you want to buy a digital camera, and you stand in the store and stare at the 27 different models a bit like a mildly senile sheep would stare at a small type printout of the theory of relativity, it is very tempting to go by whoever promises to do most. Unfortunately, very few design for, and then advertise with, good usability, and those who do are mostly lying.

There is hope, though: look at the iPod, for example. It always had less functionality than competitors, with a benefit for usability, and people have actually noticed, and they are buying these things like crazy. So not all is lost.

But most companies are just not brave enough to design for simplicity. I’ll come back to bravery below.)

So, let’s assume we agree that more features is not better to an unlimited degree, and that the “Happy User Peak” indeed does exist. (And if we don’t agree, then you’re wrong: It does, I tell you!)

The second aspect is specifically relevant to open source systems (and other open systems, by which I mean those that are ultimately influenced by a large group of people, either through direct participation or through community processes, or similar.) For simplicity’s sake, I’ll call them “open source” systems for the rest of today.

For open source systems, another factor comes in: reward.

We all do things to get rewarded. Being rewarded is being made to feel good somehow, to receive something that you like to get. The kind of rewards can be very different, but we all need rewards. (To think that open source programmers spend all their time and effort without reward would be naïve, and against human psychology.)

The rewards of open source programmers differ, of course, from those of commercial programmers. For professional programmers, the reward is (at least partly) money.

So what is the reward for open source programmers? Well, typically it is peer recognition, and the satisfaction you get from seeing someone else use your work. We all like to get the occasional mails back saying “cool stuff what you did there – I really love your work”. Or something of that kind.

Just being known for having contributed something useful to the community is sufficient reward for quite a few. Peer recognition. Pure and simple.

Now, the problem with this in this context is that there is very little peer recognition to be earned in geek communities for not doing something. Or for removing a feature.

As a result, typically the only way to enter into and be recognised in an open source community, it to add something. And that means that the amount of features in an open system will continuously increase over time:

Features

Number of features over time in an open system

Since rewards in collaborative communities are for individual bits of work, and simplicity is an overall system property, collaborative systems have a hard time rewarding simplicity.

Now, if you take this together with the Happy User Peak, you get my Natural Law from above:

Every open system develops towards the unusable.

Another part of the problem is that, even if some people can easily agree that we should fight feature creep and concentrate on the core features, they can never agree on what the core features are.

Too many people have their own favourite feature, that they see as absolutely essential for their daily work, and they will insist on it going in. And simplicity goes out the window quicker than a rat up a drainpipe.

So, what we really need is for a system developing towards the Happy User Peak, and then somebody pulling the hand brake and saying – “That’s it, folks, from now on we are only maintaining compatibility, keep the system running, oiling the cogs, but no new features. Anyone who adds a feature will be shot.”

In practice, this rarely happens. Why is that?

One problem is that it is very hard to agree where the peak is. When is the right time to stop. If your system is actually successful (meaning a good number of people find it useful enough that they actually use it), then there will always be a few hundred people pushing for something extra.

(And make no mistake: feature count is a one way street. It will only go up. The only way to get a system with fewer features is to write a new application.)

So typically a group of people discuss this, several people push the feature count (upwards, of course), the system slowly grows, we cannot agree when we have reached the peak, and one morning we all wake up and realise that we have long passed it.

We have seen it many times. Microsoft Word. Where the last few versions have been mainly about trying out new techniques for coping with an unmanageable amount of features. (How about removing some features? No? Well, I didn’t think so.)

Eclipse is also heading this way. And Java. Java started out as a small, clean, neatly designed language that really had a feel of the clarity of a single person’s design. It was far from perfect, but it clearly valued simplicity over completeness of features. And now look at the later additions. Genericity. The design of genericity in Java has special syntax for some rather esoteric cases – it really smells like design by committee. Like something where everyone had to get their favourite use case explicitly supported. And the result: while I knew a good number of people before, who could explain any and every valid Java statement, I can now write valid Java code that almost nobody on the planet can correctly explain. Josh Bloch and Neal Gafter excepted, maybe.

Even the iPod! Video? I don’t need no stinkin’ video on my MP3 player!

Deco

Above I said it takes a brave person to build a simple system. That is because people always worry that simplicity is mistaken for triviality. And rightly so: many people are impressed by anything they cannot understand (“oh, what clever people it must have been who built this”), and don’t appreciate the difficulty to engineer simplicity.

If your boss is one of those persons, then you’re in trouble. Michael Jackson – no not the singer! – has a great story about this. One of my favourite quotes:

Brilliance

from: Michael Jackson: “Software Specifications and Requirements: a lexicon of practice, principles
and prejudices”, Addison-Wesley, 1995. ISBN 0-201-87712-0 (
Amazon)

So what is the way out of this? Is there any hope? I know of only one way how this can be stopped (or at least slowed): a system needs a dictator. No committee, no group decisions. Have discussions, by all means. Talk, gather ideas.

But then one person has to make the decision. And that has to be someone who has a single plan in mind, and who is not afraid to offend people by rejecting their (individually good) ideas.

Open communities are great for development. They are a disaster for design.

14 thoughts on “Every open system develops towards the unusable

  1. I think I get it.
    I am currently enrolled in a software development course at BCIT using the BlueJ environment. A good deal of my motivation comes from the frustration of using programs/platforms/hardware that try to be everything to everyone.

    My version of the future is driven by niche programming. Large, feature rich programs built by large corporations for sale to other large corporations seems unsustainable. The end user never seems to be a marketing consideration.

    In my work (finance & mortgages), there are a lot of intricate, task specific processes that require a very tailored approach (in my opinion – not necessarily shared by management types) rather than out-of-the-box solutions.

    Are my expectations crazy? Will the bottom line eventually show bigger isn’t always better?

  2. Just look at linux operating systems, i know by design they tend to be collections of software maintained in a certain way.. but they tend to get bloated and miss the real issues when they could focus on providing some kind of Operating System.

    Here’s a quote from the linux mkswap command (i expect the FreeBSD people would never allow such things….)

    “Force – go ahead even if the command is stupid. This allows the creation of a swap area larger than the file or partition it resides on.”

    EdG

  3. Peer recognition? Blah. My take: We add features because we require them for our personal work. Patches go upstream primarily because maintaining private branches is such a burden.

  4. I fail to see how this featuritis is inherent to open-source software as is claimed. The arguments are all wrong: the act of adding features just because your competitor has one is naturally much more prevalent in commercial software where there is a “bottom-line”. Even more the unstability of the software arising from this methodology (more features=more bugs) is even stronger in the commercial world because typically a deadline has to be met.

    Also in *all* open source projects there is something like a benevolent dictator or a decision board of developers, not managers or sales people. For example new features *have* been denied in the Linux kernel because Linux Torvalds didn’t agree for a need of it (“just make a patch instead”). But of course you anti-OSS “zealots” (a word you people like to use so much, so I return the sentiment) are as quick to criticise this opposite situation, calling it un-open-source-like. It’s never good for you people, is it?

    Ed Gray>
    A single line from a unix man page should prove this point? It’s also a bit ironic you choose mkswap because that is one of the tools that follows the Unix design filosofy that says that it’s better to have a lot of inter-working tools that do their own job. As in you use fdisk+mkswap/mke2fs to prepare you drives which in windows is all in one monolitic program, the disk management.

    Really, I am in no way an OSS apologist, but jeezus people, if you want to criticize OSS try to make an actual argument against it. Good luck 😉

  5. Frederick:

    You are quite right that this does not apply specifically to open-source software, but to all products designed by committee. Including commercial software.

    Regarding the “anti-OSS zealots” remark: I think you got that quite wrong. I am involved in OSS and other non-commercial software. I am not trying to shoot it down, but to think about how to do it well.

    -mik-

  6. So, you agree the word “open” could be just dropped from the title? I’d agree completely with that 🙂

    I admit that remark was a bit over the top, I wouldn’t normally stereotype people like that (the joy of i’net communication). But it’s a sentiment that arises when I see people, like Kathy Sierra, going out of their way to critize OSS. And as I had the impression from your arguments too, which I really disagree with, the feeling was just too strong to not post a flaming remark 😉

  7. I don’t think Eclipse belongs in the category. Highly modular software with modular feature sets, a strong core that in essence is a library to facilitate extensibility then a lot of work on the modules to limit their features at some point and work on usability. That’s in essence is the key, but that happy user peak is only for one user, that peak is different for every user. For instance for a newbie programmer BlueJ might aspire towards a different happy user peak, than say an expert Java programmer that needs version control and needs to deal with a lot of different java libraries which he’s not familiar with — I love Eclipse for this. In some ways trying to make an application that fits the varying needs of many different users with different objectives, different learning curves, and so on causes this problem — and it’s with us forever, we can only improve it… never really solve it.

  8. Yes, a “happy user peak” exists. But how certain are you that it exists in the same place for every user? I do think that a given software system can be at *my* “happy user peak” – but the set of features I think the system should have may not agree with you think it should have.

  9. I don’t think we have enough experience in this area to declare that the featuritis curve is a fundamental, irreducible problem; it may be a problem that hasn’t been attacked in the right way yet. You say Eclipse is headed toward featuritis; actually, it arrived a long time ago and attacked it as an engineering problem. They adopted the plug-in concept as their starting point and have developed a variety of concepts for handling the GUI issues involved.

    The current set of solutions, despite being an organic and evolving mess, already allows Eclipse IDE users to select a specialized, manageable subset of functionality out of a system of incredible power and complexity (i.e., the entire Eclipse IDE plugin ecosystem.)

    I would not be surprised if this work eventually led to methods that scaled down to very simple programs suitable for unskilled users. We’ll see what the future brings. I believe that eventually we will bring more workable tools to bear than good taste collective self-restraint.

  10. I’m the author of an open source application used by thousands of organizations, “BugTracker.NET” (http://ifdefined.com/bugtrackernet.html). Your psychoanalysis of my motivation was exactly right.

    Saying no to new features isn’t the only way to prevent sliding over the crest of the Happy User Peak. If you work at it, you can often come up with a way to package the complexity so that it looks simple, to make the app configurable so that simple is the default but the additional features are there for those that want them. I guess I’m saying the same thing as the folks pointing out the Eclipse plugin scheme.

  11. Pingback: Michael’s Random Thoughts » Can Java be saved from death by feature bloat?

  12. Pingback: Bug Tracker Blog

Comments are closed.