Notes on software quality

Notes on software quality

Jump to:

How I think about quality

Universal signals of quality

The six signals of quality in software

The benefits of software quality

Beliefs about quality I want to disprove

Quality is impossible at scale

Anecdotal evidence that quality is impossible at scale

Companies with dedicated quality efforts

Interface quality of life improvements

How I think about quality

Quality is the absence of problems

“The absence of problems” is the best definition I can come up with for quality.

The most practical way to measure quality is to test with many different people, and have many experts take a look. If thorough testing and 100 experts can’t find a problem, the thing is probably perfect.

Quality is a spectrum up to “perfection”

Perfection is impossible. The agreed-upon best-designed software in the industry has noticeable problems. But the closer you can get, the better.

The closer you try to get to perfection, the harder it is. This is the same “diminishing returns” effect you’ll see anywhere quality is involved.

Quality relies on the organisation

Leadership and culture both define and allow quality. Even if they don’t believe they do. Even if they haven’t thought about it. Their actions and beliefs will either enable or limit quality.

Quality is a result of ability and appetite. Does the organisation have people who can produce high quality (ability)? And does the organisation allow quality to be achieved (appetite)?

~100% of the time I’ve seen an organisation make high quality software, it is because the leaders wanted it. If the leaders don’t want quality, it will be much harder.

But don’t forget that whatever you work on, there is an amount of quality you personally can achieve. Even in the most rigid design-system-controlled interface, there are still higher quality and lower quality choices you can make.

An organisation can set objective measures of quality (e.g. bugs reported). Hopefully those objective measures don’t blind the organisation to the subjective measures (e.g. “does an expert think this is high quality?”).

Quality gets harder with scale

The larger the organisation or software, the harder quality is. High quality is impossible past a certain point of scale. Some organisations are incapable of making high quality software.

Importantly, this is a natural result of scale. It’s not a complaint or a problem to be solved. It cannot be solved. It’s a trade-off, like almost everything else in software.

Quality is not necessary, but ideal

I wish every organisation cared about quality. It’s clear that many of them don’t, and don’t need to. Many commercially successful organisations don’t prioritise quality.

But quality is ideal. Everyone wants to live in a higher-quality world. It feels good to create quality. Things last longer if they’re higher quality. And over the long term, commercial success is harder if people realise your brand doesn’t represent quality.

Universal signals of quality

These are the signals of quality that I’ve found apply to everything. People and artefacts.

Appearance: In (over)simple terms, the more beautiful something is, the higher quality we think it is. I assume this is because beauty requires effort, and so does quality in general. Any evidence that we have put care into something is evidence of further care.

Association: We judge whether something is high quality by what it is associated with. A common approach to this is social proof, where someone or something is associated with other someones or somethings. We judge a thing by the company it keeps.

Cost: If someone or something costs more, we might assume that is because it is worth more. But “cost” does not have to mean “money”. If something takes more time, for example, we might assume it is of higher quality.

Performance: Performance could be quantitative like “top speed”, or qualitative like “how does it make me feel”. It could be results-focused like “did it help me complete my task”. In simple terms, if appearance is “how it looks”, performance is “how it does”.

The six signals of quality in software

Reliability: Does the software technically perform as expected, all of the time? This means no bugs, no downtime, no errors, and so on.

Speed: Does the software react to input instantly, or as near as practically possible? For those tasks that must take longer, are they as fast as possible?

Clarity: Does the person using the software understand everything?

Efficacy: Can the person do what they need to do with the software?

Efficiency: Can the person do what they need to do with the software as easily as possible?

Beauty: Is the software as aesthetically pleasing as possible?

The benefits of software quality

Some might argue that quality is an obvious good. That the reasons quality is important are naturally understood by everyone. But we’ve still got a lot of crap software, so 🤷.

Attract employees: People want to work...