Chris James - We learned this lesson already

In Why "Build vs Buy" is the Wrong Question I explained how you should categorise your systems into appropriate "subdomains" (a la Domain-Driven Design), to help better understand whether to build or buy.

Core - your business's unique competitive advantage, keep in house and invest.

Supporting - "boring" software that exists to serve your core. Because it supports something unique to your business, it tends to need to be custom built - but it's not worth over-investing in beyond what your core needs.

Generic - domains where external vendors have deep expertise you don't need to replicate. Payroll, HR, email infrastructure and so on.

The key message being that whilst you absolutely should buy generic systems, you mustn't mistake a supporting subdomain for a generic one. Trying to vendor a subdomain that is supporting your core can lead to many issues.

This post is about what happens when you do misclassify, and how the process makes it worse.

An introduction to RFP

An RFP (Request for Proposal) is a formal document sent to vendors inviting them to bid on supplying a product or service. In the context of software, it typically involves specifying your requirements in detail and asking vendors to respond to each one.

Unlike in-house software, a vendor commitment is relatively immutable. You need to fully understand what you're buying upfront, otherwise you risk an expensive and difficult mistake to unpick.

This should sound familiar

Specifying everything upfront, then committing. The Software Crisis, Waterfall, decades of failed projects.

This is not conjecture. One widely cited analysis of 5,400 IT projects found that 17% went so badly they put their company's reputation at stake, and on average IT projects run 45% over budget. Waterfall-based approaches were a key contributing factor.1

The industry's response was "agile", iterative development, fast feedback, and perhaps most importantly - acknowledging that you cannot fully specify software up front. Agile is a rejection of the idea that you can know everything.

The Agile Manifesto puts it plainly

(we have come to value) Responding to change over following a plan

This instinct to specify everything before building anything keeps resurfacing in new forms. A recent rebuttal of Spec-Driven Development (SDD) puts it well:

The what is discovered by building, not defined before it.

The RFP process makes the same assumption SDD does, that the what exists in advance, fully formed, waiting to be written down clearly enough. For your generic domains, probably. For the others, unlikely.

By committing to fixed scope, you're letting go one of the fundamental benefits of software - if you own the code, you can change it. The product might be right today, but it will likely need to evolve in the future if it's to continue to be useful.

The RFP process is waterfall in disguise

To vendor responsibly, you must specify the requirements exhaustively upfront - you are being asked to make your most consequential decisions with your least amount of knowledge. Consider the software products you admire most. They were not built from a requirements document - they were built iteratively, through cycles of building, measuring, and learning. The RFP process asks you to skip all of that.

And what's the cost of being wrong? A huge waste of engineering time spent working around the constraints of a commitment you can't easily unpick.

Analysis Paralysis

The RFP process takes time - significant time:

The average RFP timeline spans 6 to 10 weeks for most organisations, though this can extend to 12+ weeks for complex projects. 2

People agonise over spreadsheets for hours on end, hypothesising use cases that may never see the light of day. And as decades of software development have shown us, no matter how long you spend analysing, you're likely going to be wrong anyway.

It's tempting to look at a 10 week timeline and assume there's bureaucratic fat to trim. But the length isn't a function of inefficiency - it's a function of the task. Real understanding of a problem comes from building, shipping, and getting feedback from the people who actually use it. The RFP process replaces all of that with the opinions of a handful of stakeholders who won't write a line of code or click a single button in the finished product.

Even the fear of being wrong, corrupts the process. While staring at the spreadsheets, you may lean into MoSCoW prioritisation (Must have, Should have, Could have, Won't have). This model exists to manage uncertainty around scope and helps prioritise. Even though it predates "agile", it leans into one of its tenets, that not everything can and should be delivered.

In this context though, declaring something as a Should is a risky move. What if it's actually a Must and the vendor can't do it? Under that pressure and fear, everything becomes a Must. The tool designed to help make smart decisions around scope,...