The hard part was never the model - by Jennifer Jiang-Kells

Cool Things in HealthTech For Devs

SubscribeSign in

The hard part was never the model<br>Why every serious clinical AI team rebuilds the same infrastructure from scratch — and where that leaves the market

Jennifer Jiang-Kells<br>Jul 02, 2026

Share

I spent the last few months talking to the teams actually deploying clinical AI in the real world: research and innovation leads, clinicians, and engineers at Stanford, NYU, Amsterdam UMC, Cambridge University Hospitals, and UCLH (both NHS). People who’ve moved models out of a notebook and into a live hospital. Two things came up in every single conversation:<br>Every serious clinical AI project builds the same infrastructure from scratch, because no off-the-shelf option exists.

The hardest part isn’t the models. It’s the integration.

Everything below is a gross over-generalisation. Healthcare is relentlessly specific, and that specificity is itself one of the things that makes this hard — but the shape holds.<br>Thanks for reading Cool Things in HealthTech For Devs! Subscribe for free to receive new posts and support my work.

Subscribe

The layers between a model and production

When you go from model to production, there are really two layers of infrastructure to build, and then a direction you can scale in beyond them.<br>Layer 1 — single-site deployment. Get a model running on hardware at one site. Vendors exist here — the one layer where off-the-shelf options really do — and the serious ones tend to have deep academic research roots: UPMC/CMU/Pitt consortium → Abridge, MIT → Layer Health. This layer is contained, billable, and solvable; but big Epic is always lurking at the edge of it.<br>Layer 2 — multi-site integration. Connect to live clinical data, scale across sites, make it auditable. This is the inevitable next problem the moment Layer 1 works: you have a model running inferences: now what? One way or another you need data from existing systems, and ideally you need to run at more than one hospital. This is the hard part everyone talks about, and no single vendor owns it. The ecosystem is too fragmented for anyone to own the whole thing.<br>There’s another direction entirely: scale vertically instead of horizontally. Up from a single site, rather than out across many.<br>Going vertical — evaluation . Add evaluation, monitoring, and auditing on top of a single site. Stanford’s ChatEHR → MedHELM is the notable example: the rare one built entirely from scratch, in-house, and open. There’s a reason that only happens at Stanford, one of the best-resourced academic medical centres for engineering talent on earth: with no inherited infrastructure, both the deployment and evaluation layers are first-party research, and even then it takes three or four years to get the boring stuff out of the way first.<br>Startups can move faster into the eval space, though not from a standing start — they’re importing infra expertise earned elsewhere. So the speed still tracks founder pedigree, not a gap that’s closing for everyone; building good eval remains gated by rare talent. But the expectation is shifting industry-wide: evaluation is moving from a later bolt-on to the standard a Layer 1 vendor is expected to ship with, not something they add once deployment works.

“Integration” means two different things

In true healthcare fashion, “integration” means something different depending on who’s saying it.<br>To a clinical lead , integration is the gap between a model trained on retrospective data and a live system running on FHIR APIs. It’s what turns a cool concept at a local hospital into an embedded, end-to-end system that scales — the thing that proves concrete value to patients and policymakers. Integration means not just having a model, but embedding it into real-time clinical workflows. Much harder.<br>To the developers , it’s everything from the ground up — the access, the certificates, the networks, the governance. Code is the easy part. Governance, operations, and security are the invisible burden of healthcare engineering. The tech never exists in a vacuum; it lives inside systems. To know whether something is even feasible, you think about data storage and security before anything else. Integration means not just writing code, but dealing with people problems. Also much harder.<br>Both are really the same Layer 2 problem seen from two chairs — one sees the workflow, the other sees the wiring.<br>It’s a talent problem wearing a technical costume

Underneath both is the same constraint: there aren’t enough engineers who know both AI/ML and FHIR, and onboarding new devs into healthcare is slow. Sometimes there isn’t enough on the table to make them stay — and when they leave, the institutional knowledge leaves with them. In the research-to-production gap, you often have one developer in a garage carrying the whole burden of proving a model can help, and you don’t want to scare them off with FHIR — or, god forbid, HL7 feeds.<br>So the...