Now the Infrastructure Is Boring Too (And That’s Still a Compliment) | by Roeyhadad | Zencity Engineering | Jun, 2026 | MediumSitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

Zencity Engineering

Zencity uses advanced Artificial Intelligence and machine learning to help transform millions of data points into meaningful actionable insights.

Now the Infrastructure Is Boring Too (And That’s Still a Compliment)

Roeyhadad

7 min read·<br>1 hour ago

Listen

Share

This is a follow-up to How We Made Deploying a New Service Boring — and That’s a Compliment. If you haven’t read it yet, the short version: we replaced dozens of per-service Helm charts with one shared chart (zc-monochart) and made deployment of a new service a non-event. This post is about what we did next.

The problem we didn’t solve<br>After monochart landed, deploying a new service became genuinely boring. A developer creates .infra/values.yaml, pushes a branch, and the CI pipeline handles the rest. Done.<br>But deploying a service is only half the story.<br>Almost every service we run needs something from AWS. An SQS queue to process background jobs. A PostgreSQL database. A Secrets Manager secret. An S3 bucket. Sometimes several of these at once. And for all of that, the workflow looked like this:<br>Developer adds a Terraform module block to the infrastructure repo and opens a PR<br>DevOps reviews the Terraform plan and sometimes helps with edge cases. The shared modules handled the common patterns, but corner cases still needed a second pair of eyes<br>CI runs, plan reviewed, PR merged<br>Terraform applies<br>Developer gets the queue URL back via Slack, hard-codes it into their config<br>For example, adding an SQS queue meant writing something like this, in a separate repo owned by a different team:<br># tf-survey/us-east-1/13-sqs.tf ← separate repo, separate team, separate context<br>module "sqs_my_new_queue" {<br>source = "terraform-aws-modules/sqs/aws"<br>version = "4.3.1"<br>name = "MyNewQueue"<br>visibility_timeout_seconds = "5400"<br>create_dlq = true<br>tags = merge(local.global_tags, { "Name" = "MyNewQueue-sqs" })<br>}This worked. But it had two problems. First, it pulled DevOps into routine work that a developer could reason about themselves. Second, and more fundamentally, a service’s truth was now split across two repos. Deployment config in the service repo. Infrastructure config in the Terraform repo. Understanding a service fully meant context-switching between two codebases.<br>The irony wasn’t lost on us. We had made deployments boring. But the infrastructure side of the same service lived somewhere else entirely.

The insight<br>There’s a question that kept nagging at us: where does a service live?<br>The deployment side had a clean answer. After monochart, everything a service needs to run lives in .infra/values.yaml: replicas, resources, environment variables, ingress rules, autoscaling. One place: an .infra/ folder with a shared base config and small per-environment overrides, owned by the team building the service.<br>But the infrastructure side had a different answer. The SQS queue a service depends on? Terraform, in a central infrastructure repo. The database? Same. The Secrets Manager secret? Same. And so on for every AWS resource the service touches. A developer who wanted a full picture of their service had to look in at least two completely separate places (the service repo and the infra repo) and mentally stitch them together.<br>So the goal became clear: all the configuration that a service needs should live in one place, together. The deployment and the infrastructure. One place, one repo, one source of truth: a shared base config with small per-environment overrides.<br>This pointed us toward a clean model: monochart becomes the developer-facing API for everything a service needs, both to run and to be provisioned.

Press enter or click to view image in full size

Enter Crossplane<br>Crossplane is a Kubernetes-native framework for provisioning and managing cloud infrastructure and APIs. The core idea: you define custom Kubernetes resources (called Composite Resources, or XRs) that represent infrastructure, and Crossplane reconciles them into real AWS (or any cloud) resources.<br>What makes this powerful is the abstraction layer. You write CompositeResourceDefinitions(XRDs) that define the API: what parameters a team can specify. You write Compositions that define the implementation: what AWS resources actually get created. Consumers only see the API; the implementation is hidden.<br>We started with four XRDs, covering the most common needs:<br>AppWorkloadSqs -provisions an SQS queue, optionally with a DLQ<br>AppWorkloadDatabase - creates a PostgreSQL database on the shared RDS instance<br>AppWorkloadSecret -creates an AWS Secrets Manager secret<br>AppWorkloadRole - creates an IAM role with EKS Pod Identity association<br>The model is extensible by design. Adding a new resource type means writing a new XRD and Composition, with no changes required to any service. These...