How Crossplane providers work and how to implement one | Bruno Luiz da SilvaPhoto by Guillaume TECHER on UnsplashIntroduction#<br>Crossplane allows you to define Kubernetes APIs without writing controllers. Instead, users define custom resources using (usually) YAML. There are many components required for this abstraction, such as compositions and XRDs, but providers are on the heart of it: they are controllers that handle the lifecycle of an external resource (e.g. a bucket), such as creating, updating, and deleting it.<br>Usually official providers or upjet generated ones are good enough, but sometimes you might come across issues. This is when you will need to implement your own, generally in Golang.<br>This post will cover my learnings while implementing providers from scratch, from how they work and how to implement one, to best practices and examples.<br>Why create your own provider?#<br>Sometimes you will be able to find an official provider (AWS/GCP) that is decent and well maintained, but other times you won&rsquo;t since Crossplane still does not have as much coverage as Terraform. Hence, there are two options: generate a provider using upjet or implement a provider yourself.<br>Upjet leverages the Terraform ecosystem and it generates providers that hook to its providers. In many cases this is enough, especially for well maintained Terraform providers. Many of the official Upbound and Crossplane providers use it and it is the quickest way to implement a custom provider. But, there are a few caveats:<br>Not all vendors will have a proper Terraform provider, if they have one at all<br>Some Terraform providers might be incompatible with Upjet<br>Teams might want to have custom logic between reconciliation logic (e.g. emit specific metrics)<br>Generated providers can be slow or memory-intensive when Terraform overhead is a poor fit<br>If you hit one of the above, you are left with implementing a provider from scratch.<br>Start with the provider template#<br>Don&rsquo;t panic: you won&rsquo;t start fully from scratch. The Crossplane team maintains a provider template, which is a good starting point for most providers and also they maintain very popular providers (provider-http and provider-opentofu) you can base yourself on.<br>The template gives you the repository structure, build tooling and scaffolding utilities (e.g. make provider.addtype). All generated types will be placed in internal/controller/{} and those will define the reconciliation hooks described in the next section (example). The provider&rsquo;s behaviour comes from how those hooks interact with the external API.<br>Implementing the Crossplane reconcile loop#<br>ℹ️ The following assumes that default management policies are in place, as otherwise they change the lifecycle behaviour.<br>Crossplane providers use the controller pattern through a managed reconciler abstraction. Instead of a single Reconcile function, provider controllers implement a strict interface with several hooks. The Crossplane runtime manages their lifecycle, ordering, and state, handling work that a controller built with Kubebuilder would otherwise need to implement (and probably lose some hair when bugs end up surfacing).<br>Lifecycle hooks in a nutshell#<br>Setup runs once to register the controller for a &ldquo;managed-resource&rdquo; kind, but it is not part of the reconciliation loop. After it is all setup, the controller runtime will start reconciling and each time it will call Connect, Observe, and then Create, Update, or Delete when needed before calling Disconnect.<br>Besides these specific hooks in the reconcile loop, Crossplane leverages annotations to keep track of reconciliation state. The most important one is crossplane.io/external-name, which identifies the underlying resource via a stable lookup key (e.g. ID, ARN, or resource path):<br>Before Connect and Observe, the managed reconciler&rsquo;s default initializer persists metadata.name as the external name when the annotation is absent. Providers overwrite it during Create only when the external system returns a different stable lookup key.<br>Users can pre-populate it to identify and import an existing resource, although it should be used together with an Observe management policy to prevent unintended changes (docs about Crossplane resource import).<br>If management policies are set to * (the default), so that no hook is excluded, the provider lifecycle can be summarised as:

flowchart TD<br>Setup["Setup controller(once)"] --> Start<br>Start["Start reconciliation(loop)"] --> Connect<br>Connect --> Observe{"Observe"}

Observe -- "!Deleting AND !ResourceExists" --> Create["Create() external resource"]<br>Create --> CreateName["Persist external-name"]<br>CreateName --> Disconnect["Disconnect"]

Observe -- "!Deleting ANDResourceExists AND!ResourceUpToDate" --> Update["Update() external resource"]<br>Update --> UpdateStatus["Persist resource status"]<br>UpdateStatus --> Disconnect

Observe -- "!Deleting AND ResourceExists AND ResourceUpToDate" --> Disconnect

Observe -- "Deleting AND...