airflow — Green, Red, and Blue Package Skill Manual

Skip to content← Back to getcolors/airflow◆ airflowGreen · Red · Blue · Airflow · Opentofu · Ansible

#Introduction<br>The Airflow Package Skill manages one VPS running Airflow with LocalExecutor. PostgreSQL runs on the host, containers run Airflow and Caddy, WAL-G continuously archives the metadata database, and a restricted deploy key pushes DAGs from GitHub Actions.<br>Green, Red, and Blue consume the same colors.yml, render byte-identical artifacts, and manage the same state. Never run them concurrently.<br>Safe first steps build renders local files and create --dry-run prints the workflow without contacting providers. Review both before approving a real operation.

#Quick start<br>npx skills add getcolors/airflow --skill package-airflow-green<br>cp .agents/skills/package-airflow-green/green ./green<br>./green build<br>./green create --dry-run<br># after explicit approval:<br>./green createEquivalent Red and Blue skills ship red and blue launchers.

#Workflow architecture<br>create/build start ─ compute ─ smtp ─ dns ─ smtp-post ─┬─ ansible-local<br>└─ ansible-remote ─ github

delete start ─ github ─ cleanup ─ smtp-post ─ dns ─┬─ smtp<br>└─ computeGitHub publication follows remote configuration so a workflow cannot use a key before its public half is installed. Delete withdraws that credential before destroying the host.

#Desired state<br>AreaKeysAirflowairflow-host, image, admin username, SMTP From addressDAG deliverydags-repo, dags-dest, dags-branchDatabasepostgres-version, password environment variableBackupsWAL-G version, R2 bucket/endpoint, schedule, retention, freshness thresholdProvidersCompute, DNS, SMTP, and state backend selections<br>Images require explicit tags. Optional values should be omitted rather than set to REPLACE_ME. See the full configuration reference.

#Server design<br>Docker is configured before PostgreSQL so its bridge address is stable. WAL-G is installed before archiving is enabled. The deploy account is created last and is restricted to write-only rrsync access to the DAG directory.<br>Caddy is the only public route to Airflow’s API server. It computes and verifies a bcrypt password on the machine, obtains TLS with HTTP-01, and provides the single-operator authentication boundary required by this deployment.

#PostgreSQL and recovery<br>The PostgreSQL major version is pinned because WAL-G base backups do not restore across major versions. A base backup is taken during create; a persistent systemd timer takes later backups, and a separate hourly freshness check detects missing backups.<br>Keep the Fernet key COLORS_PAR_AIRFLOW_FERNET_KEY encrypts stored Airflow connections. A restore with a different key cannot decrypt them.

#DAG delivery<br>The package creates a private repository if needed, seeds it once over the operator’s SSH identity, and publishes a fresh deployment key to a GitHub Actions environment named after the profile. Existing repository contents are never overwritten.<br>The server-side key is constrained with restrict and write-only rrsync. It cannot open a shell, read DAGs back, or use sudo.

#Command reference<br>CommandEffect./COLOR buildRender only./COLOR create --dry-runPrint create graph./COLOR createConverge infrastructure, backups, Airflow, and DAG delivery./COLOR delete --dry-runPrint delete graph./COLOR deleteWithdraw credentials and destroy managed infrastructure; DAG repository remains

#State and generated files<br>.colors//<br>├── airflow-compute/<br>├── tofu-smtp/<br>├── tofu-dns/<br>├── tofu-smtp-post/<br>├── airflow-ansible-local/<br>├── airflow-ansible-remote/<br>└── airflow-github/Remote keys use /.tfstate. The three delegated ONCE stages retain their tofu-* names, making profile uniqueness essential.

#Credentials and safety<br>Credentials use COLORS_PAR_* and never belong in desired state.<br>Never export COLORS_PAR_PROFILE.<br>Keep compute-prevent-destroy: true; deletion removes the metadata database.<br>Use a dedicated, narrowly scoped backup bucket separate from state.<br>A Cloudflare zone must not be co-managed by another Colors deployment.

#Troubleshooting<br>DAG import failures under load<br>Check memory first; DAG parsing is often the first component to fail on an undersized host.<br>No backup alert arrived<br>Inspect both the base-backup timer and freshness timer. A timer that never ran cannot trigger its own OnFailure.<br>Authentication has two login screens<br>The intended public boundary is Caddy. Verify the API server is not exposed directly and SimpleAuthManager’s all-admin setting is present.