Migrating to the new Tangled knot2 | jola.dev
Home
About
Blog
Newsletter
Projects
Talks
Tangled are in the process of migrating to the new knot implementation, knot2. As a quick reminder, the knot is a self-hostable server capable of serving git repos that uses atproto to share information with the ATmosphere, and the Tangled AppView. This means you can self-host your own repo, like gitea or forgejo, while still being able to interact with other people in a single place.
I recently wrote about setting up knot and spindle. I had gotten everything working a week earlier, but it was while I was writing the blog post that I found out about knot2. Oh, well, migrating can’t be too bad, right? And there’s actually a great guide already written up, as well as tooling to help you. This write up provides a more Docker focused version of the migration guide, with my personal experience.
Prep
Okay, I hit the first challenge right off the bat. The Containerfile provided by the Tangled repo pins an amd64 arch runner image that I can’t use, so I stripped that out to clone and build the image locally. The docker image contains the migration script, so that’s why I’m building it locally first.
git clone --depth 1 https://tangled.org/tangled.org/core /tmp/tangled-core
sed -i 's|gcr.io/distroless/cc-debian13:latest@sha256:[a-f0-9]*|gcr.io/distroless/cc-debian13:latest|' /tmp/tangled-core/knot2/Containerfile
docker build -f /tmp/tangled-core/knot2/Containerfile -t knot2:local /tmp/tangled-core
docker image inspect knot2:local --format '{{.Os}}/{{.Architecture}}'
And we have to create the KNOT_MASTER_KEY. Run this and store the value in a password manager.
openssl rand -base64 32
Okay, and the final little bit of setup before we do the dry-run.
printf 'KNOT_SERVER_HOSTNAME=knot.example.com\nKNOT_REPO_SCAN_PATH=/old-repos\n' > knot-migrate.env && chmod 600 knot-migrate.env
You need to have the KNOT_MASTER_KEY in the shell, so export it, and then run.
sudo -E docker run --rm --user 0:0 \
-v "$PWD/server:/old-db" \
-v "$PWD/repositories:/old-repos" \
-v "$PWD/keys:/old-keys:ro" \
-v "$PWD/knot-migrate.env:/old.env:ro" \
-v "$PWD/data:/target" \
-e KNOT_MASTER_KEY \
--entrypoint /usr/local/bin/knot-migrate knot2:local \
--source-db /old-db/knotserver.db \
--env-file /old.env \
--host-key /old-keys/ssh_host_ed25519_key \
--plc-url https://plc.directory \
--target /target \
--dry-run
This is just a dry run. It’ll print out some information. Make sure it says everything looks good! You can run this any number of times. Here’s what my output looked like.
rehearsal: 0.0s
knot owner: did:plc:3pnunm7komgxzfwrrxdtvsvl
members to grant: 2
repos to adopt: 1
collaborator grants: 2
casbin cross-check: the acl and the tables agree
skipped repos: 0
transfer mode: copy
the filesystem checks used /var/lib/knot, since the scan path doesn't exist yet
scan path: writable
room to copy: 93.8GiB free is enough for the 3.7MiB that adoption will copy
host key algorithm: ssh-ed25519
master key: KNOT_MASTER_KEY decodes to a usable key
we left the target alone. Re-run without --dry-run to migrate.
This is the last step before we go into the “scary” part, so make sure you’re happy with the output!
The migration
Okay, game day. It’s time to do this for real. Start by stopping your knot and do a backup.
docker compose stop knot
cp -a server/knotserver.db* /var/tmp/
And here’s the non-dry-run version of the migration command.
sudo -E docker run --rm --user 0:0 \
-v "$PWD/server:/old-db" \
-v "$PWD/repositories:/old-repos" \
-v "$PWD/keys:/old-keys:ro" \
-v "$PWD/knot-migrate.env:/old.env:ro" \
-v "$PWD/data:/target" \
-e KNOT_MASTER_KEY \
--entrypoint /usr/local/bin/knot-migrate knot2:local \
--source-db /old-db/knotserver.db \
--env-file /old.env \
--host-key /old-keys/ssh_host_ed25519_key \
--plc-url https://plc.directory \
--target /target
Here’s my output so you know what to look for.
adoption: 0.0s
cobs: 0.0s
knot owner: did:plc:3pnunm7komgxzfwrrxdtvsvl
members to grant: 2
repos to adopt: 1
collaborator grants: 2
casbin cross-check: the acl and the tables agree
skipped repos: 0
adopted by copy: 1 new, 0 already present, 1 sha1, 0 sha256
member grants: 2 appended, 0 already present
registrations: 1 appended, 0 already present
collaborator grants: 2 appended, 0 already present
knot key identity: did:web:knot.cove.town
host key algorithm: ssh-ed25519
host key fingerprint: SHA256:SKiHHmsnZV+E6fSl3MpvAH2mHQS6UH48XXW7NCeIMi8
config: /target/config.toml
key archive: /target/repo-signing-keys.json
Okay! Unlike knot1, knot2 wants a nice config file with lots of explicit and clean options.
[server]
hostname = "knot.example.com"
admins = ["did:plc:YOUR_DID"]
listen_addr = "[::]:5555"
ssh_listen_addr = "[::]:2222"
ssh_host_key_file = "/data/ssh_host_key"
appview_endpoint = "https://tangled.org"
[repo]
scan_path = "/data/repos"
[secrets]
sealed_key_file =...