nordisk/gunnar

name: root-key-custody description: "Where the appliance's ed25519 ROOT SIGNING KEY should live — four options with what each protects against and what it does not, the rotation and revocation answers, why this key is unlike the three other key mechanisms in the estate, and the one ruling left open. Written for the owner to choose from; lane T10 implemented the mechanism so that any of the four works." metadata: node_type: decision

Where the root signing key lives — the owner's choice, with the trade-offs

Lane T10, 2026-09-21. The mechanism is built and merged; the custody is not decided, because it is not a lane's to decide. This page is the choice.

DECIDED 2026-09-21: t14s + oden, two copies, both 0600

The owner's ruling, verbatim: "keep the key at oden and t14s".

This is option A of section 2, held on two machines. Everything section 2 says about A stands, and one line is added to it: two copies means two machines whose compromise is a compromise of the key. The owner chose availability (either box can compose an enrolled medium, and losing one box does not lose the ability to build media the fleet accepts) over the smaller exposure of one copy. That is a legitimate trade, and it is the one written down here.

Neither box has a TPM or a hardware root of trust, so the protection is what section 2 says A's is: a compromised medium cannot author code; a compromised t14s or oden can.

Rotation under this custody

1. On t14s: dwarves root-key rotate. The old seed moves to ~/.dwarves/root/retired/<old public hex>.seed and is kept; a fresh seed takes its place.
2. Copy the new seed to oden over ssh to the same path, 0600, directory 0700, exactly as the first copy was made — and copy the retired seed to oden's retired/ too, so both boxes can still build a medium the old fleet accepts.
3. dwarves root-key show on both; the fingerprints must match each other and not the old one.
4. Compose new media under the new key. Machines installed from old media keep booting against the old key until they are reinstalled — see section 4.

Nothing travels through a log, a scratch file, a report or a chat. The copy is a single ssh stream from one 0600 file to another.

If one box is lost or suspected compromised: rotate on the other, as above, and treat every medium composed under the old key as signed by a key that may be in someone else's hands (section 5 — outrun, not revoke).

0. One page of context, so the options mean something

The appliance verifies its payload twice, and the two are not the same claim.

• blake3 says the payload is whole — the bytes are the bytes that were pinned. Checked over the whole archive at every boot and per-chunk during extraction.
• ed25519 over the blake3 root says the payload is ours — somebody holding the root signing key blessed exactly these bytes.

Everything about the first was already true and measured. The second was mechanically complete and never armed: every medium in this estate was composed with KORP_ROOT_SIGNING_KEY unset, so no root.pub and no app.sig reached the ESP, korp.require_signed=0 was baked into the signed first stage, and every appliance printed, honestly, signature-over-root=NOT-ENROLLED.

That is the estate's own threat model ATTACK 1 (edda/.nornir/installer-threat-model-2026-08-16.md), whose cmdline half was closed in August: an attacker with ESP write access writes their own payload to P2 and its blake3 into app.blake3, and the machine boots. With an anchor enrolled, that attack ends — the attacker's payload has a different blake3 root and the signature does not cover it.

The whole of what is left is a key and the ceremony around it. The key is what this page is about.

1. What makes THIS key different from the other three in the estate

The estate now mints keys in four places. They should not look different for no reason, so here is the reason.

Three of those four are per-box identities minted where they live. That is exactly why the one-medium ruling works: a twin does not need a key handed to it, it makes its own and the primary pins it. Their custody story is "the box holds its own identity, and losing a box loses one identity."

The root signing key is the opposite of all three. It is one key for the whole product line, it must be held off the machines it protects (a key on the appliance would be a key the attacker who took the appliance now has), and its compromise is not one box — it is every medium ever composed with it, past and future, because those media carry its public half burned into their ESP and will accept anything it signs forever.

Should they share machinery? Partly, and they already do more than they look:

• Format: all four are ed25519. An ed25519 public key is the same 32 bytes raw, inside an X.509 SubjectPublicKeyInfo and as an ssh-ed25519 key — lane R10 measured this (edda-wt-r10, crates/korp-installer/.nornir/key-lineage.md) and concluded the boot anchor and the X.509 provenance signer can be one key. Nothing in this page forecloses that; dwarves root-key mints a raw 32-byte seed, which is the shape all three envelopes are derivable from.
• Custody machinery: shared, and deliberately. dwarves root-key reuses the dwarves vault (Argon2id + AES-256-GCM, the constellation's one at-rest store) and the same 0600-from-the-first-byte file discipline dwarves key generate already uses for the git deploy key. No fifth secret store was invented.
• Custody STORY: must not be shared, and that is the answer to the owner's question. A per-box identity is fine minted in place by the box; a product-line root key minted in place by a build box is exactly the property section 2 asks the owner to decide about. Sharing a story here would be the accident, not the design.

2. The four options

All four work against the mechanism as built. Option A is what dwarves does if nobody chooses, and it says so every time it runs.

A — a 0600 seed file on the build box (the labelled default)

dwarves root-key generate → ~/.dwarves/root/korp-root.seed, mode 0600, directory 0700, minted from the OS CSPRNG, never printed.

• Protects against: a compromised medium. ATTACK 1 ends. An evil maid, a stolen disk, a hostile recovery shell, anyone who reaches the ESP — none can author a payload the appliance accepts.
• Does NOT protect against: a compromised build box. Anything that gets root there reads the seed and can sign media that every appliance in the fleet accepts, indistinguishably from us. This estate runs on rented UpCloud VMs with no TPM and no hardware root of trust, so there is nothing on the box that can hold the key better than the filesystem can.
• Also does not protect against: a build box that is backed up, snapshotted or imaged by the hosting provider. The seed is 32 bytes in a file; it travels in every copy of that disk.
• Cost: zero. One command.

B — sealed in the dwarves vault on the build box

dwarves vault add korp-root-key (Argon2id + AES-256-GCM at rest, passphrase from $DWARVES_VAULT_PASSPHRASE or an unlock session). dwarves root-key resolves it as rung 2 of the custody ladder.

• Protects against: everything A does, plus a disk image, a backup or a snapshot that leaves the box — the seed in those is ciphertext.
• Does NOT protect against: a compromised build box while a build is running, which is when the vault is unlocked and the passphrase is in the environment or the session cache. Nor against an attacker who reads $DWARVES_VAULT_PASSPHRASE out of a CI configuration.
• Cost: a passphrase somebody must supply, and an unattended compose needs that passphrase somewhere — which is where most of B's advantage over A goes if the answer is "in a file on the same box".

C — an offline machine, signing detached

The key never exists on the build box. The compose prints the payload's blake3 root; the root is carried to a machine that is not on any network; there, dwarves root-key sign --root <hex> --out <dir> writes root.pub and app.sig; those two files are carried back and staged.

• Protects against: everything A and B do, plus a fully compromised build box. An attacker who owns the builder can make it produce any payload they like — but they cannot get it signed without the offline machine, and the human at the offline machine is looking at a blake3 root they can compare.
• Does NOT protect against: a human who signs whatever root they are handed without checking where it came from. C's security is exactly the diligence of the person at the offline machine; mechanically it is a second box that says yes.
• Cost: every release is a manual step with a physical transfer. Real, and the honest reason most shops do not do it.

D — split custody: two people, two halves

Not implemented, and it is the one option this page is not offering as ready. Threshold ed25519 (FROST, or a naive k-of-n over the seed) means no single person can sign a medium.

• Protects against: one insider, and one compromised laptop.
• Does NOT protect against: the build box, unless it is combined with C.
• Cost: a real cryptographic dependency and a real ceremony. The LAW says reuse before invent, and nothing in the constellation does threshold signatures today. If the owner wants D, that is a lane, not a flag.

What a hardware token would change, and why it is not on the list

A YubiKey or similar holding the seed non-extractably is C's protection without C's physical transfer, and it is the shape most products land on. It is not offered here because it needs an FFI into PKCS#11 or a USB HID stack, and the LAW puts FFI last. If the owner wants it, the reachable pure-Rust shape is a token that speaks over a serial or network protocol rather than PKCS#11, and that is a lane with a purchase order attached — not a choice that can be made on this page.

3. The recommendation, for what it is worth

Superseded by the owner's ruling at the top of this page. Kept as the record of what was recommended.

B now, C for anything a customer pays for, and say so out loud in the build record either way.

The reasoning: A and B both stop ATTACK 1, which is the hole that is open today, and stopping it this week is worth more than a perfect custody story next month. B costs one passphrase over A and takes the backup/snapshot exposure off the table, which on rented VMs is not a theoretical channel. Neither one can honestly claim to survive a compromised builder, and the verb says so on every run rather than letting a green build log imply otherwise.

C is what the metered lineage deserves, because a metered medium installs the boxes customers pay for, and it is already reachable — dwarves root-key sign exists precisely so C is a procedure rather than a project.

This is a recommendation, not a decision. The owner rules.

4. Rotation

Answered, implemented, and it is the part most key ceremonies leave for later.

dwarves root-key rotate mints a fresh key and moves the old seed to ~/.dwarves/root/retired/<its public hex>.seed rather than deleting it. The file is named by its own public key so a medium found in the field can be matched to the seed that can rebuild something it will accept.

What happens to media already in the field. Nothing, and that is the point. An appliance verifies against the key baked onto its own ESP. A machine installed from a medium signed by the old key keeps booting, keeps verifying, and keeps refusing anything not signed by that old key. It does not know a rotation happened and it does not need to.

The consequence, stated plainly: after a rotation, a medium that those machines will accept can only be built from the retired seed. That is why rotation keeps it. A rotation that deleted the old key would be a fleet-wide outage with no recovery path, because an appliance with shell = false cannot be fixed by hand. The verb refuses to overwrite for the same reason, and names rotate as the word for doing it on purpose.

Rolling a machine forward onto a new key is a reinstall from a medium composed with the new key — the anchor is a property of the medium, read once at mastering. There is no in-place re-enrolment and this page is not proposing one: a running appliance that could be told to trust a new root key over the network is a running appliance that an attacker can tell to trust theirs.

5. Revocation

There is no revocation today, and there cannot be one that is only a key mechanism. Said precisely, because "we'll add revocation later" is how this kind of gap survives:

• An appliance's trust anchor is 32 bytes on its own ESP. Nothing it consults at boot is on a network — deliberately; the chain is airgap-safe and the clock work of 2026-09-14 went out of its way to keep it that way.
• So a compromised root key cannot be revoked from anywhere. It can only be outrun: compose new media under a fresh key and reinstall. Until a machine is reinstalled, it will accept anything the compromised key signs.

What partially covers it, and how far. The rollback floor (ATTACK 4, closed against replay in August) refuses a payload whose signed version is below the machine's floor, and where a TPM exists the floor is witnessed by an NV monotonic counter. That stops an attacker replaying an old genuine payload. It does not stop an attacker who holds the key from signing a new payload at a higher version, which is exactly what a key compromise buys them. So the floor is anti-replay, not anti-forgery, and it should not be reported as revocation.

The honest posture, and it is the same one Secure Boot's dbx exists to improve on: detection and reinstallation, not revocation. What would move it — and what is deliberately not being built in this lane because it is a design the owner has not asked for — is a second, offline-rotatable anchor: a short- lived signing key whose certificate is signed by the root and carried on the medium, so a compromise of the day-to-day key expires on its own. That is a design, and it belongs in front of the owner before it belongs in code.

6. The one ruling left open

T2 asks: should the SEALED lineage (gunnar-installer, the default) also refuse KORP_ROOT_SIGNING_KEY=dev? Today only the metered lineage refuses it.

The facts, gathered:

• DEVELOPMENT_SEED is korp DEVELOPMENT key NOT SECRET! — 32 bytes of ASCII published in korp-installer/src/rootkey.rs. Anybody who has read that file can sign a payload any medium enrolled with it will accept.
• Enrolling it is arguably worse than nothing: a machine anchored to the dev key reports signature-over-root as ENROLLED. An absent anchor at least says NOT-ENROLLED, which is a true statement an operator can act on.
• The argument therefore does extend past metered. gunnar-installer is the general product, not only the estate that sells.
• But widening it breaks installtest, which arms dev deliberately, on the default variant, to prove the enrollment path end to end. That is a real test of a real mechanism and it is the only thing that exercises it.

Recommendation: widen the refusal to gunnar-installer, and give installtest the community variant instead. gunnar-installer-open already exists, already takes dev, and an enrollment-path test does not care which lineage it runs on — it cares that root.pub and app.sig reach the ESP and verify. That keeps T2's rule matched to the measured fact (a published seed must never anchor a shipping medium) without losing the one test that proves the path.

The alternative — leave it as T2 implemented it — is defensible on the ground that it is the narrow rule the measured facts support, and that a rule which breaks a green test on the day it lands tends to be reverted rather than obeyed.

The owner rules. Nothing in this lane depends on the answer.