nordisk/holger

6. Pairing

Pairing is how the boxes of an estate (front, main, twin) and the downstreams find each other and decide whom to trust. The pairing code is the crate nordisk-pairing (Skidbladnir, pinned in Cargo.lock), shared with Gunnar. Holger's side is server/cli/src/paired.rs, holger-estate-front/ and server/lib/src/sync/.

The boxes and their links

                      the ONE box (installer)
                             │ udp/51821 wg-installer → tcp/8474 bootstrap (mTLS, installer CA)
                             ▼
 internet ──443──► FRONT 10.44.0.1 ──── wg0 udp/51820 (+ PSK) ────┬──► MAIN 10.44.0.2/.3
                     │                                           └──► TWIN 10.44.0.2/.3
                     │                                                 ▲
                     │                          replication door (gRPC port + 1, TLS 1.3, client cert)
                     │
                     └── holger-ds 10.46.0.1/24, udp/51830 ──► DOWNSTREAMS (sync door tcp/50070)

The front dials the boxes only at their overlay addresses (holger-estate-front/src/wg_only.rs).

Seeds and certificates

The ONE box makes one seed per box. A seed is a TOML file, <root>/pairing/seed.toml, where <root> is HOLGER_APPLIANCE_DATA or /data. It holds (Seed, nordisk-pairing seed.rs):

• estate, kind (here: Holger), tunnel (default wireguard), peers_expected;
• installer_ca_pem: the only root the box trusts;
• bootstrap_cert_pem and bootstrap_key_pem: one pair per medium, the same for every box of it;
• installer_link: the installer's WireGuard public key, the box's private key for that link, and the port (51821).

It holds no per-box value and no pre-shared key. The front's tunnel key comes in the join reply, and the pre-shared keys come in the feed (below).

How the seed reaches the box. For a medium built with a seed, the seed is a staged file (the installer parameter pairing_seed, never a kernel command line; holger-recipe/src/packaging.rs). The media the ONE box lays from carry no seed: their seed.toml holds only the marker nordisk-pairing-seed-from = "upcloud-metadata". At first boot the box reads its seed from the cloud's metadata service (169.254.169.254/metadata/v1/user_data, 36 tries 5 s apart) and writes it over the marker, mode 0600. A seed of another kind than Holger is refused ([pairing-seed-kind]).

The installer's own material (the CA certificate and key, the installer's certificate and key, the link file) is never on a medium. A sold estate keeps it as the bundle installer-material/1, sealed in the main's admin-only raw repository monetize-backup.

The ceremony at first boot

The installer dials out to every box; a box never dials the installer.

1. The box raises wg-installer and serves its bootstrap listener on tcp/8474.

2. Both ends show certificates under the installer CA: the box its bootstrap certificate (SAN bootstrap.installer); the installer its client certificate (CN installer). The box serves no other client.

3. The installer calls, in order:

   • GET /install/hello: the box answers a JoinRequest: its kind, its WireGuard public key, its capacity, and a CSR for a key the box made itself;
   • POST /install/feed: the installer sends the JoinReply: the box's name, the overlay, its own certificate, its role, its peers (the front first) and the feed, which carries the box's WireGuard pre-shared key (wg_psk; the front's feed holds wg_psk.<box>);
   • GET /install/offer, GET /install/secret-offer: what each box offers the others, fed on to every box;
   • GET /install/status, then POST /install/close.

4. The bootstrap certificate may join only a slot that has not joined yet (join-spent), and it is withdrawn once every slot has joined (join-withdrawn).

5. The weakest box becomes the twin. The role is written once, in holger-role.ron.

6. The box keeps pairing/own.key, own.pem, joined.toml (sealed at rest), signal.toml and closed. A fresh install clears them.

Trust rests on the pinned installer CA. There are no fingerprints to compare by hand.

After the close the installer can still call /install/refeed, /install/cert, /install/renew and /install/bootstrap on the same mTLS link.

The twin's replication certificate comes from the front: TwinCsr → IssueTwinCertificate → InstallTwinCertificate, recorded in pairing/certified.toml (holger-estate-front/src/certify.rs). The console bearer is derived on both ends from an x25519 key each side makes under console/ (holger-estate-front/src/console_key.rs).

Downstreams: signed invites

A downstream is not laid by the ONE box. It is enrolled by an invite from its main.

• Mint. POST /-/sync/downstreams/invite?name=<downstream>&hub_key=&endpoint= (Repository Admin). Empty hub_key and endpoint come from holger-sync/front-hub.ron. The enrolment is kept in holger-sync/downstreams/wg/<name>.ron.
• Signature. The invite is signed with the main's Ed25519 sync key (<data>/holger-sync/sync-key.pk8) as a Signed envelope of kind Invite (sync/src/invite.rs).
• Format. holger.downstream-invite/v2 (sync/src/invite.rs): upstream, downstream, downstream_key, hub_key, endpoint, address, hub_address, door, issued_at, expires_at and psk, the link's pre-shared key. An invite lives 7 days. A v1 invite (no key) is still read; it carries no PSK.
• Pre-shared key. The main makes one per enrolment (32 bytes from the OS generator) and makes a new one when the downstream's WireGuard key changes. The enrolment record is mode 0600.
• Redeem. On the downstream: POST /-/sync/upstreams/invite?upstream=<name> (feature downstream-wg), or the RON field invite: Some("/etc/holger/<name>.invite.json"), read at start.
• What the downstream keeps. <data>/holger-sync/enrolled-<upstream>.json (mode 0600), and its WireGuard key in holger-sync/wg.

There is no CLI verb for invites yet. (holger pair downstream|upstream|repair|list is a different thing: it pins the mirror mTLS certificates of a peer in holger-peers/*.ron.)

The front polls GET /-/sync/downstreams/wg every 5 s and lists each enrolled downstream on its hub holger-ds, with its pre-shared key (RelayPeer); it lists a downstream again when its key changes and refuses a blank one (holger-estate-front/src/downstreams.rs). The console's list (WgPeer) never carries the key.

WireGuard

• Estate wg0. Each box has its WireGuard key in <root>/wg. Every link between the front and a box carries a pre-shared key from the feed (wg_psk); a blank one is refused ([wg-psk-blank]). The PSK adds a symmetric secret to the classical handshake, against a later quantum break of the recorded traffic.
• Downstream hub, udp/51830. Each downstream's link carries the pre-shared key from its invite (above). A downstream enrolled before keys existed is listed without one.
• The installer link (wg-installer) has no PSK. Its payload is the installer's mutual TLS.

Certificate renewal

• Rule. A certificate is renewed at two thirds of its life. A renewal keeps the same key and the same CN and may not end earlier; anything else is refused ([renew-other-key] and its siblings).

• On the front (jera jobs, holger-estate-front/src/renew.rs):

  • front.renew.twin: the twin's replication certificate (365 days);
  • front.renew.boxes: the box certificates (180 days) and the installer CA (180 days, the same key, so pins hold). The new CA goes back into the sealed slot, and each box gets a fresh bootstrap pair through /install/bootstrap.
  • The jobs are submitted every 120 s and look at the links every 6 h. A failed look is tried again after 10 minutes, doubling; the third failure in a row raises ALERT [front-renew]. The state is in pairing/renewal.toml.

• On each box. The box renews its replication-door identity hourly (box.renew.door).

• The public certificate. The front takes Let's Encrypt (ACME) and renews it with 30 days left. GET /front/certs shows the state; POST /front/certs/try tries now.

What a twin rebuild keeps

A dead main is rebuilt from its twin: the front re-admits the box with a fresh seed from the kept bundle, the ONE box lays it again, and the new main starts empty and held, then recharges from the twin's replication door as holger-recharge (holger-estate-front/src/recharge.rs).

Kept (the ESTATE list, server/lib/src/replication.rs):

• holger-sync: the sync key, so every downstream keeps trusting its main, the downstream records with their pre-shared keys, and front-hub.ron;
• holger.ron, holger-passkeys, holger-entitlements, fleet/signups.json, the audit log and the stores;
• the sealed monetize-backup repository.

Not kept, made new on the new box: the role record, the box's own TLS material (holger-replication-issuer, holger-twin-tls, holger-recharge-tls), the recharge markers, fleet/, and the pairing state.

Still in v1

• Nothing; v1 had no pairing page.