Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/Skidbladnir

.nornir/socket-deployment.md

Skidbladnir OWNS the Build Thing SOCKETS

Upstream design: workspace_nordisk/.nornir/socket-deployment-design.md — the bulb/socket metaphor, the type→socket table, and the cross-repo split live there. This doc is the authority for what Skidbladnir actually implements, in src/socket.rs behind the sockets cargo feature (additive, off by default, never perturbs the shipping install/PKI paths).

A Build Thing is a bulb (an output artifact of a TYPE); every type has a matching socket — the fixture graph it plugs into to light up and be observed. The dwarves ⚒ pane reads the bulb's type and fires the matching socket automatically. Sockets carry a Mode { Test, Prod }, so this is a deployment/topology system and "test" is one mode.

What lives here (the ownership, complete)

Piece Where Note
Mode, BuildThingType, FixtureKind socket.rs the taxonomy
type→socket registry socket_for(BuildThingType) -> Box<dyn Socket> what the pane calls
the composition table SocketPlan::for_type one match arm per bulb type
fixture registry fixture_for_kind(FixtureKind) -> Box<dyn Fixture> one arm per kind
the four seams Socket · Fixture · NetworkBackend · Probe traits
net backend TestNet (slirp/user-net) → NetSpec mode-aware
fixtures Dhcp · Dns · WireGuardPeer · TestNet · NoCloudSeed · Host* · Unimplemented*
lifecycle light → SocketDeployment → confirm → teardown
WG key management WgKeyPanel (own key, peers, add-a-peer, handshake) Test-mode surface
probes AppAliveProbe · NodesReadyProbe · WorkloadHealthyProbe over console text
pane surface SocketDeployment::json() · SocketPlan::topology() auto-fetch, never a dead pane

Not ours: the fixture runtimes (dhcp/dns/wg payloads) are tiny tunnr appliances and the bulb boots ride tunnr's QEMU/OVMF primitive; the rendering (console grid, topology graph, keys panel, probe results) is the dwarves-demo ⚒ pane.

The plan/deployment lockstep

Socket::fixtures maps the socket's own plan().fixtures through fixture_for_kind, and SocketDeployment::compose is the single constructor. So the topology the pane draws (plan.topology()) and the deployment it reads (light().fixtures) are the same list, in the same order, by construction — a socket cannot grow a fixture in its plan that has no deployment row behind it. (It could before: TunnrTestSocket planned 6 kinds and lit 3.) Guarded by every_socket_lights_exactly_the_fixtures_its_plan_composes, which sweeps every registered type in both modes.

Two honesty rules

1. Nothing is green until it is observed — and a failure is louder than a success. Fixture::light_health may return Ready / NotApplicable / Down(why) / NotImplemented(why) — never Serving. Only SocketDeployment::confirm(console) moves a fixture, by reading the boot console into an Evidence:

Evidence means effect
Served the console proves it worked Ready → Serving (unless Prod refuses it)
Failed(line) the console proves it did not Ready/Serving → Down, quoting the line
Unknown(why) the console does not settle it, and says why no change — unobserved is not healthy

The marker vocabularies come from the real tools

Not from the happy path someone remembered. Transcribed from:

They live in the transcripts test module — success, failure, partial and silent for each fixture, every one naming its provenance.

The governing rule: a success marker must never be able to appear inside a failure or attempt line. Three real cases where the naive check got that wrong — two of them found by reading the actual tool output rather than by reproducing a bug:

line what it is old verdict
udhcpc: broadcasting select for 10.9.0.157, server 10.9.0.1 a DHCP REQUEST, carrying a real pool address read as a lease
Sending handshake_initiation an attempt nobody answered read as a completed handshake
HANDSHAKE(REKEY_TIMEOUT) boringtun's retry warning read as a completed handshake

no_success_marker_can_hide_inside_a_failure_or_attempt_marker enforces the rule structurally, so the next loose marker fails at test time rather than in the field.

DHCP now requires two things together: a bound marker (lease of … / bound to … / DHCPACK) and an address inside the pool. Either alone is a fake green — an attempt line carries a pool address, and a static fallback carries a subnet address. saw_dhcp_lease was tightened to mean what its name always claimed; it used to mean "any subnet address that is not the gateway".

WireGuard now requires a completed handshake (Received handshake_response / Sending handshake_response / latest handshake:), never an initiation.

Failure markers are matched first and they win, anywhere on the console, and they win even from Serving — a tunnel that dropped is not still serving. The first two fake-greens this caught, both regression-tested:

NotImplemented is fail-closed too: FixtureHealth::ok() is false, so a socket composing an unbuilt fixture never reports itself healthy, and refusals() gives the pane the reason to render instead of a dead button.

A resolver on a net that declares no nameserver is Down at light-up: it could not resolve anything, so it does not get to be Ready.

DNS inherits DHCP's evidence — one direction only

The DNS fixture may lean on the DHCP fixture's evidence only when it is literally the same slirp stack (FixtureRuntime::SlirpBuiltin), and then it may never be more confident than what it inherits. The relation is deliberately asymmetric:

DHCP evidence DNS (same slirp stack) why
Served Served the stack demonstrably answered
Failed Unknown(why) DHCP failing does not prove the resolver is broken — inheriting a failure as a failure would blame DNS for DHCP's problem
Unknown Unknown(why) confidence cannot be manufactured

A DNS fixture running as its own payload inherits nothing and says so: "this resolver is its own payload, so it needs its own evidence".

The WireGuard peer lifecycle

The peer rows in WgKeyPanel follow the tunnel rather than only ever going green — the fixture going Down while its peers stayed Established was the same fake green one level in.

observation peer transition
completed handshake → Established { last_handshake_secs }
tunnel proven failed/expired Established/Handshaking → Lost; NeverConnected stays put
only an initiation seen NeverConnected → Handshaking (never Established)
invalid / duplicate key rejected at add_peer, never enters the panel

Lost is deliberately distinct from NeverConnected: "it broke" and "it never worked" are different things to show an operator.

Staleness is real and is not invented. Handshake::STALE_AFTER_SECS = 180 is WireGuard's own REJECT_AFTER_TIME (boringtun src/noise/timers.rs:20) — the point at which the protocol refuses the session's keys. is_up() is false past it and the pane label becomes stale.

The honest limit: the age signal only exists in a wg show capture (latest handshake: 6 minutes, 12 seconds ago), which parse_handshake_age_secs reads. A plain boot console has no age — boringtun's Received handshake_response says a handshake happened, not how long ago. So confirm uses the parsed age when a wg show capture is present and otherwise records the observation as fresh (age 0). It does not invent a timeout where no age signal exists; a socket that never captures wg show simply cannot detect staleness, and that is stated rather than papered over.

2. Mode::Prod never silently serves a test fixture. FixtureRuntime::is_test_grade() marks the fakes. Today that is slirp/QEMU user-net: an in-process, unroutable NAT with a toy DHCP server and a DNS forwarder bolted on — a superb test fixture and not a network anyone deploys onto. A tunnr Payload (real boringtun) and the Host boot path are the real things.

In Mode::Prod, any fixture that would come up Ready/Serving on a test-grade runtime becomes Down("refused in Mode::Prod: …"). This is enforced twice — at compose (light-up) and again in confirm (the evidence path), because a genuine slirp lease is still test evidence and must not turn a Prod fixture green. NotApplicable and NotImplemented are left alone: they serve nothing, so they fake nothing. The default is fail-closed — NetworkBackend::is_test_grade() returns true unless a backend explicitly claims production fitness, so forgetting to think about Prod cannot produce a Prod-labelled test net.

What that means for a Prod tunnr socket today

Fixture Mode::Test Mode::Prod
test-net (slirp) Ready Down — refused, test-grade
dhcp Ready → Serving NotApplicable (static/seeded, serves nothing)
dns (slirp forwarder) Ready → Serving Down — refused, test-grade
wireguard-peer (wg-appliance) Ready → Serving Ready → Serving (real boringtun)
nocloud-seed Ready (pre-filled, auth-less) NotImplemented — a Prod seed needs real credentials, which this layer mints none of
kvm-boot Ready Ready

So TunnrTestSocket.light(Mode::Prod).all_ok() is false, on purpose. Prod is not implemented end to end because no production-grade NetworkBackend exists yet (bridge-tap / router-VM / qemu-socket-mesh are all future impls) and nothing mints Prod credentials. Rather than shipping a green Prod button backed by slirp, the socket refuses and says why, per fixture.

Every refusal is actionable

A reason that only says "test-grade" leaves an operator stuck, so every Down / NotImplemented reason in the system carries a concrete next step after a TO FIX: marker (REMEDY_MARKER), which the pane can render as an instruction:

gap the remedy it names
slirp net / DNS / DHCP / NAT in Prod impl NetworkBackend with is_test_grade() -> false (bridge-tap, router-VM, qemu-socket-mesh) and compose it in place of TestNet
Prod NoCloud seed give NoCloudSeedFixture a credential source — pki and openbao are already in this crate
k8s control-plane boot a Talos/k3s node on the socket's net (machine::kvm already boots VMs) and hand talos_apid its endpoint — both ends exist, the launcher between them does not
k8s worker the same launcher, with MachineConfig::worker(..) and a join token
CNI once a control-plane fixture exists, apply the CNI manifests through K8sApplier like any other workload
OCI registry run one as a tunnr payload on the socket's net, or point the cluster at an external one via MachineConfig::with_mirror

every_refusal_and_every_gap_names_what_would_fix_it sweeps every registered socket in both modes and fails if any reason is a dead end, so a new refusal cannot ship without one.

k8s playground — what is real and what is not

socket_for(K8sWorkload) → K8sSocket, composing [TestNet, K8sControlPlane, K8sWorker×2, Cni, OciRegistry] with a 4-console grid.

Real: the composition, the topology (each node's endpoint), the probes, and Skidbladnir's existing k8s limbs — k8s::KubernetesRenderer (manifests as data), k8s_apply::K8sApplier (server-side apply / scale / wait against an existing cluster, feature kubernetes), unfold::talos::MachineConfig + talos_apid (machineconfig render + apid ApplyConfiguration / Bootstrap, feature talos). See kubernetes-target-design.md, k8s-unfold.md, talos-unfold.md.

Not built — and it says so: nothing boots a cluster node for this socket. The control-plane / worker / CNI / registry fixtures light as NotImplemented carrying the exact reason, so K8sSocket.light(..).all_ok() is false and refusals() names the four gaps. The missing piece is a provisioner that boots a Talos/k3s node in a VM on the socket's net and hands talos_apid the endpoint; the renderer and the push already exist on both sides of it.

The probes are real now and are the half that can be trusted before the provisioner lands:

Both are zero-shell: they read console text the Build Thing already captures. No kubectl is ever executed — K8sApplier is the API-side counterpart.

Consuming it

The pane never needs per-cell config: type in, socket out. Adding a socket = one arm in SocketPlan::for_type + one arm in socket_for + impl Socket. Adding a fixture = impl Fixture + one arm in fixture_for_kind. Zero pane changes either way.

Worked examples: socket-deployment-guide.md (the lifecycle, the health table, how to add each kind of thing) and socket-deployment-for-idiots.md (zero-knowledge walkthrough, including why a Prod socket is unhealthy on purpose).

Status

cargo test --features sockets: 244 lib tests green (61 in socket::). Default-feature build unchanged at 183. Every rule above has a test that goes RED when the rule is reverted — verified by reverting is_test_grade (2 red), the plan/deployment lockstep (5 red), the failure-first evidence rule (3 red), the three k8s honesty rules (3 red), the WireGuard completion requirement (3 red), the DHCP bound-marker requirement (5 red), handshake staleness (3 red) the peer-follows-the-tunnel rule (2 red) and the actionable-remedy rule (3 red).

Open, in order: (1) a production NetworkBackend + a Prod credential source, which together turn Mode::Prod from "refuses, with reasons" into a real deployment; (2) the k8s node provisioner; (3) Fixture::provision as a live process/VM handle rather than a config descriptor — today light() realizes fixtures as descriptors the consumer attaches (the netdev arg, the payload config), which is honest but is not yet a supervisor.