nordisk/dwarves

The release function — reasoning

Class: reasoning. This file holds the why, the trade-offs, the publish traps as lessons, and every design we rejected. The descriptive shape nornir actually runs is in release-design.md (which by rule mentions none of this); the CLI workflow is release-guide.md. If you want to know "why not X?" — X is here. Siblings: release-design.md · release-guide.md · doctor → release-doctor-design.md / release-doctor-guide.md / release-doctor-for-idiots.md · release-for-idiots.md · release-history.md.

Why gates-as-code (free functions returning Result<()>)

A release gate is a predicate the release must satisfy, not a shell script. Each is a pure-ish free function returning Ok(())/Err so it is: unit-testable in isolation; identical whether an embedded warehouse read or an Ops.RunTestMatrix call fed it the rows; and composable — the sweep is just "run each, collect the errors." The whole point is that the gate is the specification, versioned next to the code it guards, not a wiki page CI happens to run.

Why empty-is-not-green. A repo with zero configured gates once passed gate all trivially — it ran nothing, so nothing failed. That is a silent hole: "we gate this repo" and "we gate nothing on this repo" looked identical. gate_all_outcome now requires gate_count > 0 (RAGNARÖK bug #14). A pass must mean something ran.

Why one success predicate drives everything. gate_succeeded is the single source of truth for both the per-repo loop-abort and the final rollup. The historical BUG#1 was exactly a divergence: only failed_test aborted the walk and flipped the final verdict, so a failed_bench shipped silently green. Two code paths asking "did this pass?" independently will drift; collapsing them to one predicate makes that class of bug unrepresentable.

Why a red bench run is never persisted as a baseline. The regression gate compares against history; if a red/regressed/errored run were appended to that history, it would become the baseline every future run is measured against — poisoning the very signal the gate exists to protect. So the append happens only after every bench gate passes, and the regression check runs before the append so a candidate can't be compared against itself (which would make every drop 0%).

Why rehearse-before-promote (the /sparring fake crates.io)

crates.io is irreversible — you cannot un-publish, only yank, and the version number is burned forever. A cascade that resolves on paper can still fail at the registry: a stripped [patch] leaves a version= that doesn't actually resolve, a dependent build only surfaces the break once the dependency is a real published crate rather than a path dep. So we prove the whole cascade against a writable, throwaway crates.io first: publish into /sparring in topo order, then build each dependent from /sparring — the empirical transition oracle for the one thing static analysis can't answer, does it actually publish and resolve?

Why embedded, in-process holger. Rehearsal needs a real sparse-index registry that accepts writes. Rather than orchestrate a separate holger-server process (config, ports, teardown, a second thing to fail), nornir links holger directly (embed-holger, on by default) and brings /sparring up inside its own process — one binary, no orchestration, deterministic teardown when the handle drops. A lean client build opts out and spawns a real subprocess; the same call sites handle both, so there is no divergent code path to rot.

Why the rehearsal uses uniquely-named fixture crates. /sparring mirrors crates.io as its upstream, so a real-world crate name (acme-lib) would collide on publish and resolve the real upstream instead of the just-staged one — the test would pass for the wrong reason. Fixtures are named nornir-stage-fixture-* precisely so a collision is impossible.

Why crate-level (not repo-level) topology

cargo publishes crates, not repos. A repo-level view reports facett ⇄ knut as a mutual dependency and refuses; but at crate granularity the same edges form a clean DAG (different crates in each repo point different ways). Ordering at the crate level is what turns a false "cycle, can't release" into a real, executable publish order. This is why build_order and the doctor's crate_publish_order operate over crates and SCC-condense before topo-sorting.

Why the graph-math core instead of if/else

The doctor's remedy engine is deliberately not a decision tree. It is search/planning over an outcome space Ω: a state is the vector of findings (dirty set, version-gap set, skew vector, promote-block set, cycles/SCCs); each remedy is a declarative STRIPS-style operator ⟨precondition, effect⟩ matched generically against the state. Adding a remedy is adding a row to a table, never a new conditional. The reason is maintainability under growth: an if/else doctor grows a new branch per symptom and becomes unreadable; a planner over a structured space generalizes to new symptoms by adding operators/data, with zero new arms, and the fixpoint's convergence is asserted (each AUTO operator strictly shrinks the open-symptom multiset). Full framing: release-doctor-design.md → graph-theory framing; engine: release-graph-math.md.

Why --rehearse is the oracle, not a predictor. Some effects (does a strip-then-publish actually resolve?) can't be computed statically. The planner searches the structured space but treats rehearsal as the empirical transition function for the residual: apply → observe → continue. So the engine stays declarative for what it can prove and honest about what it can't.

Why semantic blast (the pre-link check)

release = build, and a version bump is the link step. The crate-level blast answers "who depends on the changed crate" — a coarse over-approximation ("facett → 7 repos might break"). But most of those dependents never call the symbols that changed. Reading the warehouse SCIP call-graph resolves each changed/removed symbol to its exact call sites — "these 12 call sites in 3 crates call the 2 symbols that changed" — and marks symbols nobody calls as unused_changes (a provably safe delta). The reason is to distinguish might break from actually breaks, so coordination effort goes only where a real link error would occur. Detail: release-semantic-blast.md.

Why refuse a dirty tree (and why it's overridable)

nornir remembers a release as its per-repo git SHAs — that is the identity of shipped code and the whole rollback story. A dirty tree means the SHA does not fully describe what would ship, so the release could never be rolled back to exactly what was released. The pre-flight therefore refuses on any uncommitted change (refuse_if_dirty, default on). It's overridable (--allow-dirty in staging tolerates it) because rehearsal into /sparring is throwaway and a dirty tree there costs nothing; only promote to the real registry demands a committed, tagged state.

Why pure-gix for tag/commit but a cargo subprocess for publish

Git inspection and tag/commit are done in-process via gix — no git shellout — because they are cheap, well-defined tree operations we control. cargo publish stays a subprocess because cargo owns crate packaging (sources → .crate tarball → upload, feature resolution, the verify build); reimplementing that in a few lines of HTTP would be a bug farm and would drift from cargo's behavior. So the rule is: own what is ours (git plumbing, warehouse writes), delegate what is cargo's. One consequence we accept: gix 0.84 has no send-pack, so the network push is left to the operator/CI and we print the exact command — deferred deliberately rather than shelling out to git push.

The publish traps (lessons — why each guard exists)

Hard-won during the nornir 0.4.54 publish + the facett warehouse-landing fix (2026-06-21). Each looked like a different bug. Full write-up: release-publish-traps.md.

Lesson 1 — private = true (git-private) ≠ "don't publish to crates.io"

The release config's private flag conflated two orthogonal facts: is the git repo SSH-private? and is there a crates.io crate? nornir's git is SSH-private but the crate is public (cargo install nornir works). With private = true, release publish nornir hit the private-repo gate and hard-refused. Lesson: the flag must mean "has no crates.io crate" — holger/korp/ordning are genuinely crate-less and stay private; nornir is private = false. Distinguish by is there a crate?, never is the git public?

Lesson 2 — the fork-promote gate over-blocks via publish = false bench crates

patch_fork_blockers walks every Cargo.toml (depth 4) for [patch.crates-io] path/git forks and does not skip publish = false crates. So bench manifests (skade/bench, facett/bench) that legitimately patch the arrow-58 iceberg fork got scanned, and the coarse repo-level transitive closure folded that hold all the way up to the nornir crate itself — which touches no strip-on-publish fork. Lesson: a coarse "any crate in this repo rides a fork ⇒ hold the whole repo" closure over-approximates; the real fix (deferred) is to skip publish = false crates in the blocker scan. Until then the escape hatch is cargo publish -p nornir by hand — always with the self-protecting default --verify, never --no-verify (that shipped the broken 0.4.45 that got yanked). This is why promote keeps a bump-by-hand path and why the arrow-58 fork wiring (below) must never be "cleaned up."

Lesson 2b — the arrow-58 forks are load-bearing and must follow to crates.io FIRST

The iceberg = { package = "iceberg-arrow58", version = "0.9.1", path = "vendor/…" } entries and the bench [patch.crates-io] iceberg-datafusion patches are deliberate, required wiring for the arrow-58 stack — removing any breaks the build against arrow-58. The path= is dev-only; what makes the published manifest work is the version= resolving from crates.io, which only holds because the vendored forks are themselves published. Lesson: whenever a fork's arrow-58 content changes it must be bumped and re-published to crates.io before any consumer that pins it can publish — strip the path on publish and the version= must point at a live version or the consumer's verify-build fails.

Lesson 3 — a committed dev-path [patch.crates-io] breaks the server build

A committed patch pointing an already-published crate (nornir-testmatrix) at an absolute dev-drive path (/run/media/rickard/T9/git/…) works on the author's box and dies everywhere else: the nornir-server builds each repo from its own clone as an isolated nornir user that cannot and must not reach the dev checkout, so dependency resolution fails with an opaque failed to load source. Lesson — the one principle behind Traps 2 & 3: the published/served artifact must resolve every dependency from crates.io. A path= is a dev convenience that is stripped on publish or unreachable in the server sandbox; carry the real version= (and package= for a renamed fork) so stripping the path leaves a working manifest, and never commit a patch that only works on one developer's drive. This is why the no_path_patches / path_dep_versions gates exist — to catch it statically before it bites.

Rejected alternatives

Rejected: a hand-driven shell loop for the crates.io cascade. The whole-constellation release used to be a shell loop of cd repo && cargo publish && sleep 20. Rejected — never again — because it had no order guarantee, no fail-stop, no record for undo, and the sleep 20 was a guess at index-propagation lag that was sometimes too short (resolve failure) and usually too long. It became doctor --publish: report.topo.order verbatim, fail-stop on the first per-repo error, wait_for_index polling instead of sleep, and every mutating step recorded to release_changes so release undo can reverse it.

Rejected: a fixed sleep between publish phases. Replaced by wait_for_index, which polls crates.io until <crate>@<version> is the advertised max version. A sleep either races propagation (dependent's publish fails to resolve the just-published dependency) or wastes minutes; polling is correct and as fast as the registry allows.

Rejected: fail-stop that treats "already uploaded" as an error. crates.io returning "already uploaded" is the normal state of a re-run after a fail-stop, not a failure. Treating it as an error would make every resume abort on the first already-shipped crate. It is a skip instead — which is exactly what makes --bump keep an idempotent finish/resume path with no gratuitous re-bump churn on the irreversible registry.

Rejected: only checking docs drift at release. The docs_fresh gate used to merely check and fail if a managed artifact had drifted from its .nornir/ source — which meant a release could be blocked by staleness that a single render would have fixed. So the release now renders + stages first (render_and_stage_docs): render the managed docs with full SVG/depgraph assets, build the book PDF, git add every changed artifact, then run the check as the final assertion. Any drift after a render is a real bug, not staleness.

Rejected: a single lump book.pdf for release docs. (Shared with the docs subsystem.) One undifferentiated PDF mixed the what (design/guide) with the why (this file) and the record (history). Split by audience — manual / idiot_guide / unsorted — so the operator's manual stays thin and the reasoning is never thrown away, just re-homed. That routing is why this very file is a reasoning class and stays out of the manual.