nordisk/dwarves

Release & history

How a release is run, what it captures, and how the warehouse lets you read any past release back. This is the Urðr chapter — what has been — wired to the present: a release isn't just "publish the crates", it's the moment nornir pins a complete, SHA-keyed snapshot of the workspace into the warehouse so the whole state is queryable and restorable forever after.

The release run

nornir release run walks the workspace in dependency order (the topological order recorded in the dep_graph_edges Iceberg table — dependencies first) and, per repo, runs a fixed sequence. It stops at the first failure; every repo processed up to and including the failure is recorded.

nornir release run                 # every configured repo, in dep order
nornir release run --repo holger   # just one member

Per-repo phases, in order — each gated by a --skip-* flag:

1. test — cargo test --workspace (per-test outcomes captured).
2. bench — runs the repo's examples/nornir-bench.rs; the BenchRun (results + per-test outcomes) is persisted to bench_runs. A failing bench-test aborts the release.
3. docs render — regenerates README.md from the freshly-persisted bench row.
4. gate-all — runs every gate enabled in nornir.toml (regression floor, guard-intact, crate metadata, path-dep versions, …). A red gate aborts.
5. capture — see below; the fat-client capture of every source-fact projection at this repo's release SHA.
6. push — git commit + git push of the regenerated docs to origin, so the published crates' git-SHA lineage matches the public remote.
7. publish — cargo publish walks the repo's publish_order; crates already on crates.io are detected and skipped (idempotent).

The capture phase — "nothing computed is discarded"

nornir is a fat client: wherever it computes a projection of the source, that projection must land in the warehouse — never computed-and-thrown-away. The release run's capture phase (step 5, under --skip-snapshot) enforces this. For each repo, at its release git SHA, it persists, best-effort and non-fatal (a capture hiccup never fails an otherwise-green release):

The index and DWARF snapshot ids are pinned into the release_lineage row (tantivy_snapshot_id / dwarf_snapshot_id), so a release row points directly at the exact searchable index and code-fact snapshots that shipped.

Idempotent per (repo, git_sha). Re-running a release at an unchanged SHA doesn't redo or re-store the expensive projections: the capture phase looks up whether the warehouse already has an index / DWARF snapshot for this repo's SHA and, if so, reuses it (and its lineage pin) instead of re-walking the index or re-parsing the binary's DWARF — which matters because iceberg-rust 0.9 has no expire_snapshots/compaction, so the cheapest way to bound warehouse growth is to not write redundant data. Vectors are already idempotent (only changed chunks re-embed). Pass --recapture to force a fresh capture. (Knowledge facts still append per run — they're small and carry no git_sha to key on; a capture side-index to dedup them is a tracked follow-up.)

The same complete capture is available outside a release as a single verb:

nornir map <repo>   # knowledge scan + index build + index snapshot + vectors, all keyed to HEAD

And every producer verb now persists by default (so the everyday dev loop also never discards): knowledge scan and introspect symbols save unless you pass --no-save; index build auto-snapshots unless you pass --no-snapshot.

Roadmap (tracked in plan.md). Minor follow-ups remain: the release run writes lineage rows only for green repos (a failed repo aborts before its row is written, so release trace over real runs currently sees greens), and published_versions is left empty (the publish step's outcomes don't carry resolved versions yet). The gimli_snapshot_id pin (col 17) and rustdoc-JSON api capture are future lower-level/code-fact work.

Reading history back

Because every fact is an Iceberg row keyed by git SHA, "the workspace as of release X" is a warehouse read, not a git walk or a rebuild.

• Restore the search index at any SHA — rehydrate the exact index that shipped:

nornir index restore --repo holger            # latest snapshot
nornir index restore --repo holger --sha <sha> # pinned to a release

• Semantic / lexical search at a past SHA — vector_* and index_snapshots are columnar reads, so searching an old release never re-embeds or re-indexes (see vector.md, warehouse.md).

• Release lineage — the release_lineage table records one row per repo per release: release id, workspace, timestamp, the pinned dep_graph_snapshot_id, build-order index, git SHA / branch / dirty flag, gate status, test pass/fail counts, dry-run flag, and the published crate@version set.

• Bench history — nornir bench history-show <repo> reads the per-SHA bench_runs rows; docs history via nornir docs history.

Regression trace — last-good → first-bad

Because gate outcomes are historized per SHA, finding when something broke is a scan over the warehouse, with no rebuild:

nornir release trace --repo holger        # human-readable
nornir release trace --repo holger --json

It finds the last green release and the first red one for the repo (the gate's last-good → first-bad boundary) and bounds the suspect commit range — the crossover the three Norns sit on: what was good, what is broken, and the commits in between to inspect.

Flags (release run)

All capture/publish steps are on by default; each --skip-* turns one off.

Live progress is written as append-only ndjson under <workspace>/.nornir/logs/ — consumed by nornir-server's release-progress SSE stream and the urdr-threads visualizer's live pane.