mikronova/workspace_znippy

znippy — full release runbook

How to cut a complete znippy release from this control plane: nornir runs every bencher, gates on regression, renders docs, commits + tags, and publishes to crates.io; you do the one git push it deliberately leaves to the operator. The declarative gate contract is in SPEC.md; the manifest (publish order, gates, bench requirements) is in nornir.toml.

(This workspace-wide release lives here in workspace_znippy, the control plane — not in znippy/.nornir, which is the znippy repo's own docs.)

TL;DR

Two commands, from the workspace root (where the sibling repos live):

cd /run/media/rickard/T9/git
NORNIR_CONFIG=workspace_znippy/release/nornir.toml nornir release run
git -C znippy push origin master --follow-tags   # the step nornir leaves to you

There is no wrapper script — this is an all-Rust toolchain; the single non-Rust action, git push, is a deliberate manual step (see The one manual step below).

What nornir release run does (per repo, in order)

1. cargo test --workspace (serialized) — abort on failure.
2. Bench — runs xtask/examples/nornir-bench.rs (every registered Bencher: the codec/stream/archive/uring/maven benches and the stree size-sweep), parses the BenchRun JSON, persists it to the Iceberg bench_runs warehouse.
3. Render docs — rewrites the <!-- nornir:gen --> regions of README.md from the fresh bench row (don't hand-edit inside those markers).
4. Gates — regression ≤ 10 % vs history, docs-fresh, no path-patches, metadata (see SPEC.md's 5 gates).
5. Capture (skip with --skip-snapshot) — knowledge map / search index / semantic vectors / DWARF facts → Iceberg.
6. Commit + tag — pure-Rust gix commit of the rendered docs + a vX.Y.Z tag, locally.
7. cargo publish — walks publish_order (linflate → lgz/ljar/lzip/lbzip2 → znippy-zoomies → osm-katana → znippy-common → compress/decompress → plugins → iceberg → znippy → znippy-cli). "Already uploaded" is an idempotent skip.
8. Lineage — appends a release_lineage row to Iceberg.

The one manual step nornir leaves: git push

By design, nornir does not push to the remote: gix 0.84 has no send-pack, and nornir forbids git subprocesses + libgit2 (see release/publish.rs::push). It creates the commit + tag locally and prints the exact git push command — you run it (the TL;DR's second line). That keeps the toolchain pure-Rust; the crates.io publish is automated, the remote push is the one deliberate human step.

Prerequisites

• NORNIR_CONFIG must point at this config — nornir's auto-discovery is hardwired to workspace_holger/release/nornir.toml. Use the env var or nornir --config workspace_znippy/release/nornir.toml ….
• crates.io token — cargo's own auth (CARGO_REGISTRY_TOKEN or ~/.cargo/credentials); nornir shells out to cargo publish.
• Iceberg warehouse — now pinned by [storage] local_path = "workspace_znippy/.nornir" in nornir.toml, so bench/release write workspace_znippy/.nornir/warehouse (previously fell back to holger's). The warehouse is local redb-backed and is created on first use.
• A current nornir binary — cargo install --path nornir --bin nornir --force if yours is stale. The bench_runs Arrow schema evolved (now 7 fields); an older installed binary writes a mismatched batch and the local append fails (columns(N) must match fields(M)). Reinstall to match source.
• NORNIR_CONFIG must point at this config — auto-discovery is hardwired to workspace_holger/release/nornir.toml. Use the env var or --config ….
• crates.io token — cargo's own auth (CARGO_REGISTRY_TOKEN or ~/.cargo/credentials); nornir shells out to cargo publish.
• git author configured (user.name / user.email) — the tag step bails without it.
• Quiescent machine — the regression gate compares same-machine numbers; close other heavy jobs (and don't run two benches at once — concurrent benching corrupts numbers).
• Push auth — codeberg over SSH for the push step.
• NORNIR_SERVER — if set (e.g. http://127.0.0.1:7878), the standalone nornir bench run ships results to that server (which must serve the znippy workspace) instead of the local warehouse. release run always uses the local warehouse, so it is unaffected — but unset NORNIR_SERVER for ad-hoc local bench runs, or register the workspace on the server (nornir workspace register znippy --descriptor workspace_znippy/nornir-workspace.toml).
• Heavy/planet bench fixtures — a real release run runs the full bench suite (no skips): the 500 MB/2 GB streams, planet .osm.bz2, and europe-latest.osm.pbf must be present, or those benchers skip and the gate's required_results may fail. For a quick path check use NORNIR_BENCH_SKIP_HEAVY=1 NORNIR_BENCH_SKIP_PLANET=1 nornir bench run znippy.

Status — what was blocking, now cleared (2026-06-11)

• ✅ nornir compiles (the earlier viz/model.rs gix error is gone).
• ✅ znippy-common 0.9.2 published — it had diverged from 0.9.1 without a bump (the MetaSinkFactory iceberg-sink API), which would have failed the znippy-compress publish step mid-release. Bumped + published + every dependent's version requirement raised to 0.9.2.
• ✅ The decompressor crates were already perf-released this round (lbzip2 0.5.2, lgz 0.3.3, ljar 0.2.1, lzip-parallel 0.2.2, znippy-compress 0.9.2) — a full release run will idempotently skip those already-uploaded versions and pick up whatever else changed.
• ✅ Bench → persist path validated (NORNIR_BENCH_SKIP_HEAVY/PLANET=1): nornir bench run znippy persisted 21 results / 29 tests into a fresh workspace_znippy/.nornir/warehouse bench_runs (incl. stree_8mb/_128mb, lbzip2_scala, lgz_scala). Fixes that got it there: reinstall the nornir binary (schema skew), the [storage] pin above, and a one-time warehouse wipe (the old table predated the 7-field schema).

So a full release run is runnable today. Dry-run tip: run nornir bench run first (bench + persist, no gates/publish) to confirm the bench + warehouse path on a quiescent box before the real release.