Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/edda

crates/storage-git-gix/parked-gix/.nornir/write-read-cycle.md

The write and read cycle through Arrow IPC and znippy

Generated: cargo xtask svg → .nornir/assets/gunnar-znippy-cycle.svg. Generator: xtask/src/cycle_svg.rs. Do not hand-edit the SVG — every box and arrow in it is a const carrying a file:line and a wiring verdict, and editing the picture instead of the datum is how a diagram starts lying.

Read 2026-08-04 against gunnar @ master (rebased onto origin/master before the citations were finalised) and znippy @ master.

A correction that only surfaced on the rebase. The first pass was read against a checkout 21 commits behind origin/master, and four of the cited files had moved under it. One of the moves changed an answer, not just a line number: SealedSecretStore gained a real call site upstream, so the "secrets sealing is not wired" verdict below is retracted — see §1. Every file:line in this document was re-read after the rebase. A verdict is only as good as the ref it was taken at.

✔ The open question this document raised is now CLOSED (2026-08-04), and two of its findings have been acted on. Nothing measured below is retracted; what changes is that "not wired" is no longer pending.

The cold-tier fold stays unwired, deliberately. Both blockers re-verified against the published crates with cargo tree -e normal, not read off a sibling checkout: znippy-common 0.9.12 → sysinfo 0.39.6 → rayon 1.12.0, which LAW 3 forbids as ROOT LAW #0 and gunnar has 0 rows of; and openzl-sys-rs 0.2.0 is an unconditional dependency — no optional, no feature gate — whose build.rs curls an OpenZL tarball, cmake-builds it and links stdc++. Neither can be selected away. See cold-tier-decision.md and §4 of TOTAL-PROMPT.md, which no longer reads ⚠ UNRESOLVED.

The reflog archive is no longer write-only. §5's observation — that refs-archive/ is written at every compaction and read by nothing but arrow_log's own tests — turned out to have a deeper cause: the whole Arrow ref log was unwired for refs on the fs backend, which serves references through gix-ref with WriteReflog::Disable. refs.arrow was 448 bytes, schema only, in every repository. gunnar had no reflog at all. FsRefStore::transaction now journals every committed transaction, gunnar_store::history reads archive + live taking no lock, and gunnar reflog is the verb. Sealing costs 99.7 B/push against the live log's 666.2 — 6.68× — by carrying the push boundary as a push_seq column instead of a frame, which is §5's own suggestion made real.

1. The headline, corrected

Three claims this session was handed turned out to be wrong in the same direction — a mechanism described as wired that has no call site. Each was checked by grepping for call sites, not by reading the doc comment above the definition. LAW 7: the correction is downward in all three cases.

claim as handed over what the call-site grep says
"the cold-tier fold is wired: repack.rs step 4 folds after the new pack is durable and before old packs are unlinked" half true. The ordering is real and correct. The fold itself is never enabled: RepackOptions::cold_tier() has 0 call sites in any server crate — the 5 that exist are in crates/gunnar-coldtier/'s own tests (4) and bench (1), and that crate is excluded from the workspace. gc builds RepackOptions::new().
"secrets sealing is now wired (D26, SealedSecretStore over the SecretStore trait)" correct — the claim stands, my contradiction of it does not. Stores::secrets() at crates/gunnar-cli/src/server/stores.rs:279 constructs SealedSecretStore over the raw ArrowLogSecretStore, the KEK comes from Kek::for_control_dir at stores.rs:223, and crates/gunnar-cli/src/server/serve.rs:387 takes that handle. I first recorded "0 non-test call sites" against a tree 21 commits stale; retracted. What is still open is the last hop: sealing happens at the store, and __gunnar_secrets__ — sealing into the archive — is a different thing FoldRequest carries no input for.
"__gunnar_graph__, __gunnar_reach__ and __gunnar_oid__ are built but unread on the serving path" stronger than that: they are never built. GitIndexBuilder::pack_tier — the only constructor gunnar's fold uses — returns Ok(Vec::new()) from build_sections. gunnar's cold tier emits exactly one derived section, __gunnar_refs__.

And a fourth, found here rather than handed over:

__gunnar_refs__ as gunnar seals it holds one frame, not one per push. The "one RecordBatch per push" property is real in znippy and exercised by znippy's own tests. gunnar does not feed it push-by-push: ZnippyColdTier::seal creates a fresh refs.log in a staging directory and pushes the whole ref snapshot as a single batch. So a sealed archive carries the current namespace, and the push history is not in it.

2. Where every claim was verified

Absolute paths are omitted; everything is repo-relative to /home/rickard/git/gunnar except znippy-plugin-git/..., which is /home/rickard/git/znippy/znippy-plugin-git/....

2.1 The five reserved sections

section schema / layout built at produced by gunnar read by gunnar verdict
__gunnar_refs__ name, target(null=delete), peeled, symref_target, push_seq, updated_ms znippy-plugin-git/src/refs.rs:93 crates/gunnar-coldtier/src/lib.rs:186 (RefLog::new + push), sealed at :197 crates/gunnar-coldtier/src/unfold.rs:110; also the fold's own applied-output check at crates/gunnar-coldtier/src/lib.rs:257 WIRED — the only one. But both ends live in gunnar-coldtier.
__gunnar_secrets__ name, ciphertext(Binary), recipient, push_seq, updated_ms znippy-plugin-git/src/secrets.rs:89 — — NEVER EMITTED. FoldRequest carries no secrets: crates/gunnar-store/src/repack.rs:134-142, reason restated at crates/gunnar-coldtier/src/lib.rs:38-43.
__gunnar_oid__ raw, not Arrow: b"ZNPYGOID", v2, hash code, oid_len, count, then keys[i64] / rows[u64] / ords[u32] / oids[count*oid_len]. Keys are (u64::from_be_bytes(oid[..8]) ^ 1<<63) as i64 — order-preserving. The 8-byte prefix is a filter, not a key; the full oid is compared after. znippy-plugin-git/src/oid_index.rs:79 (layout doc :25-37) — — NEVER EMITTED on gunnar's path: znippy-plugin-git/src/sections.rs:249.
__gunnar_graph__ oid, parents List<Utf8>, tree, committer_time Int64, generation UInt32 (1 + max(parents present)) znippy-plugin-git/src/graph.rs:47 — — NEVER EMITTED. gunnar's ancestry is an object walk: crates/gunnar-policy/src/ancestry.rs:20 says so.
__gunnar_reach__ commit_oid, cardinality UInt64, bitmap Binary — roaring, over object ordinals; ReachPolicy::max_commits defaults to 512 (tips first, then a stride through generation order) znippy-plugin-git/src/reach.rs:65 — — NEVER EMITTED. And nothing downstream could read it: see §3.

The single mechanism under refs and secrets is znippy-plugin-git/src/pushlog.rs: frame = [schema | RecordBatch | EOS], self-contained, sync_all per append (pushlog.rs:266-281), recovery by scan_frames (:126) which reads forward while frames parse and discards a torn tail whole. Both tiers compose through one point, GitIndexBuilder::with_section → finish_sections (sections.rs:300-309).

2.2 Per git operation

push (receive-pack). crates/gunnar-cli/src/server/transport.rs:501 → crates/gunnar-wire/src/receive_pack.rs:363 → pack + .idx + .keep written by crates/gunnar-pack/src/read.rs:138 (gix_pack::Bundle::write_to_directory at :158) → connectivity + policy gate at receive_pack.rs:570-575 → CAS expectation built at receive_pack.rs:656 (MustNotExist for a create, MustExistAndMatch(old) otherwise) → atomic batch at :600 → on-disk commit at crates/gunnar-store/src/fs.rs:694 (.prepare(...) at :688, expectation pre-checked at :668). The CAS pre-check is deliberately before gix (fs.rs:651), because gix short-circuits same-value edits.

fetch / clone (upload-pack). advertise crates/gunnar-wire/src/upload_pack.rs:407 → negotiate crates/gunnar-wire/src/negotiate.rs:198 → select_objects crates/gunnar-wire/src/graph.rs:785 (a closure walk) → objects.packed() seam crates/gunnar-store/src/object.rs:77 → write_pack_copy crates/gunnar-pack/src/copy.rs:237 (Mode::PackCopyAndBaseObjects at :282, thread_limit: Some(1) at :281 per P-020) → send_pack upload_pack.rs:1170. P-001 is fully wired — the copy branch is taken whenever the store has packs (upload_pack.rs:1256), with objects_copied_from_pack / objects_recompressed as the applied output (crates/gunnar-pack/src/write.rs:84,88).

gc / repack. crates/gunnar-store/src/repack.rs:374. Ordered: snapshot :400 → write pack :494 → index + verify :505,:540 → sync_dir :543 (new pack durable) → fold :564 → drop .keep :578 → remove midx :597 → unlink old :603. The fold sits strictly between "new pack durable" and "first unlink", and a fold failure is a hard Err, never a logged warning. (The module doc at repack.rs:64-79 numbers the fold "step 4" while the code labels it "3b" — a numbering drift in the comment, not in the code.)

secrets. ensure_host_key crates/gunnar-core/src/host_key.rs:103 → SecretStore trait crates/gunnar-store/src/secret.rs:25 → ArrowLogSecretStore crates/gunnar-store/src/arrow_log.rs:1972, wrapped by SealedSecretStore (crates/gunnar-core/src/seal.rs:357, impl at :397) via Stores::secrets() at crates/gunnar-cli/src/server/stores.rs:279. crates/gunnar-cli/src/server/serve.rs:387 uses that sealed handle.

3. What is NOT wired, in one list

# hole evidence why it is not a wiring bug
1 znippy is not in gunnar's resolved dependency graph at all Cargo.toml:55 excludes crates/gunnar-coldtier; 0 znippy entries in Cargo.lock (measured 2026-08-04) deliberate — znippy's codec cmake-builds C/C++ during cargo build and cargo reads a path dep's manifest at resolve time even with the feature off
2 the cold-tier fold is never enabled RepackOptions::cold_tier() — 5 call sites, all in the excluded crates/gunnar-coldtier/ (4 tests, 1 bench); gc at crates/gunnar-cli/src/server/admin.rs:383 builds RepackOptions::new() the control plane already reports it out loud: "no znippy cold tier is built", total_cold_bytes: 0 — crates/gunnar-cli/src/server/control_plane.rs:903
3 secrets are persisted in the clear RETRACTED — the store IS sealed crates/gunnar-cli/src/server/stores.rs:279; KEK at :223; used on the boot path at crates/gunnar-cli/src/server/serve.rs:387 what remains is that __gunnar_secrets__ (the archive section) is never filled, which is hole #4, not this one
4 __gunnar_oid__ / __gunnar_graph__ / __gunnar_reach__ are never produced znippy-plugin-git/src/sections.rs:249 returns Ok(Vec::new()) for a pack tier; crates/gunnar-coldtier/src/lib.rs:197 uses pack_tier by construction, and defensibly: the pack's own .idx is the oid index, and the other two derive from object payloads a pack tier never inflates. Emitting them empty would state, in sections a consumer trusts, that the archive holds no objects.
5 nothing downstream could consume a bitmap even if one existed measured 2026-08-04: 0 roaring rows in gunnar's Cargo.lock; 0 ewah and 0 bitmap hits across gix-pack 0.71.0/0.73.0 and gix-odb 0.81.0/0.83.0 this is the missing algorithm case. want − have is a closure walk (crates/gunnar-wire/src/graph.rs:785); reach(want) ANDNOT reach(have) has no implementation on the serving side at all. crates/gunnar-wire/src/upload_pack.rs:94 and crates/gunnar-wire/src/graph.rs:17,1151 say so, in comments.
6 the sealed __gunnar_refs__ holds one frame, not one per push crates/gunnar-coldtier/src/lib.rs:186-190 — one RefLog::push of the whole snapshot; FoldRequest.refs is documented as a snapshot at crates/gunnar-store/src/repack.rs:154 a design consequence, not a defect — but it means the push-log's transaction history does not survive into a gunnar archive, only its current state

The ceiling on #4 and #5, measured rather than argued

Moved here from the front page 2026-08-05.

__gunnar_oid__ is structural: its lookup_row addresses one archive entry per object, which a pack tier does not have, while .idx addresses pack offsets, which is what a reader needs.

For __gunnar_graph__ and __gunnar_reach__ the ceiling was measured rather than argued. GUNNAR_EXCLUSION=full against the default edges walk, 3 real repositories seeded 295–400 commits behind, is 0.91–1.10× against an edges-to-edges noise band of 7–17%. The traversal they would replace is not measurable, so neither can be shown to help. __gunnar_secrets__ has no consumer.

The red-first control on #5

A grep that returns nothing looks exactly like a grep that ran against the wrong path. So: delta hits 65 files in the same two gix-pack source trees the ewah and bitmap greps returned 0 from. The greps read the trees.

4. What I could not verify

5. Regenerating

cargo xtask svg          # writes both .nornir/assets/gunnar-znippy-cycle.svg
                         #   and .nornir/assets/gunnar-data-path.svg
cargo test -p xtask --lib cycle_svg
cargo test -p xtask --lib datapath_svg

Eight guards on this generator, each seen red before being trusted (LAW 2):

guard made to fail by
a_wiring_verdict_changes_the_markup Wiring::Unwired => None in dash()
no_verdict_is_printed_as_a_label putting WIRED back into any caption. Added 2026-08-05 with the verdict pills removed: the stroke carries the verdict and nothing writes it out as words
the_markup_uses_neither_banned_word putting "proven" into any caption
every_unwired_section_renders_dashed_and_every_wired_one_does_not same — it was blind to that flip until it was strengthened from a dash count to the specific pattern and stroke of each verdict, because NotEmitted kept the count above the threshold on its own
no_evidence_is_a_placeholder setting one Evidence line to 0 — and see S-050 in stinky.md: that is the only way it can fail, so it was blind to a citation that had drifted, which is the only way this datum ever actually goes wrong. 8 of 24 had. The by-construction fix is datapath_svg's every_citation_still_points_at_what_it_names, which opens the cited file and looks for a token that must be on that line; cycle_svg does not have it yet.
the_markup_is_well_formed_enough_to_render dropping the </svg>
no_label_overflows_the_box_it_sits_in it found a real defect on its first run — generation / UInt32 1+max(parents) needed 178 px in a 172 px slot — and then a second, four lane captions running off the right edge of the canvas. Every other guard was green through all of it: valid markup is not readable markup.
rendering_is_deterministic — (purity, so the committed asset can be diffed)