The cold tier: a decision
Question: should Gunnar's cold object tier be a znippy archive? Written: 2026-08-03 · machine oden · settles the ⚠ UNRESOLVED block in TOTAL-PROMPT.md §4 and the open half of P-005. Corrected 2026-08-04 — see the block immediately below. Both dependency blockers this document rests on are gone. The recommendation does not change, and the reason it does not change is now a measurement rather than a dependency count.
The state as it ships
Moved here from the front page 2026-08-05.
The tier is wired and off by default. GUNNAR_COLD_TIER=znippy appends the pack a push landed, gunnar gc seals it with __gunnar_refs__, and ZnippyObjects — a third ObjectStore beside mem and fs — serves upload-pack out of the archive with no packfile on disk, over gix_pack::Find on gix-pack 0.73's FileData generic. Asserted by deleting objects/pack and cloning: real git clone plus negotiated fetch, fsck --full --strict clean.
Measured oden 2026-08-04, real repos, interleaved, load 20–140:
against the fs tier |
|
| clone bytes | 1.000–1.002× — entry copy intact, the P-001 canary |
| clone wall | 1.06–1.41× warm · 1.71–2.42× cold |
| negotiated fetch, depth 295–400 | 1.17–1.26× (2.2–2.6× before header() stopped inflating a whole object to read a type byte) |
| archive against the pack it replaces | 1.013–1.077× — and the pack need not stay, so the object tier is ~1.03× |
What the cold tier costs: 42 crates, a sibling ../znippy checkout (znippy-plugin-git 0.1.0 is unpublished, and cargo reads a path dependency's manifest whether or not its feature is on), and no rayon.
⚠ 2026-08-05: there is no znippy feature any more. It was an optional dependency on gunnar-coldtier from gunnar-cli, and because cargo reads a path dependency's manifest at resolve time, that one line made the sibling checkout above a hard requirement of every gunnar build on every feature set — not of the tier. It also made gunnar-cli unpublishable, gunnar-coldtier being publish = false. The tier now lives in the detached crates/gunnar-znippy, which builds the same gunnar binary with it installed; the costs in this table are what that binary pays and the default one does not. openzl-sys-rs resolves at 0.2.0, which curls and cmakes a tarball at build time; 0.3.0 vendors, and znippy's bump is unmerged.
CORRECTION, 2026-08-04 · both stated blockers are dead
Read against znippy origin/master d83b927 and release/0.9.13-prep f871d5c, re-verified here rather than taken on report.
Blocker 1 — the C toolchain (P-006, What breaks §1) — dead twice over.
openzl-sys-rsis no longer unconditional.znippy-common/Cargo.toml:60readsopenzl-sys-rs = { version = "0.3.0", optional = true }behindopenzl = ["dep:openzl-sys-rs"](:35). A consumer takingznippy-commonwithdefault-features = falsenever resolves it. §1's sharpest sentence — "There is no feature to turn it off" — is retracted; there is one.openzl-sys-rs 0.3.0is published to crates.io (checkedindex.crates.io2026-08-04:0.1.0,0.2.0,0.3.0) and vendors the OpenZL, zstd and lz4 C sources, compiled withcc. Nocurl, nocmake, no network fetch. Every line-numbered claim in §1 is against0.2.0'sbuild.rsand no longer describes the current crate.
Blocker 2 — rayon (LAW 3, What breaks §2) — dead, with one precision.
sysinfois now a dev-dependency:znippy-common/Cargo.toml:154, inside a[dev-dependencies]section opening at:123. It is not in a consumer's normal graph, sosysinfo'smultithreadcannot carry rayon into Gunnar.cargo tree -p znippy-common -i rayon→ did not match any packages.- The precision, because the blunt version would be false (LAW 7): rayon is still in the znippy workspace lock, via
znippy-plugin-python,lzip-parallel,tantivy,candle-*,typst-*andwasmtime. What is true is narrower and is the only part that matters here: rayon is out ofznippy-common's tree, and that is the tree a Gunnar consumer resolves. - The guard this document called uncompilable now has its missing half present —
tests/tests/rayon_free_law.rsincludestests/tests/rayon_law/core.rs, and that file exists. I did not run it, so "compiles" is inferred from the file being there, not observed.
What has not changed. The document's own bar is "all four, not any one":
| change-condition | 2026-08-03 | 2026-08-04 |
| 1 · OpenZL-free build | no | yes |
| 2 · rayon edge gone | no | yes (guard not run) |
| 3 · tier stores packfiles, not objects | no | no — still a design, not code |
4 · znippy-plugin-git published |
no | no — index.crates.io 404, today |
Two of four. znippy-common is published to 0.9.12 and bumping to 0.9.13; znippy-plugin-git has never been published, so crates/gunnar-coldtier's path dependency on ../../../znippy is still the only way to consume the git format, and that is what §4 forbids.
So the recommendation stands — but the dependency arithmetic is no longer what holds it up. That argument has expired. What replaces it is stronger, and it was measured today rather than reasoned about.
The measurement that now carries the decision
S-003 says Gunnar computes no server-side deltas. The obvious fix is to encode somewhere and store the result, and a sixth reserved section (__gunnar_delta__) is the obvious place to store it. Measured, that section would store something a packfile already stores, and store it worse.
Method: for each repository, HEAD~50 and HEAD pushed by stock git 2.53.0 into a fresh bare repo — the state Gunnar is really in after a push, not a synthetic fixture. Then the fetch that arm serves (want HEAD, have HEAD~50) packed two ways: pack-objects --thin (searches for deltas) against --thin --window=0 (reuses stored deltas, computes none — Gunnar's behaviour after ed9bcb9). The ratio is what a server-side encoder is worth. Then gc the same repo and re-measure. oden, 2026-08-04, byte counts, so load-independent.
| corpus | encoder worth, freshly pushed | encoder worth, after gc |
pack bytes |
| nornir | 4.8999× | 1.0000× | 8 722 799 → 7 876 401 |
| korp | 4.1053× | 1.0000× | 8 235 699 → 6 959 409 |
| dwarves | 2.1175× | 1.0000× | 9 557 956 → 9 263 952 |
| gunnar | 2.0383× | 1.0000× | 129 474 082 → 116 917 007 |
| znippy | 2.0252× | 1.0000× | 19 925 789 → 19 755 091 |
| holger | 2.6943× | 1.0000× | 3 542 296 → 3 422 940 |
| garmr | 1.7406× | 1.0000× | 23 172 479 → 22 418 069 |
| ordning | 1.1442× | 1.0000× | 847 233 → 848 440 |
the bakeoff large fixture |
4.7952× | 1.0000× | 7 997 959 → 7 879 663 |
Nine of nine reach exactly 1.0000×. After a batch-tier repack with a delta search, computing deltas at serve time is worth nothing at all, because the deltas are already in the pack and pack-copy already reuses them.
That is the whole of S-003, and its answer needs no new container: the packfile is the delta store, gc is the encoder, and the serve path already copies. Cost on disk is negative in 8 of 9 — the pack shrinks; the exception is ordning, which grew 1 207 B (+0.14%). Compare the reachability bitmap's ~2.2 B/object cost. A __gunnar_delta__ section would carry a delta DAG this document already costed at 52.2 B/object uncompressed, 31.6 B zstd (§3.2 of delta-per-row.md) — to hold what a repacked pack holds for less than nothing.
LAW 2, and it fired on me. The first version of this measurement read the fixture's own already-gc'd pack and reported 1.0000×, which I nearly wrote down as "the encoder buys nothing". It was the identity value, and it was the fixture trap in the exact form TOTAL-PROMPT.md warns about. The before column is what proves the metric can report anything other than 1.0000.
Delta rows in znippy: the shape, and what it is for
Rickard's ask is historization in znippy itself — first row the original object, following rows deltas against it, the way __gunnar_refs__ already puts one RecordBatch per push and lets the batch boundary be the transaction boundary (D18). That is a znippy storage feature and it stands on its own. It is not the fix for S-003, and the two must not be conflated:
- For gunnar's fetch path, delta rows buy nothing a repacked pack lacks. Nine of nine corpora above reach exactly 1.0000× after
gc. Whatever a delta section carried, pack-copy would already be copying the same deltas out of the packfile, for negative bytes. - For a znippy archive holding many versions of the same content, delta rows are the whole point, and no packfile is involved.
The crux question — can delta rows ride alongside the pack tier? Yes. Verified by reading, 2026-08-04: GitIndexBuilder::with_section (znippy-plugin-git/src/sections.rs:143) is a method on the builder, and the builder is constructed by either new (:82, one entry per object) or pack_tier (:123, pack-<id>.pack/.idx as entries). It is not tied to either. That is why __gunnar_refs__ and __gunnar_secrets__ already work on both: they are the server's log, not object layout.
So a sixth section is additive to a pack-tier archive, and P-001 entry-copy is untouched — the pack entries stay in the extension skip list, stored raw and returned raw. This is the only shape that may be built. The per-object tier is the one where ObjectStore::packed() returns None, which is P-005, and taking that fallback by accident was measured at 18.4× the bytes. A delta section must never be the reason an archive moves to the per-object tier.
The one edit that makes it real, and it is deliberately a single edit: RESERVED_MODULES (znippy-common/src/index.rs:361-372) is a slice, not a || chain, precisely so that adding a name is seen by both is_reserved_module (:388) and its guard test at once. The failure it exists to prevent is a name the sink writes as reserved but the manifest readers classify as data, which silently corrupts list, decompress, the iceberg sink and the manifest-count assertions. A GUNNAR_DELTA_MODULE = "__gunnar_delta__" that is not in that slice is worse than absent.
The compression rule it must not break. .pack/.idx/.midx/.bitmap are skip-listed and stored raw; a canonical object entry is "<type> <size>\0<content>" — inflated bytes that compress well and must be compressed. precompressed.rs warns in as many words against a rule of the shape "an oid-keyed entry is already compressed". A delta payload is likewise compressible and must not inherit a raw-store rule from sitting next to a pack.
Not built here. The registration, the schema and a writer are one small coherent change to znippy off origin/master, and it is not in this commit because it was not testable to the bar in the time available. The bar it has to meet is the same one everything else here meets: a red first, then a green.
What this does not settle. The 1.2458× fetch loss against gitea/Forgejo is measured on a corpus seeded by push and not repacked, so the table above says a gc should move it — but I did not run the bakeoff, so that number is predicted here, not measured. gunnar's own repack() does not do this today: crates/gunnar-store/src/repack.rs:954 runs Mode::PackCopyAndBaseObjects, which preserves deltas and computes none. Closing S-003 means giving repack() a delta search — and gunnar-delta is an entry in xtask's FORBIDDEN_IN_SERVER_GRAPH, so it has to arrive behind a feature the appliance profile does not turn on, or the guard goes red.
Recommendation: (b)
Packfiles remain the cold tier. znippy is an analytics and archival sidecar, and the git package format's real value is the queryable graph, not the bytes.
Concretely:
repack()stays as built — one consolidated packfile per repository, written throughgix-pack'sPackCopyAndBaseObjectsgenerator. It is git's own cold tier, it is representation (b) by construction, and it costs zero new crates.ZnippyObjectsis not written. Wave 9b's storage half does not happen; §4'sTieredObjectStore { incoming: packs, cold: ZnippyObjects }becomes{ incoming: packs, cold: PackObjects }and theznippyrow in §4's backend table moves from backend to export tier, next toiceberg, perP-013.- The
gitznippy package format is kept and is not wasted. Its role is the export tier: one sealed, immutable, per-repository archive published off the serving path, queryable in SQL, with the commit graph and the oid index as Arrow. That is the part of it which is genuinely better than what git has. - The graph does not need znippy to reach Gunnar.
__gunnar_graph__is plain Arrow. Gunnar already linksarrow-array/-buffer/-data/-ipc/-schema/-selectfor the ref log, so it can write and read a commit-graph table with zero new dependencies. The.znippycontainer is one carrier for that table, not its only one, and it is the expensive one.
So the answer to §4's contradiction is not a choice between (a) and (b): the choice dissolves once the cold tier is the packfile. §4 rejected packs-in-znippy; P-005 requires the packed representation; taking the packed representation without the archive satisfies both, and it is what is already on disk.
Why the contradiction cannot be resolved the other way
The tension between §4 and P-005 is not symmetric. The format that actually exists picks the side P-005 rules out.
znippy-plugin-git/src/object.rs (znippy origin/master, eaafaa1), first line of the module doc:
An archive entry's payload is the canonical serialization of a git object:
"<type> <size>\0<content>".
That is inflated content. f99d076's commit message says so directly and calls it a correction to its own premise:
a
znippy-plugin-gitarchive entry is NOT a zlib stream. It is the object's canonical form … inflated bytes that compress well and must keep being compressed.
P-005's whole argument is that an inflated cold tier forecloses pack-copy permanently for every object that ages out of the incoming tier, which over time is nearly every object. P-001 is the largest measured win in the project, and the head-to-head puts a number on what losing it costs: 108.70 MB on the wire against stock git's 5.93 MB, 18.336× worse (oden, 2026-08-03, gunnar at e60c95d, git 2.53.0). So:
- The
gitformat as built is representation (a) and cannot be the cold tier without giving up the single biggest win available. - Changing it to store pack entries — type/size varint, delta base reference, zlib stream — makes it a packfile with an archive around it, which is what §4 rejects.
The only coherent version of "(a)" is the third one, and it is worth stating because it is the thing to build if znippy ever earns its way in: store whole packfiles as archive entries, a handful of large entries per repository, with the three reserved sections indexing the objects inside them. That keeps pack-copy, amortises the per-entry overhead away, and keeps the Arrow index. It is not on the table today for the reasons in What breaks below, and it is not what any code currently does.
What is measured
All figures below are from oden, 2026-08-03. Source is named per row. Nothing here is re-derived; ratios are given rather than counts (LAW 7).
znippy's git format, from /home/rickard/scratch/cargo/git-bench-full.json (run 08:42:30Z, 32 cores)
| arm | what it says |
git_archive_60k |
60 000 objects, 12.24 MiB in → 42.1 MiB out = 3.44×; seal 1.42 s = 42 362 objects/s, 8.6 MB/s; reopen_lookup_ns 652.4; telem_cores_busy_max 10 of 32 |
git_oid_lookup_100k / 1m / 8m |
__gunnar_oid__ = 4.96 / 49.59 / 396.73 MB → 52.0 B per object, flat at all three sizes; stree batch vs binary search 1.51× / 1.78× / 2.62×; vs fst 3.92× / 4.08× / 3.87×; fst is also 1.8–2.9× slower to build and 1.17× larger |
git_sections_300k |
300 000 objects, all three sections in 2.09 s = 143 558 objects/s; oid section 14.88 MB = 52.0 B/object again |
git_reach_10k |
501 bitmaps over 30 000 objects = 3.83 MB = 7.6 KB per bitmap; ANDNOT 271 ns against a 76.5 ms walk |
git_graph_linear_200k / _merges_200k |
generation assignment 62.3 / 71.8 ms for 200 k commits, i.e. 311.6 / 359.1 ns per commit, merge-heavy costs 15% more |
The 3.44×, and the correction that has to go with it
PLAN.md records this as "a znippy archive of 60 000 small git objects came out 3.4× LARGER than the input". That is exactly what the arm measured, but the ratio is an artefact of the corpus and must not be quoted as a property of znippy.
archive_build's objects average 12.24 MiB / 60 000 = 214 bytes — synthetic blobs of the form content of object N\n, plus small trees and commits. Real git loose objects in znippy's own fixture (f99d076: 5388 objects, 35.8 MB) average 6.6 KB.
The transferable fact is the fixed per-entry cost of 522 B, and the fixed-cost model reproduces the measurement exactly: (214 + 522) / 214 = 3.44×, against a measured 42.1 / 12.24 = 3.44×. Applied to a real corpus the same 522 B is +7.9%, not +244%.
The corrected argument is stronger than the one it replaces, not weaker. A fixed per-entry cost divided by mean entry size is precisely why packs amortise znippy's overhead and objects cannot: a repository's pack tier is a handful of entries of tens of megabytes each, where 522 B is unmeasurable. Storing objects as entries puts the overhead on the one axis git has spent twenty years driving to zero.
For comparison in the other direction: a packfile's per-object cost is a few bytes of varint header, and its deltas make the total smaller. On the head-to-head fixture, git turns 528 MiB of content into 5.93 MB. No per-entry-indexed container competes with that.
Decomposition of the 522 B — partly measured, partly inferred. Measured: 52 B/object is __gunnar_oid__, confirmed at three independent sizes. Inferred: ~68 B/object for __gunnar_reach__ (512 bitmaps × ~8 KB ÷ 60 000, extrapolated from git_reach_10k's 7.6 KB/bitmap); __gunnar_graph__ unknown, 20 000 commits, never reported separately; residual ≈ 330–400 B/object attributed to znippy's own base index + lookup sub-index + fst trie. That residual is the number that decides whether the format can be tuned or the container is simply expensive, and I could not settle it by reading; see M1.
Gunnar's own, same machine, same day
gunnar.repack_1m |
87 054 objects/s, seal_ms 0.002, 1 M objects, 64 packs in, 105 621 680 B out |
repack() pack-copy, sha256 fixture |
11 of 12 objects copied from pack, 1 recompressed (the loose one, no pack residency) |
gunnar.vs_git_clone at e60c95d |
bytes 18.336× worse than git; server CPU 219.963× worse; push wins outright (0.779× wall, 0.736× CPU, 0.377× RSS) |
gunnar.negotiate_divergent |
197 s → 17.9 s at e60c95d; the clone rows did not move, so negotiation is not what a clone spends its time on |
The seal comparison is the one that matters and it goes against znippy. znippy's git-format seal runs at 42 362 objects/s where Gunnar's packfile repack() runs at 87 054 objects/s, i.e. 2.05× slower, on a workload 17× smaller (small workloads normally flatter per-object rates, so the gap is if anything understated). It also used 10 of 32 cores at peak, so the seal does not fan out. This is not a controlled comparison — different corpora, different code paths, one includes a fixture write — and it should not be quoted as one. It is enough to say that the archive is not cheaper to write than the thing it would replace.
znippy's already-compressed skip path, f99d076 on origin/master
P-005's "re-compresses at OpenZL level 19" objection is largely fixed, verified by reading znippy-common/src/precompressed.rs and the commit's own measurements:
| input | compress CPU before → after | saved |
| 5388 real git loose objects, 35.8 MB | 3.696 s → 0.297 s | 92.0% (12.4×) |
real 22.4 MB packfile + .idx |
4.055 s → 0.208 s | 94.9% (19.5×) |
Archives grow 0.17% and 1.2%, because the size fallback had been recovering that much at full codec cost. Resolution is ordered: caller ContentHint → extension table (now including pack/idx/midx/bitmap, deliberately not .rev) → magic-byte probe over the first 16 bytes. P-016 is effectively closed by this, and P-005's level-19 sub-argument with it.
Two things this does not fix, and they are the ones that decide the question:
- It saves CPU, not bytes. The 522 B/entry index overhead is untouched, being metadata rather than codec output.
- Known limit, stated in the commit rather than hidden: for an extensionless file larger than one slice, only chunk 0 is probed and chunks 1..N still reach the codec. An oid-keyed entry is extensionless by construction. It does not bite at 214 B or 6.6 KB; it would bite on a large blob.
Dependency arithmetic, measured by me 2026-08-03 with cargo tree -e normal on a scratch crate
Against the published znippy-common 0.9.12 (not the worktree):
crates in znippy-common's normal tree |
99 |
of those, absent from gunnar's Cargo.lock today |
28 |
Gunnar's appliance graph today (S-007, S-006) |
313 packages → 341, +8.9% |
| new arrow surface | the arrow meta-crate plus arrow-arith, -cast, -csv, -json, -ord, -row, -string, and csv, csv-core, atoi, ryu, six lexical-*; Gunnar today has only the six arrow-{array,buffer,data,ipc,schema,select} |
rayon 1.12.0 + rayon-core 1.13.0 |
new to Gunnar, via sysinfo 0.39.6; sysinfo/Cargo.toml:101 reads multithread = ["dep:rayon"] and znippy-common/Cargo.toml enables that feature explicitly. Gunnar's Cargo.lock has 0 rayon entries today |
openzl-sys-rs 0.2.0 |
not optional, a plain [dependencies] entry in znippy-common with no feature gate, so P-006 cannot be avoided by feature selection against the published crate |
znippy-plugin-git |
not published; the crates.io sparse index has no entry. Consuming the git format needs either a publish or the ../znippy path dep §4 forbids |
roaring 0.11.4, for comparison |
2 new crates (roaring, bytemuck); the bitmap half is nearly free if Gunnar ever wants it directly |
What breaks
Named specifically, because "it has costs" is not a decision input.
1. The appliance C-toolchain rule: breaks RETRACTED 2026-08-04
This section is history.
openzl-sys-rsis optional behind theopenzlfeature, and0.3.0vendors its C sources and is on crates.io. Everything below describes0.2.0and no longer describes the crate. See the correction block at the top.
P-006 held in full as of znippy origin/master, 2026-08-03. READ, openzl-sys-rs-0.2.0/build.rs:
:5-6fetcheshttps://github.com/facebook/openzl/archive/ca28e4bc….tar.gz, a pinned commit, at build time;:76-79does that fetch withCommand::new("curl"),:86-88withCommand::new("tar");:29-45runscmake::Config::new(...).build();:52,54,57linkstatic=zstd,static=lz4, andstdc++;:14-20falls back to building in/tmp/openzl-build-*whenOUT_DIRcannot hold a symlink, which on this box is RAM.
The sharpest form of it: §0 opens with "Zero shell, both halves. No git subprocess", and S-007 spent a wave removing gix-command from the appliance graph to make that true. Adding znippy puts curl and tar subprocesses back, at build time rather than run time, which is a different severity but the same rule. For a reproducible appliance image built in a sandbox with no network, it is disqualifying today. There is no feature to turn it off: openzl-sys-rs is unconditional in znippy-common.
2. LAW 3: breaks RETRACTED 2026-08-04
This section is history.
sysinfois a dev-dependency now (znippy-common/Cargo.toml:154), so it carries nothing into a consumer's graph, andcargo tree -p znippy-common -i rayonmatches nothing. See the correction block at the top for the precision that rayon is still in the znippy workspace lock via other members.
Linking znippy-common 0.9.12 put rayon in Gunnar's dependency graph, via sysinfo's multithread feature, which znippy-common turns on deliberately. LAW 3 makes rayon forbidden and names rayon_free_law.rs guards in skade, znippy and znippy-zoomies as the enforcement. PLAN.md:172 already records that znippy's guard has never run: its include! targets a file that does not exist, so it cannot compile and therefore cannot go red. The guard that would have caught this is exactly the one that has never been seen red.
This is znippy's to fix (drop multithread, or take the core count another way), and it is cheap. But today, adopting znippy means adopting rayon, and Gunnar is currently clean.
3. The single-writer requirement — holds, and Gunnar can supply it, at a cost
P-014 stands: zero hits for flock/LOCK_EX/try_lock/O_EXCL across the znippy workspace, and two concurrent appends corrupt the archive with no error from either writer.
Gunnar already takes a cross-process exclusive lock per repository — gunnar-store/gunnar-store.lock, the Arrow log's writer lock — and the head-to-head measured its consequence: run process-per-connection, fifteen of sixteen servers die with "the store is locked by another writer". So the guarantee exists. What adopting znippy would change is that this lock stops being an internal detail of the ref log and becomes load-bearing for cold-tier integrity too, in a component that will not report its own violation. Combined with S-019's lesson — an O_EXCL lock that does not self-clear turns a crash into a permanent outage, and PLAN.md records that Gunnar's lock does not self-clear — that is a real, unresolved operational edge to inherit for an archive that cannot detect the failure it is being protected from.
4. Per-entry overhead breaks for objects, is irrelevant for packs
522 B/entry measured, ~52 B of it in __gunnar_oid__. Fatal at 214 B/object, +7.9% at 6.6 KB/object, unmeasurable at pack granularity. See the correction above.
5. Pack-copy survives, because of (b), and only because of (b)
Under the packfile tier it survives and is measured: RepackOutcome reports objects_copied_from_pack against objects_recompressed, 11 copied / 1 recompressed on the sha256 fixture, and delta chains survive the repack. Under the git format as built — canonical inflated bytes — it does not survive at all: every cold-tier object would be objects_recompressed by construction, which is the RED state P-005's Verify describes.
6. P-013 is consistent with (b), and (b) is what it already recommends
skade's catalog measures 2.0 M ops/s at 0.31 µs p50, 492× Nessie. Excellent, and exclusive-locked, so one warehouse serves one process, and the multi-node answer is sharding by repository with a routing table rather than a REST catalog that would surrender ~500×. P-013's verdict is that iceberg's honest role is "a cold-archive and analytics tier … while the serving path never reads through it". Recommendation (b) is that verdict applied one layer down: the archive is where you publish to, never where you serve from. It also removes P-013's point 4 as a live problem — IcebergZnippyReader::extract_file does not verify checksums, silently dropping §4's advertised per-chunk blake3 guarantee — because no read path Gunnar depends on goes through it.
What (b) costs and forecloses
Costs, all small and all named:
- SQL over object bytes is given up. You cannot DuckDB the contents of a packfile. You keep SQL over the index, the commit graph and the reachability sets, which is where the analytic value actually is; nobody queries blob bytes.
- Per-chunk blake3 verification on every read is given up. Git's integrity mechanism is the oid itself and Gunnar already rehashes on the connectivity walk; the loss is detection of silent media corruption, which is real but is what a scrub pass or ZFS is for. Note that the iceberg path drops this guarantee anyway (
P-013.4). P-012stays Gunnar's problem, 23.7× lookup degradation on misses across 64 packs. It is already fixed byrepack()consolidating packs and by the multi-pack index (6ca99fd); it is not an argument for the archive.- znippy's release coupling is avoided rather than paid. §4 warns that a native format makes every schema change a znippy release before Gunnar can consume it. Under (b) that coupling exists only on the export path, where cadence does not matter.
Forecloses: honestly, very little. Nothing in (b) prevents (a) later: the §4.4/D14 swap protocol is written against the tier as a seam, the git format keeps being built and improved in znippy, and the sections are additive. The one thing (b) does foreclose is the "one file per repository, no filesystem at all" story — a pack tier is pack + idx files, so the off-filesystem appliance still needs an ObjectStore over something. That is a genuine loss and it is the strongest remaining argument for the archive.
What would have to be true to change the answer
All four, not any one. Two are met as of 2026-08-04; see the correction block at the top for what was checked and how.
- ✅ znippy ships an OpenZL-free build. A
codec-none/ store-only feature so a consumer can link the reader, the index and the reserved sections without cmake, withoutcurl, withoutlibstdc++. MET —openzlis an optional feature, andopenzl-sys-rs 0.3.0vendors its C sources anyway. - ✅ The rayon edge is gone (
sysinfo'smultithreaddropped or replaced) and znippy's ownrayon_free_law.rsguard is made to compile and seen red, so the next such edge is caught by a guard rather than by a dependency diff. MET on the edge —sysinfois a dev-dependency. The guard's missingcore.rsis present; it has not been seen red here. - The tier stores packfiles as entries, not objects — with
__gunnar_oid__mapping oid → (pack entry, offset) — so pack-copy survives, delta chains survive, and 522 B/entry is divided by tens of megabytes instead of by 214 bytes. This requires §4's "packs-in-znippy is rejected" to be retracted deliberately and in writing, on the ground that the reserved sections give back the sorted index and the SQL surface that rejection was protecting. - ❌
znippy-plugin-gitis published, so Gunnar can pin a version rather than path-dep../znippy. NOT MET —index.crates.ioreturns 404 for it, checked 2026-08-04.
If all four land, (a)-as-packs becomes the better design and the migration is a new ObjectStore behind the same seam. Until then it is a strictly larger, strictly slower container around the format it was meant to replace.
Amendment, 2026-08-04. Conditions 1 and 2 are met and the answer is still (b), which means the four conditions were never the real load-bearing argument — they were the cost side of a trade whose benefit was never measured. It has been measured now, for the one use a sixth reserved section was wanted for, and the benefit is negative: see the nine-corpus table at the top. A gc that searches for deltas puts them in the packfile, where pack-copy already reuses them, at negative bytes on disk.
Scope that claim exactly, because it is easy to over-read: what has no problem left to solve is gunnar's fetch path. Delta rows as znippy historization — first row the object, following rows deltas against it — are a separate and live piece of work, and the shape they must take is set out under Delta rows in znippy above. Condition 3 — store packfiles, not objects — remains the only shape in which the object archive would ever have been right, and it remains unbuilt.
The measurements that would settle what reading cannot
Each has a red condition, because a measurement with no red is a measurement that cannot disagree with me (LAW 2).
M1. Where the 522 B actually goes. Run znippy.git_archive_60k three times: ReachPolicy { max_commits: 0 } and no reserved builder; reach only; all three sections. Report archive_mb each time. ~3 lines in bench_kernels.rs::archive_build. Settles: whether the overhead is znippy's base index + lookup + trie (untunable from Gunnar's side) or the git sections (tunable). RED: if the sectionless archive is still
2× source on this corpus, no section tuning helps and the container itself is the cost.
M2. The same arm on a real corpus. 60 000 real loose objects from a real repository (mean ~6.6 KB), reporting archive_mb / source_mb and bytes_per_entry separately. Settles: whether Gunnar would live with 3.44× or 1.08×. RED: if bytes_per_entry moves materially with corpus, the fixed-overhead model above is wrong and both the headline and my correction of it need retracting.
M3. Packs as entries. Seal one repository's consolidated packfile + .idx as two znippy entries with all three sections attached, and compare archive_mb against pack + idx bytes, plus seal wall clock against gunnar.repack_1m at equal object count. Settles: whether change-condition 3 is live at all. RED: overhead > 1% of pack bytes, or a seal slower than the pack write it wraps.
M4. P-006's container build. cargo build the appliance feature set with a znippy backend in a network-isolated container with no cmake, and observe it fail; then assert the minimal appliance still builds in the same container. bench/Containerfile already exists and P-015 already anticipates this arm. It has never been run. RED: the isolated build succeeding would mean the OpenZL fetch is being served from a cache, and the test is measuring the cache rather than the constraint.
M5. gunnar.clone_cold_tier. P-005's own Verify, still not built: serve a full clone entirely from the cold tier, reporting bytes_out and objects_recompressed. Today the tier is a packfile and the expected value is ~0. Without this, "pack-copy survives from the cold tier" is proven at twelve objects on a fixture, not on a clone, which is the thing the whole decision rests on. This is the one I would build first.
M6. The bitmap claim, on real history. git_reach_10k reports query_speedup_x 282 256×. Nothing may be quoted from it: the baseline is a deliberately unmemoized ancestry walk, and the delta is 29 997 of 30 000 objects, i.e. the whole repository. The settling measurement is want − have on a real repository with real merge history, against Gunnar's current negotiator at e60c95d (17.9 s on negotiate_divergent, down from 197 s), reporting the object count in the delta. RED: a speedup reported over a delta of fewer than, say, 100 objects means nothing and the arm should refuse to report it.
What the git package format is for, then
It exists, it is on znippy origin/master (eaafaa1 · 4317af1 · f99d076), it is measured, and (b) does not waste it. Five answers, in descending order of how earned they are:
- The export and analytics tier. One sealed immutable archive per repository, published off the serving path into Iceberg/S3 — SQL over the whole forge, snapshots and time travel, cheap durability. This is
P-013's stated honest role for the whole iceberg/znippy axis, and it is the role in which every one of znippy's properties is an asset and none of its costs are on a hot path. The C toolchain does not matter in a batch publisher that does not ship in the appliance image. __gunnar_graph__, the commit graph as Arrow. The single most valuable piece.Refs.Logbecomes a scan, ancestry and merge-base become integer comparisons on generation numbers, and the table is readable by anything that reads Arrow. Measured: 311.6 ns/commit linear, 359.1 ns/commit merge-heavy, 200 k commits in ~65 ms. Gunnar can write and read this table with the arrow crates it already links; it does not need znippy to have it. That is the clean outcome: the work bought a schema and an algorithm, and those transfer.__gunnar_oid__, a real and reusable result. 52.0 B/object flat from 100 k to 8 M; batch lookup 1.51× / 1.78× / 2.62× over binary search with the margin growing with size exactly as the memory-latency-overlap argument predicts; 3.87–4.08× over fst, which also builds 1.8–2.9× slower. It settled the fst question with a measurement. Its home isznippy-zoomies::stree, which Gunnar could link on its own, without znippy, if a batch oid index over pack.idxfiles is ever wanted forP-012.__gunnar_reach__is built and correct, and that is all that may be said. znippy's own source says no speedup is claimed and nothing has been measured against a real repository; see M6. It is worth keeping precisely because the negotiation path is where Gunnar's remaining large numbers live.- It is the prototype of the pack-as-entry tier, if change-conditions 1–4 ever land. Nothing about the three sections assumes objects-as-entries; they index an object set, wherever its bytes sit.
The blunt version: the bytes are the weakest part of that format and the index is the strongest. (b) keeps the strongest part and declines the weakest.
What I could not determine
- I ran none of znippy's test suite. Building it requires the OpenZL fetch and cmake. Static count on
origin/master: 312#[test]functions, 44 of them inznippy-plugin-git. The brief's "233 tests verified" is not something I verified, and a static count is not a run. - The 330–400 B/object residual is inferred, not measured; see M1. It is the largest single unknown in this document.
__gunnar_graph__'s on-disk size was never reported by any arm, so the graph's share of the 522 B is unknown.- The seal comparison (42 362 vs 87 054 objects/s) is not controlled. Different corpora, different sizes, different code paths, one includes a fixture write. It is indicative.
PLAN.md:184-186is stale. It says znippy'sgitformat tests "are onfeat/git-package-format… Not merged tomaster". They are onorigin/masteras of 2026-08-03. Worth noting how it was found: the local/home/rickard/git/znippycheckout sits onfix/plugins-use-zoomies-inflate, 9 commits behindorigin/master, so every figure above was read fromorigin/masterexplicitly rather than from the working tree.P-016should be marked effectively closed byf99d076, andP-005's level-19 sub-argument retracted with it.P-006andP-014stand unchanged. Not edited here; this document touches nothing else.