nordisk/znippy

znippy-zoomies — benchmarks

Class: benchmarks (class scorecard) · the measured numbers only. Design in zoomies-design.md; the why in zoomies-reasoning.md; how to run in zoomies-guide.md.

f0 — what the scorecard measures (and on what)

One suite, run through nornir's Bencher trait in xtask/examples/nornir-bench.rs — not cargo bench / criterion. cargo run -p xtask --release --example nornir-bench prints a single BenchRun JSON line; results are historized in the Iceberg warehouse by git SHA, gated at >10% regression, and rendered into the README. Heavy scales gate behind NORNIR_BENCH_SKIP_HEAVY, the planet scale behind NORNIR_BENCH_SKIP_PLANET.

Machine for every number below: Threadripper PRO 3975WX · 32 cores · 128 MB L3.

The headline is decompression: zoomies is the only decompressor that is multi-core in every row of the capability matrix, and its serial inflate core still beats the reference. Those two tables come first; the parallel-layout, lookup, and write-tail numbers follow.

Decompressor capability matrix — the headline

The column that matters: znippy-zoomies is multi-core in every row. No other tool is.

Only zoomies has a ✅ in every applicable row. The unique cell is arbitrary markerless gzip: lgz speculatively resyncs the DEFLATE bitstream and decodes such a file in parallel — every other tool falls back to a single stream.

linflate vs miniz_oxide — serial inflate core

Single stream, single core, on real class-file data (nornir scorecard linflate_guava / linflate_scala, each reporting paired *_mbs throughput and speedup_x):

So even before any parallelism, the shared inflate core is 1.33–1.39× the reference decoder on the same bytes. Parallelism (the matrix above) multiplies on top of this.

Candidate-offset output model (B6) — A/B, ljar

163 840-entry / 10 GB corpus, best-of-5 wall. A = per-segment allocation (old); B = candidate-offset, one buffer, in-place decode + forward-seek compaction. Output byte-identical.

B removes the allocator churn: not only faster at the best case, but the variance collapses — the allocator is no longer in the hot path.

stree — id → position lookup

Sparse sorted i64 keys (OSM-shaped), 1 M random present queries, ns/query best-of-N, built with RUSTFLAGS="-C target-cpu=native":

Harness soundness: our binary-search baseline (~1050 ns at 1 GB) matches Ragnar Groot Koerkamp's published reference (~1150 ns std binary search → ~27 ns S-tree+batch+prefetch single-thread = his 40×; ~7 ns on 6 threads). A default build without target-cpu=native is ~2× slower — e.g. 1 GB best batch 96 ns, 11.2× instead of 66 ns / 16.0×.

arrow-ipc metadata seal — the write-tail cost

Not a decompression number: the single-threaded seal tail that runs after the gatling blob pipeline drains (baseline 2026-06-18, znippy 0.9.0). This is znippy's serial write ceiling, not a zoomies decode path — listed so the whole scorecard is honest.

Notes: random_lookup grows ~linearly with N — a cold call re-decodes the whole lookup, with manifest_reads_per_lookup = 3. prefix_meta is ~1% ≈ 100% at every scale. The append-capable clone ArrowIpcSinkAppend is byte-identical to the original, worst throughput delta +2.0% — inside the ≤10% gate.