nordisk/gunnar

gunnar bench/ — the benchmark & defect-finding suite

This crate measures gunnar's git server: how fast each operation is, how it scales with repository size, and — for the two storage engines — which one wins where and why. It is as much a defect finder as a scoreboard: several arms exist to catch a wrong answer (a torn object graph, a lost ref) that a timing number would hide.

bench/ links no storage engine directly. It knows a path, not an engine: an arm points GUNNAR_BENCH_GUNNAR_SERVER (or ENGINE_SERVER) at a gunnar serve binary built with a chosen engine, and drives it over git's real ext:: / ssh transport. That is what lets one binary be benchmarked as znippy (the shipped default) and gix (the A/B engine, never shipped) from the same run — the engine is chosen per pass by GUNNAR_ENGINE.

The batteries

Every timed op asserts its result is correct (a clone must fsck clean, a get must return the right bytes) — a bench that times a silently-failing op is measuring an error path.

The engine surface

Every git command lowers to a finite set of storage-trait methods (git-storage-trait::{GitOps, GitServe}). That is the whole seam — there is nothing below it. The benches and the complexity analysis cover it in full:

GitOps    put · put_pack · put_refs · put_refs_cas · get · has · size ·
          extents · refs · update_ref · reachable · gc
GitServe  read · header · sizes · head · set_head · emit_pack · select   (+ helpers)

Operations outside the engine — client-side commands (merge/rebase/diff/status), the control plane (repo/user/policy/replication), and LFS — are covered by sitetest, opstorm, and the gunnar-lfs tests, not by this engine suite.

Result 1 — algorithmic complexity (from code)

Variables: n objects, r refs, P packs (gix only), k batch, d delta depth, C commits, |W|/|H| want/have closure, s object size.

The recurring gap: znippy folds all objects into one S-tree (static search tree — znippy-zoomies/src/stree.rs, ~40× faster than binary search) so every point lookup is O(log n). gix probes packs serially, so every lookup carries an O(P) pack-count factor. That single ×P is the mechanism behind gix's slower log / blame / cat-file / fetch.

Command compositions (per engine): clone = reachable + k·get + refs; fetch = reachable(have/want) + closure gets; push = put_pack (+ ref CAS); ls-remote = refs; log/blame/cat-file = m·(get + header).

Full analysis: O-ANALYSIS.md.

Result 2 — measured, znippy vs gix

the desktop, 2026-09-01, 256 / 2048 / 16384 commits, 5 reps + warmup, both engines. engine-gix builds and serves clean. The box was not idle during the run (load-1 mean 13.4), so absolute times are provisional — but per-cell spreads are tight and non-overlapping between engines, so the ratios hold; only the absolutes would shift on a quiet box.

End-to-end commands

Per-op isolation (opsprobe)

One line: znippy is the faster reader / server (clone, reachability, refs); gix is the faster writer (push ingest) and has a lighter footprint.

Full tables + raw rows: ENGINE-MATRIX.md, runs/*/results.jsonl.

Result 3 — optimizations this suite drove (shipped znippy)

The analysis + matrix agreed on where znippy loses; both were fixed in znippy (plugins/native/znippy-plugin-git/), each proven byte-identical to the old path and across a fold-changing push before landing:

1. reachable memoized. It rebuilt two O(C) maps (by_commit, graph) on every negotiation; now folded once beside the bitmaps/graph and invalidated by the same token as commit_raw. Map-build O(C) → O(1): the root-tip query goes 1.258 ms → 0.73 µs at C = 20 000 (~1720×) and is now flat across a 417× range of C. (The full-closure walk stays O(C) — that is the bitmap-OR floor, not the maps.)
2. head / single-ref update_ref keyed. They cloned the whole BTreeMap<String,RefState> (O(r)) to read one row; a new RefLog::get(name) makes them O(r) → O(log r) — closing exactly the header/update_ref/has cells where gix was ahead.

Running it

# build one server carrying both engines (znippy default, gix behind a feature)
cargo build --release --example git-store-serve --features engine-gix
cargo build --release --example nornir-bench

cd bench
SERVER=<target>/release/examples/git-store-serve
NB=<target>/release/examples/nornir-bench

# two-engine command matrix: reps, arm prefix, sizes
ENGINE_SERVER=$SERVER BENCH_NB=$NB ./engine-matrix.sh 5 "gunnar.vs_git_" "256 2048 16384"

# per-op isolation, both engines
ENGINE_SERVER=$SERVER ./engine-ops-probe.sh 5

# a single arm, one engine
GUNNAR_ENGINE=znippy GUNNAR_BENCH_GUNNAR_SERVER=$SERVER $NB          # or =gix
NORNIR_BENCH_ONLY=gunnar.opstorm GUNNAR_OPSTORM_CLI=<gunnar-solo> $NB

Run on an idle box for publishable absolutes. A number taken under load is provisional; the arms record load-1 per run so a dirty row is labelled, not silently trusted. Add a 65536 grade (or GUNNAR_BENCH_H2H_BLOB_KIB=64 for a wide/deep-delta shape) for the large end.

Honesty notes

• Complexity marked inferred from structure: the S-tree inner constant, the redb tail B-tree, roaring-bitmap container costs, gix's .idx inner fanout. The ×P outer factor and the synchronous-vs-deferred push contrast are traced from call sites.
• |W|/|H| closure sizes are repo-topology-dependent and left symbolic.
• Every ratio above is dated and was taken under a recorded load; re-run idle before quoting an absolute.