nordisk/edda

gunnar against gitea and Forgejo

Generated: cargo xtask mashup → .nornir/assets/mashup_forge_bakeoff.svg and the table below. Generator: xtask/src/mashup.rs. Arms: bench/examples/forge/. Recipe for the standalone shell run: .nornir/bench/forge-bakeoff.sh.

Nobody types the numbers in

Every figure on the chart and in the table below is read back out of the iceberg bench_runs table through nornir bench history-show gunnar --json. There is no constant in the generator holding a measurement, and there is no fallback: an arm with no row is printed as NOT MEASURED, with the command that would produce it, rather than filled in from a remembered value.

That rule exists because of what hand-entered numbers did to this repository's own front page. It claimed 510 tests when the real count was 989, 314 crates when the appliance build resolved 278, 45 audit entries when stinky.md had 49, and "nothing has been load-tested" while P-015 was done. Every one of those was true on the day it was typed. A literal does not go stale loudly, so the fix is not to be more careful — it is to stop having literals.

What is being compared, and what is not

One engine, drawn twice. Forgejo is a gitea fork. During a real clone podman top catches both containers exec'ing the identical /usr/bin/git-upload-pack → git pack-objects --revs --thin --stdout --delta-base-offset --include-tag. The chart therefore draws gunnar against one rival bar, and that bar is whichever of the two sent fewer bytes — the comparison is never against the weaker of the pair. gunnar's server process spawned zero children under the same probe.

The znippy cold tier did not run. What is measured is gunnar on the filesystem: pack_tier returns an empty vector, RepackOptions::cold_tier() (crates/gunnar-store/src/repack.rs:277) has zero call sites in any server crate, and the control plane prints "no znippy cold tier is built". Objects are gix-odb packs on disk, the same place gitea's are. The arm records this as a znippy_wired column rather than as a silence, and it is in the conditions row below. No figure here is evidence for or against the tier. See cold-tier-decision.md.

Bytes lead, milliseconds follow. The byte counts are what one relay in front of every server counted; the wall-clock is reported beside them and is not what any verdict is computed from. On a shared box the milliseconds move with the load and the bytes do not.

Every bar is the wire, and the wire is the harsher column. One quantity for all four operations — the bytes the relay saw travel from server to client — so the four groups are comparable and one instrument produced all of them. The obvious alternative for the transfer arms is the pack that ended up on disk, and it is deliberately not what is drawn: index-pack --fix-thin completes a thin pack by copying in bases the client already held, so a Go forge's on-disk pack is larger than what it sent, while gunnar's is the same size because gunnar sends no thin pack. Scoring the fetch on the on-disk pack therefore reports a milder loss than the wire did. Both figures are in the table below, side by side, so which one the bar is can be checked rather than trusted.

What the arms cover, and what each spread says

Moved here from the front page 2026-08-05, where it was the paragraph under the chart. Every figure keeps the conditions it was measured under.

Cost leads, bytes follow. Bytes mostly tie — both engines hand out a pack git pack-objects made, so a tie is the expected answer. What does not tie is what a request costs the box, so that is the column the headline leads with, and it carries the client count it was measured at.

The ramp runs twice. c<N> is a cgroup quota of 8 CPUs across 32; c<N>-pin is 8 dedicated ones. gunnar's cost per request climbs 2.8× under the first and is flat under the second; the Go forges are flat under both. With dedicated cores gunnar's cost per request is within 2% of gitea's at 32 clients.

fetch runs in three maintenance states and under all four --delta-order choices. On a pack already 78.8% deltified, every ordering re-encodes LARGER than copying — 1.69× at best, 3.59× at worst — so monotone gc keeps the copy.

Every row carries whether the server it was measured against had a .bitmap. A push leaves none, gc writes one, and the two are different code paths on both sides of the wire.

Read is not O(client history). repack() writes a .bitmap and select_objects reads it: 0.278× the exact walk's store touches and 0.531× the shipping walk's — oden 2026-08-04, 12 real repos, client half behind.

Beyond the bake-off. All 19 workspace_nordisk repos serve in 686 MiB peak under 8 concurrent readers and 8 writers — arm gunnar.constellation, oden 2026-08-04.

How a row is made honest

Same RAM, CPU and disk parameters on all tests. bench/examples/forge/parity.rs does not take the caps on trust: it reads each container's own cgroup back and aborts the run unless the quotas that actually applied agree across the engines. Those read-back quotas — not the ones that were asked for — are the figures in the conditions row.

• One client. Stock git issues every request and judges every answer, so no server is ever asked by its own client. The version it ran is a column on every row.
• One fixture per corpus. Real history off this box, pushed to all three from the same bare repo. The tip oid is compared on every server, and each clone's object set is compared against the fixture's, before anything is scored. The object counts in the table are that check.
• One instrument. The counting relay in cargo xtask wireproxy sits in front of every engine and runs the same read/write loop, so whatever it miscounts it miscounts identically.
• Interleaved. Within a round the three engines run back to back on one fixture before the next fixture starts. With the engine on the outside, two measurements that end up in one ratio can be minutes apart, and on a box whose load moves during a run that gap is in the ratio.
• A failure is an abort. A server that never started, a server serving the wrong repository, and a relay that stopped counting each end the run rather than scoring a zero. The first setup script died on exactly that: a container that could not bind :22, and a stale token that made repository creation fail silently.

Which rows get bars

Four operations across up to three corpora and three engines is two dozen bars, and two dozen bars is a table someone has drawn rectangles on. The chart draws one bar group per operation, on the corpus where that operation's finding is sharpest, and prints the corpora it did not draw as ratios beside it.

Choosing a subset is the one move that could turn an honest measurement into a dishonest picture — pick the arms gunnar wins and every bar is true and the chart is a lie. So select_drawn builds the set by construction and check_drawn refuses to render one that hides a verdict the full set produced, or that retires the worst measured ratio in favour of a milder loss. The guard is seen red four ways in xtask/src/mashup.rs's own tests.

Every measured row is in the table below, drawn or not.

A known defect in the selection, found 2026-08-05

select_drawn's first rule picks one corpus per operation by sharpness() — the largest distance from a tie over any measured axis. On fetch, gc'd there are two corpora and they are two different maintenance states:

Only one of them gets drawn, and the front page labels it fetch, gc'd, so a reader takes it for the gc'd state rather than for one of two. Which one wins is not decided by anything about the fetch. Both rows have gunnar at 13 MB peak RSS against a rival 252 MB, so both score sharpness 19.384615 on RSS, and RSS is the largest distance from a tie on both — the two sharpnesses are equal to the bit. max_by returns the last of an equal run, and the rows are sorted by corpus name, so diverge-gcx is drawn because gcx sorts after gc.

The two rows do not say the same thing. On the run of 2026-08-05T01:57 the drawn one is 0.7201× on the wire (gunnar sends less) and the retired one is 0.9965× (a tie). On the previous committed run, 2026-08-04T23:42, the order was the other way: the drawn diverge-gcx was 1.0394× (gunnar sends more) and the retired diverge-gc was 0.9965×, the lower of the pair. Same rule, same tie-break, opposite direction — which is what an arbitrary choice looks like from the outside.

check_drawn does not catch either case, and is not wrong not to: 0.9965× is a tie by the 2% band, the tie verdict is covered on the wire axis by other drawn rows, and no verdict the full set produced goes undrawn. The hole is in the selection, not the guard. Fixing it by construction means one of:

• select per axis, so the axis a row is drawn for is the axis that chose it;
• draw both corpora of an operation whenever they disagree in verdict on any axis;
• break an exact sharpness tie on something stated rather than on sort order.

Until then, read fetch, gc'd as one of two maintenance states and take the other from the table below.

Every measured row

Conditions: 3 interleaved rounds · loadavg 155.23 at the end of the run · device read-ahead 2048 kB · gunnar cgroup 8.00 cpus / 8192 MB · gitea cgroup 8.00 cpus / 8192 MB · forgejo cgroup 8.00 cpus / 8192 MB · znippy cold tier not wired — gunnar on the filesystem

Source: iceberg bench_runs · oden · 32 cores · 2026-08-05T01:57:09.874732+00:00 · 3 interleaved rounds · loadavg 155.23

What the losing arm is

The fetch arm is S-003: the client deltifies, the server does not. gunnar has no server-side delta compression, so a fetch that a Go forge answers with a thin pack built out of deltas is answered by gunnar with whole objects. Pack copy recovers 100% of a clone and 97.6% of a fetch (P-009 in performance-design.md), which is why the clone arm is a tie and the fetch arm is not. It is drawn at its true length, in the losing colour, with the ratio printed next to it.