crates/storage-git-gix/parked-gix-2/bench-gix/walkpool-probe.rs
//! **Does one ceiling bound the box? No. Does the pool?**
//!
//! ```text
//! walkpool-probe <repo.git> [concurrency]
//! ```
//!
//! `Limits::walk_bytes` bounds one traversal. `gunnar serve` admits 256 of them
//! at once by default — a number chosen against CPU in `gunnar_ssh::admission`,
//! with nothing in it about memory. So the per-request ceiling multiplied by the
//! admission gate is the real bound, and it is measured in terabytes on a box
//! that has not got them. What the shipped server did about that was let the OOM
//! killer choose a victim, which is the one refusal an operator cannot read.
//!
//! [`gunnar_wire::WalkPool`] is the shared purse that fixes it, and this is the
//! measurement that says so: the **same** fan-out of concurrent clones, run
//! twice against the same repository in the same process, once with a pool and
//! once without, with `VmHWM` read at the end of each.
//!
//! # Why one process and two child processes
//!
//! `VmHWM` is a high-water mark that never falls, so an in-process before/after
//! would report the larger arm's peak for both. Each arm is therefore a child of
//! this one, exactly as `walkbudget-probe` re-execs per repository and for the
//! same reason.
//!
//! # Why the store is opened once and shared
//!
//! That is what a server does: one repository, many sessions. Opening a store
//! per thread would put N copies of the pack's mmap and gix's caches into the
//! measurement, and the pool does not bound those — it bounds the **walk**, so
//! the walk is what the arms are made to differ in.
//!
//! # What the arms must show, and the floor under them
//!
//! | arm | must |
//! |---|---|
//! | no pool | every clone completes, and the peak is the fan-out's |
//! | pool | the peak is materially lower, refusals are `WalkPoolFull`, and **at least one clone still completes** |
//!
//! The last clause is the one that matters. A pool of zero bounds memory
//! perfectly and is a server that answers nothing; without asserting that work
//! still got done, "the peak went down" is satisfied by the worst possible
//! outcome.
use anyhow::{bail, ensure, Context, Result};
use gix_hash::ObjectId;
use gunnar_bench::stages;
use gunnar_store::fs::FsStore;
use gunnar_store::ObjectStore;
use gunnar_wire::upload_pack::{Config, UploadPack};
use gunnar_wire::{Error, WalkPool};
use std::io::Write;
use std::sync::Arc;
/// The walk path on a repository that has a `.bitmap`.
///
/// A bitmap-answered clone never enters the walk — `walkbudget-probe` served a
/// whole 83-object clone at a budget of one — so without this every arm below
/// would be a fan-out over a code path that was not running, and the pool would
/// look like it bounded something it had never been asked about.
///
/// Written out in full rather than reached by leaving trait defaults in place
/// (P-020 rule 4): `packed` is forwarded on purpose, because the walk still
/// pack-copies and cutting that seam by accident is how `NoPackCopy` silently
/// disabled bitmaps for every arm it was the BEFORE column for.
struct NoBitmap<'a>(&'a dyn ObjectStore);
impl ObjectStore for NoBitmap<'_> {
fn hash_kind(&self) -> gix_hash::Kind {
self.0.hash_kind()
}
fn contains(&self, id: &gix_hash::oid) -> bool {
self.0.contains(id)
}
fn contains_without_rescan(&self, id: &gix_hash::oid) -> bool {
self.0.contains_without_rescan(id)
}
fn read<'b>(
&self,
id: &gix_hash::oid,
buf: &'b mut Vec<u8>,
) -> gunnar_store::Result<Option<gix_object::Data<'b>>> {
self.0.read(id, buf)
}
fn header(&self, id: &gix_hash::oid) -> gunnar_store::Result<Option<gix_object::Header>> {
self.0.header(id)
}
fn write(&self, kind: gix_object::Kind, bytes: &[u8]) -> gunnar_store::Result<ObjectId> {
self.0.write(kind, bytes)
}
fn packed(&self) -> Option<gunnar_pack::PackedObjects> {
self.0.packed()
}
/// `P-020: deliberately not forwarded.` **The one seam this type cuts, and
/// the whole reason it exists.**
///
/// The pool bounds the WALK, and a bitmap-answered clone does not walk:
/// `walkbudget-probe` served a whole 83-object clone at a budget of one
/// because `tree_closure` was never entered. Every mirror in this corpus
/// has a `.bitmap`, so without this `None` the fan-out below would be
/// thirty-two clones over a code path the pool never sees, and the
/// measurement would be of nothing.
///
/// Confined to this probe. Nothing in `crates/` may cut this seam; the
/// shipped read side keeps its bitmap, and the reason the walk still
/// matters in production is that a repository pushed into and never
/// repacked has none to keep.
fn bitmap(&self) -> Option<Arc<gunnar_store::bitmap::PackBitmap>> {
None
}
}
/// Counts the bytes and throws them away: holding thirty concurrent clones to
/// measure them would BE the measurement.
#[derive(Default)]
struct Sink {
bytes: u64,
}
impl Write for Sink {
fn write(&mut self, data: &[u8]) -> std::io::Result<usize> {
self.bytes += data.len() as u64;
Ok(data.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
fn status_kb(field: &str) -> u64 {
let Ok(text) = std::fs::read_to_string("/proc/self/status") else {
return 0;
};
text.lines()
.find_map(|line| line.strip_prefix(field)?.trim().strip_suffix(" kB"))
.and_then(|v| v.trim().parse().ok())
.unwrap_or(0)
}
fn tip(repo: &std::path::Path) -> Result<ObjectId> {
let out = std::process::Command::new("git")
.args([
"-C",
&repo.to_string_lossy(),
"rev-list",
"-n",
"1",
"--all",
])
.output()
.context("running git rev-list")?;
ensure!(out.status.success(), "git rev-list failed in {repo:?}");
String::from_utf8_lossy(&out.stdout)
.lines()
.find_map(|l| ObjectId::from_hex(l.trim().as_bytes()).ok())
.with_context(|| format!("{repo:?} holds no commit"))
}
/// What one arm's fan-out did.
struct Outcome {
served: usize,
refused_by_pool: usize,
hwm_kb: u64,
wire_bytes: u64,
}
/// `threads` concurrent clones of `repo`, sharing `pool`.
fn fan_out(repo: &std::path::Path, threads: usize, pool: Option<Arc<WalkPool>>) -> Result<Outcome> {
let want = tip(repo)?;
let kind = gunnar_store::fs::stored_hash_kind(repo)?;
let store = FsStore::open_or_init(repo, kind)?;
let objects = store.objects();
let refs = store.refs();
// A shared reference, so the `move` closures below copy the borrow rather
// than each trying to take the one store.
let refs = &refs;
let stripped = NoBitmap(&objects);
let read_side: &dyn ObjectStore = &stripped;
let format = match kind {
gix_hash::Kind::Sha1 => "sha1",
_ => "sha256",
};
let request = stages::fetch_request(&want, &[], format);
// `std::thread::scope`, the shape the other probes here use. Not gatling:
// this is the load GENERATOR standing in for N clients on N sockets, not
// work inside one request — P-004 is explicit that gunnar's throughput comes
// from serving many requests at once and not from fanning one out.
let results: Vec<Result<u64, Error>> = std::thread::scope(|scope| {
let handles: Vec<_> = (0..threads)
.map(|_| {
let pool = pool.clone();
let request = request.clone();
scope.spawn(move || {
let upload = UploadPack::new(read_side, refs).with_config(Config {
walk_pool: pool,
..Config::default()
});
let mut input = std::io::BufReader::new(std::io::Cursor::new(request));
let mut sink = Sink::default();
upload.serve_v2(&mut input, &mut sink).map(|_| sink.bytes)
})
})
.collect();
handles
.into_iter()
.map(|h| h.join().expect("a clone thread panicked"))
.collect()
});
let mut out = Outcome {
served: 0,
refused_by_pool: 0,
hwm_kb: status_kb("VmHWM:"),
wire_bytes: 0,
};
for result in results {
match result {
Ok(bytes) => {
out.served += 1;
out.wire_bytes += bytes;
}
Err(Error::WalkPoolFull { .. }) => out.refused_by_pool += 1,
// Anything else is a different fault, and counting it as "the pool
// worked" is how a probe reports a broken server as a bounded one.
Err(other) => bail!("a clone failed with {other}, which is not a pool refusal"),
}
}
Ok(out)
}
/// One arm in its own process, so `VmHWM` belongs to it alone.
fn arm(repo: &std::path::Path, threads: usize, pooled: Option<usize>) -> Result<()> {
let pool = pooled.map(WalkPool::new);
let out = fan_out(repo, threads, pool.clone())?;
println!(
"{{\"pool_bytes\":{},\"threads\":{},\"served\":{},\"refused_by_pool\":{},\
\"vmhwm_kb\":{},\"wire_bytes\":{},\"pool_peak_bytes\":{}}}",
pooled.map_or(0, |n| n),
threads,
out.served,
out.refused_by_pool,
out.hwm_kb,
out.wire_bytes,
pool.as_ref().map_or(0, |p| p.peak_bytes()),
);
Ok(())
}
/// Parse one field back out of a child's JSON line.
fn field(line: &str, name: &str) -> Option<u64> {
let at = line.find(&format!("\"{name}\":"))? + name.len() + 3;
line[at..].split([',', '}']).next()?.trim().parse().ok()
}
fn child(exe: &std::path::Path, repo: &str, threads: usize, pool: Option<usize>) -> Result<String> {
let mut cmd = std::process::Command::new(exe);
cmd.arg(repo)
.arg(threads.to_string())
.env("WALKPOOL_CHILD", "1");
if let Some(bytes) = pool {
cmd.env("WALKPOOL_BYTES", bytes.to_string());
}
let out = cmd.output().context("re-running this probe")?;
std::io::stderr().write_all(&out.stderr)?;
ensure!(
out.status.success(),
"the {} arm failed, so no number from it is a measurement",
if pool.is_some() { "pooled" } else { "unpooled" }
);
Ok(String::from_utf8_lossy(&out.stdout).trim().to_owned())
}
fn main() -> Result<()> {
let args: Vec<String> = std::env::args().skip(1).collect();
let Some(repo) = args.first() else {
bail!("walkpool-probe <repo.git> [concurrency] [purse-multiple]");
};
let threads: usize = args.get(1).and_then(|n| n.parse().ok()).unwrap_or(32);
if std::env::var("WALKPOOL_CHILD").is_ok() {
let pool = std::env::var("WALKPOOL_BYTES")
.ok()
.and_then(|v| v.parse().ok());
return arm(std::path::Path::new(repo), threads, pool);
}
let exe = std::env::current_exe().context("finding this binary")?;
// ── the hostile fan-out with nothing in its way ────────────────────────
let open = child(&exe, repo, threads, None)?;
let open_hwm = field(&open, "vmhwm_kb").context("the unpooled arm printed no VmHWM")?;
let open_served = field(&open, "served").unwrap_or(0);
ensure!(
open_served as usize == threads,
"the unpooled arm served {open_served} of {threads} clones. Every arm below compares \
against it, so it has to be the whole fan-out actually happening."
);
// One clone's worth, measured rather than assumed: the same fan-out at a
// concurrency of one.
let single = child(&exe, repo, 1, None)?;
let single_hwm = field(&single, "vmhwm_kb").context("the single arm printed no VmHWM")?;
ensure!(
open_hwm > single_hwm,
"{threads} concurrent clones peaked at {open_hwm} kB and one peaked at {single_hwm} kB. \
The fan-out costs nothing on this corpus, so it cannot show a pool bounding it — use a \
bigger repository."
);
// ── and the same fan-out against a purse that holds a few ──────────────
//
// A few and not exactly two, and the reason is a measured property of the
// pool rather than a rounding preference. Reservations are taken **as a
// walk grows**, because the walk's size is not known at the door, so a
// purse tight against a single walk lets every walk in the fan-out climb
// together and then refuses all of them with nobody finished. MEASURED oden
// 2026-08-05: four concurrent 14 042 000-object clones against a purse of
// two served **zero**, while thirty-two 352 396-object clones against the
// same ratio served two — the difference is that thirty-two threads on
// thirty-two cores do not advance in lockstep and four do.
//
// That is a bin-packing problem incremental reservation cannot solve, and
// the answer is architectural rather than arithmetical: `WalkPool` is the
// backstop and `--max-concurrent-transfers` is what stops the fan-out from
// being wide enough to jam it. See `WalkPool`'s own doc. The multiple here
// is the smallest one at which progress is deterministic on this box.
let multiple: usize = args.get(2).and_then(|n| n.parse().ok()).unwrap_or(4);
let purse = multiple * (single_hwm as usize * 1024);
let held = child(&exe, repo, threads, Some(purse))?;
let held_hwm = field(&held, "vmhwm_kb").context("the pooled arm printed no VmHWM")?;
let held_served = field(&held, "served").unwrap_or(0);
let refused = field(&held, "refused_by_pool").unwrap_or(0);
ensure!(
held_served > 0,
"the pooled arm served NOTHING. A pool that refuses every clone bounds memory perfectly \
and is not a server; that outcome satisfies every peak assertion below and is the worst \
one available."
);
ensure!(
refused > 0,
"the pooled arm refused nothing, so the purse of {purse} bytes was never the binding \
constraint and this arm is not about the pool"
);
ensure!(
held_hwm < open_hwm,
"the pooled fan-out peaked at {held_hwm} kB against the unpooled {open_hwm} kB — the \
pool did not bound anything"
);
eprintln!(
"\n── {threads} concurrent bitmap-less clones of {repo} ──\n\
one clone alone VmHWM {single_hwm:>9} kB\n\
{threads:>3} at once, no pool VmHWM {open_hwm:>9} kB served {open_served}\n\
{threads:>3} at once, {multiple}x pool ({purse} B) VmHWM {held_hwm:>9} kB served {held_served}, \
refused {refused} ✓\n\
the pool held the peak to {:.2}x one clone where the open fan-out reached {:.2}x, and \
every refusal was WalkPoolFull.",
held_hwm as f64 / single_hwm as f64,
open_hwm as f64 / single_hwm as f64,
);
Ok(())
}