crates/storage-git-gix/parked-gix-2/gunnar-store-gix/tests/repack_under_load.rs
#![cfg(feature = "fs")]
//! **`gc` while somebody is pushing.** The surface both 2026-08-05 sweeps named
//! as never swept.
//!
//! Everything in `tests/repack.rs` runs a repack against a repository nobody is
//! touching. That is the easy half, and it is the half that was already right:
//! the swap protocol snapshots what it will delete before it decides what to
//! carry, so a pack that lands mid-repack is copied twice at worst and never
//! unlinked unread. This file is the other half.
//!
//! # What is measured here, and what is not
//!
//! Two *processes* cannot repack and serve the same repository at once:
//! `FsStore` takes the Arrow log's `O_EXCL` lock on open, and `gunnar gc` opens
//! the store exactly as a transfer does. So the cross-process question answers
//! itself, and the interesting failure there is what the loser is *told*, which
//! is `gunnar-ssh`'s and is guarded in
//! `a_transfer_refused_before_its_first_byte_tells_the_client_why`.
//!
//! What is left, and what these tests are about, is **one process, one store**.
//! `repack()` takes `&self` and its `repack_lock` excludes only other repacks,
//! so a repack running against a store another thread is writing to is a
//! reachable state of the library, not a hypothetical.
//!
//! # What this file found, and what closed it
//!
//! `gunnar gc --prune` destroyed the objects of any push whose reachability walk
//! it overlapped. Not sometimes: **every time**, at every gate width from 200 µs
//! to 200 ms, because the walk and the window do not race — the walk simply
//! looks into a window that is open. gunnar has no receive-pack quarantine
//! (`a_refused_push_leaves_its_objects_in_the_object_database` pins that
//! deliberately), so between the object write and the ref transaction a push's
//! whole payload is unreferenced and reads as garbage.
//!
//! It was worse than one lost push. The destroyed round still committed its ref
//! transaction, leaving a reference pointing at nothing, and from then on
//! **every** `--prune` on that repository failed outright — `connectivity` walks
//! from the refs and a broken tip is an error, so one lost push disabled
//! maintenance for good. Stock git 2.53.0 does the same (`gc --prune=now`
//! against a hand-broken ref exits 128), so that half is parity and is left
//! alone; what is fixed is not getting there.
//!
//! The fix is `RepackOptions::prune_grace`, defaulting to
//! [`DEFAULT_PRUNE_GRACE`] — stock git's `gc.pruneExpire`. Both tests below now
//! run **two arms**: the default grace, which must lose nothing, and
//! `Duration::ZERO`, which must still lose, because a guard whose harness has
//! stopped being able to see the defect is not a guard (LAW 2).
//!
//! # `gc` against a repository being written by `mirror sync`: it cannot happen
//!
//! The last thing both sweeps asked for, and there is no test here for it
//! because there is nothing to race. Traced 2026-08-05, all three mirror paths:
//!
//! * **server-side inbound** — `Mirror.Add`, `Mirror.Sync`, `Mirror.Status` and
//! `Mirror.Remove` every one return `ControlError::Unimplemented(MIRROR_GAP)`.
//! Mirroring needs `gunnar-client`, which an S crate must never link (§2), so
//! the verb that would write into a served repository does not exist.
//! * **`gunnar mirror sync`** is a *client* verb. It writes into a
//! `MirrorRegistry` under `mirror_root`, a client-side directory, never into a
//! repository the server serves.
//! * **outbound replication** (`server::mirror::replicate_once`) does touch a
//! served repository, and it opens it with a plain `gix::open` **and not
//! `FsStore`** — deliberately, and its own comment says why: `FsStore` takes
//! the Arrow log's `O_EXCL` lock and a second handle would fight the transport
//! for it. So the replicator is a *reader*, and it is the reader case
//! `a_reader_mid_transfer_survives_a_gc_unlinking_the_packs_under_it` already
//! covers, by the same POSIX argument.
//!
//! Written down rather than left as a gap, because the third of those is one
//! commit away from being a writer, and on that day this paragraph is wrong and
//! should stop the change.
//!
//! # LAW 2 note on the concurrency tests
//!
//! A concurrency test that passes once proves nothing, and neither does one that
//! races and hopes. The writer and the repacker here **rendezvous**: the writer
//! says "my objects are in the ODB and my reference has not moved" and blocks
//! until the repacker has completed one repack in exactly that state. An earlier
//! draft only started both threads and was green 5 times in 10 — guaranteeing
//! that a repack *completed* does not guarantee its walk fell inside a window.
use std::path::{Path, PathBuf};
use std::process::Command;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::Arc;
use std::time::Duration;
use gix_hash::{Kind, ObjectId};
use gunnar_store::connectivity;
use gunnar_store::fs::FsStore;
use gunnar_store::repack::{DeltaPolicy, RepackOptions, DEFAULT_PRUNE_GRACE};
use gunnar_store::{FullName, ObjectStore, RefEdit, RefStore, Target};
/// How many write-then-publish rounds the concurrent tests run.
///
/// **Not a probability knob, and the first draft of this constant said it was.**
/// Every round rendezvouses with a repack, so the number of overlaps is the
/// number of rounds exactly and raising it buys nothing but time. What it has to
/// be is greater than 1: the zero-grace arm loses its **first** round and then
/// its dangling reference makes every later repack fail, so a run of 1 could not
/// tell "the grace works" from "only one repack ever ran".
///
/// Swept in `sweep_the_prune_race` at 4, 8, 16 and 32 with both graces. Under
/// `Duration::ZERO` the loss is 1 and the completed repacks are 1 at every size;
/// under the default grace the loss is 0 and the completed repacks equal the
/// rounds at every size. The result does not move with the count, so 8 is kept
/// as the cheapest size that is not 1.
const ROUNDS: usize = 8;
/// The gate widths the sweep runs, and the floor evidence behind
/// [`DEFAULT_PRUNE_GRACE`].
///
/// The "gate" is the span a real push holds objects in the ODB with no reference
/// on them: index the pack, verify a signature, run a protection rule, run a
/// pre-receive hook. 200 µs is a gate that does nothing; 200 ms is a hook that
/// shells out. The point of sweeping it is that the loss under a zero grace is
/// **flat across all four** — this is not a narrow race that a fast server
/// escapes.
const GATES: [Duration; 4] = [
Duration::from_micros(200),
Duration::from_millis(2),
Duration::from_millis(20),
Duration::from_millis(200),
];
fn git_available() -> bool {
Command::new("git")
.arg("--version")
.output()
.map(|o| o.status.success())
.unwrap_or(false)
}
fn git(cwd: &Path, args: &[&str]) -> String {
let out = Command::new("git")
.current_dir(cwd)
.args(args)
.output()
.unwrap_or_else(|err| panic!("running git {args:?}: {err}"));
assert!(
out.status.success(),
"git {args:?} failed in {}: {}{}",
cwd.display(),
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
struct Fixture {
_dir: tempfile::TempDir,
git_dir: PathBuf,
work: PathBuf,
store: Arc<FsStore>,
}
/// The production shape: many pushes, many packs, no repack, no bitmap.
///
/// `receive.unpackLimit=1` because git otherwise unpacks a small push into loose
/// objects and the multi-pack state this whole question is about never arises.
/// `git clone --bare` is **not** used to build this: it repacks and writes no
/// `.bitmap`, so a fixture made that way cannot reproduce anything.
fn fixture(pushes: usize) -> Fixture {
let dir = tempfile::tempdir().expect("temporary directory");
let git_dir = dir.path().join("repo.git");
let store =
Arc::new(FsStore::open_or_init(&git_dir, Kind::Sha1).expect("create the bare repo"));
git(
dir.path(),
&[
"--git-dir",
git_dir.to_str().unwrap(),
"config",
"receive.unpackLimit",
"1",
],
);
let work = dir.path().join("work");
std::fs::create_dir_all(&work).expect("create the working tree");
git(&work, &["init", "--quiet", "--initial-branch=main", "."]);
git(&work, &["config", "user.email", "lane@gunnar.invalid"]);
git(&work, &["config", "user.name", "unswept lane"]);
for round in 0..pushes {
std::fs::write(work.join("hello.txt"), format!("round {round}\n")).expect("write");
std::fs::write(work.join("big.txt"), "x".repeat(4096 * (round + 1))).expect("write");
git(&work, &["add", "hello.txt", "big.txt"]);
git(
&work,
&["commit", "--quiet", "-m", &format!("commit {round}")],
);
git(
&work,
&[
"push",
"--quiet",
git_dir.to_str().unwrap(),
"HEAD:refs/heads/main",
],
);
}
Fixture {
_dir: dir,
git_dir,
work,
store,
}
}
/// Every object in the repository, **read through the real `git` binary**.
///
/// Not through gunnar's own iteration, and for the reason `tests/repack.rs`
/// gives at the same helper: the set a repack must preserve is the set git can
/// see, and asking gunnar for it would let one bug hide another.
fn all_ids(git_dir: &Path) -> Vec<ObjectId> {
git(
git_dir,
&[
"--git-dir",
git_dir.to_str().unwrap(),
"cat-file",
"--batch-check=%(objectname)",
"--batch-all-objects",
],
)
.lines()
.filter(|line| !line.is_empty())
.map(|line| ObjectId::from_hex(line.as_bytes()).expect("an object id"))
.collect()
}
/// The pack count on disk, which is the only thing that makes the fixture the
/// production shape rather than a clone.
fn packs(git_dir: &Path) -> usize {
std::fs::read_dir(git_dir.join("objects/pack"))
.expect("pack dir")
.filter_map(|e| e.ok())
.filter(|e| e.path().extension().is_some_and(|x| x == "pack"))
.count()
}
// ─────────────────────────────────────────────────────────────────────────────
// 1. The window, made deterministic
// ─────────────────────────────────────────────────────────────────────────────
/// **`gc --prune` must not destroy the objects of a push that has not updated
/// its reference yet.** It did until 2026-08-05.
///
/// This is the state `a_refused_push_leaves_its_objects_in_the_object_database`
/// already describes, reached the other way. gunnar has **no quarantine**:
/// objects land in the ODB before the gate runs, so between `index_thin_pack`
/// finishing and `RefStore::transaction` committing, a push's whole payload is
/// unreferenced and reads as garbage to any reachability walk. `connectivity`
/// walks references and nothing else — no reflog, no mtime — so the only thing
/// that can tell that payload from garbage is *when it was written*.
///
/// The construction here is not a simulation of the window: it **is** the
/// window. Objects present, reference not yet pointing at them, byte-identical
/// to what a concurrent walk would see.
///
/// # Seen red, in the test
///
/// The second arm is the red half and it runs every time: the same repository,
/// the same walk, `prune_grace(Duration::ZERO)`, and the objects go. If the
/// grace were removed from `repack()` the first arm fails; if the harness ever
/// stopped being able to reach the objects at all — a fixture that never pushed,
/// a `contains` that always answers `false` — the second arm fails. Neither can
/// be green for the same wrong reason.
#[test]
fn prune_spares_a_push_that_has_landed_its_objects_and_not_yet_its_ref() {
if !git_available() {
eprintln!("skipping: no git on PATH");
return;
}
for (grace, must_survive) in [(DEFAULT_PRUNE_GRACE, true), (Duration::ZERO, false)] {
let f = fixture(12);
assert_eq!(
packs(&f.git_dir),
12,
"the fixture must be 12 pushes / 12 packs, or it cannot reproduce anything"
);
// A thirteenth push whose objects land and whose reference then goes
// away again: exactly the mid-push state, and exactly the refused-push
// state.
std::fs::write(f.work.join("hello.txt"), "the thirteenth\n").expect("write");
git(&f.work, &["add", "hello.txt"]);
git(&f.work, &["commit", "--quiet", "-m", "in flight"]);
let in_flight = git(&f.work, &["rev-parse", "HEAD"]);
git(
&f.work,
&[
"push",
"--quiet",
f.git_dir.to_str().unwrap(),
"HEAD:refs/heads/in-flight",
],
);
git(
&f.work,
&[
"push",
"--quiet",
"--delete",
f.git_dir.to_str().unwrap(),
"refs/heads/in-flight",
],
);
let tip = ObjectId::from_hex(in_flight.as_bytes()).expect("a commit id");
f.store.objects().refresh();
// The control. Without it a prune that dropped nothing would satisfy
// every line below, and so would a push that never reached the server.
assert!(
f.store.objects().contains(&tip),
"the in-flight commit is not in the object store, so this test would prove nothing"
);
let outcome = f
.store
.repack(
&RepackOptions::new()
.delta_policy(DeltaPolicy::Preserve)
.prune_unreachable(true)
.prune_grace(grace),
)
.expect("the pruning repack");
f.store.objects().refresh();
let survived = f.store.objects().contains(&tip);
assert_eq!(
survived, must_survive,
"grace {grace:?}: the in-flight commit {in_flight} survived={survived}, \
expected {must_survive}: {outcome:?}"
);
// Not only the tip. A commit is three objects and a prune that kept the
// commit while dropping its tree is a repository git cannot read.
if must_survive {
assert_eq!(
outcome.objects_pruned, 0,
"the grace spared the tip and pruned something else out from under it: \
{outcome:?}"
);
let fsck = Command::new("git")
.current_dir(&f.git_dir)
.args(["fsck", "--connectivity-only", "--no-dangling"])
.output()
.expect("run git fsck");
assert!(
fsck.status.success(),
"git 2.53.0 refused the repository a graced prune produced: {}{}",
String::from_utf8_lossy(&fsck.stdout),
String::from_utf8_lossy(&fsck.stderr)
);
} else {
assert!(
outcome.objects_pruned > 0,
"the zero-grace arm reclaimed nothing, so it is not the red half of \
anything: {outcome:?}"
);
}
}
}
// ─────────────────────────────────────────────────────────────────────────────
// 2. The same window, concurrently, through gunnar's own write sequence
// ─────────────────────────────────────────────────────────────────────────────
/// **A pruning repack running against a live gunnar push must not destroy that
/// push.** It did until 2026-08-05.
///
/// # Why the writer is not `git push`
///
/// The first version of this test had a thread running real `git push` against
/// the repository while another repacked with `--prune`. It was green, and it
/// was green **for the wrong reason**: `git push` to a local path spawns stock
/// `git-receive-pack`, which stages into `objects/incoming-*` and migrates out
/// of it only after the hooks have run. Stock git was protecting itself from
/// gunnar's prune. The probe was measuring git's quarantine, not gunnar's
/// window.
///
/// So the writer here does what **gunnar's** receive-pack does, in gunnar's
/// order, which is the sequence
/// `a_refused_push_leaves_its_objects_in_the_object_database` already pins:
/// write every object into the ODB, *then* run the gate, *then* move the
/// reference. There is no quarantine anywhere in that list.
///
/// Each round publishes an **independent root commit**, deliberately. The first
/// version chained them onto one branch, and every tip but the last was then
/// held alive by the tip after it, which is exactly the shape that would have
/// made this green over the defect.
///
/// # What is asserted
///
/// Not "no error came back": before the fix both threads returned `Ok`
/// throughout, and the ref transaction for the destroyed round **succeeded**,
/// leaving a reference pointing at nothing. So what is checked is object
/// presence, through gunnar and then confirmed through the `git` binary, because
/// a gunnar-side lookup bug must not be able to invent a loss or hide one.
///
/// # Seen red, in the test
///
/// The zero-grace arm is the red half and it runs every time. It also pins the
/// second half of the old defect: under a zero grace exactly **one** repack
/// completes, because the reference the destroyed round left behind makes every
/// later walk fail. Under the default grace all eight complete.
#[test]
fn a_concurrent_pruning_repack_does_not_destroy_a_gunnar_push() {
if !git_available() {
eprintln!("skipping: no git on PATH");
return;
}
let graced = measure_concurrent_prune(ROUNDS, Duration::from_millis(2), DEFAULT_PRUNE_GRACE);
assert_eq!(
graced.rounds, ROUNDS,
"only {} of {ROUNDS} rounds ran; nothing was raced",
graced.rounds
);
// The headline first, the control after: a broken rendezvous cannot report
// a loss, so checking `repacks` first would replace "a push was destroyed"
// with "the harness is wrong" in the failure the fix regressing produces.
assert_eq!(
graced.losses, 0,
"a pruning repack under the default grace destroyed {} of {ROUNDS} live pushes; \
repack errors after the first loss: {:?}",
graced.losses, graced.repack_errors
);
assert_eq!(
graced.repacks, ROUNDS,
"the rendezvous is broken: {} repacks completed inside {ROUNDS} windows, so this is \
not the measurement it claims to be. Errors: {:?}",
graced.repacks, graced.repack_errors
);
// The red half. Same harness, same rendezvous, grace removed.
let raw = measure_concurrent_prune(ROUNDS, Duration::from_millis(2), Duration::ZERO);
assert!(
raw.losses > 0,
"the zero-grace arm destroyed nothing either, so the arm above is not evidence of \
anything: the harness can no longer see the defect it guards"
);
assert_eq!(
raw.repacks, 1,
"the zero-grace arm completed {} repacks; it used to complete exactly 1, because the \
dangling reference its first loss leaves behind fails every later walk. If that has \
changed, the second half of this finding has moved. Errors: {:?}",
raw.repacks, raw.repack_errors
);
}
/// What one `(rounds, gate, grace)` run produced.
struct Run {
rounds: usize,
repacks: usize,
losses: usize,
repack_errors: Vec<String>,
}
/// Run `rounds` of "gunnar writes a commit, gate runs, ref moves" against a
/// pruning repack that rendezvouses with each window.
///
/// One writer for both tests and for the sweep, because the only difference
/// between them is two parameters and the whole point is that the result is a
/// function of those parameters (LAW 5).
fn measure_concurrent_prune(rounds: usize, gate: Duration, grace: Duration) -> Run {
use std::sync::mpsc;
let f = fixture(12);
// The rendezvous. The writer says "my objects are in the ODB and my
// reference has not moved" and waits; the repacker does exactly one repack
// in that state and says so. Nothing else is synchronised — not the object
// write, not the walk, not the transaction — so what is removed is the
// *luck*, not the mechanism.
let (in_window_tx, in_window_rx) = mpsc::channel::<()>();
let (repacked_tx, repacked_rx) = mpsc::channel::<()>();
let repacker = {
let store = Arc::clone(&f.store);
std::thread::spawn(move || {
let mut ran = 0usize;
let mut errors: Vec<String> = Vec::new();
// Ends when the writer drops its sender, which is the writer
// finishing rather than a timeout.
while in_window_rx.recv().is_ok() {
match store.repack(
&RepackOptions::new()
.delta_policy(DeltaPolicy::Preserve)
.prune_unreachable(true)
.prune_grace(grace),
) {
Ok(_) => ran += 1,
Err(err) => errors.push(err.to_string()),
}
if repacked_tx.send(()).is_err() {
break;
}
}
(ran, errors)
})
};
let mut published: Vec<(String, ObjectId)> = Vec::new();
let mut done = 0usize;
for round in 0..rounds {
let objects = f.store.objects();
// ── gunnar's receive-pack, step 1: the objects land in the ODB ──────
let blob = objects
.write(
gix_object::Kind::Blob,
format!("round {round}\n").as_bytes(),
)
.expect("write the blob");
let tree = objects
.write(gix_object::Kind::Tree, &tree_bytes(&[("f.txt", blob)]))
.expect("write the tree");
let commit = objects
.write(gix_object::Kind::Commit, &commit_bytes(tree, round))
.expect("write the commit");
// ── step 2: the gate. This is the window, and it is not zero. ───────
in_window_tx.send(()).expect("the repacker is alive");
std::thread::sleep(gate);
repacked_rx.recv().expect("the repacker answered");
// ── step 3: and only now does the reference move ────────────────────
let name = format!("refs/heads/round-{round}");
let full = FullName::try_from(name.as_str()).expect("a valid ref name");
f.store
.refs()
.transaction(vec![RefEdit::create(full, Target::Object(commit))])
.expect("the ref transaction");
published.push((name, commit));
done += 1;
}
drop(in_window_tx);
let (repacks, repack_errors) = repacker.join().expect("the repacker thread");
// The applied output: how many published references now point at an object
// that is not there. Read through gunnar, then confirmed through git, so a
// gunnar-side lookup bug cannot hide a real loss or invent one.
f.store.objects().refresh();
let objects = f.store.objects();
let mut losses = 0usize;
for (name, commit) in &published {
if !objects.contains(commit) {
losses += 1;
let seen = Command::new("git")
.current_dir(&f.git_dir)
.args(["--git-dir", f.git_dir.to_str().unwrap(), "cat-file", "-e"])
.arg(commit.to_string())
.output()
.expect("run git cat-file");
assert!(
!seen.status.success(),
"gunnar says {name} -> {commit} is gone and git says it is there; one of the \
two readers is wrong, which is a different and worse bug"
);
}
}
Run {
rounds: done,
repacks,
losses,
repack_errors,
}
}
/// Minimal tree object bytes: `100644 <name>\0<20-byte oid>`.
fn tree_bytes(entries: &[(&str, ObjectId)]) -> Vec<u8> {
let mut out = Vec::new();
for (name, id) in entries {
out.extend_from_slice(b"100644 ");
out.extend_from_slice(name.as_bytes());
out.push(0);
out.extend_from_slice(id.as_slice());
}
out
}
/// Minimal commit object bytes. No parent: each round publishes an independent
/// root commit, deliberately, so that losing one is a loss and not an ancestor
/// still held alive by a later reference. The first version of this test chained
/// them onto one branch, and every tip but the last was reachable from the tip
/// after it — which is exactly the shape that made it green over the defect.
fn commit_bytes(tree: ObjectId, round: usize) -> Vec<u8> {
let when = 1_700_000_000 + round as u64;
format!(
"tree {tree}\n\
author unswept lane <lane@gunnar.invalid> {when} +0000\n\
committer unswept lane <lane@gunnar.invalid> {when} +0000\n\
\n\
round {round}\n"
)
.into_bytes()
}
/// The parameter sweep behind [`ROUNDS`] and [`DEFAULT_PRUNE_GRACE`]'s floor,
/// run on demand.
///
/// Two parameters, both swept, because a single point would not distinguish "the
/// grace fixes it" from "this round count and this gate width happen not to
/// reproduce it":
///
/// * **rounds** 4, 8, 16, 32 at a 2 ms gate — does the count matter?
/// * **gate** 200 µs … 200 ms at 8 rounds — is this a narrow race a fast server
/// escapes?
///
/// It is `#[ignore]`d because it is a measurement rather than a guard, but it
/// still asserts the shape its own headline rests on, so it cannot print
/// nonsense and be believed.
///
/// # What this replaced, and why
///
/// There was a `the_window_widens_with_the_gate` here that compared the loss
/// rate at a 200 µs gate against a 20 ms gate and asserted the wider one lost at
/// least as much. It was green, and it was **blind**: instrumenting it showed
/// exactly one repack completing in each arm, because a free-running repacker on
/// this fixture is slower than the writer's whole loop. It was comparing two
/// single-repack runs and reading the difference as a trend. The rendezvous is
/// what replaced it — every round now has exactly one repack inside it by
/// construction, so the counts mean what they say.
#[test]
#[ignore = "a measurement, not a guard; run explicitly with --ignored"]
fn sweep_the_prune_race() {
if !git_available() {
eprintln!("skipping: no git on PATH");
return;
}
eprintln!("rounds sweep, gate 2ms");
for rounds in [4usize, 8, 16, 32] {
let graced =
measure_concurrent_prune(rounds, Duration::from_millis(2), DEFAULT_PRUNE_GRACE);
let raw = measure_concurrent_prune(rounds, Duration::from_millis(2), Duration::ZERO);
eprintln!(
" rounds={rounds:2} grace=2w: repacks={:2} losses={} grace=0: repacks={:2} \
losses={}",
graced.repacks, graced.losses, raw.repacks, raw.losses
);
assert_eq!(
graced.losses, 0,
"the graced arm lost a push at {rounds} rounds"
);
assert!(
raw.losses > 0,
"the zero-grace arm lost nothing at {rounds} rounds"
);
}
eprintln!("gate sweep, {ROUNDS} rounds");
for gate in GATES {
let graced = measure_concurrent_prune(ROUNDS, gate, DEFAULT_PRUNE_GRACE);
let raw = measure_concurrent_prune(ROUNDS, gate, Duration::ZERO);
eprintln!(
" gate={gate:>8?} grace=2w: repacks={:2} losses={} grace=0: repacks={:2} \
losses={}",
graced.repacks, graced.losses, raw.repacks, raw.losses
);
assert_eq!(
graced.losses, 0,
"the graced arm lost a push at gate {gate:?}"
);
assert!(
raw.losses > 0,
"the zero-grace arm lost nothing at gate {gate:?}"
);
}
}
/// **The same finding, on a real repository instead of a built fixture.**
///
/// `fixture(12)` is a synthetic 12-push repository, and a defect that only
/// reproduces on synthetic data is a defect in the fixture. This runs the
/// window against a **copy** of a real one from the shared production corpus at
/// `/home/rickard/scratch/gunnar-voucher-fx/pushed-corpus` — `tunnr.git` is the
/// **sha256** corpus, so it also carries the finding across the hash split that
/// every other test here takes at sha1.
///
/// `#[ignore]`d and env-driven because it needs a path that is not in the
/// repository. It copies before it touches anything: six other lanes read that
/// corpus and a `--prune` against the original would be somebody else's bad
/// afternoon.
///
/// GUNNAR_CORPUS=/home/rickard/scratch/gunnar-voucher-fx/pushed-corpus/tunnr.git \
/// cargo test -p gunnar-store --features fs,repack-delta \
/// --test repack_under_load on_a_real_repository -- --ignored --nocapture
///
/// MEASURED 2026-08-05 on oden, tunnr.git (12 packs, no bitmap, sha256): the
/// in-flight commit is destroyed under `Duration::ZERO` and survives under
/// [`DEFAULT_PRUNE_GRACE`], exactly as on the built fixture. The window is the
/// repository's, not the fixture's.
#[test]
#[ignore = "needs GUNNAR_CORPUS; run explicitly with --ignored"]
fn on_a_real_repository_the_window_is_the_same() {
if !git_available() {
eprintln!("skipping: no git on PATH");
return;
}
let Ok(source) = std::env::var("GUNNAR_CORPUS") else {
panic!("set GUNNAR_CORPUS to a bare repository in the production shape");
};
let source = PathBuf::from(source);
assert!(
source.join("objects/pack").is_dir(),
"{} is not a bare repository",
source.display()
);
for (grace, must_survive) in [(DEFAULT_PRUNE_GRACE, true), (Duration::ZERO, false)] {
// A COPY. The corpus is shared with six other lanes.
let dir = tempfile::tempdir().expect("temporary directory");
let git_dir = dir.path().join("repo.git");
let copied = Command::new("cp")
.args(["-a", source.to_str().unwrap(), git_dir.to_str().unwrap()])
.status()
.expect("copy the corpus repository");
assert!(copied.success(), "the corpus copy failed");
let packs = packs(&git_dir);
assert!(
packs >= 5,
"{} holds {packs} pack(s); a corpus that has been repacked cannot show this",
source.display()
);
let kind = if git(&git_dir, &["rev-parse", "--show-object-format"]) == "sha256" {
Kind::Sha256
} else {
Kind::Sha1
};
let store = FsStore::open_or_init(&git_dir, kind).expect("open the copy");
// An unreferenced object, written exactly where a push's payload lands:
// in the ODB, with no reference on it yet.
let orphan = store
.objects()
.write(gix_object::Kind::Blob, b"an in-flight push's payload\n")
.expect("write the orphan");
store.objects().refresh();
assert!(
store.objects().contains(&orphan),
"the orphan is not in the store, so this run would prove nothing"
);
let outcome = store
.repack(
&RepackOptions::new()
.delta_policy(DeltaPolicy::Preserve)
.prune_unreachable(true)
.prune_grace(grace),
)
.expect("the pruning repack");
store.objects().refresh();
let survived = store.objects().contains(&orphan);
eprintln!(
" {} ({kind:?}, {packs} packs) grace {grace:?}: orphan survived={survived}, \
pruned={}",
source.display(),
outcome.objects_pruned
);
assert_eq!(
survived,
must_survive,
"grace {grace:?} on {}: survived={survived}, expected {must_survive}: {outcome:?}",
source.display()
);
// And the repository is still one stock git will read, whichever arm.
let fsck = Command::new("git")
.current_dir(&git_dir)
.args(["fsck", "--connectivity-only", "--no-dangling"])
.output()
.expect("run git fsck");
assert!(
fsck.status.success(),
"git 2.53.0 refused the repository after grace {grace:?}: {}{}",
String::from_utf8_lossy(&fsck.stdout),
String::from_utf8_lossy(&fsck.stderr)
);
}
}
/// **The default `gc`, with no `--prune`, came up clean against a live writer.**
///
/// The other half, and the one that is a good result rather than a defect.
/// `repack()` without pruning deletes only what it snapshotted and carries
/// everything it enumerated, so a push that lands mid-repack is at worst copied
/// twice. Saying so matters because it makes the finding above specific: the
/// danger was `--prune`, not `gc`, and a report that said "gc is unsafe under
/// load" would send someone to rewrite a swap protocol that is correct.
///
/// # This is a smoke test, and NO red half was found for it
///
/// Stated plainly rather than dressed up, because a green nobody has seen red is
/// not a result (LAW 2). Three probes were tried on 2026-08-05 and none turned
/// the loss assertion red:
///
/// 1. **adding `.prune_unreachable(true)` to the repacker.** Still green — and
/// that is the whole reason the test above had to stop using `git push`. The
/// writer here is stock `git-receive-pack`, which quarantines in
/// `objects/incoming-*`, so it protects itself from gunnar's prune. This test
/// therefore cannot see the prune defect at all.
/// 2. **deleting the `self.objects().refresh()` before the object iteration** —
/// the 2026-08-03 data-loss shape `repack()`'s own comment describes. Still
/// green.
/// 3. **re-snapshotting the pack directory at retire time**, so the deletion set
/// is newer than the object set, which is that same shape made unmissable. It
/// went red on the **control** (`no repack ran`), not on the loss assertion:
/// every repack failed its own verification step and deleted nothing. That is
/// a control firing for the wrong reason, and it is also the answer — the
/// verification in step 3 of the swap protocol refuses before the first
/// unlink, so the failure mode is a refused repack rather than a lost object.
///
/// So what this test actually establishes is narrow and worth having: a
/// non-pruning repack, running against real concurrent `git push` traffic, leaves
/// every pre-existing object, every published commit and `git fsck` intact. What
/// it does **not** establish is that it would notice if that stopped being true.
/// The thing that would notice is `repack()`'s own verification, and the third
/// probe is the evidence that verification is live.
#[test]
fn a_repack_without_prune_against_a_live_writer_keeps_every_object() {
if !git_available() {
eprintln!("skipping: no git on PATH");
return;
}
let f = fixture(12);
let before = all_ids(&f.git_dir);
assert!(
before.len() > 30,
"the fixture holds only {} objects; too small to say anything",
before.len()
);
let stop = Arc::new(AtomicBool::new(false));
let pushes_done = Arc::new(AtomicUsize::new(0));
let writer = {
let work = f.work.clone();
let git_dir = f.git_dir.clone();
let stop = Arc::clone(&stop);
let done = Arc::clone(&pushes_done);
std::thread::spawn(move || {
let mut tips = Vec::new();
for round in 0..ROUNDS {
if stop.load(Ordering::Relaxed) {
break;
}
std::fs::write(work.join("hello.txt"), format!("safe {round}\n")).expect("write");
git(&work, &["add", "hello.txt"]);
git(
&work,
&["commit", "--quiet", "-m", &format!("safe {round}")],
);
tips.push(git(&work, &["rev-parse", "HEAD"]));
git(
&work,
&[
"push",
"--quiet",
git_dir.to_str().unwrap(),
"HEAD:refs/heads/main",
],
);
done.fetch_add(1, Ordering::Relaxed);
}
tips
})
};
let repacker = {
let store = Arc::clone(&f.store);
let stop = Arc::clone(&stop);
std::thread::spawn(move || {
let mut ran = 0usize;
for _ in 0..ROUNDS {
if stop.load(Ordering::Relaxed) {
break;
}
if store
.repack(&RepackOptions::new().delta_policy(DeltaPolicy::Preserve))
.is_ok()
{
ran += 1;
}
}
ran
})
};
let tips = writer.join().expect("the writer thread");
stop.store(true, Ordering::Relaxed);
let repacks = repacker.join().expect("the repacker thread");
assert!(
pushes_done.load(Ordering::Relaxed) >= ROUNDS / 2,
"the writer only managed {} pushes; nothing was raced",
pushes_done.load(Ordering::Relaxed)
);
assert!(repacks > 0, "no repack ran, so nothing was raced");
f.store.objects().refresh();
let objects = f.store.objects();
// Nothing that existed before the storm is gone.
let mut lost: Vec<ObjectId> = Vec::new();
for id in &before {
if !objects.contains(id) {
lost.push(*id);
}
}
assert!(
lost.is_empty(),
"a non-pruning repack lost {} of {} pre-existing objects: {lost:?}",
lost.len(),
before.len()
);
// Nor is anything the writer published.
let mut lost_tips = Vec::new();
for tip in &tips {
let id = ObjectId::from_hex(tip.as_bytes()).expect("a commit id");
if !objects.contains(&id) {
lost_tips.push(tip.clone());
}
}
assert!(
lost_tips.is_empty(),
"a non-pruning repack destroyed {} of {} published commits: {lost_tips:?}",
lost_tips.len(),
tips.len()
);
connectivity::reachable(&objects, &f.store.refs())
.expect("every reference must still reach its whole closure after a concurrent repack");
// And stock git agrees, which is the judge on anything gunnar wrote.
let out = Command::new("git")
.current_dir(&f.git_dir)
.args(["fsck", "--connectivity-only", "--no-dangling"])
.output()
.expect("run git fsck");
assert!(
out.status.success(),
"git fsck refused the repository a concurrent repack produced: {}{}",
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
);
}
/// **A reader mid-transfer against a `gc` that is unlinking the packs under
/// it.** The first question both sweeps asked, and the one the grace period
/// does *not* answer.
///
/// A fetch that has negotiated is reading objects out of packs by id. A
/// concurrent `gc` writes a new pack and then **unlinks** every old one. The
/// grace period is no help here: the objects at risk are old, in old packs, and
/// what threatens them is not the prune decision but the `unlink` that follows
/// any fold at all.
///
/// # The answer, and it is POSIX's rather than gunnar's
///
/// A reader that has the pack **mapped** keeps reading it after the directory
/// entry is gone: `unlink` removes the name, not the inode, and the mapping
/// holds the last reference. `FsStore` reaches `gix-odb`, which mmaps pack
/// data, so a transfer that has begun cannot be starved by a fold that
/// completes underneath it. That is a **documented position**, not a mechanism
/// gunnar implements, and it has one consequence worth writing down: it does
/// not extend to a reader that has *not yet opened* the pack. Such a reader
/// takes a fresh snapshot and finds the new pack, which holds the same objects,
/// so it is fine for a different reason — and neither reason is "the repack
/// waits", because it does not.
///
/// # What is asserted
///
/// Applied output: every object id read back **byte for byte**, against what it
/// held before the storm, by a reader that took its id list before the first
/// repack and never refreshed. Not `is_ok()` — a read that returned the wrong
/// bytes, or a zero-length body, is `Ok` and is the failure this exists to
/// find.
///
/// The prune is on with a **zero grace**, deliberately: that is the most
/// destructive fold gunnar can perform, it unlinks the most, and it is the only
/// setting under which the reader's packs can go away while nothing new holds
/// their contents. Everything the reader looks at is reachable from a
/// reference, so a correct prune keeps all of it.
///
/// # One fold, and that is the whole event
///
/// MEASURED: the repacker loops until told to stop and completes **one**
/// rewriting fold, because the second finds a single consolidated pack with
/// nothing to do. That is not a weakness of the fixture — it is the shape of
/// the thing. The event under test is the moment twelve `.pack` files lose
/// their directory entries while a reader holds them open, and it happens
/// exactly once per fold. The reader's forty passes are what make sure it is
/// still reading afterwards, not an attempt to catch a second one.
///
/// # Seen red
///
/// The reader is checked against the recorded bytes, so corrupting one recorded
/// body turns it red on that id. What could NOT be made red: no probe was found
/// that makes the mapped-file read fail, which is the point — the guarantee is
/// the kernel's. What this test would catch is gunnar losing it, by copying
/// packs instead of mapping them, or by closing a handle mid-transfer.
#[test]
fn a_reader_mid_transfer_survives_a_gc_unlinking_the_packs_under_it() {
if !git_available() {
eprintln!("skipping: no git on PATH");
return;
}
let f = fixture(12);
let ids = all_ids(&f.git_dir);
assert!(
ids.len() > 30,
"the fixture holds only {} objects; too small to say anything",
ids.len()
);
// The reader's own view, taken BEFORE any repack, exactly as a transfer
// that has already negotiated holds one.
let objects = f.store.objects();
let mut before: Vec<(ObjectId, Vec<u8>)> = Vec::new();
let mut buf = Vec::new();
for id in &ids {
let data = objects
.read(id, &mut buf)
.expect("read")
.unwrap_or_else(|| panic!("the fixture does not hold {id}"));
before.push((*id, data.data.to_vec()));
}
let stop = Arc::new(AtomicBool::new(false));
let folds = Arc::new(AtomicUsize::new(0));
let repacker = {
let store = Arc::clone(&f.store);
let stop = Arc::clone(&stop);
let folds = Arc::clone(&folds);
std::thread::spawn(move || {
let mut errors: Vec<String> = Vec::new();
while !stop.load(Ordering::Relaxed) {
match store.repack(
&RepackOptions::new()
.delta_policy(DeltaPolicy::Preserve)
.prune_unreachable(true)
.prune_grace(Duration::ZERO),
) {
Ok(outcome) if outcome.rewrote => {
folds.fetch_add(1, Ordering::Relaxed);
}
Ok(_) => {}
Err(err) => errors.push(err.to_string()),
}
}
errors
})
};
// The reader keeps reading through the storm, off the handle it already
// had. No `refresh()`: a transfer does not re-open the object database
// between two entries of the pack it is writing.
let mut mismatches: Vec<String> = Vec::new();
let mut rounds = 0usize;
for _ in 0..40 {
for (id, expected) in &before {
match objects.read(id, &mut buf) {
Ok(Some(data)) if data.data == expected.as_slice() => {}
Ok(Some(data)) => mismatches.push(format!(
"{id}: {} bytes, expected {}",
data.data.len(),
expected.len()
)),
Ok(None) => mismatches.push(format!("{id}: gone mid-transfer")),
Err(err) => mismatches.push(format!("{id}: {err}")),
}
}
rounds += 1;
}
stop.store(true, Ordering::Relaxed);
let errors = repacker.join().expect("the repacker thread");
// The controls, both of them. A run where the repacker never folded, or
// where the reader never looked, would satisfy the assertion below over an
// untouched repository.
assert!(
folds.load(Ordering::Relaxed) > 0,
"no repack rewrote anything, so no pack was ever unlinked under the reader. \
Repack errors: {errors:?}"
);
assert!(rounds > 0 && !before.is_empty());
assert!(
mismatches.is_empty(),
"a reader holding an open object database lost {} of {} objects to a concurrent \
pruning gc, over {rounds} passes and {} folds. unlink(2) removes a name, not an \
inode, so a mapped pack survives its directory entry — this says gunnar has \
stopped mapping, or has closed a handle mid-transfer: {mismatches:?}",
mismatches.len(),
before.len(),
folds.load(Ordering::Relaxed)
);
}
/// **Two repacks at once, looped.** `repack_lock` is a plain in-process mutex,
/// so this is what it is for; the question is whether the *loser* is left in a
/// consistent state rather than whether the two serialise.
///
/// # Seen red
///
/// Replacing the `repack_lock` guard in `FsStore::repack` with a no-op turns
/// this red, usually as a `MissingObject` from the verification step: two runs
/// snapshot the same packs, both write a new one, and the second unlinks what
/// the first was still carrying.
#[test]
fn two_concurrent_repacks_leave_every_object_readable() {
if !git_available() {
eprintln!("skipping: no git on PATH");
return;
}
let f = fixture(12);
let before = all_ids(&f.git_dir);
let mut errors: Vec<String> = Vec::new();
for _round in 0..8 {
let a = {
let store = Arc::clone(&f.store);
std::thread::spawn(move || {
store
.repack(&RepackOptions::new().delta_policy(DeltaPolicy::Preserve))
.map_err(|err| err.to_string())
})
};
let b = {
let store = Arc::clone(&f.store);
std::thread::spawn(move || {
store
.repack(&RepackOptions::new().delta_policy(DeltaPolicy::Preserve))
.map_err(|err| err.to_string())
})
};
for outcome in [a.join().expect("thread a"), b.join().expect("thread b")] {
if let Err(err) = outcome {
errors.push(err);
}
}
}
assert!(
errors.is_empty(),
"concurrent repacks reported {} failures: {errors:?}",
errors.len()
);
f.store.objects().refresh();
let objects = f.store.objects();
let mut lost: Vec<ObjectId> = Vec::new();
for id in &before {
if !objects.contains(id) {
lost.push(*id);
}
}
assert!(
lost.is_empty(),
"two concurrent repacks lost {} of {} objects: {lost:?}",
lost.len(),
before.len()
);
let out = Command::new("git")
.current_dir(&f.git_dir)
.args(["fsck", "--connectivity-only", "--no-dangling"])
.output()
.expect("run git fsck");
assert!(
out.status.success(),
"git fsck refused what two concurrent repacks produced: {}{}",
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
);
}