crates/storage-git-gix/src/odb.rs
//! The `gix-odb` handle pool and the pack-directory staleness watch (`P-012`).
//!
//! # What was actually costing the misses
//!
//! `P-012` predicted that a lookup which misses is slow because it must be
//! disproved against **every** pack index, so cost is linear in pack count.
//! That is real, and it is **not** the largest term. Measured on oden,
//! 2026-08-03, one million distinct objects, misses per second (see
//! `.nornir/performance-design.md` `P-012` for how each row was taken):
//!
//! | | 1 pack | 64 packs |
//! |---|---|---|
//! | as this crate did it before | **78 183** | **3 376** |
//! | pooled handles, `RefreshMode::Never` | 436 196 | 60 337 |
//! | …and a multi-pack index | *n/a* | 432 113 |
//!
//! **A miss got 5.6× faster at ONE pack**, where there is no `O(M)` term at all
//! to explain it. The cause is `gix_odb`'s default
//! [`RefreshMode::AfterAllIndicesLoaded`]: when a lookup falls through every
//! loaded index, `Handle::contains` calls `load_one_index`, which calls
//! `consolidate_with_disk_state`, which is a **re-scan of `objects/` and
//! `objects/pack` from disk, behind the store's global write lock, on every
//! single miss**.
//! That is the ~10 µs a miss spends at one pack and the great majority of the
//! ~300 µs it spends at 64, and it is what a `have` line from a client that
//! gunnar has never heard of pays.
//!
//! The second term is this crate's own: the store it was lifted from built a **fresh
//! `gix-odb` handle for every call**. `Store::to_handle_arc()` runs
//! `collect_snapshot()`, which clones one `Arc` per loaded index into a new
//! `Vec`, so handle construction is itself `O(packs)`, and it was being paid on
//! hits too, where there is no refresh to blame: hits ran at 1 073 582/s over
//! one pack and **100 323/s** over 64.
//!
//! Only the third term is the one `P-012` named, and it is the one left standing
//! once the other two are gone: 436 196 → 60 337 across 64 packs, i.e. 7.2×. A
//! **multi-pack index** removes it, which is what takes the last row back to
//! 432 113, inside 1% of the single-pack rate.
//!
//! **[`crate::midx`] is now lifted and wired**, and it is the second half of
//! this module: on its own it makes the miss path *worse* while the default
//! refresh mode is on (MEASURED oden 2026-08-03, -28%), and it only pays once
//! this module has stopped the per-miss directory scan. The order the two
//! landed in is therefore not an accident. [`OdbPool::index_slots`] is how you
//! tell which of the two shapes a store is actually in.
//!
//! # What this module does
//!
//! 1. **Pools handles** instead of building one per call, bounded at
//! [`POOL_CAPACITY`]. `gix_odb`'s handle is `Send` but not `Sync`, its
//! snapshot and inflate state being `RefCell`s, so it cannot be shared;
//! a lease is the shape that fits. Memory is bounded by the pool rather than
//! by concurrency, which is the second half of what `P-012` asks for.
//! 2. **Sets [`RefreshMode::Never`] on every pooled handle**, so a miss is a
//! binary search and not a directory scan. This is git's own answer:
//! `has_object()` passes `OBJECT_INFO_QUICK` by default and only re-scans
//! when the caller opts in with `HAS_OBJECT_RECHECK_PACKED`, precisely
//! because misses are the common case on a server.
//! 3. **Puts the re-scan back exactly where it is needed**, via a one-`statx`
//! watch on `objects/pack`; see [`OdbPool::refresh_if_stale`]. git makes the
//! *writer* call `reprepare_packed_git()`; gunnar cannot, because the code
//! that drops a pushed pack into `objects/pack` lives in another crate and
//! reaches this one only through the storage trait. So the
//! check is automatic, and it is exact.
//!
//! [`RefreshMode::AfterAllIndicesLoaded`]: gix_odb::store::RefreshMode::AfterAllIndicesLoaded
//! [`RefreshMode::Never`]: gix_odb::store::RefreshMode::Never
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex, MutexGuard, RwLock};
use std::time::{Duration, SystemTime};
/// The handle type every lease hands out.
///
/// Wrapped in `gix_odb::Cache`, and that wrapper is the only place a **delta
/// base cache** can be attached: `set_pack_cache` is a method on `Cache`, and
/// the bare `store::Handle` underneath it has no cache slot at all, so its
/// `gix_pack::Find::try_find` takes the trait's default body and passes
/// `gix_pack::cache::Never`. `Cache` derefs to the handle, so
/// `refresh_never()` and everything else still reach through it unchanged.
/// See [`DELTA_BASE_CACHE_BYTES`].
pub type OdbHandle = gix_odb::Cache<gix_odb::store::Handle<Arc<gix_odb::Store>>>;
/// How many idle handles are kept.
///
/// A handle holds one `Arc` per loaded pack index plus an inflate buffer, so the
/// cap is what stops `P-012`'s "memory grows linearly with concurrency". Beyond
/// it, a returned handle is dropped rather than kept: the pool exists to remove
/// a per-call allocation, not to be a cache with its own eviction policy.
const POOL_CAPACITY: usize = 64;
/// The delta base cache each pooled handle carries, in bytes.
///
/// # Why there has to be one at all
///
/// `gix-odb`'s `new_pack_cache` defaults to `None`, and `Cache::try_find` then
/// passes `gix_pack::cache::Never` (`gix-odb-0.83.0/src/cache.rs:102` and
/// `:228`). Every handle in this pool therefore re-resolved a delta chain from
/// its whole base on **every** read: object N of a chain re-inflated the root
/// and re-applied N patches, which is `O(depth²)` over a range and is precisely
/// what git's delta base cache (`packfile.c:1167-1325`, inserting every
/// intermediate on the way back up at `:1715-1719`) exists to avoid.
///
/// MEASURED on oden, 2026-08-04, `bench/examples/deltachain-probe.rs` over
/// nornir's own 13 431-object pack, every object read once on a cold handle in
/// pack order:
///
/// | cache | wall | vs none |
/// |---|---:|---:|
/// | none (what this was) | 0.963 s | 1.00× |
/// | 256 KiB | 0.588 s | 1.64× |
/// | 1 MiB | 0.405 s | 2.38× |
/// | **4 MiB** | **0.352 s** | **2.74×** |
/// | 16 MiB | 0.281 s | 3.43× |
/// | 64 MiB | 0.285 s | 3.37× |
///
/// # Why 4 MiB and not git's 96
///
/// `core.deltaBaseCacheLimit` is 96 MiB and git forks a process per connection,
/// so that number is per *request*. This pool holds up to [`POOL_CAPACITY`]
/// handles **per repository**, so the same number here would be 6 GiB against
/// an RSS that is currently one of gunnar's better figures (97 MB on a push
/// where gitea took 391 MB). 4 MiB caps the pool at 256 MiB under full
/// concurrency and takes 2.74× of the available 3.43×.
///
/// The cap is a ceiling and not a reservation: [`crate::delta_cache::DeltaBaseCache`]
/// allocates as it fills, so an idle pool pays nothing.
///
/// # What it is actually worth end to end, which is much less
///
/// Stated here so nobody reads the 2.74× as a serve-path number. It is not.
/// Clone and fetch answer from the `.bitmap` and copy pack entries verbatim
/// (`P-001`), and neither resolves a chain, so **neither touches this cache at
/// all**. The one hot path that does is receive-pack's `connectivity` walk.
/// MEASURED the same day, one real 3 362-object push, interleaved A/B on the
/// same binary with this constant at 4 MiB and at 0:
///
/// * `connectivity` 13.73 ms → 10.50 ms, **1.31×**
/// * the whole push 219.5 ms → 216.1 ms, **1.016×**
///
/// So this is a correct-by-construction fix worth ~1.5% of a push today, and
/// the number to watch is what it will be worth to a repository whose reads do
/// not come out of a bitmap.
const DELTA_BASE_CACHE_BYTES: usize = 4 * 1024 * 1024;
/// How many times a lookup may re-take a handle before answering a miss.
///
/// **Not one, and the difference is a correctness bug rather than a tuning
/// knob.** The retry exists because the handle in hand can predate the pack
/// being asked about; the *replacement* handle can predate it too, if another
/// request's scan landed between this thread's pack hitting the disk and this
/// thread's retry taking a lease. One retry therefore covers one interleaving,
/// and a repository with ten writers on it produces more than one.
///
/// `gunnar.year` hit this once in a fifty-two week run — week 51, ten users,
/// 1153 packs — as the same `missing necessary objects … (peeling a tag)` the
/// slot table used to produce. The slot table was ruled out by the server's own
/// log: it had grown six times to 2216 slots, twice what was needed. **That the
/// single retry is what remained is inference, not measurement**; what IS
/// measured is that one retry cannot cover two interleavings, which
/// `a_miss_retries_until_it_has_seen_a_settled_generation` shows.
///
/// Four, because each attempt is only reached after a miss that has already
/// been proven to have raced, so the loop cannot spin on a genuinely absent
/// object: an absent object races nobody and leaves on the first pass.
const LOOKUP_ATTEMPTS: usize = 4;
/// How stale a directory mtime must be before it may be used as a memo.
///
/// A filesystem timestamp is not written from a fine-grained clock: Linux stamps
/// inodes from `ktime_get_coarse_real_ts64()`, which advances once per timer
/// tick, and NFS and older filesystems are coarser still. One second is the
/// classic worst case. So two changes inside one tick are indistinguishable, and
/// the memo below is only sound if the mtime it records was **already in the
/// past** when it was taken: only then must every later change move it.
///
/// This is git's own "racily clean" rule, applied to a directory instead of to
/// an index entry. The cost is that for one second after a pack lands, a miss
/// pays the full consolidation, which is exactly what it paid *always*, before
/// this module. The new path is therefore never slower than the old one, and is
/// two orders of magnitude faster as soon as the repository is quiet.
const RACY_WINDOW: Duration = Duration::from_secs(1);
/// What was known about `objects/pack` when the store last scanned it.
#[derive(Default, Clone, Copy)]
struct Watch {
/// The mtime read **immediately before** that scan.
///
/// Before, not after: the scan is then never older than the memo it
/// validates, so "the mtime has not moved since" implies "the scan saw
/// everything that is there now".
mtime: Option<SystemTime>,
/// Whether that mtime was old enough at the time to be relied on. See
/// [`RACY_WINDOW`].
trusted: bool,
}
/// A bounded pool of `gix-odb` handles over one repository, plus the watch that
/// decides when their snapshot may have gone stale.
pub struct OdbPool {
/// Swappable, because the slot table inside it cannot grow and the pack
/// directory can. See [`OdbPool::grow_slots_if_full`].
store: RwLock<Arc<gix_odb::Store>>,
/// `objects/`, kept so the store can be rebuilt with a bigger slot table.
objects_dir: PathBuf,
pack_dir: PathBuf,
hash_kind: gix_hash::Kind,
idle: Mutex<Vec<OdbHandle>>,
/// Bumped by every refresh. A lease taken before the bump is discarded on
/// return rather than pooled, because its snapshot predates the scan.
generation: AtomicU64,
watch: Mutex<Watch>,
/// How many times a miss has consulted the filesystem about its own
/// staleness — i.e. how many `statx` calls [`OdbPool::refresh_if_stale`]
/// has made.
///
/// This is the *observable* for [`ObjectStore::contains_without_rescan`]
/// ([`OdbPool::lookup_without_rescan`]), and it is a counter rather than a
/// test double because
/// nothing else can see the difference. The whole intervention is defined
/// to leave every answer, every wire byte and every call count unchanged;
/// what it changes is a syscall, so a syscall is what has to be counted.
/// Exposed as [`OdbPool::rescan_counts`].
rescan_probes: AtomicU64,
/// Of those, how many found the directory moved and paid a full
/// consolidation. The expensive half; the probe above is the syscall.
rescans: AtomicU64,
/// How many times the slot table has been grown, and how many slots it has
/// now. Reported by [`OdbPool::slot_table`] so the condition is
/// observable from a test and from a bench arm, rather than only inferable
/// from objects going missing.
regrows: AtomicU64,
slots: AtomicU64,
}
impl std::fmt::Debug for OdbPool {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("OdbPool")
.field("pack_dir", &self.pack_dir)
.field("generation", &self.generation.load(Ordering::Relaxed))
.field("slots", &self.slots.load(Ordering::Relaxed))
.field("regrows", &self.regrows.load(Ordering::Relaxed))
.finish()
}
}
impl OdbPool {
/// Wrap `store`, and scan once so the first lookup is neither wrong nor slow.
///
/// The eager scan matters: a handle built from an uninitialised store has an
/// empty index list, so without it the very first `contains` would report a
/// present object absent, be corrected by the staleness path, and only then
/// answer. Still *correct*, the retry sees to that, but it would put a
/// directory scan in front of the first request of every repository.
pub fn new(
store: Arc<gix_odb::Store>,
git_dir: &Path,
hash_kind: gix_hash::Kind,
) -> Self {
let slots = store.metrics().unused_slots as u64;
let pool = OdbPool {
store: RwLock::new(store),
objects_dir: git_dir.join("objects"),
pack_dir: git_dir.join("objects").join("pack"),
hash_kind,
idle: Mutex::new(Vec::new()),
generation: AtomicU64::new(0),
watch: Mutex::new(Watch::default()),
rescan_probes: AtomicU64::new(0),
rescans: AtomicU64::new(0),
regrows: AtomicU64::new(0),
slots: AtomicU64::new(slots),
};
pool.refresh();
pool
}
/// The underlying `gix-odb` store, for the one caller that needs a handle
/// of its own rather than a lease: `P-001`'s pack-copy source.
///
/// A lease will not do there. The pack generator holds its handle across a
/// whole clone and clones it per worker, and `entry_by_location` hands out
/// slices pointing into a mapped pack, so that handle needs
/// `prevent_pack_unload()`, which a pooled handle deliberately does not
/// have (it would pin every pack a repack replaced for as long as the
/// handle sat idle in the pool).
pub fn store(&self) -> Arc<gix_odb::Store> {
Arc::clone(&self.store.read().unwrap_or_else(poisoned))
}
/// `(slots, times grown)`. See [`OdbPool::grow_slots_if_full`].
pub fn slot_table(&self) -> (u64, u64) {
(
self.slots.load(Ordering::Relaxed),
self.regrows.load(Ordering::Relaxed),
)
}
/// How many **pack indices** the store's snapshot holds.
///
/// This is the `M` in `P-012`'s `O(M)`: the number of separately-searched
/// index structures a miss walks before it can say no. It equals the number
/// of `.idx` files — **unless** a `multi-pack-index` is present, in which
/// case `gix-odb` folds every index it names into a single entry and this
/// drops to one (`load_index.rs:267`, gated on `use_multi_pack_index`).
///
/// **A count, not a clock** (§6), and the only thing that distinguishes the
/// two arms: they answer identically by construction, so nothing else can
/// tell a multi-pack index that is being routed through from one that was
/// written and ignored. See [`crate::midx`].
/// `known_reachable_indices` alone, and **not** plus
/// `open_reachable_indices`: gix counts an index that is mapped in both
/// places (`metrics.rs:20-42`), so adding them double-counts every loaded
/// index. Caught by the test asserting this equals the `.idx` count.
pub fn index_slots(&self) -> usize {
self.store().metrics().known_reachable_indices
}
/// Run `f` against **one** pooled handle, with no retry.
///
/// For an operation that is many reads rather than one lookup — a closure
/// walk, or handing a base source to the pack indexer. The retry loop in
/// [`OdbPool::lookup`] is keyed on a single answer being a miss, which is
/// not a question a walk can be asked; and a walk wants one handle for its
/// whole length anyway, because that is what makes the delta base cache
/// worth having. A pack landing mid-walk is covered by the caller's own
/// [`OdbPool::refresh`] on the write path, not here.
pub fn with_handle<T>(&self, f: impl FnOnce(&OdbHandle) -> T) -> T {
let lease = self.lease();
f(lease.handle())
}
/// Borrow a handle for one operation.
fn lease(&self) -> Lease<'_> {
let generation = self.generation.load(Ordering::Acquire);
let handle = self
.idle
.lock()
.unwrap_or_else(poisoned)
.pop()
.unwrap_or_else(|| {
let mut handle = gix_odb::Cache::from(self.store().to_handle_arc());
// The whole point: a miss must not scan the directory. See the
// module docs for what that was costing.
handle.refresh_never();
handle.set_pack_cache(|| {
Box::new(crate::delta_cache::DeltaBaseCache::new(
DELTA_BASE_CACHE_BYTES,
))
});
handle
});
Lease {
pool: self,
handle: Some(handle),
generation,
}
}
/// Run `op` against a pooled handle, retrying once if it missed and the
/// pack directory may have changed since those handles were built.
///
/// `hit` decides what counts as an answer. For a fallible lookup an **error
/// is deliberately not a hit**: `gix-odb` reports an unresolvable ref-delta
/// base as `DeltaBaseMissing` rather than as `Ok(None)`, and a base that
/// landed in a pack written after the snapshot is exactly the case the
/// retry exists for.
///
/// # Two ways a handle can predate the pack, not one
///
/// The `mtime` watch answers "has a pack appeared since the last scan?".
/// That is the whole question on a quiet store and only half of it on a busy
/// one, and the missing half **reported a present object missing on a live
/// server** (found by `gunnar.endurance`, 2026-08-04: a `git push` of a tag
/// was refused with `missing necessary objects … is not in this repository`
/// while the object sat in a pack on disk, and the identical push succeeded
/// against a restarted server with nothing else changed):
///
/// ```text
/// thread A thread B
/// lease() → generation G, pops handle H
/// (H's snapshot predates the new pack)
/// refresh(): bump to G+1,
/// drain the idle pool, and
/// record the NEW mtime
/// op(H) → miss
/// refresh_if_stale(): the watch is fresh
/// (B just wrote it) → "nothing has
/// changed" → no retry, miss returned
/// ```
///
/// A's handle came from before B's scan, so B's refresh is precisely what
/// makes A's answer wrong — and it is also what makes A's staleness check
/// say everything is fine. The mtime cannot see this: it is not a question
/// about the directory, it is a question about **this handle**.
///
/// So a miss retries when *either* the directory moved *or* the pool's
/// generation moved under this lookup. The generation is already maintained
/// for exactly this reason on the return path ([`Lease::drop`] refuses to
/// pool a handle whose snapshot predates a scan); this reads the same
/// counter on the way out instead of only on the way back.
///
/// Cost in the common case is one relaxed-ish atomic load on a path that
/// has already missed.
pub fn lookup<T>(&self, op: impl FnMut(&OdbHandle) -> T, hit: impl Fn(&T) -> bool) -> T {
self.lookup_with(OnMiss::AskTheDisk, op, hit)
}
/// [`OdbPool::lookup`], for a caller whose miss **is the answer**.
///
/// Same retry loop, one term removed: a miss does not go and ask the
/// filesystem whether a pack has appeared. See
/// [`GitOps::has`](git_storage_trait::GitOps::has) — a client's `have` line —
/// is the caller this exists for, and a stale `false` is safe there.
///
/// **The generation check stays.** It is an atomic load, not a syscall, and
/// what it catches is not staleness-in-general but *this handle* having been
/// leased from before another request's scan — the interleaving
/// `a_miss_retries_when_another_request_refreshed_underneath_it` pins. It is
/// free and it makes the answer strictly better, so there is no case for
/// dropping it along with the syscall.
pub fn lookup_without_rescan<T>(
&self,
op: impl FnMut(&OdbHandle) -> T,
hit: impl Fn(&T) -> bool,
) -> T {
self.lookup_with(OnMiss::Answer, op, hit)
}
fn lookup_with<T>(
&self,
on_miss: OnMiss,
mut op: impl FnMut(&OdbHandle) -> T,
hit: impl Fn(&T) -> bool,
) -> T {
let mut answer = None;
for _ in 0..LOOKUP_ATTEMPTS {
let (result, generation) = {
let lease = self.lease();
(op(lease.handle()), lease.generation)
};
if hit(&result) {
return result;
}
// Order matters: ask about the handle first, because
// `refresh_if_stale` has the side effect of making the watch look
// current. `&&` short-circuits left to right, so `OnMiss::Answer`
// reaches no syscall at all — which is the whole change.
let raced = generation != self.generation.load(Ordering::Acquire);
let stale = !raced && on_miss == OnMiss::AskTheDisk && self.refresh_if_stale();
answer = Some(result);
if !raced && !stale {
break;
}
}
answer.expect("LOOKUP_ATTEMPTS is not zero, so the loop ran at least once")
}
/// `(staleness probes, of those, full consolidations)`. See
/// [`OdbPool::rescan_probes`]'s field docs.
pub fn rescan_counts(&self) -> (u64, u64) {
(
self.rescan_probes.load(Ordering::Relaxed),
self.rescans.load(Ordering::Relaxed),
)
}
/// Consolidate from disk if `objects/pack` may have changed since the last
/// scan; report whether it did.
///
/// The fast answer is one `statx`. It is only taken when the recorded mtime
/// is [trusted](RACY_WINDOW), so "unchanged" here really does mean "no pack
/// has appeared" and a miss may be reported as a miss.
fn refresh_if_stale(&self) -> bool {
self.rescan_probes.fetch_add(1, Ordering::Relaxed);
let watch = *self.lock_watch();
if watch.trusted && watch.mtime == mtime_of(&self.pack_dir) {
return false;
}
self.rescans.fetch_add(1, Ordering::Relaxed);
self.refresh();
true
}
/// Re-scan `objects/` from disk and invalidate every pooled handle.
///
/// This is `reprepare_packed_git()`. It is public through
/// [`OdbPool::refresh`] for the one case the automatic watch
/// cannot cover, which is another program writing into the repository. A
/// stock `git` binary run by an operator, say. gunnar's own writers are covered by the
/// watch, and by the exclusive store lock that stops a second gunnar process
/// existing at all.
pub fn refresh(&self) {
// Read the mtime BEFORE the scan; see `Watch::mtime`.
let mtime = mtime_of(&self.pack_dir);
let trusted = mtime.is_some_and(|m| {
SystemTime::now()
.duration_since(m)
.is_ok_and(|age| age > RACY_WINDOW)
});
// The slot table BEFORE the scan, so a directory that has outgrown it is
// enlarged rather than scanned into an error nobody can see. See
// `grow_slots_if_full`.
self.grow_slots_if_full();
// `gix-odb` exposes no "rescan now"; the consolidation is reachable only
// by making a handle that still has the default refresh mode fall
// through every index. The null oid is the id no object can have (the
// hash of no content is not zero), so this asks for a guaranteed miss
// and takes the scan as its side effect.
let prober = self.store().to_handle_arc();
let null = gix_hash::ObjectId::null(self.hash_kind);
let _ = gix_object::Exists::exists(&prober, null.as_ref());
drop(prober);
// …and again afterwards. The scan is what fills the slots, so a
// repository that arrived at exactly-full only reveals it here, and
// leaving it full means the NEXT pack is the one that vanishes.
self.grow_slots_if_full();
// Bump first, so a lease in flight is discarded on return rather than
// pooled with a pre-scan snapshot.
self.generation.fetch_add(1, Ordering::AcqRel);
self.idle.lock().unwrap_or_else(poisoned).clear();
*self.lock_watch() = Watch { mtime, trusted };
}
/// **Rebuild the store with a bigger slot table when the current one is
/// full.** This is the fix for the defect `gunnar.endurance` found.
///
/// # What goes wrong without it
///
/// `gix_odb::Store::at_opts` sizes its slot map **once, at open**, from
/// `Slots::AsNeededByDiskState { multiplier: 1.1, minimum: 32 }`, and the
/// map never grows. `gunnar repo create` leaves a repository with **zero**
/// packs, so `gunnar serve` opens it at `max(0 × 1.1, 32)` = **32 slots**,
/// and every push adds a pack. Nothing removes one while the server is up:
/// `gunnar gc` cannot run against a live server, and the multi-pack index
/// that would fold them into a single slot is written by
/// `FsStore::open_or_init` and by nothing else.
///
/// On the push that makes the **33rd** pack, `consolidate_with_disk_state`
/// returns `Error::InsufficientSlots { current: 32, needed: 1 }` — and
/// **two layers throw it away**. `gix_pack::Find::contains` ends in
/// `Err(_) => return false` (gix-odb `find.rs`, carrying its own `TODO:
/// Maybe that should change?`), and [`OdbPool::refresh`] reaches the
/// consolidation as `let _ = Exists::exists(…)`. Nothing is logged, nothing
/// fails, and the store's snapshot is **frozen at 32 packs for the life of
/// the process**. Every object in the 33rd pack and beyond is reported
/// absent by a store that is looking straight at it.
///
/// The symptom is a `git push` refused with
///
/// ```text
/// refs/tags/end-v7: missing necessary objects:
/// object <oid> is not in this repository (peeling a tag)
/// ```
///
/// for an object `git cat-file -t` finds in a pack on the server's own
/// disk. It reproduced on cycle 7 with 33 packs in three of three runs at
/// load averages 21, 35 and 50 — the same cycle at every load, because it
/// is a **count** and not a race — and a restart cured it, because
/// re-opening re-sizes the table from disk *and* writes a multi-pack index
/// that collapses 33 indices into one slot. That second effect is why the
/// "a restarted server took 47 packs one tag at a time" experiment could
/// not have failed: it had 35 free slots and needed 14.
///
/// # Why growing, and not a bigger constant
///
/// A constant only moves the cliff. A long-lived server accumulates packs
/// for as long as it is up, so any fixed table is eventually the same bug
/// with a larger number in it. Doubling is the shape that has no cliff.
///
/// # Why `unused_slots == 0` and not the error
///
/// The error is not reachable: neither `Exists::exists` nor
/// `gix_pack::Find::contains` returns one, by design of those traits.
/// `Store::metrics().unused_slots` is public, is exactly the quantity that
/// decides whether the next index can be placed, and gix's own
/// documentation on `metrics()` says to use it "to decide whether a new
/// instance should be created". Acting at zero rather than at the failure
/// means the pack that would have vanished never gets the chance to.
///
/// Zero free slots is the *only* trigger, so a repository that never fills
/// its table never pays for this — and one that does pays a re-open, which
/// is what a restart was costing an operator anyway.
fn grow_slots_if_full(&self) {
{
let current = self.store();
if current.metrics().unused_slots > 0 {
return;
}
}
let mut store = self.store.write().unwrap_or_else(poisoned);
// Re-checked under the lock: two refreshes can race here, and the
// second must not rebuild a table the first has already grown.
if store.metrics().unused_slots > 0 {
return;
}
let current = self.slots.load(Ordering::Relaxed).max(1);
// Twice the table, and at least twice what is on disk — a directory
// that grew by more than the table while nobody was looking is then
// still covered in one rebuild rather than in a series of them.
let want = (current * 2).max(count_packs(&self.pack_dir) as u64 * 2 + 2);
let want = want.min(MAX_SLOTS);
if want <= current {
// Already at gix-odb's ceiling of 2^15-1 indices. Nothing can be
// done here; say so, loudly, because the alternative is objects
// quietly going missing.
tracing::error!(
slots = current,
packs = count_packs(&self.pack_dir),
"the object database's slot table is full and cannot be grown further; objects \
in packs beyond it will be reported absent. Repack this repository."
);
return;
}
match gix_odb::Store::at_opts(
self.objects_dir.clone(),
self.hash_kind,
&mut std::iter::empty(),
gix_odb::store::init::Options {
slots: gix_odb::store::init::Slots::Given(want as u16),
..Default::default()
},
) {
Ok(bigger) => {
tracing::info!(
from = current,
to = want,
packs = count_packs(&self.pack_dir),
"grew the object database's slot table"
);
*store = Arc::new(bigger);
self.slots.store(want, Ordering::Relaxed);
self.regrows.fetch_add(1, Ordering::Relaxed);
// Every pooled handle belongs to the OLD store. `refresh` will
// clear the pool a moment later on its own path, but this is
// also called from `new`, so it is done here too rather than
// relying on the caller.
self.idle.lock().unwrap_or_else(poisoned).clear();
}
Err(err) => {
tracing::error!(
error = %err,
slots = current,
want,
"could not grow the object database's slot table"
);
}
}
}
fn lock_watch(&self) -> MutexGuard<'_, Watch> {
self.watch.lock().unwrap_or_else(poisoned)
}
}
/// `gix-odb` refuses more than `2^15 - 1` slots (`PackId::max_indices()`), and
/// `Slots::Given` takes a `u16`, so this is the ceiling on both counts.
const MAX_SLOTS: u64 = (1 << 15) - 1;
/// What a lookup should do when it falls through every loaded index.
///
/// The two variants are the two call sites `P-012` conflated, and they have
/// opposite correct answers. See
/// See [`OdbPool::lookup_without_rescan`].
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum OnMiss {
/// Treat the miss as evidence this store has not looked at the disk lately,
/// and check before answering. A `want`, a delta base, an id the closure
/// walk already found: a stale `false` for any of those is a refused fetch.
AskTheDisk,
/// Return the miss. A client's `have` that the server does not hold is not
/// a stale snapshot, it is the client telling you about its own commits.
Answer,
}
/// A handle borrowed from the pool for the length of one operation.
struct Lease<'a> {
pool: &'a OdbPool,
/// `Some` for the whole lifetime; the `Option` exists so `Drop` can move it.
handle: Option<OdbHandle>,
generation: u64,
}
impl Lease<'_> {
fn handle(&self) -> &OdbHandle {
self.handle
.as_ref()
.expect("a lease holds its handle until it is dropped")
}
}
impl Drop for Lease<'_> {
fn drop(&mut self) {
let Some(handle) = self.handle.take() else {
return;
};
if self.generation != self.pool.generation.load(Ordering::Acquire) {
// A refresh happened while this was out; its snapshot is older than
// the scan, so it must not go back into the pool.
return;
}
let mut idle = self.pool.idle.lock().unwrap_or_else(poisoned);
if idle.len() < POOL_CAPACITY {
idle.push(handle);
}
}
}
/// A poisoned pool mutex only means some other lookup panicked mid-flight; the
/// data behind it is a plain `Vec` of handles and is not left inconsistent.
fn poisoned<T>(err: std::sync::PoisonError<T>) -> T {
err.into_inner()
}
fn mtime_of(dir: &Path) -> Option<SystemTime> {
std::fs::metadata(dir).and_then(|m| m.modified()).ok()
}
/// How many `.pack` files are in `dir`.
pub fn count_packs(dir: &Path) -> usize {
match std::fs::read_dir(dir) {
Ok(entries) => entries
.filter_map(std::result::Result::ok)
.filter(|e| e.path().extension().is_some_and(|ext| ext == "pack"))
.count(),
Err(_) => 0,
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::AtomicUsize;
/// A pool over an empty object directory. Enough for the question below,
/// which is about the retry *decision* and not about what is found.
fn pool(dir: &Path) -> OdbPool {
std::fs::create_dir_all(dir.join("objects/pack")).expect("objects/pack");
let store = gix_odb::Store::at_opts(
dir.join("objects"),
gix_hash::Kind::Sha1,
&mut std::iter::empty(),
gix_odb::store::init::Options::default(),
)
.expect("open the store");
OdbPool::new(Arc::new(store), dir, gix_hash::Kind::Sha1)
}
/// A miss on a quiet store must NOT retry. This is the half that was
/// already right, and it is here so the test below is a comparison rather
/// than an assertion about a number nobody has seen move.
#[test]
fn a_miss_on_an_unchanged_pack_directory_does_not_retry() {
let dir = tempfile::tempdir().expect("tempdir");
let pool = pool(dir.path());
// The watch has to be trusted for the fast "nothing changed" answer to
// be reachable at all; it only becomes so once the recorded mtime is
// older than RACY_WINDOW.
std::thread::sleep(RACY_WINDOW + Duration::from_millis(50));
pool.refresh();
let calls = AtomicUsize::new(0);
let _ = pool.lookup(
|_| {
calls.fetch_add(1, Ordering::Relaxed);
false
},
|found: &bool| *found,
);
assert_eq!(
calls.load(Ordering::Relaxed),
1,
"a miss with nothing changed must be answered once, not re-run"
);
}
/// **Two refreshes, which is what more than one writer produces.**
///
/// The retry above covers a handle that predates one scan. It cannot cover
/// a handle that predates a scan whose *replacement* predates the next one,
/// and that is not a hypothetical on a repository with ten people pushing
/// to it: every push refreshes.
///
/// **Seen RED** with the single-shot retry: this asserts `3 == 2`, and the
/// second attempt's miss is returned as the answer. On a live server that
/// is a push refused for an object that is sitting in a pack on disk.
#[test]
fn a_miss_retries_until_it_has_seen_a_settled_generation() {
let dir = tempfile::tempdir().expect("tempdir");
let pool = pool(dir.path());
std::thread::sleep(RACY_WINDOW + Duration::from_millis(50));
pool.refresh();
let calls = AtomicUsize::new(0);
let _ = pool.lookup(
|_| {
// Another request's refresh on each of the first two attempts,
// landing while this lookup is between its handle and its
// staleness check.
if calls.fetch_add(1, Ordering::Relaxed) < 2 {
pool.refresh();
}
false
},
|found: &bool| *found,
);
assert_eq!(
calls.load(Ordering::Relaxed),
3,
"a lookup must keep re-taking a handle while the pool keeps moving under it. One \
retry covers one interleaving; a repository with several writers produces more \
than one, and the miss that comes back is an object the server is holding."
);
}
/// **The race, made deterministic.** A refresh by another request while
/// this lookup is holding a handle from before that scan.
///
/// Reproduced without threads by refreshing from inside `op` itself, which
/// is exactly the interleaving: the handle in hand predates the scan, and
/// the watch the staleness check consults has just been written by that
/// same scan and therefore says "nothing has changed".
///
/// **Seen RED:** with the retry keyed on the mtime watch alone this asserts
/// `2 == 1` — one call, the stale handle's miss returned as the answer. On
/// a live server that is a `git push` refused with "missing necessary
/// objects" for an object sitting in a pack on disk, which is how
/// `gunnar.endurance` found it.
#[test]
fn a_miss_retries_when_another_request_refreshed_underneath_it() {
let dir = tempfile::tempdir().expect("tempdir");
let pool = pool(dir.path());
std::thread::sleep(RACY_WINDOW + Duration::from_millis(50));
pool.refresh();
let calls = AtomicUsize::new(0);
let _ = pool.lookup(
|_| {
if calls.fetch_add(1, Ordering::Relaxed) == 0 {
// Another request's refresh, landing while this lookup is
// between its handle and its staleness check.
pool.refresh();
}
false
},
|found: &bool| *found,
);
assert_eq!(
calls.load(Ordering::Relaxed),
2,
"a handle that predates another request's scan must be re-tried; the mtime watch \
cannot see this, because the very scan that invalidated the handle is what makes \
the watch look current"
);
}
}