plugins/native/znippy-plugin-git/src/gc.rs
//! Garbage collection, as a trait with two implementations.
//!
//! Both reclaim the same thing: the payload of entries nothing points at any
//! more. `supersede_as_delta` replaces a generation's bytes with a delta and
//! drops its index rows, but the old *blob* stays in the file, unreferenced —
//! measured in znippy-common, an 8 056 624-byte archive went to 8 057 731 after
//! 4 000 000 bytes of live payload became 11. Reclaiming that is what a GC is
//! here for.
//!
//! ## Neither implementation copies a git object through a codec
//!
//! Both route the copy through
//! [`znippy_common::compact_archive`](znippy_common::compact_archive), which
//! copies every live chunk's on-disk bytes **verbatim**: no decode, no
//! re-encode, no delta recomputation, `compressed` / `uncompressed_size` /
//! blake3 carried unchanged, and a delta chunk staying a delta chunk at the same
//! depth. Nothing in this module reimplements that loop (LAW 5); the two
//! implementations differ only in *where the result ends up*.
//!
//! ## The two, and the one real difference between them
//!
//! | | [`CompactInPlace`] (A) | [`NewGeneration`] (B, the default) |
//! |---|---|---|
//! | result | the same path, rewritten | a **new** file, `x.znippy` → `x.g1.znippy` |
//! | commit point | `rename(staged, archive)` | `rename(work, x.g1.znippy)`, then `unlink(x.znippy)` |
//! | verified before committing? | **no** — the rename has already happened | **yes** — the new generation is read back in full first |
//! | interrupted at any byte | the original is still there, dead payload and all | the original is still there, and so is the debris |
//!
//! ### Crash safety, by construction, in both
//!
//! Nothing is ever written over live data. A: `compact_archive` stages a
//! `*.compact-<pid>-<nanos>` file *beside* the destination and the only mutation
//! of the archive's own name is one `rename(2)`, which is atomic — so an
//! interruption at any byte leaves the original serving and a stray staged file
//! behind. B: the compaction runs against a **hard link** to the original inode,
//! so the original *name* is untouched for the whole run; the new generation is
//! then verified, renamed into place, and only after that is the old name
//! unlinked. There is no window in either where a reader finds no archive.
//!
//! B's extra guarantee is the one A structurally cannot give: A commits by
//! rename before anything has read the result back, so if the compacted file
//! were bad, the good one is already gone. B verifies first and deletes last.
//! That is why B is the default.
//!
//! ## The third ending, not yet written
//!
//! B's last step deletes the superseded generation. [`retire_to_holger`] is the
//! alternative: ship it to nordisk's artifact server instead, because a sealed
//! archive is already the shape an artifact store wants. **That function is
//! deliberately empty** — signature and contract only. Read its doc comment for
//! why the shape works; do not add a client to this file.
//!
//! ## What "verified" means here
//!
//! Every entry is **read back through the ordinary reader** with
//! `ZnippyArchive::extract_file_verified` — delta chains reconstructed, blake3
//! checked against the index — plus a set comparison of the manifest
//! (`relative_path` and `uncompressed_size`) against the original, because an
//! archive that verifies against its own index but has lost an entry is
//! internally consistent and still wrong.
//!
//! It is deliberately **not** [`znippy_common::verify_archive_integrity`], and
//! that is a finding rather than a preference: on the fixture in
//! [`tests::both_implementations_reclaim_the_dead_payload_and_change_no_entry`]
//! — four generations, three of them superseded into deltas — that function
//! reports "3 corrupt entries of 4" on a perfectly good archive, because its
//! `decompress_archive` path does not reconstruct delta chunks. It cannot be
//! used to gate a GC of any archive `supersede_as_delta` has touched, which is
//! every archive a GC is worth running on.
use std::path::{Path, PathBuf};
use anyhow::{Context, Result, anyhow, bail};
use znippy_common::{
ArtifactMeta, CompactReport, ZnippyArchive, compact_archive, get_all_files_meta,
};
/// What one GC run did. Owned by the `git-storage-trait` contract; re-exported
/// here so `crate::gc::GcReport` stays a valid path. Every field is measured
/// off the filesystem or off the archive after the fact — see the trait crate
/// for per-field docs, and note `retired_packs` is always `0` from a bare
/// [`Gc::run`]: `GitOps::gc` fills it in.
pub use git_storage_trait::GcReport;
/// Reclaim the dead payload in a znippy archive.
pub trait Gc {
fn run(&self, archive: &Path) -> Result<GcReport>;
fn name(&self) -> &'static str;
}
// ── the third ending: retire the old generation to holger ─────────────────────
/// Where a pensioned generation goes. Deliberately just an address and a name:
/// what the endpoint means is the transport's business, and no transport is
/// decided.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct HolgerTarget {
/// Where to reach the holger instance. Opaque to this crate.
pub endpoint: String,
/// The repository the generation belongs to, so the artifact is filed under
/// something meaningful at the destination.
pub repository: String,
}
/// What a completed retirement reports. Every field is about the archive that
/// moved; nothing here describes an index, because a sealed archive carries its
/// own and no index is shipped separately.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RetirementReceipt {
/// The name the generation was filed under at the destination.
pub artifact: String,
pub bytes_shipped: u64,
/// The archive's digest, as computed locally and as confirmed by holger.
/// They are the same value or the retirement failed.
pub digest_hex: String,
/// Holger read the archive back and it verified. Never `false` on success —
/// the local copy is not removed until this is true.
pub verified_at_destination: bool,
/// Whether the local file was unlinked afterwards.
pub local_copy_removed: bool,
}
/// **Retire a pensioned generation to holger instead of deleting it. NOT
/// IMPLEMENTED — deliberately.**
///
/// This is the alternative ending to [`NewGeneration`]. That implementation
/// seals a new generation, proves it, and then unlinks the old one (step 5). The
/// old one is not worthless: it is a complete, sealed, self-describing archive
/// of a real state of a real repository. Shipping it to holger — nordisk's
/// artifact/package server — keeps it as an artifact rather than destroying it,
/// and the local disk is freed either way.
///
/// # What makes this viable, which is the part worth recording now
///
/// * **A sealed archive never changes.** That is the whole property. Retiring
/// one is therefore a **copy and a delete**, not a migration: there is no
/// schema to translate, no writer to quiesce, no reader to redirect
/// mid-flight, and no state at the destination that can go stale relative to
/// the source. An artifact store wants exactly this shape — immutable,
/// self-contained, addressable by digest — and a sealed `.znippy` already is
/// it.
/// * **It can be verified at the destination before the local copy goes.** The
/// archive carries its own index and a blake3 per chunk, so holger can read
/// back what it received and prove it, with no help from here. The ordering is
/// the same one [`NewGeneration`] already uses and for the same reason: ship,
/// verify, *then* delete. There is never a moment when the only copy is the
/// one in flight.
/// * **It composes with GC rather than replacing it.** The generation being
/// retired has already been superseded by a proven newer one, so nothing is
/// serving from it. Retirement is not on any read path.
///
/// # What is deliberately NOT here
///
/// No client, no connection, no holger dependency, no digest algorithm choice,
/// no naming scheme at the destination. Those are the decisions this stub is
/// holding a place for. [`tests::retiring_to_holger_is_deliberately_empty`]
/// asserts the emptiness so a half-implementation cannot pass for a finished
/// one.
///
/// # Panics
///
/// Always.
pub fn retire_to_holger(_generation: &Path, _target: &HolgerTarget) -> Result<RetirementReceipt> {
todo!(
"retire a pensioned generation to holger: deliberately empty. The contract is settled \
(ship the sealed archive, have holger verify it, only then unlink the local copy); \
the transport, the digest and the destination naming are not. Do not add a holger \
client from this file without deciding them first."
)
}
// ── Impl A — CompactInPlace ───────────────────────────────────────────────────
/// **Delegate to znippy's own compaction, in place.**
///
/// A thin wrapper over [`znippy_common::compact_archive`] and deliberately
/// nothing more: that function is the one that knows how to copy a live chunk
/// without touching its codec, and a second copy of that loop in this crate
/// would be a second thing to keep correct.
///
/// The archive keeps its name. An interruption leaves the original serving,
/// which is `compact_archive`'s own contract (stage beside, one atomic rename),
/// asserted in `tests/gc_crash.rs` by killing a real process mid-copy.
///
/// It cannot verify before committing: by the time there is a compacted file to
/// read, the rename has already replaced the original. The post-hoc check this
/// runs is therefore a report, not a gate — use [`NewGeneration`] when the
/// difference matters, which is normally.
#[derive(Debug, Clone, Copy)]
pub struct CompactInPlace {
/// Read the result back and blake3-check it after the rename. On by default;
/// it is a full pass over the archive, so a caller compacting a very large
/// archive on a timer may turn it off.
pub verify: bool,
}
impl Default for CompactInPlace {
/// `verify: true`. Written out rather than derived, because a derived
/// `Default` would be `false` — the safe-looking spelling giving the less
/// safe object.
fn default() -> Self {
Self { verify: true }
}
}
impl CompactInPlace {
/// With the post-hoc verification on.
pub fn new() -> Self {
Self::default()
}
}
impl Gc for CompactInPlace {
fn run(&self, archive: &Path) -> Result<GcReport> {
let CompactReport {
bytes_before,
bytes_after,
rows,
delta_rows,
} = compact_archive(archive)
.with_context(|| format!("compacting {} in place", archive.display()))?;
let verified = if self.verify {
read_back_every_entry(archive).with_context(|| {
format!(
"verifying {} after an in-place compaction — the original is already gone, \
which is exactly the window NewGeneration closes",
archive.display()
)
})?;
true
} else {
false
};
Ok(GcReport {
strategy: self.name(),
archive: archive.to_path_buf(),
retired: None,
bytes_before,
bytes_after,
rows,
delta_rows,
verified,
retired_packs: 0,
})
}
fn name(&self) -> &'static str {
"CompactInPlace"
}
}
// ── Impl B — NewGeneration ────────────────────────────────────────────────────
/// Where a [`NewGeneration`] run may be stopped, for the crash-safety guards.
///
/// Stopping returns an `Err` and leaves the filesystem in **exactly** the state a
/// process death at that point would leave it — nothing is unwound. That is the
/// point: the test then asserts what a reader finds.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum StopAfter {
#[default]
Never,
/// The work copy has been compacted; nothing has been verified or renamed.
Compact,
/// The new generation has been verified but not yet renamed into place.
Verify,
/// The new generation is in place under its own name; the old archive has
/// **not** been unlinked. Both are on disk and both are readable.
Rename,
}
/// **GC into a new generation, keeping the old one until the new one is proven.**
///
/// `repo.znippy` becomes `repo.g1.znippy`, then `repo.g2.znippy`, and so on. The
/// old file is unlinked only after the new one has been read back in full.
///
/// The sequence, and why each step is where it is:
///
/// 1. **`link(archive, work)`** — a hard link, so the work name and the archive
/// name are the same inode and not one byte is copied. This is what lets step
/// 2 be `compact_archive` verbatim (LAW 5) while the archive's own name stays
/// untouched: `compact_archive` renames over the name it was given, and the
/// name it is given here is the work link, not the archive.
/// 2. **`compact_archive(work)`** — stages beside `work` and renames over it.
/// The archive's inode is still fully referenced by the archive's own name
/// throughout, so every reader of `repo.znippy` is unaffected.
/// 3. **verify** — [`verify_archive_integrity`] over the new file, plus a
/// manifest set comparison against the original. Both are read back off disk.
/// 4. **`rename(work, repo.gN.znippy)`** — atomic; the new generation now has
/// its permanent name.
/// 5. **`unlink(repo.znippy)`** — the old index and old data go, and only now.
///
/// A death at any point leaves at least one complete, readable archive:
/// before 4 the original under its own name (plus debris), after 4 and before 5
/// both of them, after 5 the new generation. Asserted, per step, in
/// [`tests::an_interruption_at_every_step_leaves_a_readable_archive`].
#[derive(Debug, Clone, Copy, Default)]
pub struct NewGeneration {
/// Test-only interruption point. [`StopAfter::Never`] in every real run.
pub stop_after: StopAfter,
}
impl NewGeneration {
pub fn new() -> Self {
Self::default()
}
/// Stop the run at `stop`, leaving the filesystem as a crash there would.
pub fn stopping_after(stop: StopAfter) -> Self {
Self { stop_after: stop }
}
}
/// The default GC. B, because it is the one that verifies before it deletes.
pub fn default_gc() -> NewGeneration {
NewGeneration::new()
}
/// `repo.znippy` → `repo.g1.znippy` → `repo.g2.znippy` → …
///
/// The generation lives in the stem rather than in the extension so the file is
/// still a `.znippy` to everything that dispatches on extension. A stem that
/// already ends in `.gN` is advanced; anything else starts at `g1`.
pub fn next_generation(archive: &Path) -> Result<PathBuf> {
let name = archive
.file_name()
.and_then(|n| n.to_str())
.ok_or_else(|| anyhow!("{} has no usable file name", archive.display()))?;
let (stem, ext) = match name.rsplit_once('.') {
Some((s, e)) if !s.is_empty() => (s, Some(e)),
_ => (name, None),
};
let next = match stem.rsplit_once('.') {
Some((head, marker)) if is_generation(marker) => {
format!("{head}.g{}", marker[1..].parse::<u64>().unwrap_or(0) + 1)
}
_ => format!("{stem}.g1"),
};
let file = match ext {
Some(e) => format!("{next}.{e}"),
None => next,
};
Ok(archive.with_file_name(file))
}
fn is_generation(s: &str) -> bool {
s.len() > 1 && s.starts_with('g') && s[1..].bytes().all(|b| b.is_ascii_digit())
}
/// Read every entry back through the ordinary reader, reconstructing delta
/// chains and checking each entry's blake3 against the index. Returns the bytes
/// that came out.
///
/// This is the expensive half of a GC and it is the half that makes the cheap
/// half safe: it is one full decode pass over the archive.
pub(crate) fn read_back_every_entry(archive: &Path) -> Result<u64> {
let manifest = manifest(archive)?;
let ar = ZnippyArchive::open(archive)
.with_context(|| format!("opening {} to verify it", archive.display()))?;
let mut bytes = 0u64;
for (path, expected) in &manifest {
let got = ar.extract_file_verified(path).with_context(|| {
format!(
"{} does not read back {path} — the new generation is not usable",
archive.display()
)
})?;
if got.len() as u64 != *expected {
bail!(
"{}: {path} reads back {} bytes, the index says {expected}",
archive.display(),
got.len()
);
}
bytes += got.len() as u64;
}
Ok(bytes)
}
/// The manifest as a comparable set: path and uncompressed size, sorted.
fn manifest(archive: &Path) -> Result<Vec<(String, u64)>> {
let mut m: Vec<(String, u64)> = get_all_files_meta(archive)
.with_context(|| format!("listing {}", archive.display()))?
.into_iter()
.map(
|ArtifactMeta {
relative_path,
uncompressed_size,
..
}| { (relative_path, uncompressed_size) },
)
.collect();
m.sort();
Ok(m)
}
fn unique_sibling(archive: &Path, tag: &str) -> PathBuf {
let unique = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
let mut p = archive.as_os_str().to_owned();
p.push(format!(".{tag}-{}-{unique}", std::process::id()));
PathBuf::from(p)
}
/// fsync the directory, so the rename and the unlink are durable and not just
/// visible. Best-effort: a filesystem that refuses to open a directory is not a
/// reason to fail a GC that has otherwise succeeded.
fn sync_dir(path: &Path) {
if let Some(parent) = path.parent()
&& let Ok(f) = std::fs::File::open(parent)
{
let _ = f.sync_all();
}
}
impl Gc for NewGeneration {
fn run(&self, archive: &Path) -> Result<GcReport> {
let bytes_before = std::fs::metadata(archive)
.with_context(|| format!("stat {}", archive.display()))?
.len();
let before = manifest(archive)?;
let target = next_generation(archive)?;
if target.exists() {
bail!(
"{} already exists — a previous GC left a generation behind, or two are running \
at once",
target.display()
);
}
// 1. A second name for the same inode. No bytes move; the archive's own
// name keeps pointing at the original for the whole compaction.
let work = unique_sibling(archive, "gc");
std::fs::hard_link(archive, &work).with_context(|| {
format!(
"hard-linking {} to {} — a new generation is produced by compacting a second \
name for the same inode, which needs both on one filesystem",
archive.display(),
work.display()
)
})?;
// 2. znippy's own compaction, verbatim, against the work name.
let compacted = compact_archive(&work);
let CompactReport {
bytes_after,
rows,
delta_rows,
..
} = match compacted {
Ok(r) => r,
Err(e) => {
let _ = std::fs::remove_file(&work);
return Err(e.context(format!(
"compacting a new generation of {}",
archive.display()
)));
}
};
if self.stop_after == StopAfter::Compact {
bail!("interrupted after compact (test)");
}
// 3. Read the result back before anything is committed. On any failure
// the work copy goes and the original is untouched — that is the
// whole difference from A.
if let Err(e) = read_back_every_entry(&work) {
let _ = std::fs::remove_file(&work);
return Err(e.context(format!(
"the new generation of {} did not verify — nothing was replaced",
archive.display()
)));
}
let after = manifest(&work)?;
if after != before {
let _ = std::fs::remove_file(&work);
bail!(
"the new generation of {} carries {} entries, the original {} — nothing was \
replaced",
archive.display(),
after.len(),
before.len()
);
}
if self.stop_after == StopAfter::Verify {
bail!("interrupted after verify (test)");
}
// 4. The new generation takes its permanent name.
std::fs::rename(&work, &target)
.with_context(|| format!("naming the new generation {}", target.display()))?;
sync_dir(&target);
if self.stop_after == StopAfter::Rename {
bail!("interrupted after rename (test)");
}
// 5. Only now: the old index and the old data.
std::fs::remove_file(archive).with_context(|| format!("retiring {}", archive.display()))?;
sync_dir(archive);
Ok(GcReport {
strategy: self.name(),
archive: target,
retired: Some(archive.to_path_buf()),
bytes_before,
bytes_after,
rows,
delta_rows,
verified: true,
retired_packs: 0,
})
}
fn name(&self) -> &'static str {
"NewGeneration"
}
}
#[cfg(test)]
mod tests {
use super::*;
use znippy_common::{
SupersedeOutcome, ZnippyArchive, ZnippyReader, create_archive, read_delta_map,
supersede_as_delta,
};
/// Four generations of a pack, three of them superseded into deltas, so the
/// archive holds real dead payload for a GC to find. Modelled on
/// znippy-common's own `compaction_reclaims_the_superseded_blob` fixture.
fn fixture(dir: &Path, name: &str) -> (PathBuf, Vec<Vec<u8>>) {
let archive = dir.join(name);
let mut st = 0x5151_2323_abcd_ef01u64;
let base: Vec<u8> = (0..600_000u32)
.map(|_| {
st = st.wrapping_add(0x9e37_79b9_7f4a_7c15);
let mut z = st;
z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9);
(z ^ (z >> 27)) as u8
})
.collect();
let mut gens: Vec<Vec<u8>> = vec![base];
for g in 1..4 {
let mut next = gens[g - 1].clone();
next.extend_from_slice(format!("generation {g} tail ").repeat(300).as_bytes());
gens.push(next);
}
let files: Vec<(String, Vec<u8>)> = gens
.iter()
.enumerate()
.map(|(i, b)| (format!("pack-{i}.pack"), b.clone()))
.collect();
create_archive(&archive, &files, 3).unwrap();
for i in (0..3).rev() {
let out = supersede_as_delta(
&archive,
&format!("pack-{i}.pack"),
&format!("pack-{}.pack", i + 1),
3 - 1 - i,
3,
)
.unwrap();
assert!(
matches!(out, SupersedeOutcome::Delta { .. }),
"gen {i}: {out:?}"
);
}
(archive, gens)
}
fn reads_back(archive: &Path, gens: &[Vec<u8>]) {
let ar = ZnippyArchive::open(archive)
.unwrap_or_else(|e| panic!("{} does not open: {e}", archive.display()));
for (i, want) in gens.iter().enumerate() {
assert_eq!(
&ar.extract_file(&format!("pack-{i}.pack")).unwrap(),
want,
"{} lost generation {i}",
archive.display()
);
}
}
/// Both implementations reclaim the dead payload and neither changes a byte
/// any entry reads back as — delta chunks at their original depth included.
///
/// Asserted on applied output: the file size on disk, and every generation
/// extracted and compared byte for byte through the ordinary reader.
///
/// Seen RED by having `NewGeneration::run` `rename(archive, target)` instead
/// of renaming the compacted work file — i.e. publishing the *old* file
/// under the new generation's name: "called `Result::unwrap()` on an `Err`
/// value: retiring /tmp/…/NewGeneration.znippy … No such file or directory
/// (os error 2)" — the rename had already consumed the name step 5 goes on
/// to unlink. Restored.
#[test]
fn both_implementations_reclaim_the_dead_payload_and_change_no_entry() {
let dir = tempfile::tempdir().unwrap();
let raw = |gens: &[Vec<u8>]| gens.iter().map(|g| g.len() as u64).sum::<u64>();
for gc in [
&CompactInPlace::new() as &dyn Gc,
&NewGeneration::new() as &dyn Gc,
] {
let (archive, gens) = fixture(dir.path(), &format!("{}.znippy", gc.name()));
let before = std::fs::metadata(&archive).unwrap().len();
let report = gc.run(&archive).unwrap();
assert_eq!(report.strategy, gc.name());
assert_eq!(
report.rows,
4,
"{}: a GC must not change the row count",
gc.name()
);
assert_eq!(
report.delta_rows,
3,
"{}: the delta map must travel",
gc.name()
);
assert_eq!(report.bytes_before, before);
let on_disk = std::fs::metadata(&report.archive).unwrap().len();
assert_eq!(
report.bytes_after,
on_disk,
"{}: reported size is not the size",
gc.name()
);
assert!(
on_disk * 2 < raw(&gens),
"{}: the new archive is {on_disk} bytes for {} bytes of generations — the dead \
payload was not reclaimed",
gc.name(),
raw(&gens)
);
reads_back(&report.archive, &gens);
let map = read_delta_map(&report.archive).unwrap();
assert!(
map.iter().all(|(p, _, _)| p != "pack-3.pack"),
"{}: the live generation went behind a link: {map:?}",
gc.name()
);
}
}
/// The difference between A and B, as an assertion rather than a claim: A
/// keeps the name and has no retired file; B produces `x.g1.znippy`, retires
/// `x.znippy`, and the retired name is gone from the filesystem.
///
/// Seen RED by removing step 5 (the `remove_file(archive)`), so the old
/// generation survived its replacement: "B kept the old generation after
/// proving the new one". Restored.
#[test]
fn a_renames_over_the_original_and_b_produces_a_new_generation() {
let dir = tempfile::tempdir().unwrap();
let (a_path, gens) = fixture(dir.path(), "a.znippy");
let a = CompactInPlace::new().run(&a_path).unwrap();
assert_eq!(a.archive, a_path, "A moved the archive");
assert_eq!(a.retired, None);
assert!(a_path.exists(), "A removed the archive it compacted");
reads_back(&a_path, &gens);
let (b_path, gens) = fixture(dir.path(), "b.znippy");
let b = NewGeneration::new().run(&b_path).unwrap();
assert_eq!(b.archive, dir.path().join("b.g1.znippy"));
assert_eq!(b.retired, Some(b_path.clone()));
assert!(
!b_path.exists(),
"B kept the old generation after proving the new one"
);
assert!(b.archive.exists());
reads_back(&b.archive, &gens);
assert!(b.verified, "B must not report an unverified success");
// And again, from the new generation: g1 → g2.
let b2 = NewGeneration::new().run(&b.archive).unwrap();
assert_eq!(b2.archive, dir.path().join("b.g2.znippy"));
assert!(!b.archive.exists());
reads_back(&b2.archive, &gens);
}
/// **A crash at any step of B leaves a complete, readable archive.**
///
/// Not simulated debris: the run really stops at that step and nothing is
/// unwound, so the filesystem is in the state a `kill -9` there produces.
/// Asserted on applied output — every generation extracted and compared byte
/// for byte from whichever archive is supposed to be serving.
///
/// Seen RED by moving the `remove_file(archive)` to just after the hard link
/// (i.e. retiring the old name before the new one exists): at
/// `StopAfter::Compact` the assertion fired with "compact: the original was
/// unlinked early" — the very window this ordering exists to close.
/// Restored.
#[test]
fn an_interruption_at_every_step_leaves_a_readable_archive() {
let dir = tempfile::tempdir().unwrap();
for (step, label) in [
(StopAfter::Compact, "compact"),
(StopAfter::Verify, "verify"),
(StopAfter::Rename, "rename"),
] {
let (archive, gens) = fixture(dir.path(), &format!("{label}.znippy"));
let target = next_generation(&archive).unwrap();
let err = NewGeneration::stopping_after(step)
.run(&archive)
.expect_err("the interruption did not stop the run");
assert!(err.to_string().contains(label), "wrong stop: {err}");
// The original is still there and still serves, at every step.
assert!(archive.exists(), "{label}: the original was unlinked early");
reads_back(&archive, &gens);
match step {
StopAfter::Rename => {
// After the rename both names exist and both serve. That is
// the only step where two complete archives are on disk.
assert!(target.exists(), "the renamed generation is missing");
reads_back(&target, &gens);
}
_ => assert!(
!target.exists(),
"{label}: a generation took its permanent name before it was proven"
),
}
}
}
/// A stale generation left by an earlier interruption is refused rather than
/// silently overwritten — the one case where B could destroy a good archive.
///
/// Seen RED by deleting the `target.exists()` check: the run succeeded and
/// overwrote the debris — "overwrote a generation: GcReport { strategy:
/// \"NewGeneration\", archive: \"…/x.g1.znippy\", retired: Some(\"…/x.znippy\"),
/// bytes_before: 2440284, bytes_after: 624084, rows: 4, delta_rows: 3,
/// verified: true }". Had the debris been a *real* earlier generation, it
/// would have gone the same way. Restored.
#[test]
fn an_existing_generation_is_never_overwritten() {
let dir = tempfile::tempdir().unwrap();
let (archive, gens) = fixture(dir.path(), "x.znippy");
let target = next_generation(&archive).unwrap();
// Debris from an interrupted earlier run, in the place the next
// generation wants.
std::fs::write(&target, b"not an archive").unwrap();
let err = NewGeneration::new()
.run(&archive)
.expect_err("overwrote a generation");
assert!(
err.to_string().contains("already exists"),
"wrong error: {err}"
);
assert_eq!(std::fs::read(&target).unwrap(), b"not an archive");
reads_back(&archive, &gens);
}
/// **The holger retirement is asserted to be empty**, so it cannot be
/// mistaken for an oversight. If this starts failing, `retire_to_holger`
/// grew a body — which is the intended future. Delete this test in the same
/// commit that does it, on purpose.
///
/// Seen RED by replacing the `todo!()` with `Ok(RetirementReceipt { … })`:
/// "the holger retirement no longer panics — if it was implemented, delete
/// this test in the same commit: ()". Restored.
#[test]
fn retiring_to_holger_is_deliberately_empty() {
let target = HolgerTarget {
endpoint: "unspecified".into(),
repository: "nordisk/znippy".into(),
};
let outcome = std::panic::catch_unwind(|| {
let _ = retire_to_holger(Path::new("/nonexistent.g1.znippy"), &target);
});
let payload = outcome.expect_err(
"the holger retirement no longer panics — if it was implemented, delete this test in \
the same commit",
);
let msg = payload
.downcast_ref::<String>()
.cloned()
.or_else(|| payload.downcast_ref::<&str>().map(|s| s.to_string()))
.unwrap_or_default();
assert!(
msg.contains("deliberately empty"),
"it panics, but not as the placeholder: {msg}"
);
}
/// The generation names advance, and a `.znippy` stays a `.znippy`.
///
/// Seen RED by making `is_generation` always false, so a marker is never
/// recognised and never advances: "assertion `left == right` failed / left:
/// \"/srv/repo.g1.g1.znippy\" / right: \"/srv/repo.g2.znippy\"". Restored.
#[test]
fn generation_names_advance_in_the_stem() {
let n = |s: &str| {
next_generation(Path::new(s))
.unwrap()
.to_string_lossy()
.into_owned()
};
assert_eq!(n("/srv/repo.znippy"), "/srv/repo.g1.znippy");
assert_eq!(n("/srv/repo.g1.znippy"), "/srv/repo.g2.znippy");
assert_eq!(n("/srv/repo.g9.znippy"), "/srv/repo.g10.znippy");
// Not a generation marker: `git` is not `g<digits>`.
assert_eq!(n("/srv/repo.git.znippy"), "/srv/repo.git.g1.znippy");
assert_eq!(n("/srv/repo"), "/srv/repo.g1");
}
}