.nornir/archive-design.md

archive — design

Class: design · Function axis: f2. States only the shape we ship; the why and the znippy-consolidation plan are in archive-reasoning.md.

The `Archive` trait — the engine seam

pub trait Archive (archive.rs:86) is the one seam every caller uses; the engine behind it is swappable without touching the pipeline.

method	signature	what it does
`pack`	`pack(staging, version, out) -> BundleManifest` (`archive.rs:88`)	seal a staging tree into a bundle, return the manifest
`list`	`list(bundle) -> Vec<String>` (`archive.rs:91`)	list relative entry paths
`extract_one`	`extract_one(bundle, entry, dest)` (`archive.rs:94`)	extract one entry atomically (temp → rename)
`extract_all`	`extract_all(bundle, dest_dir)` (`archive.rs:96`)	extract the whole bundle, tree preserved
`read_entry`	`read_entry(bundle, entry) -> Vec<u8>` (`archive.rs:102`)	read one entry's bytes (znippy overrides for O(1))
`verify_entries`	`verify_entries(bundle, manifest) -> Vec<String>` (`archive.rs:117`)	return the bad-entry paths (missing and extra both count)
`delta`	`delta(old, new_staging, version) -> Patch` (`archive.rs:143`)	binary delta against a prior manifest (znippy)
`apply`	`apply(old_bundle, patch, new_bundle) -> BundleManifest` (`archive.rs:151`)	reconstruct a new bundle from an old one + a patch (znippy)

The content-addressed manifest

Every bundle carries a sidecar <bundle>.manifest.json (archive::manifest_path, archive.rs:338; load_manifest, archive.rs:343):

EntryDigest { path, sha256, len } (archive.rs:41) — one per entry, the content-addressed unit.
BundleManifest { version, entries, root_sha256 } (archive.rs:52); verify_root() (archive.rs:65) recomputes the root over the sorted entries and compares.
Hash helpers: sha256_hex / sha256_file (streaming) — thin seams that CONSUME the shared nornir-hash leaf (sha256_hex / sha256_file / Sha256Hasher / hex_encode) rather than hand-rolling Sha256::digest + a {b:02x} hex loop (L5 dedup); digests are byte-identical to the retired twin.

This manifest is what fetch::verify_on_receipt (f3) gates the unpack on.

`TarZstd` — the always-on default

pub struct TarZstd { level } (archive.rs:192; new() → zstd level 3, archive.rs:198). Pure Rust (tar + zstd), always builds and runs offline. impl Archive (archive.rs:203):

pack (archive.rs:204) — two-pass: content-digest the tree (collect_digests), stream tar→zstd into out.tmp, atomic rename, write the sidecar manifest.
verify_entries (archive.rs:296) — a single streaming pass (O(total bytes)), hashing each entry on the fly; an archive entry absent from the manifest is pushed bad (archive.rs:329), a manifest entry absent from the archive is pushed bad (archive.rs:332).
read_entry uses the trait default (temp-extract one entry).

`ZnippyArchive` — feature `znippy` (REAL znippy-core, one canonical format)

pub struct ZnippyArchive { level } (#[cfg(feature = "znippy")]) is a thin adapter over the real znippy stack (../znippy) — L5 dedup: there is now ONE archive format in the constellation, not a Skidbladnir look-alike twin. A bundle sealed here IS a genuine .znippy archive, readable by the znippy CLI / DuckDB / Polars. On-disk format (the real znippy layout):

[ zstd/OpenZL chunk blob region ][ Arrow-IPC sub-index(es) ][ manifest + global footer (ZNPYMIDX) ]

seal — pack content-digests the tree for the SHA-256 sidecar manifest, then znippy_compress::compress_dir writes the real archive (seal renames the forced-.znippy output onto the caller's exact path). Per-chunk blake3 is znippy's own integrity (Design law 3).
selective read — read_entry calls znippy_common::get_file: locate the entry via the lookup sub-index, pread only its chunk blobs, blake3-verify each, reassemble — never decompresses the whole archive. list uses znippy_common::ZnippyArchive::open(..).list_files(); extract_all uses the parallel gatling decompress_archive (blake3-verified, zip-slip-safe, fails on any corrupt file).
delta — delta keeps entries whose hash is unchanged and ships only new blobs (deduped); apply reconstructs the target tree (kept entries via get_file from the old bundle, added from the patch), then re-seals it as a fresh real znippy bundle — the result reproduces the target's content exactly (same per-entry SHA-256 → same manifest root_sha256 as a clean pack). Patch carries kept/added/added_blobs; shipped_bytes() and reused() are the metrics.

Security guards

zip-slip — reject_unsafe_entry_path rejects absolute paths and .. that climbs above the root; enforced on every znippy extract_one/read_entry/apply. decompress_archive applies its own safe_output_path gate on full extract.
extra-entry / tamper — verify_entries treats extra and missing entries as bad; a tampered archive byte fails znippy's per-chunk blake3 (surfaced by read_entry/verify_archive_integrity); a tampered manifest fails verify_root.

Contract (inject-assert tests)

pack_then_extract_roundtrip_with_manifest_checksums, tampered_manifest_fails_root_verify, verify_rejects_extra_entry_absent_from_manifest (TarZstd); and over REAL znippy-core: znippy_seal_verify_unpack_roundtrip_over_znippy_core, znippy_tampered_byte_fails_verification (RED-when-broken), znippy_content_addressing_identical_entries_round_trip, znippy_selective_read_returns_one_entry, znippy_delta_round_trip_reproduces_target_and_ships_only_changes, znippy_extract_rejects_zip_slip_paths, znippy_full_extract_round_trips.