nordisk/znippy

znippy decompress — reasoning

Class: reasoning. The why behind the read path and every alternative we rejected. The descriptive shape is in decompress-design.md; how to drive it is decompress-guide.md.

f0 — why the read path has no reader thread

The compress path needs a reader thread because its input is a stream it must split into slices. The read path does not: the index already carries every blob_offset, so there is nothing to discover sequentially. That single fact drives the whole shape below.

Rejected: a reader thread + work channel (mirror the compress engine)

The symmetric design would reuse the Gatling engine — one reader fans blobs to N workers over a channel. We rejected it: it would reintroduce a serial front (the reader) and a channel hop for data that is already random-access on disk. Instead N workers share one AtomicU64 row cursor and fetch_add their next row, then pread its blob directly. There is no reader, no channel, no slot pool — the disk's own queue depth is the parallelism, and NVMe gets real concurrent I/O instead of one thread feeding many.

Rejected: streaming decompression (a windowed decoder per file)

A classic decompressor streams: keep a sliding window, pull compressed bytes, emit plaintext. We rejected streaming on the read path because every chunk is a complete, independently sized unit whose uncompressed size the index states. So a worker decompress_into a reused buffer in one shot (or copies the raw bytes when compressed = false), with no window state to carry and nothing to allocate per call (Law 2). Streaming would buy nothing here and cost buffer management.

Rejected: verifying after the whole file is reassembled

Integrity could be checked once per file, after all its chunks land. We verify per chunk, against the 32-byte blake3 the index carries for that chunk_seq, the instant the chunk is decoded — before it is placed at fdata_offset. Reasons: a corrupt chunk is caught at its source row (not blamed on a whole file), and because checksums are bound to chunk_seq and not thread identity (Law 3), the verdict is identical no matter how many workers ran or in what order they finished. Deterministic verification is a property of the data, not the run.

Rejected: trusting archive paths on extract

A path inside an archive is untrusted input. We rejected writing it out verbatim: safe_output_path rejects absolute paths and .. traversal (zip-slip) before any pwrite, and a corrupt archive returns an error rather than panicking (tests safe_output_path_rejects_zip_slip_and_absolute_paths, decompress_corrupt_archive_errors_not_panics). The cost is a per-path check; the alternative is a directory-escape write primitive, which is not a trade we make.

Why selective restore lives in the same engine

decompress_archive_filtered (by pkg_type / repo) is not a second code path — it filters which index rows enter the shared cursor, then runs the identical worker loop. Rejected: a bespoke selective extractor. Keeping one engine means the filter can never diverge from the full-restore semantics; "extract a slice" is just "extract, over fewer rows."