nordisk/znippy-zoomies

lzip-parallel — guide

Class: guide · Function: lzip-parallel. Build, run, tune. The shape is in lzip-parallel-design.md; the why in lzip-parallel-reasoning.md. The JAR-subset twin is ljar-guide.md.

Build

CARGO_TARGET_DIR=/mnt/work4t/cargo-target/zoomies cargo build --release -p lzip-parallel

Binary lands at target/release/lzip.

Run

lzip archive.zip                  # extract every entry into ./ (each its own file)
lzip archive.zip -o out/          # extract into a target dir
lzip pkg.whl --filter METADATA    # only entries whose name contains the needle

Any file zip wrote decodes — wheels, .jar, plain .zip (JAR ⊂ ZIP). Entries are extracted to independent files by the parallel writer (gatling::parwrite::extract_entries_unordered, lzip-parallel/src/bin/lzip.rs:146); a filter excludes non-matching entries at the Central Directory level, so they are never read or inflated.

Tune

• Worker count defaults to physical cores; the balanced fan-out self-dispatches with no barrier, so a skewed archive still fills every core (heaviest entry claimed first).
• Streaming decode holds only one 64-entry batch of output live at a time (BATCH_SIZE, lzip-parallel/src/reader.rs:20) — peak RAM stays bounded on huge ZIPs.

Library

// in-memory decode (parallel; streams internally above 10 MB):
let entries = lzip_parallel::decompress_zip(&bytes)?;                 // lib.rs:58
// exact-name set (wheels: pull just METADATA + RECORD):
let meta = lzip_parallel::decompress_zip_filter_set(&bytes, &["METADATA", "RECORD"])?;  // lib.rs:94
// all files of a type:
let py = lzip_parallel::decompress_zip_filter_suffixes(&bytes, &[".py", ".pyi"])?;      // lib.rs:111
// stream any Read+Seek without loading the file into RAM:
let f = std::fs::File::open("archive.zip")?;
let entries: Vec<_> = lzip_parallel::decompress_zip_stream(f)?;       // lib.rs:126

Bench

Driven by nornir through the shared harness — see decompressors-guide.md. HEAVY only (no smoke): nornir bench run znippy-zoomies, then nornir docs book znippy-zoomies to fill the A/B/C mashup.