nordisk/znippy-zoomies

Decompressors — guide

Class: guide · Function: decompressors. How to build, run, and bench the parallel decoders. Chosen design in decompressors-design.md; the crushing numbers in benchmarks.md.

Build

cargo build --release -p lgz -p lbzip2 -p ljar -p lzip-parallel   # the four CLIs
cargo build --release -p linflate                                  # the DEFLATE core (lib)

The binaries land at target/release/{lgz,lbunzip2,ljar,lzip}.

Run the CLIs

# multi-member gzip → stdout (or -o file)
lgz input.gz > output
lgz input.gz -o output

# bzip2 → output path
lbunzip2 input.bz2 output

# JAR / ZIP → extract
ljar archive.jar
lzip archive.zip

All four saturate every core; watch htop — flat 100 % across cores is the contract. Rivals: they read the same stock formats as gzip/bzip2/unzip, so any file those tools produce, these decode — just in parallel.

Use as a library

// multi-member gzip, keep only members matching a needle:
let members = lgz::decompress_tar_gz_filter(&bytes, "config")?;   // Vec<(name, bytes)>

// JAR / ZIP entries:
let entries = ljar::decompress_jar(&bytes)?;                      // Vec<JarEntry>
let entries = lzip_parallel::decompress_zip(&bytes)?;             // Vec<ZipEntry>

// raw DEFLATE core (one segment):
let out = linflate::inflate_to_vec(&deflate_bytes, expected_len)?;

Run the HEAVY bake-off

The head-to-head lives in examples/nornir-bench.rs and is driven by nornir:

nornir bench run znippy-zoomies                 # HEAVY: all cores, max memory, one BenchRun row
NORNIR_ZOOMIES_CORPUS_MB=1024 nornir bench run znippy-zoomies   # explicit heavy corpus (default)

It compresses one deterministic corpus into every rival's format, then races every decompressor on the box — znippy-gz/znippy-bz2 against gzip/pigz, bzip2/lbzip2/pbzip2, and the cross-format zstd/lz4/xz/pixz/brotli/plzip. Rivals not installed are skipped and named. There is no small/smoke mode (heavy-bench law).

Then render the numbers:

nornir docs book znippy-zoomies                 # fills the mashup marker → docs/manual.pdf

See benchmarks.md for the recorded table (host CPU, cores, and peak RSS recorded per rival) and how to read it.