lgz — crate card
Reverse-engineered (from source — verify, don't trust the name)
- One-line pitch: A pure-Rust parallel gzip decompressor that parallelizes across DEFLATE full-flush boundaries with a zero-copy ring-slot pipeline.
- Does: Pipeline is Reader → Main(carry+split) → N Workers → Collector → Writer with ring-slot recycle. Locates full-flush markers (
00 00 FF FF) where the LZ77 window resets so each segment is independently decompressible; streams frompigz/bgzf/Z_FULL_FLUSH parallelize fully, plaingzipoutput falls back to single-threaded flate2. Public API:decompress_gz,decompress_gz_capped,decompress_gz_into,decompress_gz_stream,deflate_offset,GzError, plus size-guardoutput_too_large/OUTPUT_TOO_LARGE. Modules:deflate_scan,chunk,parallel,speculative. - Key deps: intra-workspace
linflate(the actual DEFLATE core),flate2(fallback + header handling),tar. Dev:flate2,miniz_oxide. - Shape: lib + bin
lgz(705 LoC); ~2.1k LoC. Featuretiming.PARALLEL_THRESHOLD= 4 MB below which it stays single-threaded. - Observed role in the workspace: Gzip decode backend for
osm-katana(gzip-compressed OSM XML). Architecture explicitly mirrors lbzip2 and ljar; shares the DEFLATE core with them vialinflate.
Fill in (one line each — only the human knows these)
- Primary consumer(s):
- Why it exists (vs the obvious alternative):
- Hard constraint (airgap / no-C-deps / no-FFI / pure-Rust / etc.):
- Release cadence & independence (rides a parent? own repo someday?):
- Status (production / prototype / experimental / stubbed):
- Must-NOT-change invariant:
- Biggest known gap or TODO:
- Anything a newcomer would get wrong about it: