nordisk/znippy-zoomies

lbzip2 library readers — guide

Class: guide · Function: lbzip2-readers. How to call the three bzip2 library twins. Shape in lbzip2-readers-design.md; the CLI in lbunzip2-guide.md.

Build

CARGO_TARGET_DIR=/mnt/work4t/cargo-target/zoomies cargo build --release -p lbzip2

Pick a reader

use std::io::Read;

// 1. Whole slice in, whole Vec out (you already have the bytes, want it all):
let out: Vec<u8> = lbzip2::parallel::decompress_parallel(&bytes)?;   // parallel.rs:15

// 2. Read-trait streaming over any Read source (bounded memory, 100 MB chunks):
let mut r = lbzip2::reader::StreamingBz2Read::new(file)?;            // reader.rs:30
let mut xml = Vec::new();
r.read_to_end(&mut xml)?;

// 3. Read-trait over an mmap'd / in-memory slice (64-block batches):
let mut r = lbzip2::reader::ParallelBz2Read::new(&mmap)?;            // reader.rs:205

When to use which

• decompress_parallel — you hold the whole .bz2 and want the whole output; the fastest, but resident memory is the full decompressed size.
• StreamingBz2Read — a Read source (socket, pipe, file) and bounded memory matters; only one ~100 MB chunk's decoded output is live at once.
• ParallelBz2Read — an mmap'd file you want to stream through Read without materialising all output; decodes in 64-block batches.

All three read anything bzip2/lbzip2/pbzip2 wrote and CRC-check every block.

Bench

These share the lbzip2 crate benches (cargo bench -p lbzip2); the headline head-to-head is the CLI in the README A/B/C mashup — see decompressors-guide.md.