nordisk/znippy-zoomies

Magazine → chunk_revolver consolidation (kill the fork)

znippy's "Magazine" slot pool (znippy-common/src/slotpool.rs:194) is a FORK of the canonical gatling::revolver — its own header says it "Replaces the per-thread ChunkRevolver rings." Same gatling-style principle: add the missing mode to the canonical engine, then drop the fork.

Phase 1 is DONE (2026-08-25, gatling/src/revolver/coalesce.rs). Phase 2 is open and is znippy-side work.

The capability Magazine had that chunk_revolver lacked

Coalescing / shared-slot packing. Magazine packs many small files into ONE fixed-size slot as variable-width slices (invariant: a slice ⊆ one file AND ⊆ one slot), with per-slot reference counting (AtomicUsize armed at publish, freed when the last worker release_ones) and a blocking claim() (backpressure). chunk_revolver was 1-write = 1 fixed-size slot, no ref-count, non-blocking try_get_chunk.

Phase 1 — coalescing mode in chunk_revolver — DONE

gatling/src/revolver/coalesce.rs, additive: ChunkRevolver::new / try_get_chunk / return_chunk / Chunk are untouched and tests/chunk_revolver.rs still passes.

Three deliberate divergences from the plan sketch and from the fork:

1. No slot_ring: Vec<u8>. A coalescing slot has no ring affinity worth preserving (its consumers are N workers on one global queue, not one worker on one ring), so the free list is flat. arm_coalescing (revolver.rs:105) drains the per-ring free lists into it, which is also what makes the two modes mutually exclusive by construction rather than by a runtime flag: try_get_chunk on a coalescing pool can only return None.
2. Mutex<Vec<u32>> + Condvar, not a channel. Touched once per slot, never per slice; keeps crossbeam out of this leaf crate.
3. publish_into recycles the slice buffer. The fork allocates a fresh Vec<Round> per claimed slot and grows it one payload at a time; publish_into drains into a caller-owned buffer and returns its own to a pooled scratch stack.

Proof: tests/chunk_revolver_coalescing.rs (5 tests — coalescing lifecycle with the slot freed only on the Nth release, 4 workers reading published slices against a still-packing reader, un-published drop returns the slot, empty publish frees immediately, no cross-mode double-issue). NOT proven: miri. The plan asks for a miri run over the ref-count orderings; this box has no usable nightly (cargo +nightly miri → "Missing manifest in toolchain 'nightly-x86_64-unknown-linux-gnu'"). Orderings are byte-identical to the fork's (store(n, Release) / fetch_sub(1, AcqRel)), which is an argument, not a proof.

Bench: benches/revolver_coalescing.rs, best of 9 interleaved rounds over 133 650 real files (985.7 MiB, mean 7 733 B/file) on an AMD Ryzen AI 7 PRO 360 (8C/16T), 16 workers, 64 MiB reservation for every arm:

Canonical beat the fork replica in all five interleaved runs taken, by 5–15%, and allocates 7.9× less. Honest negative: coalescing is not a throughput win over fixed-slot once the corpus is warm — the pipeline is memcpy-bound and every arm lands within noise. Cold, coalescing LOSES (1404 vs 1872 MiB/s): 8 MiB slots cycling 64 MiB blow a cache that 64 KiB slots keep hot. The reason to have the mode is the capability — 64 MiB buys 1 024 tiny files in flight fixed-slot, or ~8 700 coalesced — not the speed.

Phase 2 — switch the znippy consumer + delete Magazine — OPEN (znippy-side)

Magazine is consumed at three sites, all in one file, exactly as the plan predicted ("sole consumer"):

• znippy-compress/src/slot_packer.rs:660 — let pool = Magazine::from_plan(plan);
• znippy-compress/src/slot_packer.rs:821 — let pool = Magazine::from_plan(plan);
• znippy-compress/src/slot_packer.rs:1119 — pool: &'p Magazine (the pack helper)

plus type-only mentions in the same file (Round at :111, :197, :216; Ejector at :153, :200; Clip at :1121) and one test-side PoolPlan import at tests/tests/precompressed_big_files.rs:48.

Mechanical map, then delete znippy-common/src/slotpool.rs:

Magazine::from_plan(plan)   -> ChunkRevolver::with_coalescing(plan.num_slots,
                                                              plan.slot_size,
                                                              plan.slice_size)
Magazine::claim()           -> ChunkRevolver::claim_slot()
Magazine::returner()        -> ChunkRevolver::returner()
Clip                        -> Filler          (same methods)
Clip::publish()             -> Filler::publish_into(&mut reused)   [preferred]
Round                       -> PackedSlice     (same seven fields, same unsafe as_slice)
Ejector                     -> Returner
Ejector::release_one        -> Returner::release_one

PoolPlan stays in znippy: DEFAULT_SLOT_SIZE / DEFAULT_NUM_SLOTS / slice_size_for are znippy's output-affecting cut policy, not pool mechanics, and their tests live there. Keep the type, move only the pool it constructs. Add znippy-zoomies (or rotaryengine directly) to znippy-compress's manifest if absent.

Gates, unchanged: round-trip BYTE-IDENTICAL (fallback_byte_identical_to_io_uring etc.), and a before/after RELEASE bench of small-file archiving (xtask nornir-bench archive_*) — revert on >1–2% regression. The zero-copy pwrite path must stay identical-or-better.

§2 of the slot-pool memory plan — disk-backed slots — still proposed

.nornir/slot-pool-memory-plan.md:91 proposes a disk-backed slot mode (tmp-file-backed slots that can be evicted under pressure) on top of §1's PoolPlan. Untouched by Phase 1 and independent of it: it is a change of where a slot's bytes live, not of how slots are claimed, packed or released. If it is ever built, it belongs in coalesce.rs behind the same Filler/PackedSlice surface so no consumer changes.