mikronova/modgunn

warehouse — reasoning

Class: reasoning. Why the warehouse seam is shaped this way, and the forks not taken. The descriptive shape is in warehouse-design.md.

Why over skade, never nornir

modgunn needs the shared warehouse (advisories, snapshots, SBOM) but must not depend on nornir — that edge is the cycle the whole crate exists to break (see reasoning.md). skade is an embedded-Iceberg leaf, so modgunn → skade is cycle-free, and skade re-exports arrow/iceberg so modgunn needs no direct arrow/iceberg deps. modgunn reaches nornir's data by opening the same physical catalog and reading nornir's sbom_components table directly — data coupling through a shared table, not code coupling through an API. That is the only way holger + nornir can both consume modgunn while modgunn stays a leaf.

Why the whole seam is behind a feature (off by default)

skade pulls arrow/iceberg/redb — a heavy tree holger's lean gate and static-holger must never carry. So warehouse is opt-in; the default scan build has no warehouse at all, and Engine A takes an in-memory AdvisoryDb instead. The world-delta site and nornir opt in. The DataFusion fleet-join is a second wall (warehouse-sql) because DataFusion (+ the C liblzma via async-compression) is heavier still and only pays off at S3/S4 scale where loading the whole advisory db into RAM stops being affordable.

Why a read-only copied-snapshot open

redb is single-writer. A live nornir-server holds the write lock, so a modgunn reader that tried to open read/write would block or fail. open_read_only copies catalog.redb into an auto-cleaned temp dir and reads that — a consistent point-in-time snapshot with zero contention. It works because skade stamps absolute file:// data paths, so the copied catalog still points at the live data files; only the tiny metadata db is copied. This mirrors nornir's own open_read_only, so the two agree on the pattern.

Why range-native rows and drift-tolerant reads

Advisories are stored as ranges (see feeds-reasoning.md); the warehouse round-trips the range JSON plus alias/withdrawn/source/CVSS/CWE columns. Reads go through str_col, which tolerates missing columns, so schema v2 could add columns additively without a migration — an old reader ignores new columns, a new reader defaults absent ones. Iceberg appends are additive by nature, which is what makes db_rev generations cheap: each import is an append, old generations stay for time-travel/audit (A31/A34), and load_advisory_db simply returns the latest rev.

Why modgunn consumes CDC rather than computing it

iceberg-rust 0.9.1 has no native incremental/changelog scan — only time-travel by snapshot id. Rather than reconstruct CDC itself, modgunn leans on skade, which already walks the snapshot lineage (parent_snapshot_id) and returns a DeltaPlan (added data files = inserts, equality-delete files = deletes). modgunn just polls commit_seq, then table_delta(from). When upstream iceberg-rust ships a native changelog scan, skade swaps its internals and modgunn's API does not change — the seam is deliberately thin so that swap is invisible here.

Why the incremental scan exists at all

At S3/S4 scale recomputing the whole fleet on every advisory import is wasteful — only the neighborhood a read_delta touched actually changed. scan_sbom_delta(from) scans only the components added since a watermark and returns the next one, so a monitor polls commit_seq → scan_sbom_delta and does O(delta) work, not O(fleet). This is the same reason the heavy Engine-B materializations (blast-radius, owner-clusters) are designed to refresh off the CDC delta rather than from scratch.