mikronova/modgunn

warehouse — design

Class: design · Function axis: f0 the seam · f1 tables & round-trip · f2 the fleet scan · f3 the CDC seam. Strict: only the shape we run. The why is in warehouse-reasoning.md; the system view in design.md.

f0 — the seam

The warehouse function (feature warehouse, off by default) gives modgunn Iceberg access over skade only. Warehouse (warehouse.rs:147) opens the shared physical catalog (catalog.redb + warehouse/) in the nornir namespace — the same bytes nornir writes, reached through skade's tables, never through nornir's API. Two opens:

• open(root) (warehouse.rs:161) — read/write.
• open_read_only(root) (warehouse.rs:175) — copies catalog.redb into an auto-cleaned temp dir when the live catalog is locked by another writer (a running nornir-server), so a reader never contends. skade stamps absolute file:// data paths, so a copied catalog still resolves live data.

ensure_modgunn_tables (warehouse.rs:218) creates modgunn's owned tables idempotently.

Reuse: warehouse.uses = [feeds, worlddelta, rollup]. It round-trips feeds' Advisorys, persists worlddelta's WorldSnapshots, and produces the FleetFindings rollup folds.

f1 — tables & round-trip

modgunn OWNs (writes) these tables and READs nornir's sbom_components:

behavior_features is the A23 package-behaviour VERDICT ledger: append_package_behavior( ecosystem, &PackageBehavior) stores the ecosystem-tagged coords + the full feature vector (install hooks / network-in-install / URL count / max token entropy / sensitive APIs, the Vec/Set fields comma-joined) + the derived verdict (worst finding severity + the comma-joined finding ids, folded from PackageBehavior::findings() — the SAME fold the CLI prints, no twin). load_package_behaviors() returns the latest [BehaviorRecord] per (ecosystem, subject) (append-only, last wins), whose inner PackageBehavior round-trips field-identical to the stored one (mutation-verified). Written behind the warehouse feature only — the lean scan/default gate (holger / static-holger) pulls no skade/arrow tree. The CLI append→load round-trip is behavior --warehouse → warehouse behavior-log.

append_advisories(db_rev, advisories) writes a new generation; load_advisory_db(eco) returns the latest db_rev, filtered by ecosystem (a single-partition scan when partitioned). Reads are drift-tolerant (str_col), so schema columns are added additively without breaking the reader. Engine-B capture helpers capture_world_snapshot (warehouse.rs:894, gated scip+warehouse) and record_behavior_delta_findings (warehouse.rs:919) persist a WorldSnapshot and its deltas in one call.

warehouse compiles WITHOUT scip. Every method that returns or consumes an Engine-B (scip) type — load_vuln_sinks, load_world_snapshot, reachable_vuln_findings, and append_vuln_findings_gated (the b1 reachability grader) — is #[cfg(feature = "scip")], so a lean consumer that takes only scan + warehouse (holger's Tools#3 warehouse-scan) builds the advisory/warehouse surface without pulling the heavy Engine-B graph plane. Before this the refs were ungated, so --features warehouse (and holger's dependency) failed to compile with unresolved import crate::scip. The CLI's warehouse scan --gate-on <world> needs scip and now fails loudly with that guidance when it is off, instead of silently dropping the flag.

f2 — the fleet scan

scan_fleet (warehouse.rs:434) joins the whole sbom_components table (every repo) × the latest cve_advisories and returns FleetFinding (warehouse.rs:945) rows { repo, ecosystem, name, version, advisory_id, severity }. Two variants:

• in-memory (default) — loads the advisory db and range-matches in Rust;
• scan_fleet_sql (warehouse.rs:484, feature warehouse-sql) — pushes the name = package equi-join down to skade/DataFusion, then range-matches + latest-rev in Rust on the narrowed candidates. A test proves it returns exactly the in-memory result.

sbom_drift(snap_a, snap_b) (warehouse.rs:652) reports (entered, left) (name, version) sets via the snapshot_id column (A30).

f3 — the CDC seam

modgunn consumes CDC; skade reconstructs it from the manifest layer.

• commit_seq (warehouse.rs:827) — a cheap poll to notice the catalog changed.
• current_snapshot_id(table) (warehouse.rs:835) — the table's latest snapshot (the watermark).
• table_delta(table, from) (warehouse.rs:846) — the rows changed since from + the new watermark, over skade's read_delta/DeltaPlan (added data files = inserts, equality-delete files = deletes).
• scan_sbom_delta(from) (warehouse.rs:462) — the incremental scan: only the components added since a snapshot, returning findings + the next watermark to poll.

Surface

The modgunn binary (main.rs) exposes warehouse import (→ append_advisories), warehouse scan [--since] [--vex] [--sql] (→ scan_fleet / scan_sbom_delta) and warehouse stats (→ load_advisory_db). DEFAULT_ECOSYSTEM = "cargo" (warehouse.rs:940) is assumed until nornir adds an ecosystem column to sbom_components.