mikronova/modgunn

baddie-detection — design (the flagship join)

Class: design · Axis: b0 the join contract · b1 reachable-vuln · b2 malicious / typosquat · b3 KEV / yank cascade · b4 leak path · b5 the verdict trichotomy. Strict: the shape we run + the data modgunn OWES the Mímir lens. The why is in the master plan A8/A10; the ingest side in feeds-design.md.

Ownership boundary. modgunn OWNS the data (the cve_advisories rows, keyed for join) and the verdict semantics (refuse/hold/act per node). The Mímir lens + the red-marked graph rendering is nornir = smid's — this doc is the join spec + the table contract smid builds that lens against. modgunn never renders the graph; it guarantees the joinable rows exist.

b0 — the join contract (what already runs, and on what)

The present-on-disk base join is live today: Warehouse::scan_fleet equi-joins modgunn's cve_advisories against nornir's sbom_components on (ecosystem, name, version) and emits FleetFinding { repo, ecosystem, name, version, advisory_id, severity }. That is the "the dangerous thing is present in this repo" layer. Baddie-detection is the set of escalation joins layered on top of it that turn present into actually dangerous, according to the reports — each one narrows a FleetFinding set by a reachability / lineage predicate.

Data law for join-readiness (the thing modgunn owes smid): every advisory row must carry coords the join keys on. A CVE row keys on (ecosystem, package) + a version range; a yank/deprecation row keys on (ecosystem, package) + an exact version — registry::parse_crates_yanks / parse_npm_deprecations / parse_pypi_yanks emit exactly that (affected_versions = [version], ecosystem ∈ {cargo, npm, pypi}, source ∈ {crates.io, npm, pypi}), so a yanked version already surfaces through scan_fleet as a FleetFinding with advisory_id = MODGUNN-YANK-*. No new plumbing — the yank rides the CVE seam (L5). smid's lens filters on the advisory_id prefix + source to bucket the finding kind.

Canonical-ecosystem contract (coordination with smid). The equi-join keys on the ecosystem label, so both sides MUST agree on it. modgunn's advisory plane is canonical: OSV normalizes to pypi/cargo/npm/… (osv::normalize_ecosystem) and the version comparator keys on those (versions::compare). Two known aliases diverge on the component side and must be canonicalized before the join fires: pip ≡ pypi (the manifest parser tags Python components pip) and crates.io ≡ cargo. versions::compare already folds pip→PEP 440 so range-matching is label-robust. As of L6 the name/ecosystem join key itself is canonicalized inside Advisory::matches (and the indexed Engine-B bucket key, so A≡B holds): scan::canonical_eco folds pip→pypi, and scan::canonical_pkg applies PEP 503 to PyPI names (case-insensitive; any -/_/. run → a single -) on BOTH sides — so an advisory keyed django / zope-interface (the normalized form OSV/PyPA ship) now matches a component spelled Django / zope.interface or labelled pip. Before L6 this was a silent exact-string compare — a systemic PyPI false negative (every real project spelling missed its CVE). Non-PyPI ecosystems stay verbatim/case-sensitive (collapsing separators would corrupt a Maven group.artifact). The fleet SQL pre-filter (scan_fleet_sql) still equi-joins on the raw name in-warehouse (it can't apply PEP 503 ecosystem-blind, and sbom_components has no ecosystem column anyway), so full fleet-scale PyPI canonicalization lands with that column; the in-memory scan_fleet path and per-artifact matches are correct now. Today the fleet join (scan_fleet) assumes DEFAULT_ECOSYSTEM = cargo for every component — sbom_components carries no ecosystem column yet (nornir's SBOM is cargo metadata). So the npm/pypi advisory + yank rows are populated and correct, but only surface at fleet scale once nornir's multi-ecosystem SBOM cutover adds the ecosystem column AND modgunn reads it (canonicalizing pip→pypi at read time). Until then the multi-ecosystem cascade is verified per-artifact (Advisory::matches), not fleet-wide. This is the one cross-repo item the baddie-detection join is blocked on.

b1 — reachable-vuln (A8, the 80–90%-FP killer)

reachable-vuln = vuln_findings × sbom_components × dep_edges × call_edges. A FleetFinding for a CVE is escalated to 🔴 only if the CVE'd symbol is transitively CALLED from a repo entry point, not merely present on disk:

1. scan_fleet → the candidate FleetFinding set (CVE present in a resolved component).
2. For each candidate, resolve the CVE → its affected symbol(s) — the sink. This is now REAL data, not a TODO: the Go vuln DB ships symbol-level granularity in OSV affected[].ecosystem_specific.imports[].symbols, parsed by osv::parse_osv_vuln_symbols into VulnSymbol{id,ecosystem,package,symbol,severity} and persisted to the modgunn-owned vuln_symbols table (`warehouse import-vuln-symbols --go-osv
3. Multi-source BFS/DFS from the repo's entry points over scip_symbols (the call graph) — the A8 closure. A candidate whose sink is not in the closure is demoted (present-but-unreachable).
4. Reachable ⇒ 🔴; present-but-unreachable ⇒ 🟠 hold (still worth surfacing — a future call edge makes it live, which is the A10 sink-newly-reachable set-diff across two world_snapshots).

Already implemented (Engine B, scip.rs) — the primitives the lens calls, not TODO: the closure is WorldSnapshot::reachable_from (multi-source forward BFS entrypoints→callees over the caller→callee call_adjacency); the A8 gate itself is scip::reachability_gated_vulns; the A10 promotion is SinkNewlyReachableTripwire (the after.contains && !before.contains set-diff); ranking within a verdict bucket uses the blast_radius PageRank (widely-depended-on hub ranks top). These were adversarially audited (2026-07-26) and confirmed correct on direction, multi-source seeding, visited-marking, and cycle-termination — so smid's lens can rely on them rather than reimplement the traversal. modgunn owns the closure + the demotion predicate; smid renders the reachable path RED and the unreachable node amber in the graph.

WIRED end-to-end (2026-07-27). The join now runs off persisted data, not just in-memory tests: Warehouse::reachable_vuln_findings(snapshot) reconstructs the whole WorldSnapshot from the warehouse — scip_symbols (via read_scip_symbols), the NEW call_edges table (the symbol call graph, persisted by capture_world_snapshot + read by read_call_edges — was previously dropped on capture), dep_edges, and the vulnerable sinks (vuln_symbols → VulnSink via load_vuln_sinks) — then runs reachability_gated_vulns/unreachable_vulns and returns (reachable RED, suppressed). CLI: warehouse reachable-vuln --snapshot <id>. The b1 join-input table list above is now fully materialized (call_edges was the missing one). End-to-end tested (reachable_vuln_join_reds_only_the_called_symbol): a captured snapshot whose entrypoint calls one CVE'd symbol but not another → the called one is RED, the orphaned one suppressed, all through a warehouse reload.

b2 — malicious / typosquat dep

OpenSSF-malicious × dep_edges + the typosquat/dep-confusion detector (Engine B #8). A MAL-* row (from feeds::parse_malicious, imported Critical) matching any node in the repo's dep_edges closure is an unconditional 🔴 — malware needs no reachability gate (its install is the payload; see behavior A23 install-hook features). A typosquat candidate (edit- distance-near a popular package, newly-introduced) is 🟠 hold pending human confirm.

b3 — KEV / yank cascade (the new leg this pass)

kev/yank × release_lineage. Two sub-joins over world_snapshots (the per-(ecosystem,name) release history):

• KEV cascade — a FleetFinding whose advisory carries kev = true (CISA Known-Exploited) that is also reachable (b1) is the top-priority 🔴: confirmed-exploited and reachable.
• Public-exploit rank — an advisory lifted by feeds::enrich_exploited (a PoC-in-GitHub public PoC → severity floored at High, [public-exploit] summary marker) ranks a reachable 🔴 above a version-only High: exploitability-in-practice is the tiebreaker between two otherwise-equal-severity findings. Schema-free, so smid's lens reads it off the existing severity + summary columns — a finding whose summary contains [public-exploit] sorts first within its verdict bucket. The evidence ladder the rank encodes: KEV (in-the-wild) > public-PoC > version-only.

• Yank / deprecation cascade — a MODGUNN-YANK-* / MODGUNN-DEPRECATED-* FleetFinding means the repo pins a version the registry itself pulled back. Verdict by lineage:

  • The yanked/deprecated version is the one currently resolved in sbom_components → 🟠 hold: you are shipping a release the ecosystem retracted; pin forward. (Deprecation, being advisory-only, floors at 🟠; a yank that the registry scrubbed for malware — rare, detected by a co-located MAL-* alias on the same coords via merge_by_alias — inherits b2's 🔴.)
  • A newer, un-yanked version exists → the fix is a version bump to the earliest clean successor, which modgunn now computes at ingest (registry::earliest_clean_successor, per-ecosystem ordering) and writes into the advisory summary (…; upgrade to X), so it rides the round-tripping summary column and smid's lens renders it with no extra query. When the whole tail is yanked, the summary says (no clean successor) — an honest "no fix forward", never a yanked target. This is the actionable remediation the cascade exists to produce.
  • The yanked version is only a transitive dependency (reached through dep_edges, not a direct manifest pin) → still 🟠, but attributed to the intermediate dependent that pins it, so the fix lands in the right place.

Because the yank row is already a cve_advisories row keyed to an exact version, the cascade's base layer needs no new query — scan_fleet yields it; the cascade is the world_snapshots lineage lookup smid layers on to pick the successor version + attribute the transitive case.

b4 — leak path

warehouse_access_edges × call_edges. A symbol that reads a warehouse-access credential/edge and is transitively reachable from an untrusted entry point (a network handler) is a data-exfil path → 🔴. Owned by Engine B's taint-path source→sink (taint_path_findings); listed here for completeness — the same reachability machinery as b1, different sink class.

b6 — active-malware IOC match (the garm-side axis)

ioc_indicators × workspace_observables. Beyond vulnerable packages, the IOC corpus (ioc.rs; abuse.ch URLhaus/Feodo/SSLBL/ThreatFox → the ioc_indicators table) flags what is actively malicious — no reachability gate, because a match IS the compromise (like the b2 malicious dep). Two consumer sides, one table:

• Supply-chain (dwarves) side. A resolved dependency's download URL matching a URLhaus row, or a released artifact's hash matching an SSLBL/MalwareBazaar/ThreatFox hash → 🔴: a dependency served from a known malware-distribution URL, or a build output byte-identical to a known-bad sample. The join key is the indicator value (URL / sha256|sha1 / domain) against the workspace's SBOM/artifact metadata.
• Garm side. A workspace/host's egress IP/domain (from a network-policy manifest, a compose file, an observed connection) matching a Feodo C2 IP or a ThreatFox ip:port/domain → 🔴: talking to a botnet controller. modgunn owns the corpus + the match predicate; garm renders/acts.

Verdict: an IOC hit is refuse 🔴 unconditionally (active malware, not a latent vuln) — the strongest grade, ranked with KEV+reachable at the top of b3's evidence ladder. Join-readiness: the indicator is stored (kind, value) with value normalized (hashes lowercased, IPs dotted-quad). modgunn owns the match predicate — ioc::IocIndex (build from the corpus, matches(kind, value) O(1) with the same normalization) is the b6 analogue of Advisory::matches; the Mímir lens calls it against each workspace observable rather than reimplementing the lookup. This is the one axis whose workspace-observable side is not yet a modgunn table — it is the SBOM/artifact-metadata and the network-policy manifests the consumer already holds; modgunn supplies the authoritative bad-list.

WIRED end-to-end (2026-07-27). b6 is no longer a bare lookup: Warehouse::ioc_scan takes raw observables, classifies each by shape (IocKind::infer), matches it against the persisted corpus rebuilt via ioc_index, and returns IocHit rows carrying the SAME BaddieVerdict the CVE axes emit (🔴 refuse, axis ioc, rank 9) — so one renderer colours every node. CLI: warehouse ioc-scan --snapshot <id> --observables <file>, non-zero exit on any hit. A CIDR containment hit reports the network that matched rather than the queried IP, so the operator sees which bad block the egress fell into. A corpus row with an unreadable kind label is skipped AND counted — a shrunken bad-list must never read as a clean sweep.

b5 — the verdict trichotomy (aligns with the Trust lens)

Every node the lens colours carries one verdict, matching the shipped Trust lens palette:

modgunn computes the verdict (the predicate above, over rows it owns); smid's Mímir lens maps verdict → colour and renders the red-marked graph. The two never duplicate the decision — modgunn is the authority on whether a node is a baddie, nornir on how the baddie is drawn.

IMPLEMENTED (2026-07-27) — src/baddie.rs, and PERSISTED. Until this pass the second half of that sentence lived only in prose: vuln_findings stored a severity and an advisory id, so the lens had to re-derive “is this refuse or hold?” from id prefixes and summary markers — exactly the duplicated decision this doc forbids. baddie::classify is now the ONE implementation, and its output is written into every vuln_findings row.

No new trichotomy enum. refuse/hold/act is the SAME three-way decision core::Decision already carries constellation-wide (what holger's gate speaks), so baddie reuses it instead of twinning a vocabulary (L5). The stored verdict column holds Decision::as_str(); the mapping the lens applies is fixed:

The three columns modgunn now owes smid

vuln_findings gained verdict, axis and rank (see vuln_findings_schema), written by the single funnel append_graded_vuln_findings — a row cannot be written without a verdict:

• verdict — Block/Warn/Pass per the table above.

• axis — which leg fired, a baddie::BaddieAxis label: present-vuln · reachable-vuln (b1) · malicious-dep (b2) · kev-cascade (b3) · yank-cascade (b3) · leak-path (b4) · ioc (b6) · clean. BaddieAxis::from_label round-trips it; an unrecognized label reads back None, never a silently-wrong axis.

• rank — 0..=9, the position on the evidence ladder, used to sort WITHIN a verdict bucket so two 🔴s still order by how strong the evidence is:

Warehouse::read_vuln_findings returns StoredFinding rows already sorted strongest-evidence-first (refuse before hold, then descending rank) — the order the lens renders in, so smid does no sorting either.

The law this axis rests on: Unknown ≠ Unreachable

baddie::Reachability has three states, not two, and the distinction is load-bearing. Unreachable means the closure was walked and the sink is not called — only then may a finding be demoted to rank 2. Unknown means no closure ran for this advisory: the present-on-disk base join did no reachability analysis at all, and even after a b1 run most advisories have no symbol-level sink to gate on (only the Go vuln DB ships ecosystem_specific.imports today). Grading those Unreachable — the tempting “if it isn't reachable it must be dead” shortcut — would silently demote every non-Go CVE in the fleet on no evidence, burying real work under demoted noise. So Unknown holds at rank 3, strictly above proven-dead at rank 2. baddie::grade_by_closure is the single place that decision is made, and the red-when-reverted test an_advisory_the_closure_never_saw_grades_unknown_not_unreachable pins it.

For the same reason Unknown never yields refuse: refuse is reserved for evidence-backed danger (reachable · in-the-wild · malware · IOC), which is the entire point of the b1 join. The two escalations that skip the reachability gate on purpose are b2 malware (the install is the payload, so “the function is never called” is irrelevant — checked FIRST so an unreachable MAL-* is never demoted) and b3 KEV (confirmed exploited in the wild is direct evidence; it refuses even with no closure, but a KEV the closure proved dead holds).

Surface

• warehouse scan --persist <snapshot> — materialize the fleet findings with verdicts at Reachability::Unknown (honest: no closure walked).
• warehouse scan --persist <snapshot> --gate-on <world_snapshot> — additionally grade each finding through the b1 call closure of a captured world snapshot (Warehouse::append_vuln_findings_gated): proven-called ⇒ 🔴, proven-dead ⇒ 🟠 demoted, no sink ⇒ Unknown hold. Prints N refuse 🔴 / M hold 🟠.
• warehouse findings-log --snapshot <id> — read the decided rows back, colour-marked, in lens order.

Open (next iterations)

• ✅ b1 materialisation DONE: warehouse scan --persist <snapshot> writes the (VEX-suppressed) fleet findings to the vuln_findings table (Warehouse::append_vuln_findings), keyed by purl (pkg:{ecosystem}/{name}@{version}) so the Mímir lens joins sbom_components directly, reading a stored verdict instead of recomputing the scan.
• ✅ b5 verdict materialisation DONE (2026-07-27): the stored row now carries the DECIDED verdict/axis/rank (baddie::classify) instead of leaving the lens to re-derive refuse-vs-hold from id prefixes — see b5 above.
• ✅ b1 demotion wired into the persist DONE (2026-07-27): warehouse scan --persist <snap> --gate-on <world_snapshot> grades every fleet finding through the A8 call closure (Warehouse::append_vuln_findings_gated) — proven-called 🔴, proven-dead 🟠 demoted, no symbol-level sink → Unknown (never demoted on no evidence).
• ✅ b3 remediation DONE: the earliest-clean-successor is computed at ingest from the registry's own version list (registry::earliest_clean_successor) and carried in the yank advisory summary — no world_snapshots query needed for the common case, since the crates.io/PyPI document already lists every version + its yank state. (A cross-generation world_snapshots lineage lookup would only add successors published after the yank doc was captured — a later refinement.)