mikronova/funnel-core

funnel-core — Plan-DAG ↔ Component-DAG wiring

The funnel knows what was planned. Mímir knows what the code is. Nothing joined them, so the question "which code does this plan touch, and what is the blast radius?" had no answer. This is funnel-core's half of that join: the wiring module — a bipartite plan_node ↔ component layer, the traversal that walks it into the component graph, and the report that names every plan node the wiring does not cover.

Implements the funnel-core side of workspace_nordisk/.nornir/funnel-dag-component-wiring-design.md (P1). Additive and migration-safe: one new module, one new Event variant, one new Funnel field, one new PlanDag method. No existing type's shape or function's signature changes; every pre-existing log row replays byte-identically.

1. The three graphs

2. Edge direction — the one thing to get right

A ComponentEdge { from, to } means from depends on to: the Cargo / dep_graph_edges DSM reading, where the dependent is the source (from_repo → to_repo). So deps_of walks forward (what a component needs) and dependents_of walks backward (who needs it). blast_radius is the transitive-dependents closure — change this component and these break — the pure twin of Mimir.AffectedByChange.

This is the opposite orientation to PlanDagEdge, where from → to means "from must finish before to" (dependency first). The two graphs come from different worlds and the seam refuses to paper over it: a nornir adapter feeding dep_graph_edges rows in must orient them dependent → dependency, and the direction is locked by a test that asserts the dependency closure of a root is a different set from its blast radius — reverse the traversal and it goes RED.

Both closures are BFS with a visited set, so a cyclic component graph (call edges routinely are — mutual recursion) terminates instead of spinning.

3. The seam: ComponentGraph

funnel-core is pure (FC-7): no RPC, no storage, no clock. So it never fetches the component graph — a host implements a trait and gets the traversal for free:

pub trait ComponentGraph {
    fn has_component(&self, id: &ComponentId) -> bool;
    fn deps_of(&self, id: &ComponentId) -> Vec<ComponentId>;
    fn dependents_of(&self, id: &ComponentId) -> Vec<ComponentId>;
    fn components(&self) -> Option<Vec<ComponentId>> { None }   // provided
    fn blast_radius(&self, id: &ComponentId) -> Vec<ComponentId> { … }        // provided
    fn dependency_closure(&self, id: &ComponentId) -> Vec<ComponentId> { … }  // provided
}

Two implementations are expected: nornir's, over live Mimir.DepsOf/DependentsOf, and the in-memory ComponentDag shipped here for the offline/test/fixture path. ComponentDag is not a second source of truth — it is the materialised form of Mímir rows a caller builds when it wants the whole graph in hand.

components() defaulting to None is load-bearing, not laziness: a lazy RPC-backed graph cannot enumerate its nodes, and None says unknown, which is honestly distinct from Some(vec![]) ("the graph is empty ⇒ every component is claimed"). It flows straight through into WiringReport::unplanned: Option<Vec<ComponentId>>.

A ComponentId is deliberately an opaque String, not a funnel id_newtype: the component namespace belongs to Mímir (repo/member names — "funnel-core", "nornir"), so funnel-core neither mints these nor validates their shape.

4. Wires are events, and they fail closed

One additive variant, last-write-wins per (plan_node, component):

Event::PlanComponentWired { plan_node, component, origin, status, confidence, why, ts }

origin (drawn | inferred | declared) and status (proposed | confirmed | rejected) are free-form wire tokens, exactly like IdeaModeSet::mode, so a new provenance or lifecycle state never needs a schema bump. They are read typed via Wire::origin_kind / Wire::status_kind, which fail closed:

• an explicit status token always wins;
• an empty status falls back to the origin's natural status — drawn ⇒ confirmed (a human dragging a wire is the confirmation), everything else ⇒ proposed;
• an unknown origin falls back to proposed.

So no wire this build cannot fully understand ever binds by accident. Only Confirmed binds — a suggestion is not a mapping.

There is no un-wire event: the log is append-only, so a wire is retracted by appending a later row for the same pair with status = "rejected". That is not bookkeeping pedantry — it preserves the fact that a suggestion was considered and thrown away, which is what stops infer_declared re-proposing it forever.

Validation asymmetry, on purpose. apply rejects a wire whose plan_node is not a real card (an idea or a task node — the same card_exists check CardAssigned uses), because a wire to a step that does not exist would silently inflate a plan's wiring coverage. It does not validate the component: the funnel does not own that namespace and holds no component graph, and a plan must be wireable ahead of the code. An id the graph turns out not to hold surfaces as a dangling wire in the report instead.

5. The traversal

plan node ──▶ bound components ──▶ transitive dependents
  (wiring)      (touched)             (affected)

• Wiring::blast_radius(plan_node, &graph) — one node.
• Wiring::cone_blast_radius(&plan_dag, plan_node, &graph) — the node plus every Plan-DAG node downstream of it (PlanDag::descendants, added for this): "if I start here, what code does the rest of this plan touch, and what ripples out of it?"

PlanDag::descendants is a plain BFS rather than the topological fold bottlenecks uses, so it still answers on a malformed cyclic edge set (where topo_order fails and bottlenecks returns nothing) instead of silently reporting an empty cone.

A component that is directly touched is never also listed as affected — no double-counting. A missing (dangling) component is reported but never followed: a closure over a phantom id would be fiction.

6. The honesty contract (the point of the whole thing)

A plan node with no component binding must never silently succeed. Two mechanisms:

BlastRadius::unmapped_nodes. An empty affected list is ambiguous — it can mean "nothing downstream" or "we have no idea". The traversal therefore reports every unbound node in the cone, and is_complete() is the one-call reading. Two tests lock the distinction: an unwired node yields affected = [] and is_complete() == false; a wired leaf yields affected = [] and is_complete() == true. Collapse the two and one of them goes RED.

WiringReport. Every node of the Plan-DAG lands in exactly one of mapped / unmapped — nothing is dropped, which is what makes the unmapped list trustworthy. Each unmapped node carries a reason, ordered by how much attention it needs:

Plus the roll-ups: is_fully_mapped() (the gate), coverage(), unmapped_ids(), unmapped_tasks() (task nodes only, for callers that treat an idea card as a container), and done_but_unmapped() — the loud row: a task node that is Done yet never wired means work landed in code the plan cannot point at. That one filters to task nodes on purpose: a mother idea card reads Done as soon as it is decomposed (its Planned lane), which is not finished work, so including ideas would drown the signal.

The other direction is unplanned — components no plan node claims. A component with churn and no plan wire is unplanned work.

7. The pure inference (declared)

The funnel already records file-glob targets on every NodeAdded, and a ComponentNode carries an optional path. Wiring::infer_declared(&funnel, &component_dag, ts) matches the two and returns the PlanComponentWired events to apply+append. Three rules, in order: the glob matches the component's path; the target names a file inside the component's path (funnel-core/src owns funnel-core/src/wiring.rs); the glob matches the component's name. That double rule matters because nornir's targets field is in practice overloaded — planner.rs::plan_from_component writes a component name, decompose.rs writes a file path — and this matches both.

It emits proposals, never bindings, and skips any pair that already has a wire — including a rejected one — so re-running is idempotent and never resurrects a suggestion a human threw away. Mímir's symbol-level inference is the impure half and arrives over the same seam as inferred wires.

ts is a parameter because funnel-core reads no clock (FC-7).

8. Fan-out

None. Every function here is a single-pass BTree-ordered projection over data the caller already holds — the cold render/report path, not a hot loop, and the graphs are hundreds of nodes, not millions. If a future consumer ever needs the wiring report fanned out across many plans, it is znippy gatling — never rayon, never thread::spawn, never a hand-rolled thread::scope. funnel-core itself stays dependency-free of all three.

HANDOFF — the nornir-side change (for the nornir agent)

funnel-core owns the types, the fold and the traversal. Three things must land on the nornir side, none of which funnel-core can do (they touch storage, RPC and the workspace graph). funnel-core is already green and pushed; nothing below blocks it.

Component ids are the workspace repo/member names (WorkspaceGraph::component_names() — nornir, funnel-core, facett, …), which is exactly what ComponentId carries. No translation layer is needed.

H0 — Event is now an OPEN enum (do this first)

This spec should have predicted the break it caused, and did not. Recorded here so the next variant cannot repeat it.

Adding PlanComponentWired broke nornir (lib) with E0004 at nornir/src/warehouse/funnel.rs:310 (build_batch) and :780 (event_kind_str), and through it dwarves --features embed-nornir. The cause was not the variant — it was that funnel_core::Event was a closed enum while the crate's own charter promises "new event kinds append at the bottom". Every future variant would have done the same.

Event is therefore now #[non_exhaustive]. Every match on an Event outside funnel-core needs a wildcard arm, including matches that list every variant that exists today. The two nornir sites above need one each, and what the arm does is not a detail:

Better: stop matching. event_kind_str in warehouse/funnel.rs:779 and the eleven KIND_* consts at :76-86 can be deleted outright and replaced with funnel_core::Event::kind_str(). It is a total accessor that returns byte-identical tokens (they are serde's #[serde(tag = "kind")] variant names — the same PascalCase strings the KIND_* consts already hold, verified against them), it is locked by a test that asserts kind_str() == the serde tag for every variant, and it never needs touching again. funnel_core::Event::KINDS is the full token set for validating a stored column. Do this and H1 below is the only work a new variant ever costs nornir.

Other enums that became open in the same change — check nornir for exhaustive matches on these too, they need wildcards for the same reason: ItemKind, CardType, FunnelMode, and the new ComponentEdgeKind / WireOrigin / UnmappedReason. Deliberately left closed, because a new variant there is a real semantic break that should stop your build: NodeStatus, PlanStatus, TriageDecision, RunOutcome, WireStatus, DagStatus, EdgeKind, Lane, HistoryStatus, Sort.

H1 — persist the new event

Event::PlanComponentWired { plan_node: String, component: String, origin: String, status: String, confidence: Option<f64>, why: Option<String>, ts }.

1. nornir/src/warehouse/iceberg_schema.rs → pub fn funnel_events() -> Result<Schema>: append four optional fields after the existing keystone columns (mode, member, role, from_lane, to_lane) — plan_node: string, component: string, wire_origin: string, wire_status: string, confidence: double. Optional ⇒ migration-safe; the table auto-evolves forward on the first write via iceberg::ensure_table_schema, the same route field 28 (item_kind) took. (why reuses the existing why column; plan_node cannot reuse node_id, which is a typed NodeId column — a wire may address an i-NNN idea card.)

2. nornir/src/warehouse/funnel.rs:

   • the kind token is "PlanComponentWired" — PascalCase, matching serde's variant name and the existing KIND_* consts. (An earlier draft of this spec said plan_component_wired; that was wrong and would have made the row unreadable by parse_batch.) Better still, take H0's advice and delete the KIND_* consts in favour of Event::kind_str(), and this line stops being a decision at all;
   • add the new COL_* index constants (append only — the comment there is explicit that the indices match funnel_events field order) and the write arm in fn build_batch(table, events) -> Result<RecordBatch> + the read arm in fn parse_batch(batch) -> Result<Vec<Event>>.
   • parse_batch must read a missing/NULL wire_origin/wire_status back as "" — the empty token is meaningful (it resolves through WireOrigin::default_status), so it must not be coerced to "proposed" on the way in.

3. nornir/src/warehouse/access_scan.rs — no change: the pair ("append_event", funnel_events, Write) already covers it.

No change is needed in funnel/store.rs: Store::record already routes any Event through funnel.apply → append_event, and apply's new arm does the validation.

Rule for the next variant (this is the part the spec was missing): a new funnel_core::Event variant costs a consumer nothing if it uses Event::kind_str() and a wildcard arm, and costs it a build outage otherwise. Any future funnel-core change that adds a variant must land with this section re-read, and any consumer-side match ev must be written open from the start.

H2 — implement ComponentGraph over the workspace graph

New file nornir/src/funnel/component_graph.rs (or a mimir.rs impl block):

pub struct WorkspaceComponentGraph<'a>(pub &'a nornir_depgraph::WorkspaceGraph);

impl funnel_core::ComponentGraph for WorkspaceComponentGraph<'_> {
    fn has_component(&self, id: &ComponentId) -> bool { self.0.has_component(id.as_str()) }
    fn deps_of(&self, id: &ComponentId) -> Vec<ComponentId>       { /* edges from == id → to */ }
    fn dependents_of(&self, id: &ComponentId) -> Vec<ComponentId> { /* edges to == id → from */ }
    fn components(&self) -> Option<Vec<ComponentId>> {
        Some(self.0.component_names().into_iter().map(ComponentId::new).collect())
    }
}

Direction check (do not skip): CrossRepoEdge { from, to } in edda/crates/nornir-depgraph must be read as from depends on to. If it is the other way round, swap the two bodies above — the whole blast radius inverts otherwise. Assert it with one test over a known pair (nornir depends on funnel-core, so dependents_of("funnel-core") must contain "nornir", and deps_of("funnel-core") must not).

Do not re-derive the closure: blast_radius / dependency_closure are provided methods and already cycle-safe. Overriding blast_radius with a call to WorkspaceGraph::affected_by_change is allowed only if it returns the same set — if it does, prefer it (one implementation of impact in the workspace) and lock the agreement with a test.

H3 — surface it

• CLI (nornir/src/bin/nornir.rs, enum FunnelOp): Wire { node, component }, Unwire { node, component } (⇒ funnel_core::reject_wire), and Wiring { plan } printing WiringReport::state_json(). Wire builds funnel_core::draw_wire(..) and hands it to Store::record.
• gRPC (nornir/proto/nornir.proto, service Funnel): rpc Wiring (PlanQuery) returns (JsonResponse) whose body is WiringReport::state_json(), plus rpc WireNode (WireRequest) returns (Empty). Keep the JSON blob shape — it is what the pane reads.
• Viz (nornir/src/viz/funnel_plandag.rs / funnel_tab.rs): the pane already calls PlanDag::from_funnel(&store.funnel, sel_plan) in fn load_plan_dag. Add a sibling fn load_wiring(&self) -> Option<funnel_core::Wiring> (Wiring::from_funnel(&store.funnel)) and render WiringReport::of(&dag, &wiring, &graph). Render unmapped visibly — a plan node with no binding must look different from a wired one, or the honesty in the data dies at the pixel layer.
• Auto-propose: nornir funnel wire --infer ⇒ Wiring::infer_declared(&funnel, &dag, Utc::now()) → Store::persist_events. Idempotent; safe to run on every plan.

The dwarves/brokkr wiring canvas (plan_component_edges persistence in the P1 design doc) should persist through this event, not a private table — otherwise the funnel log and the canvas disagree about what is wired.