mikronova/funnel-core

funnel-core — design

funnel-core is the pure workflow-DAG core of the nornir funnel. It was lifted out of nornir/src/funnel/ in the #22 facets-migration wave so the DAG primitives could be shared without the warehouse.

Charter

• Pure and deterministic (FC-7). Only serde-serialisable data + topological readiness + projections. Nothing that touches storage, RPC, a model or a clock. Same events + same query ⇒ bit-identical output, so every surface is golden-testable with no warehouse and no GPU.
• Append-only event log. Every mutation is one [Event]; the funnel state is a fold of the log via [Funnel::apply]. New event kinds append at the bottom; old rows keep replaying (serde #[serde(default)] / #[serde(tag = "kind")]).
• Additive, migration-safe. New fields default so every pre-existing log row reads back unchanged (ItemKind::Idea for pre-FI rows, Idea::mode = None, …).

Open vs closed enums (a new Event variant must not break a consumer)

Migration-safety on the wire is only half of it. Event was a closed enum while its own charter promised "new event kinds append at the bottom" — so adding PlanComponentWired for the wiring seam instantly broke nornir's two exhaustive matches (E0004) and, through it, dwarves --features embed-nornir. Any future variant would have done the same. The source-level contract now matches the wire contract:

• Open (#[non_exhaustive], consumers need a wildcard arm): Event, ItemKind, CardType, FunnelMode, ComponentEdgeKind, WireOrigin, UnmappedReason — vocabularies that grow by design, and whose parsers already fail open (ItemKind::parse → Idea, FunnelMode::parse → None, an unknown WireOrigin → Proposed).
• Closed (deliberately, so a new variant does stop a consumer's build until a human decides what it means): NodeStatus, PlanStatus, TriageDecision, RunOutcome, WireStatus, DagStatus, EdgeKind, Lane, HistoryStatus, Sort — complete state machines and fixed render palettes.

The better fix than a wildcard is not matching at all: Event::kind_str() is a total accessor returning byte-identical tokens to serde's #[serde(tag = "kind")] (pinned by a test over every variant), and Event::KINDS is the whole vocabulary. A consumer built on those two pays nothing for a new variant. The open-enum guarantee itself is locked by a compile_fail doc test on Event: an exhaustive match from outside the crate must not compile, so deleting #[non_exhaustive] turns that doc test red.

The seven surfaces

The one materialised [Funnel] projects into seven pure views, each its own module:

The mode and Plan-DAG critical-path extensions have their own design note: plan-dag-critical-path.md (chosen design + the mode-gate rule). The Plan-DAG ↔ Component-DAG wiring has dag-wiring-design.md (chosen design + the nornir-side handoff spec).

Why a separate crate

nornir's funnel dragged in Iceberg, the codegen planner and the demo. ordning wanted the same idea→plan→node primitives for its case-graduation seam but not the storage. Extracting the pure core lets both consume ONE definition of the funnel DAG at independent release cadences (polyrepo rule: a crate MOVE is additive) — and keeps the projection logic unit-testable in isolation.

The DAG invariant (apply keeps every plan a DAG)

[Funnel::apply] rejects events that reference unknown ids, would duplicate an edge, create a self-loop, or close a dependency cycle — an EdgeAdded from → to is refused when to already reaches from (a BFS over the plan's current edges). This closes a real stall: a cycle (a→b, b→c, c→a) is invisible to the self-loop / duplicate-edge guards because each edge is individually valid, yet once one lands, topo_ready / simulate_flow skip the whole plan forever and it permanently stalls. Guarding the closing edge at write time keeps the log a DAG, so every accepted plan stays drainable and topo_order never trips on production data. The check is O(V + E) per edge on the cold write path — off the projection hot paths.

Honesty note — the one relaxed invariant. apply still does not enforce a node-status lifecycle (e.g. Done → Pending): a NodeStatusChanged sets the status as given. Callers (nornir, the flow simulator) drive statuses monotonically, so this is a deliberately-relaxed invariant, not a gap — documented here so the log's guarantees stay honest.

The temporal seam (growth = the log's running structure)

The Plan-DAG projections above all read one materialised Funnel snapshot — a still frame. DagGrowth::from_events adds the time axis: it replays the log and records the first step + timestamp at which each node and edge entered the DAG. Because the log is append-only (there is no remove-node / remove-edge event), growth is monotone — a node once born stays, so a "first appearance" stamp is a true birth and node_count_at/edge_count_at are non-decreasing. This is deliberately not another snapshot projection and not the executor's mutable ready-set (topo_ready, which is the live dispatch frontier): it is the structural history the funnel pane animates the DAG growing along and scrubs. It is composed from the existing fold + [PlanDag::from_funnel] (no reimplemented projection logic, so it can never disagree with a snapshot) and runs on the cold render path. A rejected event births nothing — matching the store, which applys before it appends, so a real log never contains one.

Two clocks over one seam — batch and stream. from_events is the batch clock: it re-projects the whole DAG after every event, a single pass built for a cold load or a full history scrub. A live pane that wants the DAG to grow in real time can't afford to call it every tick — that re-replays the entire log each time (O(steps²) over a session). So the seam has a second, incremental entry: DagGrowth::new(sel) opens an empty stream and extend(&mut self, &Event) folds one event into it, keeping the running Funnel accumulator inside the stream, projecting the single new frame, and appending only the first-seen births — O(from_funnel) per tick, no re-fold. The design constraint is that the two paths must never diverge: they are independent implementations (batch global-sorts the births at the end; the stream appends an intra-step-sorted batch per tick), yet produce byte-identical timelines on the same log — a chosen invariant, tested as a RED-when-broken equivalence guard, that lets a renderer switch between load (batch) and live (stream) freely. The stream state (accumulated Funnel + seen sets) is an implementation detail: serde-skipped (the wire form is just the births) and excluded from PartialEq (two logs with the same births are equal however they were built — and Funnel is neither Serialize nor Eq). Because from_events retains that accumulator too, the paths compose: hydrate a pane from the log so far with a batch fold, then extend new ticks onto it.

The wiring seam (the funnel's only outward-facing graph join)

Every other surface above is a projection of the funnel and nothing else. The wiring module is the exception: it joins the Plan-DAG to a Component-DAG the funnel does not own and cannot fetch (it lives in Mímir/nornir, behind RPC and storage — everything FC-7 forbids). The seam is therefore a trait, ComponentGraph: a host implements three methods over whatever it has, and the traversal (blast_radius, cone_blast_radius) comes for free and stays pure. The in-memory ComponentDag shipped here is the offline/fixture implementation, not a second source of truth.

Two invariants keep the join honest, both tested RED-when-reverted:

• The empty answer is disambiguated. A blast radius with no affected components means either "nothing downstream" or "nobody wired this plan node". Those are different facts, so BlastRadius carries unmapped_nodes and is_complete() alongside the list — a plan node with no component binding can never be read as a successful, empty result.
• Only a Confirmed wire binds, and unknown tokens fail closed. origin and status are free-form wire tokens (the IdeaModeSet::mode precedent, so a new provenance never needs a schema bump); an unreadable token resolves to Proposed, which does not bind. A machine suggestion is never silently promoted to a fact.

Event::PlanComponentWired is the one new (additive) variant, folded last-write-wins per (plan_node, component) into Funnel::wires. There is no un-wire event — the log is append-only, so a wire is retracted with a later rejected row, which is also what lets Wiring::infer_declared stay idempotent without re-proposing a rejected suggestion. apply validates the plan side (the node must be a real card) but deliberately not the component side: the funnel does not own that namespace and a plan must be wireable ahead of the code, so an unknown component surfaces as a dangling wire in WiringReport rather than a rejected event.

Chosen design + the nornir-side handoff spec: dag-wiring-design.md.

Docs

This .nornir/ is the source of the repo's README and docs (single-source, class-tagged): design.md (this file) · guide.md (the API walk-through) · for-idiots.md (the beginner tour), plus the per-feature design notes plan-dag-critical-path.md and dag-wiring-design.md. The renderer emits the thin README + the full book from these same files.