nordisk/nornir

Funnel task decomposition with the compiler-as-oracle (EPIC #45)

⚠ Ownership moved (doctrine 2026-07-15, PLAN #5): the funnel ACTION lives in dwarves. The LLM-driven funnel ACTION — decompose / plan-drive / prompt-as- mother / executor-drive / autonomous kick — now lives in dwarves (dwarves/crates/dwarves/src/funnel_action.rs + dwarves-mcp: ClaudeCaller + decompose + autonom_kick). Use dwarves funnel-action. nornir's funnel ACTION verbs (funnel decompose|prompt|run|plan-from, and the DAG-mutating intake/edit verbs) are retained as still-working shims that print a deprecation banner (funnel::funnel_action_moved_banner, classified by funnel_op_is_action) — full removal is a later pass. The QUERY / READ surface stays in nornir and is unchanged: funnel show|next|history|cards|board + the viz Funnel tab (all warehouse-backed views) — nornir stays queryable. The design below documents the retained implementation.

Design + guide for src/funnel/decompose.rs: turning a funnel task into ≤20 atomic, testable subtasks, RAG-anchored to real code, then driving each through a local-LLM generate → judge → retry loop where the compiler is the accept/reject oracle. Companion to .nornir/bakeoff-judge.md (the CodegenJudge/AnswerJudge oracle, #46), .nornir/generator-backends.md (the Generator trait, #39), and .nornir/funnel-intake.md (the funnel DAG).

1. The problem it solves

A funnel item (an idea / error / test) is too big to hand to a small local model and say "write the code". The planner (src/funnel/planner.rs) already turns an item into a component-level plan from the dep graph (the blast radius), without a big model writing it. EPIC #45 goes one level finer: it breaks the task into atomic, file/fn-level subtasks, each with an acceptance check, and then lets a small local model fill in the code — but only accepts a subtask when the compiler + tests say it's right. No cloud, no big model: the model proposes, the compiler disposes.

2. The pipeline

 task text ─┬─ retrieve_context (RAG) ──→ anchors (file/fn snippets)
            │      ├ code index (symbol_facts/call_edges)  ← always (model-free)
            │      └ vector index (Embedder)               ← #[cfg(vector)]
            ├─ decompose ──────────────→ Vec<Subtask>  (≤20, each anchored)
            ├─ store_subtasks ─────────→ child funnel DAG nodes under parent
            └─ for each subtask:
                 run_subtask_loop ─ Generator → CodegenJudge oracle
                   accept (judge 1.0) → node Done
                   reject (judge 0.0/partial) → retry w/ compiler error
                                                 + more context, bounded
                   exhausted → node Failed (flagged)

It mirrors the planner's spirit — the structure is grounded in the code index, not hallucinated — but adds the codegen + judge oracle on top.

3. RAG retrieval (the anchor)

Retriever is the seam. Each leg returns ranked Anchors (symbol, file, line, kind, excerpt, score, source):

• CodeIndexRetriever over a KnowledgeView (symbol_facts + call_edges) — always present, 100% offline + model-free. It scores a symbol by lexical token overlap between the query and the symbol's name/file/signature (a name hit outranks a signature hit outranks a file hit), so "retry the http fetch" anchors onto a fn retry_fetch in http.rs.
• Vector leg (#[cfg(feature = "vector")]) — layers the embedder/ANN index on top when compiled, exactly like the planner's embedder leg. Absent on the default build, so decompose still works with neither model nor network.

retrieve(retrievers, query, k) fans out, merges, dedups by (file, symbol) (keeping the best score), and returns the global top-k — deterministic tie-breaks ((file, line)) so tests assert exact rankings.

4. Decomposition

decompose(task, retrievers, top_k, max) retrieves the top anchors and emits one Subtask per anchor (each concrete symbol a task touches is one testable unit), capped at max.min(MAX_SUBTASKS=20). Each Subtask carries: an imperative title, the target_file + anchor_symbol it touches, an acceptance description, and its own single-anchor RetrievedContext (so the generator is grounded on exactly the symbol it must edit). An empty index still yields one whole-task fallback subtask so the loop always runs.

5. The per-subtask loop (generate → judge → retry)

run_subtask_loop(parent_task, sub, generator, judge, max_rounds):

1. build_codegen_prompt — task + subtask + anchor context; on a retry it also carries the previous round's compiler/test error ("Fix it.").
2. ask the Generator (GenRequest, 1024-token cap) for a file body / patch.
3. evaluate with the AnswerJudge oracle (CodegenJudge: apply → cargo build → cargo test → Verdict{compiled, tests_passed/total, score, message}).
4. accept on the first Verdict::accepted() (compiled + every test passes) → stop; else re-feed verdict.message and retry up to max_rounds.

Returns a SubtaskVerdict (accepted/failed, the best Verdict, every Round's prompt-excerpt + verdict — the audit trail). The judge is the oracle: a non-compiling answer is a 0.0 reject, never a hard error; an Err means the generator itself failed (model missing), which a real --gen run surfaces.

6. Storing subtasks as child funnel DAG nodes

store_subtasks(store, idea_id, decomp) weaves the decomposition into the funnel as one child plan under the parent item: one node per subtask (in order, chained by dependency edges so they dispatch top-down), node_kind from the parent's ItemKind (code:write / code:fix / test:write), targets = [target_file], and the subtask's state_json stored in node params (LAW 6). apply_verdicts(...) then writes each loop verdict back as a NodeStatusChanged — accept → Done, reject → Failed (a flagged failure), with the judge's message/score in why.

7. The offline simulation

simulate(task, view, generator, judge, top_k, max_subtasks, max_rounds) runs the WHOLE loop and returns a SimulationReport (the decomposition + every verdict + the accept/total tally), with render() (the human subtask-DAG view) and to_json() (LAW 6). It needs no model and no network: pass generator("mock")? and a CodegenJudge over a fixture crate — the compiler is the oracle.

CLI

# fully offline, no model, no network — the fixture crate is created in a temp dir
nornir funnel decompose "make add() correct" --fixture

# against a real crate dir, with a real local backend, persisting the subtasks
nornir funnel decompose i-003 --repo ./mycrate --gen candle:qwen2-0.5b --store

# JSON state_json instead of the DAG view
nornir funnel decompose "…" --fixture --json

Flags: --fixture (self-contained temp crate), --repo <dir> (scan + judge a real crate), --gen <spec> (mock default → $NORNIR_GEN_BACKEND → mock), --max-subtasks (≤20), --rounds, --top-k, --store (persist child nodes + verdicts; needs task to be a funnel item id), --json.

The shipped mock Generator echoes the prompt, which is never valid Rust → it always rejects (proof the oracle isn't a rubber stamp). To see the green path without a model, run cargo run --example funnel_decompose, which uses a deterministic canned generator returning a known-good body for one subtask and a non-compiling body for another — both judged by the SAME real oracle.

8. Tests (matrix-wired, inject-assert)

12 unit tests in decompose.rs + tests/funnel_decompose.rs (aspect funnel, published as test_results rows so they show in the matrix with the real scores as metric):

Real input → real asserted output, all model-free + network-free (cargo, already required to run the tests, is the oracle).

9. Test matrix aspect: Aspect::FunnelDecompose (EPIC #45b)

The funnel decompose pipeline is wired into the nornir-testmatrix engine as its own Aspect::FunnelDecompose (label "funnel").

What it does

run_aspect(repo_root, Aspect::FunnelDecompose) shells:

cargo test --test funnel_decompose -- --test-threads=1

in the workspace root, captures output, and parses pass/fail counts from libtest lines. The metric is the number of passing pipeline properties (so the matrix grid shows the integer count, not just green/red). The aspect is in both Aspect::ALL and Aspect::DEFAULT (it's fast — all offline/no-network).

CLI

nornir test run nornir --aspects funnel

Viz: "🔬 Run decompose test" button (Funnel tab)

The Funnel tab's decompose_panel() adds a local-mode-only 🔬 Run decompose test button. On click it shells the same cargo test --test funnel_decompose in the workspace root, parses the 5 named subtask results, and shows them inline:

🔬 funnel decompose: 5/5 pass
  ✅ decompose.anchors_real_files   score=1.0
  ✅ rag.retrieves_relevant         score=1.0
  ✅ loop.accepts_good_patch        score=1.0
  ✅ loop.rejects_noncompiling      score=1.0
  ✅ simulate.end_to_end_offline    score=1.0

LAW #6: state_json["decompose"]

{
  "decompose": {
    "running": false,
    "run_count": 5,
    "pass_count": 5,
    "rows": [
      {"name": "decompose.anchors_real_files", "status": "pass", "metric": 1.0, "message": "..."},
      ...
    ]
  }
}

Tests (tests/viz_funnel_decompose.rs)

10. Files

• src/funnel/decompose.rs — the pipeline (retrieval, decompose, loop, store, simulate) + 12 unit tests.
• src/funnel/mod.rs — re-exports (decompose_task, simulate, Retriever, Subtask, SubtaskVerdict, …).
• src/bin/nornir.rs — nornir funnel decompose (DecomposeArgs, run_funnel_decompose, build_decompose_fixture).
• examples/funnel_decompose.rs — the offline demo showing both verdict paths.
• tests/funnel_decompose.rs — the funnel-aspect matrix test.
• crates/nornir-testmatrix/src/aspect.rs — Aspect::FunnelDecompose + run_funnel_decompose + parse_funnel_decompose.
• src/viz/funnel_tab.rs — decompose_panel() + state_json["decompose"] + run_decompose_test_for_test().
• tests/viz_funnel_decompose.rs — inject-assert + heavy tests for the above.