nordisk/nornir

MCP tool testing — design + guide

The exhaustive test harness that exercises every nornir MCP tool against a temp warehouse seeded with a small, known dataset, asserts each tool's structured output matches the seed (inject-and-assert, LAW 1 — never merely "didn't panic"), and emits a machine-readable coverage manifest so there are no silent gaps. This file is a durable design note (a chapter of nornir docs book), the companion to the agent-facing airgapped-agent.md and mcp-from-claude-code.md.

Source: tests/support/mcp_harness.rs (the shared harness body — JSON-RPC plumbing, the exercise! macro, the coverage gate), tests/support/mcp_seed.rs (the seeder), tests/support/genworkspace.rs (the synthetic workspace generator). Two thin #[test] entry points call the same harness so they can never drift: tests/mcp_matrix.rs (mcp_every_tool_functionally_asserted — the matrix-wired home, runs under nornir test / EPIC L) and tests/mcp_tools.rs (mcp_every_tool_exercised_and_asserted — the standalone alias). Tools under test: src/bin/nornir-mcp.rs (NornirServer, 56 #[tool] methods, rmcp). Manifest: tests/mcp_tool_coverage.json. Authoritative per-tool reference: mcp-tools.md.

Run it

cargo test --features mcp --test mcp_matrix mcp_every_tool_functionally_asserted
# …or the standalone alias:
cargo test --features mcp --test mcp_tools

The harness self-discovers the registered tool surface over the wire (tools/list), so a newly added #[tool] that the harness forgets to cover fails the run ("LAW: no silent gaps — these registered tools were never exercised: …"). Adding a tool therefore forces adding its coverage row.

To probe a seeded workspace by hand (it stays alive 120 s, prints its paths):

cargo test --features mcp --test mcp_tools seed_only -- --ignored --nocapture

The harness writes the coverage manifest atomically (temp + rename) because cargo test may run the mcp_matrix and mcp_tools binaries concurrently and both publish the same tests/mcp_tool_coverage.json — last-writer-wins on identical bytes, never a torn file.

Why drive the real binary over stdio

The harness spawns the actual nornir-mcp binary and speaks JSON-RPC over its stdio transport, rather than constructing NornirServer in-process and calling methods directly. This exercises the genuinely registered surface — #[tool]/#[tool_router] macro registration, the JSON input schemas, argument deserialization, and the CallToolResult serialization — exactly as a real MCP client (Claude Code, Aider, opencode) sees it. An in-process call would skip all of that and could pass while the wire surface is broken.

initialize → notifications/initialized → tools/list → N× tools/call. Each tool is called twice: once with valid args (output asserted against the seed) and once with bad args (asserted to surface a clear error a weak/on-prem model can recover from — the recovery requirement).

The seed (known ground truth)

mcp_seed.rs builds a self-contained workspace on disk and seeds every warehouse table the embedded tools read from:

• A real cargo workspace — the 10-repo named holger diamond from genworkspace (Shape::NamedHolgerTree). Ground-truth edges: holger → {snippy, io-buf, codec-utils}, bench-suite → holger, decomp-* → codec-utils, io-buf deliberately outside codec-utils' blast radius. Drives repos_list, the dependency-Mímir tools (deps_of, dependents_of, affected_by_change, build_order, dep_path, dep_graph_svg, repo_overview), guard_*, the release_gate_* set, docs_*, and search.
• A syn knowledge scan of the holger crate → knowledge_*. Every generated crate defines pub fn metric() + pub const SELF_VALUE, so metric is a guaranteed hit.
• A synthetic DWARF facts snapshot for holger (no binary needed) → dwarf_*. Known edges: holger::engine::run → step → finish.
• A doc_exports row (README/md) → docs_history.
• A bench_history.jsonl with two runs → bench_history, release_gate_nexus_floor, release_gate_no_regression, regression_trace.
• A built Tantivy index over the workspace → search.

The warehouse is opened → seeded → closed before the MCP spawns: skade-katalog's catalog.redb is single-writer, so the test must release the write lock before nornir-mcp opens it embedded. The seed writes to the exact path the spawned MCP resolves via Loaded::warehouse_root() — with the home-derived default that is <HOME>/.nornir/workspaces/<name>/builds (the per-workspace layout adopted 2026-06-12), i.e. <root>/.nornir/workspaces/default/builds for the harness (config has no workspace_<name> path component → name default). Seeding the old <root>/.nornir/warehouse left every warehouse tool reading an empty table (the drift this seeder now guards against). For the same reason the harness runs the MCP with NORNIR_MCP_NO_TELEMETRY=1 — the per-call telemetry writer would otherwise hold the same redb open and starve every embedded warehouse read (repo_overview, knowledge_*, dwarf_*, docs_history, changed_since_last_release, regression_trace, …). The server now defaults telemetry off in embedded mode for exactly this reason (see NornirServer::new and the telemetry_off gate).

Coverage strategies (per tool)

Each tool's verdict is recorded with a strategy explaining how it was driven:

The coverage manifest

tests/mcp_tool_coverage.json is written on every run: one CoverageRow per registered tool — {tool, verdict, strategy, asserted, bad_args_errors, detail} plus a header (total_tools, passed, skipped, failed, generated_at). The harness asserts failed == 0, that every registered tool has exactly one row, and that the manifest invents no tool that isn't registered. The rows are shaped to stream 1:1 into a future warehouse mcp_tool_coverage table (LAW 5: persist to the warehouse) — emit the manifest now, wire the table later.

Current verdict: 56 tools — 53 passed, 3 skipped (needs-server), 0 failed.

The block_on-within-runtime trap (and the fix)

Embedded warehouse tools must not open the warehouse or drive an async skade-katalog query directly on the rmcp async worker thread. IcebergWarehouse carries its own tokio runtime and opening it (and the async table scans) calls block_on; doing that on a thread already driving the rmcp runtime panics with "Cannot start a runtime from within a runtime". The worker dies, the tools/call response is never sent, and the client hangs forever.

The fix (the established pattern across the knowledge/dwarf/docs_history tools): wrap the warehouse work in tokio::task::spawn_blocking, and drive any async query inside it via the warehouse's own runtime, wh.block_on(fut). This harness is what surfaced the trap in changed_since_last_release and regression_trace, which previously hung the whole suite; both are now off-loaded the same way. Rule of thumb: any new #[tool] that touches the embedded warehouse goes through spawn_blocking (+ wh.block_on for async queries) — never a bare IcebergWarehouse::open / .await on the worker.

Adding a tool

1. Add the #[tool] to NornirServer. If it reads the embedded warehouse, wrap it in spawn_blocking (see above).
2. Seed any data it needs in mcp_seed.rs.
3. Add an exercise!(…) (or an explicit cov.record(…) for error-path/server tools) in mcp_harness.rs with a valid-args assertion against the seed and a bad-args case. The "no silent gaps" assertion (both --test mcp_matrix and --test mcp_tools) will fail the build until you do.