docs/v2/README.md

dwarves v2 — one library, four doors, the graph-package lens

dwarves is the forge and operator tool of the constellation. Since the cargo door landed, every command-line binary is a few lines around one library function, dwarves::cli::run_named, and the first repo lens, cargo lens graph-package, writes a byte-stable graph file into any Rust repo's target/dwarves/.

This document describes the code on main as it is today. Paths are repo-relative in dwarves unless another repo is named. One section is marked "in progress" and one "planned"; everything else is what runs now.

1. Glossary

word	meaning
verb	a dwarves subcommand (`dwarves doctor`, `dwarves lens graph-package`, `dwarves tune show`, …)
door	a binary that reaches the verbs: `dwarves`, `dwarves-cli`, `cargo dwarves`, `cargo lens`
forge	a running `dwarves-server`: the `dwarf.v1` gRPC service on `:7900`
fat / thin	where a verb's WORK runs: here (fat) or on a forge (thin)
DATA source	where lenses, dupes, coverage and feeds read from: a nornir/Mímir server or a local directory
Mímir	nornir's code graph; see nornir `docs/v2/mimir/README.md`
mimir dump	the Arrow IPC files `mimir-driver` writes per compiled crate (`.nodes.arrow`, `.edges.arrow`, …)
moniker	a node id of the code graph: `rust-analyzer cargo <package> <version> <descriptors>`
znippy	our archive format (one file, many entries, zstd/openzl)
modgunn	our graph vocabulary crate: `DepNode`, `DepEdge`, `Symbol`, `CallEdge`
row	a declared Build Thing artifact (`[[build.kernel]]`, `[[build.installer]]`, …) under `.nornir/profiles/<row>/`
Heimdall	our sandbox: measured seccomp profiles per program; `dwarves tune` is its tuner
edda	the repo of `nornir-build-thing`, the Build Thing engine dwarves links

2. Flow

  dwarves  dwarves-cli  cargo-dwarves  cargo-lens         dwarves-gui
     │         │             │             │                   │
     └─────────┴──────┬──────┴─────────────┘                   │ gRPC only
                      ▼                                        ▼
            dwarves::cli::run_named(bin, args)        forge :7900 ──proxy──► nornir :7878
                      │                                CodeRoom.Answer, Mimir.*, Warehouse
        ┌─────────────┼──────────────────┐
        ▼             ▼                  ▼
  lens graph-package  tune show        every other verb
        │             │                  (WORK: --server/$DWARVES_SERVER else local)
        │             └─ .nornir/profiles/<row>/kernel-needs.json
        │                (edda KernelNeeds::load, schema nornir.kernel-needs/1)
        ▼
  cargo metadata + Cargo.lock ─┐
  cargo-mimir-dump (*.edges.arrow, CALLS) ─┤ join on monikers
                                           ▼
                 target/dwarves/graph-package.znippy
                 dep_nodes · dep_edges · symbols · call_edges · graph-package.json

3. The entry points

binary	source	what it does	required feature
`dwarves`	`crates/dwarves/src/main.rs`	`dwarves::cli::run_named(env!("CARGO_BIN_NAME"), args)`	`cli`
`dwarves-cli`	same `main.rs`, second `[[bin]]`	the same; a thin-client file name	`cli`
`cargo-dwarves`	`crates/dwarves/src/bin/cargo-dwarves.rs`	`cli::cargo_dwarves_args` drops the `dwarves` argument cargo inserts, then `run_named`	`cli`
`cargo-lens`	`crates/dwarves/src/bin/cargo-lens.rs`	`cli::cargo_lens_args` inserts the `lens` verb when cargo did not, then `run_named`	`cli`
`dwarves-server`	`crates/dwarves/src/bin/dwarves-server.rs`	the forge on `:7900`; with `embed-nornir` also nornir's full gRPC stack on `:7878` in the same process	`server`

So cargo dwarves <verb> is dwarves <verb> and cargo lens <x> is dwarves lens <x>; there is one verb dispatch, run_named in crates/dwarves/src/cli.rs. The GUI (dwarves-gui/) does not go through dwarves::cli; it talks gRPC to a forge (section 7).

4. `lens graph-package`

crates/dwarves/src/lens_graph.rs; the CLI arguments in crates/dwarves/src/cli.rs (GraphPackageArgs, lens_graph_package_cmd); the tests in crates/dwarves/tests/lens_graph_package.rs.

Run in any cargo workspace:

cd <any rust repo>
cargo lens graph-package            # human report
cargo lens graph-package --json     # JSON summary on stdout (out, sha256, counts, source_graph)

4.1 Inputs

input	rule
`Cargo.toml`	`--manifest-path`, else cargo's own search from the cwd
`Cargo.lock`	with a lock the resolve runs `--locked` (a stale lock is refused, never rewritten); without one, the lock cargo has to write is kept under `target/dwarves/Cargo.lock` and removed from the repo again when the run ends, also on error
mimir dump	`--mimir-dump <dir>` reads an existing dump; else `cargo-mimir-dump` is run, found via `--mimir-bin`, then `$DWARVES_MIMIR_DUMP`, then `PATH`; `--no-source` asks for the package graph only
output dir	`--out-dir`, default `<workspace root>/target/dwarves` (always the repo's own `target/dwarves`, not `$CARGO_TARGET_DIR`)

The repo is never edited. cargo-mimir-dump is nornir's producer (nornir/crates/mimir-driver): it runs cargo check with mimir-driver as RUSTC_WRAPPER on its pinned nightly (nightly-2026-10-05, needs rustc-dev, llvm-tools-preview, rust-src). When it cannot be found the file still carries the package graph and the summary says source_graph.state = not_available with what is needed.

4.2 The two halves and the join

half	source	code
package graph	`cargo metadata` (the resolve) plus `Cargo.lock` (checksums)	`lens_graph.rs`, the resolve step
source graph	the dump's `*.edges.arrow` files, rows with `kind = "CALLS"`, deduplicated on `(from, to)` keeping the largest `count`	`lens_graph.rs` `read_call_edges`
join	the callee's package is parsed out of its moniker (`rust-analyzer cargo <package> <version> …`, `moniker_package`) and looked up in the package graph	`lens_graph.rs`

The symbols' ids and the call edges' from / to are the driver's monikers, byte for byte; nothing is re-keyed.

4.3 Output: `target/dwarves/graph-package.znippy`

One znippy archive (znippy_common::create_archive_to_vec) holding four Arrow IPC file-format tables and one JSON entry, written in a fixed order. The entries and columns are named after modgunn's graph vocabulary (modgunn::scip):

entry	modgunn	columns
`dep_nodes.arrow`	`DepNode`	`key` utf8 (`cargo:name@version`), `ecosystem` utf8 (`cargo`), `name` utf8, `version` utf8, `source_kind` utf8 (`path` · `registry` · `git`), `source` utf8 (registry/git URL, or the path relative to the workspace root), `checksum` utf8 null (from `Cargo.lock`), `member` bool
`dep_edges.arrow`	`DepEdge`	`from` utf8, `to` utf8 (keys), `kind` utf8 (`normal` · `dev` · `build`), `target` utf8 null (the `cfg(…)` or triple)
`symbols.arrow`	`Symbol`	`id` utf8 (the moniker), `display` utf8 (its descriptors), `package` utf8 (the owning `DepNode` key; `lang:<crate>` for the toolchain's own crates; `""` when the moniker names none)
`call_edges.arrow`	`CallEdge`	`from` utf8, `to` utf8 (symbol ids), `from_package` utf8, `to_package` utf8 (keys), `to_source_kind` utf8 (`path` · `registry` · `git` · `lang` · `unknown`), `count` u32
`graph-package.json`	–	`schema`, the counts, `source_graph.state` (`complete` · `not_asked` · `not_available`) with `why`, and after packing `out` and `sha256`

Every Arrow table carries schema metadata dwarves.graph_package.schema = "2" and dwarves.graph_package.table = <entry name without .arrow> (lens_graph.rs SCHEMA_VERSION, packages_schema, deps_schema, symbols_schema, calls_schema).

Only symbols named by a CALLS edge are in symbols.arrow; there are no spans, no file paths, no commit and no text in the file. The JSON summary carries the sha256 of the archive bytes.

4.4 Byte stability

Rows are sorted, there are no timestamps and no absolute paths, and the entries are written in a fixed order, so two runs over the same tree give the same bytes. A customer's file and Holger's own re-run can be compared by hash.

4.5 Tests (`crates/dwarves/tests/lens_graph_package.rs`)

test	checks
`cargo_dwarves_drops_the_subcommand_cargo_inserts`	the cargo door's argument handling
`cargo_dwarves_help_and_version_work`, `cargo_lens_help_works_both_ways`	help and `--version` through both doors
`package_graph_is_exact_and_the_repo_is_untouched`	the exact package graph with and without a lock; the repo is left as found
`a_call_into_a_dependency_is_tagged_with_that_dependency`	a call edge into a dependency carries that dependency's key, from a real dump fixture
`the_file_is_byte_stable_across_runs_and_locations`	two runs, two locations, identical bytes
`without_a_source_graph_it_says_what_is_missing`	the `not_available` message names the producer
`live_mimir_dump_on_the_fixture` (ignored)	the real `cargo-mimir-dump` on the fixture, via `$DWARVES_MIMIR_DUMP`
`real_outside_crate_itertools` (ignored)	a crates.io crate fetched and dumped end to end

Fixture: crates/dwarves/tests/fixtures/graph-package/.

4.6 What it needs to build

Feature lens-graph = arrow + znippy-common (with openzl) (crates/dwarves/Cargo.toml). No GraphBLAS, no gRPC, no nornir. A dwarves built without it answers lens graph-package with exit code 2 and the line "built without the lens-graph feature; install with --features cli,lens-graph" (cli.rs).

5. Slim install

cargo install --path crates/dwarves --no-default-features --features cli,lens-graph

gives dwarves, dwarves-cli, cargo-dwarves and cargo-lens with the lens and nothing of the forge. The default feature set is the full tool (crates/dwarves/Cargo.toml default): cli, lens-graph, cli-install, vault, embed-nornir, embed-skade, build-image, kernel-build, package-build, wasm-stage, runs-on-container, release-build. embed-nornir pulls server, warehouse, lens-feeds and nornir's full-server (the fat node, which links SuiteSparse:GraphBLAS 7 for the embedded code room). Features are fixed at install time; a verb whose feature is not built errors and names the feature.

6. `tune show` and `kernel-needs.json`

crates/dwarves/src/tune.rs, crates/dwarves/src/tune/review.rs.

dwarves tune is the Heimdall tuner, thin over edda's measure step (nornir_build_thing::measure). Verbs: probe, gen, check, approve, show. show is read-only: the review of the profiles on disk under .nornir/profiles/<row>/.

The kernel join: show also reads .nornir/profiles/<row>/kernel-needs.json through edda's validator (nornir_build_thing::syscall_join::KernelNeeds::load, schema nornir.kernel-needs/1, file name KERNEL_NEEDS_FILE). The document holds schema, row, kernel, calls (measured syscalls per program and libc with the kernel options that gate them) and options (each option with its callers). The review prints, per enabled option, which programs call through it and which other options also cover the call.

A row with a saved kernel join and no measure.toml is a join-only row (a row of another repo whose programs were measured outside Build Thing's measure step, as Holger's scan worker): tune show accepts it and renders the kernel review alone (review::has_kernel_join_only, kernel_only_review).

The writer of the file is edda's kernel needs <row> --save (edda/crates/nornir-build-thing/src/kernel_trim.rs, KernelNeeds::save in syscall_join.rs), run through a repo's <repo>-build-thing door.

When the saved join carries the possible calls (edda's kernel needs read a program's graph.json, see below), the review also has them (KernelReview::possible, outside_possible): every enabled option a possible call needs is a refusal line with the call's path (CONFIG_X: tunnr-cni can call signalx (possible, unexercised: main > on_signal), edda's PossibleVeto::render), the possible calls are printed as possible, unexercised: …, and measured calls outside the graph as measured but outside the graph (a graph gap to review): ….

6.1 The possible calls: `graph.json` and the cross-check

The MAX of a program's system calls comes from the code graph: nornir's mimir-room <lake> possible --bin <name> --out graph.json (nornir docs/v2/mimir/README.md section 10.2) walks from the program's entry points to the system boundary and writes the Graph JSON. It lives beside the program's profiles: .nornir/profiles/<row>/<program>/graph.json. A measured profile is the MIN a test saw.

reader	what it does with the max	where
`dwarves-tune-graph --graph <graph.json> --profile <probe.json\|oci.json> [--json]`	`read_graph` reads the file, `cross_check` compares: `confirmed` = seen and possible; `untested` = possible, unseen (direct paths first; the possible-but-unexercised calls a test could still drive); `gaps` = seen, not possible (a graph gap: the hint names the unresolved sites, `dlopen`, missing libraries and unresolved imports); `near_miss` = seen only on error-path runs, with the graph's view. `guides_only` is always true: the report never widens a filter	`crates/dwarves-tune-graph/src/{lib.rs,xcheck.rs,main.rs}`
`dwarves tune app run` with `graph` in the spec	the same cross-check of the run's evidence, in the app result (`AppResult::xcheck`)	`crates/dwarves/src/tune_app.rs`
`dwarves tune show`	the possible lines of the saved kernel join (above)	`crates/dwarves/src/tune/review.rs`

Mímir's file has extra fields the reader ignores (format, per call uncertain, per unresolved site callee, reason, targets). Its unresolved_sites holds the raw syscall doors and the call sites the walk widened (dyn dispatch, calls through closures and fn pointers); the gap hint counts them as unresolved sites. The binary walk inside dwarves-tune-graph (walk.rs, srcgraph/) stays the owner's stub and is not used for this.

7. dwarves-gui: the code room views

dwarves-gui/src/. Every code-graph pane asks nornir's code room through the forge and draws the answer; nothing is computed or read from disk in the GUI.

view	file	asks	draws
Coderoom tab	`coderoom.rs`	`Viz.CrateGraph`, `Coderoom.SnapshotDiff` per consumed repo; `Coderoom.Compat` and `Coderoom.WhatBreaks` come back `not_wired`	facett's shared `CoderoomPane` (`facett-coderoom-view`) behind the `RoomCoderoom` seam: one node per `name@version`, joins with dependency/call/type-use counts, the "via" line with workspace, repo@seq, commit, node and edge counts
graph source pane	`graph_source.rs`	`Knowledge.Source { moniker }` for the clicked node	the item's repo, file, span and the text of the span as the server read it at the consumed commit
room seam	`room.rs`	`CodeRoom.Answer` at `<forge> / <workspace>`	the `RoomAnswer` state chip and evidence; a non-measured state is drawn as the state, never as a number
Mímir lenses	`mimir_lens.rs`	`nornir_warehouse_embed::mimir::*` services through the forge (`RemoteLensSource::ask_once`, `drive_until_answered`)	the lens tables
kernel scene	`kernel_scene.rs`	nothing remote: the committed Kconfig fragments	a preset's symbol set as a 3-D graph

Data path: UI → forge :7900 → proxy → embedded nornir :7878 → room (crates/dwarves/src/warehouse_proxy.rs CodeRoomProxy). The forge address and token come from the GUI's settings.

8. Fat and thin today

The rule is in one module, crates/dwarves/src/source.rs (dwarves::source::select_env), called from cli.rs (selection):

question	rule
WORK (forge verbs: run, trigger, buildthing, release-build, …)	`--server <url>`, else `$DWARVES_SERVER`, else local (fat)
DATA (lenses, dupes, coverage, feeds, bench, funnel context, …)	`--mimir <url>`, else `$NORNIR_SERVER`, else the WORK url (a forge on `:7900` proxies its nornir's warehouse surface), else the local directory
local DATA directory	the verb's own flag, else `$NORNIR_WAREHOUSE`, else the workspace's nornir builds dir when the run is inside a workspace, else the repo's `target/dwarves/`
`--local`	both local, said out loud; refused beside `--server` / `--mimir`
tokens	WORK: `$DWARVES_SERVER_TOKEN`, then `~/.dwarves/token`; remote DATA: `$NORNIR_SERVER_TOKEN` when the address came from `--mimir` / `$NORNIR_SERVER`, else the forge's bearer

The mode is chosen at run time; the features are chosen at install time. cargo dwarves --server <url> <verb> is thin, plain cargo dwarves <verb> is fat. An unreachable server is an error, never a silent switch to local data.

A queued lens answer is waited for, bounded: nornir answers a lens over a room it has not built yet as queued; the CLI re-asks with a backoff until the answer is a measurement or the bound is reached (nornir_warehouse_embed::mimir::until_answered, crates/dwarves/src/room_read.rs lens_bound). The bound is $DWARVES_LENS_WAIT seconds, default 120 (room_read.rs ENV_LENS_WAIT, DEFAULT_LENS_WAIT_SECS). Past it the command prints that the lens is still queued on the server after N seconds, says to re-run later or wait longer with $DWARVES_LENS_WAIT=<secs>, and exits non-zero; a gate never passes on an answer that is not a measurement (cli.rs print_lens_and_gate).

dwarves serve runs the same bootstrap as the dwarves-server binary: with embed-nornir the embedded nornir on :7878 comes up beside the forge on :7900 (cli.rs serve_cmd).

9. In progress (phase 2 of the cargo door)

Worked on now by the dw-cargo lane; this section is updated when it merges.

dwarves::lens: every lens through one call over the selected DataSource, local (nornir's mimir_lens::answer in process) or remote, same typed answer.
The GUI's own lens switchboard (mimir_lens.rs ask_once) and re-ask loop (drive_until_answered) fold onto the CLI's room_read and nornir_warehouse_embed's until_answered; lens graph-package reaches the GUI through the same library call.
The selector feeds lens graph-package (today lens_graph_package_cmd builds its GraphOptions from the verb's own flags).

Next (planned, not built)

spans (file, lines), the commit and the member's remote in the znippy, as separate entries so the four tables stay byte-stable; an optional --embed-text.
cargo lens reach <package@version> <function> over target/dwarves/, reusing modgunn's reachability and VEX output.
GraphBLAS for cargo install dwarves on a plain machine: a bundled build or a clear error naming the system package.
Short cargo names beside cargo dwarves / cargo lens.