nordisk/dwarves

⚒ brokkr — the CI/build-server — design

📍 THIS DOC LIVES IN dwarves (moved from nornir/.nornir/, 2026-07-22, move=delete). The code it documents is dwarves' own (crates/dwarves/src/…), so the doc follows the code. nornir keeps only the read-only viz surface that renders this engine's output. Source links below are dwarves-repo-relative.

Class: design · Function axis: f0 (the whole: pipeline-as-data → run → historize → notify) · f1 the model + descriptor · f2 the plan (DAG + matrix) · f3 the run engine + StepRunner seam · f4 the host wiring (CI_BUILD job, real runner, trigger bridge) · f5 the read surface (5 facets + Tab::Brokkr in nornir viz; the dwarves forge CLI + badge). Strict: this file states only what brokkr is. Sibling how-to: brokkr-guide.md; zero-knowledge walkthrough: brokkr-for-idiots.md.

Named for Brokkr, the dwarf-smith who forged Mjölnir — the one who builds.

f0 — brokkr, the whole (dwarves::brokkr, crates/dwarves/src/brokkr/)

A repo declares its CI/build pipeline as DATA in a brokkr.toml: triggers, an optional build matrix, ordered stages of DAG-ordered steps, cross-project artifact chains, secrets and notify routing. brokkr discovers + validates it, topo-orders the steps into a plan, fans that plan out over the matrix, runs each step through one execution seam (cargo build_variant / a bollard container / a VM boot), historizes every run and step, and notifies on the configured outcomes. It is the twin of nornir-build-thing: pipeline-as-data is to CI what BuildVariantSet is to a single build.

The engine is the deliberately dep-light leaf crate brokkr (serde / petgraph / anyhow / chrono / sha2 only — no iceberg, arrow, eframe, gix or bollard), the dwarves::brokkr module consumed host-side by nornir. Everything heavy lives behind the StepRunner trait in the dwarves host, so a leaf repo's xtask can pull just the model + planner cheaply. Zero shell: runners are cargo/bollard Rust APIs, never sh -c.

Reuse map (what brokkr uses). The leaf mirrors nornir's own engines: the DAG is petgraph::toposort exactly like funnel-core::topo_ready; the descriptor is the twin of BuildVariantSet::discover/validate; the facet Elm cores follow build-thing's elm.rs pattern; history rows are Iceberg-shaped like build_provenance. It is used by the dwarves forge CLI, the host engine (a CI_BUILD nornir-jobs record), the monitor poll loop (the trigger bridge), and the viz (Tab::Brokkr, five facets).

f1 — the model + descriptor (pipeline as data)

A brokkr.toml is rooted at [pipeline] (PipelineFile, model.rs:18) — mirroring build-thing's [build] root. A Pipeline (model.rs:26) carries:

• trigger — Vec<Trigger> (model.rs:54). A TriggerKind (model.rs:71) is webhook / poll / cron / manual, gated by events + a branch glob.
• matrix — named axes → value lists (Matrix, model.rs:86). combinations() (model.rs:99) is the cartesian product (a BTreeMap per cell, so order is deterministic); an empty matrix yields exactly one empty cell.
• stage — Vec<Stage> (model.rs:121), each a named group of Steps.
• Step (model.rs:131): the DAG node id (name), needs (edges, may cross stages), a run: RunKind, published artifacts, an optional report (junit), and an optional when gate.
• RunKind (model.rs:156, externally tagged) is variant = "server" / container = { name, cmd } / vm = "debian". ContainerRun.cmd is an argv vector — never a shell string (model.rs:179).
• depends — Vec<ArtifactDep> (model.rs:200): a TeamCity-style cross-project chain, snapshot (build against upstream source) or artifact (consume upstream outputs), pinned or latest-green.
• secrets — a Secret (model.rs:232) is a pointer (env:NAME / vault:path), never a literal; resolved at run time, never stored in history.
• notify — on outcomes × via transports (Notify, model.rs:240).

PipelineDescriptor (descriptor.rs:18) discovers <repo_root>/brokkr.toml (discover, descriptor.rs:49) and validate() (descriptor.rs:77) is the front door — no job is ever enqueued for a structurally-broken pipeline. It rejects, with a helpful message: an empty name, no steps, a duplicate step name (the DAG id must be unique), a needs edge to an unknown step, a self-needs, an empty container image or argv, a cron trigger with no expression, a webhook/poll trigger that gates on nothing.

f2 — the plan: DAG + matrix fan-out

Dag::build (dag.rs:47) builds a petgraph::DiGraph with a need → step edge and toposorts it; a cycle is a hard error naming the offending node (dag.rs:81) — exactly how funnel-core reports. ready_after(done) (dag.rs:104) is brokkr's topo_ready: given the done-set, the steps whose every need is satisfied — the concurrent scheduler's "what can run now?". roots() (dag.rs:113) is the first independent wave.

PipelinePlan::build (plan.rs:50) fans the DAG out over matrix.combinations(): a rust × target matrix runs every step once per cell, so the host enqueues one job per (cell, step). PlannedStep::label (plan.rs:28) is the stable step[k=v,k=v] id.

f3 — the run engine + the StepRunner seam

run_pipeline (engine.rs:74) walks the topo order for one matrix cell:

• A step runs once all its needs passed. A failed step poisons its transitive dependents — they are recorded Skipped, never handed to the runner (fail-fast, engine.rs:92).
• A step whose when gate does not match the run's branch/event context is Skipped but its dependents still run — a gated skip is not a failure (engine.rs:117).
• Each step is executed through StepRunner::run (runner.rs:52), producing a StepOutcome { ok, summary, artifact_id }. The outcome and a wall-clock duration are historized as a BuildRow.

It returns a fully-historized RunReport (engine.rs:22): one PipelineRunRow + a BuildRow per step. run_matrix (engine.rs:183) runs run_pipeline once per cell, suffixing the run_id with the cell index.

The StepRunner trait is brokkr's one execution boundary — the dep-light core never links cargo/bollard/qemu. Tests supply a FakeRunner (runner.rs:62) that records order and returns scripted outcomes, so the whole planner/engine is proven without a toolchain.

The leaf-owned side models each own a pure, tested decision the host backs with real state:

f4 — host wiring (nornir-only)

The dwarves host module is the host side the leaf's seams plug into:

• run_pipeline_job (engine.rs:26) — the invariant every engine op is a nornir-jobs record: it opens a kind::CI_BUILD (jobs.rs:177) parent job (target = pipeline, workspace = project), runs the DAG through the leaf engine, stamps each BuildRow's job_ref back to the parent id, and finishes/fails the parent with brokkr_runs:<run_id> as its result_ref.
• RealRunner (runner_real.rs:25) — the real StepRunner: a Variant step calls build-thing's build_variant (runner_real.rs:52, spawns cargo build --message-format=json, content-hashes the executables, captures a provenance attestation — zero shell); container/vm steps defer to a fallback runner. DemoRunner (mod.rs:31) greens every step deterministically for the toolchain-free path.
• trigger_monitor — bridges the nornir monitor poll loop to runs: runs_for_report (trigger_monitor.rs:38) maps a FetchReport.changed project with a poll trigger to a TriggeredRun. The pure firing decision stays in the leaf (brokkr::trigger).
• store — historizes every run + step. The P1 MVP appends JSONL under <warehouse>/brokkr/ (append_history, store.rs:23; read_runs, store.rs:55); the Iceberg brokkr_runs/brokkr_steps tables are a drop-in P3 upgrade behind the same function.

f5 — the read surface: 5 facets, one tab, the CLI

The five facett Elm pure cores each hold all observable state in one serializable model, mutate only through explicit methods, and expose a state_json a viz pane + the headless robot harness read:

The results facet is Elm to the letter: BrokkrMsg::Run(name) (elm.rs:64) returns a BrokkrEffect::RunPipeline (elm.rs:76) the host executes and feeds back as RunFinished; clicking an unknown pipeline is refused with a logged reason.

Tab::Brokkr composes all five behind a sub-view selector (BrokkrView brokkr_tab.rs:39). Pressing Run (run_pipeline, brokkr_tab.rs:235) drives the pipeline through the injected StepRunner and folds the one historized result into every facet at once, assigning + releasing an agent; the DAG renders through facett_graphview::MetroView (metro_map, brokkr_tab.rs:418 — one transit line per stage, stations lit green as steps pass). state_json (brokkr_tab.rs:203) emits one slice per facet, so tests/viz_brokkr_robot.rs drives the whole tab headless.

The forge CLI now lives in the dwarves tool (the nornir brokkr verb was retired 2026-07-18, move=delete). It reads the local brokkr.toml off the filesystem — no server round-trip. Each leaf emits a CommandOutcome (the CLI twin of a facet's state_json), printed as JSON with --json or a human render.

Feature flags

The CLI, host engine, RealRunner, DemoRunner and JSONL store are always present in the nornir binary (no gate) — brokkr is a first-class subsystem, not an opt-in.