crates/nornir-testmatrix/src/inventory.rs
//! # inventory — *what does this constellation measure, and what does it claim?*
//!
//! A hand-written list of benches and matrix rows drifts the day after it is
//! written; that is the failure [`crate::discover`] exists to kill for the test
//! surface. This module does the same for the two things that had no verb:
//!
//! * **every bench target** — cargo `[[bench]]`, criterion, a `nornir-bench`
//! ARM (`register_bench!`), and the house `examples/bench*.rs` form; including
//! the ones in **detached manifests** that open their own `[workspace]` and
//! are therefore invisible to the repo root's `cargo metadata` (gunnar's
//! `bench/` and `bench-stock/`, edda's `bench/`, funnel-core's, ordning's).
//! * **every test-matrix row** — the appliance grid this crate DECLARES
//! ([`APPLIANCE_PRODUCTS`] × [`APPLIANCE_PROOFS`]) against what the
//! constellation actually EMITS, plus every other component/check pair some
//! test emits. A declaration nobody emits is the exact shape of a green
//! report over an empty grid, so it gets its own verdict
//! ([`State::DeclaredNotEmitted`]) instead of being counted as coverage.
//!
//! ## Why here and not in a new `nornir-inventory` crate
//!
//! The trait boundary decided it, not convenience:
//!
//! 1. The OUTPUT is [`TestResultRow`] — the row schema this crate owns and
//! every [`TestSink`](crate::TestSink) already takes. An inventory line IS a
//! matrix row (`aspect = "inventory"`). A sibling crate would depend on this
//! one for the row type anyway, and would then own half a vocabulary whose
//! other half lives here.
//! 2. The row half is *about this crate's own API*: it has to know what an
//! emission call site looks like ([`EMIT_FNS`]) and what the declared grid is
//! ([`crate::appliance`]). That knowledge cannot move to a sibling without
//! moving the model with it.
//! 3. The INPUT is `cargo metadata` parsed with `serde_json` plus a source-text
//! scan — both already in this crate ([`crate::gatedtests::parse_metadata`],
//! [`crate::citations`]), both inside its charter dep budget (`serde` +
//! `serde_json` + `anyhow`). Nothing new is pulled in; a sibling would twin
//! the parser.
//!
//! The one argument for a sibling crate was reusing `nornir-bench`'s
//! `find_bench_example` location rule. That crate pulls `chrono` + `inventory` +
//! `sysinfo`, which is exactly the weight the `[dependencies]` block here exists
//! to keep out, so the three conventional locations are restated as
//! [`BENCH_EXAMPLE_LOCATIONS`] with a citation and a test that pins them.
//!
//! ## The verb
//!
//! ```text
//! cargo run --release --example inventory -- --roots ~/git/nornir ~/git/gunnar …
//! ```
//!
//! The rendered table is a *render*; this module is the truth. Nothing here
//! reads a checked-in list of expected benches — every line comes from
//! `cargo metadata` or from the bytes of a source file.
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
use std::process::Command;
use anyhow::{anyhow, Context, Result};
use serde::{Deserialize, Serialize};
use crate::appliance::{APPLIANCE_PRODUCTS, APPLIANCE_PROOFS};
use crate::harness::now_micros;
use crate::model::{status, TestResultRow};
/// The aspect tag every inventory row carries, so a warehouse query can select
/// the inventory out of `test_results` without a second table.
pub const ASPECT_INVENTORY: &str = "inventory";
/// The three conventional `nornir-bench` entrypoints, in the order
/// `nornir_bench::runner::find_bench_example` probes them. Restated (not
/// imported) because `nornir-bench` is outside this crate's dep budget — see the
/// module docs. `bench_example_locations_match_the_harness` pins the list.
pub const BENCH_EXAMPLE_LOCATIONS: [&str; 3] =
["examples/nornir-bench.rs", "xtask/examples/nornir-bench.rs", "bench/examples/nornir-bench.rs"];
/// The functions and macros that EMIT a matrix row. `functional_status` /
/// `functional_row` are this crate's own two; `assert_emit` / `emit` / `matrix`
/// are the local wrapper macros every repo hand-rolls over them (nornir's
/// `assert_emit!` doctrine — `nornir-selftest/src/selftest.rs`); `stamp_row` is
/// the shared harness builder in [`crate::harness`], and `cell` is what it used
/// to be called AND what holger and korp still call their own hand-rolled twins
/// of it — the name is kept here for discovery, never for use.
pub const EMIT_FNS: [&str; 8] = [
"functional_status",
"functional_row",
"functional_na",
"assert_emit",
"emit",
"matrix",
"cell",
"stamp_row",
];
/// The one emitter that declares a check **does not apply** here —
/// [`functional_na`](crate::functional::functional_na)`(component, check,
/// reason)`. Its rows are discovered like any other, and then graded
/// [`State::NotApplicable`] instead of [`State::Present`]: a stage the product
/// has not got is not evidence that it climbed it.
pub const NA_FN: &str = "functional_na";
/// How many lines past its first one an emission call may span and still be
/// read.
///
/// It was 10, and 10 was measured wrong on 2026-09-10: korp's
/// `identity_survives_replacement` emitter (whose third argument is an
/// `if`/`else` over two `format!`s) and nornir's `identity_minted` emitter
/// (whose detail is a four-argument `format!`) both run to thirteen lines, so
/// the argument list never closed inside the window and BOTH rows came back
/// "no emitter in any scanned repo" — while sitting, spelled out, in the
/// product's own test. A cap that silently drops the emitters it cannot read
/// is the same failure this module exists to catch.
///
/// A cap there must still be — the scan is textual and an unbalanced paren in
/// one file must not swallow the rest of it — but it belongs where no
/// hand-written call reaches: a 60-line argument list is a code-review problem,
/// not a discovery problem.
pub const EMIT_CALL_MAX_LINES: usize = 60;
/// The word this project's vocabulary forbids for a matrix coordinate — the
/// model is *variant → features → settings/sub-settings → typ*. A rendered
/// inventory that speaks it is a bug with a name; [`forbidden_vocabulary`]
/// finds it and a test fails on it.
pub const FORBIDDEN_WORD: &str = "cell";
/// **The declaration that turns `dark` into `gated`.** An `#[ignore]`d emitter
/// is normally a row nothing writes — [`State::DeclaredNotEmitted`], red. But
/// some proofs cannot run in `cargo test` on any box and never will: enforcing
/// firmware, a signing key in a HSM, a second physical NIC. Those tests are not
/// forgotten, they are GATED, and the difference has to be declared where the
/// gate is:
///
/// ```ignore
/// #[test]
/// #[ignore = "requires: enforcing firmware on real hardware"]
/// fn secure_boot_refuses_the_unsigned_image() { … }
/// ```
///
/// The scanner reads the ignore reason. `requires: <what>` → [`State::Gated`],
/// and the `<what>` is carried onto the row so the report says what would have
/// to exist for the row to go green. No marker → still `declared-not-emitted`:
/// silence about the reason is exactly the shape this module refuses.
pub const GATE_MARKER: &str = "requires:";
/// The component-name marker the appliance grid hangs on (mirrors
/// `appliance::COMPONENT_SUFFIX`, which is private).
const APPLIANCE_SUFFIX: &str = "-appliance";
// ─── the model ─────────────────────────────────────────────────────────────
/// How a bench is declared. Four genuinely different mechanisms, each with its
/// own way of going silent.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum BenchKind {
/// A `[[bench]]` target — libtest `#[bench]`, or a `harness = false` main
/// that is not criterion.
CargoBench,
/// A target whose source drives `criterion`.
Criterion,
/// A `nornir-bench` ARM: a `Bencher` registered with `register_bench!` /
/// `register_bench_ordered!`. Its id is the string the runner logs.
NornirBenchArm,
/// The house form: an example target named `bench`/`bench_*`, or the
/// `nornir-bench` entrypoint example, run by hand or by an xtask.
HouseExample,
}
impl BenchKind {
#[must_use]
pub fn label(self) -> &'static str {
match self {
BenchKind::CargoBench => "cargo-bench",
BenchKind::Criterion => "criterion",
BenchKind::NornirBenchArm => "nornir-bench arm",
BenchKind::HouseExample => "house example",
}
}
}
/// The verdicts an inventory line carries. A thing is here, it is not here, it
/// was CLAIMED and produces nothing — which is the verdict a green-looking
/// report hides — or it is declared and honestly out of reach here.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum State {
/// Declared, and it produces something.
Present,
/// Nothing of this kind exists here. A repo with no benches yields exactly
/// one of these — never an empty table, which reads as "not looked at".
Absent,
/// Declared, and nothing emits/produces it: a row id no runnable test
/// writes, a `Bencher` nobody registers, or a bench target that does not
/// compile.
DeclaredNotEmitted,
/// Declared, its only emitter is `#[ignore]`d, **and the ignore says what
/// it requires** ([`GATE_MARKER`]). Not proof and never counted as one —
/// but not a forgotten test either, and the report must not read the same
/// for "nobody wrote it" and "no box here has enforcing firmware". The
/// requirement travels on the row.
Gated,
/// Declared, and the product's own running test says **this product has no
/// such stage**, with the reason ([`NA_FN`]). Not a hole anyone can go
/// fill, and never counted as proof.
NotApplicable,
}
impl State {
#[must_use]
pub fn label(self) -> &'static str {
match self {
State::Present => "present",
State::Absent => "absent",
State::DeclaredNotEmitted => "declared-not-emitted",
State::Gated => "gated",
State::NotApplicable => "n/a",
}
}
/// The matrix status a row of this state carries. `declared-not-emitted` is
/// RED — it is the finding, not a footnote. `gated` is `skip`: it did not
/// run, it said why, and a skip is not a pass.
#[must_use]
pub fn status(self) -> &'static str {
match self {
State::Present => status::PASS,
State::Absent | State::Gated | State::NotApplicable => status::SKIP,
State::DeclaredNotEmitted => status::FAIL,
}
}
}
/// One discovered bench.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct BenchTarget {
/// The repo checkout's directory name.
pub repo: String,
/// The cargo package it belongs to (best effort for an arm: the nearest
/// enclosing manifest).
pub krate: String,
pub kind: BenchKind,
/// The unique identifier: the arm id (`gunnar.opstorm`) for a nornir-bench
/// arm, else `<package>::<target name>`. Empty on an `absent` placeholder.
pub id: String,
/// What it measures — the first line of the file's `//!` module doc. Empty
/// when undocumented; never guessed.
pub measures: String,
/// The manifest that declares it, relative to the repo root. Empty for an
/// arm (an arm is declared by a macro, not by a manifest).
pub manifest: String,
/// The source file, relative to the repo root.
pub src: String,
/// True when the declaring manifest opens its own `[workspace]` and is
/// therefore invisible to the repo root's `cargo metadata`.
pub detached: bool,
/// `Some(false)` when a `--no-run` probe was run and failed, `Some(true)`
/// when it passed, `None` when no probe was asked for.
pub compiles: Option<bool>,
pub state: State,
/// Why the state is what it is.
pub note: String,
}
/// One test-matrix row id: where it is declared, and whether anything emits it.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct MatrixRowDecl {
pub repo: String,
/// The cargo package the declaration or the emitter lives in.
pub krate: String,
/// The test that emits it (or should), relative to the repo root. Empty
/// when nothing anywhere names the row.
pub test: String,
/// `<component>/<check>` — the coordinate a [`TestResultRow`] carries.
pub row_id: String,
/// What the row claims.
pub claim: String,
pub state: State,
pub note: String,
}
/// One emission call site found in the sources.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct EmitSite {
pub repo: String,
pub krate: String,
/// Path relative to the repo root.
pub file: String,
pub component: String,
pub check: String,
/// The enclosing `fn` name, empty when the call is not inside one.
pub test_fn: String,
/// True when the enclosing test carries `#[ignore]` — it emits nothing in a
/// default run.
pub ignored: bool,
/// What the `#[ignore]` reason declares this emitter needs, when it declares
/// one ([`GATE_MARKER`]). Empty for a live emitter and for an `#[ignore]`
/// that says nothing — the second is the case that stays red.
#[serde(default)]
pub requires: String,
/// The reason this call site gives for the check NOT APPLYING here — the
/// third argument of a [`NA_FN`] call. Empty for every other emitter.
#[serde(default)]
pub na_reason: String,
}
impl EmitSite {
#[must_use]
pub fn row_id(&self) -> String {
format!("{}/{}", self.component, self.check)
}
}
/// Everything found in one repo checkout.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct RepoInventory {
pub repo: String,
pub root: String,
pub benches: Vec<BenchTarget>,
pub rows: Vec<MatrixRowDecl>,
pub emits: Vec<EmitSite>,
/// A repo whose `cargo metadata` refused — recorded, never silently skipped.
pub errors: Vec<String>,
}
/// The whole constellation.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct Inventory {
pub repos: Vec<RepoInventory>,
}
impl Inventory {
/// Real benches (an `absent` placeholder is not one).
#[must_use]
pub fn bench_count(&self) -> usize {
self.repos.iter().flat_map(|r| r.benches.iter()).filter(|b| !b.id.is_empty()).count()
}
#[must_use]
pub fn row_count(&self) -> usize {
self.repos.iter().map(|r| r.rows.len()).sum()
}
/// Every declared row nothing emits — the cross-check against an "N/N" claim.
#[must_use]
pub fn declared_not_emitted(&self) -> Vec<&MatrixRowDecl> {
self.repos
.iter()
.flat_map(|r| r.rows.iter())
.filter(|r| r.state == State::DeclaredNotEmitted)
.collect()
}
/// Every declared row whose only emitter is `#[ignore]`d but SAYS what it
/// requires ([`GATE_MARKER`]) — honestly out of reach here, never counted
/// as proof, and never confused with a row nobody wrote.
#[must_use]
pub fn gated(&self) -> Vec<&MatrixRowDecl> {
self.repos
.iter()
.flat_map(|r| r.rows.iter())
.filter(|r| r.state == State::Gated)
.collect()
}
/// Every row a product declared **not applicable**, with its reason
/// ([`NA_FN`]).
#[must_use]
pub fn not_applicable(&self) -> Vec<&MatrixRowDecl> {
self.repos
.iter()
.flat_map(|r| r.rows.iter())
.filter(|r| r.state == State::NotApplicable)
.collect()
}
/// Rows whose coordinate is a sentence rather than a key — a row id with a
/// space in it cannot be a column heading, a grep target or a warehouse key.
#[must_use]
pub fn prose_rows(&self) -> Vec<&MatrixRowDecl> {
self.repos
.iter()
.flat_map(|r| r.rows.iter())
.filter(|r| r.row_id.contains(' '))
.collect()
}
/// Every bench a `--no-run` probe proved does not compile, plus every
/// `Bencher` impl no `register_bench!` links.
#[must_use]
pub fn broken_benches(&self) -> Vec<&BenchTarget> {
self.repos
.iter()
.flat_map(|r| r.benches.iter())
.filter(|b| b.state == State::DeclaredNotEmitted)
.collect()
}
}
// ─── cargo metadata → targets ──────────────────────────────────────────────
/// One cargo target as `cargo metadata` reports it — the subset that matters.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MetaTarget {
pub package: String,
pub kind: String,
pub name: String,
pub src_path: PathBuf,
pub manifest_path: PathBuf,
}
/// Parse `cargo metadata --no-deps --format-version 1` into every target of
/// every workspace member. Pure — the test feeds it a literal.
pub fn parse_targets(json: &str) -> Result<Vec<MetaTarget>> {
let v: serde_json::Value = serde_json::from_str(json).context("parse cargo metadata JSON")?;
let pkgs = v
.get("packages")
.and_then(|p| p.as_array())
.ok_or_else(|| anyhow!("cargo metadata has no `packages` array"))?;
let mut out = Vec::new();
for p in pkgs {
let package = p.get("name").and_then(|n| n.as_str()).unwrap_or_default().to_string();
let manifest_path =
PathBuf::from(p.get("manifest_path").and_then(|m| m.as_str()).unwrap_or_default());
let Some(targets) = p.get("targets").and_then(|t| t.as_array()) else { continue };
for t in targets {
let name = t.get("name").and_then(|n| n.as_str()).unwrap_or_default().to_string();
let src_path =
PathBuf::from(t.get("src_path").and_then(|s| s.as_str()).unwrap_or_default());
let kinds = t.get("kind").and_then(|k| k.as_array()).map(|a| {
a.iter().filter_map(|k| k.as_str()).map(str::to_string).collect::<Vec<_>>()
});
for kind in kinds.unwrap_or_default() {
out.push(MetaTarget {
package: package.clone(),
kind,
name: name.clone(),
src_path: src_path.clone(),
manifest_path: manifest_path.clone(),
});
}
}
}
Ok(out)
}
/// Ask cargo for the targets of the workspace rooted at `dir`.
pub fn workspace_targets(dir: &Path) -> Result<Vec<MetaTarget>> {
let run = |offline: bool| -> Result<std::process::Output> {
let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".into());
let mut c = Command::new(cargo);
c.args(["metadata", "--no-deps", "--format-version", "1"]);
if offline {
c.arg("--offline");
}
c.current_dir(dir)
.output()
.with_context(|| format!("spawn cargo metadata in {}", dir.display()))
};
// Offline first: the constellation is path-dep heavy and this keeps a
// survey from touching the network. Fall back rather than report a phantom
// empty repo.
let out = run(true)?;
if out.status.success() {
return parse_targets(&String::from_utf8_lossy(&out.stdout));
}
let out = run(false)?;
if !out.status.success() {
return Err(anyhow!(
"cargo metadata failed in {}: {}",
dir.display(),
String::from_utf8_lossy(&out.stderr).trim().lines().next_back().unwrap_or("")
));
}
parse_targets(&String::from_utf8_lossy(&out.stdout))
}
/// Every manifest directory in `root` cargo must be asked about separately: the
/// repo root, plus every `Cargo.toml` that opens its own `[workspace]` — a
/// DETACHED manifest (gunnar's `bench/`, `bench-stock/`, edda's `bench/`).
pub fn manifest_roots(root: &Path) -> Vec<PathBuf> {
let mut out = Vec::new();
if root.join("Cargo.toml").is_file() {
out.push(root.to_path_buf());
}
for dir in walk_dirs(root) {
if dir != *root
&& dir.join("Cargo.toml").is_file()
&& declares_workspace(&dir.join("Cargo.toml"))
{
out.push(dir);
}
}
out.sort();
out.dedup();
out
}
fn walk_dirs(root: &Path) -> Vec<PathBuf> {
let mut out = Vec::new();
let mut stack = vec![root.to_path_buf()];
while let Some(dir) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&dir) else { continue };
for e in rd.filter_map(|e| e.ok()) {
let p = e.path();
if !p.is_dir() {
continue;
}
let Some(n) = p.file_name().and_then(|n| n.to_str()) else { continue };
if skip_dir(n) {
continue;
}
out.push(p.clone());
stack.push(p);
}
}
out
}
/// Directories the walk never enters. `fixtures` is in the list on purpose: a
/// test fixture's fake bench is not a bench this constellation runs, and
/// counting it would put a lie in the inventory of the crate that owns the
/// scanner. (A fixture repo handed in as a ROOT is still walked — the skip is on
/// descending *into* a `fixtures` dir, not on being one.)
fn skip_dir(name: &str) -> bool {
matches!(name, "target" | "vendor" | "node_modules" | "old" | "fixtures")
|| name.starts_with('.')
}
/// Does this manifest open its own `[workspace]`? A textual test on purpose:
/// the crate has no TOML parser (charter dep budget) and the question is
/// line-shaped — `[workspace]` alone on a line, outside a comment.
#[must_use]
pub fn declares_workspace(manifest: &Path) -> bool {
let Ok(text) = std::fs::read_to_string(manifest) else { return false };
text.lines().any(|l| l.trim() == "[workspace]")
}
// ─── source scanning ───────────────────────────────────────────────────────
/// The first line of a file's `//!` module doc — "what it measures", straight
/// from the source rather than from a list somebody maintains.
#[must_use]
pub fn module_doc_first_line(src: &str) -> String {
for line in src.lines() {
let t = line.trim();
if t.is_empty() || t.starts_with("//!") && t.len() == 3 {
continue;
}
if let Some(rest) = t.strip_prefix("//!") {
let rest = rest.trim();
if rest.is_empty() {
continue;
}
return strip_markup(rest);
}
if t.starts_with("#!") || t.starts_with("//") {
continue;
}
break;
}
String::new()
}
/// Drop the markdown a doc line carries so a table column stays a column.
fn strip_markup(s: &str) -> String {
let mut out = String::with_capacity(s.len());
for ch in s.chars() {
match ch {
'`' | '*' | '[' | ']' => {}
_ => out.push(ch),
}
}
out.trim_start_matches('#').trim().to_string()
}
/// Does this source drive criterion?
#[must_use]
pub fn uses_criterion(src: &str) -> bool {
src.contains("criterion_group!")
|| src.contains("criterion_main!")
|| src.contains("Criterion::")
|| src.contains("use criterion")
}
/// One `nornir-bench` arm found by reading the source.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct ArmDecl {
pub id: String,
/// The `Bencher` type, when the id came from an `impl` block. Empty when the
/// id came from the registration expression itself (the `id:` field form).
pub ty: String,
/// True when a `register_bench!` / `register_bench_ordered!` links it.
/// `inventory` collects at LINK time, so an unregistered impl never runs.
pub registered: bool,
}
/// Scan one Rust source for `nornir-bench` arms.
///
/// Pure text, no `syn`: the dep budget has no parser and [`crate::citations`]
/// already establishes the scanner idiom. Two shapes are in the tree and both
/// are real, so both are looked for:
///
/// * the **unit-struct** form — `impl … Bencher for T { fn id(&self) ->
/// &'static str { "autonom.function_scan" } }` plus `register_bench!(T…)`;
/// * the **id-field** form — `register_bench!(Foo { id: "znippy.archive_…", … })`,
/// where `fn id()` returns `self.id` and the literal is in the registration.
#[must_use]
pub fn scan_bench_arms(src: &str) -> Vec<ArmDecl> {
let registered_tys = registered_types(src);
let mut out: BTreeSet<ArmDecl> = BTreeSet::new();
// Shape 1: the impl block's `fn id` literal. Detected on the MASK so a
// doc-comment ```ignore example is not read as an implementation.
let masked = mask_comments_and_strings(src);
let mlines: Vec<&str> = masked.lines().collect();
let lines: Vec<&str> = src.lines().collect();
// …and not read inside a `#[cfg(test)]` module either. A `Bencher` written
// to drive the runner's own unit tests is a FIXTURE, not an arm nobody
// registered: it must never link, and reporting it as a declaration that
// produces nothing puts a permanent red on a list of real work.
//
// MEASURED 2026-09-10: `nornir-bench`'s own `PinAware` — the fixture that
// proves the single-core pass sets and clears the pin flag — was the whole
// of edda's "declared-not-emitted" column, and the only way to clear it
// would have been to register a test double as a real bench. Same rule as
// `skip_dir`'s `fixtures`, and the same rule the appliance grid learned:
// an instrument's own sample is not evidence.
let test_mod_at = first_cfg_test_line(&mlines);
for (i, raw) in mlines.iter().enumerate() {
if test_mod_at.is_some_and(|t| i >= t) {
break;
}
let Some(ty) = impl_bencher_for(raw.trim()) else { continue };
let mut id = String::new();
for j in (i + 1)..(i + 200).min(lines.len()) {
let l = mlines.get(j).copied().unwrap_or_default().trim();
if l.starts_with("impl ") && impl_bencher_for(l).is_some() {
break;
}
if l.contains("fn id(") {
for k in j..(j + 4).min(lines.len()) {
if let Some(lit) = first_str_literal(lines[k]) {
id = lit;
break;
}
}
break;
}
}
if !id.is_empty() {
let registered = registered_tys.contains(&ty);
out.insert(ArmDecl { id, ty, registered });
}
}
// Shape 2: the `id:` FIELD of the registration expression. Deliberately the
// field and not "any identifier-shaped literal in the expression" — a
// bencher's constructor also carries SQL template names, corpus names and
// scenario labels, and taking those produced arms (`call-chain`,
// `change-radius`, `def-use`) that no runner has ever emitted. An
// over-collecting scan puts phantoms in an inventory whose whole job is to
// be believed.
for (ty, args) in registration_args(src) {
if let Some(id) = id_field(&args) {
out.insert(ArmDecl { id, ty, registered: true });
}
}
out.into_iter().collect()
}
/// The value of an `id:` field in a struct-literal registration expression:
/// `Foo { id: "znippy.archive_small_10k", n: 10_000 }` → the literal.
#[must_use]
pub fn id_field(args: &str) -> Option<String> {
let mut from = 0usize;
while let Some(at) = args[from..].find("id") {
let at = from + at;
from = at + 2;
// Must be a whole field name, not the tail of `build_id` / `run_id`.
let before_ok = at == 0
|| !args[..at]
.chars()
.next_back()
.is_some_and(|c| c.is_ascii_alphanumeric() || c == '_');
if !before_ok {
continue;
}
let rest = args[at + 2..].trim_start();
let Some(rest) = rest.strip_prefix(':') else { continue };
let rest = rest.trim_start();
if !rest.starts_with('"') {
continue;
}
if let Some(lit) = first_str_literal(rest) {
if looks_like_arm_id(&lit) {
return Some(lit);
}
}
}
None
}
/// How many registrations a file makes whose id this scan could NOT read out of
/// the source (no in-file `Bencher` impl with a literal, no `id:` field). Not a
/// silent zero: the verb prints one line per such file so the count is visible.
#[must_use]
pub fn unresolved_registrations(src: &str) -> usize {
let regs = registration_args(src);
let resolved_by_field = regs.iter().filter(|(_, a)| id_field(a).is_some()).count();
let impl_ids: BTreeSet<String> = scan_bench_arms(src)
.into_iter()
.filter(|a| !a.ty.is_empty() && a.registered)
.map(|a| a.ty)
.collect();
let resolved_by_impl = regs
.iter()
.filter(|(ty, a)| id_field(a).is_none() && impl_ids.contains(ty))
.count();
regs.len().saturating_sub(resolved_by_field + resolved_by_impl)
}
/// The line index of the first `#[cfg(test)]` — everything after it is the
/// file's own test module, and nothing there links into a release binary.
/// `None` when the file has no test module at all.
fn first_cfg_test_line(mlines: &[&str]) -> Option<usize> {
mlines.iter().position(|l| {
let t = l.trim();
t.starts_with("#[cfg(test)]") || t.starts_with("#![cfg(test)]")
})
}
/// `impl nornir_bench::api::Bencher for ScanFold {` → `ScanFold`.
fn impl_bencher_for(line: &str) -> Option<String> {
if !line.starts_with("impl ") {
return None;
}
let (head, tail) = line.split_once(" for ")?;
let trait_name = head.rsplit("::").next().unwrap_or(head).trim();
if !trait_name.ends_with("Bencher") {
return None;
}
let ty = tail
.split_whitespace()
.next()
.unwrap_or_default()
.trim_end_matches('{')
.trim_end_matches(|c: char| c == '<' || c == ',')
.trim();
(!ty.is_empty()).then(|| ty.to_string())
}
/// Every `register_bench!` / `register_bench_ordered!` call: `(type, raw args)`.
#[must_use]
pub fn registration_args(src: &str) -> Vec<(String, String)> {
// Find the call sites on the MASK (so a doc example or a test's string
// fixture is not a registration) and read the arguments out of the ORIGINAL
// at the same offsets.
let masked = mask_comments_and_strings(src);
let mut out = Vec::new();
for (needle, skip_first) in [("register_bench!", false), ("register_bench_ordered!", true)] {
let mut from = 0usize;
while let Some(at) = masked[from..].find(needle) {
let start = from + at + needle.len();
from = start;
let Some(open) = masked[start..].find('(') else { continue };
let open = start + open;
let Some(close) = balanced_close(&masked, open) else { continue };
let mut args = &src[open + 1..close];
if skip_first {
if let Some((_, tail)) = args.split_once(',') {
args = tail;
}
}
let ty = args
.trim()
.split(['(', ':', ' ', '{', ',', '<'])
.next()
.unwrap_or_default()
.trim()
.to_string();
out.push((ty, args.to_string()));
from = close;
}
}
out
}
/// The types named by a registration.
#[must_use]
pub fn registered_types(src: &str) -> BTreeSet<String> {
registration_args(src).into_iter().map(|(ty, _)| ty).filter(|t| !t.is_empty()).collect()
}
/// The index of the `)` matching the `(` at `open`.
fn balanced_close(src: &str, open: usize) -> Option<usize> {
let bytes = src.as_bytes();
let mut depth = 0i32;
let mut in_str = false;
let mut i = open;
while i < bytes.len() {
let c = bytes[i];
if in_str {
if c == b'\\' {
i += 2;
continue;
}
if c == b'"' {
in_str = false;
}
} else {
match c {
b'"' => in_str = true,
b'(' => depth += 1,
b')' => {
depth -= 1;
if depth == 0 {
return Some(i);
}
}
_ => {}
}
}
i += 1;
}
None
}
/// An arm id is `<repo>.<scenario>` by convention — a dotted or dashed
/// identifier, never a sentence and never a path.
#[must_use]
pub fn looks_like_arm_id(s: &str) -> bool {
if s.len() < 3 || s.contains(' ') || s.contains('/') {
return false;
}
if !s.contains('.') && !s.contains('-') && !s.contains('_') {
return false;
}
if !s.chars().next().is_some_and(|c| c.is_ascii_alphabetic()) {
return false;
}
if s.ends_with(".rs") || s.ends_with(".toml") || s.ends_with(".json") {
return false;
}
s.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '_' | '.' | '-'))
}
/// A copy of `src` with every comment and every string-literal BODY replaced by
/// spaces, byte offsets preserved.
///
/// The scanner needs both halves of this: it must not find `register_bench!` or
/// `impl … Bencher for T` inside a doc comment's ```ignore example (measured:
/// that alone invented 20 phantom registrations in this very file and an
/// unregistered `PinAware` arm in `nornir-bench`'s own API docs), and it must
/// not find one inside a test's string fixture. So detection runs over the
/// MASK and extraction runs over the ORIGINAL at the same offsets — which is
/// only sound because the mask is byte-for-byte the same length.
///
/// Raw strings (`r"…"`, `r#"…"#`) are handled; char literals are not masked,
/// which is safe because nothing looked for here is a single character.
#[must_use]
pub fn mask_comments_and_strings(src: &str) -> String {
let b = src.as_bytes();
let mut out: Vec<u8> = b.to_vec();
let blank = |out: &mut Vec<u8>, from: usize, to: usize| {
for x in out.iter_mut().take(to.min(b.len())).skip(from) {
if *x != b'\n' {
*x = b' ';
}
}
};
let mut i = 0usize;
while i < b.len() {
// Line comment.
if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'/' {
let end = src[i..].find('\n').map(|n| i + n).unwrap_or(b.len());
blank(&mut out, i, end);
i = end;
continue;
}
// Block comment (Rust nests them).
if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'*' {
let start = i;
let mut depth = 1;
i += 2;
while i < b.len() && depth > 0 {
if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'*' {
depth += 1;
i += 2;
} else if b[i] == b'*' && i + 1 < b.len() && b[i + 1] == b'/' {
depth -= 1;
i += 2;
} else {
i += 1;
}
}
blank(&mut out, start, i);
continue;
}
// Raw string.
if b[i] == b'r' && i + 1 < b.len() && (b[i + 1] == b'"' || b[i + 1] == b'#') {
let mut h = i + 1;
let mut hashes = 0;
while h < b.len() && b[h] == b'#' {
hashes += 1;
h += 1;
}
if h < b.len() && b[h] == b'"' {
let body = h + 1;
let mut term = String::from('"');
term.push_str(&"#".repeat(hashes));
let end = src[body..].find(&term).map(|n| body + n).unwrap_or(b.len());
blank(&mut out, body, end);
i = end + term.len();
continue;
}
}
// Ordinary string.
if b[i] == b'"' {
let body = i + 1;
let mut j = body;
while j < b.len() {
if b[j] == b'\\' {
j += 2;
continue;
}
if b[j] == b'"' {
break;
}
j += 1;
}
blank(&mut out, body, j.min(b.len()));
i = j + 1;
continue;
}
i += 1;
}
String::from_utf8(out).unwrap_or_else(|_| src.to_string())
}
/// The first `"…"` literal on a line.
fn first_str_literal(line: &str) -> Option<String> {
str_literals(line).into_iter().next()
}
/// Every `"…"` literal in a fragment.
fn str_literals(s: &str) -> Vec<String> {
let mut out = Vec::new();
let chars: Vec<char> = s.chars().collect();
let mut i = 0;
while i < chars.len() {
if chars[i] == '"' {
let mut j = i + 1;
let mut lit = String::new();
while j < chars.len() && chars[j] != '"' {
if chars[j] == '\\' {
j += 1;
if j >= chars.len() {
break;
}
}
lit.push(chars[j]);
j += 1;
}
out.push(lit);
i = j + 1;
} else {
i += 1;
}
}
out
}
// ─── matrix-row emission scanning ──────────────────────────────────────────
/// One emission call site as the source scan found it, before it is attached to
/// a repo.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct ScannedEmit {
pub component: String,
pub check: String,
/// The enclosing `fn`, empty when the call is not inside one.
pub test_fn: String,
/// The enclosing `fn` carries `#[ignore]`.
pub ignored: bool,
/// What that `#[ignore]` declared it requires ([`GATE_MARKER`]), empty when
/// it declared nothing.
pub requires: String,
/// The reason a [`NA_FN`] call gives for the check not applying, empty for
/// every other callee.
pub na_reason: String,
}
/// Scan one source for emission call sites: `(component, check)` pairs, the
/// enclosing `fn`, whether that `fn` is `#[ignore]`d, and — when it is — what
/// its ignore reason says it REQUIRES.
///
/// The rule is the shape the API forces — `functional_status(component, check,
/// ok, detail)` and every local `assert_emit!` wrapper over it take two string
/// literals first. A call whose first two arguments are not literals is not
/// guessed at; it is simply not a discovered row.
#[must_use]
pub fn scan_emit_sites(src: &str) -> Vec<ScannedEmit> {
let masked = mask_comments_and_strings(src);
let mlines: Vec<&str> = masked.lines().collect();
let lines: Vec<&str> = src.lines().collect();
let mut out: Vec<ScannedEmit> = Vec::new();
let mut cur_fn = String::new();
let mut pending_ignore = false;
let mut pending_requires = String::new();
let mut cur_ignored = false;
let mut cur_requires = String::new();
for (i, mraw) in mlines.iter().enumerate() {
let line = mraw.trim();
if line.starts_with("#[ignore") || line.starts_with("#[cfg_attr") && line.contains("ignore")
{
pending_ignore = true;
// The reason is a string literal, so the MASK has blanked it —
// read it out of the original line at the same index.
pending_requires =
ignore_requirement(lines.get(i).copied().unwrap_or_default()).unwrap_or_default();
continue;
}
if let Some(rest) = line.strip_prefix("fn ").or_else(|| {
line.strip_prefix("pub fn ").or_else(|| line.strip_prefix("async fn "))
}) {
cur_fn = rest.split(['(', '<']).next().unwrap_or_default().trim().to_string();
cur_ignored = pending_ignore;
cur_requires = std::mem::take(&mut pending_requires);
pending_ignore = false;
continue;
}
if line.starts_with("#[") {
continue;
}
if line.starts_with("//") {
continue;
}
let Some((call, callee)) = emit_call_at(line) else { continue };
// Detection ran on the mask; the ARGUMENTS come from the original line
// at the same offsets. Widen to the whole statement so a multi-line call
// still yields both coordinates.
let raw = lines.get(i).copied().unwrap_or_default();
let off = raw.len() - raw.trim_start().len() + call;
let mut window = String::from(&raw[off.min(raw.len())..]);
for l in lines.iter().skip(i + 1).take(EMIT_CALL_MAX_LINES) {
if window.matches('(').count() <= window.matches(')').count() {
break;
}
window.push(' ');
window.push_str(l.trim());
}
// The first TWO TOP-LEVEL arguments must each be a bare string literal.
// Taking "the first two literals anywhere in the call" is wrong and was
// measurably wrong: where the component is a variable, it paired the
// CHECK with the DETAIL and invented rows like
// `identity_minted/pubkey={key1}`. A call whose coordinate is computed
// is not guessed at — it is simply not a discovered row.
let Some(args) = top_level_args(&window) else { continue };
// WHICH arguments carry the coordinate depends on the callee.
// `functional_status(component, check, ok, detail)` puts it first;
// `harness::cell(run_id, repo, component, aspect, test_name, …)` puts it
// at 2 and 4. Reading "the first two" out of a `cell` call produced the
// row `run-1/holger` — a run id and a repo name, which is not a
// coordinate at all.
let (ci, ki) =
if callee == "cell" || callee == "stamp_row" { (2usize, 4usize) } else { (0usize, 1usize) };
if args.len() <= ki {
continue;
}
let (Some(component), Some(check)) = (bare_literal(&args[ci]), bare_literal(&args[ki]))
else {
continue;
};
if component.is_empty() || check.is_empty() || component.contains(' ') {
continue;
}
out.push(ScannedEmit {
component,
check,
test_fn: cur_fn.clone(),
ignored: cur_ignored,
requires: if cur_ignored { cur_requires.clone() } else { String::new() },
na_reason: if callee == NA_FN {
args.get(2).and_then(|a| bare_literal(a)).unwrap_or_else(|| "—".to_string())
} else {
String::new()
},
});
}
out.sort();
out.dedup();
out
}
/// The `requires: <what>` an `#[ignore = "…"]` declares, if it declares one.
///
/// Only the ignore's own reason is read — a test that wants the `gated` grade
/// has to say so at the gate, not in a comment somewhere near it.
#[must_use]
pub fn ignore_requirement(attr_line: &str) -> Option<String> {
let lit = str_literals(attr_line).into_iter().next()?;
let at = lit.find(GATE_MARKER)?;
let what = lit[at + GATE_MARKER.len()..].trim();
(!what.is_empty()).then(|| what.to_string())
}
/// Split a call's argument list into top-level arguments. `text` must start at
/// the callee (`functional_status("a", "b", …)`); returns `None` when the call's
/// parentheses never close inside the window.
#[must_use]
pub fn top_level_args(text: &str) -> Option<Vec<String>> {
let open = text.find('(')?;
let close = balanced_close(text, open)?;
let inner = &text[open + 1..close];
let mut out = Vec::new();
let mut depth = 0i32;
let mut in_str = false;
let mut cur = String::new();
let mut chars = inner.chars().peekable();
while let Some(c) = chars.next() {
if in_str {
cur.push(c);
if c == '\\' {
if let Some(n) = chars.next() {
cur.push(n);
}
} else if c == '"' {
in_str = false;
}
continue;
}
match c {
'"' => {
in_str = true;
cur.push(c);
}
'(' | '[' | '{' => {
depth += 1;
cur.push(c);
}
')' | ']' | '}' => {
depth -= 1;
cur.push(c);
}
',' if depth == 0 => {
out.push(cur.trim().to_string());
cur = String::new();
}
_ => cur.push(c),
}
}
if !cur.trim().is_empty() {
out.push(cur.trim().to_string());
}
Some(out)
}
/// The content of `arg` iff the whole argument is one plain string literal —
/// not `format!(…)`, not a `const`, not `&x`.
#[must_use]
pub fn bare_literal(arg: &str) -> Option<String> {
let t = arg.trim();
if t.len() < 2 || !t.starts_with('"') || !t.ends_with('"') {
return None;
}
let lits = str_literals(t);
// Exactly one literal spanning the whole argument.
(lits.len() == 1).then(|| lits.into_iter().next().unwrap_or_default())
}
/// Byte offset (into the trimmed line) and name of an emission callee, if any.
/// Requires the name to be a whole identifier followed by `(` or `!(` so
/// `cell(` matches and `parcell(` does not.
fn emit_call_at(line: &str) -> Option<(usize, &'static str)> {
let mut best: Option<(usize, &'static str)> = None;
for f in EMIT_FNS {
let mut from = 0usize;
while let Some(at) = line[from..].find(f) {
let at = from + at;
let before_ok = at == 0
|| !line[..at]
.chars()
.next_back()
.is_some_and(|c| c.is_ascii_alphanumeric() || c == '_');
let rest = &line[at + f.len()..];
let after_ok = rest.starts_with('(') || rest.starts_with("!(");
if before_ok && after_ok && best.is_none_or(|(b, _)| at < b) {
best = Some((at, f));
}
from = at + f.len();
}
}
best
}
// ─── the walk ──────────────────────────────────────────────────────────────
/// Walk one repo checkout: its benches and its emission sites. The declared
/// matrix rows are resolved afterwards, by [`resolve_rows`], because a row
/// declared for one product may be emitted from another repo.
pub fn scan_repo(root: &Path, probe_compile: bool) -> RepoInventory {
let repo = root.file_name().and_then(|n| n.to_str()).unwrap_or_default().to_string();
let mut inv = RepoInventory {
repo: repo.clone(),
root: root.display().to_string(),
benches: Vec::new(),
rows: Vec::new(),
emits: Vec::new(),
errors: Vec::new(),
};
// 1. The manifests — including the detached ones.
let mut seen_src: BTreeSet<PathBuf> = BTreeSet::new();
for md in manifest_roots(root) {
let detached = md != *root;
let targets = match workspace_targets(&md) {
Ok(t) => t,
Err(e) => {
inv.errors.push(format!("{}: {e}", rel(root, &md)));
continue;
}
};
for t in targets {
let is_bench = t.kind == "bench";
let is_house = t.kind == "example"
&& (t.name == "bench"
|| t.name.starts_with("bench_")
|| t.name.starts_with("bench-")
|| t.name == "nornir-bench"
|| t.name.ends_with("-bench"));
if !is_bench && !is_house {
continue;
}
if !seen_src.insert(t.src_path.clone()) {
continue;
}
let src = std::fs::read_to_string(&t.src_path).unwrap_or_default();
let kind = if uses_criterion(&src) {
BenchKind::Criterion
} else if is_bench {
BenchKind::CargoBench
} else {
BenchKind::HouseExample
};
let note = if t.name == "nornir-bench" || t.name.ends_with("-bench") {
"nornir-bench entrypoint — its arms are listed as their own rows".to_string()
} else {
String::new()
};
inv.benches.push(BenchTarget {
repo: repo.clone(),
krate: t.package.clone(),
kind,
id: format!("{}::{}", t.package, t.name),
measures: module_doc_first_line(&src),
manifest: rel(root, &t.manifest_path),
src: rel(root, &t.src_path),
detached,
compiles: None,
state: State::Present,
note,
});
}
}
// 2. The ARMS and the EMISSIONS — one pass over the tree.
for src_path in rust_sources(root) {
let Ok(src) = std::fs::read_to_string(&src_path) else { continue };
let krate = nearest_package(root, &src_path);
let relp = rel(root, &src_path);
let doc = module_doc_first_line(&src);
if src.contains("register_bench") || src.contains("Bencher") {
for arm in scan_bench_arms(&src) {
let (state, note) = if arm.registered {
(State::Present, format!("register_bench!({})", arm.ty))
} else {
(
State::DeclaredNotEmitted,
format!(
"impl Bencher for {} — no register_bench!, inventory links nothing",
arm.ty
),
)
};
inv.benches.push(BenchTarget {
repo: repo.clone(),
krate: krate.clone(),
kind: BenchKind::NornirBenchArm,
id: arm.id,
measures: doc.clone(),
manifest: String::new(),
src: relp.clone(),
detached: false,
compiles: None,
state,
note,
});
}
// Registrations whose id this scan could not read. Reported as ONE
// line per file rather than dropped: a bench the inventory cannot
// name is a hole in the inventory, and a hole nobody prints is
// indistinguishable from no hole at all.
let unresolved = unresolved_registrations(&src);
if unresolved > 0 {
inv.benches.push(BenchTarget {
repo: repo.clone(),
krate: krate.clone(),
kind: BenchKind::NornirBenchArm,
id: format!("{relp}::<unnamed>"),
measures: doc.clone(),
manifest: String::new(),
src: relp.clone(),
detached: false,
compiles: None,
state: State::Present,
note: format!(
"{unresolved} register_bench! call(s) whose id is not a literal in this file — run the arm to learn its id"
),
});
}
}
if src.contains("functional_status")
|| src.contains("assert_emit")
|| src.contains("TestResultRow")
|| src.contains("nornir_testmatrix")
{
for e in scan_emit_sites(&src) {
inv.emits.push(EmitSite {
repo: repo.clone(),
krate: krate.clone(),
file: relp.clone(),
component: e.component,
check: e.check,
test_fn: e.test_fn,
ignored: e.ignored,
requires: e.requires,
na_reason: e.na_reason,
});
}
}
}
if probe_compile {
probe_bench_compile(root, &mut inv);
}
// 3. A repo with nothing is one `absent` line, never an empty table.
if inv.benches.is_empty() {
inv.benches.push(BenchTarget {
repo: repo.clone(),
krate: String::new(),
kind: BenchKind::CargoBench,
id: String::new(),
measures: String::new(),
manifest: String::new(),
src: String::new(),
detached: false,
compiles: None,
state: State::Absent,
note: "no bench target of any kind under this checkout".into(),
});
}
inv.benches.sort_by(|a, b| (&a.krate, &a.id, &a.src).cmp(&(&b.krate, &b.id, &b.src)));
inv.benches.dedup_by(|a, b| a.id == b.id && a.src == b.src && a.kind == b.kind);
inv.emits.sort();
inv.emits.dedup();
inv
}
impl PartialOrd for EmitSite {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}
impl Ord for EmitSite {
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
(&self.repo, &self.component, &self.check, &self.file, &self.test_fn).cmp(&(
&other.repo,
&other.component,
&other.check,
&other.file,
&other.test_fn,
))
}
}
/// Resolve the DECLARED appliance grid against every emission found anywhere in
/// the constellation, and attach each row to the product's repo.
///
/// The declaration is [`APPLIANCE_PRODUCTS`] × [`APPLIANCE_PROOFS`] — 40 rows —
/// and it is the only central declaration in the suite. Anything else a repo
/// emits is reported as its own `present` row under the repo that emits it.
pub fn resolve_rows(inv: &mut Inventory) {
let products: Vec<&str> = APPLIANCE_PRODUCTS.to_vec();
let proofs: Vec<&str> = APPLIANCE_PROOFS.to_vec();
resolve_rows_with(inv, &products, &proofs);
}
/// [`resolve_rows`] against a roster given by the caller. The seam exists so a
/// fixture constellation can be asserted byte-for-byte against a two-row grid
/// instead of the canonical forty — and so a future roster (the chain matrix's
/// forms, say) can reuse the same resolution.
pub fn resolve_rows_with(inv: &mut Inventory, products: &[&str], proofs: &[&str]) {
// Index every emission by (component, check).
let mut by_row: BTreeMap<String, Vec<EmitSite>> = BTreeMap::new();
for repo in &inv.repos {
for e in &repo.emits {
by_row.entry(e.row_id()).or_default().push(e.clone());
}
}
let scanned: BTreeSet<String> = inv.repos.iter().map(|r| r.repo.clone()).collect();
for product in products.iter().copied() {
for proof in proofs.iter().copied() {
// A product's emitters may name the bare component or a form
// (`gunnar-appliance/container`), so match on the prefix.
let prefix = format!("{product}{APPLIANCE_SUFFIX}");
let hits: Vec<&EmitSite> = by_row
.iter()
.filter(|(k, _)| {
k.starts_with(&prefix) && k.rsplit('/').next().is_some_and(|c| c == proof)
})
.flat_map(|(_, v)| v.iter())
.collect();
// A proof counts only when the PRODUCT'S OWN repo emits it.
//
// MEASURED 2026-09-10, and it is the whole reason this rule exists:
// without it the grid read holger 8/8 GREEN, and every one of those
// eight "emitters" was a unit-test fixture inside
// `nornir-testmatrix/src/appliance.rs` — the instrument's own sample
// rows, counted as the product's evidence. That is the same shape as
// the claim this inventory was built to check, one level down.
let live: Vec<&&EmitSite> = hits
.iter()
.filter(|e| !e.ignored && e.repo == product && e.na_reason.is_empty())
.collect();
let na: Vec<&&EmitSite> =
hits.iter().filter(|e| !e.ignored && e.repo == product && !e.na_reason.is_empty()).collect();
let foreign: Vec<&&EmitSite> =
hits.iter().filter(|e| !e.ignored && e.repo != product).collect();
let (state, test, krate, note) = if let Some(e) = live.first() {
(
State::Present,
e.file.clone(),
e.krate.clone(),
format!("emitted by {}", if e.test_fn.is_empty() { "—" } else { &e.test_fn }),
)
} else if let Some(e) = na.first() {
// The product's own running test says this rung is not one it
// has. A claim somebody made out loud, with a reason — the one
// thing that is neither evidence nor a hole.
(
State::NotApplicable,
e.file.clone(),
e.krate.clone(),
format!("{} — declared by {}", e.na_reason, e.test_fn),
)
} else if let Some(e) = hits.iter().find(|e| e.repo == product && !e.requires.is_empty())
{
// The product's own test declares what the rung needs. Not
// proof — but not a forgotten test either, and the report must
// say which.
(
State::Gated,
e.file.clone(),
e.krate.clone(),
format!("{GATE_MARKER} {} — declared by {}", e.requires, e.test_fn),
)
} else if let Some(e) = foreign.first() {
(
State::DeclaredNotEmitted,
format!("{}/{}", e.repo, e.file),
e.krate.clone(),
format!(
"the only emitter is outside {product} — {}/{} ({}); another repo's fixture is not this product's proof",
e.repo,
e.file,
if e.test_fn.is_empty() { "—" } else { &e.test_fn }
),
)
} else if let Some(e) = hits.first() {
(
State::DeclaredNotEmitted,
e.file.clone(),
e.krate.clone(),
format!("only emitter is #[ignore]d: {}", e.test_fn),
)
} else if scanned.contains(product) {
(
State::DeclaredNotEmitted,
String::new(),
String::new(),
"declared by the appliance roster, no emitter in any scanned repo".to_string(),
)
} else {
(
State::DeclaredNotEmitted,
String::new(),
String::new(),
format!("declared by the appliance roster, {product} not among the scanned roots"),
)
};
let row = MatrixRowDecl {
repo: product.to_string(),
krate,
test,
row_id: format!("{prefix}/{proof}"),
claim: proof_claim(proof).to_string(),
state,
note,
};
match inv.repos.iter_mut().find(|r| r.repo == product) {
Some(r) => r.rows.push(row),
None => {
inv.repos.push(RepoInventory {
repo: product.to_string(),
root: String::new(),
benches: Vec::new(),
rows: vec![row],
emits: Vec::new(),
errors: vec![format!("{product} is a declared appliance product but was not among the scanned roots")],
});
}
}
}
}
// Every other emitted component/check pair, under the repo that emits it.
let declared: BTreeSet<String> = inv
.repos
.iter()
.flat_map(|r| r.rows.iter())
.map(|r| r.row_id.clone())
.collect();
for repo in inv.repos.iter_mut() {
let mut extra: BTreeMap<String, EmitSite> = BTreeMap::new();
for e in &repo.emits {
let id = e.row_id();
if declared.contains(&id) {
continue;
}
extra.entry(id).or_insert_with(|| e.clone());
}
for (id, e) in extra {
// A row whose component names ANOTHER product is this repo talking
// about someone else's appliance — true as a fact about the source,
// misleading as a line of evidence. Say which it is.
let cross = products
.iter()
.find(|p| **p != repo.repo && id.starts_with(&format!("{p}{APPLIANCE_SUFFIX}")));
let mut note = if e.ignored && !e.requires.is_empty() {
format!("{GATE_MARKER} {} — declared by {}", e.requires, e.test_fn)
} else if e.ignored {
format!("only emitter is #[ignore]d: {}", e.test_fn)
} else if !e.na_reason.is_empty() {
format!("{} — declared by {}", e.na_reason, e.test_fn)
} else {
format!("emitted by {}", if e.test_fn.is_empty() { "—" } else { &e.test_fn })
};
if let Some(p) = cross {
note.push_str(&format!(
" · names {p}'s appliance from inside {} — not {p}'s evidence",
repo.repo
));
}
// A coordinate is a KEY. korp and ordning pass whole sentences as
// the `check` argument, which makes the row id unusable as one and
// breaks any column that tries to hold it. Reported, not silently
// reshaped: the id stays verbatim because it is what the row
// actually carries.
if id.contains(' ') {
note.push_str(" · the coordinate carries prose, not a key");
}
repo.rows.push(MatrixRowDecl {
repo: repo.repo.clone(),
krate: e.krate.clone(),
test: e.file.clone(),
row_id: id,
claim: String::new(),
state: match () {
// A cross-repo claim is red whatever the emitter's own
// grade: it is not this repo's row to make.
() if cross.is_some() => State::DeclaredNotEmitted,
() if e.ignored && !e.requires.is_empty() => State::Gated,
() if e.ignored => State::DeclaredNotEmitted,
() if !e.na_reason.is_empty() => State::NotApplicable,
() => State::Present,
},
note,
});
}
repo.rows.sort_by(|a, b| (&a.row_id, &a.test).cmp(&(&b.row_id, &b.test)));
repo.rows.dedup_by(|a, b| a.row_id == b.row_id);
}
}
/// What each rung of the appliance ladder claims — lifted from
/// [`APPLIANCE_PROOFS`]' own doc so the claim column is not a second copy.
fn proof_claim(proof: &str) -> &'static str {
match proof {
"identity_ready" => "the server binary ran the ceremony (host form)",
"identity_minted" => "the container robot minted an identity",
"identity_survives_replacement" => "identity survives container replacement",
"identity_survives_upgrade" => "identity survives an image upgrade",
"tunnel_up_traffic_crossed" => "handshake seen and at least one packet sealed",
"ping_through_tunnel" => "ICMP echo through the tunnel, both directions",
"install_ok" => "the installer ISO laid the product onto a disk under KVM",
"secure_boot_armed" => "enforcing firmware boots the signed installer, refuses the unsigned",
_ => "",
}
}
/// `cargo bench --no-run` per manifest that owns a bench target — the question
/// "is this declared bench even buildable?". Expensive; opt-in.
fn probe_bench_compile(root: &Path, inv: &mut RepoInventory) {
let manifests: BTreeSet<String> =
inv.benches.iter().filter(|b| !b.manifest.is_empty()).map(|b| b.manifest.clone()).collect();
let mut verdict: BTreeMap<String, bool> = BTreeMap::new();
for m in manifests {
let full = root.join(&m);
let dir = full.parent().map(Path::to_path_buf).unwrap_or_else(|| root.to_path_buf());
let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".into());
let ok = Command::new(cargo)
.args(["bench", "--no-run", "--manifest-path"])
.arg(&full)
.current_dir(&dir)
.output()
.map(|o| o.status.success())
.unwrap_or(false);
verdict.insert(m, ok);
}
for b in inv.benches.iter_mut() {
let Some(ok) = verdict.get(&b.manifest) else { continue };
b.compiles = Some(*ok);
if !*ok {
b.state = State::DeclaredNotEmitted;
if b.note.is_empty() {
b.note = "cargo bench --no-run failed for this manifest".into();
}
}
}
}
fn rust_sources(root: &Path) -> Vec<PathBuf> {
let mut out = Vec::new();
let mut dirs = walk_dirs(root);
dirs.push(root.to_path_buf());
for dir in dirs {
let Ok(rd) = std::fs::read_dir(&dir) else { continue };
for e in rd.filter_map(|e| e.ok()) {
let p = e.path();
if p.is_file() && p.extension().is_some_and(|x| x == "rs") {
out.push(p);
}
}
}
out.sort();
out
}
/// The package a source belongs to: the nearest ancestor `Cargo.toml`'s
/// `name = "…"`. Textual for the same reason [`declares_workspace`] is.
fn nearest_package(root: &Path, src: &Path) -> String {
let mut dir = src.parent();
while let Some(d) = dir {
let man = d.join("Cargo.toml");
if man.is_file() {
if let Ok(text) = std::fs::read_to_string(&man) {
for line in text.lines() {
let t = line.trim();
if let Some(rest) = t.strip_prefix("name") {
if let Some(rest) = rest.trim_start().strip_prefix('=') {
if let Some(lit) = first_str_literal(rest) {
return lit;
}
}
}
}
}
}
if d == root {
break;
}
dir = d.parent();
}
String::new()
}
fn rel(root: &Path, p: &Path) -> String {
p.strip_prefix(root).unwrap_or(p).display().to_string()
}
/// Inventory every root, in the order given, then resolve the declared grid.
pub fn scan_roots(roots: &[PathBuf], probe_compile: bool) -> Inventory {
let mut inv = Inventory { repos: roots.iter().map(|r| scan_repo(r, probe_compile)).collect() };
resolve_rows(&mut inv);
inv
}
/// [`scan_roots`] against a caller-given roster — the fixture door.
pub fn scan_roots_with(
roots: &[PathBuf],
probe_compile: bool,
products: &[&str],
proofs: &[&str],
) -> Inventory {
let mut inv = Inventory { repos: roots.iter().map(|r| scan_repo(r, probe_compile)).collect() };
resolve_rows_with(&mut inv, products, proofs);
inv
}
// ─── rendering ─────────────────────────────────────────────────────────────
/// The crisp three-column table: col-1 a short unique identifier, col-2 the
/// status, col-3 the technical detail for the rest of the line.
#[must_use]
pub fn render(inv: &Inventory) -> String {
let mut out = String::new();
out.push_str("SUMMARY\n");
for repo in &inv.repos {
out.push_str(&line(&repo.repo, repo_state(repo), &repo_summary(repo)));
}
out.push('\n');
out.push_str("BENCHES\n");
for repo in &inv.repos {
for b in &repo.benches {
if b.id.is_empty() {
out.push_str(&line(&repo.repo, State::Absent, &b.note));
continue;
}
let mut detail = format!("{} · {}", b.kind.label(), b.src);
if b.detached {
detail.push_str(" · detached manifest");
}
if !b.measures.is_empty() {
detail.push_str(" · ");
detail.push_str(&b.measures);
}
if !b.note.is_empty() {
detail.push_str(" · ");
detail.push_str(&b.note);
}
out.push_str(&line(&format!("{}·{}", repo.repo, b.id), b.state, &detail));
}
}
out.push_str("\nROWS\n");
let mut any = false;
for repo in &inv.repos {
for r in &repo.rows {
any = true;
let mut detail = if r.test.is_empty() { "—".to_string() } else { r.test.clone() };
if !r.claim.is_empty() {
detail.push_str(" · ");
detail.push_str(&r.claim);
}
if !r.note.is_empty() {
detail.push_str(" · ");
detail.push_str(&r.note);
}
out.push_str(&line(&format!("{}·{}", repo.repo, r.row_id), r.state, &detail));
}
}
if !any {
for repo in &inv.repos {
out.push_str(&line(&repo.repo, State::Absent, "no test-matrix row declared here"));
}
}
let errs: Vec<(&str, &String)> =
inv.repos.iter().flat_map(|r| r.errors.iter().map(move |e| (r.repo.as_str(), e))).collect();
if !errs.is_empty() {
out.push_str("\nREFUSED\n");
for (repo, e) in errs {
out.push_str(&line(&format!("{repo}·metadata"), State::Absent, e));
}
}
out
}
/// One rendered line. Column 1 pads to 46 and column 2 to 21, so column 3 always
/// starts at the same offset and keeps the rest of the line.
fn line(id: &str, state: State, detail: &str) -> String {
format!("{:<46} {:<21} {}\n", id, state.label(), detail.trim())
}
/// **One repo, one line** — the scoreboard the whole render exists to move.
///
/// Counted, never typed: how many benches of each mechanism, how many rows, and
/// how many of those rows are the finding. A repo's line reads `present` only
/// when nothing in it is `declared-not-emitted`; that is what "green" means
/// here, and it is the only claim this summary makes.
fn repo_state(repo: &RepoInventory) -> State {
if repo.rows.iter().any(|r| r.state == State::DeclaredNotEmitted)
|| repo.benches.iter().any(|b| b.state == State::DeclaredNotEmitted)
{
return State::DeclaredNotEmitted;
}
if repo.rows.is_empty() && repo.benches.iter().all(|b| b.state == State::Absent) {
return State::Absent;
}
State::Present
}
fn repo_summary(repo: &RepoInventory) -> String {
let benches = || repo.benches.iter().filter(|b| !b.id.is_empty());
let of_kind = |k: BenchKind| benches().filter(|b| b.kind == k).count();
let rows_in = |s: State| repo.rows.iter().filter(|r| r.state == s).count();
format!(
"{} arms · {} cargo · {} criterion · {} house · {} detached | {} rows · {} declared-not-emitted · {} gated · {} n/a · {} prose",
of_kind(BenchKind::NornirBenchArm),
of_kind(BenchKind::CargoBench),
of_kind(BenchKind::Criterion),
of_kind(BenchKind::HouseExample),
benches().filter(|b| b.detached).count(),
repo.rows.len(),
rows_in(State::DeclaredNotEmitted),
rows_in(State::Gated),
rows_in(State::NotApplicable),
repo.rows.iter().filter(|r| r.row_id.contains(' ')).count(),
)
}
/// The inventory as JSON — the machine half of the same truth.
pub fn to_json(inv: &Inventory) -> Result<String> {
serde_json::to_string_pretty(inv).context("serialise inventory")
}
/// The inventory as matrix rows, so a [`TestSink`](crate::TestSink) persists it
/// beside the test results it describes.
#[must_use]
pub fn to_rows(inv: &Inventory, run_id: &str) -> Vec<TestResultRow> {
let ts = now_micros();
let mut out = Vec::new();
for repo in &inv.repos {
for b in repo.benches.iter().filter(|b| !b.id.is_empty()) {
out.push(TestResultRow {
run_id: run_id.into(),
repo: repo.repo.clone(),
suite: b.krate.clone(),
test_name: b.id.clone(),
status: b.state.status().into(),
duration_ms: 0.0,
ts_micros: ts,
message: format!("{} · {}", b.kind.label(), b.measures),
aspect: ASPECT_INVENTORY.into(),
metric: 0.0,
});
}
for r in &repo.rows {
out.push(TestResultRow {
run_id: run_id.into(),
repo: repo.repo.clone(),
suite: r.krate.clone(),
test_name: r.row_id.clone(),
status: r.state.status().into(),
duration_ms: 0.0,
ts_micros: ts,
message: r.note.clone(),
aspect: ASPECT_INVENTORY.into(),
metric: 0.0,
});
}
}
out
}
/// Lines of `text` that speak the forbidden coordinate word. The rendered
/// inventory must never use it — the model's word is *typ*.
#[must_use]
pub fn forbidden_vocabulary(text: &str) -> Vec<String> {
text.lines()
.filter(|l| {
l.split(|c: char| !c.is_ascii_alphabetic())
.any(|w| w.eq_ignore_ascii_case(FORBIDDEN_WORD))
})
.map(str::to_string)
.collect()
}
/// Every fixed string the renderer itself can put on a line — the vocabulary
/// this module OWNS, as opposed to the identifiers it quotes out of the repos.
///
/// The distinction is load-bearing. holger really does have a test called
/// `the_declared_cell_survives_replacement_and_upgrade_and_fails_loud`, and
/// tunnr a row called `every_cell_honest`; quoting those verbatim is the
/// inventory being accurate, not the inventory adopting the word. So the hard
/// guard runs over THIS list ([`the_renderers_own_vocabulary_is_clean`]), and
/// the scan over rendered text ([`forbidden_vocabulary`]) is a report about the
/// constellation, not a verdict on the renderer.
#[must_use]
pub fn renderer_vocabulary() -> Vec<&'static str> {
let mut v = vec![
"SUMMARY",
"arms · cargo · criterion · house · detached | rows · declared-not-emitted · gated · n/a · prose",
"BENCHES",
"ROWS",
"REFUSED",
"declared by",
GATE_MARKER,
"detached manifest",
"no bench target of any kind under this checkout",
"no test-matrix row declared here",
"nornir-bench entrypoint — its arms are listed as their own rows",
"cargo bench --no-run failed for this manifest",
"declared by the appliance roster, no emitter in any scanned repo",
"only emitter is #[ignore]d",
"emitted by",
"impl Bencher for",
"no register_bench!, inventory links nothing",
"register_bench! call(s) whose id is not a literal in this file",
];
v.extend(
[BenchKind::CargoBench, BenchKind::Criterion, BenchKind::NornirBenchArm, BenchKind::HouseExample]
.map(BenchKind::label),
);
v.extend(
[State::Present, State::Absent, State::DeclaredNotEmitted, State::Gated, State::NotApplicable]
.map(State::label),
);
for p in APPLIANCE_PROOFS {
v.push(proof_claim(p));
}
v
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn module_doc_is_the_first_line_of_the_bang_doc() {
let src = "//! **Bench the scan hot path** (feature `bench`) — parse a blob\n//! and more.\nuse x;\n";
assert_eq!(
module_doc_first_line(src),
"Bench the scan hot path (feature bench) — parse a blob"
);
assert_eq!(module_doc_first_line("use x;\n"), "", "no doc is empty, never a guess");
}
#[test]
fn a_unit_struct_arm_is_its_id_literal() {
let src = "//! measures the fold\nstruct ScanFold;\nimpl nornir_bench::api::Bencher for ScanFold {\n fn id(&self) -> &'static str {\n \"autonom.function_scan\"\n }\n}\nnornir_bench::register_bench!(ScanFold::default());\n";
let arms = scan_bench_arms(src);
assert_eq!(arms.len(), 1, "{arms:?}");
assert_eq!(arms[0].id, "autonom.function_scan");
assert_eq!(arms[0].ty, "ScanFold");
assert!(arms[0].registered, "register_bench! names the type");
}
#[test]
fn an_id_field_arm_is_read_out_of_the_registration() {
let src = "register_bench!(Archive { id: \"znippy.archive_small_10k\", n: 10_000 });\n";
let arms = scan_bench_arms(src);
assert_eq!(arms.len(), 1, "{arms:?}");
assert_eq!(arms[0].id, "znippy.archive_small_10k");
assert!(arms[0].registered);
}
#[test]
fn an_unregistered_bencher_impl_is_reported_as_declared_not_emitted() {
let src = "impl Bencher for Orphan {\n fn id(&self) -> &'static str { \"x.orphan\" }\n}\n";
let arms = scan_bench_arms(src);
assert_eq!(arms.len(), 1);
assert!(!arms[0].registered, "inventory links at LINK time — say so");
}
#[test]
fn ordered_registration_skips_the_order_argument() {
let types = registered_types("register_bench_ordered!(1, Stage2PbfToGeo);\n");
assert!(types.contains("Stage2PbfToGeo"), "got {types:?}");
}
#[test]
fn an_emit_site_carries_its_test_and_whether_it_is_ignored() {
let src = r#"
#[test]
fn live() {
functional_status("gunnar-appliance", "identity_ready", true, "minted");
}
#[test]
#[ignore = "heavy"]
fn dark() {
functional_status("holger-appliance/container", "install_ok", true, "");
}
"#;
let sites = scan_emit_sites(src);
let live = sites.iter().find(|s| s.check == "identity_ready").expect("live emit");
assert_eq!(live.component, "gunnar-appliance");
assert_eq!(live.test_fn, "live");
assert!(!live.ignored, "not ignored");
let dark = sites.iter().find(|s| s.check == "install_ok").expect("ignored emit");
assert_eq!(dark.component, "holger-appliance/container");
assert_eq!(dark.test_fn, "dark");
assert!(dark.ignored, "an #[ignore]d test emits nothing in a default run");
assert!(dark.requires.is_empty(), "`heavy` declares no requirement — it stays dark");
}
/// RED-BEFORE-GREEN: the two ignored tests must not read alike.
///
/// One is a test somebody forgot; the other cannot run on any box here and
/// says so. Reporting both as `declared-not-emitted` puts an unfixable row
/// on the same list as a fixable one, and the list then stops being a list
/// anybody works through.
#[test]
fn an_ignore_that_names_what_it_requires_is_gated_not_dark() {
let src = r#"
#[test]
#[ignore = "requires: enforcing firmware on real hardware"]
fn armed() {
functional_status("holger-appliance/iso-kvm", "secure_boot_armed", true, "");
}
#[test]
#[ignore = "heavy: takes a while"]
fn forgotten() {
functional_status("holger-appliance/iso-kvm", "install_ok", true, "");
}
"#;
let sites = scan_emit_sites(src);
let gated = sites.iter().find(|s| s.check == "secure_boot_armed").expect("emit");
assert_eq!(gated.requires, "enforcing firmware on real hardware", "the WHAT travels");
let dark = sites.iter().find(|s| s.check == "install_ok").expect("emit");
assert!(dark.requires.is_empty(), "a reason without the marker declares nothing");
// …and the grade follows the declaration, per repo.
let mut inv = Inventory {
repos: vec![RepoInventory {
repo: "holger".into(),
root: "/tmp/holger".into(),
benches: Vec::new(),
rows: Vec::new(),
emits: sites
.into_iter()
.map(|e| EmitSite {
repo: "holger".into(),
krate: "xtask".into(),
file: "xtask/tests/iso.rs".into(),
component: e.component,
check: e.check,
test_fn: e.test_fn,
ignored: e.ignored,
requires: e.requires,
na_reason: e.na_reason,
})
.collect(),
errors: Vec::new(),
}],
};
resolve_rows_with(&mut inv, &["holger"], &["secure_boot_armed", "install_ok"]);
let row = |id: &str| {
inv.repos
.iter()
.flat_map(|r| r.rows.iter())
.find(|r| r.row_id == id)
.unwrap_or_else(|| panic!("row {id}"))
.clone()
};
let armed = row("holger-appliance/secure_boot_armed");
assert_eq!(armed.state, State::Gated);
assert!(armed.note.contains("enforcing firmware"), "the requirement is ON the row: {}", armed.note);
assert_eq!(armed.state.status(), "skip", "gated did not run — and a skip is not a pass");
assert_eq!(row("holger-appliance/install_ok").state, State::DeclaredNotEmitted);
// Each emitter also yields its FORM row (`…-appliance/iso-kvm/<proof>`),
// which is a different coordinate and carries the same grade — so the
// two lists are two long, and the grades still separate cleanly.
assert_eq!(row("holger-appliance/iso-kvm/secure_boot_armed").state, State::Gated);
assert_eq!(row("holger-appliance/iso-kvm/install_ok").state, State::DeclaredNotEmitted);
assert_eq!(inv.gated().len(), 2);
assert_eq!(inv.declared_not_emitted().len(), 2, "the gated rows are off the fix list");
}
/// RED-BEFORE-GREEN: a rung the product HAS NOT GOT is neither proof nor a
/// hole, and the reason is read straight out of the call.
#[test]
fn a_declared_n_a_rung_reads_as_n_a_and_carries_its_reason() {
let src = "#[test]\nfn ladder() {\n functional_na(\"nornir-appliance\", \"secure_boot_armed\", \"nornir declares no bootable target\");\n}\n";
let sites = scan_emit_sites(src);
assert_eq!(sites.len(), 1, "{sites:?}");
assert_eq!(sites[0].na_reason, "nornir declares no bootable target");
let mut inv = Inventory {
repos: vec![RepoInventory {
repo: "nornir".into(),
root: "/tmp/nornir".into(),
benches: Vec::new(),
rows: Vec::new(),
emits: vec![EmitSite {
repo: "nornir".into(),
krate: "nornir".into(),
file: "tests/appliance_ladder.rs".into(),
component: sites[0].component.clone(),
check: sites[0].check.clone(),
test_fn: sites[0].test_fn.clone(),
ignored: false,
requires: String::new(),
na_reason: sites[0].na_reason.clone(),
}],
errors: Vec::new(),
}],
};
resolve_rows_with(&mut inv, &["nornir"], &["secure_boot_armed"]);
let row = &inv.repos[0].rows[0];
assert_eq!(row.state, State::NotApplicable, "not present — nothing was proven");
assert!(row.note.contains("no bootable target"), "the reason is the content: {}", row.note);
assert!(inv.declared_not_emitted().is_empty(), "an n/a is not on the fix list");
assert_eq!(inv.not_applicable().len(), 1);
}
#[test]
fn a_substring_is_not_an_emit_call() {
assert!(emit_call_at("let parcell(x);").is_none(), "the name must be whole, not a tail");
assert_eq!(emit_call_at("cell(\"a\", \"b\")").map(|(_, f)| f), Some("cell"));
assert_eq!(
emit_call_at("assert_emit!(\"a\", \"b\", true)").map(|(_, f)| f),
Some("assert_emit")
);
}
#[test]
fn a_cell_call_yields_the_component_and_the_test_name_not_the_run_id() {
// `cell(run_id, repo, component, aspect, test_name, status, ms, msg)` —
// reading "the first two literals" here produced the row `run-1/holger`.
let src = "fn t() {\n let r = cell(\"run-1\", \"holger\", \"holger-server\", \"unit\", \"holger::lib\", \"fail\", 12.5, \"boom\");\n}\n";
let sites = scan_emit_sites(src);
assert_eq!(sites.len(), 1, "{sites:?}");
assert_eq!(sites[0].component, "holger-server", "the component, not the run id");
assert_eq!(sites[0].check, "holger::lib", "the test name, not the repo");
}
/// RED-BEFORE-GREEN: the runner's own test double is not an unregistered arm.
#[test]
fn a_bencher_inside_a_test_module_is_a_fixture_not_a_declaration() {
let src = "struct Real;\nimpl Bencher for Real {\n fn id(&self) -> &'static str { \"demo.real\" }\n}\nregister_bench!(Real);\n\n#[cfg(test)]\nmod tests {\n struct PinAware;\n impl Bencher for PinAware {\n fn id(&self) -> &'static str { \"demo.pin_aware\" }\n }\n}\n";
let arms = scan_bench_arms(src);
assert_eq!(arms.len(), 1, "only the real arm: {arms:?}");
assert_eq!(arms[0].id, "demo.real");
assert!(arms[0].registered);
// …and the guard is not vacuous: outside the test module it IS a finding.
let loose = src.replace("#[cfg(test)]\n", "");
assert_eq!(scan_bench_arms(&loose).len(), 2, "an unregistered impl in live code is red");
}
#[test]
fn a_doc_comment_example_is_not_a_registration() {
let src = "//! ```ignore\n//! impl nornir::bench::api::Bencher for Demo { fn id(&self) -> &'static str { \"demo.pin_aware\" } }\n//! nornir::register_bench!(Demo);\n//! ```\nfn real() {}\n";
assert!(scan_bench_arms(src).is_empty(), "prose is not code: {:?}", scan_bench_arms(src));
assert_eq!(unresolved_registrations(src), 0, "and it is not a counted hole either");
}
#[test]
fn a_string_fixture_is_not_a_registration() {
let src = "#[test]\nfn t() {\n let s = \"register_bench!(Ghost { id: \\\"x.ghost\\\" });\";\n let _ = s;\n}\n";
assert!(scan_bench_arms(src).is_empty(), "a test's fixture text is not a bench");
assert_eq!(unresolved_registrations(src), 0);
}
#[test]
fn the_mask_keeps_byte_offsets() {
let src = "let a = \"xx\"; // note\nfn f() {}\n";
let m = mask_comments_and_strings(src);
assert_eq!(m.len(), src.len(), "offsets must survive masking");
assert_eq!(m.lines().count(), src.lines().count());
assert!(!m.contains("note"), "the comment is gone");
assert!(!m.contains("xx"), "the literal body is gone");
assert!(m.contains("fn f() {}"), "the code is untouched");
}
#[test]
fn an_arm_id_is_not_a_sentence_or_a_path() {
assert!(looks_like_arm_id("gunnar.opstorm"));
assert!(looks_like_arm_id("saga-replay-fold"));
assert!(!looks_like_arm_id("the appliance boots"));
assert!(!looks_like_arm_id("bench/examples/x.rs"));
assert!(!looks_like_arm_id("main.rs"));
assert!(!looks_like_arm_id("plain"));
}
#[test]
fn criterion_is_recognised_by_what_the_source_drives() {
assert!(uses_criterion("criterion_group!(benches, x);\n"));
assert!(uses_criterion("use criterion::Criterion;\n"));
assert!(!uses_criterion("fn main() {}\n"));
}
#[test]
fn a_detached_manifest_opens_its_own_workspace() {
let dir = std::env::temp_dir().join(format!("ntm-inv-{}", crate::model::new_run_id()));
std::fs::create_dir_all(&dir).unwrap();
let man = dir.join("Cargo.toml");
std::fs::write(&man, "[package]\nname = \"x\"\n\n[workspace]\n").unwrap();
assert!(declares_workspace(&man));
std::fs::write(&man, "[package]\nname = \"x\"\n").unwrap();
assert!(!declares_workspace(&man), "a plain member is not a detached root");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_render_never_speaks_the_forbidden_word() {
let inv = Inventory {
repos: vec![RepoInventory {
repo: "empty".into(),
root: "/tmp/empty".into(),
benches: vec![BenchTarget {
repo: "empty".into(),
krate: String::new(),
kind: BenchKind::CargoBench,
id: String::new(),
measures: String::new(),
manifest: String::new(),
src: String::new(),
detached: false,
compiles: None,
state: State::Absent,
note: "no bench target of any kind under this checkout".into(),
}],
rows: Vec::new(),
emits: Vec::new(),
errors: Vec::new(),
}],
};
let text = render(&inv);
assert!(forbidden_vocabulary(&text).is_empty(), "the model's word is typ: {text}");
assert!(text.contains("absent"), "a repo with no benches gets one absent line");
}
#[test]
fn the_renderers_own_vocabulary_is_clean() {
let hits: Vec<&str> = renderer_vocabulary()
.into_iter()
.filter(|s| !forbidden_vocabulary(s).is_empty())
.collect();
assert!(hits.is_empty(), "the model's word is typ, never {FORBIDDEN_WORD:?}: {hits:?}");
// Not vacuous: the checker does fire on the word.
assert_eq!(forbidden_vocabulary("one cell of the grid").len(), 1);
}
#[test]
fn an_id_comes_from_the_id_field_and_not_from_any_literal_nearby() {
// The real shape that produced phantom arms before this was tightened:
// a bencher constructor carrying scenario labels next to its id.
let src = "register_bench!(SearchSqlTemplate { id: \"nornir.mimir.search.sql\", templates: &[\"call-chain\", \"change-radius\", \"def-use\"] });\n";
let arms = scan_bench_arms(src);
assert_eq!(arms.len(), 1, "one registration is one arm: {arms:?}");
assert_eq!(arms[0].id, "nornir.mimir.search.sql");
assert_eq!(unresolved_registrations(src), 0);
}
#[test]
fn a_registration_whose_id_is_computed_is_counted_not_invented() {
let src = "register_bench!(make_bench(size));\n";
assert!(scan_bench_arms(src).is_empty(), "no literal, no guess");
assert_eq!(unresolved_registrations(src), 1, "the hole is counted, not dropped");
}
/// RED-BEFORE-GREEN: a rustfmt-shaped call is not a shorter call.
///
/// The real shape that was being dropped: two literal coordinates, then a
/// verdict expression, then a `format!` detail broken over several lines —
/// thirteen lines all told, which is what `cargo fmt` produces from an
/// ordinary emit and what the 10-line window silently refused.
#[test]
fn an_emit_call_spread_over_a_dozen_lines_is_still_found() {
let src = "fn t() {\n functional_status(\n \"nornir-appliance\",\n \"identity_minted\",\n minted && shaped,\n &format!(\n \"host form: minted \\\n {} + {}\",\n a.display(),\n b.display(),\n c,\n ),\n );\n}\n";
let sites = scan_emit_sites(src);
assert_eq!(sites.len(), 1, "the call spans 12 lines and is one row: {sites:?}");
assert_eq!(sites[0].component, "nornir-appliance");
assert_eq!(sites[0].check, "identity_minted");
assert!(EMIT_CALL_MAX_LINES >= 12, "and the cap is why");
}
#[test]
fn a_computed_component_does_not_pair_the_check_with_the_detail() {
// holger's real call: the component is a variable, so the first two
// LITERALS are the check and the detail — which used to invent the row
// `identity_minted/pubkey={key1}`.
let src = "fn f() {\n functional_status(&comp, \"identity_minted\", true, &format!(\"pubkey={key1}\"));\n}\n";
assert!(
scan_emit_sites(src).is_empty(),
"a computed coordinate is not a discovered row: {:?}",
scan_emit_sites(src)
);
// …and the literal form still works.
let ok = "fn f() {\n functional_status(\"holger-appliance\", \"identity_minted\", true, \"x\");\n}\n";
assert_eq!(scan_emit_sites(ok).len(), 1);
}
#[test]
fn top_level_args_ignores_commas_inside_nested_calls_and_strings() {
let args = top_level_args("f(\"a,b\", g(1, 2), [3, 4])").expect("balanced");
assert_eq!(args.len(), 3, "{args:?}");
assert_eq!(bare_literal(&args[0]).as_deref(), Some("a,b"));
assert_eq!(bare_literal(&args[1]), None, "a call is not a literal");
}
#[test]
fn bench_example_locations_match_the_harness() {
assert_eq!(BENCH_EXAMPLE_LOCATIONS[0], "examples/nornir-bench.rs");
assert_eq!(BENCH_EXAMPLE_LOCATIONS[1], "xtask/examples/nornir-bench.rs");
assert_eq!(BENCH_EXAMPLE_LOCATIONS[2], "bench/examples/nornir-bench.rs");
}
#[test]
fn another_repos_fixture_is_not_this_products_proof() {
// The regression this rule was written for: the matrix crate's own
// sample rows read as holger's evidence and the grid went 8/8 green.
let mut inv = Inventory {
repos: vec![RepoInventory {
repo: "edda".into(),
root: "/tmp/edda".into(),
benches: Vec::new(),
rows: Vec::new(),
emits: vec![EmitSite {
repo: "edda".into(),
krate: "nornir-testmatrix".into(),
file: "src/appliance.rs".into(),
component: "holger-appliance".into(),
check: "install_ok".into(),
test_fn: "the_form_vocabulary_folds_onto_the_grid".into(),
ignored: false,
requires: String::new(),
na_reason: String::new(),
}],
errors: Vec::new(),
}],
};
resolve_rows_with(&mut inv, &["holger"], &["install_ok"]);
let row = inv
.repos
.iter()
.flat_map(|r| r.rows.iter())
.find(|r| r.row_id == "holger-appliance/install_ok")
.expect("the declared row exists");
assert_eq!(row.state, State::DeclaredNotEmitted, "a fixture in edda is not holger's proof");
assert!(row.note.contains("outside holger"), "and it says where the emitter really is: {}", row.note);
}
#[test]
fn the_declared_grid_is_the_roster_product() {
let mut inv = Inventory::default();
resolve_rows(&mut inv);
assert_eq!(
inv.row_count(),
APPLIANCE_PRODUCTS.len() * APPLIANCE_PROOFS.len(),
"the appliance roster floors the grid at 5 x 8"
);
assert_eq!(
inv.declared_not_emitted().len(),
inv.row_count(),
"with nothing scanned, every declared row is unemitted — never silently green"
);
}
}