crates/nornir-testmatrix/src/testsurface.rs
//! **Which features gate tests, and does anything declare a run of them?**
//!
//! [`gatedtests`](crate::gatedtests) next door answers a narrower question: does
//! this `tests/*.rs` compile to an empty binary under a plain `cargo test`, and
//! if not, does a declared arm rescue it? It walks test TARGETS, and that is
//! precisely where it stops seeing.
//!
//! ## The hole it cannot see (MEASURED 2026-08-25, nine repos)
//!
//! A `#[cfg(feature = "x")] pub mod m;` in `lib.rs` puts `m`'s `#[test]` fns in
//! the **library** target. There is no `[[test]]` target to walk, no
//! `required-features` to read, and no crate-level `#![cfg]` to extract — so no
//! walk built on `CrateManifest::tests` has ever looked at one.
//!
//! Skidbladnir ~350 · knut 165 · edda 96 · korp-collectors ~70 · draupnir ~54 ·
//! modgunn 49 · gunnar 28 · korp ~20 · skade 19 — **roughly 850 tests that no
//! declared variant enables**. The one that cost something: Skidbladnir's
//! `src/vmsuite.rs` held 23 pure judge tests behind `matrix`, `judge_active`
//! graded systemd's transitional `deactivating` as *stopped*, and a suite case of
//! `systemctl stop` → `Sleep(1500)` → collect passed on "it stopped" for any
//! service that drains longer than 1.5 s. A real service on that box sat in
//! `deactivating` for 6.4 s. Three weeks, green matrix.
//!
//! ## Three sources of "gates a test"
//!
//! | source | shape | seen before? |
//! |---|---|---|
//! | integration target | `tests/x.rs` with `#![cfg(feature = "f")]` | yes — `audit_repo` |
//! | cargo target | `[[test]]` / `[[bench]]` `required-features` | cargo skips it silently, exit 0 |
//! | **library unit tests** | `#[cfg(feature = "f")]` on a `mod`, inline or file | **nothing, anywhere** |
//!
//! ## Three sources of "the feature is ON", and the one that makes a liar of a scan
//!
//! A feature-closure scan reads `[build] offers` and each variant's `features`.
//! That is two of the three. The third is a **dependency EDGE**:
//!
//! ```toml
//! # holger/server/cli/Cargo.toml:71 — unconditional
//! server-lib = { package = "holger-server-lib", path = "../lib", features = ["sign"] }
//! # holger/server/cli/Cargo.toml:99, holger/server/ui/Cargo.toml:110 — via `bench`
//! bench-scenarios = { package = "holger-bench-scenarios", features = ["served"], optional = true }
//! ```
//!
//! Neither `sign` nor `served` is written by any `[[build.variant]]`, and
//! `holger/.nornir/build-variants.toml:176` and `:207` therefore say in prose that
//! both are *"DECLARED-BUT-NEVER-BUILT"*. **Both claims are false.** `sign` is on
//! for all six variants that build `holger-server-cli` — the edge is not even
//! optional. `served` is on for `full` (`full = [… "bench" …]`, cli:36) and for
//! `ui-embedded` (`gui-embedded = [… "bench" …]`, ui:113).
//!
//! A detector that reports a lit feature as dark is a detector people switch off,
//! so this one resolves the edges: [`resolve`] runs cargo's own propagation —
//! optional-dep activation through `dep:alias`, edge `features`, edge
//! `default-features`, and the `alias/feat` / weak `alias?/feat` cross-crate
//! strings — to a fixed point over the workspace graph.
//!
//! ## …and therefore THREE verdicts, not two
//!
//! Resolving the edges is not the same as calling the feature green, and the
//! difference is measured, not argued. On `tests/fixtures/dep_edge_lit`, verified
//! by running cargo 2026-08-25:
//!
//! * `cargo build -p driver --features bench` **compiles** `engine` with `served`;
//! * `cargo test -p driver --features bench` runs **0** of `engine`'s tests.
//!
//! Cargo never builds a dependency's test targets. So an edge-lit feature's tests
//! are compiled by the forge and still never executed by it. That is neither
//! [`RunVerdict::Dark`] nor [`RunVerdict::Run`]; it is
//! [`RunVerdict::CompiledOnly`], and it is reported with the sibling and the
//! configuration that lit it, so the fix (name the package in the variant's
//! `test-packages`, or declare an arm) is the next thing a reader reaches for.
//!
//! ## What this proves, and what it does not
//!
//! It proves the DECLARATION loop: a gated test surface is named by something the
//! repo declares. Whether the forge then *runs* that declaration is one level up —
//! pass the variants a `brokkr.toml` step actually names as `run_variants` and the
//! rows say so. Nothing here compiles anything: one `cargo metadata`, a `src/**`
//! read, and set arithmetic.
//!
//! ```no_run
//! use nornir_testmatrix::{workspace_crates, audit_test_surface, MatrixArms};
//! use std::path::Path;
//!
//! let root = Path::new(".");
//! let crates = workspace_crates(root).unwrap();
//! let arms = MatrixArms::load(root).unwrap_or_default();
//! // `variants` come from `nornir_build_thing::BuildVariantSet` — mapped to
//! // `VariantSpec` by the caller, because this crate's dependency budget is
//! // serde + anyhow and build-thing is a dev-dependency only.
//! let report = audit_test_surface(root, "myrepo", "run-1", &crates, &[], &arms, None);
//! assert!(report.is_green(), "{}", report.summary());
//! ```
use std::collections::{BTreeMap, BTreeSet, VecDeque};
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use crate::gatedtests::{
count_test_fns, extract_crate_cfg, is_test_attr, parse_cfg_expr, strip_comments, CrateManifest,
DepKind, MatrixArm, MatrixArms,
};
use crate::harness::now_micros;
use crate::model::{status, TestResultRow};
use crate::targetcheck::VariantSpec;
/// The matrix aspect every row this module emits carries.
pub const ASPECT_TEST_SURFACE: &str = "test-surface";
// ─── the scan: where a gate can sit ────────────────────────────────────────
/// Where a feature was found gating tests.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum GateSite {
/// A `tests/*.rs` whose crate-level `#![cfg(...)]` names the feature.
IntegrationTarget,
/// A `[[test]]` / `[[bench]]` `required-features` entry. Cargo **silently
/// skips** such a target when the features are off: no warning, no line in
/// `cargo test` output, exit 0.
RequiredFeatures,
/// A `#[cfg(feature = ...)]` on a `mod` — inline or file — inside a non-test
/// target, so the `#[test]` fns land in the library (or bin) target. **The
/// hole nothing could see.**
LibraryUnitTests,
}
impl GateSite {
/// Short tag for a report line.
pub fn tag(self) -> &'static str {
match self {
GateSite::IntegrationTarget => "integration-target",
GateSite::RequiredFeatures => "required-features",
GateSite::LibraryUnitTests => "library-unit-tests",
}
}
}
/// One feature, in one package, found gating `#[test]` fns.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct GatedSurface {
/// The package the tests live in — the package whose tests must be RUN for
/// them to count, which is not necessarily the package a variant builds.
pub package: String,
/// The gating feature, as declared in `package`'s own `[features]` table.
pub feature: String,
/// Where the gate sits.
pub site: GateSite,
/// How many `#[test]`-family fns this gate hides.
pub tests: usize,
/// Human-readable location, repo-relative where possible.
pub evidence: String,
}
/// One gated block of library `#[test]` fns: the AND of every feature gate in
/// force at its position.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UnitGate {
/// Every feature that must be on for these tests to compile. More than one
/// means nested or `all(...)` gates; the tests are attributed to each,
/// because turning on one of two ANDed features still leaves them dark.
pub features: BTreeSet<String>,
/// `#[test]`-family fns under this gate.
pub tests: usize,
/// The file they live in.
pub file: PathBuf,
/// The line of the first one.
pub line: usize,
}
/// **The library-unit-test scan** — walk a non-test target's module tree from its
/// root source file and report every `#[test]` fn that sits under a
/// `#[cfg(feature = ...)]`.
///
/// Four gate shapes, because they all occur in the constellation and each hides
/// the tests just as thoroughly:
///
/// ```ignore
/// #[cfg(feature = "matrix")] pub mod vmsuite; // file module (Skidbladnir)
/// #[cfg(feature = "gui")] mod robot { #[test] … } // inline module (holger)
/// mod deep; // nested: inherits its parent's gate
/// #[cfg(feature = "solo")] #[test] fn one() {} // the gate on the fn itself
/// ```
///
/// Nesting is ANDed: a `mod` inside a gated `mod` needs both. Brace depth is
/// tracked with a string-literal-aware scanner, so a `{` inside a `"…"` or a
/// `r#"…"#` cannot close a module early — the failure mode that makes a
/// depth-counting scan attribute half a file to the wrong gate.
pub fn scan_unit_gates(root: &Path) -> Vec<UnitGate> {
let mut acc: BTreeMap<(PathBuf, Vec<String>), (usize, usize)> = BTreeMap::new();
let mut seen = BTreeSet::new();
walk_module(root, &BTreeSet::new(), 0, &mut seen, &mut acc);
let mut out: Vec<UnitGate> = acc
.into_iter()
.map(|((file, feats), (tests, line))| UnitGate {
features: feats.into_iter().collect(),
tests,
file,
line,
})
.collect();
out.sort_by(|a, b| (&a.file, a.line).cmp(&(&b.file, b.line)));
out
}
fn walk_module(
file: &Path,
inherited: &BTreeSet<String>,
depth: usize,
seen: &mut BTreeSet<PathBuf>,
acc: &mut BTreeMap<(PathBuf, Vec<String>), (usize, usize)>,
) {
if depth > 12 || !seen.insert(file.to_path_buf()) {
return;
}
let Ok(raw) = std::fs::read_to_string(file) else {
return;
};
let src = strip_comments(&raw);
let lines: Vec<&str> = src.lines().collect();
let depths = depths_per_line(&src);
// Frames pushed by an INLINE `#[cfg(...)] mod x { … }`: the brace depth the
// `mod` line itself sits at, plus the features it adds.
let mut stack: Vec<(usize, BTreeSet<String>)> = Vec::new();
// Features collected from `#[cfg(...)]` attribute lines waiting for the item
// they decorate.
let mut pending: BTreeSet<String> = BTreeSet::new();
let mut i = 0usize;
while i < lines.len() {
let d = depths.get(i).copied().unwrap_or(0);
while stack.last().map(|(fd, _)| d <= *fd).unwrap_or(false) {
stack.pop();
}
let t = lines[i].trim();
if t.is_empty() {
i += 1;
continue;
}
// A `#[test]`-family attribute: the gate in force here is everything.
if is_test_attr(t) {
let mut gate: BTreeSet<String> = inherited.clone();
for (_, f) in &stack {
gate.extend(f.iter().cloned());
}
gate.extend(pending.iter().cloned());
if !gate.is_empty() {
let key = (file.to_path_buf(), gate.into_iter().collect::<Vec<_>>());
let e = acc.entry(key).or_insert((0, i + 1));
e.0 += 1;
}
i += 1;
continue;
}
// Inner attributes belong to the file, not to the next item.
if t.starts_with("#![") {
i += 1;
continue;
}
if t.starts_with("#[") {
// An attribute may wrap: join until the brackets balance, so a
// `#[cfg(all(\n feature = "a",\n feature = "b"))]` is still read.
let (joined, consumed) = join_attribute(&lines, i);
if let Some(body) = outer_cfg_body(&joined) {
if let Some(e) = parse_cfg_expr(&body) {
pending.extend(e.features());
}
}
i += consumed;
continue;
}
// An item line. Whatever `pending` holds applies to it.
let mut gate_here: BTreeSet<String> = inherited.clone();
for (_, f) in &stack {
gate_here.extend(f.iter().cloned());
}
gate_here.extend(pending.iter().cloned());
if let Some(name) = file_mod_decl(t) {
if let Some(child) = module_file(file, name) {
walk_module(&child, &gate_here, depth + 1, seen, acc);
}
} else if inline_mod_decl(t).is_some() {
stack.push((d, pending.clone()));
}
pending.clear();
i += 1;
}
}
/// Join a wrapped attribute starting at `start` until its `[` … `]` balances.
/// Returns the joined text and how many lines it spans (always ≥ 1).
fn join_attribute(lines: &[&str], start: usize) -> (String, usize) {
let mut joined = String::new();
let mut depth = 0isize;
let mut n = 0usize;
for line in lines.iter().skip(start) {
joined.push_str(line.trim());
n += 1;
for c in line.chars() {
match c {
'[' | '(' => depth += 1,
']' | ')' => depth -= 1,
_ => {}
}
}
if depth <= 0 || n >= 16 {
break;
}
}
(joined, n.max(1))
}
/// The body of an OUTER `#[cfg(…)]` attribute — the text [`parse_cfg_expr`] wants.
///
/// Deliberately NOT a twin of [`extract_crate_cfg`]: that one walks a file's
/// PROLOGUE for INNER `#![cfg]` attributes and stops at the first item, which by
/// construction can never see a `#[cfg]` sitting on a `mod` halfway down `lib.rs`.
/// `#[cfg_attr(...)]` is not matched — it decorates conditionally, it does not gate.
pub fn outer_cfg_body(line: &str) -> Option<String> {
let t = line.trim();
let inner = t.strip_prefix("#[cfg(")?.strip_suffix(")]")?;
(!inner.trim().is_empty()).then(|| inner.trim().to_string())
}
/// `mod x;` / `pub mod x;` / `pub(crate) mod x;` → `x`. A brace-bodied module
/// returns `None` (see [`inline_mod_decl`]).
pub fn file_mod_decl(line: &str) -> Option<&str> {
let name = mod_name(line)?;
line.trim().ends_with(';').then_some(name)
}
/// `mod x {` / `pub mod x {` → `x`. A file module returns `None`.
pub fn inline_mod_decl(line: &str) -> Option<&str> {
let name = mod_name(line)?;
line.trim().ends_with('{').then_some(name)
}
fn mod_name(line: &str) -> Option<&str> {
let t = line.trim();
let rest = [
"pub(crate) mod ",
"pub(super) mod ",
"pub(self) mod ",
"pub mod ",
"mod ",
]
.iter()
.find_map(|p| t.strip_prefix(p))?;
let name = rest
.trim()
.trim_end_matches('{')
.trim_end_matches(';')
.trim();
(!name.is_empty() && name.chars().all(|c| c.is_alphanumeric() || c == '_')).then_some(name)
}
/// The file that holds `mod <name>;` declared inside `parent`.
pub fn module_file(parent: &Path, name: &str) -> Option<PathBuf> {
let stem = parent.file_stem()?.to_str()?;
let dir = if matches!(stem, "lib" | "mod" | "main") {
parent.parent()?.to_path_buf()
} else {
parent.parent()?.join(stem)
};
let flat = dir.join(format!("{name}.rs"));
if flat.is_file() {
return Some(flat);
}
let nested = dir.join(name).join("mod.rs");
nested.is_file().then_some(nested)
}
/// Brace depth at the START of each line, counting only braces that are really
/// code: string literals (`"…"`, `r"…"`, `r#"…"#`), char literals (`'{'`) and
/// escapes are skipped, and a lifetime (`'a`) is not mistaken for one.
///
/// Comments are expected to be stripped already. Without this, one `"{"` in a
/// format string shifts every subsequent depth by one and a scan attributes the
/// rest of the file to whichever gated module happened to be open.
pub fn depths_per_line(src: &str) -> Vec<usize> {
let c: Vec<char> = src.chars().collect();
let mut out = vec![0usize];
let mut depth = 0usize;
let mut i = 0usize;
while i < c.len() {
match c[i] {
'\n' => {
out.push(depth);
i += 1;
}
'{' => {
depth += 1;
i += 1;
}
'}' => {
depth = depth.saturating_sub(1);
i += 1;
}
'r' if matches!(c.get(i + 1), Some('"') | Some('#')) => {
if let Some(next) = skip_raw_string(&c, i) {
// A raw string may span lines; push a depth for each.
for k in i..next {
if c[k] == '\n' {
out.push(depth);
}
}
i = next;
} else {
i += 1;
}
}
'"' => {
let mut k = i + 1;
while k < c.len() {
if c[k] == '\\' {
k += 2;
continue;
}
if c[k] == '"' {
k += 1;
break;
}
if c[k] == '\n' {
out.push(depth);
}
k += 1;
}
i = k;
}
'\'' => {
// A char literal is `'x'` or `'\n'`; anything else is a lifetime.
if c.get(i + 1) == Some(&'\\') {
let mut k = i + 2;
while k < c.len() && c[k] != '\'' {
k += 1;
}
i = k + 1;
} else if c.get(i + 2) == Some(&'\'') {
i += 3;
} else {
i += 1;
}
}
_ => i += 1,
}
}
out
}
/// Past the closing delimiter of a raw string starting at `i` (`r"…"`, `r#"…"#`).
fn skip_raw_string(c: &[char], i: usize) -> Option<usize> {
let mut k = i + 1;
let mut hashes = 0usize;
while c.get(k) == Some(&'#') {
hashes += 1;
k += 1;
}
if c.get(k) != Some(&'"') {
return None;
}
k += 1;
while k < c.len() {
if c[k] == '"' {
let mut h = 0usize;
while h < hashes && c.get(k + 1 + h) == Some(&'#') {
h += 1;
}
if h == hashes {
return Some(k + 1 + hashes);
}
}
k += 1;
}
Some(c.len())
}
/// **Every feature that gates a test, across every workspace member and all three
/// sources.** Pure: reads sources, compiles nothing.
pub fn gated_surfaces(crates: &[CrateManifest]) -> Vec<GatedSurface> {
let mut out = Vec::new();
for k in crates {
let rel = |p: &Path| -> String {
k.manifest_path
.parent()
.and_then(|d| p.strip_prefix(d).ok())
.unwrap_or(p)
.display()
.to_string()
};
// 1 + 2: the test targets.
for t in &k.tests {
for f in &t.required_features {
if !k.features.contains_key(f) {
continue;
}
out.push(GatedSurface {
package: k.name.clone(),
feature: f.clone(),
site: GateSite::RequiredFeatures,
tests: std::fs::read_to_string(&t.src_path)
.map(|s| count_test_fns(&s))
.unwrap_or(0),
evidence: format!("`required-features` of test target `{}`", t.name),
});
}
let Ok(src) = std::fs::read_to_string(&t.src_path) else {
continue;
};
let Some(body) = extract_crate_cfg(&src) else {
continue;
};
let Some(expr) = parse_cfg_expr(&body) else {
continue;
};
let n = count_test_fns(&src);
for f in expr.features() {
if !k.features.contains_key(&f) {
continue;
}
out.push(GatedSurface {
package: k.name.clone(),
feature: f,
site: GateSite::IntegrationTarget,
tests: n,
evidence: format!("{n} `#[test]` fn(s) in {}, `#![cfg({body})]`", rel(&t.src_path)),
});
}
}
// 3: the library / bin unit tests — the hole.
for root in &k.src_roots {
for g in scan_unit_gates(root) {
for f in &g.features {
if !k.features.contains_key(f) {
continue;
}
out.push(GatedSurface {
package: k.name.clone(),
feature: f.clone(),
site: GateSite::LibraryUnitTests,
tests: g.tests,
evidence: format!(
"{} `#[test]` fn(s) at {}:{}, gated on {}",
g.tests,
rel(&g.file),
g.line,
g.features
.iter()
.map(|x| format!("`{x}`"))
.collect::<Vec<_>>()
.join(" + ")
),
});
}
}
}
}
out.sort_by(|a, b| {
(&a.package, &a.feature, a.site, &a.evidence).cmp(&(&b.package, &b.feature, b.site, &b.evidence))
});
out.dedup();
out
}
// ─── the declared side: what turns a feature on ────────────────────────────
/// Where a configuration came from — the fix for a red row depends on it.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum ConfigOrigin {
/// A plain `cargo test` over the workspace.
Default,
/// A declared `[[build.variant]]`, modelled as its brokkr `tests` step.
Variant,
/// A declared `.nornir/testmatrix-arms.json` arm.
Arm,
}
/// One configuration the repo declares, as a TEST run: which packages' test
/// targets it builds, and what features each package is seeded with.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct TestConfig {
/// Human label used in report lines (the variant or arm name).
pub label: String,
/// Where it came from.
pub origin: ConfigOrigin,
/// The packages whose **test targets** this run compiles and executes.
/// Everything else in the graph is a dependency: compiled, never tested.
pub selected: BTreeSet<String>,
/// Seeds per package (`--features`, plus `default` unless suppressed).
pub seeds: BTreeMap<String, Vec<String>>,
/// Whether this configuration is named by a pipeline step, when the caller
/// knows. `None` = not asked.
pub run_by_pipeline: Option<bool>,
}
/// A resolved configuration: the features actually on, per package, after
/// cargo's propagation across the workspace graph.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Resolved {
/// package → features on.
pub enabled: BTreeMap<String, BTreeSet<String>>,
/// `(package, feature)` → the sibling whose dependency edge introduced it.
/// Absent when the feature came from the package's own seeds.
pub via_edge: BTreeMap<(String, String), String>,
}
/// The transitive closure of a crate's own features, **keeping** the tokens
/// [`crate::feature_closure`] discards: `dep:x` (activates an optional dep) and
/// `x/f` / `x?/f` (turns `f` on in dep `x`). Those tokens are the whole mechanism
/// by which a sibling's feature comes on, so a resolver cannot throw them away.
///
/// Returns `(own features, dep tokens)`.
pub fn closure_split(
table: &BTreeMap<String, Vec<String>>,
seeds: &[String],
) -> (BTreeSet<String>, BTreeSet<String>) {
let mut own: BTreeSet<String> = BTreeSet::new();
let mut tokens: BTreeSet<String> = BTreeSet::new();
let mut queue: VecDeque<String> = seeds.iter().cloned().collect();
while let Some(f) = queue.pop_front() {
if f.starts_with("dep:") || f.contains('/') {
tokens.insert(f);
continue;
}
if !own.insert(f.clone()) {
continue;
}
if let Some(children) = table.get(&f) {
queue.extend(children.iter().cloned());
}
}
if !table.contains_key("default") {
own.remove("default");
}
(own, tokens)
}
/// `alias/feat` → `(alias, feat, weak=false)`; `alias?/feat` → weak.
fn split_cross(tok: &str) -> Option<(&str, &str, bool)> {
let (lhs, rhs) = tok.split_once('/')?;
match lhs.strip_suffix('?') {
Some(a) => Some((a, rhs, true)),
None => Some((lhs, rhs, false)),
}
}
/// **Run cargo's feature propagation over the workspace graph to a fixed point.**
///
/// Four ways a feature reaches a sibling, all of them measured in the
/// constellation and none of them visible to a scan that reads only variants:
///
/// 1. an **unconditional** edge with `features = [...]` — `holger/server/cli/Cargo.toml:71`;
/// 2. an **optional** edge activated by `dep:alias`, carrying `features = [...]`
/// — `holger/server/cli/Cargo.toml:99`, `server/ui/Cargo.toml:110`;
/// 3. the edge's `default-features` (on unless the manifest says otherwise);
/// 4. a cross-crate feature string `alias/feat` in the parent's own `[features]`
/// table, or its weak form `alias?/feat` which applies only if the dep is
/// already active.
///
/// Only WORKSPACE members are followed — a feature turned on in a crates.io
/// dependency is not a test surface this repo can run.
///
/// `[dev-dependencies]` edges are followed only out of packages the configuration
/// **selects**: a `cargo build` does not build dev-deps at all, and a
/// `cargo test -p X` builds X's but not those of a package it merely links.
pub fn resolve(crates: &[CrateManifest], cfg: &TestConfig) -> Resolved {
let by: BTreeMap<&str, &CrateManifest> = crates.iter().map(|c| (c.name.as_str(), c)).collect();
let mut r = Resolved::default();
for (pkg, seeds) in &cfg.seeds {
let Some(m) = by.get(pkg.as_str()) else {
continue;
};
let (own, _) = closure_split(&m.features, seeds);
r.enabled.entry(pkg.clone()).or_default().extend(own);
}
// Fixed point. Bounded by (packages × features), and the constellation's
// widest workspace is well under a thousand of each.
for _ in 0..64 {
let mut changed = false;
let names: Vec<String> = r.enabled.keys().cloned().collect();
for pkg in names {
let Some(m) = by.get(pkg.as_str()) else {
continue;
};
let on = r.enabled.get(&pkg).cloned().unwrap_or_default();
// Re-close so `dep:`/`x/f` tokens reachable from the enabled set are
// in hand (the seeds' closure dropped them into `tokens` once, but
// the set may have grown since through another edge).
let seeds: Vec<String> = on.iter().cloned().collect();
let (_, tokens) = closure_split(&m.features, &seeds);
for edge in &m.deps {
if edge.kind == DepKind::Dev && !cfg.selected.contains(&pkg) {
continue;
}
let Some(target) = by.get(edge.package.as_str()) else {
continue; // outside the workspace
};
let activated = !edge.optional
|| tokens.contains(&format!("dep:{}", edge.alias))
|| on.contains(&edge.alias)
|| tokens
.iter()
.filter_map(|t| split_cross(t))
.any(|(a, _, weak)| a == edge.alias && !weak);
if !activated {
continue;
}
let mut dep_seeds: Vec<String> = edge.features.clone();
if edge.uses_default_features {
dep_seeds.push("default".to_string());
}
for (a, f, _) in tokens.iter().filter_map(|t| split_cross(t)) {
if a == edge.alias {
dep_seeds.push(f.to_string());
}
}
let (add, _) = closure_split(&target.features, &dep_seeds);
let slot = r.enabled.entry(edge.package.clone()).or_default();
for f in add {
if slot.insert(f.clone()) {
changed = true;
r.via_edge
.entry((edge.package.clone(), f))
.or_insert_with(|| pkg.clone());
}
}
}
}
if !changed {
break;
}
}
r
}
/// The baseline configuration: `cargo test --workspace`, every member selected
/// with its own `default` features.
///
/// Deliberately the GENEROUS reading of "a human typed `cargo test`" — a repo
/// with a root package would run fewer members, and calling a feature dark that
/// some plain invocation does run is the false positive this module exists to
/// avoid.
pub fn default_config(crates: &[CrateManifest]) -> TestConfig {
TestConfig {
label: "cargo test --workspace".to_string(),
origin: ConfigOrigin::Default,
selected: crates.iter().map(|c| c.name.clone()).collect(),
seeds: crates
.iter()
.map(|c| (c.name.clone(), vec!["default".to_string()]))
.collect(),
run_by_pipeline: None,
}
}
/// A declared variant, modelled as the brokkr `tests` step that runs it:
/// `cargo test -p <package> [-p <test-package>…] --features …`.
///
/// A variant with no `package` (a single-crate repo) selects every member, which
/// is what a bare `cargo test --features …` at such a repo's root does.
pub fn config_for_variant(
crates: &[CrateManifest],
v: &VariantSpec,
test_packages: &[String],
run_by_pipeline: Option<bool>,
) -> TestConfig {
let mut selected: BTreeSet<String> = BTreeSet::new();
let mut seeds: BTreeMap<String, Vec<String>> = BTreeMap::new();
let mut feats = v.features.clone();
if !v.no_default_features {
feats.push("default".to_string());
}
match &v.package {
Some(p) => {
selected.insert(p.clone());
seeds.insert(p.clone(), feats);
for extra in test_packages {
selected.insert(extra.clone());
seeds
.entry(extra.clone())
.or_insert_with(|| vec!["default".to_string()]);
}
}
None => {
for c in crates {
selected.insert(c.name.clone());
seeds.insert(c.name.clone(), feats.clone());
}
}
}
TestConfig {
label: format!("variant `{}`", v.name),
origin: ConfigOrigin::Variant,
selected,
seeds,
run_by_pipeline,
}
}
/// A declared matrix arm, as the `cargo test …` it stands for.
pub fn config_for_arm(crates: &[CrateManifest], a: &MatrixArm) -> TestConfig {
let mut feats = a.features.clone();
if !a.no_default_features {
feats.push("default".to_string());
}
let members: Vec<String> = match &a.package {
Some(p) => vec![p.clone()],
None => crates.iter().map(|c| c.name.clone()).collect(),
};
let seeds = members
.iter()
.map(|m| {
let f = if a.all_features {
crates
.iter()
.find(|c| &c.name == m)
.map(|c| c.features.keys().cloned().collect())
.unwrap_or_default()
} else {
feats.clone()
};
(m.clone(), f)
})
.collect();
TestConfig {
label: format!("arm `{}`", a.command()),
origin: ConfigOrigin::Arm,
selected: members.into_iter().collect(),
seeds,
run_by_pipeline: None,
}
}
// ─── the verdict ───────────────────────────────────────────────────────────
/// What a repo actually does with a gated test surface.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum RunVerdict {
/// A declared configuration selects the package that OWNS the tests with the
/// feature on: the tests are compiled and executed.
Run,
/// The feature is on — through a sibling's dependency edge — but nothing
/// selects the owning package, so cargo compiles the code and never builds
/// its test targets. **Verified by running cargo, not inferred:**
/// `cargo test -p driver --features bench` on `tests/fixtures/dep_edge_lit`
/// runs 0 of `engine`'s 2 `served` tests while `cargo build` compiles them.
CompiledOnly,
/// Nothing declares the feature at all. Only a human typing `--features`
/// ever compiles these tests, and nothing at all runs them.
Dark,
}
/// One feature's whole story, folded across every site it gates.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct SurfaceRow {
/// The package whose tests these are.
pub package: String,
/// The gating feature.
pub feature: String,
/// Total `#[test]` fns behind it (max per site, not summed across the same
/// file seen twice).
pub tests: usize,
/// Every site the gate was found at.
pub sites: Vec<GateSite>,
/// The human-readable evidence lines.
pub evidence: Vec<String>,
/// The verdict.
pub verdict: RunVerdict,
/// The configurations that turn the feature on, with how.
pub lit_by: Vec<String>,
}
/// The report [`audit_test_surface`] produces.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct TestSurfaceReport {
/// The repo audited.
pub repo: String,
/// The run this belongs to.
pub run_id: String,
/// One row per (package, feature) that gates tests.
pub rows: Vec<SurfaceRow>,
/// How many workspace members were scanned — asserted by callers so a green
/// over an empty world is not mistaken for a green.
pub packages: usize,
/// The configurations considered, by label.
pub configs: Vec<String>,
}
impl TestSurfaceReport {
/// Rows nothing declares at all.
pub fn dark(&self) -> Vec<&SurfaceRow> {
self.rows
.iter()
.filter(|r| r.verdict == RunVerdict::Dark)
.collect()
}
/// Rows whose code is compiled by a declared build but whose tests nothing
/// runs.
pub fn compiled_only(&self) -> Vec<&SurfaceRow> {
self.rows
.iter()
.filter(|r| r.verdict == RunVerdict::CompiledOnly)
.collect()
}
/// Total `#[test]` fns behind a gate nothing declares.
pub fn dark_tests(&self) -> usize {
self.dark().iter().map(|r| r.tests).sum()
}
/// Total `#[test]` fns compiled by a declared build and run by nothing.
pub fn compiled_only_tests(&self) -> usize {
self.compiled_only().iter().map(|r| r.tests).sum()
}
/// Green when nothing is [`RunVerdict::Dark`]. [`RunVerdict::CompiledOnly`]
/// is deliberately NOT red: the code is built on every push, the declaration
/// loop is closed, and what is missing is a `test-packages` entry or an arm.
/// Calling it red would put a repo in the position of switching the whole
/// detector off to get past a row it cannot fix in one edit.
pub fn is_green(&self) -> bool {
self.dark().is_empty()
}
/// One line per row, dark first.
pub fn summary(&self) -> String {
if self.rows.is_empty() {
return format!(
"test-surface {}: no feature gates any test across {} package(s)",
self.repo, self.packages
);
}
let mut s = format!(
"test-surface {}: {} gated feature(s) over {} package(s) — {} DARK ({} test fns), \
{} compiled-but-never-run ({} test fns)\n",
self.repo,
self.rows.len(),
self.packages,
self.dark().len(),
self.dark_tests(),
self.compiled_only().len(),
self.compiled_only_tests(),
);
let mut rows: Vec<&SurfaceRow> = self.rows.iter().collect();
rows.sort_by_key(|r| (r.verdict, std::cmp::Reverse(r.tests)));
for r in rows {
let tag = match r.verdict {
RunVerdict::Run => "RUN",
RunVerdict::CompiledOnly => "COMPILED-ONLY",
RunVerdict::Dark => "DARK",
};
s.push_str(&format!(
" {tag:<14} {}/{} {} test fn(s)\n",
r.package, r.feature, r.tests
));
for e in &r.evidence {
s.push_str(&format!(" – {e}\n"));
}
for l in &r.lit_by {
s.push_str(&format!(" + {l}\n"));
}
}
s
}
/// The matrix rows: `pass` for RUN, `listed` for COMPILED-ONLY (the tests
/// exist and have not run — exactly what that status means), `fail` for DARK.
pub fn matrix_rows(&self) -> Vec<TestResultRow> {
let ts = now_micros();
self.rows
.iter()
.map(|r| TestResultRow {
run_id: self.run_id.clone(),
repo: self.repo.clone(),
suite: r.package.clone(),
test_name: format!("gate:{}", r.feature),
status: match r.verdict {
RunVerdict::Run => status::PASS,
RunVerdict::CompiledOnly => status::LISTED,
RunVerdict::Dark => status::FAIL,
}
.to_string(),
duration_ms: 0.0,
ts_micros: ts,
message: {
let mut m = r.evidence.join(" · ");
if !r.lit_by.is_empty() {
m.push_str(" | lit by: ");
m.push_str(&r.lit_by.join(", "));
}
m
},
aspect: ASPECT_TEST_SURFACE.to_string(),
metric: r.tests as f64,
})
.collect()
}
}
/// **The audit.** Scan every gate, resolve every declared configuration, and
/// subtract.
///
/// `variants` are [`VariantSpec`]s mapped by the caller from
/// `nornir_build_thing::BuildVariantSet` — this crate keeps build-thing as a
/// dev-dependency only (a normal edge is a `cargo metadata` cycle, measured
/// 2026-08-15), exactly as [`crate::configs_from_variants`] already does.
///
/// `run_variants`, when given, is the set of variant names a `brokkr.toml` step
/// actually names; a variant outside it lights nothing, because a declaration no
/// pipeline runs is a file somebody has to remember to build by hand.
pub fn audit_test_surface(
_repo_root: &Path,
repo: &str,
run_id: &str,
crates: &[CrateManifest],
variants: &[(VariantSpec, Vec<String>)],
arms: &MatrixArms,
run_variants: Option<&BTreeSet<String>>,
) -> TestSurfaceReport {
let mut configs = vec![default_config(crates)];
for (v, test_packages) in variants {
let run = run_variants.map(|s| s.contains(&v.name));
if run == Some(false) {
continue;
}
configs.push(config_for_variant(crates, v, test_packages, run));
}
for a in &arms.arms {
configs.push(config_for_arm(crates, a));
}
let resolved: Vec<(&TestConfig, Resolved)> =
configs.iter().map(|c| (c, resolve(crates, c))).collect();
// Fold the raw surfaces into one row per (package, feature).
let mut folded: BTreeMap<(String, String), SurfaceRow> = BTreeMap::new();
for g in gated_surfaces(crates) {
let e = folded
.entry((g.package.clone(), g.feature.clone()))
.or_insert_with(|| SurfaceRow {
package: g.package.clone(),
feature: g.feature.clone(),
tests: 0,
sites: Vec::new(),
evidence: Vec::new(),
verdict: RunVerdict::Dark,
lit_by: Vec::new(),
});
e.tests += g.tests;
if !e.sites.contains(&g.site) {
e.sites.push(g.site);
}
e.evidence.push(g.evidence);
}
for row in folded.values_mut() {
let mut best = RunVerdict::Dark;
for (cfg, res) in &resolved {
let Some(on) = res.enabled.get(&row.package) else {
continue;
};
if !on.contains(&row.feature) {
continue;
}
let via = res
.via_edge
.get(&(row.package.clone(), row.feature.clone()))
.cloned();
if cfg.selected.contains(&row.package) {
best = RunVerdict::Run;
row.lit_by.push(match &via {
Some(from) => format!(
"{} selects `{}` and `{}` arrives on the dependency edge from `{from}` \
— RUN",
cfg.label, row.package, row.feature
),
None => format!("{} selects `{}` with `{}` — RUN", cfg.label, row.package, row.feature),
});
} else {
if best == RunVerdict::Dark {
best = RunVerdict::CompiledOnly;
}
row.lit_by.push(format!(
"{} turns `{}` on in `{}` through the dependency edge from `{}`, but does \
not select `{}` — cargo builds the code and never builds its test targets, \
so these tests are COMPILED and never RUN. Fix: add `{}` to that variant's \
`test-packages`, or declare an arm that runs it.",
cfg.label,
row.feature,
row.package,
via.as_deref().unwrap_or("?"),
row.package,
row.package,
));
}
}
row.verdict = best;
}
TestSurfaceReport {
repo: repo.to_string(),
run_id: run_id.to_string(),
rows: folded.into_values().collect(),
packages: crates.len(),
configs: configs.into_iter().map(|c| c.label).collect(),
}
}
// ─── hole 2: the declared test surface vs. the dependency closure ──────────
/// The workspace members reachable from `pkg` through non-dev dependency edges —
/// the packages a `cargo build -p pkg` compiles.
///
/// This is the honest bound on a variant's `test-packages`: a package INSIDE the
/// closure is already compiled by the build with feature values cargo derived
/// from the variant's own `--features`, so adding it to the test selection runs
/// its tests in the configuration the artifact ships. A package OUTSIDE it drags
/// its own default features into the unified graph and makes the test run a
/// configuration nothing ships — the drift that inheriting the variant's feature
/// resolution exists to prevent.
pub fn dep_closure(crates: &[CrateManifest], pkg: &str) -> BTreeSet<String> {
let by: BTreeMap<&str, &CrateManifest> = crates.iter().map(|c| (c.name.as_str(), c)).collect();
let mut seen = BTreeSet::new();
let mut queue = VecDeque::from(vec![pkg.to_string()]);
while let Some(p) = queue.pop_front() {
if !seen.insert(p.clone()) {
continue;
}
let Some(m) = by.get(p.as_str()) else {
continue;
};
for e in &m.deps {
if e.kind == DepKind::Dev {
continue;
}
if by.contains_key(e.package.as_str()) {
queue.push_back(e.package.clone());
}
}
}
seen.remove(pkg);
seen
}
/// One thing wrong with a variant's declared sibling test surface.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum SurfaceGap {
/// The variant names a package that is NOT in its build's dependency
/// closure: testing it here is a configuration nothing ships.
Outsider {
/// The variant.
variant: String,
/// The package it should not have named.
package: String,
},
/// A workspace member the variant's build compiles, holding `tests`
/// `#[test]` fns, that no variant selects for testing.
Unclaimed {
/// The variant that builds it.
variant: String,
/// The package whose tests nothing runs.
package: String,
/// How many `#[test]` fns are at stake.
tests: usize,
},
}
/// **Hole 2's detector.** Difference each variant's declared `test-packages`
/// against the dependency closure its build already compiles.
///
/// `tests_per_package` is how many `#[test]` fns each member holds — the caller
/// has it from [`crate::list_tests`] or a source count; a package with none is
/// not reported, because widening a test selection to reach zero tests buys
/// nothing.
pub fn surface_gaps(
crates: &[CrateManifest],
variants: &[(VariantSpec, Vec<String>)],
tests_per_package: &BTreeMap<String, usize>,
) -> Vec<SurfaceGap> {
let mut out = Vec::new();
for (v, declared) in variants {
let Some(pkg) = v.package.as_deref() else {
continue; // a variant with no package already selects everything
};
let closure = dep_closure(crates, pkg);
for d in declared {
if !closure.contains(d) {
out.push(SurfaceGap::Outsider {
variant: v.name.clone(),
package: d.clone(),
});
}
}
for c in &closure {
if declared.contains(c) {
continue;
}
let n = tests_per_package.get(c).copied().unwrap_or(0);
if n == 0 {
continue;
}
out.push(SurfaceGap::Unclaimed {
variant: v.name.clone(),
package: c.clone(),
tests: n,
});
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn depths_ignore_braces_inside_literals() {
let src = "fn a() {\n let s = \"{{{\";\n let c = '{';\n let r = r#\"}}\"#;\n}\nfn b() {}\n";
let d = depths_per_line(src);
// line 0 starts at 0; the body lines are all at 1; the `}` line is at 1;
// `fn b` is back at 0. A naive counter would have drifted to 3 by line 2.
assert_eq!(&d[..6], &[0, 1, 1, 1, 1, 0], "{d:?}");
}
#[test]
fn a_lifetime_is_not_a_char_literal() {
let src = "fn a<'x>(v: &'x str) {\n let _ = v;\n}\nfn b() {}\n";
let d = depths_per_line(src);
assert_eq!(&d[..4], &[0, 1, 1, 0], "{d:?}");
}
#[test]
fn a_wrapped_cfg_attribute_is_joined() {
let lines = vec![
"#[cfg(all(",
" feature = \"a\",",
" feature = \"b\"",
"))]",
"pub mod m;",
];
let (joined, n) = join_attribute(&lines, 0);
assert_eq!(n, 4, "{joined}");
let body = outer_cfg_body(&joined).expect("body: {joined}");
let feats = parse_cfg_expr(&body).expect("parse").features();
assert_eq!(
feats.into_iter().collect::<Vec<_>>(),
vec!["a".to_string(), "b".to_string()]
);
}
#[test]
fn cross_crate_feature_strings_split_weak_from_strong() {
assert_eq!(split_cross("engine/probe"), Some(("engine", "probe", false)));
assert_eq!(split_cross("engine?/probe"), Some(("engine", "probe", true)));
assert_eq!(split_cross("dep:engine"), None);
assert_eq!(split_cross("plain"), None);
}
#[test]
fn closure_split_keeps_the_tokens_feature_closure_throws_away() {
let table: BTreeMap<String, Vec<String>> = [
("default".to_string(), vec![]),
(
"bench".to_string(),
vec!["dep:engine-lib".to_string(), "engine-lib/probe".to_string()],
),
]
.into_iter()
.collect();
let (own, tokens) = closure_split(&table, &["bench".to_string()]);
assert!(own.contains("bench") && own.len() == 1, "{own:?}");
assert!(tokens.contains("dep:engine-lib"), "{tokens:?}");
assert!(tokens.contains("engine-lib/probe"), "{tokens:?}");
// The existing closure drops both — which is why a resolver could not be
// built on it.
let plain = crate::feature_closure(&table, &["bench".to_string()]);
assert_eq!(plain.into_iter().collect::<Vec<_>>(), vec!["bench"]);
}
}