crates/nornir-testmatrix/src/targetcheck.rs
//! **Does every declared test target COMPILE?** — the step before every other
//! guard in this crate.
//!
//! # The hole this closes
//!
//! `cargo check` is not a proxy for `cargo test`, and **a test target that does
//! not compile is reported by nothing**. `cargo test` does not list it, the
//! matrix does not row it, and the gated-tests guard ([`crate::gatedtests`]) is
//! looking at a different failure entirely — a file that compiles to an *empty*
//! binary. A file that does not compile at all never reaches that question.
//!
//! Four independent instances, all measured 2026-08-22:
//!
//! 1. **korp `--features server`** — `cargo check --features server` was clean;
//! `cargo test --features server --lib` failed with 4× `E0308` (two tonic
//! versions in one signature). `control::assets_tests`, gated on that
//! feature, had therefore **never run**.
//! 2. **nornir** — 26 of 98 declared test targets could not compile: the whole
//! `viz_*` / `robot_*` surface, i.e. the entire robot-UI assertion surface.
//! `cargo test` never listed one of them.
//! 3. **korp `tests/robot_scene.rs`** — two `#[test]`s behind
//! `#![cfg(all(feature = "server", feature = "client"))]`. Its rescue arm had
//! been RED for 25 days; nothing said so.
//! 4. **korp-ui** — the `client,map3d-gpu,collector-feeds
//! --no-default-features` combination did not compile at all, and that
//! combination is *declared*, in the repo's own `.nornir/build-variants.toml`.
//! No suite built it.
//!
//! # What this module asks, and how
//!
//! For each **configuration** ([`CompileConfig`]) it runs one
//!
//! ```text
//! cargo build --tests --keep-going --message-format=json [-p P] [--features …]
//! ```
//!
//! `--tests` selects every target with `test = true` — lib-as-unittest,
//! bin-as-unittest, and every `tests/*.rs` — so the answer covers the same
//! targets `cargo test` would run. `--keep-going` is load-bearing: without it
//! cargo stops at the first broken target and the other twenty-five stay
//! invisible, which is the exact failure mode being fixed. `cargo test
//! --no-run` cannot be used — it rejects `--keep-going` (measured, cargo 1.97.1)
//! and would report one target where there are twenty-six.
//!
//! The verdict per target comes from the **JSON stream**, never from an exit
//! code: a `compiler-artifact` naming the target is proof it built, a
//! `compiler-message` at level `error` naming it is proof it did not, and a
//! declared target that appears in neither is *not accounted for* — a third
//! state, not a silent pass. An exit code cannot tell those apart.
//!
//! # Which configurations, and why not a hand-written list
//!
//! A list written by hand is a guess that rots. [`configs_from_metadata`]
//! enumerates from the repo's own data:
//!
//! * **default** — one workspace-wide build, what a plain `cargo test` compiles.
//! * **`--all-features`** — the opposite corner.
//! * **every `required-features` a test target declares** — a target cargo
//! *silently skips* when its features are absent, which is invisible in the
//! default build.
//!
//! and two more come from data the repo already keeps:
//!
//! * **every arm in `.nornir/testmatrix-arms.json`** ([`configs_from_arms`]) —
//! the rescue arms the gated-tests guard already trusts.
//! * **every variant in `.nornir/build-variants.toml`**
//! ([`configs_from_variants`]) — the combinations the repo *ships*. Instance 4
//! above is declared there and nowhere else, so this is the only enumeration
//! that would have caught it.
//!
//! What is **not** covered, stated plainly: the feature *powerset* (that is
//! [`crate::Aspect::FeaturePowerset`]'s job and is exponential), cross-compile
//! targets, non-dev profiles, and any combination a human runs but no file in
//! the repo declares. The claim is "every combination this repo declares",
//! not "every combination that exists".
//!
//! `configs_from_variants` takes [`VariantSpec`]s rather than reading the TOML
//! itself: `nornir-build-thing::BuildVariantSet` is the one writer of that file
//! and this crate's dependency budget is `serde` + `anyhow` (no `toml`). The
//! caller maps one type to the other in five lines.
//!
//! # Cost
//!
//! Compiling is the expensive part, so this is emphatically **not** something to
//! hang off a default `cargo test` — it belongs in an xtask verb or a declared
//! arm run where the arms are declared. See the crate README for the measured
//! per-repo wall-clock.
//!
//! ```no_run
//! use nornir_testmatrix::{workspace_crates, configs_from_metadata, check_targets};
//! let root = std::path::Path::new(".");
//! let crates = workspace_crates(root).unwrap();
//! let configs = configs_from_metadata(&crates);
//! let report = check_targets(root, &crates, &configs);
//! assert!(report.is_green(), "{}", report.summary());
//! ```
use std::collections::{BTreeMap, BTreeSet};
use std::path::Path;
use std::process::Command;
use std::time::Instant;
use serde::{Deserialize, Serialize};
use crate::armcheck::arm_args;
use crate::gatedtests::{CrateManifest, MatrixArm, MatrixArms};
use crate::harness::{now_micros, stamp_row};
use crate::model::{status, TestResultRow};
/// The matrix aspect these rows land on.
pub const ASPECT_TARGET_COMPILE: &str = "target-compile";
// ─── configurations ────────────────────────────────────────────────────────
/// Where a checked configuration came from. Recorded because a failure's fix
/// depends on it: a broken `BuildVariant` is a shipped artifact that does not
/// build, a broken `RequiredFeatures` config is a test nobody has ever run.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
pub enum ConfigSource {
/// A plain workspace build — what `cargo test` compiles.
Default,
/// `--all-features`.
AllFeatures,
/// A `required-features` set some test target declares.
RequiredFeatures,
/// An arm from `.nornir/testmatrix-arms.json`.
DeclaredArm,
/// A variant from `.nornir/build-variants.toml` — a combination the repo ships.
BuildVariant,
}
impl ConfigSource {
/// A short tag for the report.
pub fn tag(self) -> &'static str {
match self {
ConfigSource::Default => "default",
ConfigSource::AllFeatures => "all-features",
ConfigSource::RequiredFeatures => "required-features",
ConfigSource::DeclaredArm => "declared-arm",
ConfigSource::BuildVariant => "build-variant",
}
}
}
/// **How hard to compile.** Measured on oden 2026-08-22, nornir at `origin/main`,
/// 32 cores / `CARGO_BUILD_JOBS=8` / `CARGO_INCREMENTAL=0`, cold target dir:
///
/// | depth | argv | wall-clock | target dir | broken targets found |
/// |---|---|---|---|---|
/// | [`Depth::Check`] | `cargo check --all-targets` | **2 min 20 s** | **2.8 GB** | **26** |
/// | [`Depth::Build`] | `cargo build --tests` | 9 min 51 s | ~170 GB | 26 |
///
/// Same twenty-six, 4.2× the wall-clock and 60× the disk. So `Check` is the
/// default: a guard nobody can afford to run is a guard that gets switched off,
/// and 170 GB per repo is how `/home` hit 100 % five times in one night.
///
/// `Build` still exists and is not redundant — it is the only one that reaches
/// codegen and the linker, so it is what catches a monomorphisation error in a
/// generic that nothing instantiates during checking, a `const` evaluated at
/// codegen, and a missing native library. Ask for it where the artifact matters;
/// `Check` answers "would `cargo test` have been able to build this" for
/// everything that is a type or name error, which is every instance measured so
/// far.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
pub enum Depth {
/// `cargo check --all-targets` — type-check every target (tests, benches,
/// examples, bins) without codegen. The default.
Check,
/// `cargo build --tests` — all the way through codegen and the linker.
Build,
}
impl Default for Depth {
fn default() -> Self {
Depth::Check
}
}
impl Depth {
/// A short tag for the report.
pub fn tag(self) -> &'static str {
match self {
Depth::Check => "check",
Depth::Build => "build",
}
}
}
/// One feature configuration to compile every test target under.
///
/// The feature/package half is a [`MatrixArm`] on purpose — the arms file, the
/// arm verifier and this guard must render `--features` / `--no-default-features`
/// / `--all-features` identically or a "configuration" here would mean something
/// different from a "configuration" there. [`arm_args`] is the single writer.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CompileConfig {
/// Human handle for the report (`default`, `korp-ui`, `-p korp --features server`).
pub label: String,
/// Where this configuration was enumerated from.
pub source: ConfigSource,
/// The package + feature selection.
pub arm: MatrixArm,
/// Pass `--workspace` (only meaningful when the arm names no package).
pub workspace: bool,
/// How hard to compile. [`Depth::Check`] by default — see [`Depth`] for the
/// measured 4.2×-wall-clock / 60×-disk difference and what `Build` buys.
pub depth: Depth,
/// `--target <triple>`, carried from [`VariantSpec::target`]. Part of a
/// configuration's IDENTITY: a host and a wasm build of the same package with
/// the same features are two artifacts, not one.
pub target: Option<String>,
/// `--manifest-path <dir>/Cargo.toml`, carried from
/// [`VariantSpec::manifest_dir`] — how a DETACHED workspace is reached at all.
pub manifest_dir: Option<String>,
}
impl CompileConfig {
/// A workspace-wide configuration with the given features.
pub fn workspace_wide(label: impl Into<String>, source: ConfigSource, arm: MatrixArm) -> Self {
Self {
label: label.into(),
source,
arm,
workspace: true,
depth: Depth::default(),
target: None,
manifest_dir: None,
}
}
/// The cargo argv, **derived from [`arm_args`]** rather than re-rendered, so
/// this guard and [`crate::verify_arm`] can never disagree about what a
/// feature selection means.
///
/// `arm_args` yields cargo's *test* half; the verb is swapped to `build` and
/// `--tests` added, because `cargo test --no-run` rejects `--keep-going` and
/// without `--keep-going` only the first broken target is ever named.
pub fn cargo_args(&self) -> Vec<String> {
let mut v = arm_args(&self.arm);
debug_assert_eq!(v.first().map(String::as_str), Some("test"));
v[0] = match self.depth {
Depth::Check => "check".to_string(),
Depth::Build => "build".to_string(),
};
// A DETACHED workspace is reached by pointing cargo at its own manifest.
// `--workspace` is then wrong by construction (it would mean *that* other
// workspace), so the two are mutually exclusive here rather than in the
// caller — one writer for "which workspace am I compiling".
if let Some(dir) = &self.manifest_dir {
v.push("--manifest-path".to_string());
v.push(format!("{}/Cargo.toml", dir.trim_end_matches('/')));
} else if self.workspace && self.arm.package.is_none() {
v.insert(1, "--workspace".to_string());
}
// The variant's own cross-compile triple. Without it a wasm-only artifact is
// checked for the HOST, which cannot work and does not say so.
if let Some(t) = &self.target {
v.push("--target".to_string());
v.push(t.clone());
}
// `--all-targets` (check) is a superset of `--tests` (build): it also
// type-checks benches and examples, which cost nothing extra without
// codegen. `cargo build --all-targets` would LINK them, so the build
// depth stays narrowed to the targets this guard is about.
v.push(match self.depth {
Depth::Check => "--all-targets".to_string(),
Depth::Build => "--tests".to_string(),
});
v.push("--keep-going".to_string());
v.push("--message-format=json".to_string());
v
}
/// The same configuration, compiled all the way through codegen and the
/// linker instead of only type-checked.
pub fn built(mut self) -> Self {
self.depth = Depth::Build;
self
}
/// The command line, rendered from the argv (never a twin of it).
pub fn command(&self) -> String {
format!("cargo {}", self.cargo_args().join(" "))
}
/// The features this configuration turns on for `package`, using that
/// package's own feature table — the same resolution the gated-tests guard
/// uses, so `required-features` are judged against the real closure and not
/// against the literal `--features` list.
pub fn enabled_for(&self, krate: &CrateManifest) -> BTreeSet<String> {
krate.arm_set(&self.arm)
}
/// Does this configuration compile `package` at all?
pub fn selects(&self, package: &str) -> bool {
match &self.arm.package {
None => true,
Some(p) => p == package,
}
}
}
/// The half of a `nornir-build-thing` `BuildVariant` this guard needs.
///
/// A plain data hand-off: this crate does not depend on `nornir-build-thing`
/// (that edge is a `cargo metadata` cycle — measured 2026-08-15) and carries no
/// TOML parser, so the caller that already owns `BuildVariantSet` maps one to
/// the other.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct VariantSpec {
/// The variant's declared name.
pub name: String,
/// `-p <package>`.
pub package: Option<String>,
/// `--features a,b`.
pub features: Vec<String>,
/// `--no-default-features`.
pub no_default_features: bool,
/// `--target <triple>` — the variant's own cross-compile target
/// (`BuildVariant::target`). Dropping it at this seam made a wasm-only variant
/// run as a HOST `cargo check`, which answers "did not match any packages" and
/// read on the report as a shipped artifact that does not build.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub target: Option<String>,
/// The directory holding the `Cargo.toml` of the workspace this variant's
/// package belongs to, relative to the repo root — rendered as
/// `--manifest-path <dir>/Cargo.toml`. `None` (the normal case) means the
/// package is a member of the repo-root workspace.
///
/// **This is NOT `BuildVariant::member`, and the difference is measured, not
/// assumed.** `member` is the *suite member* an artifact IS — a node in the suite
/// dep graph and the dedicated system-user / service name, a logical handle and
/// not a path. skade declares `member = "skade"` on BOTH its variants, while
/// `skade-remote`'s package `skade-katalog-serve` lives in `serve/`: treating
/// `member` as a manifest directory would be right by accident for one variant
/// and wrong for the other. A detached workspace has to have its location
/// declared, so it gets a key of its own.
///
/// ⚠ **`None` here does not mean the repo declared nothing.** The mapping into
/// this field is only as good as the `BuildVariant` the caller parsed with, and
/// `BuildVariant` carries no `#[serde(deny_unknown_fields)]`: a
/// `nornir-build-thing` older than the key parses `manifest_dir = "serve"` and
/// throws it away without a word. MEASURED 2026-08-26 — the published
/// `nornir-build-thing 0.1.8` contains zero occurrences of `manifest_dir`, and
/// skade is the one repo in the constellation that consumes the crate from
/// crates.io (`skade/skade/Cargo.toml:206`, `version = "0.1.4"`) rather than
/// path-depping edda's tree, so skade's two declared `manifest_dir` lines are
/// dropped on the floor for anything built against its dev-dep. That is why
/// [`CompileReport::summary`] names the version-skew possibility out loud
/// instead of only telling the reader to add a line they may already have.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub manifest_dir: Option<String>,
}
/// Enumerate the configurations a repo's **cargo metadata** implies: the default
/// build, `--all-features`, and one per distinct `required-features` set any test
/// target declares.
///
/// The `required-features` half is the non-obvious one: cargo *silently skips* a
/// target whose required features are off. It does not warn, it does not fail,
/// and it does not appear in `cargo test` output — so without a configuration
/// that turns those features on, such a target is never compiled by anything.
pub fn configs_from_metadata(crates: &[CrateManifest]) -> Vec<CompileConfig> {
let mut out = vec![
CompileConfig::workspace_wide("default", ConfigSource::Default, MatrixArm::default()),
CompileConfig::workspace_wide("all-features", ConfigSource::AllFeatures, {
let mut a = MatrixArm::default();
a.all_features = true;
a
}),
];
let mut seen: BTreeSet<(String, Vec<String>)> = BTreeSet::new();
for k in crates {
for t in &k.tests {
if t.required_features.is_empty() {
continue;
}
let mut feats = t.required_features.clone();
feats.sort();
feats.dedup();
if !seen.insert((k.name.clone(), feats.clone())) {
continue;
}
let arm = MatrixArm::features(feats.clone()).package(k.name.clone());
out.push(CompileConfig {
label: format!("required-features {} [{}]", k.name, feats.join(",")),
source: ConfigSource::RequiredFeatures,
arm,
workspace: false,
depth: Depth::default(),
target: None,
manifest_dir: None,
});
}
}
out
}
/// Every declared rescue arm, as a configuration to compile under.
///
/// [`crate::verify_arm`] already compiles each arm — but only the arm's own
/// `--test <name>` slice of it. Here the arm's *features* are taken as a
/// configuration and **every** target of the selected package is compiled under
/// them, which is a strictly larger question.
pub fn configs_from_arms(arms: &MatrixArms) -> Vec<CompileConfig> {
arms.arms
.iter()
.map(|a| {
let mut arm = a.clone();
// The arm's `--test` narrowing is deliberately dropped: this guard
// asks about every target the configuration compiles, not one file.
arm.test = None;
CompileConfig {
label: format!("arm {}", a.command()),
source: ConfigSource::DeclaredArm,
arm,
workspace: true,
depth: Depth::default(),
target: None,
manifest_dir: None,
}
})
.collect()
}
/// Every declared build variant, as a configuration.
///
/// This is the enumeration that catches "the binary the owner actually runs does
/// not compile": the combination is declared in `.nornir/build-variants.toml`
/// and in no suite, so nothing else ever builds it.
pub fn configs_from_variants(variants: &[VariantSpec]) -> Vec<CompileConfig> {
// The dedup key carries the TARGET TRIPLE and the MANIFEST LOCATION as well as
// package+features. It used to be package+features+no-default only, which
// collapsed a host build and a wasm build of one package into a single
// configuration — two shipped artifacts, one check, and whichever variant came
// second was never compiled by anything.
type Key = (Option<String>, Vec<String>, bool, Option<String>, Option<String>);
let mut seen: BTreeSet<Key> = BTreeSet::new();
let mut out = Vec::new();
for v in variants {
let mut feats = v.features.clone();
feats.sort();
feats.dedup();
let key: Key = (
v.package.clone(),
feats.clone(),
v.no_default_features,
v.target.clone(),
v.manifest_dir.clone(),
);
if !seen.insert(key) {
continue;
}
let mut arm = MatrixArm::features(feats);
arm.package = v.package.clone();
arm.no_default_features = v.no_default_features;
out.push(CompileConfig {
label: format!("variant {}", v.name),
source: ConfigSource::BuildVariant,
arm,
workspace: v.package.is_none(),
depth: Depth::default(),
target: v.target.clone(),
manifest_dir: v.manifest_dir.clone(),
});
}
out
}
/// Drop configurations that are duplicates of one already in the list (same
/// package + features + flags), keeping the FIRST — so the cheap `default` build
/// wins over an arm that says the same thing.
pub fn dedup_configs(configs: Vec<CompileConfig>) -> Vec<CompileConfig> {
let mut seen: BTreeSet<Vec<String>> = BTreeSet::new();
configs.into_iter().filter(|c| seen.insert(c.cargo_args())).collect()
}
// ─── the cargo JSON stream (pure) ──────────────────────────────────────────
/// One target the compiler complained about.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct TargetError {
/// Package the target belongs to (from `package_id`).
pub package: String,
/// Target name (`viz_tabs_are_reachable`, `nornir`, …).
pub target: String,
/// `test`, `lib`, `bin` …
pub kind: String,
/// The FIRST error's one-line message — the thing a reader needs.
pub first_error: String,
}
/// **Warnings that mean a test silently stopped being a test.**
///
/// Not a general "warnings are errors" policy — that belongs to
/// [`crate::Aspect::Clippy`], and turning one on here would drown the signal.
/// These are the specific lints whose *entire* observable consequence is that
/// something a reader believes is running is not.
///
/// `duplicate_macro_attributes` is here because of a case measured on
/// 2026-08-22: korp's `korp_state_exposes_all_views` had not run for months. A
/// test function had been inserted between a `#[test]` attribute and the
/// function it belonged to, so one function carried two `#[test]`s and this one
/// carried none. It compiled, the suite was green throughout, and the only
/// signal in the whole build was a single `duplicate_macro_attributes` warning
/// nobody reads. That is one level finer than a target that does not compile —
/// a test that compiles, is listed nowhere, and draws no complaint — but it is
/// the same family, and the signal already existed. This makes it RED.
pub const SILENTLY_FATAL_LINTS: &[&str] = &["duplicate_macro_attributes"];
/// What one `cargo build --tests --message-format=json` stream says.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct BuildOutcome {
/// `(package, target, kind)` for every artifact that was produced.
pub built: BTreeSet<(String, String, String)>,
/// One entry per target that emitted at least one `level: error`.
pub errors: Vec<TargetError>,
/// `build-finished`'s own verdict, when cargo emitted one.
pub finished_ok: Option<bool>,
}
/// Package name out of a cargo `package_id`.
///
/// Handles both spellings cargo has used: the modern PURL-ish
/// `path+file:///x/y#nornir-flight@0.3.0` (and its short form
/// `path+file:///x/nornir-flight#0.3.0`) and the legacy
/// `nornir-flight 0.3.0 (path+file:///…)`.
pub fn package_name_from_id(id: &str) -> String {
if let Some(hash) = id.rfind('#') {
let tail = &id[hash + 1..];
if let Some((name, _ver)) = tail.split_once('@') {
return name.to_string();
}
// `…/the-name#1.2.3` — the name is the last path segment before `#`.
if tail.chars().next().is_some_and(|c| c.is_ascii_digit()) {
let head = &id[..hash];
if let Some(seg) = head.rsplit('/').next() {
return seg.to_string();
}
}
return tail.to_string();
}
id.split_whitespace().next().unwrap_or(id).to_string()
}
/// Parse a cargo JSON stream. **Pure** — no subprocess, so the accounting that
/// decides pass/fail is unit-testable without a compiler.
///
/// Tolerant of interleaved non-JSON progress lines and of unknown fields; NOT
/// tolerant of an absent verdict — a target that appears in neither `built` nor
/// `errors` stays unaccounted for rather than being read as fine.
pub fn parse_build_stream(stdout: &str) -> BuildOutcome {
let mut out = BuildOutcome::default();
let mut first: BTreeMap<(String, String, String), String> = BTreeMap::new();
let mut order: Vec<(String, String, String)> = Vec::new();
for line in stdout.lines() {
let line = line.trim();
if !line.starts_with('{') {
continue;
}
let Ok(v) = serde_json::from_str::<serde_json::Value>(line) else { continue };
match v.get("reason").and_then(|r| r.as_str()) {
Some("compiler-artifact") => {
let pkg = package_name_from_id(
v.get("package_id").and_then(|p| p.as_str()).unwrap_or_default(),
);
let name = v
.get("target")
.and_then(|t| t.get("name"))
.and_then(|n| n.as_str())
.unwrap_or_default()
.to_string();
let kind = v
.get("target")
.and_then(|t| t.get("kind"))
.and_then(|k| k.as_array())
.and_then(|a| a.first())
.and_then(|k| k.as_str())
.unwrap_or_default()
.to_string();
if name.is_empty() {
continue;
}
out.built.insert((pkg, name, kind));
}
Some("compiler-message") => {
let level = v
.get("message")
.and_then(|m| m.get("level"))
.and_then(|l| l.as_str())
.unwrap_or_default();
let lint = v
.get("message")
.and_then(|m| m.get("code"))
.and_then(|c| c.get("code"))
.and_then(|c| c.as_str())
.unwrap_or_default();
let fatal_warning = SILENTLY_FATAL_LINTS.contains(&lint);
if level != "error" && level != "error: internal compiler error" && !fatal_warning
{
continue;
}
let pkg = package_name_from_id(
v.get("package_id").and_then(|p| p.as_str()).unwrap_or_default(),
);
let name = v
.get("target")
.and_then(|t| t.get("name"))
.and_then(|n| n.as_str())
.unwrap_or_default()
.to_string();
let kind = v
.get("target")
.and_then(|t| t.get("kind"))
.and_then(|k| k.as_array())
.and_then(|a| a.first())
.and_then(|k| k.as_str())
.unwrap_or_default()
.to_string();
let msg = v
.get("message")
.and_then(|m| m.get("message"))
.and_then(|m| m.as_str())
.unwrap_or("(no message)");
let detail = match (lint.is_empty(), fatal_warning) {
(true, _) => msg.to_string(),
(false, false) => format!("{lint}: {msg}"),
// Spelled out, because a reader who sees a WARNING quoted as
// a failure needs to know why it is not merely noise.
(false, true) => format!(
"{lint}: {msg} — a warning, and RED here because its whole \
consequence is that something a reader believes is running is \
not"
),
};
// FIRST wins: `insert` would overwrite it with the last error,
// which is invariably `aborting due to N previous errors` — the
// one line that says nothing about what is wrong.
let key = (pkg, name, kind);
if !first.contains_key(&key) {
first.insert(key.clone(), detail);
order.push(key);
}
}
Some("build-finished") => {
out.finished_ok = v.get("success").and_then(|s| s.as_bool());
}
_ => {}
}
}
for key in order {
let detail = first.remove(&key).unwrap_or_default();
// A target that ALSO produced an artifact compiled despite the message
// (an error emitted by a different codegen unit of the same name is not
// a thing; but a `lib` artifact plus a `lib` unittest error IS, because
// the two share a target name and differ only by `profile.test`).
out.errors.push(TargetError {
package: key.0,
target: key.1,
kind: key.2,
first_error: detail,
});
}
out
}
// ─── verdicts ──────────────────────────────────────────────────────────────
/// What became of one declared target under one configuration.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum TargetState {
/// cargo produced an artifact for it.
Compiled,
/// cargo emitted a `level: error` naming it.
Failed,
/// Its `required-features` are not enabled here — cargo skipped it silently.
NotSelected,
/// Neither built nor named: something upstream broke first, so this target's
/// state is **unknown**. Never read as a pass.
NotAccountedFor,
}
impl TargetState {
/// The matrix status this state maps to.
pub fn status(self) -> &'static str {
match self {
TargetState::Compiled => status::PASS,
TargetState::Failed => status::FAIL,
TargetState::NotSelected => status::SKIP,
TargetState::NotAccountedFor => status::FAIL,
}
}
}
/// One declared target × one configuration.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct TargetVerdict {
/// Owning package.
pub package: String,
/// Target name.
pub target: String,
/// The configuration's label.
pub config: String,
/// Where the configuration came from.
pub source: ConfigSource,
/// The verdict.
pub state: TargetState,
/// The first compiler error, when there is one.
pub detail: String,
}
/// What happened to one whole configuration.
#[derive(Debug, Clone, PartialEq)]
pub struct ConfigOutcome {
/// The configuration's label.
pub label: String,
/// Where it came from.
pub source: ConfigSource,
/// The rendered command.
pub command: String,
/// Did cargo even start? `false` for e.g. a feature name that no longer
/// exists — which is itself a finding, not an excuse to skip the config.
pub cargo_ran: bool,
/// Why cargo could not run (empty when it did).
pub cargo_error: String,
/// Wall-clock milliseconds.
pub duration_ms: f64,
/// Errors against targets this guard does not track as test targets
/// (`lib`/`bin`/`example`) — a broken lib is why everything under it is
/// `NotAccountedFor`, so it must be named.
pub other_errors: Vec<TargetError>,
}
/// The whole answer for one repo.
#[derive(Debug, Clone)]
pub struct CompileReport {
/// Repo handle (for rows).
pub repo: String,
/// Run identity (for rows).
pub run_id: String,
/// One per configuration, in the order they were run.
pub configs: Vec<ConfigOutcome>,
/// How many `test`-kind targets the workspace DECLARES.
pub declared_test_targets: usize,
/// Every (target × configuration) verdict.
pub verdicts: Vec<TargetVerdict>,
}
impl CompileReport {
/// Targets that failed to compile in at least one configuration.
pub fn broken(&self) -> Vec<&TargetVerdict> {
self.verdicts.iter().filter(|v| v.state == TargetState::Failed).collect()
}
/// Targets that were never accounted for anywhere — the state that must not
/// be silently read as a pass.
pub fn unaccounted(&self) -> Vec<&TargetVerdict> {
self.verdicts.iter().filter(|v| v.state == TargetState::NotAccountedFor).collect()
}
/// Declared targets that compiled in **no** configuration. A target may be
/// legitimately `NotSelected` in most configurations, but if it never once
/// compiles then nothing in this repo has ever built it.
pub fn never_compiled(&self) -> Vec<(String, String)> {
let ok: BTreeSet<(&str, &str)> = self
.verdicts
.iter()
.filter(|v| v.state == TargetState::Compiled)
.map(|v| (v.package.as_str(), v.target.as_str()))
.collect();
let all: BTreeSet<(&str, &str)> =
self.verdicts.iter().map(|v| (v.package.as_str(), v.target.as_str())).collect();
all.difference(&ok).map(|(p, t)| ((*p).to_string(), (*t).to_string())).collect()
}
/// How many distinct declared targets compiled at least once.
pub fn compiled_targets(&self) -> usize {
self.verdicts
.iter()
.filter(|v| v.state == TargetState::Compiled)
.map(|v| (v.package.as_str(), v.target.as_str()))
.collect::<BTreeSet<_>>()
.len()
}
/// Configurations cargo refused to even start.
pub fn dead_configs(&self) -> Vec<&ConfigOutcome> {
self.configs.iter().filter(|c| !c.cargo_ran).collect()
}
/// The dead configurations that are a **DECLARATION gap, not a repo defect**:
/// cargo refused because the package is not a member of the workspace this
/// command was rooted at. That is what a detached workspace (`skade/`, `serve/`,
/// `rest/`) and a deliberately `exclude`d cross-compile package look like from
/// the repo root, and reporting them beside real compile failures made the guard
/// blame code for a missing `manifest_dir` line.
///
/// They still fail [`is_green`](Self::is_green) — a gap NAMED is not a gap
/// forgiven — but [`summary`](Self::summary) now says what to declare.
pub fn configs_outside_this_workspace(&self) -> Vec<&ConfigOutcome> {
self.dead_configs()
.into_iter()
.filter(|c| outside_this_workspace(&c.cargo_error))
.collect()
}
/// The dead configurations that really are failures of the repo — an unknown
/// feature name, an unloadable manifest, a cargo that would not spawn. Everything
/// [`dead_configs`](Self::dead_configs) reports minus the declaration gaps.
pub fn dead_configs_that_are_real_failures(&self) -> Vec<&ConfigOutcome> {
self.dead_configs()
.into_iter()
.filter(|c| !outside_this_workspace(&c.cargo_error))
.collect()
}
/// **The gate.**
///
/// False when anything failed to compile, when a declared target was never
/// accounted for, when a configuration could not be run — and, deliberately,
/// when the run examined NOTHING. A scan of zero targets or zero
/// configurations is the empty-scan false green this whole module exists to
/// stop; it fails here rather than printing a reassuring nothing.
pub fn is_green(&self) -> bool {
self.declared_test_targets > 0
&& !self.configs.is_empty()
&& !self.verdicts.is_empty()
&& self.compiled_targets() > 0
&& self.dead_configs().is_empty()
&& self.broken().is_empty()
&& self.unaccounted().is_empty()
&& self.never_compiled().is_empty()
&& self.configs.iter().all(|c| c.other_errors.is_empty())
}
/// A paste-into-a-failure report that names every target, its configuration
/// and its first error. Never a count on its own.
pub fn summary(&self) -> String {
let mut s = String::new();
if self.declared_test_targets == 0 {
s.push_str(&format!(
"EMPTY SCAN in {}: the workspace declares ZERO test targets, so this guard \
proved nothing. That is a failure, not a pass — a run that examines \
nothing is shaped exactly like a clean one.\n",
self.repo
));
return s;
}
if self.configs.is_empty() || self.verdicts.is_empty() {
s.push_str(&format!(
"EMPTY SCAN in {}: {} declared test target(s) but {} configuration(s) and \
{} verdict(s) — nothing was compiled.\n",
self.repo,
self.declared_test_targets,
self.configs.len(),
self.verdicts.len()
));
return s;
}
if self.is_green() {
return format!(
"{}: all {} declared test target(s) compile, across {} configuration(s) \
({}) — {} target×config verdict(s).\n",
self.repo,
self.declared_test_targets,
self.configs.len(),
self.configs
.iter()
.map(|c| c.label.as_str())
.collect::<Vec<_>>()
.join(", "),
self.verdicts.len()
);
}
s.push_str(&format!(
"TEST TARGETS THAT DO NOT COMPILE — a test target that does not compile is \
reported by nothing: `cargo test` does not list it and the matrix has no row \
for it.\n\nrepo {}: {} declared test target(s), {} compiled at least once, \
{} configuration(s) examined.\n",
self.repo,
self.declared_test_targets,
self.compiled_targets(),
self.configs.len()
));
for c in self.dead_configs_that_are_real_failures() {
s.push_str(&format!(
"\n CONFIGURATION WOULD NOT RUN — {} [{}]\n {}\n {}\n",
c.label,
c.source.tag(),
c.command,
c.cargo_error
));
}
// Separated on purpose: these are NOT "the shipped artifact does not build".
// They are a variant declaring a package this workspace root cannot reach,
// which is what a detached workspace or an `exclude`d cross-compile package
// looks like from here. The fix is one declared line, and the report has to
// say WHICH line — a gap named beats a gap hidden, and a gap misattributed is
// worse than either.
for c in self.configs_outside_this_workspace() {
s.push_str(&format!(
"\n CONFIGURATION NOT REACHABLE FROM THIS WORKSPACE ROOT — {} [{}]\n \
{}\n {}\n This is a DECLARATION gap, not a compile failure: the \
variant names a package that is not a member of the workspace at the \
repo root (a detached workspace such as `skade/` / `serve/` / `rest/`, \
or a package deliberately `exclude`d because it only builds for another \
target). Give the variant a `manifest_dir` naming the directory whose \
`Cargo.toml` owns that package — and a `target` if it is cross-compile \
only — and this configuration becomes runnable instead of dead.\n \
IF YOU ALREADY DECLARED `manifest_dir` AND THIS LINE HAS NOT CHANGED, \
the key is being DROPPED before it reaches this guard, not ignored by \
it. `nornir_build_thing::BuildVariant` carries no \
`#[serde(deny_unknown_fields)]`, so a `nornir-build-thing` older than \
the key parses the line and discards it in SILENCE — measured on the \
published 0.1.8, which contains zero occurrences of `manifest_dir`. \
Check the version your repo actually LINKS (`cargo tree -i \
nornir-build-thing`), not the one in edda's tree, and confirm the \
mapping into `VariantSpec::manifest_dir` is not `None`.\n",
c.label,
c.source.tag(),
c.command,
c.cargo_error
));
}
let broken = self.broken();
if !broken.is_empty() {
s.push_str(&format!("\n {} target×config FAILURE(S):\n", broken.len()));
for v in broken {
s.push_str(&format!(
" {}::{} [{} / {}]\n {}\n",
v.package,
v.target,
v.config,
v.source.tag(),
v.detail
));
}
}
for c in &self.configs {
for e in &c.other_errors {
s.push_str(&format!(
"\n NON-TEST TARGET BROKEN under {} — {}::{} ({})\n {}\n",
c.label, e.package, e.target, e.kind, e.first_error
));
}
}
let never = self.never_compiled();
if !never.is_empty() {
s.push_str(&format!(
"\n {} declared target(s) that compiled in NO configuration — nothing in \
this repo has ever built them:\n",
never.len()
));
for (p, t) in &never {
s.push_str(&format!(" {p}::{t}\n"));
}
}
let unacc = self.unaccounted();
if !unacc.is_empty() {
let mut seen: BTreeSet<(&str, &str)> = BTreeSet::new();
let mut lines = Vec::new();
for v in &unacc {
if seen.insert((v.package.as_str(), v.target.as_str())) {
lines.push(format!(" {}::{} [{}]\n", v.package, v.target, v.config));
}
}
s.push_str(&format!(
"\n {} target×config verdict(s) UNACCOUNTED FOR — cargo neither built them \
nor complained about them, which means something upstream broke first and \
their state is unknown. Unknown is not a pass:\n",
unacc.len()
));
for l in lines {
s.push_str(&l);
}
}
s
}
/// The matrix rows — one per target×configuration verdict.
pub fn rows(&self) -> Vec<TestResultRow> {
let mut rows: Vec<TestResultRow> = self
.verdicts
.iter()
.map(|v| {
stamp_row(
&self.run_id,
&self.repo,
&v.package,
ASPECT_TARGET_COMPILE,
&format!("{}::{} @ {}", v.package, v.target, v.config),
v.state.status(),
0.0,
&v.detail,
)
})
.collect();
// The empty scan needs a row of its own — otherwise a run that examined
// nothing publishes an empty row set, which reads as "all green".
if self.declared_test_targets == 0 || self.verdicts.is_empty() {
rows.push(stamp_row(
&self.run_id,
&self.repo,
&self.repo,
ASPECT_TARGET_COMPILE,
"target-compile::scan",
status::FAIL,
0.0,
"examined ZERO test targets — an empty scan is a failure, not a pass",
));
}
rows
}
}
// ─── the run ───────────────────────────────────────────────────────────────
/// Where a compile check's build output goes. A dedicated sibling of the
/// caller's target dir: the caller may itself be running under `cargo test`,
/// which holds the build lock on its own target dir for the whole invocation, so
/// a child cargo pointed at the same directory does not fail — it BLOCKS,
/// forever, and the guard looks like a hang instead of a result. Same reasoning
/// (and the same env override policy) as [`crate::arm_target_dir`].
pub fn compile_target_dir(repo_root: &Path) -> std::path::PathBuf {
if let Ok(explicit) = std::env::var("NORNIR_TARGETCHECK_TARGET_DIR") {
if !explicit.is_empty() {
return std::path::PathBuf::from(explicit);
}
}
match std::env::var("CARGO_TARGET_DIR") {
Ok(t) if !t.is_empty() => std::path::PathBuf::from(t).join("targetcheck"),
_ => repo_root.join("target").join("targetcheck"),
}
}
/// Run ONE configuration and return `(outcome, stream)`.
pub fn run_config(repo_root: &Path, config: &CompileConfig) -> (ConfigOutcome, BuildOutcome) {
let t0 = Instant::now();
let args = config.cargo_args();
let out = Command::new(std::env::var("CARGO").unwrap_or_else(|_| "cargo".into()))
.args(&args)
.current_dir(repo_root)
.env("CARGO_TARGET_DIR", compile_target_dir(repo_root))
.env_remove("RUSTC_WORKSPACE_WRAPPER")
.output();
let duration_ms = t0.elapsed().as_secs_f64() * 1000.0;
let mut outcome = ConfigOutcome {
label: config.label.clone(),
source: config.source,
command: config.command(),
cargo_ran: true,
cargo_error: String::new(),
duration_ms,
other_errors: Vec::new(),
};
let out = match out {
Ok(o) => o,
Err(e) => {
outcome.cargo_ran = false;
outcome.cargo_error = format!("could not spawn cargo: {e}");
return (outcome, BuildOutcome::default());
}
};
let stdout = String::from_utf8_lossy(&out.stdout).to_string();
let build = parse_build_stream(&stdout);
// cargo can fail BEFORE compiling anything — an unknown feature, a manifest
// it cannot load. That emits no JSON at all, and the difference between it
// and a clean build is invisible in the exit code alone.
if build.built.is_empty() && build.errors.is_empty() && build.finished_ok.is_none() {
outcome.cargo_ran = false;
let stderr = String::from_utf8_lossy(&out.stderr);
let msg: Vec<&str> = stderr
.lines()
.filter(|l| l.contains("error") || l.contains("Caused by"))
.take(4)
.collect();
outcome.cargo_error = if msg.is_empty() {
format!("cargo produced no JSON and exited {:?}", out.status.code())
} else {
msg.join(" | ")
};
}
(outcome, build)
}
/// **Compile every declared test target of `crates` under every `config`.**
///
/// The verdict per target comes from the JSON stream, so a target that cargo
/// never mentioned is `NotAccountedFor` rather than assumed fine.
pub fn check_targets(
repo_root: &Path,
crates: &[CrateManifest],
configs: &[CompileConfig],
) -> CompileReport {
let repo = repo_root
.file_name()
.map(|s| s.to_string_lossy().to_string())
.unwrap_or_else(|| repo_root.display().to_string());
let run_id = crate::model::new_run_id();
let declared_test_targets: usize = crates.iter().map(|k| k.tests.len()).sum();
let mut outcomes = Vec::new();
let mut verdicts = Vec::new();
for config in configs {
let (mut outcome, build) = run_config(repo_root, config);
let test_names: BTreeSet<(&str, &str)> = crates
.iter()
.flat_map(|k| k.tests.iter().map(move |t| (k.name.as_str(), t.name.as_str())))
.collect();
outcome.other_errors = build
.errors
.iter()
.filter(|e| !test_names.contains(&(e.package.as_str(), e.target.as_str())))
.cloned()
.collect();
for k in crates {
if !config.selects(&k.name) {
continue;
}
let enabled = config.enabled_for(k);
for t in &k.tests {
let (state, detail) = if let Some(e) = build
.errors
.iter()
.find(|e| e.package == k.name && e.target == t.name)
{
(TargetState::Failed, e.first_error.clone())
} else if build
.built
.iter()
.any(|(p, n, _)| p == &k.name && n == &t.name)
{
(TargetState::Compiled, String::new())
} else if !t.required_features.is_empty()
&& !t.required_features.iter().all(|f| enabled.contains(f))
{
(
TargetState::NotSelected,
format!(
"required-features [{}] not enabled here",
t.required_features.join(",")
),
)
} else if !outcome.cargo_ran {
(
TargetState::NotAccountedFor,
format!("configuration would not run: {}", outcome.cargo_error),
)
} else {
(
TargetState::NotAccountedFor,
"cargo neither built it nor complained about it".to_string(),
)
};
verdicts.push(TargetVerdict {
package: k.name.clone(),
target: t.name.clone(),
config: config.label.clone(),
source: config.source,
state,
detail,
});
}
}
outcomes.push(outcome);
}
CompileReport { repo, run_id, configs: outcomes, declared_test_targets, verdicts }
}
/// The one-liner a repo's own guard test calls: panic, naming every target, if
/// any declared test target does not compile in every configuration the repo
/// declares.
///
/// Reads `.nornir/testmatrix-arms.json` when present; build variants must be fed
/// in by the caller (see [`configs_from_variants`]).
pub fn assert_all_test_targets_compile(repo_root: &Path, extra: &[CompileConfig]) {
let crates = match crate::gatedtests::workspace_crates(repo_root) {
Ok(c) => c,
Err(e) => panic!("cargo metadata failed in {}: {e}", repo_root.display()),
};
let mut configs = configs_from_metadata(&crates);
if let Ok(Some(arms)) = MatrixArms::read(repo_root) {
configs.extend(configs_from_arms(&arms));
}
configs.extend(extra.iter().cloned());
let configs = dedup_configs(configs);
let report = check_targets(repo_root, &crates, &configs);
assert!(report.is_green(), "{}", report.summary());
}
/// Timestamp helper re-export used by callers building their own rows.
pub fn stamp() -> i64 {
now_micros()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::gatedtests::TestTarget;
fn krate(name: &str, tests: &[(&str, &[&str])]) -> CrateManifest {
CrateManifest {
name: name.to_string(),
// Widened by the gated-tests work: these three carry the SECOND and THIRD
// sources of "which feature is on" — the library target's own `#[cfg]`ed
// modules, and features turned on along a dependency EDGE. This fixture
// exercises neither, so empty is the honest value, not a placeholder.
src_roots: Vec::new(),
deps: Vec::new(),
manifest_path: Default::default(),
features: BTreeMap::from([
("default".to_string(), vec!["client".to_string()]),
("client".to_string(), vec![]),
("server".to_string(), vec![]),
("gui".to_string(), vec![]),
]),
tests: tests
.iter()
.map(|(n, rf)| TestTarget {
name: (*n).to_string(),
src_path: std::path::PathBuf::from(format!("tests/{n}.rs")),
required_features: rf.iter().map(|s| (*s).to_string()).collect(),
})
.collect(),
}
}
/// The argv must be `cargo build --tests --keep-going`, and its feature half
/// must be the one `arm_args` renders — not a second rendering that can
/// drift from the arms file's meaning.
#[test]
fn the_argv_is_build_tests_keep_going_and_reuses_arm_args() {
let c = CompileConfig::workspace_wide(
"default",
ConfigSource::Default,
MatrixArm::default(),
);
assert_eq!(
c.cargo_args(),
vec!["check", "--workspace", "--all-targets", "--keep-going", "--message-format=json"],
"the DEFAULT depth is check: same 26 findings on nornir at 4.2x less \
wall-clock and 60x less disk than a full build"
);
assert_eq!(
c.clone().built().cargo_args(),
vec!["build", "--workspace", "--tests", "--keep-going", "--message-format=json"],
"and the build depth still reaches codegen and the linker"
);
let mut arm = MatrixArm::features(["server", "client"]).package("korp");
arm.no_default_features = true;
let c = CompileConfig {
label: "korp-ui".into(),
source: ConfigSource::BuildVariant,
arm: arm.clone(),
workspace: false,
depth: Depth::default(),
target: None,
manifest_dir: None,
};
let args = c.clone().built().cargo_args();
assert_eq!(args[0], "build", "the verb is build: `cargo test` rejects --keep-going");
assert!(args.contains(&"--keep-going".to_string()), "{args:?}");
assert!(!args.contains(&"--workspace".to_string()), "a -p config is not workspace-wide");
// The feature half is arm_args' own output, minus the verb.
let mut expect = arm_args(&arm);
expect[0] = "build".into();
assert_eq!(expect[0], args[0]);
assert_eq!(&args[..expect.len()], &expect[..], "the feature half is arm_args'");
assert!(c.command().starts_with("cargo check "), "{}", c.command());
assert!(c.built().command().starts_with("cargo build "));
}
/// `required-features` is the enumeration that cannot come from a list a
/// human writes: cargo SKIPS such a target silently, so the only place it is
/// declared is the manifest.
#[test]
fn required_features_become_their_own_configuration() {
let crates = vec![
krate("korp", &[("plain", &[]), ("robot_scene", &["server", "client"])]),
krate("other", &[("dup", &["client", "server"])]),
];
let cfgs = configs_from_metadata(&crates);
assert_eq!(cfgs[0].source, ConfigSource::Default);
assert_eq!(cfgs[1].source, ConfigSource::AllFeatures);
let rf: Vec<&CompileConfig> =
cfgs.iter().filter(|c| c.source == ConfigSource::RequiredFeatures).collect();
assert_eq!(rf.len(), 2, "one per (package, feature-set): {cfgs:?}");
let cmds: Vec<String> = rf.iter().map(|c| c.command()).collect();
assert!(
cmds.iter().any(|c| c.contains("-p korp") && c.contains("--features client,server")),
"features are sorted so the same set is one config: {cmds:?}"
);
// A workspace whose targets declare no required-features yields exactly
// the two corners and nothing invented.
let plain = configs_from_metadata(&[krate("p", &[("a", &[])])]);
assert_eq!(plain.len(), 2, "{plain:?}");
}
/// A build variant is a combination the repo SHIPS. Duplicates collapse, and
/// `--no-default-features` survives — korp-ui's failing combination is
/// exactly `client,map3d-gpu --no-default-features`.
#[test]
fn build_variants_become_configurations_with_their_flags_intact() {
let specs = vec![
VariantSpec {
name: "korp-ui".into(),
package: Some("korp".into()),
features: vec!["client".into(), "map3d-gpu".into(), "collector-feeds".into()],
no_default_features: true,
target: None,
manifest_dir: None,
},
VariantSpec {
name: "korp-ui-again".into(),
package: Some("korp".into()),
features: vec!["collector-feeds".into(), "client".into(), "map3d-gpu".into()],
no_default_features: true,
target: None,
manifest_dir: None,
},
];
let cfgs = configs_from_variants(&specs);
assert_eq!(cfgs.len(), 1, "the same set in another order is the same build: {cfgs:?}");
let cmd = cfgs[0].command();
assert!(cmd.contains("--no-default-features"), "{cmd}");
assert!(cmd.contains("--features client,collector-feeds,map3d-gpu"), "{cmd}");
assert_eq!(cfgs[0].source, ConfigSource::BuildVariant);
}
#[test]
fn package_ids_parse_in_every_spelling_cargo_has_used() {
assert_eq!(
package_name_from_id("path+file:///home/x/nornir#nornir-flight@0.3.0"),
"nornir-flight"
);
assert_eq!(
package_name_from_id("path+file:///home/x/nornir-flight#0.3.0"),
"nornir-flight"
);
assert_eq!(
package_name_from_id("nornir-flight 0.3.0 (path+file:///home/x)"),
"nornir-flight"
);
assert_eq!(package_name_from_id("registry+https://x#serde@1.0.0"), "serde");
}
/// The stream parser must separate BUILT from FAILED by what cargo said, and
/// keep the FIRST error per target — a target with forty errors is named
/// once, with the error a reader needs.
#[test]
fn the_stream_parser_names_the_broken_target_and_its_first_error() {
let stream = concat!(
r#"{"reason":"compiler-artifact","package_id":"path+file:///r#nornir@1.0.0","target":{"name":"good_test","kind":["test"]},"profile":{"test":true},"executable":"/t/good"}"#,
"\n",
" Compiling nornir v1.0.0\n",
r#"{"reason":"compiler-message","package_id":"path+file:///r#nornir@1.0.0","target":{"name":"viz_tabs","kind":["test"]},"message":{"level":"error","code":{"code":"E0433"},"message":"failed to resolve: use of undeclared crate `build_state`"}}"#,
"\n",
r#"{"reason":"compiler-message","package_id":"path+file:///r#nornir@1.0.0","target":{"name":"viz_tabs","kind":["test"]},"message":{"level":"error","message":"aborting due to 36 previous errors"}}"#,
"\n",
r#"{"reason":"compiler-message","package_id":"path+file:///r#nornir@1.0.0","target":{"name":"viz_tabs","kind":["test"]},"message":{"level":"warning","message":"unused import"}}"#,
"\n",
r#"{"reason":"build-finished","success":false}"#,
);
let o = parse_build_stream(stream);
assert!(o.built.contains(&("nornir".into(), "good_test".into(), "test".into())));
assert_eq!(o.errors.len(), 1, "one entry per target, not per error: {:?}", o.errors);
assert_eq!(o.errors[0].target, "viz_tabs");
assert_eq!(
o.errors[0].first_error,
"E0433: failed to resolve: use of undeclared crate `build_state`",
"the FIRST error, with its code — not `aborting due to 36 previous errors`"
);
assert_eq!(o.finished_ok, Some(false));
// A warning-only stream has no errors at all.
let warn = r#"{"reason":"compiler-message","package_id":"p#a@1.0.0","target":{"name":"t","kind":["test"]},"message":{"level":"warning","message":"unused"}}"#;
assert!(parse_build_stream(warn).errors.is_empty());
}
/// A `#[test]` attribute stranded above the wrong function makes one function
/// carry two and its neighbour carry none. It compiles, the suite stays
/// green, and the ONLY signal in the whole build is one
/// `duplicate_macro_attributes` warning. Measured on korp 2026-08-22:
/// `korp_state_exposes_all_views` had not run for months.
///
/// So that one lint is RED here — and an ordinary warning still is not, or
/// the signal drowns.
#[test]
fn the_lint_that_means_a_test_stopped_being_a_test_is_red() {
let stream = concat!(
r#"{"reason":"compiler-message","package_id":"p#korp@1.0.0","target":{"name":"korp_state","kind":["test"]},"message":{"level":"warning","code":{"code":"duplicate_macro_attributes"},"message":"duplicated attribute"}}"#,
"\n",
r#"{"reason":"compiler-message","package_id":"p#korp@1.0.0","target":{"name":"other","kind":["test"]},"message":{"level":"warning","code":{"code":"unused_imports"},"message":"unused import: `std::fmt`"}}"#,
"\n",
r#"{"reason":"build-finished","success":true}"#,
);
let o = parse_build_stream(stream);
assert_eq!(o.errors.len(), 1, "an ordinary warning is NOT a failure: {:?}", o.errors);
assert_eq!(o.errors[0].target, "korp_state");
assert!(
o.errors[0].first_error.contains("duplicate_macro_attributes"),
"{}",
o.errors[0].first_error
);
assert!(
o.errors[0].first_error.contains("believes is running is not"),
"a warning quoted as a failure must say why it is not noise: {}",
o.errors[0].first_error
);
// And cargo said the build SUCCEEDED — which is exactly the problem.
assert_eq!(o.finished_ok, Some(true));
}
fn report_with(verdicts: Vec<TargetVerdict>, declared: usize) -> CompileReport {
CompileReport {
repo: "testrepo".into(),
run_id: "run-1".into(),
configs: vec![ConfigOutcome {
label: "default".into(),
source: ConfigSource::Default,
command: "cargo build --workspace --tests".into(),
cargo_ran: true,
cargo_error: String::new(),
duration_ms: 1.0,
other_errors: Vec::new(),
}],
declared_test_targets: declared,
verdicts,
}
}
fn verdict(target: &str, state: TargetState) -> TargetVerdict {
TargetVerdict {
package: "p".into(),
target: target.into(),
config: "default".into(),
source: ConfigSource::Default,
state,
detail: if state == TargetState::Failed { "E0308: mismatched types".into() } else { String::new() },
}
}
/// **The empty scan.** A run that examined nothing must FAIL. This is the
/// constellation's signature bug and the reason this guard exists at all —
/// so it is asserted directly, both on the verdict and on the emitted rows.
#[test]
fn a_run_that_examined_nothing_is_red_not_green() {
let empty = report_with(Vec::new(), 0);
assert!(!empty.is_green(), "zero declared targets is an EMPTY SCAN, never a pass");
assert!(empty.summary().contains("EMPTY SCAN"), "{}", empty.summary());
let rows = empty.rows();
assert_eq!(rows.len(), 1, "an empty scan still publishes a row: {rows:?}");
assert_eq!(rows[0].status, status::FAIL, "and that row is RED");
// Declared targets but no verdicts is the same hole from the other side.
let no_verdicts = report_with(Vec::new(), 12);
assert!(!no_verdicts.is_green());
assert!(no_verdicts.summary().contains("EMPTY SCAN"), "{}", no_verdicts.summary());
// Config list empty, verdicts empty: also red.
let mut no_configs = report_with(vec![verdict("a", TargetState::Compiled)], 1);
no_configs.configs.clear();
assert!(!no_configs.is_green());
}
/// A green run is only green when something actually compiled.
#[test]
fn green_requires_a_real_compiled_target() {
let good = report_with(vec![verdict("a", TargetState::Compiled)], 1);
assert!(good.is_green(), "{}", good.summary());
assert!(good.summary().contains("all 1 declared test target(s) compile"), "{}", good.summary());
// Every target merely SKIPPED is not a pass — nothing was proved.
let all_skipped = report_with(vec![verdict("a", TargetState::NotSelected)], 1);
assert!(!all_skipped.is_green(), "a target that never compiles anywhere is red");
assert!(all_skipped.summary().contains("compiled in NO configuration"), "{}", all_skipped.summary());
}
/// A failure must be NAMED, with its configuration and its first error — an
/// exit code is not a report.
#[test]
fn a_broken_target_is_named_with_its_config_and_first_error() {
let r = report_with(
vec![verdict("good", TargetState::Compiled), verdict("viz_tabs", TargetState::Failed)],
2,
);
assert!(!r.is_green());
let s = r.summary();
assert!(s.contains("p::viz_tabs"), "{s}");
assert!(s.contains("default"), "the configuration is named: {s}");
assert!(s.contains("E0308: mismatched types"), "the first error is quoted: {s}");
let rows = r.rows();
assert_eq!(rows.len(), 2);
assert_eq!(rows.iter().filter(|x| x.status == status::FAIL).count(), 1);
assert!(rows.iter().any(|x| x.test_name.contains("viz_tabs") && x.aspect == ASPECT_TARGET_COMPILE));
}
/// "cargo said nothing about it" is a THIRD state. Reading it as a pass is
/// how twenty-five broken targets hide behind one broken lib.
#[test]
fn a_target_cargo_never_mentioned_is_unknown_and_unknown_is_red() {
let r = report_with(
vec![
verdict("a", TargetState::Compiled),
verdict("b", TargetState::NotAccountedFor),
],
2,
);
assert!(!r.is_green());
assert!(r.summary().contains("UNACCOUNTED FOR"), "{}", r.summary());
assert_eq!(r.unaccounted().len(), 1);
}
/// A configuration cargo refuses to start (a feature that no longer exists)
/// is a finding, not a skip.
#[test]
fn a_configuration_cargo_will_not_run_is_red_and_named() {
let mut r = report_with(vec![verdict("a", TargetState::Compiled)], 1);
r.configs.push(ConfigOutcome {
label: "variant korp-ui".into(),
source: ConfigSource::BuildVariant,
command: "cargo build -p korp --features gone --tests".into(),
cargo_ran: false,
cargo_error: "error: none of the selected packages contains these features: gone"
.into(),
duration_ms: 5.0,
other_errors: Vec::new(),
});
assert!(!r.is_green());
let s = r.summary();
assert!(s.contains("CONFIGURATION WOULD NOT RUN"), "{s}");
assert!(s.contains("variant korp-ui"), "{s}");
assert!(s.contains("contains these features: gone"), "{s}");
}
/// A broken lib/bin is why its test targets are unaccounted for, so it must
/// be named too — instance 4 (korp-ui's own binary) is a `bin`, not a test.
#[test]
fn a_broken_non_test_target_is_named_as_well() {
let mut r = report_with(vec![verdict("a", TargetState::Compiled)], 1);
r.configs[0].other_errors.push(TargetError {
package: "korp".into(),
target: "korp-ui".into(),
kind: "bin".into(),
first_error: "E0432: unresolved import `crate::map3d`".into(),
});
assert!(!r.is_green(), "a shipped binary that does not compile is red");
let s = r.summary();
assert!(s.contains("NON-TEST TARGET BROKEN"), "{s}");
assert!(s.contains("korp::korp-ui"), "{s}");
assert!(s.contains("E0432"), "{s}");
}
/// Two configurations that render the same argv are one build, not two —
/// compiling is the expensive part.
#[test]
fn identical_configurations_collapse_and_the_first_wins() {
let a = CompileConfig::workspace_wide("default", ConfigSource::Default, MatrixArm::default());
let b = CompileConfig::workspace_wide(
"arm cargo test",
ConfigSource::DeclaredArm,
MatrixArm::default(),
);
let c = CompileConfig::workspace_wide(
"all-features",
ConfigSource::AllFeatures,
{
let mut m = MatrixArm::default();
m.all_features = true;
m
},
);
let got = dedup_configs(vec![a, b, c]);
assert_eq!(got.len(), 2, "{got:?}");
assert_eq!(got[0].source, ConfigSource::Default, "the first spelling wins");
}
/// An arm's `--test` narrowing is dropped: this guard asks about every target
/// the configuration compiles, which is a bigger question than the arm's own.
#[test]
fn an_arm_becomes_a_whole_configuration_not_one_file() {
let arms = MatrixArms::new([MatrixArm::features(["gui"])
.package("holger-ui")
.test("robot_ui_app")]);
let cfgs = configs_from_arms(&arms);
assert_eq!(cfgs.len(), 1);
assert!(cfgs[0].arm.test.is_none(), "the --test narrowing is dropped");
assert!(
!cfgs[0].command().contains("--test robot_ui_app"),
"the file narrowing is gone from the argv, but the whole-target flag stays: {}",
cfgs[0].command()
);
assert!(cfgs[0].command().contains("--all-targets"), "{}", cfgs[0].command());
assert!(cfgs[0].command().contains("-p holger-ui --features gui"), "{}", cfgs[0].command());
assert!(cfgs[0].label.contains("robot_ui_app"), "the label still says which arm");
}
}
#[cfg(test)]
mod detached_workspace_tests {
use super::*;
fn wasm_variant() -> VariantSpec {
VariantSpec {
name: "knut-bifrost-wasm-guest".into(),
package: Some("knut-bifrost-wasm-guest".into()),
features: vec![],
no_default_features: false,
target: Some("wasm32-unknown-unknown".into()),
manifest_dir: None,
}
}
fn detached_variant() -> VariantSpec {
VariantSpec {
name: "skade-remote".into(),
// NOTE the package name and the directory DISAGREE — that is the whole
// point of the key, see `VariantSpec::manifest_dir`.
package: Some("skade-katalog-serve".into()),
features: vec![],
no_default_features: false,
target: None,
manifest_dir: Some("serve".into()),
}
}
/// RED-when-broken: `BuildVariant::target` (the cross-compile triple) was
/// DROPPED at the `VariantSpec` seam, so a wasm-only variant was run as a HOST
/// `cargo check` and answered "did not match any packages" — reported as a dead
/// configuration, i.e. as if the shipped artifact did not build.
#[test]
fn a_cross_compile_variant_renders_its_target_triple() {
let cfgs = configs_from_variants(&[wasm_variant()]);
assert_eq!(cfgs.len(), 1);
let args = cfgs[0].cargo_args();
let i = args.iter().position(|a| a == "--target").unwrap_or_else(|| {
panic!("the variant's `target` must reach the argv, got {args:?}")
});
assert_eq!(args.get(i + 1).map(String::as_str), Some("wasm32-unknown-unknown"));
assert!(cfgs[0].command().contains("--target wasm32-unknown-unknown"));
}
/// RED-when-broken: a variant whose package lives in a DETACHED workspace
/// (`skade/`, `serve/`, `rest/` — the deliberate pattern that keeps the core
/// library's manifest and lockfile clean) can never be reached by `-p <pkg>`
/// from the repo root, whatever features are passed. It needs a manifest
/// location, and the variant now declares one.
#[test]
fn a_detached_workspace_variant_renders_its_manifest_path() {
let cfgs = configs_from_variants(&[detached_variant()]);
let args = cfgs[0].cargo_args();
let i = args
.iter()
.position(|a| a == "--manifest-path")
.unwrap_or_else(|| panic!("the variant's `manifest_dir` must reach the argv: {args:?}"));
assert_eq!(args.get(i + 1).map(String::as_str), Some("serve/Cargo.toml"));
assert!(
!args.iter().any(|a| a == "--workspace"),
"a detached-workspace config must not also say --workspace: {args:?}"
);
}
/// RED-when-broken: the dedup key was `(package, features, no_default_features)`
/// only, so a HOST variant and a WASM variant of the same package with the same
/// features collapsed into ONE configuration — and whichever came second was
/// silently never checked. Two artifacts, one check.
#[test]
fn a_host_and_a_wasm_variant_of_one_package_are_two_configurations() {
let host = VariantSpec {
name: "guest-host".into(),
package: Some("knut-bifrost-wasm-guest".into()),
features: vec![],
no_default_features: false,
target: None,
manifest_dir: None,
};
let cfgs = configs_from_variants(&[host, wasm_variant()]);
assert_eq!(
cfgs.len(),
2,
"the cross-compile triple is part of a configuration's identity: {:?}",
cfgs.iter().map(|c| c.command()).collect::<Vec<_>>()
);
}
/// RED-when-broken: cargo's refusal to select a package that is not a member of
/// THIS workspace was folded into `dead_configs()` alongside genuine compile
/// failures, so a detector gap read on the report as "the shipped artifact does
/// not build". It is a DECLARATION gap and must be named as one — while still
/// failing the gate, because a gap named is not a gap forgiven.
#[test]
fn a_package_outside_this_workspace_is_named_as_a_declaration_gap() {
// The two real cargo messages, verbatim in shape.
assert!(outside_this_workspace(
"error: package(s) `skade-katalog-serve` not found in workspace \
`/home/alice/git/skade/Cargo.toml`"
));
assert!(outside_this_workspace(
"error: cannot specify features for packages outside of workspace"
));
assert!(outside_this_workspace("error: did not match any packages"));
// A genuine build failure is NOT this.
assert!(!outside_this_workspace(
"error: none of the selected packages contains these features: gone"
));
assert!(!outside_this_workspace("error: could not compile `korp` due to 3 errors"));
let report = CompileReport {
repo: "skade".into(),
run_id: "r".into(),
declared_test_targets: 1,
verdicts: vec![TargetVerdict {
package: "skade-katalog".into(),
target: "t".into(),
config: "default".into(),
source: ConfigSource::Default,
state: TargetState::Compiled,
detail: String::new(),
}],
configs: vec![
ConfigOutcome {
label: "variant skade-remote".into(),
source: ConfigSource::BuildVariant,
command: "cargo check -p skade-katalog-serve".into(),
cargo_ran: false,
cargo_error: "error: did not match any packages".into(),
duration_ms: 1.0,
other_errors: vec![],
},
ConfigOutcome {
label: "variant broken".into(),
source: ConfigSource::BuildVariant,
command: "cargo check -p korp".into(),
cargo_ran: false,
cargo_error: "error: none of the selected packages contains these \
features: gone"
.into(),
duration_ms: 1.0,
other_errors: vec![],
},
],
};
let gaps = report.configs_outside_this_workspace();
assert_eq!(gaps.len(), 1, "exactly the detached one is a declaration gap");
assert_eq!(gaps[0].label, "variant skade-remote");
let real = report.dead_configs_that_are_real_failures();
assert_eq!(real.len(), 1, "the feature-name failure stays a real dead config");
assert_eq!(real[0].label, "variant broken");
assert!(!report.is_green(), "both still fail the gate — a gap NAMED, not forgiven");
let s = report.summary();
assert!(
s.contains("manifest_dir"),
"the summary must say what to DECLARE, not just that it failed:\n{s}"
);
}
}
/// **Did cargo refuse because the package is not in THIS workspace?**
///
/// The one writer of that judgement, so the report's classification and the
/// summary's advice can never disagree. Matched on cargo's own wording, of which
/// there are three shapes in the wild (measured against skade's two detached-
/// workspace variants and knut's `exclude`d wasm guest):
///
/// * `did not match any packages` — `-p <pkg>` for a package the workspace root
/// does not contain (an `exclude`d member, or a member of a detached workspace);
/// * `cannot specify features for packages outside of workspace` — the same, when
/// the configuration also carries `--features`;
/// * `not found in workspace` — newer cargo's spelling of the first.
///
/// A genuine failure of the repo must NOT match: an unknown feature name
/// (`none of the selected packages contains these features`) and a compile error
/// (`could not compile`) stay real dead configurations.
pub fn outside_this_workspace(cargo_error: &str) -> bool {
let e = cargo_error.to_ascii_lowercase();
e.contains("did not match any packages")
|| e.contains("packages outside of workspace")
|| e.contains("not found in workspace")
}