server/lib/src/twin.rs
//! **THE TWIN'S ROLE** — plan §3 W9, `.nornir/stinky/02-twin-growth.md` §4.1 and
//! S2-3.
//!
//! `private-holger-ops/estate.toml` declares a twin beside the primary and is
//! emphatic that it runs THE SAME BINARY and THE SAME CONFIG — byte-identical,
//! a refusal rather than a note if it drifts. That part was right and is
//! untouched here. What was missing, MEASURED on `main` at `2fb3602`, is that
//! the twin had no **role**: nothing made it refuse a write, nothing kept its
//! product doors off the public interface, and nothing made it follow anything.
//! Installed today it was a *second empty primary* whose public-shaped doors
//! were closed by a firewall rule and nothing else.
//!
//! gunnar's scar is the law here. Its metered appliance had an UN-metered twin
//! (`STINK-LIST-2026-09-19.md:130-131`); promoted, it would have stopped
//! enforcing entitlements and said nothing. The lesson holger takes is that a
//! follower must be **declared** and must **refuse by name**, not be trusted to
//! behave because no client happens to know its address.
//!
//! ## What a role is, and what it is not
//!
//! * [`NodeRole::Primary`] is the **default, and an absent key can never mean
//! `Twin`**. Reading "no role stated" as "follower" would turn a config typo
//! into a server that refuses its operator's writes; reading it as `Primary`
//! keeps every config written before this key existed doing exactly what it
//! did. [`an_absent_role_is_primary_not_twin`](tests::an_absent_role_is_primary_not_twin)
//! is the guard.
//! * [`NodeRole::Twin`] **names its primary**. A refusal that says "no" is an
//! outage; a refusal that says "this box is a twin of `<primary>`, write
//! there" is an instruction. The name is therefore mandatory in the variant,
//! not an `Option`.
//!
//! ## Three behaviours, all keyed off that one declaration
//!
//! 1. **Every write and admin mutation is refused by name** — `holger-twin-1`,
//! at the same two choke points `OperationMode::ReadOnly` already sits on
//! (`crate::grpc::HolgerGrpc::authorize_write` / `authorize_admin`) and on
//! the HTTP gateway's write gate. Reads are untouched: a twin is a follower,
//! not a box that is off.
//! 2. **No public door; the control plane where gunnar keeps it.**
//! [`NodeRole::effective_bind`] rewrites every `ron_http_url` (the public
//! HTTP/OCI doors) onto the loopback interface, keeping the port, and
//! `crate::wire_holger` applies it to the whole graph — so a door added to
//! the config later is loopbacked too, without anyone remembering to. The
//! gRPC CONTROL door (`ron_url`) stays on its configured address
//! ([`NodeRole::effective_control_bind`]; owner ruling 2026-09-27, "look like
//! gunnar": gunnar's twin keeps :50051 on the utility address) — every write
//! and admin mutation there is still refused by name, and the front can
//! certify a RESTARTED twin and recharge a dead main against it.
//! The config file stays byte-identical to the primary's, which is what
//! `[estate-twin-drift]` demands; the REWRITE is at wiring time.
//! 3. **The twin PULLS; the primary dials nobody.** This is gunnar's proven
//! direction (`twin-transaction-log.md:74-84`, `DATA-SET-GROWTH-FLOW.md:65`
//! F13), and stinky 02 §4.2 records that holger's owed driver was specified
//! the other way round. Here the direction is a TYPE fact: only a `Twin`
//! yields a [`FollowPlan`], and the only thing that holds a [`PrimaryView`]
//! is the follower. A `Primary` has no field naming a twin, so there is
//! nothing for it to dial.
//!
//! ## The transport, stated honestly
//!
//! [`PrimaryView`] is what the follower needs FROM the primary. The
//! implementation shipped here is [`LocalPrimary`] — a primary data root this
//! process can read (the laptop-testable form, and the form a restore takes
//! when the primary's volume is attached). The gRPC form is `holger-remote`,
//! plan W11, still owed; it plugs in here and changes nothing above it. Saying
//! this out loud is the point: stinky 02 S2-1 measured that holger has NO
//! network replication at all, and a module that implied otherwise would be the
//! green-looking dead appliance the serving marker exists to catch.
//!
//! **The network form exists now (2026-09-27)**, beside this seam rather than
//! through it: [`crate::twin_follow`] mirrors every repository and pulls the
//! estate over the primary's replication door, mutual TLS 1.3 under the
//! primary's OWN issuer ([`crate::twin_tls`], `nordisk-twin`), and a dead main
//! is recharged from its twin ([`crate::recharge_lane`]). The role is LIVE —
//! [`TwinPosture::become_twin`]; only the loopback bind waits for a start.
//!
//! ## `twin ≥ primary` — a runtime reading, not a declaration (?6, S2-10)
//!
//! `metering::statvfs` and `AdminService.Capacity` landed 2026-09-20, so the
//! fill reading the forced order waits on now EXISTS. [`store_verdict`] states
//! the invariant over two [`Capacity`] readings: a twin whose SERVABLE
//! `available_bytes` is below the primary's `used_by_holger_bytes` cannot hold
//! the data, and is reported **by name** (`holger-twin-2`) on the twin door and
//! in the log. It is a WARNING and never a refusal — refusing here would take a
//! restore source offline for a disk that is merely small, which is the
//! opposite of what a twin is for. Nothing in this module resizes anything:
//! `xtask grow --apply` needs the hot-plug kernel and is the owner's.
use crate::lock::{LockOrRecover as _, RwLockOrRecover as _};
use std::path::{Path, PathBuf};
use std::sync::{Arc, RwLock};
use serde::{Deserialize, Serialize};
use crate::capacity::Capacity;
/// **What a twin heard of its primary** — the shared fence gate's
/// `nordisk_twin::fence::Hearing` in an estate build (`estate`, H2's estate cut).
#[cfg(feature = "estate")]
pub use nordisk_twin::fence::Hearing;
#[cfg(feature = "estate")]
use nordisk_twin::fence::{IDLE as HEARD_IDLE, UNREACHABLE as HEARD_UNREACHABLE};
/// **What a twin heard of its primary**, field for field nordisk-twin's
/// `fence::Hearing` (a build without `estate` links no nordisk-twin; the
/// `TwinStatus` answer is the same).
#[cfg(not(feature = "estate"))]
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Hearing {
pub twin: bool,
pub primary: String,
pub last_answer_unix_ms: i64,
pub answer_ago_ms: u64,
pub primary_head: u64,
pub state: String,
}
#[cfg(not(feature = "estate"))]
const HEARD_IDLE: &str = "idle";
#[cfg(not(feature = "estate"))]
const HEARD_UNREACHABLE: &str = "unreachable";
/// The loopback host every twin door is rewritten onto. IPv4 by house law —
/// holger's estate runs no IPv6 (`korp-start` ruling).
pub const TWIN_LOOPBACK_HOST: &str = "127.0.0.1";
/// How often a follower asks its primary, when the config does not say.
/// gunnar's twin polls every 2 s over a utility network
/// (`twin-transaction-log.md:285`); holger's unit of replication is a whole
/// repository rather than a transaction-log frame, so the default here is a
/// minute — slow enough not to re-list a large store continuously, fast enough
/// that "caught up" means something on the day of a promotion.
pub const DEFAULT_POLL_SECONDS: u64 = 60;
/// **What this box IS** — the `role:` key of the RON config.
///
/// Absent ⇒ [`NodeRole::Primary`]. See the module docs for why that direction
/// of default is the only safe one.
#[derive(Serialize, Deserialize, Clone, Debug, Default, PartialEq, Eq)]
pub enum NodeRole {
/// The box that serves and accepts writes. The default, and what every
/// config written before this key existed still means.
#[default]
Primary,
/// A follower of `primary`. Refuses writes by name, binds only loopback,
/// and pulls.
Twin {
/// The primary this box follows, as an address a client would use
/// (`"holger-primary.utility:50051"`). It is NAMED in every refusal, so
/// an operator who wrote to the wrong box is told which is the right
/// one.
primary: String,
/// Seconds between follow cycles. Absent ⇒ [`DEFAULT_POLL_SECONDS`].
#[serde(default = "default_poll_seconds")]
poll_seconds: u64,
},
}
fn default_poll_seconds() -> u64 {
DEFAULT_POLL_SECONDS
}
impl NodeRole {
/// True when this box is a follower.
pub fn is_twin(&self) -> bool {
matches!(self, NodeRole::Twin { .. })
}
/// The primary this box follows, or `None` on a primary.
pub fn primary(&self) -> Option<&str> {
match self {
NodeRole::Primary => None,
NodeRole::Twin { primary, .. } => Some(primary.as_str()),
}
}
/// The human half of the refusal a write gets on a twin — it says what this
/// box is AND where the write belongs. `None` on a primary, which refuses
/// nothing on account of its role.
pub fn write_refusal(&self) -> Option<String> {
self.primary().map(|p| {
format!(
"this server is a TWIN and accepts no writes; it follows the primary \
`{p}` — write there"
)
})
}
/// The address a door actually binds on.
///
/// A primary binds what the config says. A twin keeps the PORT and moves to
/// [`TWIN_LOOPBACK_HOST`], so the config file stays byte-identical to the
/// primary's (`[estate-twin-drift]`) while the process holds no public
/// socket. An address with no `:port` is replaced wholesale — there is no
/// port to preserve, and leaving a bare public host would be the one door
/// that stayed public.
/// The address the gRPC CONTROL door binds on: what the config says, on a
/// twin too (owner ruling 2026-09-27 — gunnar's twin keeps its control
/// plane on the utility address). A twin's control door refuses every write
/// and admin mutation by name (`holger-twin-1`) and answers the pairing and
/// recharge calls (`TwinCsr`, `InstallTwinCertificate`, `IssueTwinCertificate`
/// for a recharge, `TwinStatus`) — the calls a front must make of a twin that
/// has restarted.
pub fn effective_control_bind(&self, configured: &str) -> String {
configured.to_string()
}
/// The address a PUBLIC (HTTP/OCI) door binds on. See [`Self::effective_control_bind`]
/// for the gRPC control door.
///
pub fn effective_bind(&self, configured: &str) -> String {
if !self.is_twin() {
return configured.to_string();
}
match configured.rsplit_once(':') {
Some((_, port)) if !port.is_empty() && port.chars().all(|c| c.is_ascii_digit()) => {
format!("{TWIN_LOOPBACK_HOST}:{port}")
}
_ => TWIN_LOOPBACK_HOST.to_string(),
}
}
/// **The direction law, as a type.** A twin yields a plan for dialling its
/// primary; a primary yields `None` and holds no address of any follower, so
/// there is nothing it could dial even if something wanted it to.
pub fn follow_plan(&self) -> Option<FollowPlan> {
match self {
NodeRole::Primary => None,
NodeRole::Twin {
primary,
poll_seconds,
} => Some(FollowPlan {
primary: primary.clone(),
poll: std::time::Duration::from_secs((*poll_seconds).max(1)),
}),
}
}
}
/// What a follower needs in order to run: who to ask, and how often.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FollowPlan {
/// The primary's address. Only a twin ever holds one.
pub primary: String,
/// The interval between cycles.
pub poll: std::time::Duration,
}
/// **The `twin ≥ primary` invariant, as a reading rather than a declaration.**
///
/// `private-holger-ops/estate.toml`'s `[estate-twin-store]` refuses a *declared*
/// twin volume smaller than the primary's; nothing checked the *made* disk,
/// because until 2026-09-20 holger had no fill reading at all. It has one now
/// ([`crate::metering::statvfs`], [`crate::capacity::measure`]), so the
/// invariant can be stated against the running boxes.
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq)]
#[serde(tag = "verdict", rename_all = "snake_case")]
pub enum StoreVerdict {
/// The primary has not been read yet — the follower has not completed a
/// cycle, or this box is a primary and the question does not apply.
/// Deliberately NOT "sufficient": an unasked question is not a pass.
Unknown,
/// The twin can hold what the primary holds today.
Sufficient {
twin_available_bytes: u64,
primary_used_bytes: u64,
},
/// The twin cannot. **A warning, never a refusal** — see the module docs.
TooSmall {
twin_available_bytes: u64,
primary_used_bytes: u64,
short_by_bytes: u64,
},
}
impl StoreVerdict {
/// The named warning line, or `None` when there is nothing to warn about.
/// The code is `holger-twin-2` and it is the ONLY place that sentence is
/// written, so the door, the log and a test all say the same thing.
pub fn warning(&self) -> Option<String> {
match self {
StoreVerdict::TooSmall {
twin_available_bytes,
primary_used_bytes,
short_by_bytes,
} => Some(holger_errcode::TWIN_STORE_TOO_SMALL.tag(format!(
"this twin has {twin_available_bytes} bytes available and its primary has \
already used {primary_used_bytes}: it is short by {short_by_bytes} bytes. \
A promotion of this twin could not hold the data. Nothing is refused — \
grow the twin's volume before it is promoted."
))),
_ => None,
}
}
/// True when the invariant is known to hold.
pub fn is_sufficient(&self) -> bool {
matches!(self, StoreVerdict::Sufficient { .. })
}
}
/// State the invariant over two live readings. `primary` is what the primary
/// reported, `twin` is this box's own measurement.
pub fn store_verdict(primary: &Capacity, twin: &Capacity) -> StoreVerdict {
if twin.available_bytes >= primary.used_by_holger_bytes {
StoreVerdict::Sufficient {
twin_available_bytes: twin.available_bytes,
primary_used_bytes: primary.used_by_holger_bytes,
}
} else {
StoreVerdict::TooSmall {
twin_available_bytes: twin.available_bytes,
primary_used_bytes: primary.used_by_holger_bytes,
short_by_bytes: primary.used_by_holger_bytes - twin.available_bytes,
}
}
}
/// **The posture this boot has**, shared with every door: the role, the latest
/// [`StoreVerdict`] the follower produced, and what the replication lane saw.
///
/// One object rather than two fields on five structs, so a new door asks one
/// question. A primary carries one too — [`NodeRole::Primary`] with an
/// [`StoreVerdict::Unknown`] verdict — because "this is a primary" is an answer
/// the twin door has to be able to give.
///
/// **The role is LIVE** (stinky 14 §2 correction 3, §P3): `BecomeTwin` records
/// the election and then [`TwinPosture::become_twin`] switches it here, so every
/// door that asks [`TwinPosture::write_refusal`] refuses by name from that
/// instant — no restart, and no window in which a just-elected twin serves a
/// write. What stays a START-time fact is the loopback bind of the public doors
/// ([`NodeRole::effective_bind`] runs in `wire_holger`): a running process does
/// not rebind a door underneath its clients, so until the next start a
/// live-elected twin's doors stay where they were bound — refusing every write
/// by name, serving reads. gunnar keeps its git listener the same way.
#[derive(Debug)]
pub struct TwinPosture {
role: RwLock<NodeRole>,
verdict: RwLock<StoreVerdict>,
lane: std::sync::Mutex<Lane>,
/// `Some(source)` while this box awaits (or is in) a recharge: it serves
/// NOTHING until the recharge is whole (`holger-twin-15`).
awaiting: RwLock<Option<String>>,
/// `Some(epoch)` once this primary was FENCED (a pulled `fence_primary`): every
/// door refuses writes from then on, until the box is re-imaged or recharged.
fenced: RwLock<Option<u64>>,
/// What stands down when this box becomes a twin live (the job pool, stinky
/// SB #34) — called by [`TwinPosture::become_twin`] on every road to it.
on_twin: OnTwin,
/// Wakes the follow loop before its next tick ([`TwinPosture::wake_follow`]).
follow_wake: tokio::sync::Notify,
}
/// The stand-down hooks [`TwinPosture::become_twin`] calls.
#[derive(Default)]
struct OnTwin(std::sync::Mutex<Vec<Box<dyn Fn() + Send + Sync>>>);
impl std::fmt::Debug for OnTwin {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "OnTwin({})", self.0.lock_or_recover().len())
}
}
/// What the replication lane saw, for `GET /-/twin`: the door's refusals (named,
/// `holger-twin-12`), the principals it admitted, and how this box follows.
#[derive(Debug, Default, Clone)]
struct Lane {
refused: u64,
last_refusal: String,
admitted: std::collections::BTreeMap<String, u64>,
follow: String,
/// Unix seconds of the last follow cycle that reached the primary and was
/// whole; 0 = never.
last_whole: i64,
/// The last cycle that reached the primary and was NOT whole: when, and what.
last_partial: i64,
partial_why: String,
/// ★ What this twin last HEARD of its primary (fence before recharge, A7 row 4,
/// Gunnar R-13): Unix ms of the last cycle that reached it, of the last that did
/// not, and what the primary listed then (its estate files). 0 = never.
reached_ms: i64,
unreached_ms: i64,
reached_holds: u64,
}
impl Default for TwinPosture {
fn default() -> Self {
Self::new(NodeRole::Primary)
}
}
impl TwinPosture {
pub fn new(role: NodeRole) -> Self {
Self {
role: RwLock::new(role),
verdict: RwLock::new(StoreVerdict::Unknown),
lane: Default::default(),
awaiting: RwLock::new(None),
fenced: RwLock::new(None),
on_twin: OnTwin::default(),
follow_wake: tokio::sync::Notify::new(),
}
}
/// **Call `f` when this box becomes a twin live** — what a primary-only plane
/// (the job pool) registers so it stands down whichever road the election
/// came by (`BecomeTwin`, the pairing loop, a re-point). `f` must not block.
pub fn on_become_twin(&self, f: Box<dyn Fn() + Send + Sync>) {
self.on_twin.0.lock_or_recover().push(f);
}
/// **Something the follow waited for is here** (its replication credential): the follow
/// loop's next cycle is due now, not at its next tick.
pub fn wake_follow(&self) {
// `notify_one` keeps a permit when no rest is waiting, so a credential that lands
// while a cycle runs still ends the NEXT rest at once.
self.follow_wake.notify_one();
}
/// **The follow loop's rest between two cycles**: `every`, or less when
/// [`TwinPosture::wake_follow`] was called.
pub async fn rest_between_follows(&self, every: std::time::Duration) {
tokio::select! {
_ = tokio::time::sleep(every) => {}
_ = self.follow_wake.notified() => {}
}
}
/// The role NOW. A poisoned lock reads the value it held.
pub fn role(&self) -> NodeRole {
self.role.read_or_recover().clone()
}
/// **Switch this running box to `role`, live.** Called by `BecomeTwin` AFTER
/// the election is recorded, so the record and the posture never disagree
/// across a restart.
pub fn become_twin(&self, role: NodeRole) {
let twin = role.is_twin();
*self.role.write_or_recover() = role;
if twin {
for f in self.on_twin.0.lock_or_recover().iter() {
f();
}
}
}
pub fn is_twin(&self) -> bool {
self.role.read_or_recover().is_twin()
}
/// The write refusal this posture imposes, or `None` on a primary that serves.
pub fn write_refusal(&self) -> Option<String> {
if let Some(epoch) = self.fenced() {
return Some(format!("this primary is FENCED (epoch {epoch}): it takes no writes; its estate is being recharged or replaced"));
}
self.role.read_or_recover().write_refusal()
}
/// **Fence this primary** at `epoch`: every door refuses writes from now on. Live, no
/// restart; it lasts for this process (a recharge or a re-image ends it).
pub fn fence(&self, epoch: u64) {
let mut f = self.fenced.write_or_recover();
*f = Some(f.map_or(epoch, |had| had.max(epoch)));
}
/// The epoch this primary was fenced at, if it was.
pub fn fenced(&self) -> Option<u64> {
*self.fenced.read_or_recover()
}
/// The replication door refused a peer (already tagged).
pub fn record_door_refusal(&self, line: String) {
let mut l = self.lane.lock_or_recover();
l.refused += 1;
l.last_refusal = line;
}
/// The replication door admitted a peer whose certificate names `cn`.
pub fn record_door_admitted(&self, cn: &str) {
*self
.lane
.lock_or_recover()
.admitted
.entry(cn.to_string())
.or_default() += 1;
}
/// How this box's follow last dialled (`mtls <door>`, `plain <endpoint>`).
pub fn record_follow(&self, how: String) {
self.lane.lock_or_recover().follow = how;
}
/// A follow cycle reached the primary and was whole.
pub fn record_follow_whole(&self, now_unix: i64) {
self.lane.lock_or_recover().last_whole = now_unix;
}
/// A follow cycle reached the primary and left `why` behind. A cycle that
/// could not REACH the primary is not this: a dead primary is not a loss.
pub fn record_follow_partial(&self, now_unix: i64, why: String) {
let mut l = self.lane.lock_or_recover();
l.last_partial = now_unix;
l.partial_why = why;
}
/// A follow cycle REACHED the primary at `now_ms` (whole or not), which listed
/// `holds` estate files: the primary answers.
pub fn record_reached(&self, now_ms: i64, holds: u64) {
let mut l = self.lane.lock_or_recover();
l.reached_ms = now_ms;
l.reached_holds = holds;
}
/// A follow cycle could NOT reach the primary at `now_ms`.
pub fn record_unreached(&self, now_ms: i64) {
self.lane.lock_or_recover().unreached_ms = now_ms;
}
/// **What this twin heard of its primary, as of `now_ms`** — the shared fence
/// gate's `nordisk_twin::fence::Hearing`, measured on THIS box's clock.
pub fn hearing(&self, now_ms: i64) -> Hearing {
let l = self.lane.lock_or_recover();
let role = self.role();
let state = match (l.reached_ms, l.unreached_ms) {
(0, 0) => HEARD_IDLE,
(r, u) if u > r => HEARD_UNREACHABLE,
_ => "following",
};
Hearing {
twin: role.is_twin(),
primary: role.primary().unwrap_or_default().to_string(),
last_answer_unix_ms: l.reached_ms,
answer_ago_ms: if l.reached_ms > 0 {
u64::try_from(now_ms.saturating_sub(l.reached_ms)).unwrap_or(0)
} else {
0
},
primary_head: l.reached_holds,
state: state.to_string(),
}
}
/// **Can this twin give everything back?** `(complete, why, last_whole)`:
/// complete when its last cycle that reached the primary was whole.
pub fn completeness(&self) -> (bool, String, i64) {
let l = self.lane.lock_or_recover();
if l.last_whole == 0 && l.last_partial == 0 {
return (
false,
"this twin has never completed a follow cycle".into(),
0,
);
}
if l.last_partial >= l.last_whole {
return (
false,
format!(
"its last follow cycle that reached the primary was not whole: {}",
l.partial_why
),
l.last_whole,
);
}
(true, String::new(), l.last_whole)
}
/// **The replication door's lane, for the admin view**: how many peers it
/// refused, the last refusal (already tagged), and every principal it
/// admitted with its count.
pub fn door_lane(&self) -> (u64, String, std::collections::BTreeMap<String, u64>) {
let l = self.lane.lock_or_recover();
(l.refused, l.last_refusal.clone(), l.admitted.clone())
}
/// How this box last followed.
pub fn follow(&self) -> String {
self.lane.lock_or_recover().follow.clone()
}
/// **This box awaits (or is in) a recharge from `source`** — from this call
/// on it serves nothing but health, `/-/twin` and the pairing/recharge doors.
pub fn await_recharge(&self, source: &str) {
*self.awaiting.write_or_recover() = Some(source.to_string());
}
/// The recharge is whole: serve.
pub fn recharged(&self) {
*self.awaiting.write_or_recover() = None;
}
/// The source a pending recharge waits for.
pub fn awaiting_from(&self) -> Option<String> {
self.awaiting.read_or_recover().clone()
}
/// **The refusal of ANY product request on a box awaiting its recharge**
/// (`holger-twin-15`), or `None` when it serves.
pub fn serve_refusal(&self) -> Option<String> {
self.awaiting_from().map(|from| {
holger_errcode::TWIN_AWAITING_RECHARGE.tag(format!(
"this box is held for its recharge from its twin (`{from}`) and serves nothing until the recharge is whole — \
half a box served as whole is the worse fault. The recharge (the ONE box's pulled \
recharge_from_twin order, or `holger-server recharge`) releases it"
))
})
}
/// **THE `serving` SEAM** — false while a recharge is pending. What holger's
/// grow machine folds into `GrowStatus.serving` (monetize's
/// `restore_decision` reads a box that cannot serve as a restore target).
pub fn serving(&self) -> bool {
self.awaiting.read_or_recover().is_none()
}
/// The latest verdict. A poisoned lock reads as [`StoreVerdict::Unknown`] —
/// a health field must not panic the door it is reported on.
pub fn verdict(&self) -> StoreVerdict {
self.verdict.read_or_recover().clone()
}
/// Record a fresh verdict and emit its warning, if it has one.
pub fn record_verdict(&self, verdict: StoreVerdict) {
if let Some(warning) = verdict.warning() {
log::warn!("{warning}");
}
{
let mut slot = self.verdict.write_or_recover();
*slot = verdict;
}
}
/// **The twin door's body** — what `GET /-/twin` answers. JSON, read-only,
/// no secret in it: the role, the primary it follows, the invariant's
/// current verdict with its named warning when there is one, and the
/// replication lane (refusals by name, admitted principals, the follow).
pub fn door_json(&self) -> String {
let verdict = self.verdict();
let warning = verdict.warning().unwrap_or_default();
let role = self.role();
let lane = self.lane.lock_or_recover().clone();
serde_json::json!({
"role": if role.is_twin() { "twin" } else { "primary" },
"primary": role.primary().unwrap_or(""),
"accepts_writes": !role.is_twin() && self.serving(),
"serving": self.serving(),
"awaiting_recharge_from": self.awaiting_from().unwrap_or_default(),
"store": verdict,
"warning": warning,
"replication": {
"refused": lane.refused,
"last_refusal": lane.last_refusal,
"admitted": lane.admitted,
"follow": lane.follow,
},
})
.to_string()
}
}
/// **What a follower needs FROM its primary.** The seam the wire eventually
/// plugs into; see the module docs for what exists today and what is owed.
///
/// Read-only by construction: there is no method here that writes to the
/// primary, because a follower that could write to its primary is not a
/// follower.
pub trait PrimaryView: Send + Sync {
/// The primary's address, for messages and logs.
fn endpoint(&self) -> &str;
/// The primary's own capacity reading — the left-hand side of
/// [`store_verdict`].
fn capacity(&self) -> Result<Capacity, String>;
/// The primary's data root, whose side stores a twin must hold (W10).
fn data_root(&self) -> Result<PathBuf, String>;
}
/// A primary this process can READ ON DISK — the laptop-testable form, and the
/// form a restore takes when the primary's volume is attached to the twin.
///
/// This is not a claim that holger replicates over a network. It does not
/// (stinky 02 S2-1); `holger-remote` (plan W11) is the wire implementation of
/// this same trait and is owed.
pub struct LocalPrimary {
endpoint: String,
data_root: PathBuf,
capacity: Capacity,
}
impl LocalPrimary {
pub fn new(
endpoint: impl Into<String>,
data_root: impl Into<PathBuf>,
capacity: Capacity,
) -> Self {
Self {
endpoint: endpoint.into(),
data_root: data_root.into(),
capacity,
}
}
}
impl PrimaryView for LocalPrimary {
fn endpoint(&self) -> &str {
&self.endpoint
}
fn capacity(&self) -> Result<Capacity, String> {
Ok(self.capacity.clone())
}
fn data_root(&self) -> Result<PathBuf, String> {
Ok(self.data_root.clone())
}
}
/// What one follow cycle did.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct FollowReport {
/// The primary that was asked.
pub primary: String,
/// Everything the estate replication moved or found already present.
pub estate: crate::replication::EstateReport,
/// The `twin ≥ primary` reading taken during the cycle.
pub verdict: StoreVerdict,
}
/// **One follow cycle: the twin asks, copies and measures.**
///
/// The order matters. The estate is pulled FIRST and the verdict taken after,
/// so the reading describes the store as it now is rather than as it was before
/// the copy. Nothing here writes to the primary, and the primary is never told
/// this happened — it is a read of the primary and a write of the twin, which
/// is the whole of gunnar's F13.
pub fn follow_once(
view: &dyn PrimaryView,
posture: &TwinPosture,
twin_root: &Path,
opts: &crate::replication::ReplicationOptions,
) -> Result<FollowReport, String> {
let primary_root = view.data_root()?;
let estate = crate::replication::replicate_estate(&primary_root, twin_root, opts)?;
let primary_capacity = view.capacity()?;
let twin_capacity = crate::capacity::measure(twin_root, &[])?;
let verdict = store_verdict(&primary_capacity, &twin_capacity);
posture.record_verdict(verdict.clone());
Ok(FollowReport {
primary: view.endpoint().to_string(),
estate,
verdict,
})
}
/// A shared posture, for the doors.
pub type SharedPosture = Arc<TwinPosture>;
// ─────────────────────────────────────────────────────────────────────────────
// THE ROLE COMES ONLY FROM THE PAIRING — NEVER FROM AN ENVIRONMENT OR A CONFIG
// ─────────────────────────────────────────────────────────────────────────────
/// The environment variable that once carried a role into the server — now
/// REFUSED by name (stinky SB #36): the role is the installer's election only.
pub const ROLE_ENV: &str = "HOLGER_ROLE";
/// The once-paired primary address of an environment twin — refused too.
pub const PRIMARY_ENV: &str = "HOLGER_PRIMARY";
/// The once-paired poll interval of an environment twin — refused too.
pub const POLL_ENV: &str = "HOLGER_TWIN_POLL_SECONDS";
/// **A role named anywhere but the pairing's election is refused by name**
/// (stinky SB #36, `holger-twin-21`).
///
/// The role is a BOOTSTRAP fact the installer's door elects (the weakest box is
/// the twin, nordisk-pairing `elect::weakest`) and the box records once
/// ([`elect`], `holger-role.ron`). An environment variable or a config key that
/// names a role is a second declaration of what the box IS; reading it is how a
/// box comes up in the role nobody elected, and IGNORING it leaves an operator
/// believing a setting holds. So both are refused, naming what to remove:
///
/// * `get` looks up the environment ([`ROLE_ENV`], [`PRIMARY_ENV`],
/// [`POLL_ENV`]); an empty or blank value names nothing;
/// * `config_role` is the config's `role:` key, `None` when the key is absent.
///
/// Taken as a lookup so it is testable without mutating the process environment.
pub fn refuse_declared_role(
get: impl Fn(&str) -> Option<String>,
config_role: Option<&NodeRole>,
) -> Result<(), String> {
for key in [ROLE_ENV, PRIMARY_ENV, POLL_ENV] {
if get(key).is_some_and(|v| !v.trim().is_empty()) {
return Err(holger_errcode::TWIN_ROLE_NOT_FROM_PAIRING.tag(format!(
"the environment sets {key}, and a role comes only from the pairing's election \
(the installer's verdict, recorded in {ROLE_RECORD}); remove {key} from the \
installation's environment"
)));
}
}
if let Some(role) = config_role {
return Err(holger_errcode::TWIN_ROLE_NOT_FROM_PAIRING.tag(format!(
"the config names `role: {}`, and a role comes only from the pairing's election \
(the installer's verdict, recorded in {ROLE_RECORD}); remove the `role:` key — the \
twin's config is the primary's byte for byte",
if role.is_twin() {
"Twin(..)"
} else {
"Primary"
}
)));
}
Ok(())
}
/// [`refuse_declared_role`] against the real process environment.
pub fn refuse_declared_role_in_env(config_role: Option<&NodeRole>) -> Result<(), String> {
refuse_declared_role(|k| std::env::var(k).ok(), config_role)
}
// ─────────────────────────────────────────────────────────────────────────────
// THE ROLE IS A BOOTSTRAP FACT — ELECTED ONCE, RECORDED, NEVER RE-EVALUATED
// ─────────────────────────────────────────────────────────────────────────────
/// The file the election is recorded in, under the data root.
///
/// gunnar's `twin.json` in holger's vocabulary. RON rather than JSON because
/// every other declaration holger reads is RON, and the role is read on the same
/// boot path as the config.
pub const ROLE_RECORD: &str = "holger-role.ron";
/// **What this box was ELECTED to be, and when.**
///
/// Owner rulings 2026-09-21, verbatim: *"the weakest machine must become the
/// twin. PERIOD"* and *"IT must not CHANGE role later ! so its a bootstrap
/// thing from xtask"*.
///
/// Two consequences, and they are the whole design:
///
/// * The role is **not on the medium**. Both boxes install from ONE identical
/// ISO with no role parameter at all, which is what *"it MUST use the SAME
/// iso"* means literally. The role is conferred afterwards, over the control
/// plane, onto the box a measurement says is weakest.
/// * The role is **not in `holger.ron`**. `private-holger-ops`'
/// `[estate-twin-drift]` requires the twin's config to be byte-identical to
/// the primary's, and writing the role into the file would mean writing an
/// exception into the one rule that guards gunnar's un-metered-twin scar.
///
/// So it is STATE THE BOX ACQUIRES, beside the config rather than in it, and
/// written `create_new` so a second election is a refusal rather than an
/// overwrite — the same mechanism the promotion engine's seal uses.
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq)]
pub struct RoleRecord {
/// What this box was elected to be.
pub role: NodeRole,
/// Unix seconds. An election is a moment, and a record with no moment
/// cannot be told apart from a file somebody copied.
pub elected_at: u64,
/// WHO elected it — the authenticated identity the control plane presented.
pub elected_by: String,
/// The MEASUREMENT that decided it, in the electing side's own words
/// ("2 cores / 1 GB RAM / 20 GB hdd vs 2 / 4 / 20 maxiops"). Free text on
/// purpose: this record is read by an operator on the worst day, and a
/// number with no units is not a reason.
pub reason: String,
}
/// Where the record lives for a given data root.
pub fn role_record_path(data_root: &Path) -> PathBuf {
data_root.join(ROLE_RECORD)
}
/// **Read the election, if this box has had one.** `Ok(None)` means no box-level
/// role has ever been conferred, which is the state every box installs in.
///
/// A record that cannot be PARSED is an `Err` and never a `None`: a corrupt role
/// file read as "no role" is a twin coming up as a primary, which is the exact
/// failure this whole module exists to prevent.
pub fn recorded_role(data_root: &Path) -> Result<Option<RoleRecord>, String> {
let path = role_record_path(data_root);
match std::fs::read_to_string(&path) {
Ok(text) => ron::from_str::<RoleRecord>(&text)
.map(Some)
.map_err(|e| format!("the role record {} is not readable: {e}", path.display())),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None),
Err(e) => Err(format!("reading the role record {}: {e}", path.display())),
}
}
/// **A box is FRESH when it holds nothing of its own.**
///
/// Only a fresh box may be elected a twin: electing one that already serves
/// would put its history behind a follower's refusals and start refusing its
/// clients' writes, by name, with no warning to whoever was writing.
///
/// "Nothing of its own" is read against an ALLOW-LIST of what counts as history
/// (lane HOL-312-HARDEN): content in one of the [`HISTORY_STORES`], in a store or
/// the audit dir the box's config names under its data root, an audit chain, or a
/// znippy archive. A file no history kind names is never history, so no file a
/// feed or a start writes before the election can refuse it (`holger-twin-5`).
/// Deliberately generous about the CONFIG — every installed box has one, and a
/// box is not un-fresh for having been configured.
pub fn is_fresh(data_root: &Path) -> bool {
held_state(data_root).is_none()
}
/// **The first thing that makes a box un-fresh**, by path — what `holger-twin-5`
/// names, so an operator is told WHICH history would disappear, not only that
/// some would. `None` on a fresh box.
pub fn held_state(data_root: &Path) -> Option<PathBuf> {
held_state_except(data_root, &[])
}
/// [`held_state`], not counting the top-level names in `except` — what a
/// RECHARGE asks: its own audit of the requests it refused while it waited is
/// not history a recharge could overwrite (segments are named per process and
/// the estate pull never deletes), and the recharge's own markers are its own.
/// Whether the first-login record at `p` reads and names no buyer who registered under it.
fn unused_first_login(p: &Path) -> bool {
std::fs::read(p)
.ok()
.and_then(|b| serde_json::from_slice::<serde_json::Value>(&b).ok())
.is_some_and(|v| v.is_object() && v.get("used_by").is_none_or(serde_json::Value::is_null))
}
pub fn held_state_except(data_root: &Path, except: &[&str]) -> Option<PathBuf> {
held_state_with(data_root, except, &[])
}
/// [`held_state_except`], with the stores the RUNNING config names (`stores`: every repository's
/// storage place and storage endpoint, [`crate::Holger::history_stores`]) counted beside the ones
/// the config on disk names — a box whose config is not at `<data_root>/holger.ron` still has
/// its repositories' bytes counted.
pub fn held_state_with(data_root: &Path, except: &[&str], stores: &[PathBuf]) -> Option<PathBuf> {
// CONTENT is a byte somebody put there: a non-empty file anywhere below.
// An empty directory, or the zero-byte audit segment a start opens, is the
// shape of a box that has STARTED, not of one that has served (stinky 14
// §2 correction 3, MEASURED 2026-09-27: one start of an empty box left
// `audit/audit-….arrows` at 0 bytes, `jobs/jobs.redb` and
// `holger-jobs/jobs.redb` — and `BecomeTwin` refused it `holger-twin-5`,
// so the one-medium box could never be elected after its first boot).
fn content(p: &Path) -> Option<PathBuf> {
match std::fs::metadata(p) {
Ok(m) if m.is_dir() => std::fs::read_dir(p)
.ok()?
.flatten()
.find_map(|e| content(&e.path())),
// The passkey store's session MAC key is minted by every START of
// a box with the passkey plane on (the people are holger's since
// holger-ui's thin-layer move) — scaffolding, like `jobs`, not
// history: a store that has enrolled nobody holds nothing but it.
// Its credentials, invites and revocations still count.
Ok(_)
if p.file_name().is_some_and(|n| n == "session.key")
&& p.parent()
.and_then(Path::file_name)
.is_some_and(|d| d == "holger-passkeys") =>
{
None
}
// The shop's first-login record (`holger-passkeys/first-admin.json`, lane P1): every box
// of a sold estate is fed the invite at boot, BEFORE the installer elects the twin, so a
// record nobody registered under is the feed's, not history (fast holger 2026-10-28:
// the twin's election was refused `holger-twin-5`). One a buyer used still counts.
Ok(_) if p.ends_with(crate::auth::FIRST_ADMIN_MARKER_REL) && unused_first_login(p) => {
None
}
// The trust roots every start writes from the config (`plugins/trust/<stem>.der`,
// `Holger::materialize_pem_trust_roots`, lane W2): the shipped declaration's korp
// root, config and not history. Without this every estate box refused its twin
// election after its first start (`holger-twin-5`, lane D3). A plugin module
// beside it still counts.
Ok(_)
if p.parent()
.and_then(Path::file_name)
.is_some_and(|d| d == "trust")
&& p.parent()
.and_then(Path::parent)
.and_then(Path::file_name)
.is_some_and(|d| d == "plugins") =>
{
None
}
// The sync key every estate box mints at its start (lane HOL-WGDS): a key minted
// at the start is no history (no downstream, no ledger, no invite); anything
// beside it in the sync state is. The twin's follow and a recharge bring the
// main's own key over it (lane OX-WGDS2).
Ok(_)
if p.file_name().is_some_and(|n| n == SYNC_KEY)
&& p.parent()
.and_then(Path::file_name)
.is_some_and(|d| d == SYNC_DIR) =>
{
None
}
// What the installer feeds every box BEFORE it elects the twin (lane OX-WGDS3, the KVM
// gate 2026-10-28: the twin was refused `holger-twin-5` over both, and the site never
// paired): the front's offered downstream interface, kept for the invites
// (`sync::downstreams::keep_front_hub`), and monetize's verifying key in the meter
// plugin's slot (`fleet_box::METER_KEY_SLOT`, the seed's or the late feed's
// `monetize_pubkey`). Feed, not history; an enrolment or a reading beside them counts.
Ok(_)
if p.file_name().is_some_and(|n| n == FRONT_HUB)
&& p.parent()
.and_then(Path::file_name)
.is_some_and(|d| d == SYNC_DIR) =>
{
None
}
Ok(_)
if p.ends_with(METER_KEY)
&& p.parent()
.and_then(Path::parent)
.and_then(Path::file_name)
.is_some_and(|d| d == "meters") =>
{
None
}
// The Sigstore trust root every start seeds from the medium (lane HOL-OCI-EVIDENCE,
// `sigstore_trust::seed`: `holger-trust/` and the trust product's floor marker under
// `holger-update-state/`) is the medium's, not history (lane HOL-REBUILT-SYNC-KEY, the
// KVM gate 2026-10-31: the main laid again after its death was not held for its
// recharge, served under a fresh sync key and was never filled). A root an import
// installed, and every other floor, still counts.
Ok(_) if seeded_from_the_medium(p) => None,
// The audit INDEX (`audit/index/`, HOL-AUDIT c3dbbef5) is built from the chain at every
// start and deleting it is always safe: derived, not history (the chain, `audit.jsonl`,
// still counts). Without this a started box was refused its twin election `holger-twin-5`.
// Under ANY audit dir (the box's is `holger-audit`): the index dir sits beside the chain.
Ok(_)
if p.parent()
.and_then(Path::file_name)
.is_some_and(|d| d == crate::audit::arrow_ipc::INDEX_DIR)
&& p.parent().and_then(Path::parent).is_some_and(|a| {
a.join(crate::audit::AUDIT_FILE).is_file()
|| a.file_name()
.is_some_and(|n| n == "audit" || n == "holger-audit")
}) =>
{
None
}
Ok(m) if m.len() > 0 => Some(p.to_path_buf()),
_ => None,
}
}
// Only what the ALLOW-LIST names is history (lane HOL-312-HARDEN): a file no history kind
// names is the feed's or a start's, whatever a future feed writes.
let counted = |rel: &Path| {
rel.components().next().is_some_and(|c| {
let top = c.as_os_str().to_string_lossy();
!START_SCAFFOLDING.contains(&top.as_ref()) && !except.contains(&top.as_ref())
})
};
// 1. The stores of history, each by its own name.
for rel in HISTORY_STORES {
if counted(Path::new(rel)) {
if let Some(p) = content(&data_root.join(rel)) {
return Some(p);
}
}
}
// 2. The artifact stores and the audit dir the box's config names under its data root.
for dir in configured_history(data_root).iter().chain(stores) {
// A store that IS the data root (or holds it) is judged by the kinds below: counting it
// whole would make every fed file history again.
if data_root.starts_with(dir) {
continue;
}
if dir.strip_prefix(data_root).is_ok_and(&counted) {
if let Some(p) = content(dir) {
return Some(p);
}
}
}
// 3. By kind, anywhere else on the box: an audit chain (and the segments beside it), and a
// znippy archive (the artifact store a config names under a path of its own).
let mut tops: Vec<PathBuf> = std::fs::read_dir(data_root)
.into_iter()
.flatten()
.flatten()
.filter(|e| {
let n = e.file_name().to_string_lossy().to_string();
// A store of history is judged by its own rule (the medium's trust root is a znippy
// archive and no history).
!n.starts_with('.') && counted(Path::new(&n)) && !HISTORY_STORES.contains(&n.as_str())
})
.map(|e| e.path())
.collect();
tops.sort();
tops.iter().find_map(|top| {
let is_audit_dir = top.join(crate::audit::AUDIT_FILE).is_file()
|| top
.file_name()
.is_some_and(|n| n == "audit" || n == "holger-audit");
if top.is_dir() && is_audit_dir {
if let Some(p) = content(top) {
return Some(p);
}
}
archive(top)
})
}
/// **The stores of history**, by their path under the data root: every
/// [`crate::replication::ESTATE`] item but the config and the [`START_SCAFFOLDING`]
/// (`every_estate_item_is_a_history_store_or_named_not_history` holds it), the Sigstore trust
/// root an import installed (the medium's seed is not history), and the front report main
/// heard (`front_report::REPORT_FILE`; the proof key the feed keeps beside it is not).
/// A NEW store of history is added HERE; anything this list and [`held_state_except`]'s
/// kinds do not name is never history (lane HOL-312-HARDEN, HOL-TWIN-HELD-ALLOWLIST).
pub const HISTORY_STORES: &[&str] = &[
"holger-attestations",
"holger-audit",
"holger-props",
"holger-sboms",
"holger-prod-store",
"holger-update-state",
"holger-import-receipts",
"plugins",
"holger-passkeys",
"holger-entitlements",
"meters",
"holger-purges",
"fleet/signups.json",
"fleet/sold-repositories.json",
"holger-sync",
TRUST_DIR,
"holger-front/report.ron",
];
/// The ESTATE items that are NOT history: the served config (every installed box has one).
#[cfg(test)]
const NOT_HISTORY_ESTATE: &[&str] = &["holger.ron"];
/// The first non-empty znippy archive at or below `p` (an artifact store, whatever the config
/// names its directory).
fn archive(p: &Path) -> Option<PathBuf> {
match std::fs::metadata(p) {
Ok(m) if m.is_dir() => {
let mut kids: Vec<PathBuf> = std::fs::read_dir(p)
.ok()?
.flatten()
.map(|e| e.path())
.collect();
kids.sort();
kids.iter().find_map(|k| archive(k))
}
Ok(m) if m.len() > 0 && p.extension().is_some_and(|x| x == "znippy") => {
Some(p.to_path_buf())
}
_ => None,
}
}
/// The config keys that name where a repository's bytes live: a storage endpoint's `ron_path`, a
/// repository's `ron_data_dir`, `ron_archive_path` and `ron_writable_archive`.
const STORE_KEYS: &[&str] = &[
"ron_path",
"ron_data_dir",
"ron_archive_path",
"ron_writable_archive",
];
/// The storage paths (`ron_path`, a repository's places) and the audit dir (`audit.dir`) the config at
/// `<data_root>/holger.ron` names, anchored under the data root when relative. Read leniently:
/// a config this cannot read names nothing here, and the stores of history and the kinds still
/// count.
fn configured_history(data_root: &Path) -> Vec<PathBuf> {
fn walk(v: &ron::Value, out: &mut Vec<String>) {
match v {
ron::Value::Map(m) => {
// A repository that names no place of its own keeps its files in the default
// (`repository::Repository::storage_places`: `/var/lib/holger/<name>`).
let key = |k: &str| {
m.iter()
.find(|(mk, _)| matches!(mk, ron::Value::String(s) if s == k))
.map(|(_, v)| v)
};
if key("ron_repo_type").is_some()
&& !STORE_KEYS.iter().any(|k| {
key(k).is_some_and(|v| {
!matches!(v, ron::Value::Option(None) | ron::Value::Unit)
})
})
{
if let Some(ron::Value::String(n)) = key("ron_name") {
out.push(format!("/var/lib/holger/{n}"));
}
}
for (k, v) in m.iter() {
match (k, v) {
(ron::Value::String(k), ron::Value::String(s))
if STORE_KEYS.contains(&k.as_str()) =>
{
out.push(s.clone())
}
(ron::Value::String(k), ron::Value::Option(Some(s)))
if STORE_KEYS.contains(&k.as_str()) =>
{
if let ron::Value::String(s) = s.as_ref() {
out.push(s.clone())
}
}
(ron::Value::String(k), ron::Value::Map(a)) if k == "audit" => {
for (k2, v2) in a.iter() {
if let (ron::Value::String(k2), ron::Value::String(d)) = (k2, v2) {
if k2 == "dir" {
out.push(d.clone());
}
}
}
}
_ => walk(v, out),
}
}
}
ron::Value::Seq(s) => s.iter().for_each(|v| walk(v, out)),
ron::Value::Option(Some(v)) => walk(v, out),
_ => {}
}
}
let Ok(text) = std::fs::read_to_string(data_root.join("holger.ron")) else {
return Vec::new();
};
let mut named = Vec::new();
match ron::from_str::<ron::Value>(&text) {
Ok(v) => walk(&v, &mut named),
// A config the generic reader refuses (an extension it does not know): its storage
// paths by their key, so a store is still counted.
Err(_) => {
for key in STORE_KEYS {
let mut rest = text.as_str();
while let Some(i) = rest.find(key) {
rest = &rest[i + key.len()..];
let r = rest.trim_start();
let Some(r) = r.strip_prefix(':') else {
continue;
};
let r = r.trim_start();
let r = r.strip_prefix("Some(").map(str::trim_start).unwrap_or(r);
let Some(r) = r.strip_prefix('"') else {
continue;
};
if let Some(end) = r.find('"') {
named.push(r[..end].to_string());
}
}
}
}
}
named
.into_iter()
.filter(|s| !s.trim().is_empty())
.map(|s| {
let p = Path::new(s.trim());
if p.is_absolute() {
p.to_path_buf()
} else {
data_root.join(p)
}
})
.collect()
}
/// The Sigstore trust root's directory (`sigstore_trust::DIR`), the record of what is installed
/// there (`sigstore_trust::INSTALLED_FILE`), the import floors' directory
/// (`import_door::UPDATE_STATE_DIR`) and the trust product's floor under it (`RollbackStore`:
/// `<supplier>/<product>`, each segment sanitized: `holger.sigstore-trust/v1` →
/// `holger.sigstore-trust_v1`).
const TRUST_DIR: &str = "holger-trust";
const TRUST_INSTALLED: &str = "installed.json";
const UPDATE_STATE: &str = "holger-update-state";
const TRUST_FLOOR: [&str; 2] = ["holger", "holger.sigstore-trust_v1"];
/// The sequence of the trust root the MEDIUM seeded under `trust_dir` (its `installed.json`
/// says `from: "medium"`), `None` when nothing is installed there or an import installed it.
fn medium_trust_sequence(trust_dir: &Path) -> Option<u64> {
let v: serde_json::Value =
serde_json::from_slice(&std::fs::read(trust_dir.join(TRUST_INSTALLED)).ok()?).ok()?;
if v.get("from")?.as_str()? != "medium" {
return None;
}
v.get("sequence")?.as_u64()
}
/// Is `p` what a start's seed of the medium's trust root wrote: a file in `holger-trust/` while
/// the medium's root is installed, or the trust product's floor marker of exactly that sequence?
fn seeded_from_the_medium(p: &Path) -> bool {
fn name(q: Option<&Path>) -> Option<&str> {
q.and_then(Path::file_name).and_then(|n| n.to_str())
}
let parent = p.parent();
if name(parent) == Some(TRUST_DIR) {
return parent.and_then(medium_trust_sequence).is_some();
}
let product = parent;
let supplier = product.and_then(Path::parent);
let state = supplier.and_then(Path::parent);
if name(product) != Some(TRUST_FLOOR[1])
|| name(supplier) != Some(TRUST_FLOOR[0])
|| name(state) != Some(UPDATE_STATE)
{
return false;
}
let Some(seq) = p
.file_name()
.and_then(|n| n.to_str())
.and_then(|n| n.strip_suffix(".accepted"))
.and_then(|n| n.parse::<u64>().ok())
else {
return false;
};
state
.and_then(Path::parent)
.and_then(|root| medium_trust_sequence(&root.join(TRUST_DIR)))
== Some(seq)
}
/// The sync state's directory (server-lib `sync::STATE_DIR`) and the key every estate box
/// mints at its start (`sync::KEY_FILE`, lane HOL-WGDS).
const SYNC_DIR: &str = "holger-sync";
const SYNC_KEY: &str = "sync-key.pk8";
/// The front's offered downstream interface as every estate box keeps it (`sync::downstreams::FRONT_HUB_FILE`).
const FRONT_HUB: &str = "front-hub.ron";
/// monetize's verifying key in the meter plugin's slot, under `meters/` (`fleet_box::METER_KEY_SLOT`).
const METER_KEY: &str = "monetize/key.hex";
/// **What every start lays down before anything is served**: the durable job
/// queue (`<data_root>/jobs`) and the promotion job ledger (`holger-jobs`) are
/// opened — as non-empty redb files — on the first boot of an empty box. Their
/// presence is not history; the history they could record (a promotion, a
/// queued job) always leaves bytes in a store [`is_fresh`] does count. The read gate's
/// posture record (`posture/read-gate`, HCON-5) is written by every start too: what it says is
/// this boot's posture, not history. The scan sandbox (`sandbox`, `holger_sandbox::media::Layout`)
/// is laid from the running slot's medium by every start before it serves: the medium's, never
/// replicated, and a re-laid main that counted it was never held for its twin (lane OX-WGDS4).
pub const START_SCAFFOLDING: &[&str] = &["jobs", "holger-jobs", "posture", "sandbox"];
/// **Elect this box, once.**
///
/// Refuses, by name, when:
/// * a role has already been recorded (`holger-twin-4`) — *"IT must not CHANGE
/// role later"*. Re-electing the SAME role is refused too: a second election
/// is a second election, and a caller that thinks it needs one has lost track
/// of which box it is talking to.
/// * the box is not fresh and the role being conferred is `Twin`
/// (`holger-twin-5`).
///
/// The write is `create_new`, so two callers racing cannot both win.
pub fn elect(
data_root: &Path,
role: NodeRole,
elected_by: &str,
reason: &str,
now_unix: u64,
) -> Result<RoleRecord, String> {
if let Some(existing) = recorded_role(data_root)? {
return Err(holger_errcode::TWIN_ROLE_ALREADY_ELECTED.tag(format!(
"this box was elected `{}` at {} by `{}` ({}). A role is a BOOTSTRAP fact and is \
never re-evaluated; to change it, rebuild the box.",
if existing.role.is_twin() {
"twin"
} else {
"primary"
},
existing.elected_at,
existing.elected_by,
existing.reason
)));
}
if let (true, Some(held)) = (role.is_twin(), held_state(data_root)) {
return Err(holger_errcode::TWIN_BOX_NOT_FRESH.tag(format!(
"{} already holds state of its own ({}). Electing it a twin would put that history \
behind a follower's refusals and start refusing its clients' writes by name. \
Elect a freshly installed box.",
data_root.display(),
held.display()
)));
}
let record = RoleRecord {
role,
elected_at: now_unix,
elected_by: elected_by.to_string(),
reason: reason.to_string(),
};
let text = ron::ser::to_string_pretty(&record, ron::ser::PrettyConfig::default())
.map_err(|e| format!("serialising the role record: {e}"))?;
std::fs::create_dir_all(data_root)
.map_err(|e| format!("creating {}: {e}", data_root.display()))?;
let path = role_record_path(data_root);
// Created only if absent — two callers racing must not both believe they won — and
// WHOLE (R10, the way Gunnar keeps its state): staged and synced, then hard-linked
// into place (`link(2)` refuses an existing name), the directory synced. A power cut
// can no longer leave an EMPTY role record that every later boot refuses.
durable_new(&path, text.as_bytes()).map_err(|e| match e.kind() {
std::io::ErrorKind::AlreadyExists => {
holger_errcode::TWIN_ROLE_ALREADY_ELECTED.tag(format!(
"{} already exists — this box has had its election",
path.display()
))
}
_ => format!("writing {}: {e}", path.display()),
})?;
Ok(record)
}
/// **Re-point a twin at the main recharged from it** (stinky 14 §P3). NOT a new
/// election and not a role change — the box stays a twin, as the owner ruled it
/// must; only WHICH primary it follows moves, because the one it was elected
/// beside is dead and a fresh main now holds everything it held. Refused unless
/// this box's recorded role is `Twin`. The record keeps its election (moment,
/// elector) and says the re-point in its reason, so the worst-day reader sees
/// both. Written whole, then renamed.
pub fn repoint(data_root: &Path, new_primary: &str, now_unix: u64) -> Result<RoleRecord, String> {
let Some(mut record) = recorded_role(data_root)? else {
return Err(holger_errcode::TWIN_PAIRING_REFUSED
.tag("this box has no election on record, so it is no twin to re-point"));
};
let NodeRole::Twin {
primary,
poll_seconds,
} = &record.role
else {
return Err(holger_errcode::TWIN_PAIRING_REFUSED
.tag("this box was elected a PRIMARY; only a twin is re-pointed"));
};
let new_primary = new_primary.trim();
if new_primary.is_empty() {
return Err(holger_errcode::TWIN_PAIRING_REFUSED
.tag("a re-point that names no primary is not a re-point"));
}
record.reason = format!("{} · re-pointed from `{primary}` to `{new_primary}` at {now_unix} (its primary died; the main recharged from this twin)", record.reason);
record.role = NodeRole::Twin {
primary: new_primary.to_string(),
poll_seconds: *poll_seconds,
};
let text = ron::ser::to_string_pretty(&record, ron::ser::PrettyConfig::default())
.map_err(|e| format!("serialising the role record: {e}"))?;
let path = role_record_path(data_root);
let tmp = path.with_extension("ron.part");
durable_stage(&tmp, text.as_bytes()).map_err(|e| format!("writing {}: {e}", tmp.display()))?;
std::fs::rename(&tmp, &path)
.map_err(|e| format!("{} → {}: {e}", tmp.display(), path.display()))?;
sync_parent(&path).map_err(|e| format!("syncing the directory of {}: {e}", path.display()))?;
Ok(record)
}
/// Write `bytes` to `tmp` and fsync it (std only: server-lib links no libc crate).
fn durable_stage(tmp: &Path, bytes: &[u8]) -> std::io::Result<()> {
use std::io::Write as _;
let mut f = std::fs::File::create(tmp)?;
f.write_all(bytes)?;
f.sync_all()
}
/// fsync the directory `path` lives in, so its rename or link survives a power cut.
fn sync_parent(path: &Path) -> std::io::Result<()> {
std::fs::File::open(path.parent().unwrap_or(Path::new(".")))?.sync_all()
}
/// edda `appliance_pid1::durable::write_new`, std only: staged and synced, hard-linked
/// into place (refused when it exists), the staging name removed, the directory synced.
fn durable_new(path: &Path, bytes: &[u8]) -> std::io::Result<()> {
let tmp = path.with_extension(format!("ron.{}.new", std::process::id()));
durable_stage(&tmp, bytes)?;
let linked = std::fs::hard_link(&tmp, path);
let _ = std::fs::remove_file(&tmp);
linked?;
sync_parent(path)
}
/// **The role this boot runs with: the pairing's election, recorded on this box,
/// else `Primary`** (stinky SB #36). Nothing else names a role — an environment
/// or a config that tries is refused before this is asked
/// ([`refuse_declared_role`]). A box no installer elected (a single box, a test)
/// is a primary, which is the safe direction of the default; a record that
/// cannot be read is an `Err`, never a primary.
pub fn effective_role(data_root: &Path) -> Result<NodeRole, String> {
Ok(recorded_role(data_root)?
.map(|r| r.role)
.unwrap_or_default())
}
#[cfg(test)]
mod tests {
use super::*;
fn twin(primary: &str) -> NodeRole {
NodeRole::Twin {
primary: primary.to_string(),
poll_seconds: DEFAULT_POLL_SECONDS,
}
}
/// ★ **A twin whose credential arrives follows at once, not at its next tick** (FAST-REL,
/// 2026-10-25, Holger's fast under load): the follow's first cycle ran before the front had
/// certified the twin (`holger-twin-13`, no replication credential), then the loop slept
/// its whole 60 s; the certificate landed 3 s later and the pairing's `follow` step waited
/// out the rest ("this twin has never completed a follow cycle"). The install wakes the
/// rest; a wake that comes while a cycle runs is kept for the next rest, never lost.
#[tokio::test]
async fn a_twin_whose_credential_arrives_follows_at_once_not_at_its_next_tick() {
use std::time::{Duration, Instant};
let p = Arc::new(TwinPosture::new(twin("10.44.0.2:50051")));
let resting = p.clone();
let rest = tokio::spawn(async move {
let t = Instant::now();
resting.rest_between_follows(Duration::from_secs(60)).await;
t.elapsed()
});
tokio::time::sleep(Duration::from_millis(100)).await;
p.wake_follow();
let took = tokio::time::timeout(Duration::from_secs(5), rest)
.await
.expect("the install wakes a resting follow")
.unwrap();
assert!(took < Duration::from_secs(5), "woken after {took:?}");
p.wake_follow();
tokio::time::timeout(
Duration::from_secs(5),
p.rest_between_follows(Duration::from_secs(60)),
)
.await
.expect("a wake during a cycle is kept for the next rest");
assert!(
tokio::time::timeout(
Duration::from_millis(300),
p.rest_between_follows(Duration::from_secs(60))
)
.await
.is_err(),
"with no wake the follow rests its tick"
);
}
/// **An absent `role:` is `Primary`, and no shape of an absent key can spell
/// `Twin`.** The dangerous direction of this default is the other one: a
/// config that lost its role key and came up as a follower would refuse its
/// operator's writes and name a primary it was never told about.
#[test]
fn an_absent_role_is_primary_not_twin() {
// The Rust default.
assert_eq!(NodeRole::default(), NodeRole::Primary);
assert!(!NodeRole::default().is_twin());
assert_eq!(NodeRole::default().primary(), None);
assert!(NodeRole::default().write_refusal().is_none());
// The RON default, through the real container: a config that says
// nothing about a role parses as a primary.
#[derive(Deserialize, Default)]
struct Node {
#[serde(default)]
role: NodeRole,
}
let silent: Node = ron::from_str("(role: Primary)").expect("explicit primary parses");
assert_eq!(silent.role, NodeRole::Primary);
let absent: Node = ron::from_str("()").expect("an absent role parses");
assert_eq!(absent.role, NodeRole::Primary, "an absent key is a PRIMARY");
// And `Twin` is only ever reached by naming it AND naming a primary:
// there is no bare `Twin` for a half-written config to land on.
let declared: Node =
ron::from_str("(role: Twin(primary: \"holger-primary.utility:50051\"))")
.expect("a declared twin parses");
assert_eq!(
declared.role.primary(),
Some("holger-primary.utility:50051")
);
assert!(
ron::from_str::<Node>("(role: Twin())").is_err(),
"a Twin that names no primary is not a configuration"
);
}
/// **A twin holds no public socket.** Every configured door keeps its port
/// and moves to loopback; a primary's doors are untouched. A door with no
/// port at all is replaced wholesale rather than left public.
#[test]
fn a_twin_binds_no_public_door() {
let t = twin("holger-primary.utility:50051");
for configured in ["0.0.0.0:50051", "10.4.0.7:8443", "0.0.0.0:80"] {
let bound = t.effective_bind(configured);
let port = configured
.rsplit_once(':')
.expect("test inputs carry a port")
.1;
assert_eq!(
bound,
format!("{TWIN_LOOPBACK_HOST}:{port}"),
"{configured}"
);
assert!(
bound.starts_with(TWIN_LOOPBACK_HOST),
"a twin door must be loopback: {bound}"
);
}
// No port to keep ⇒ loopback anyway, never the public host.
assert_eq!(t.effective_bind("0.0.0.0"), TWIN_LOOPBACK_HOST);
assert_eq!(t.effective_bind(""), TWIN_LOOPBACK_HOST);
// The CONTROL door stays where the config puts it, on a twin too.
assert_eq!(
t.effective_control_bind("10.13.8.3:50051"),
"10.13.8.3:50051"
);
// A primary binds exactly what its config says — this rewrite is the
// ROLE's, not a global policy.
assert_eq!(
NodeRole::Primary.effective_bind("0.0.0.0:50051"),
"0.0.0.0:50051"
);
}
/// **The direction law.** Only a twin produces a plan to dial anything; a
/// primary produces `None`, and there is no field on `NodeRole::Primary`
/// that could name a follower, so a primary has no address to dial even in
/// principle. gunnar's F13, as a type.
#[test]
fn a_primary_never_dials_its_twin() {
assert!(
NodeRole::Primary.follow_plan().is_none(),
"a primary dials nobody"
);
assert_eq!(
NodeRole::Primary.primary(),
None,
"a primary names no peer at all — not a primary, and not a twin"
);
let plan = twin("holger-primary.utility:50051")
.follow_plan()
.expect("a twin follows somebody");
assert_eq!(plan.primary, "holger-primary.utility:50051");
assert_eq!(
plan.poll,
std::time::Duration::from_secs(DEFAULT_POLL_SECONDS)
);
// A poll of zero would be a hot loop, not a disabled follower: clamped.
let hot = NodeRole::Twin {
primary: "p:50051".into(),
poll_seconds: 0,
}
.follow_plan()
.expect("still a twin");
assert_eq!(hot.poll, std::time::Duration::from_secs(1));
}
/// ★ **The role comes only from the pairing's election** (stinky SB #36): an
/// environment (`HOLGER_ROLE`, `HOLGER_PRIMARY`, `HOLGER_TWIN_POLL_SECONDS`)
/// or a config (`role:`) that names a role is refused BY NAME (`holger-twin-21`),
/// never read — a role is the installer's verdict, recorded on the box.
#[test]
fn a_role_named_by_the_environment_or_the_config_is_refused_by_name() {
for (key, value) in [
(ROLE_ENV, "twin"),
(ROLE_ENV, "primary"),
(PRIMARY_ENV, "p:50051"),
(POLL_ENV, "5"),
] {
let e = refuse_declared_role(|k| (k == key).then(|| value.to_string()), None)
.expect_err("an environment role is refused");
assert!(
e.contains("holger-twin-21") && e.contains(key),
"{key}: {e}"
);
}
assert_eq!(
refuse_declared_role(|_| Some(" ".into()), None),
Ok(()),
"an empty variable names nothing"
);
assert_eq!(
refuse_declared_role(|_| None, None),
Ok(()),
"nothing named: the election decides"
);
for role in [
NodeRole::Primary,
NodeRole::Twin {
primary: "p:50051".into(),
poll_seconds: 1,
},
] {
let e =
refuse_declared_role(|_| None, Some(&role)).expect_err("a config role is refused");
assert!(e.contains("holger-twin-21") && e.contains("role:"), "{e}");
}
}
fn cap(available: u64, used: u64) -> Capacity {
Capacity {
total_bytes: available + used,
available_bytes: available,
used_by_holger_bytes: used,
path: PathBuf::from("/data"),
cpu_cores: 2,
ram_bytes: 1 << 30,
}
}
/// **`twin ≥ primary` is reported, never enforced.** A short twin produces
/// the frozen `holger-twin-2` warning naming both numbers and the shortfall;
/// an adequate one produces no warning; and an unread primary is `Unknown`,
/// which is NOT a pass.
#[test]
fn a_short_twin_is_named_and_never_refused() {
let short = store_verdict(&cap(0, 900), &cap(100, 0));
assert!(!short.is_sufficient());
let warning = short.warning().expect("a short twin warns");
assert_eq!(
holger_errcode::parse(&warning).map(|c| c.code),
Some(holger_errcode::TWIN_STORE_TOO_SMALL.code),
"{warning}"
);
assert!(warning.contains("800"), "the shortfall is named: {warning}");
assert!(warning.contains("Nothing is refused"), "{warning}");
let ok = store_verdict(&cap(0, 900), &cap(900, 0));
assert!(ok.is_sufficient());
assert!(ok.warning().is_none(), "an adequate twin says nothing");
assert!(
!StoreVerdict::Unknown.is_sufficient(),
"an unasked question is not a pass"
);
// The door reports it, and a primary's door says so plainly.
let posture = TwinPosture::new(twin("holger-primary.utility:50051"));
posture.record_verdict(short);
let body = posture.door_json();
assert!(body.contains("\"role\":\"twin\""), "{body}");
assert!(body.contains("holger-primary.utility:50051"), "{body}");
assert!(body.contains("\"accepts_writes\":false"), "{body}");
assert!(
body.contains(holger_errcode::TWIN_STORE_TOO_SMALL.code),
"{body}"
);
let primary = TwinPosture::default();
let body = primary.door_json();
assert!(body.contains("\"role\":\"primary\""), "{body}");
assert!(body.contains("\"accepts_writes\":true"), "{body}");
}
// ── W10: replication that can actually make a twin ───────────────────────
/// ★★ **THE RESTORE PATH, END TO END: holger.rs is gone.**
///
/// The owner's order is one sentence — *"when main dies, it can be fed from
/// the twin… the twin has config + data"* — and until 2026-09-21 nothing in
/// holger could perform it: `replicate_estate` had NO caller outside its own
/// tests, so there was no verb, in any binary, that moved a twin's estate
/// onto a new box. `holger-server estate --from --to` is that verb, and this
/// test is the whole story it has to survive.
///
/// 1. The primary holds evidence, a property and its served config.
/// 2. The twin follows it.
/// 3. **The primary is DELETED** — not emptied, removed, the way a
/// terminated VM is.
/// 4. A new, empty box is fed FROM THE TWIN, which is the same call with the
/// roots swapped, because a restore that ran different code from the
/// follow would be code exercised once, on the worst day.
/// 5. The new box holds it all, read back through the stores that own it.
#[cfg(feature = "intake")]
#[test]
fn a_twin_can_feed_a_new_primary_after_the_old_one_is_gone() {
use crate::attest::{ArtifactDigest, AttestStore};
use traits::ArtifactId;
let root = scratch("restore");
let dead = root.join("holger-rs");
let twin = root.join("twin");
let reborn = root.join("the-new-box");
std::fs::create_dir_all(&dead).expect("the primary");
let id = ArtifactId {
namespace: Some("acme".to_string()),
name: "widget".into(),
version: "1.0.0".into(),
};
let payload = b"bytes the estate is evidence about";
let digest = ArtifactDigest::of_sha256(payload);
let attest = AttestStore::new(dead.join("holger-attestations"));
let dir = attest.digest_dir(&digest);
std::fs::create_dir_all(&dir).expect("evidence dir");
std::fs::write(dir.join("gate-1.dsse.json"), b"{\"signatures\":[]}").expect("evidence");
crate::properties::PropertyStore::new(dead.join("holger-props"))
.set("prod", &id, "keep", vec!["forever".to_string()])
.expect("a property");
let config = b"(repositories: [], exposed_endpoints: [], storage_endpoints: [])";
std::fs::write(dead.join("holger.ron"), config).expect("served config");
// 2. THE TWIN FOLLOWS.
let followed = crate::replication::replicate_estate(
&dead,
&twin,
&crate::replication::ReplicationOptions::executing(),
)
.expect("the twin follows");
assert!(followed.is_complete() && followed.copied() > 0);
// 3. THE PRIMARY IS GONE.
std::fs::remove_dir_all(&dead).expect("the machine is terminated");
assert!(
!dead.exists(),
"there is no restore source but the twin now"
);
// 4. FEED A NEW BOX FROM THE TWIN — the same call, roots swapped.
let restored = crate::replication::replicate_estate(
&twin,
&reborn,
&crate::replication::ReplicationOptions::executing(),
)
.expect("the new box is fed from the twin");
assert!(restored.is_complete(), "{:?}", restored.items);
assert!(
restored.copied() > 0,
"a restore that copied nothing is not a restore"
);
// 5. READ IT BACK THROUGH THE STORES THAT OWN IT, on the new box.
let new_attest = AttestStore::new(reborn.join("holger-attestations"));
assert!(
new_attest
.digest_dir(&digest)
.join("gate-1.dsse.json")
.exists(),
"the new primary serves bytes nothing has ever attested"
);
assert_eq!(
crate::properties::PropertyStore::new(reborn.join("holger-props"))
.get("prod", &id)
.get("keep")
.cloned(),
Some(vec!["forever".to_string()]),
"the retention pin did not survive the restore"
);
assert_eq!(
std::fs::read(reborn.join("holger.ron")).expect("the config came back"),
config,
"the restored box has no config to serve"
);
// ★ AND THE HALF IT DOES NOT CARRY, SAID OUT LOUD. The artifacts are not
// here: they move with `replicate --from --to`. A restore that reported
// success over an artifact-less box would be the worst lie holger could
// tell.
assert!(
restored.unreplicated.is_empty(),
"this fixture has no artifact store; a real one is named, not moved"
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★★ **ONE MEDIUM, NO ROLE PARAMETER: the role is conferred at BOOTSTRAP.**
///
/// Owner rulings 2026-09-21: *"the weakest machine must become the twin.
/// PERIOD"* and *"IT must not CHANGE role later ! so its a bootstrap thing
/// from xtask"*. Both boxes install from ONE identical ISO and come up as
/// primaries — because an absent role is `Primary`, always — and the control
/// plane then tells the weaker one what it is.
#[test]
fn two_boxes_from_one_medium_and_the_weaker_is_told_what_it_is() {
let root = scratch("elect");
let strong = root.join("holger-server");
let weak = root.join("holger-twin");
for b in [&strong, &weak] {
std::fs::create_dir_all(b).expect("an installed box");
// What the ONE medium lays down: a config, and nothing else.
std::fs::write(b.join("holger.ron"), b"(repositories: [])").expect("the config");
}
// Out of the box, with no parameter anywhere, BOTH are primaries.
for b in [&strong, &weak] {
assert_eq!(
effective_role(b).expect("a fresh box reads"),
NodeRole::Primary,
"a box installed from an unparameterised medium is a primary"
);
assert!(
recorded_role(b).unwrap().is_none(),
"no election has happened"
);
assert!(is_fresh(b), "a box with only a config is fresh");
}
// The control plane measures and elects the weaker one. ONCE.
let record = elect(
&weak,
NodeRole::Twin {
primary: "grpc.holger.rs:50051".into(),
poll_seconds: 60,
},
"console",
"1 core / 1 GB / 20 GB hdd vs 2 / 4 / 20 maxiops",
1_700_000_000,
)
.expect("the weaker box is elected");
assert_eq!(record.elected_by, "console");
assert!(
record.reason.contains("hdd"),
"the measurement is recorded verbatim"
);
// From the next boot it IS a twin — with a config that never changed.
let effective = effective_role(&weak).expect("reads back");
assert!(effective.is_twin());
assert_eq!(effective.primary(), Some("grpc.holger.rs:50051"));
assert_eq!(effective.effective_bind("0.0.0.0:50051"), "127.0.0.1:50051");
assert!(effective.write_refusal().is_some());
// …and the primary is untouched by its twin's election.
assert_eq!(effective_role(&strong).unwrap(), NodeRole::Primary);
let _ = std::fs::remove_dir_all(&root);
}
/// **The audit index is no history** (HOL-AUDIT c3dbbef5 builds `audit/index/` at every start;
/// since then a freshly started box was refused its twin election `holger-twin-5`, and the
/// twin's recharge tests were red): the index is derived from the chain and deleting it is
/// always safe, so a box whose only audit bytes are the index is fresh. A chain line counts.
#[test]
fn the_audit_index_a_start_builds_is_no_history_and_a_chain_line_is() {
let root = scratch("audit-index");
let b = root.join("holger-twin");
let index = b.join("audit").join(crate::audit::arrow_ipc::INDEX_DIR);
std::fs::create_dir_all(&index).unwrap();
std::fs::write(b.join("holger.ron"), b"(repositories: [])").unwrap();
std::fs::write(index.join("MANIFEST.json"), br#"{"segments":[]}"#).unwrap();
std::fs::write(index.join("seg-000000.arrow"), b"ARROW1").unwrap();
assert_eq!(
held_state(&b),
None,
"the index a start builds is not history"
);
assert!(is_fresh(&b));
std::fs::write(b.join("audit").join("audit.jsonl"), b"{\"seq\":1}\n").unwrap();
assert_eq!(
held_state(&b),
Some(b.join("audit").join("audit.jsonl")),
"a chain line is history"
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★★ **A ROLE IS NEVER RE-EVALUATED.** *"IT must not CHANGE role later!"* —
/// so a second election is a refusal by name, including a second election to
/// the SAME role, because a caller that thinks it needs one has lost track of
/// which box it is talking to.
#[test]
fn a_second_election_is_refused_by_name() {
let root = scratch("reelect");
let box_ = root.join("box");
std::fs::create_dir_all(&box_).unwrap();
let twin_role = NodeRole::Twin {
primary: "p:50051".into(),
poll_seconds: 60,
};
elect(
&box_,
twin_role.clone(),
"console",
"measured",
1_700_000_000,
)
.expect("the first election");
for (role, what) in [
(NodeRole::Primary, "a promotion"),
(twin_role.clone(), "the same role again"),
] {
let err = elect(&box_, role, "console", "measured again", 1_700_000_100)
.expect_err("a second election must refuse");
assert!(
err.contains(holger_errcode::TWIN_ROLE_ALREADY_ELECTED.code),
"{what} was not refused by name: {err}"
);
assert!(
err.contains("console"),
"the refusal does not say who elected it: {err}"
);
}
// A corrupt record is an ERROR, never a silent "no role" — that would be
// a twin coming up as a primary.
std::fs::write(role_record_path(&box_), b"this is not a role record").unwrap();
assert!(
recorded_role(&box_).is_err(),
"a corrupt record must not read as absent"
);
assert!(effective_role(&box_).is_err());
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **A started box is still fresh with the audit dir a box really has**
/// (`holger-audit`, the default and the estate's): the zero-byte chain a start
/// opens and the audit index (derived from the chain) are no history; one
/// chained record is.
#[test]
fn the_audit_index_under_the_real_audit_dir_is_no_history() {
let root = scratch("index-real-dir");
for dir in ["holger-audit", "audit", "my-audit"] {
let box_root = root.join(dir);
let audit = box_root.join(dir);
let idx = audit.join(crate::audit::arrow_ipc::INDEX_DIR);
std::fs::create_dir_all(&idx).unwrap();
std::fs::write(audit.join(crate::audit::AUDIT_FILE), b"").unwrap();
std::fs::write(idx.join(crate::audit::arrow_ipc::MANIFEST), b"{\"rows\":0}").unwrap();
std::fs::write(idx.join("seg-00000000000000000000.arrow"), b"ARROW1").unwrap();
assert_eq!(
held_state(&box_root),
None,
"{dir}: a started box with its index is fresh"
);
std::fs::write(audit.join(crate::audit::AUDIT_FILE), b"{\"seq\":0}\n").unwrap();
assert!(
held_state(&box_root).is_some(),
"{dir}: a chained record is history"
);
}
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **Only a FRESH box may be elected a twin**, and a recorded role that
/// disagrees with the environment is a refusal rather than a precedence
/// decision.
#[test]
fn a_serving_box_is_not_elected_and_a_disagreement_is_refused() {
let root = scratch("fresh");
let serving = root.join("serving");
std::fs::create_dir_all(serving.join("holger-audit")).unwrap();
std::fs::write(serving.join("holger-audit").join("seg-0.arrow"), b"history").unwrap();
assert!(
!is_fresh(&serving),
"a box with audit segments is not fresh"
);
let err = elect(
&serving,
NodeRole::Twin {
primary: "p:50051".into(),
poll_seconds: 60,
},
"console",
"measured",
1_700_000_000,
)
.expect_err("a serving box must not be elected a twin");
assert!(
err.contains(holger_errcode::TWIN_BOX_NOT_FRESH.code),
"{err}"
);
// An artifact store makes a box un-fresh too — it is the biggest thing a
// box can hold and it is named by no ESTATE entry.
let stored = root.join("stored");
std::fs::create_dir_all(stored.join("store")).unwrap();
std::fs::write(stored.join("store").join("crates.znippy"), b"bytes").unwrap();
assert!(!is_fresh(&stored));
// A recorded twin is a twin whatever an environment says: the
// environment is refused before the record is read (SB #36).
let elected = root.join("elected");
std::fs::create_dir_all(&elected).unwrap();
elect(
&elected,
NodeRole::Twin {
primary: "p:50051".into(),
poll_seconds: 60,
},
"console",
"measured",
1_700_000_000,
)
.unwrap();
assert!(effective_role(&elected).unwrap().is_twin());
let refused =
refuse_declared_role(|k| (k == ROLE_ENV).then(|| "primary".to_string()), None)
.expect_err("an environment role is refused");
assert!(
refused.contains(holger_errcode::TWIN_ROLE_NOT_FROM_PAIRING.code),
"{refused}"
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **A role never travels.** The record must not be replicated: a restore
/// that carried it would bring the dead primary's twin-hood onto the new
/// box, which would come up refusing every write and naming a machine that
/// no longer exists.
#[test]
fn the_role_record_is_never_replicated() {
let root = scratch("norole");
let twin = root.join("twin");
let reborn = root.join("reborn");
std::fs::create_dir_all(&twin).unwrap();
elect(
&twin,
NodeRole::Twin {
primary: "dead.holger.rs:50051".into(),
poll_seconds: 60,
},
"console",
"measured",
1_700_000_000,
)
.unwrap();
std::fs::write(twin.join("holger.ron"), b"(repositories: [])").unwrap();
let report = crate::replication::replicate_estate(
&twin,
&reborn,
&crate::replication::ReplicationOptions::executing(),
)
.expect("the restore runs");
assert!(
!report.unreplicated.contains(&ROLE_RECORD.to_string()),
"the role record is EXCLUDED on purpose, not reported as uncovered"
);
assert!(
!role_record_path(&reborn).exists(),
"the restored box inherited its dead primary's twin-hood"
);
assert_eq!(
effective_role(&reborn).unwrap(),
NodeRole::Primary,
"a box fed from a twin comes up a PRIMARY"
);
let _ = std::fs::remove_dir_all(&root);
}
fn scratch(tag: &str) -> PathBuf {
let p = std::env::temp_dir().join(format!(
"holger-twin-{tag}-{}",
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.expect("clock")
.as_nanos()
));
std::fs::create_dir_all(&p).expect("scratch root");
p
}
/// **`replicate_pairs` moves the BYTES; this moves the reasons to believe
/// them.** Plan §3 W10, stinky 02 S2-4.
///
/// A twin built from the artifact engine alone holds artifacts and nothing
/// that says where they came from: no evidence store, no audit segments, no
/// properties, no SBOMs, no sealed PROD store, no bundle state, no served
/// config. Promoted, it serves bytes NOTHING has ever attested and a history
/// that begins at the promotion — gunnar's un-metered-twin scar in a
/// different coat.
///
/// So: promote an artifact on A, attach evidence, a property and an SBOM to
/// it, write an audit segment and a served config — then replicate the
/// estate to B and read every one of them back **through the store that owns
/// it**, at the path that store itself computes on B. Then replicate again
/// and assert the second run is a pure no-op, which is the idempotency the
/// artifact half already has.
#[cfg(feature = "intake")]
#[test]
fn a_replicated_twin_holds_the_evidence_not_just_the_bytes() {
use crate::attest::{ArtifactDigest, AttestStore};
use crate::audit::{AuditAction, AuditEvent, AuditLog};
use traits::ArtifactId;
let root = scratch("estate");
let a = root.join("primary");
let b = root.join("twin");
std::fs::create_dir_all(&a).expect("primary root");
let id = ArtifactId {
namespace: Some("acme".to_string()),
name: "widget".into(),
version: "1.0.0".into(),
};
let payload = b"the artifact bytes a twin would serve after a promotion";
let digest = ArtifactDigest::of_sha256(payload);
// 1. PROMOTED — the sealed PROD store and its immutable ref.
let engine = crate::promote::PromotionEngine::new(a.join("holger-prod-store"));
engine
.seal("dev", "prod", &id, payload, "tester", 1_700_000_000)
.expect("the artifact is promoted on the primary");
assert!(engine.is_promoted("prod", &id));
// 2. EVIDENCE — a DSSE envelope filed about exactly those bytes. Written
// at the path the store itself names, so the read-back on B is the
// store's own path rule and not a second copy of it.
let a_attest = AttestStore::new(a.join("holger-attestations"));
let envelope = br#"{"payloadType":"application/vnd.in-toto+json","payload":"e30=","signatures":[{"keyid":"gate-1","sig":"AA=="}]}"#;
let evidence_dir = a_attest.digest_dir(&digest);
std::fs::create_dir_all(&evidence_dir).expect("evidence dir");
std::fs::write(evidence_dir.join("gate-1.dsse.json"), envelope).expect("file the evidence");
// 3. A PROPERTY — the retention pin is one of these, and so is the
// promotion engine's required property.
let a_props = crate::properties::PropertyStore::new(a.join("holger-props"));
a_props
.set("prod", &id, "keep", vec!["forever".to_string()])
.expect("the property is set on the primary");
// 4. AN SBOM.
let a_sboms = crate::sbom::SbomStore::new(a.join("holger-sboms"));
let sbom = br#"{"bomFormat":"CycloneDX","specVersion":"1.5","components":[]}"#;
a_sboms
.attach("prod", &id, sbom)
.expect("the SBOM is attached on the primary");
// 5. THE AUDIT FILE — append-only, the history.
{
let log = crate::audit::FileAuditLog::new(a.join("holger-audit"))
.expect("audit sink on the primary");
log.record(AuditEvent::new(
"somebody",
AuditAction::Download,
"prod",
"acme/widget@1.0.0",
"10.0.0.9:4242",
200,
payload.len() as u64,
))
.expect("record");
log.flush().expect("flush the segment to disk");
}
// 6. THE SERVED CONFIG — byte-identical is the estate's rule.
let config = b"(repositories: [], exposed_endpoints: [], storage_endpoints: [])";
std::fs::write(a.join("holger.ron"), config).expect("served config");
// ── The dry run writes NOTHING. ──
let preview = crate::replication::replicate_estate(
&a,
&b,
&crate::replication::ReplicationOptions::preview(),
)
.expect("a preview runs");
assert!(preview.would_copy() > 0, "there is an estate to copy");
assert_eq!(preview.copied(), 0, "a dry run copies nothing");
assert!(!b.exists(), "a dry run does not even make the twin's root");
// ── The real run. ──
let run = crate::replication::replicate_estate(
&a,
&b,
&crate::replication::ReplicationOptions::executing(),
)
.expect("the estate replicates");
assert!(run.is_complete(), "nothing may fail: {:?}", run.items);
assert!(run.copied() > 0);
// ── Now read every one of them back FROM THE TWIN, each through the
// store that owns it. This is the assertion the whole lane exists for.
// ──
// artifacts + the promotion seal
let b_engine = crate::promote::PromotionEngine::new(b.join("holger-prod-store"));
assert!(
b_engine.is_promoted("prod", &id),
"the twin must hold the PROMOTION, not merely the bytes"
);
assert_eq!(
b_engine
.fetch("prod", &id)
.expect("the twin serves it")
.as_deref(),
Some(&payload[..]),
"and the sealed bytes must be the same bytes"
);
// evidence
let b_attest = AttestStore::new(b.join("holger-attestations"));
let filed = b_attest.digest_dir(&digest).join("gate-1.dsse.json");
assert_eq!(
std::fs::read(&filed)
.expect("the twin holds the evidence")
.as_slice(),
&envelope[..],
"the twin must hold the EVIDENCE about the artifact it serves"
);
// property
let b_props = crate::properties::PropertyStore::new(b.join("holger-props"));
assert_eq!(
b_props.get("prod", &id).get("keep").cloned(),
Some(vec!["forever".to_string()]),
"the twin must hold the properties — the retention pin is one of these"
);
// SBOM
let b_sboms = crate::sbom::SbomStore::new(b.join("holger-sboms"));
assert_eq!(
b_sboms.fetch("prod", &id).expect("sbom read").as_deref(),
Some(&sbom[..]),
"the twin must hold the SBOM"
);
// audit segments — read back through the reader, not off the filesystem
let replayed = crate::audit::FileAuditLog::read_dir(b.join("holger-audit"))
.expect("the twin's audit segments replay");
assert_eq!(replayed.len(), 1, "the twin must hold the HISTORY");
assert_eq!(replayed[0].ident, "somebody");
assert_eq!(replayed[0].artifact, "acme/widget@1.0.0");
// served config, byte-identical
assert_eq!(
std::fs::read(b.join("holger.ron")).expect("the twin holds the config"),
config,
"`[estate-twin-drift]`: byte-identical or it is not a twin"
);
// Every declared item is accounted for by NAME — a store that silently
// stopped being replicated would show up here as a missing row.
for item in crate::replication::ESTATE {
let row = run
.item(item.path)
.unwrap_or_else(|| panic!("no row for {}", item.path));
assert!(
!row.what.is_empty(),
"{}: a row must say what is lost",
item.path
);
}
// An absence is reported as an absence, never as a success: this
// deployment hosts no update bundles and no plugins.
assert!(
run.item("holger-update-state")
.expect("row")
.absent_on_primary
);
assert!(run.item("plugins").expect("row").absent_on_primary);
// ── A fresh replicate is a NO-OP. Idempotent by content id, exactly as
// the artifact half is. ──
let again = crate::replication::replicate_estate(
&a,
&b,
&crate::replication::ReplicationOptions::executing(),
)
.expect("a second run");
assert!(
again.is_noop(),
"a second replicate over an unchanged primary must write nothing: \
copied={} would_copy={} failed={}",
again.copied(),
again.would_copy(),
again.failed()
);
assert_eq!(
again.skipped(),
run.copied(),
"every file skipped by content id"
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **A started box with the passkey plane on is still fresh** — its
/// minted session key is scaffolding (a laid-again main recharging from its
/// twin was refused `holding … holger-passkeys/session.key` on the mock,
/// 2026-10-04) — while a store that enrolled somebody is history.
#[test]
fn a_bare_passkey_session_key_is_scaffolding_and_an_enrolment_is_not() {
let root =
std::env::temp_dir().join(format!("holger-fresh-passkeys-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
std::fs::create_dir_all(root.join("holger-passkeys")).unwrap();
std::fs::write(root.join("holger-passkeys/session.key"), [7u8; 32]).unwrap();
assert!(
is_fresh(&root),
"a bare session key is not held state: {:?}",
held_state(&root)
);
std::fs::write(root.join("holger-passkeys/credentials.json"), b"[{}]").unwrap();
assert_eq!(
held_state(&root),
Some(root.join("holger-passkeys/credentials.json"))
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **A box fed the shop's first-login invite before its election is still fresh** (fast
/// holger RED 2026-10-28, lane OX-TRAIT2): every box of a sold estate is fed the invite at
/// boot, before the installer elects the twin, so the twin's passkey door wrote the record
/// `holger-passkeys/first-admin.json` and its election was refused `holger-twin-5`. A record
/// nobody used is the feed's, not history; one a buyer registered under still is.
#[test]
fn an_unused_first_login_record_is_scaffolding_and_a_used_one_is_not() {
let root =
std::env::temp_dir().join(format!("holger-fresh-first-admin-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
std::fs::create_dir_all(root.join("holger-passkeys")).unwrap();
let marker = root.join(crate::auth::FIRST_ADMIN_MARKER_REL);
std::fs::write(
&marker,
br#"{"sha256":"ab","at":1,"seeded":true,"display":"acme","ttl_secs":3600}"#,
)
.unwrap();
assert!(
is_fresh(&root),
"a fed first-login record nobody used is not held state: {:?}",
held_state(&root)
);
std::fs::write(&marker, br#"{"sha256":"ab","at":1,"seeded":true,"display":"acme","ttl_secs":3600,"used_by":"passkey:acme"}"#).unwrap();
assert_eq!(
held_state(&root),
Some(marker.clone()),
"a buyer registered under it: history"
);
std::fs::write(&marker, b"not json").unwrap();
assert_eq!(
held_state(&root),
Some(marker),
"a record that cannot be read counts"
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **The scan sandbox a start lays from the medium is not history** (lane OX-WGDS4, KVM gate
/// d47: a main laid again after its death was fed `recharge_from` and was NOT held: "this box
/// holds data of its own (/data/holger/sandbox/laid.toml): it is not held, and it serves", so
/// it served empty under a fresh sync key and was never filled from its twin). Every start
/// lays `<data_root>/sandbox` from the running slot (`holger_sandbox::media::Layout`: crun, the
/// worker's and ClamAV's rootfs, `laid.toml`) before it serves; the medium's, not the box's.
#[test]
fn the_sandbox_a_start_lays_from_the_medium_is_scaffolding() {
let root =
std::env::temp_dir().join(format!("holger-fresh-sandbox-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
std::fs::create_dir_all(root.join("sandbox/worker/rootfs/bin")).unwrap();
std::fs::write(
root.join("sandbox/laid.toml"),
b"[entries]\ncrun = \"ab\"\n",
)
.unwrap();
std::fs::write(root.join("sandbox/crun"), b"\x7fELF").unwrap();
std::fs::write(root.join("sandbox/worker/rootfs/bin/scan"), b"\x7fELF").unwrap();
assert!(
is_fresh(&root),
"the medium's sandbox is not held state: {:?}",
held_state(&root)
);
std::fs::create_dir_all(root.join("repositories/bundles")).unwrap();
std::fs::write(root.join("repositories/bundles/a.znippy"), b"x").unwrap();
assert!(!is_fresh(&root), "an artifact beside it still is");
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **The Sigstore trust root a start seeds from the medium is not history** (lane
/// HOL-REBUILT-SYNC-KEY, the KVM gate `gate holger --downstreams` 2026-10-31 15:43Z on holger
/// 70235ded: the main laid again after its death was NOT held for its recharge, served empty
/// under a fresh sync key, and every downstream pinned to the old one would refuse it). Every
/// start of a box whose medium carries the trust root (HOL-OCI-EVIDENCE, `sandbox_boot`)
/// writes `holger-trust/` (the bundle, its policy, `installed.json` from `medium`) and the
/// trust product's floor marker under `holger-update-state/`. The medium's, not the box's.
/// A root an import installed, or another product's floor, is still history.
#[test]
fn the_trust_root_a_start_seeds_from_the_medium_is_scaffolding_and_an_import_is_not() {
let root = std::env::temp_dir().join(format!(
"holger-fresh-sigstore-trust-{}",
std::process::id()
));
let _ = std::fs::remove_dir_all(&root);
let installed = |from: &str| {
format!(
r#"{{"bundle_id":"b-7","sequence":7,"trusted_root_sha256":"ab","from":"{from}","at":"2026-10-31T00:00:00Z","path":"/x"}}"#
)
};
std::fs::create_dir_all(root.join("holger-trust")).unwrap();
std::fs::write(
root.join("holger-trust/sigstore-trust.holger.znippy"),
b"bundle",
)
.unwrap();
std::fs::write(root.join("holger-trust/bundle-trust.ron"), b"(policy)").unwrap();
std::fs::write(
root.join("holger-trust/installed.json"),
installed("medium"),
)
.unwrap();
let floor = root.join("holger-update-state/holger/holger.sigstore-trust_v1");
std::fs::create_dir_all(&floor).unwrap();
std::fs::write(
floor.join(format!("{:020}.accepted", 7)),
br#"{"sequence":7,"bundle_id":"b-7","core_digest":"cd","accepted_at":"2026-10-31T00:00:00Z"}"#,
)
.unwrap();
assert!(
is_fresh(&root),
"the medium's trust root and its floor are not held state: {:?}",
held_state(&root)
);
assert_eq!(
held_state_except(
&root,
&["holger-audit", "holger-recharged", "holger-recharging"]
),
None,
"what a recharge asks: fresh"
);
// A floor marker of another sequence is an import's: history.
let other = floor.join(format!("{:020}.accepted", 8));
std::fs::write(&other, b"{}").unwrap();
assert_eq!(held_state(&root), Some(other.clone()));
std::fs::remove_file(&other).unwrap();
// Another product's floor (an intel import) is history.
let intel = root.join("holger-update-state/holger/holger.intel_v1");
std::fs::create_dir_all(&intel).unwrap();
std::fs::write(intel.join(format!("{:020}.accepted", 3)), b"{}").unwrap();
assert!(!is_fresh(&root), "an intel import's floor is history");
std::fs::remove_dir_all(&intel).unwrap();
// A root an import installed is history.
std::fs::write(
root.join("holger-trust/installed.json"),
installed("import"),
)
.unwrap();
assert!(
!is_fresh(&root),
"an imported trust root is history: {:?}",
held_state(&root)
);
let _ = std::fs::remove_dir_all(&root);
}
/// The names [`seeded_from_the_medium`] reads are the seed's own (`sigstore_trust`,
/// `import_door`); `RollbackStore` sanitizes the product's `/` to `_`.
#[cfg(feature = "intake")]
#[test]
fn the_trust_seed_names_are_the_seeds_own() {
assert_eq!(TRUST_DIR, crate::sigstore_trust::DIR);
assert_eq!(TRUST_INSTALLED, crate::sigstore_trust::INSTALLED_FILE);
assert_eq!(UPDATE_STATE, crate::import_door::UPDATE_STATE_DIR);
assert_eq!(TRUST_FLOOR[0], crate::sigstore_trust::SUPPLIER);
assert_eq!(
TRUST_FLOOR[1],
crate::sigstore_trust::TRUST_PRODUCT.replace('/', "_")
);
}
/// ★ **A started box of the shipped declaration is still fresh** (lane D3): every
/// start writes the staged korp root as `plugins/trust/<stem>.der` (W2,
/// `Holger::materialize_pem_trust_roots`) — config, not history — so the twin's
/// election after a start must not be refused `holger-twin-5`. A loaded plugin
/// module beside it is still history.
#[test]
fn the_trust_root_a_start_writes_is_scaffolding_and_a_plugin_is_not() {
let root = std::env::temp_dir().join(format!("holger-fresh-trust-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
std::fs::create_dir_all(root.join("plugins/trust")).unwrap();
std::fs::write(
root.join("plugins/trust/korp-root-ca.der"),
[0x30u8, 0x82, 1, 2],
)
.unwrap();
assert!(
is_fresh(&root),
"the config's trust root is not held state: {:?}",
held_state(&root)
);
std::fs::write(root.join("plugins/meter.wasm"), b"\0asm").unwrap();
assert_eq!(held_state(&root), Some(root.join("plugins/meter.wasm")));
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **Guard: everything the feed and the pairing loop write before the election, at once,
/// leaves a box fresh** (lane HOL-311-RELEASE). Each of these refused a twin election once
/// (`holger-twin-5`): the sync key a start mints, the front's offered downstream interface,
/// monetize's meter key, the unused first-login record, the passkey session key, and the
/// front-report key. A new file the feed writes before the election belongs in this list.
#[test]
fn everything_fed_before_the_election_together_leaves_a_box_fresh() {
let root = std::env::temp_dir().join(format!("holger-fresh-allfed-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
let fed: &[(&str, &[u8])] = &[
(
"holger-sync/front-hub.ron",
b"(key: \"K=\", endpoint: \"203.0.113.4:51830\")",
),
(
"meters/monetize/key.hex",
b"abababababababababababababababababababababababababababababababab",
),
(
"holger-passkeys/session.key",
b"0123456789abcdef0123456789abcdef",
),
(
"holger-front/report.key",
b"abababababababababababababababababababababababababababababababab",
),
];
for (rel, bytes) in fed {
let p = root.join(rel);
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
std::fs::write(&p, bytes).unwrap();
}
let first = root.join(crate::auth::FIRST_ADMIN_MARKER_REL);
std::fs::create_dir_all(first.parent().unwrap()).unwrap();
std::fs::write(&first, b"{\"used_by\": null}").unwrap();
assert!(
is_fresh(&root),
"every file fed before the election, together, is no history: {:?}",
held_state(&root)
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★★ **History is an ALLOW-LIST** (lane HOL-312-HARDEN, HOL-TWIN-HELD-ALLOWLIST). Every
/// refused election so far (`holger-twin-5`: the sync key, the front hub, the meter key, the
/// first-login record, the passkey session key, the trust roots, the audit index, and in
/// 3.1.1 `holger-front/report.key`) was a file the feed or a start wrote that the deny-list
/// did not know. So a file no history kind names is NOT history, whatever a future feed
/// writes, and every history kind still counts on its own.
#[test]
fn history_is_an_allow_list_and_an_unknown_file_is_not_history() {
let root = scratch("allow-list");
let unknown = root.join("unknown");
std::fs::create_dir_all(&unknown).unwrap();
std::fs::write(unknown.join("holger.ron"), b"(repositories: [])").unwrap();
for rel in [
"holger-some-future-feed/token.key",
"holger-front/another-fed.key",
"fleet/room-want.json",
"holger-peers/vm2.ron",
"holger-time/floor",
"repo-signing/release.pk8",
"a-top-level-file.ron",
] {
let p = unknown.join(rel);
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
std::fs::write(&p, b"fed before the election").unwrap();
}
assert_eq!(
held_state(&unknown),
None,
"a file no history kind names is not history"
);
let config = |store: &str| {
format!(
"(repositories: [], storages: [(ron_name: \"kv\", ron_storage_type: \"rocksdb\", ron_path: \"{store}\")])"
)
};
let history: &[(&str, Option<String>)] = &[
("holger-attestations/a.dsse", None),
("holger-audit/audit.jsonl", None),
("holger-audit/seg-0.arrow", None),
("my-audit/audit.jsonl", None),
("holger-props/p.ron", None),
("holger-sboms/s.json", None),
("holger-prod-store/x", None),
(
"holger-update-state/holger/holger.intel_v1/00000000000000000003.accepted",
None,
),
("holger-import-receipts/r.json", None),
("plugins/meter.wasm", None),
("holger-passkeys/credentials.json", None),
("holger-entitlements/e.json", None),
("meters/monetize/reading.json", None),
("holger-purges/p.json", None),
("fleet/signups.json", None),
("fleet/sold-repositories.json", None),
("holger-sync/downstreams/lab-1.ron", None),
("holger-trust/bundle-trust.ron", None),
("holger-front/report.ron", None),
("repositories/bundles/a.znippy", None),
("stored/store/crates.znippy", None),
("kv-store/CURRENT", Some(config("kv-store"))),
];
for (i, (rel, cfg)) in history.iter().enumerate() {
let b = root.join(format!("h{i}"));
std::fs::create_dir_all(&b).unwrap();
let cfg = cfg.clone().unwrap_or_else(|| "(repositories: [])".into());
std::fs::write(b.join("holger.ron"), cfg).unwrap();
let p = b.join(rel);
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
std::fs::write(&p, b"history").unwrap();
assert_eq!(held_state(&b), Some(p.clone()), "{rel} is history");
}
// A configured store named by its absolute path counts the same.
let abs = root.join("abs");
std::fs::create_dir_all(abs.join("kv/db")).unwrap();
std::fs::write(
abs.join("holger.ron"),
config(&abs.join("kv").display().to_string()),
)
.unwrap();
std::fs::write(abs.join("kv/db/000001.sst"), b"history").unwrap();
assert_eq!(held_state(&abs), Some(abs.join("kv/db/000001.sst")));
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **A repository's loose-file store is history wherever its config names it** (lane
/// HOL-312-HARDEN, the allow-list's first gate: `every_rpc_on_the_wire` saw a box holding a
/// raw artifact under `ron_data_dir` taken for a recharge target, `holger-twin-13` where
/// `holger-twin-16` was due). The config on disk names it (`ron_data_dir`), the running
/// config names it ([`held_state_with`], the follower's stores), and a storage path that IS
/// the data root does not make every file history.
#[test]
fn a_repository_store_the_config_names_is_history() {
let root = scratch("repo-store");
let b = root.join("box");
let artifact = b.join("raw/crates/serde-1.0.crate");
std::fs::create_dir_all(artifact.parent().unwrap()).unwrap();
std::fs::write(&artifact, b"bytes").unwrap();
std::fs::write(b.join("unknown.fed"), b"fed").unwrap();
// No config on disk, no store named: the loose file is no kind of history.
assert_eq!(held_state(&b), None);
// The running config names it.
assert_eq!(
held_state_with(&b, &[], &[b.join("raw"), b.clone()]),
Some(artifact.clone()),
"the running config's store"
);
// The config on disk names it; its storage endpoint is the data root itself.
let d = b.display();
std::fs::write(
b.join("holger.ron"),
format!(
"(repositories: [(ron_name: \"raw\", ron_repo_type: \"raw\", ron_data_dir: Some(\"{d}/raw\"))], storage_endpoints: [(ron_name: \"s\", ron_storage_type: \"znippy\", ron_path: \"{d}\")])"
),
)
.unwrap();
assert_eq!(held_state(&b), Some(artifact.clone()), "the config on disk");
std::fs::remove_file(&artifact).unwrap();
assert_eq!(
held_state(&b),
None,
"a storage path that is the data root is not every file"
);
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **Every estate item is a store of history or named as none** (lane HOL-312-HARDEN): a
/// new ESTATE item is a decision, not a silent hole in the allow-list. The front report the
/// list names is the one `front_report` writes.
#[test]
fn every_estate_item_is_a_history_store_or_named_not_history() {
for item in crate::replication::ESTATE {
assert!(
HISTORY_STORES.contains(&item.path)
|| START_SCAFFOLDING.contains(&item.path)
|| NOT_HISTORY_ESTATE.contains(&item.path),
"the estate item {} is neither history nor named as none",
item.path
);
}
let report = format!(
"{}/{}",
crate::front_report::DIR,
crate::front_report::REPORT_FILE
);
assert!(HISTORY_STORES.contains(&report.as_str()));
assert!(!HISTORY_STORES
.iter()
.any(|s| s.ends_with(crate::front_report::KEY_FILE)));
}
/// ★ **The front-report key the feed writes before the election is not history** (lane
/// HOL-311-RELEASE, the 3.1.1 KVM gate on dwarf 2026-11-02: the twin's election was refused
/// `holger-twin-5` over `holger-front/report.key` from its first try, the front's certify of
/// the twin failed `holger-twin-14`, and the site was never paired): the pairing loop keeps
/// the key of the front's report proof (`front_report::keep_key`, HOL-FLOW-FRONT) on every
/// box from the feed. A report main heard (`report.ron`) is still history.
#[test]
fn the_front_report_key_the_feed_writes_is_not_history() {
let root =
std::env::temp_dir().join(format!("holger-fresh-frontkey-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
std::fs::create_dir_all(root.join(crate::front_report::DIR)).unwrap();
std::fs::write(
root.join(crate::front_report::DIR)
.join(crate::front_report::KEY_FILE),
b"ab".repeat(32),
)
.unwrap();
assert!(
is_fresh(&root),
"the fed front-report key is not held state: {:?}",
held_state(&root)
);
let heard = root
.join(crate::front_report::DIR)
.join(crate::front_report::REPORT_FILE);
std::fs::write(&heard, b"(report: (), heard_ms: 1)").unwrap();
assert_eq!(held_state(&root), Some(heard), "a heard report is history");
let _ = std::fs::remove_dir_all(&root);
}
/// ★ **What the installer feeds every box before the election is not history** (lane
/// OX-WGDS3, the KVM gate on dwarf 2026-10-28: the twin's election was refused `holger-twin-5`
/// over `holger-sync/front-hub.ron`, then over `meters/monetize/key.hex`, and the site never
/// paired): the front's offered downstream interface (`sync::downstreams::keep_front_hub`) and
/// monetize's verifying key in the meter plugin's slot (`fleet_box::METER_KEY_SLOT`, the
/// seed's or the late feed's `monetize_pubkey`). A downstream enrolled, or a meter reading,
/// beside them is still history.
#[test]
fn what_the_installer_feeds_every_box_before_the_election_is_not_history() {
let root = std::env::temp_dir().join(format!("holger-fresh-fed-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
std::fs::create_dir_all(root.join("holger-sync")).unwrap();
std::fs::create_dir_all(root.join("meters/monetize")).unwrap();
std::fs::write(
root.join("holger-sync/front-hub.ron"),
b"(key: \"K=\", endpoint: \"203.0.113.4:51830\")",
)
.unwrap();
std::fs::write(root.join("meters/monetize/key.hex"), b"ab".repeat(32)).unwrap();
assert!(
is_fresh(&root),
"the fed front offer and meter key are not held state: {:?}",
held_state(&root)
);
std::fs::create_dir_all(root.join("holger-sync/downstreams")).unwrap();
std::fs::write(
root.join("holger-sync/downstreams/lab-1.ron"),
b"(name: \"lab-1\")",
)
.unwrap();
assert_eq!(
held_state(&root),
Some(root.join("holger-sync/downstreams/lab-1.ron")),
"an enrolment is history"
);
std::fs::remove_file(root.join("holger-sync/downstreams/lab-1.ron")).unwrap();
std::fs::write(root.join("meters/monetize/reading.json"), b"{}").unwrap();
assert_eq!(
held_state(&root),
Some(root.join("meters/monetize/reading.json")),
"a meter reading is history"
);
let _ = std::fs::remove_dir_all(&root);
}
}