nordisk/Skidbladnir

PROMOTE — guide

How to describe your environments, and how to move an application between them without breaking production. Design + the precise contract: promote-design.md. Never done this before: promote-for-idiots.md.

skidbladnir = { version = "0.1", features = ["promote"] }      # pure core
skidbladnir = { version = "0.1", features = ["promote-k8s"] }  # + live kube-rs ops

1. Describe your ladder

One file, in Skíðblaðnir's schema. Yours, not Njord's — the crate knows no rung names.

apiVersion: skidbladnir.nordisk/promote.v1
meta:
  customer: Contoso
  owner: ops@contoso.example
slots:
  - name: bench
    namespace: contoso-bench
    hosts: [bench.contoso.example]
    role: development
    guarded: false
    promotes_to: [staging-eu]

  - name: staging-eu
    namespace: contoso-stg
    hosts: [stg.contoso.example]
    role: pre-production
    guarded: false
    promotes_to: [apex]

  - name: apex
    namespace: contoso
    hosts: [contoso.example, www.contoso.example]
    role: production
    guarded: true
    meta: {ticket-queue: SRE}

Four fields are required and there is no default for any of them: name, namespace, role, guarded. That is on purpose — an environment that does not say what it is, is an error, not a test slot.

promotes_to is the direction code travels. Data travels the other way.

Load it:

use skidbladnir::promote::*;

let ladder = Ladder::from_file(Path::new("env/contoso/ladder.yaml"))?;

Ladder::from_yaml(source, text) and from_json(source, text) take a source label that appears in every error message, so a validation failure names the file.

2. Declare a baseline, then layer config over it

Never build a fence out of a config file alone. Declare the guarded slots in code, and let the file layer over them:

let baseline = SlotRegistry::from_slots(vec![
    Slot::new("apex", "contoso", SlotRole::Production, true)
        .host("contoso.example")
        .host("www.contoso.example"),
])?;

let registry = SlotRegistry::load_over(&baseline, ladder, DemotionPolicy::Refuse)?;

The file may add slots and may raise a slot to guarded. It may not lower or delete a guarded one: Refuse fails the load, ReportAndKeepFenced loads with the slot still fenced and the attempt in registry.demotions().

Print them. They also appear in Topology::notes and in every guard refusal:

for d in registry.demotions() { eprintln!("{}", d.line()); }

3. Clear the fence before you touch anything

let guard = Guard::new(&registry)
    .with_override(Override::flagged())      // the --allow-prod flag was passed
    .with_env(EnvSource::Process);           // and PROMOTE_ALLOW_PROD must be "1"

let cleared = guard.assert(&TargetRef::Slot("staging-eu"))?;
if let Some(b) = cleared.banner() { eprintln!("{b}"); }

Picking the wrong variant cannot make the answer wrong — the registry matches across every identity a slot has. Url only looks at the host, and everything printed is credential-stripped.

Clear every target of the operation, not just the first:

let cleared = guard.assert_all(&[
    TargetRef::Namespace(&ns),
    TargetRef::Url(&database_url),
    TargetRef::Url(&object_store_endpoint),
])?;

Clearing only the namespace is how a lab-guarded restore writes to a production database.

Unknown targets

By default an identifier no slot claims is refused. For a genuinely read-only verb, opt out explicitly:

let g = Guard::new(&registry).allow_unknown();   // list / inspect / diff ONLY

The clearance is a capability

Cleared cannot be constructed outside the guard, and every mutating call takes one. It also remembers what it cleared:

cleared.covers("contoso-stg");                // true
cleared.covers("contoso-stg-20260721-1432");  // true — a generation of it
cleared.covers("contoso");                    // false — different slot

4. Ask the ladder before you move

match registry.check_promotion("bench", "apex") {
    Hop::Legal => …,
    Hop::SkipsRungs(rungs) => …,      // refuse, or override and REPORT the rungs
    Hop::WrongDirection => …,         // that is a data copy, not a promotion
    _ => …,
}

Or in one call, fence included:

let (cleared, skipped) = guard.assert_promotion("staging-eu", "apex", allow_skip)?;
if !skipped.is_empty() {
    eprintln!("⚠ skipped {}", skipped.join(", "));   // the override does not silence it
}

Data goes the other way, and upward is not overridable:

let cleared = guard.assert_data_flow("apex", "bench")?;   // ok
let err     = guard.assert_data_flow("bench", "apex");    // always an error

5. Implement AppData — once, in your own repo

Next to the code that knows your schema. The ship never reads your payload.

impl AppData for MyApp {
    fn describe(&self) -> &str { "yvra" }

    fn dump<'a>(&'a self, slot: &'a Slot, staging: &'a Path)
        -> Fut<'a, Result<DumpReport>>
    {
        Box::pin(async move {
            let tables = self.dump_tables(slot, staging).await?;
            let mut r = DumpReport::new()
                .part(CapturedPart::complete("database", Some(tables as u64)));

            r = match self.fetch_objects(slot, staging).await {
                Ok(n)  => r.part(CapturedPart::complete("objects", Some(n))),
                Err(e) => r.part(CapturedPart::skipped("objects", &format!("{e:#}"))),
            };

            Ok(r.watermark(self.read_watermark(slot).await?)
                .app_meta(json!({ "schema_version": 51, "tables": … })))
        })
    }
    …
}

Rules to keep

• Declare every part. A part you never declare cannot be reported missing.
• skipped is the honest form of "degrade quietly". Never return a report that says complete when a leg did not run.
• items: None means unknown, never zero.
• watermark returning Ok(None) means unknown, and the ship treats that as "may discard newer data". Do not invent a value to make a check pass.
• app_meta is your opaque channel. Schema version, table list, object keys — anything that must ride in the manifest. The ship stores and round-trips it and never reads a key out of it.
• neutralise_after_load returns what it changed, so the operator is told.

The five capture states

Anything but Complete must carry a detail.

6. Seal, verify, unseal

let report = app.dump(&slot, &staging).await?;
report.require_complete()?;              // refuse early if you want to

let sealed = seal(&TarZstd::default(), &SealSpec {
    staging: &staging,
    out:     &bundle,
    source_slot: &slot.name,
    version: "20260727",
    taken_at: &now_rfc3339(),
    app: app.describe(),
}, &report)?;

The verdict is written into <bundle>.promote.json, so three days later nobody has to remember what scrolled past.

let v = verify(&TarZstd::default(), &bundle)?;
v.intact();      // do the BYTES verify?
v.is_usable();   // intact AND the capture was whole

Those are different questions. A partial bundle is byte-perfect.

let u = unseal(&TarZstd::default(), &bundle, &staging, LoadPolicy::strict())?;

strict() refuses an incomplete or corrupt bundle. accept_incomplete() / accept_corrupt() are deliberate opt-ins, and u.completeness() still tells you.

7. The watermark gate

let bundle_wm = u.watermark();                          // from the manifest
let target_wm = app.watermark(&target_slot).await?;     // from the live slot

match bundle_wm.map(|b| b.would_discard(target_wm.as_ref())) {
    Some(Discard::Nothing) => { /* proceed */ }
    Some(d) => {
        eprintln!("{}", d.line());
        confirm_or_abort()?;         // includes Discard::Unknown
    }
    None => confirm_or_abort()?,     // the bundle has no watermark either
}

requires_confirmation() is true for everything except Nothing. Including Unknown — the whole point.

Give your watermark a magnitude (row counts, object counts) as well as a mark. A deletion does not move a timestamp, it removes one, so a mark alone cannot see that two hundred rows vanished from the target.

8. Roll an image, with a rollback

let ops = KubeReleaseOps::connect().await?;       // feature promote-k8s
let plan = RolloutPlan::new(ReleaseRef::new(&ns, "web", "web"), "registry/app:v2")
    .wait(Duration::from_secs(240));

for line in plan.describe() { println!("  ↪ {line}"); }   // this IS the dry run

let outcome = rollout(&ops, &cleared, &plan).await?;
println!("{}", outcome.line());
outcome.into_result()?;                                    // only Rolled is success

Do not collapse these into "failed". They are different phone calls.

9. Generations — blue/green

let g = Generation::new("contoso-stg", now_unix)?;   // contoso-stg-20260721-1432

let plan = switch_plan(
    "contoso-stg",
    ops.host_owner("stg.contoso.example").await?.as_deref(),  // who owns it now
    &g.namespace,
    "stg.contoso.example",
    &g.namespace,      // the service name
    3000,
    "stg-tls",
)?;

for s in &plan { println!("  ↪ {}", s.describe()); }          // the dry run
let log = run_generation_plan(&ops, &cleared, &plan).await?;  // the real thing

The order is wake → point → drop, and it is fixed. Switching before waking opens a window where the host resolves to a scaled-down generation.

The old generation is not put to sleep by a switch — that is sleep_plan(ns), a separate decision, so that switching back stays a matter of seconds.

Rolling back is just another switch:

let back = switch_plan("contoso-stg", Some(&new_ns), &previous_ns, host, &previous_ns, 3000, tls)?;

RAISE_LADDER names the phases of bringing a whole new generation up, in order: Rbac → Namespace → Tls → Storage → Secrets → Release → Data → SwitchHost. Walk it; do not reorder it. SwitchHost is the commit point.

10. Helm

promote does not render charts and will not shell out to helm. Render where the chart is owned and hand over the result:

let docs = helm_template_in_your_own_pipeline()?;         // or KubernetesRenderer
ops.apply(&cleared, &ns, &ReleaseSource::Manifests { docs }).await?;

ReleaseSource::Chart(ChartRef { … }) is a declaration — carried, compared and reported, never executed. Passing it to apply returns a refusal naming the fix.

Reading a release's stored values needs no chart and no helm binary:

let values = stored_release_values(client, &ns, "myapp").await?;

That is what helm get values would answer, read out of the sh.helm.release.v1.* Secret — so it runs as a plain Kubernetes Job.

11. Show the ladder in a UI

let mut obs = Observations::new();
obs.insert("staging-eu".into(), Observation::green(&now_rfc3339()).revision("v1.4.2"));
obs.insert("bench".into(),
    Observation::failed("rollout-timeout", "3/5 ready after 240s", &now_rfc3339()));

let topo = registry.topology(&obs);       // serde, both ways

Hand topo to a facett-pipeline adapter. It has nodes, edges, a computed rank for layout, declared roles, guarded, your meta, and a per-rung state. No Skíðblaðnir UI dependency exists or should be added.

Honesty rules the panel must respect, because the engine enforces them:

• a rung with no observation is NeverRun — never render it green;
• Unknown is its own state ("we could not look"), not a failure and not health;
• Failed and Unknown always carry a code — show it;
• topo.notes carries any config that tried to lower the fence. Show that too.

topo.not_green() is your "needs attention" list, and it includes the never-run.

12. Gotchas

A dry run is a plan, not a flag. *_plan() and describe() are pure. If you find yourself adding a Mode::DryRun branch inside something that talks to a cluster, you have re-created the bug this design removed.

Ok(None) from watermark is not "safe". It is "unknown", and unknown blocks.

An empty DumpReport is not success. It is what a dump that never ran looks like.

intact() is not is_usable(). A partial bundle passes a checksum.

A clearance is not transferable. cleared.covers(ns) must be true, or the call refuses.

Two keys means exactly "1". PROMOTE_ALLOW_PROD=true does not arm the override, on purpose.

Never author a rank. It is computed from the edges. Two sources of truth about order is how they disagree.