nordisk/Skidbladnir

tier0 common-infra — design

Class: design · Function axis: f6c (product / ComponentSpec) + f8 (orchestrate waves) + the openbao feature. States only the shape we ship — the app-agnostic platform bring-up (rebirth-ladder Tier 0.5). The why this substrate and the rejected all-Helm approach are in the reasoning notes; the beginner walk-through is tier0-common-infra-for-idiots.md.

What it is

The common-infra ComponentSpec set — the shared cluster substrate every lake app sits on, brought up before any app, in dependency order. Four app-agnostic components, each a [ComponentSpec] DESCRIBED (not linked) exactly like holger/tunnr, so they render to k8s manifests through the ONE k8s renderer and order through the ONE wave engine — no new machinery, just the two shipped limbs composed.

The set (product.rs)

ComponentSpec::ingress_nginx() · cert_manager() · holger_registry() · openbao(), collected by common_infra() -> Vec<ComponentSpec> (deliberately SEPARATE from registry(), the lake app products):

Wave ordering (common_infra_waves)

common_infra_waves() -> Result<Vec<Vec<String>>> maps each spec through as_product() (its after → depends_on) and hands the set to the SHARED crate::orchestrate::start_waves — the ONE Kahn topo sort (sources first; stable within a wave). There is no second ordering to drift (L5). For the shipped set the waves are exactly:

wave 0: [cert-manager, ingress-nginx]   # the two roots (no deps)
wave 1: [holger-registry, openbao]      # each after its wave-0 deps

A caller brings up wave 0 in parallel, waits for readiness, then wave 1. A mis-declared after that never converges is a hard cycle error, not a silent hang.

The renderer half (pure) vs the live back half (deferred)

Everything above is pure and laptop-testable: the waves are computed offline; each spec renders to manifest DATA via renderer_for(Backend::Kubernetes) (a StatefulSet/Deployment + Service + PVC, in the component's own namespace, with the pinned image). The deferred back half is the live cluster apply (k8s_apply.rs, kubernetes feature — server-side apply / scale / rollout-wait).

Production-shaped object sets (infra.rs) — beyond the generic workload

The generic k8s renderer projects a ComponentSpec onto ONE workload (Deployment/StatefulSet + Service + PVC) — right for a lake app, but an infra component needs its whole real object graph (RBAC, an IngressClass, a LoadBalancer Service — all outside the workload projection). So infra::ingress_nginx_objects() renders the complete production set for the ingress controller — 18 objects (9 core + 9 admission-webhook) — reusing ComponentSpec::ingress_nginx() for the identity constants (name/namespace/image/ports — ONE source, L5). The core nine, in apply order:

1. ServiceAccount ingress-nginx — the identity the controller runs as.

2. ConfigMap ingress-nginx-controller — controller tunables (the Deployment's --configmap).

3. ClusterRole ingress-nginx — cluster-wide watch of Ingresses/IngressClasses/EndpointSlices + ingresses/status update.

4. ClusterRoleBinding — SA → ClusterRole.

5. Role ingress-nginx — namespace leader-election (coordination.k8s.io leases) + local config.

6. RoleBinding — SA → Role.

7. Deployment ingress-nginx-controller — runs as the SA, wired to its ConfigMap + IngressClass

   • election id, ports 80/443/8443(webhook).

8. Service ingress-nginx-controller — a LoadBalancer publishing 80/443.

9. IngressClass nginx — controller k8s.io/ingress-nginx.

RBAC is rendered coherent: both bindings reference their role AND the one ServiceAccount the Deployment runs as, so the grants actually reach the pod (asserted).

The admission webhook (9 more objects) makes ingress-nginx fully production-shaped — the apiserver validates every Ingress on CREATE/UPDATE before it is admitted: a ValidatingWebhookConfiguration (validate.nginx.ingress.kubernetes.io, clientConfig → the admission Service), an admission Service (ingress-nginx-controller-admission, routing to the controller's webhook port), a dedicated admission ServiceAccount + ClusterRole/Binding (update the webhook config's caBundle) + Role/Binding (create the serving-cert Secret), and the two upstream kube-webhook-certgen Jobs — a create Job (mint the self-signed serving cert) + a patch Job (inject its CA into the webhook config's caBundle), both running AS the admission SA. This self-provisions the serving cert via the Jobs rather than coupling to cert-manager, so ingress-nginx stays a wave-0 root (no new dependency) and matches the shipped ingress-nginx manifests. The webhook clientConfig→Service link and the cert-Job→SA→RBAC coherence are asserted.

cert-manager — infra::cert_manager_objects() (the symmetric TLS-issuer graph)

cert-manager is THREE components, so its production set is deeper — 28 objects rendered from ComponentSpec::cert_manager() (the webhook/cainjector images are the controller image with the component swapped, so the version pin lives in ONE place, L5):

• controller — ServiceAccount + ClusterRole (issuers/certificates/orders/challenges/secrets + */status) + ClusterRoleBinding + a kube-system leader-election Role/RoleBinding (leases) + Deployment (runs as the SA) + a metrics Service (9402).
• webhook — ServiceAccount + ClusterRole (subjectaccessreviews) + ClusterRoleBinding + a dynamic-serving Role/RoleBinding (its serving-cert Secret) + Deployment + a 443 Service + the ValidatingWebhookConfiguration + MutatingWebhookConfiguration (clientConfig → the webhook Service, guarding the cert-manager.io/acme.cert-manager.io groups).
• cainjector — ServiceAccount + ClusterRole (webhookconfigs + certificates/secrets) + ClusterRoleBinding + a leader-election Role/RoleBinding + Deployment.
• six CRD registrations — Certificate/CertificateRequest/Issuer/ClusterIssuer ( cert-manager.io) + Order/Challenge (acme.cert-manager.io), each a real named CustomResourceDefinition with group/scope/names.

CRD schemas are DEFERRED (the flagged CRD-schema-heavy leg): each CRD's v1 uses openAPIV3Schema: {type: object, x-kubernetes-preserve-unknown-fields: true} + an explicit # SCHEMA DEFERRED marker — an "accept anything" schema, NOT the hand-authored OpenAPI validation (a separate slice). Honest: the registration is applyable; strict validation is not modelled.

Per-component RBAC is asserted coherent (each ClusterRoleBinding binds its ClusterRole to the SA its Deployment runs as). Live apply of either set is the deferred back half.

OpenBao structural-populate hook (openbao.rs, openbao feature)

Once the openbao component's wave is up, its structure is populated (never secret values — those are tillsynia-admin's job). This is expressed as a pure, ordered plan:

• PopulateStep — EnableKubernetesAuth · ConfigureKubernetes{host} · Policy{name,hcl} · KubernetesRole{role,sa,ns,policies,ttl}.
• PopulatePlan::env_steps(env, sa, ns) — the per-env pair (a scoped read policy + a 1h-TTL k8s-auth role, both named for the env). The ONE per-env definition; OpenBao::provision_env now delegates to it (L5, behaviour unchanged).
• PopulatePlan::for_platform(host, &[(env,sa,ns)]) — the full structural populate: enable + configure the k8s auth method, then env_steps per environment.
• OpenBao::apply(&plan) — the thin, idempotent live executor (the deferred back half; needs a reachable OpenBao).

The plan is DATA, so a bring-up asserts the exact structure it will create — ordering, per-env scoping (one env's policy never grants another's path), and step count — entirely offline.

Tests (RED-when-broken)

• product.rs::common_infra_waves_are_dependency_ordered — pins the exact 2-wave shape + the flat order's dep edges (drop/mis-name an after → the shape changes → fail).
• product.rs::common_infra_specs_carry_their_infra_shape — image/namespace/stateful/PVC/deps per component.
• product.rs::openbao_infra_spec_renders_a_k8s_statefulset_service_pvc — the pure renderer half: a StatefulSet (not Deployment) + Service + PVC, namespace openbao, the pinned image, PVC 10Gi.
• openbao.rs::populate_plan_for_platform_is_ordered_and_env_scoped (feature openbao) — ordered setup-then-per-env steps, per-env HCL scoping isolation, exact count.
• openbao.rs::env_steps_is_the_single_per_env_definition (feature openbao) — the platform plan's per-env tail IS env_steps (one definition, L5).
• tests/testmatrix_rows.rs::common_infra_bringup_is_wave_ordered_and_renders — green tier0-common-infra matrix row (waves + OpenBao StatefulSet render).
• infra.rs::ingress_nginx_renders_the_full_production_object_set — all nine kinds present + the exact count (a missing object fails); …_rbac_binds_the_service_account_to_its_roles — both bindings reference the role AND the SA the Deployment runs as; …_rbac_rules_cover_the_controller_essentials — Ingress/IngressClass/EndpointSlice + status rules + leader-election leases; …_service_ingressclass_and_image_are_production_wired — LoadBalancer 80/443, IngressClass nginx, the ConfigMap ref, and the image sourced from the spec; …_admission_webhook_points_at_admission_service_and_wires_cert — the ValidatingWebhookConfiguration clientConfig → the admission Service, the Service → the controller webhook port, and the create/patch Jobs run as the admission SA whose RBAC creates the Secret + updates the webhook config (18-object count pinned). Green tier0-ingress-nginx matrix row.
• infra.rs::cert_manager_renders_all_three_components_full_graph — all three components' objects + both webhook configs + six CRDs + the exact 28-object count; …_rbac_binds_each_sa_to_its_deployment — per-component CRB→ClusterRole + Deployment serviceAccountName + leader-election leases; …_crds_are_registered_with_schema_deferred — every CRD carries preserve-unknown-fields + the SCHEMA DEFERRED marker (a silently hand-authored schema fails); …_webhook_wiring_and_image_pin — both admission configs point at the webhook Service and the webhook/cainjector images are the version-pinned swaps. Green tier0-cert-manager matrix row.