mikronova/tunnr-etcd

Rusternetes on tunnr: can tunnr be the installer for a Rusternetes node?

Findings from reading the tunnr source (tree at commit e364198, 2026-10-19) and the public Rusternetes README. This brief is written for an outside reviewer who has not seen either codebase.

1. Goal

We want an installer that boots from an ISO (or a cloud image), lays a Kubernetes node onto bare metal or a VM unattended, and comes up as a working Rusternetes cluster member. The question is whether our own appliance framework, tunnr, is the right base.

2. What Rusternetes is

• Kubernetes rewritten from scratch in Rust, Apache-2.0 (github.com/calfonso/rusternetes). It is about 216k lines and has 3,100+ tests.
• It claims 415 of 441 Kubernetes v1.35 conformance tests passing (94.1%, via Sonobuoy). Outside commenters dispute the numbers, so they need checking.
• It has 10 crates: api-server (Axum, TLS/mTLS, RBAC, webhooks), controller-manager (31 controllers), scheduler, kubelet, kube-proxy, storage, common, kubectl, cloud-providers, and rusternetes.
• All-in-one binary: rusternetes --data-dir ./cluster.db runs every component as a tokio task in one process, using embedded SQLite. Redis is an option, and etcd (or "Rhino", an etcd-compatible shim over SQL) is used for multi-node setups.
• The kubelet does not use CRI. It talks to the Docker Engine API through bollard, so it needs a Docker or Podman daemon. On Linux, Podman must be rootful because kube-proxy needs it.
• kube-proxy works in iptables mode only, with host networking.
• Pod networking: each pod gets a pause container that holds its network namespace. The repo has a cni-config directory, but the README does not say which CNI plugin is used.
• DNS and certificates: shell scripts (generate-certs.sh, bootstrap-cluster.sh) set up the certificates and CoreDNS. Today's install path is Docker or Podman Compose, or the all-in-one binary.

3. What tunnr is

tunnr is our framework for minimal, distroless Linux appliances. Its rules:

• No shell, no package manager, and no general-purpose userspace.
• One static Rust binary runs as PID 1.
• The kernel is the only non-Rust part. It is built from a pinned source with a checksum.
• Everything is proven by booting under QEMU + OVMF + KVM and checking markers on the serial console.

3.1 The installer: crates/korp-installer (~49k lines, pure Rust)

This is the piece that matters for us.

• korp-installer binary (PID 1 on the install medium). It boots, detects disks, UEFI and the GPU from sysfs, and writes a GPT by hand (no sgdisk or parted). The layout is:

  • P1: an ESP (FAT32, written by a pure-Rust builder).
  • P2: the rootfs, which holds the app as a raw blake3-chunked .znippy archive that is never extracted.
  • P3: optional korp-data at the end of the disk, formatted FAT32 or XFS. The XFS formatter is our own pure-Rust korp-mkfs-xfs.

It also handles multi-disk layouts (layout.rs, with rules for which disks become /data), mdraid RAID1, and UEFI plus legacy BIOS boot (korp-boot-bios). After writing, it re-reads the payload from the target disk and checks it against the hash pin and the ed25519 signature, and refuses the install on a mismatch. It narrates every step over serial (INSTALLER: …, INSTALL-OK).

• korp-start binary (PID 1 on the installed disk, the root of trust). On every boot it:

  1. Prints a banner.
  2. Runs a three-part self-test: the archive's blake3 root, the ed25519 signature over that root, and a blake3 check of every chunk.
  3. Mounts /data.
  4. memfd_create + fexecves the verified static app binary straight from RAM.

Because exec replaces the process, the app becomes PID 1. The app's argv comes from korp.args= on the kernel cmdline or from the kernel -- tail.

• Configuration through korp.* kernel cmdline tokens (cmdline.rs): korp.appliance=lean|fat, korp.entry=, korp.znippy=, korp.require_signed=, korp.require_sealed=, korp.seal_pcrs= (TPM-sealed WireGuard identity), korp.data.fs=fat|xfs, korp.log_udp=, and others.

• Also present: Secure Boot signing and customer-key enrollment (secureboot.rs), TPM sealing (via nornir-tpm), and a live-update and rollback path (live.rs, rollback.rs).

3.2 Other parts of tunnr

• tunnr-vm: a pure-Rust harness that builds the boot artifact (kernel + newc initramfs) and boots it under QEMU/OVMF/KVM.
• tunnr-demo: one appliance image boots as either server or client. The only difference is a ~2 MiB NoCloud (CIDATA) seed volume that carries the hostname, instance-id and WireGuard config. This is a natural fit for "the same image becomes a control-plane or a worker node."
• payloads/wg-appliance: a WireGuard-first PID-1, with a capability-dropping sandbox and a seccomp filter.
• tunnr-build / tunnr-build-thing: the build recipe expressed as data, including the kernel.

3.3 Kernel build (in our sibling repo edda, crate nornir-build-thing)

• There are four presets:

Every preset gets a hardened (KSPP-style) profile on top. tunnr's appliance kernel uses nuked.

• kernel_requires.rs: each feature can declare the kernel options it needs (KconfigReq: symbol, value, reason). The build merges them after the preset and refuses by name when a preset contradicts one. It also writes a kernel.why.toml file that says why each option is set.

4. What "container support" in tunnr actually is

I looked for it specifically. tunnr has container images, but no container runtime.

5. How well they fit

6. Options

A. Pure tunnr, with a Rust runtime built in (best fit with tunnr's rules)

1. Ship the Rusternetes all-in-one binary (SQLite in /data) as the korp-start payload. Run it as the child of a small PID-1 wrapper that does the reaping and mounts, rather than as PID 1 itself.
2. Replace the kubelet's bollard/Docker client with an in-process OCI runtime built on youki's libcontainer crate, and wire it to korp.container_backend=youki. Pulling images needs a registry client (for example the oci-client crate) and an overlayfs snapshotter.
3. Add a k8s kernel variant with the needed options declared.
4. Networking: a minimal built-in bridge + veth setup (no external CNI binaries). kube-proxy programs nftables over netlink (for example with the rustables crate) instead of shelling out to iptables.
5. The NoCloud seed carries the node role, cluster CA and join token.

Cost: big. It means changing the Rusternetes kubelet, its networking and kube-proxy, effectively a fork.

B. tunnr installer + a Buildroot rootfs with Podman (fast, but breaks the rules) Keep korp-installer for the disk layout and trust chain. The P2 payload becomes a Buildroot rootfs with rootful Podman, iptables and CNI plugins, and Rusternetes runs unchanged against the Podman socket. tunnr already has a Buildroot road (variants/buildroot). Cost: small to medium. The price is breaking tunnr's "no general userspace" rule and losing the "one verified binary" property.

C. Hybrid Start with B to prove Rusternetes works on our installed disk and to check the conformance claims on our own hardware. Then move piece by piece to A (runtime first, then networking).

7. Risks

• Conformance is self-reported (94%), and the numbers are disputed. Run Sonobuoy ourselves before relying on it.
• Rusternetes is young and maintained by one person. A runtime change in its kubelet means owning a fork.
• Becoming PID 1: korp-start's fexecve makes the app PID 1. Either Rusternetes gets PID-1 duties, or korp-start gains a supervise mode.
• The nuked kernel is a security feature. A k8s kernel is necessarily much larger (namespaces, netfilter, overlayfs), so the hardened profile's refusals have to be re-measured for it.
• Multi-node clusters need etcd (or Rhino plus a database). The all-in-one SQLite mode is single-node only.

8. Questions for the reviewer

1. Is A (Rust runtime inside, no daemons) worth the fork cost, or is C the sane path?
2. Is youki's libcontainer mature enough to embed in a kubelet as a library rather than as a CLI?
3. Pod networking without CNI binaries: is an in-process bridge + veth + nftables (over netlink) realistic for a conformance-passing node?
4. Should the app run as PID 1 directly, or should there always be a supervising PID-1 in front of a complex app like this?
5. For multi-node clusters: a tiny etcd-compatible store inside the appliance, or one external store?