nordisk/edda

Building your own Linux kernel — for beginners

Class: idiot · Zero-knowledge walkthrough of the kernel build kind. The commands live in kernel-guide.md.

The one-sentence version

You write down which parts of Linux you want, and a container downloads the official Linux source code and builds exactly that kernel for you.

What is a kernel, and what is a bzImage?

The kernel is the part of Linux that talks to the hardware. It is the first program a machine runs. Everything else — your shell, your web server — asks the kernel to do things.

A bzImage is that kernel as one file. Roughly 13 MB for a normal one. When a machine boots, this file is what gets loaded.

What is a "kconfig symbol"?

Linux is built from thousands of optional parts. Each part has a switch. The switch has a name that starts with CONFIG_. That name is a kconfig symbol.

Three examples:

• CONFIG_TUN=y — "build the virtual-network-card driver into the kernel."
• CONFIG_SOUND=n — "leave sound out."
• # CONFIG_DRM is not set — the same thing as =n, just the other spelling Linux uses when it writes the file itself.

y means yes, inside the kernel file. n means no, not at all. There is a third answer, m, meaning "build it as a separate file to load later" — a module. Remember that word; it has a sting in it further down.

A file full of these lines is a config. Linux 6.12.104 has 17 704 of these switches. Nobody sets them by hand.

The catch: switches depend on other switches

CONFIG_VIRTIO_BLK (a virtual hard disk) needs CONFIG_BLOCK (the disk layer) and CONFIG_PCI (the bus the disk hangs off). If you ask for the disk but not the bus, Linux does not argue with you — it quietly turns your switch back off. That silence is the whole problem this system is built around.

"Make it smaller" is three different knives — but only two of them cut here

People say "trim the kernel" as if it were one job. It is three jobs, at three different depths. None of them can do another one's work.

Knives 1 and 2 are built and measured. Knife 3 does not exist on our machines, and that was found out by trying it, not by guessing:

• The switch that does it, CONFIG_LD_DEAD_CODE_DATA_ELIMINATION, is only offered on arm, mips, riscv and powerpc chips. Our machines are x86_64. Asking for it now gets a clear refusal instead of doing nothing quietly — a setting that silently does nothing is worse, because the receipt makes it look like it worked.
• The other candidate, CONFIG_TRIM_UNUSED_KSYMS, only exists in kernels that allow modules. Our small kernels forbid modules. So it cannot be there either.
• The nearest substitute is a different compiler, clang, with a feature called LTO. It works, it boots — and it made the small kernel bigger: 2 016 256 bytes against gcc's 1 811 456. That is 204 800 bytes more, 11.3 %. LTO is a speed tool, not a size tool.

So: two knives, not three.

The four presets

A preset is a saved answer. You pick one word.

(All on Linux 6.12.104, on one machine, with the source already downloaded. slim is 7.6 times smaller than normal. The solver takes another 23.3 % off slim.)

The 20 480 bytes that decide everything

Look at full and normal. They are almost the same size — 20 480 bytes apart — and that tiny difference is the entire point of this system.

full is the plain generic kernel. It boots. But on a generic kernel, CONFIG_TUN (the virtual network card our appliances send their encrypted traffic through) is built as a module — a separate file, loaded later. Our appliances have no separate files. So a full kernel starts up, works, and has no network tunnel at all.

normal is full plus a short list of switches that were each written down after something broke. It costs 20 480 bytes and buys a machine that can actually do its job. That is what those 20 kilobytes are.

We did not read this off a config file. We booted the thing and asked it. See below.

Why a smaller kernel is not automatically a working kernel

After the build, a check called the canary runs. It reads your config back and asks, line by line: did every switch I asked for actually stay on? If one was quietly turned off, the build stops and prints its name.

The canary is good. It also has a hole you must understand:

The canary can only check what you WROTE DOWN.

If you never mentioned a switch, the canary has nothing to compare against, so it says nothing about it.

Here is the trap, and it is a real one. CONFIG_BINFMT_ELF is the switch that lets Linux run a program at all. Not a driver — the ability to start software. tinyconfig leaves it off. Nobody thinks to write it down, because nobody thinks of "running programs" as a feature.

A kernel without it:

• builds fine,
• passes every canary line,
• boots,
• and then cannot start a single program. It is not a smaller kernel. It is a brick.

"But surely the solver catches that?" — no. This was measured.

This is the most important thing on this page.

The solver (knife 2) reads Linux's own rules and works out which parent switches must be on. It is genuinely clever. So the obvious idea is: throw away the hand-written list of parent switches and let the solver find them all.

That was tried. The hand-written list had 16 lines. The solver found 13 of them by itself. It missed three: CONFIG_BINFMT_ELF, CONFIG_MULTIUSER and CONFIG_UNIX.

Those are exactly the three the canary cannot catch — and for exactly the same reason. Nothing in the request mentions them. The solver answers "what must be on for what you asked for to work". It cannot answer "what did you forget to ask for".

That kernel was 1 369 088 bytes, passed every check, and could not run a single program. The boot test refused it: did NOT reach PID 1.

A solver is a knife, not an oracle. It makes what you declared as small as it can be. It does not discover what you failed to declare.

The only thing that catches this is actually booting it. That is the probe: the built kernel is started inside a virtual machine (QEMU), and a tiny program written in C is handed to it as the very first thing to run. If that program prints its greeting on the serial line, then the kernel booted and it can run software. Those are two separate facts, and the probe proves both.

The probe also names each device it looks for as present or ABSENT. It never just prints what it found — a report that only lists successes reads identically on a kernel that has nothing. That is how full's missing network tunnel was caught: the probe asked the running kernel, and it answered tun ABSENT.

One complete worked example

Goal: build the smallest kernel that still boots, and prove it boots.

1. Write the declaration

In your repo's .nornir/build-variants.toml:

[[build.kernel]]
name = "my-first-kernel"
version = "6.12.104"
tarball_sha256 = "949d166a…"          # the fingerprint of the official source file
preset = "slim"
toolchain_image = ""                   # empty name + the id below = "the image on this box"
toolchain_image_sha256 = "388a7816…"   # the toolchain container's id
toolchain_packages = []                # it already has everything → no internet needed

The tarball_sha256 is not optional. It is the fingerprint of the source download. Without it you would be compiling whatever the network happened to hand you. Leave it out and the build refuses, by name.

2. It checks the disk before it downloads anything

It wants 6 GiB free. A slim build's working directory peaks at 1.9 GB, plus a 148 MB source file. If the disk is too small it says so now, not after 80 seconds of compiling.

3. It fetches the source — once

The 148 MB file is saved in a shared cache, under a filename that is its own fingerprint. Build three different presets of the same version and it downloads once. A stale cache entry cannot exist: if the bytes do not match the name, it is not a hit, it is garbage, and it gets fetched again.

4. It compiles, inside a box with walls

The compile happens in a container limited to 8 CPU cores and 8 GiB of memory. This is not decoration. On 2026-08-20 an unlimited build reached 33 GB and took the whole machine down. mem_limit_mb = 0 is refused: "unlimited" is not a limit.

5. The canary speaks — and this is the good part

The first time anyone ran slim, it stopped after 18 seconds with twelve lines:

KERNEL-CONFIG-REFUSED: CONFIG_EFI_STUB=y did not survive olddefconfig
KERNEL-CONFIG-REFUSED: CONFIG_TUN=y did not survive olddefconfig
…
KERNEL-CONFIG-REFUSED-TOTAL: 12 option(s) did not hold

That is not a failure. That is the answer to a question. Each line means "you asked for this, and its parent switch is off, so it was dropped." The fix is to turn the parents on: CONFIG_PCI, CONFIG_BLOCK, CONFIG_NET, and so on.

Those twelve answers became a permanent file, deps-tinyconfig.config. Every line in it was written by a refusal, not by a human reading documentation. You will not have to repeat this for slim or nuked — it is done. You will see it again the day you ask for something new.

6. The build finishes

Three files land in .kernel-out/my-first-kernel/:

• bzImage — the kernel, 1 811 456 bytes;
• kernel.config — every switch, as it ended up;
• my-first-kernel.kernel-pin.json — the receipt: the fingerprint of the source, of the config, of the kernel, which preset it was, and which container built it.

7. Boot it — the step that catches the brick

Point the probe at what you just built and run it. It starts QEMU inside the same container and reads the serial line. A good run says:

NORDISK-PROBE-PID1-ALIVE
NORDISK-PROBE-DEV console present
NORDISK-PROBE-DEV tun present
NORDISK-PROBE-DEV virtio-blk present
NORDISK-PROBE-DEVICES

Measured: slim reaches this in 2.1 seconds, nuked in 2.1, normal in 4.2. full boots too — and says tun ABSENT, which is the whole lesson of this page in one line.

Now you know your kernel works. Not because it compiled — because it ran.

8. Optional: turn the solver on

Add one line and rebuild:

solver = "klocalizer"

The container reads Linux's own rules, works out the smallest set of switches that still satisfies what you asked for, and hands that to the build. slim drops from 1 811 456 to 1 389 568 bytes. Two runs produced byte-identical kernels.

The first run takes longer — 148 s instead of 80 — because it must first turn Linux's rules into a 19 949 986-byte puzzle for the solver. That puzzle is saved in the same cache the source file lives in, so every later build on the same Linux version gets it free (103 s).

It also writes a file, deps-6.12.104.config, listing every parent switch it had to turn on. That is a list a human can read and commit — the same list that was once closed by hand, done automatically.

Then boot it. Always boot it. See the measured brick above for why.

A note on "closed" and "open" worlds

One detail decides whether the solver helps or hurts. Left to itself, the solver treats every switch you did not mention as free to turn on, and it happily turns on network-card and filesystem drivers nobody asked for — 472 extra switches in the measured run. Written the right way, with every unmentioned switch explicitly turned off first, the same solve adds 78.

A naively-wired minifier does not make your kernel smaller. It makes it bigger. Ours writes the closed version; you do not have to do anything. But if you ever read that the solver "added mlx5", this is why.

Building with the internet unplugged

A kernel build needs three things off the network: the source tarball, the solver formula (only if you turned the solver on), and the toolchain — the box with the compiler in it. The first two were already solved: they live in a cache, named after their own checksum, and you can simply copy the files onto the machine.

The toolchain was the one that was not. It is a container image, and an image that is not on the machine leaves podman with exactly one move: go and fetch it. On a machine with no internet, that is where the build stops.

So the toolchain now travels the same way the tarball does — as a file.

On a machine that HAS the toolchain:

ID=$(podman image inspect --format '{{.Id}}' nordisk-kernel-toolchain)
mkdir -p ~/.cache/nordisk/kernel/images
podman save --format docker-archive -o ~/.cache/nordisk/kernel/images/$ID.tar \
    nordisk-kernel-toolchain

Copy that one file to the offline machine, into the same place (<cache>/images/<ID>.tar), and put those same 64 characters in the declaration:

toolchain_image_sha256 = "<ID>"

That is the whole procedure. The build then loads the image out of the file and compiles with no network at all.

The trap that cost this a day: two different 64-character numbers

Podman will tell you two long hex numbers about an image and they are not interchangeable:

Measured on oden: save an image, load it on another machine, and the ID comes back the same while the digest comes back different — podman rebuilds the manifest when it loads, so the digest describes the copy, not the original. Pin the ID. If you pin the digest, the offline machine will tell you, by name, that the archive holds a different image than the one you asked for — and it will refuse rather than quietly use it.

What it says when something is missing

There is no quiet fallback here, on purpose. If the toolchain is not on the machine and no archive is either, the build stops with

NORDISK-TOOLCHAIN-MISSING: kernel 'x': the toolchain image `…` is not in this
box's image store, and no staged archive is either.

and then tells you the exact path to put the file at and the exact command that makes it. A build that quietly downloaded a toolchain instead would look identical to a working offline build, which is the same as having no offline build at all.

You can click all this

There is a Kernel tab in the dwarves app with a slowly rotating 3D picture of the switches each preset turns on and off. It is a real picture of the committed files, not a decoration — though the lines between switches are drawn from their names, not from Linux's real rule graph, and the tab says so on itself.

Careful

• A green build is not a working kernel. Only the probe says that.
• A KERNEL-CONFIG-REFUSED list is a to-do list, not a stack trace. Read it as "here are the parents you still owe."
• The solver does not replace the boot test. It cannot see what nobody wrote down — measured, three switches, one brick.
• Never remove tarball_sha256 to "make it work". It is the difference between building Linux and building whatever arrived over the wire.
• Pin the image ID, never the manifest digest. They are both 64 hex characters and only one of them means the same image on two machines.
• A log can lie. The probe once reported a successful boot for a kernel that was never started: the compiler had failed, and its error message quoted the source line containing the greeting text. The probe now checks that the step actually succeeded before it reads a single line of output. Take the lesson with you.

More

kernel-guide.md has every field, every refusal, and the commands. The reasoning lives in workspace_nordisk/.nornir/kernel-trim-design.md.