crates/nornir-build-thing/src/image_writer.rs
//! **Cloud MEDIUM writers** — turn an already-installed RAW disk image into the
//! import format each cloud actually accepts, in **pure Rust**, with ZERO
//! external process. No `qemu-img`, no `tar(1)`, no `gzip(1)`, no `dd`: every
//! byte below is emitted from Rust + `std` (with `tar`/`flate2`/`sha2` as
//! LIBRARIES, never as subprocesses), the same rule [`crate::iso_writer`]
//! follows for ISO-9660. There is no `Command::new` in this file.
//!
//! # Why this module exists
//!
//! [`crate::target::presets`] has declared `MediumBring::{ImportRaw,
//! ImportQcow2, ImportVhd, Several}` for months and **nothing built any of
//! them**. `iso_writer` masters an ISO, and an ISO was the ONLY medium this
//! constellation could produce — so `azure-gen1`, `azure-gen2`, `aws-nitro`,
//! `aws-xen-hvm`, `gce-virtio`, `gce-gvnic`, `ovh-public-cloud` and
//! `scaleway-instance` could not be tested anywhere, *not even wrongly*. A
//! preset whose medium has no writer is a row that can only ever be read.
//!
//! # One install, four media
//!
//! The input is the disk the installer already wrote — `korp-installer`'s
//! `tests/kvm_install.rs` boots the installer ISO under QEMU+OVMF with a blank
//! target disk attached and reads that disk back afterwards; **that installed
//! disk is the cloud image**. Nothing here installs anything: each writer is a
//! pure `raw disk → import artifact` transform.
//!
//! | [`CloudMedium`] | who takes it | what comes out |
//! |---|---|---|
//! | [`Raw`](CloudMedium::Raw) | AWS `import-snapshot` → `register-image` | the same bytes, **validated and named** |
//! | [`Vhd`](CloudMedium::Vhd) | Azure (Gen1 + Gen2) | fixed-size VHD: data + a 512-byte `conectix` footer |
//! | [`GceTarGz`](CloudMedium::GceTarGz) | GCE `images create` | a gzipped tar whose single member is named exactly `disk.raw` |
//! | [`Qcow2`](CloudMedium::Qcow2) | OVH / Scaleway / OpenStack glance | qcow2 **v3**, uncompressed, unbacked, sparse |
//!
//! # The two rules that cost money to learn late
//!
//! * **Azure refuses a VHD whose virtual size is not a whole number of MiB** —
//! and it refuses it *after* the upload, which is the most expensive place to
//! find out. [`write_vhd`] rounds the virtual size up to the next MiB, writes
//! the zero padding, and RECORDS both numbers
//! ([`MediumProduct::virtual_size`], [`MediumProduct::padded`]).
//! [`VhdFooter::parse`] refuses a footer that breaks the rule, so it is
//! checked on the way back in too.
//! * **The file inside a GCE tarball must be named exactly `disk.raw`.** That
//! single name is the whole contract — [`GCE_MEMBER_NAME`] is the one place it
//! is spelled, and [`tar_gz_member_names`] reads it back off the produced
//! artifact rather than trusting the writer.
//!
//! # Where this sits beside the ISO
//!
//! [`MediumBoot::Gpt`](crate::installer::MediumBoot::Gpt) is the ISO's answer to
//! the same customer: an ISO that ALSO carries a GPT ESP in its reserved system
//! area, so a UEFI-only cloud can import the identical artifact as a disk. That
//! covers the case where the medium the cloud imports *is* the installer. This
//! module covers the other one — the disk the installer already produced, in the
//! container format the cloud's importer demands. The two are complements, not
//! rivals, and both must exist for the roster's rows to be reachable.
//!
//! # Determinism
//!
//! Like `iso_writer`, everything here is content-deterministic: a fixed default
//! timestamp ([`VHD_TIMESTAMP_DEFAULT`]), fixed tar metadata (mode 0644, uid/gid
//! 0, mtime 0), and a VHD unique id DERIVED from the SHA-256 of the padded image
//! rather than randomly minted — so the same installed disk converts
//! byte-identically every time. That is what lets a content-addressed build
//! attest a medium at all.
//!
//! # Proof
//!
//! The point of a medium writer is that its product **boots**. Every writer here
//! is proven by booting what it produced back under QEMU — `format=vpc` for the
//! VHD, `raw` for raw, the tarball unpacked to `disk.raw`, `qcow2` for qcow2 —
//! in `crates/cloud-media-bootproof/tests/image_writer_boot.rs`. QEMU is a hypervisor, not a build tool, and
//! it is the ONE sanctioned external in the whole lane; it appears only in an
//! `#[ignore]`d test, never on the writing path. The byte-level structure, the
//! named refusals, and a full pure-Rust qcow2 read-back live in the cheap,
//! always-running `tests/image_writer.rs`.
use std::fs::File;
use std::io::{BufReader, BufWriter, Read, Seek, SeekFrom, Write};
use std::path::{Path, PathBuf};
use anyhow::{bail, Context, Result};
use sha2::{Digest, Sha256};
use crate::target::presets::{MediumBring, TargetPreset, PRESETS};
// ─────────────────────────────────────────────────────────────────────────────
// Shared vocabulary
// ─────────────────────────────────────────────────────────────────────────────
/// The 512-byte logical sector every disk image here is measured in.
pub const SECTOR: u64 = 512;
/// One mebibyte — Azure's virtual-size quantum.
pub const MIB: u64 = 1024 * 1024;
/// One gibibyte — GCE's *recommended* image quantum (reported, never enforced;
/// see [`MediumProduct::gib_aligned`]).
pub const GIB: u64 = 1024 * MIB;
/// Streaming buffer for the copy/hash passes (1 MiB).
const COPY_CHUNK: usize = 1024 * 1024;
/// The name the single file inside a GCE tarball MUST have. `gcloud compute
/// images create --source-uri` looks for this and nothing else; a tarball whose
/// member is called `image.raw`, `./disk.raw` or `disk.raw.001` is rejected with
/// a message that never mentions the name. One spelling, one place.
pub const GCE_MEMBER_NAME: &str = "disk.raw";
// ── the named refusals ───────────────────────────────────────────────────────
// A red probe asserts on THESE constants, never on a prose sentence a later
// edit can reword out from under it.
/// A source that is not a regular file, is empty, or is not a whole number of
/// 512-byte sectors.
pub const ERR_SOURCE: &str = "not a usable raw disk image";
/// The footer's first 8 bytes are not `conectix`.
pub const ERR_VHD_COOKIE: &str = "VHD footer cookie is not `conectix`";
/// The footer's stored checksum does not equal the recomputed one.
pub const ERR_VHD_CHECKSUM: &str = "VHD footer checksum does not verify";
/// The virtual size is not a whole number of MiB — Azure refuses this AFTER the
/// upload, so it is refused here instead.
pub const ERR_VHD_NOT_MIB: &str = "VHD virtual size is not a whole number of MiB";
/// The file is shorter than the 512-byte footer it must end with.
pub const ERR_VHD_SHORT: &str = "shorter than the 512-byte VHD footer";
/// The footer says something other than disk type 2 (fixed).
pub const ERR_VHD_NOT_FIXED: &str = "VHD is not disk type 2 (fixed)";
/// The tarball's single member is not named [`GCE_MEMBER_NAME`].
pub const ERR_GCE_MEMBER: &str = "GCE tarball member is not named `disk.raw`";
/// The first four bytes are not the qcow2 magic.
pub const ERR_QCOW2_MAGIC: &str = "not a qcow2 image (bad magic)";
/// The header version is not 3.
pub const ERR_QCOW2_VERSION: &str = "qcow2 version is not 3";
/// The image declares an incompatible feature bit this reader does not
/// implement (compression, external data file, extended L2).
pub const ERR_QCOW2_FEATURES: &str = "qcow2 declares incompatible features";
/// The four import media a cloud takes an already-installed disk in.
///
/// This is the WRITER-side vocabulary. It is joined to the roster's declared
/// [`MediumBring`] by [`CloudMedium::for_bring`] — the whole reason the enum
/// exists is that a preset row saying `ImportVhd` must be able to reach a thing
/// that actually writes a VHD.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum CloudMedium {
/// AWS `import-snapshot` → `register-image`. The installed bytes, unchanged.
Raw,
/// Azure: a **fixed-size** VHD (data + 512-byte `conectix` footer).
Vhd,
/// GCE: `disk.raw` inside a gzipped tar.
GceTarGz,
/// OVH / Scaleway / OpenStack glance: qcow2 v3, uncompressed, unbacked.
Qcow2,
}
impl CloudMedium {
/// Every medium, in table order — so a matrix test cannot forget one.
pub const ALL: [CloudMedium; 4] =
[CloudMedium::Raw, CloudMedium::Vhd, CloudMedium::GceTarGz, CloudMedium::Qcow2];
/// The artifact's file extension (no leading dot).
pub fn file_extension(self) -> &'static str {
match self {
CloudMedium::Raw => "raw",
CloudMedium::Vhd => "vhd",
CloudMedium::GceTarGz => "tar.gz",
CloudMedium::Qcow2 => "qcow2",
}
}
/// The stable word a receipt / CLI flag spells this medium with.
pub fn as_str(self) -> &'static str {
match self {
CloudMedium::Raw => "raw",
CloudMedium::Vhd => "vhd",
CloudMedium::GceTarGz => "tar.gz",
CloudMedium::Qcow2 => "qcow2",
}
}
/// The `-drive format=` word QEMU boots this artifact with.
///
/// [`GceTarGz`](Self::GceTarGz) says `raw` because a tarball is not a disk:
/// the proof unpacks it to `disk.raw` first and boots THAT — which is
/// exactly what GCE does with it too.
pub fn qemu_format(self) -> &'static str {
match self {
CloudMedium::Raw | CloudMedium::GceTarGz => "raw",
CloudMedium::Vhd => "vpc",
CloudMedium::Qcow2 => "qcow2",
}
}
/// Whether the artifact is itself a block image QEMU (or a cloud) can attach
/// directly. False for the GCE tarball, which must be unpacked first.
pub fn is_attachable(self) -> bool {
!matches!(self, CloudMedium::GceTarGz)
}
/// **The medium a path's extension names**, or `None` when it names none.
///
/// The one reader of file extensions in this module, so a caller that has
/// only a path — `InstallerBuild::kvm_disk_args` is the one that matters —
/// gets the same answer as a caller holding the variant. `None` is a real
/// answer and not a failure: a harness hands this seam a blank disk it just
/// created, which has no format to detect, and the caller's own default
/// (`raw`) is right for that.
///
/// Deliberately NOT a content sniff. Reading a `conectix` footer off the
/// file would be a second, disagreeing source of truth about an artifact
/// this module just wrote, and the writer already knows.
pub fn for_extension(path: impl AsRef<str>) -> Option<Self> {
let p = path.as_ref();
let ext = p.rsplit_once('.').map(|(_, e)| e.to_ascii_lowercase())?;
match ext.as_str() {
"vhd" => Some(CloudMedium::Vhd),
"qcow2" => Some(CloudMedium::Qcow2),
"raw" | "img" => Some(CloudMedium::Raw),
// `.gz` alone is not the GCE medium — that one is `disk.raw.tar.gz`
// and is never attached as a drive anyway (`is_attachable`).
"gz" if p.ends_with(".tar.gz") => Some(CloudMedium::GceTarGz),
_ => None,
}
}
/// **The rendered `-drive` argv that attaches this medium to QEMU.**
///
/// One writer for the format word, because the alternative is already in
/// the tree and already wrong: `InstallerBuild::kvm_disk_args`
/// (`crate::installer`) SNIFFS the format from the file extension —
/// `if disk_path.ends_with(".qcow2") { "qcow2" } else { "raw" }` — so a
/// `.vhd` is attached as `format=raw` and the firmware finds a `conectix`
/// footer where a partition table should be. That door predates this
/// module and is not this lane's file to edit; when it delegates here, the
/// sniff and its blind spot go away.
///
/// [`GceTarGz`](Self::GceTarGz) has no argv at all — a tarball is not a
/// block device. Unpack it with [`unpack_gce_tar_gz`] and attach the
/// resulting `disk.raw` as [`Raw`](Self::Raw), which is what GCE does.
pub fn qemu_drive_arg(self, disk_path: &Path) -> Option<Vec<String>> {
self.is_attachable().then(|| {
vec![
"-drive".to_string(),
format!("file={},if=virtio,format={}", disk_path.display(), self.qemu_format()),
]
})
}
/// **Who takes it, in one phrase** — the sentence a door prints when it
/// reports or refuses a medium. Kept beside the variants rather than in
/// every caller (LAW 5), exactly as [`MediumBring::describe`] is.
pub fn describe(self) -> &'static str {
match self {
CloudMedium::Raw => "a raw disk image (AWS import-snapshot then register-image)",
CloudMedium::Vhd => "a fixed-size VHD (Azure Gen1/Gen2)",
CloudMedium::GceTarGz => "disk.raw inside a gzipped tar (GCE images create)",
CloudMedium::Qcow2 => "a qcow2 v3 image (OVH / Scaleway / OpenStack glance)",
}
}
/// **The join to the roster.** A preset row declares how its platform takes
/// a medium; this answers which writer produces it, or `None` when the
/// platform takes an attached ISO ([`MediumBring::IsoCdrom`] — that is
/// [`crate::iso_writer`]'s job), is bare metal, or is undocumented.
///
/// [`MediumBring::Several`] carries a free-text phrase because the platform
/// genuinely accepts more than one shape; the phrase is matched for the
/// medium this constellation actually produces (GCE's `disk.raw` tar.gz,
/// then AWS' raw). Anything unrecognised is `None` rather than a guess, and
/// the wildcard arm means a variant added to `MediumBring` later compiles
/// here instead of breaking a file this lane does not own.
pub fn for_bring(bring: MediumBring) -> Option<CloudMedium> {
match bring {
MediumBring::ImportRaw => Some(CloudMedium::Raw),
MediumBring::ImportVhd => Some(CloudMedium::Vhd),
MediumBring::ImportQcow2 => Some(CloudMedium::Qcow2),
MediumBring::Several(how) if how.contains(GCE_MEMBER_NAME) => {
Some(CloudMedium::GceTarGz)
}
MediumBring::Several(how) if how.contains("raw") => Some(CloudMedium::Raw),
_ => None,
}
}
/// The medium this preset's platform imports, if any.
pub fn for_preset(p: &TargetPreset) -> Option<CloudMedium> {
CloudMedium::for_bring(p.medium)
}
/// **Every roster row this medium serves**, derived from `PRESETS` rather
/// than hand-listed — a preset added later joins its writer with no edit
/// here, which is the whole point of deriving it.
pub fn presets(self) -> Vec<&'static TargetPreset> {
PRESETS.iter().filter(|p| CloudMedium::for_preset(p) == Some(self)).collect()
}
}
/// What a writer actually produced — the APPLIED OUTPUT, measured off the file
/// that now exists, not the request that was made.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MediumProduct {
/// Which writer made it.
pub medium: CloudMedium,
/// Where it landed.
pub path: PathBuf,
/// Bytes of the source raw disk.
pub source_size: u64,
/// The disk size the CLOUD will see. Equals `source_size` except for
/// [`CloudMedium::Vhd`], where it is rounded up to the next MiB.
pub virtual_size: u64,
/// Zero bytes appended to reach [`virtual_size`](Self::virtual_size).
pub padded: u64,
/// Bytes the artifact occupies on disk. For a sparse qcow2 or a compressed
/// tarball this is far smaller than `virtual_size`, and that difference is
/// the upload a customer does not pay for.
pub file_size: u64,
/// SHA-256 over the **virtual** image (source + padding) — the content
/// identity that is the same across all four media of one install, and the
/// number a read-back must reproduce.
pub content_sha256: [u8; 32],
/// Whether [`virtual_size`](Self::virtual_size) is a whole number of GiB.
/// GCE prefers it; nothing here enforces it, and this field is the honest
/// report rather than a silent rounding.
pub gib_aligned: bool,
}
impl MediumProduct {
/// The content identity as lowercase hex — through the shared encoder, not a
/// private `write!("{b:02x}")` loop (LAW 5).
pub fn content_hex(&self) -> String {
nornir_hash::hex_encode(&self.content_sha256)
}
}
/// Round `n` up to the next multiple of `q` (`q` must be non-zero).
fn round_up(n: u64, q: u64) -> u64 {
n.div_ceil(q) * q
}
/// **The one validated door onto a source disk.** Every writer starts here, so
/// "what counts as an installable raw disk" has exactly one definition and no
/// writer can be laxer than another.
///
/// Refuses — by name, [`ERR_SOURCE`] — anything that is not a regular file, is
/// empty, or is not a whole number of 512-byte sectors. A partial sector is not
/// a disk and every cloud importer rejects one; finding out here costs nothing,
/// finding out after an upload costs an hour.
pub fn validate_source(src: &Path) -> Result<u64> {
let meta = std::fs::metadata(src)
.with_context(|| format!("{ERR_SOURCE}: cannot stat {}", src.display()))?;
if !meta.is_file() {
bail!("{ERR_SOURCE}: {} is not a regular file", src.display());
}
let len = meta.len();
if len == 0 {
bail!("{ERR_SOURCE}: {} is empty", src.display());
}
if len % SECTOR != 0 {
bail!(
"{ERR_SOURCE}: {} is {len} bytes, not a whole number of {SECTOR}-byte sectors",
src.display()
);
}
Ok(len)
}
/// What a partition table looks like in the first two sectors of an image.
///
/// Reported, never enforced: an installer's disk carries a protective-MBR
/// `0x55AA` and a GPT header, but a legitimately partitionless image (a
/// single-filesystem cloud volume) is not an error. The proofs assert on this so
/// a "converted" image that LOST its partition table cannot pass as one that
/// kept it — the failure a byte-count check would never see.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct DiskProbe {
/// `0x55 0xAA` at bytes 510..512.
pub mbr_signature: bool,
/// `EFI PART` at the start of LBA 1.
pub gpt_header: bool,
}
/// Read the first two sectors of `src` and report what a partition table would
/// look like there. See [`DiskProbe`].
pub fn probe_disk(src: &Path) -> Result<DiskProbe> {
let mut f = File::open(src).with_context(|| format!("open {}", src.display()))?;
let mut head = [0u8; 2 * SECTOR as usize];
f.read_exact(&mut head)
.with_context(|| format!("read the first two sectors of {}", src.display()))?;
Ok(DiskProbe {
mbr_signature: head[510] == 0x55 && head[511] == 0xAA,
gpt_header: &head[512..520] == b"EFI PART",
})
}
/// **The one door.** Write `src` as `medium` to `dst`.
///
/// Every per-medium function below is reachable directly; this exists so a
/// matrix (`for m in CloudMedium::ALL`) needs no four-arm `match` at the call
/// site, which is the twin that lets one medium quietly go missing.
pub fn write_medium(medium: CloudMedium, src: &Path, dst: &Path) -> Result<MediumProduct> {
match medium {
CloudMedium::Raw => write_raw(src, dst),
CloudMedium::Vhd => write_vhd(src, dst),
CloudMedium::GceTarGz => write_gce_tar_gz(src, dst),
CloudMedium::Qcow2 => write_qcow2(src, dst),
}
}
/// **One install, four media** — write every [`CloudMedium`] of `src` into
/// `out_dir` as `<stem>.<ext>`, in [`CloudMedium::ALL`] order.
///
/// This is the shape a `[[build.installer]]` cell wants: the KVM install proof
/// produced ONE disk, and the roster needs it in four containers. Returns the
/// products in the same order, so a receipt can record all four against the one
/// [`MediumProduct::content_sha256`] they share.
pub fn write_all_media(src: &Path, out_dir: &Path, stem: &str) -> Result<Vec<MediumProduct>> {
std::fs::create_dir_all(out_dir)
.with_context(|| format!("create the medium output dir {}", out_dir.display()))?;
let mut out = Vec::with_capacity(CloudMedium::ALL.len());
for m in CloudMedium::ALL {
let dst = out_dir.join(format!("{stem}.{}", m.file_extension()));
out.push(write_medium(m, src, &dst)?);
}
Ok(out)
}
/// A reader that hashes every byte it hands out — so a disk image is read ONCE
/// to be both written into a container and content-identified. The alternative
/// (a second full pass to hash) doubles the I/O on the largest thing in the
/// build.
struct HashingReader<'a, R: Read> {
inner: R,
hasher: &'a mut Sha256,
}
impl<R: Read> Read for HashingReader<'_, R> {
fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
let n = self.inner.read(buf)?;
self.hasher.update(&buf[..n]);
Ok(n)
}
}
/// Stream `src` into `sink`, padding with zeros to `virtual_size`, hashing every
/// byte that goes past. Returns the SHA-256 of the virtual image.
///
/// The one copy loop the raw and VHD writers share: two subtly different loops
/// would give "what content identity means here" two answers.
fn stream_padded<W: Write>(src: &Path, sink: &mut W, virtual_size: u64) -> Result<[u8; 32]> {
let mut f = BufReader::new(File::open(src).with_context(|| format!("open {}", src.display()))?);
let mut hasher = Sha256::new();
let mut buf = vec![0u8; COPY_CHUNK];
let mut done = 0u64;
loop {
let n = f.read(&mut buf).with_context(|| format!("read {}", src.display()))?;
if n == 0 {
break;
}
hasher.update(&buf[..n]);
sink.write_all(&buf[..n]).context("write the image body")?;
done += n as u64;
}
let mut left = virtual_size.saturating_sub(done);
if left > 0 {
let zeros = vec![0u8; COPY_CHUNK.min(left as usize)];
while left > 0 {
let n = (zeros.len() as u64).min(left) as usize;
hasher.update(&zeros[..n]);
sink.write_all(&zeros[..n]).context("write the virtual-size padding")?;
left -= n as u64;
}
}
Ok(hasher.finalize().into())
}
// ─────────────────────────────────────────────────────────────────────────────
// 1. RAW — AWS import-snapshot then register-image
// ─────────────────────────────────────────────────────────────────────────────
/// **AWS' medium: the raw disk, validated and named.**
///
/// A passthrough is still a PATH, not a "do nothing". `aws ec2 import-snapshot`
/// takes the block image byte-for-byte, so there is no transform — but the
/// source is validated ([`validate_source`]), the bytes are hashed so the
/// artifact carries the same [`MediumProduct::content_sha256`] every other
/// medium of this install carries, and the product is NAMED so a proof can boot
/// it and a receipt can record it. "For raw we skip the writer" is precisely how
/// a medium ends up with no test.
pub fn write_raw(src: &Path, dst: &Path) -> Result<MediumProduct> {
let source_size = validate_source(src)?;
let mut out =
BufWriter::new(File::create(dst).with_context(|| format!("create {}", dst.display()))?);
let content_sha256 = stream_padded(src, &mut out, source_size)?;
out.flush().context("flush the raw image")?;
drop(out);
Ok(MediumProduct {
medium: CloudMedium::Raw,
path: dst.to_path_buf(),
source_size,
virtual_size: source_size,
padded: 0,
file_size: std::fs::metadata(dst)?.len(),
content_sha256,
gib_aligned: source_size % GIB == 0,
})
}
// ─────────────────────────────────────────────────────────────────────────────
// 2. FIXED-SIZE VHD — Azure
// ─────────────────────────────────────────────────────────────────────────────
/// The 8-byte cookie every VHD footer starts with.
pub const VHD_COOKIE: [u8; 8] = *b"conectix";
/// A VHD footer is exactly 512 bytes.
pub const VHD_FOOTER_LEN: usize = 512;
/// Disk type 2 — **fixed**. Azure takes fixed VHDs only; a dynamic (type 3) VHD
/// is refused at import.
pub const VHD_TYPE_FIXED: u32 = 2;
/// `data offset` for a fixed disk: there is no dynamic-disk header to point at.
pub const VHD_DATA_OFFSET_FIXED: u64 = 0xFFFF_FFFF_FFFF_FFFF;
/// Feature bits: bit 1 (`0x2`) is the spec's "reserved, must be set" bit.
pub const VHD_FEATURES: u32 = 0x0000_0002;
/// File-format version 1.0.
pub const VHD_FORMAT_VERSION: u32 = 0x0001_0000;
/// Creator host OS `Wi2k` — what every Windows-written VHD carries, and what
/// Azure's importer has always been happy with.
pub const VHD_HOST_OS_WINDOWS: u32 = 0x5769_326B;
/// VHD timestamps count seconds since **2000-01-01T00:00:00Z**, not the Unix
/// epoch. This is that epoch expressed as a Unix time, for a caller converting.
pub const VHD_EPOCH_UNIX: i64 = 946_684_800;
/// The deterministic default timestamp: 2024-01-01T00:00:00Z in the VHD epoch
/// (24 years, 6 of them leap → 8766 days). Fixed on purpose — the same installed
/// disk must convert byte-identically, which is what lets a content-addressed
/// build attest a medium at all (the same rule as `iso_writer`'s fixed volume
/// dates).
pub const VHD_TIMESTAMP_DEFAULT: u32 = 8766 * 86_400;
/// Knobs a caller may want to move; every default is deterministic.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct VhdOptions {
/// Seconds since the VHD epoch (see [`VHD_EPOCH_UNIX`]).
pub timestamp: u32,
/// Four bytes naming who wrote it.
pub creator_app: [u8; 4],
/// Creator version, `major << 16 | minor`.
pub creator_version: u32,
/// Creator host OS (see [`VHD_HOST_OS_WINDOWS`]).
pub creator_host_os: u32,
/// Override the disk's unique id. `None` (the default) DERIVES it from the
/// SHA-256 of the padded image, so it is unique per content and identical
/// across re-runs — a random v4 here would make every conversion of one
/// install a different file, and a content-addressed build could attest
/// none of them.
pub unique_id: Option<[u8; 16]>,
}
impl Default for VhdOptions {
fn default() -> Self {
Self {
timestamp: VHD_TIMESTAMP_DEFAULT,
creator_app: *b"nrdk",
creator_version: 0x0001_0000,
creator_host_os: VHD_HOST_OS_WINDOWS,
unique_id: None,
}
}
}
/// A VHD footer, decoded. Every field is big-endian on disk.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct VhdFooter {
pub cookie: [u8; 8],
pub features: u32,
pub format_version: u32,
pub data_offset: u64,
pub timestamp: u32,
pub creator_app: [u8; 4],
pub creator_version: u32,
pub creator_host_os: u32,
/// Virtual size at creation.
pub original_size: u64,
/// Virtual size now; equal to `original_size` for a disk nobody resized.
pub current_size: u64,
pub cylinders: u16,
pub heads: u8,
pub sectors_per_track: u8,
pub disk_type: u32,
/// As stored. [`VhdFooter::to_bytes`] recomputes it; [`VhdFooter::parse`]
/// verifies it.
pub checksum: u32,
pub unique_id: [u8; 16],
pub saved_state: u8,
}
/// **The VHD CHS geometry algorithm, verbatim from the VHD spec.**
///
/// It matters: Hyper-V (and therefore Azure) derives a disk's addressable size
/// from `cylinders * heads * sectors_per_track * 512`, so a geometry that does
/// not cover the declared size is a disk the platform believes is smaller than
/// it is. The `65535 * 16 * 255` clamp is the spec's own — CHS cannot address
/// beyond ~127 GiB, and larger disks are legitimately clamped and read their
/// size from the size fields instead.
pub fn vhd_chs(total_sectors: u64) -> (u16, u8, u8) {
const MAX: u64 = 65535 * 16 * 255;
let ts = total_sectors.min(MAX);
let (spt, heads, cyl_times_heads): (u64, u64, u64);
if ts >= 65535 * 16 * 63 {
spt = 255;
heads = 16;
cyl_times_heads = ts / spt;
} else {
let mut s = 17u64;
let mut cth = ts / s;
let mut h = cth.div_ceil(1024);
if h < 4 {
h = 4;
}
if cth >= h * 1024 || h > 16 {
s = 31;
h = 16;
cth = ts / s;
}
if cth >= h * 1024 {
s = 63;
h = 16;
cth = ts / s;
}
spt = s;
heads = h;
cyl_times_heads = cth;
}
let cylinders = cyl_times_heads / heads;
(cylinders as u16, heads as u8, spt as u8)
}
/// The VHD footer checksum: **one's complement of the sum of every footer byte
/// with the checksum field zeroed**.
///
/// Its own public function on purpose — this is the thing a red probe corrupts,
/// and a checksum computed inline in the writer and again inline in the reader
/// is two implementations that can agree while both being wrong.
pub fn vhd_checksum(footer: &[u8; VHD_FOOTER_LEN]) -> u32 {
let mut sum: u32 = 0;
for (i, b) in footer.iter().enumerate() {
// Bytes 64..68 are the checksum field itself — summed as zero.
if (64..68).contains(&i) {
continue;
}
sum = sum.wrapping_add(*b as u32);
}
!sum
}
impl VhdFooter {
/// The footer for a **fixed** disk of `virtual_size` bytes.
///
/// # Errors
/// [`ERR_VHD_NOT_MIB`] when `virtual_size` is not a whole number of MiB. The
/// constructor refuses it rather than the writer, so a caller assembling a
/// footer by hand cannot skip the rule Azure enforces after the upload.
pub fn fixed(virtual_size: u64, unique_id: [u8; 16], opts: &VhdOptions) -> Result<VhdFooter> {
if virtual_size % MIB != 0 {
bail!(
"{ERR_VHD_NOT_MIB}: {virtual_size} bytes is {} B past {} MiB — Azure rejects this \
AFTER the upload",
virtual_size % MIB,
virtual_size / MIB
);
}
let (cylinders, heads, sectors_per_track) = vhd_chs(virtual_size / SECTOR);
Ok(VhdFooter {
cookie: VHD_COOKIE,
features: VHD_FEATURES,
format_version: VHD_FORMAT_VERSION,
data_offset: VHD_DATA_OFFSET_FIXED,
timestamp: opts.timestamp,
creator_app: opts.creator_app,
creator_version: opts.creator_version,
creator_host_os: opts.creator_host_os,
original_size: virtual_size,
current_size: virtual_size,
cylinders,
heads,
sectors_per_track,
disk_type: VHD_TYPE_FIXED,
checksum: 0,
unique_id,
saved_state: 0,
})
}
/// Render the 512 on-disk bytes, stamping the checksum last.
pub fn to_bytes(&self) -> [u8; VHD_FOOTER_LEN] {
let mut b = [0u8; VHD_FOOTER_LEN];
b[0..8].copy_from_slice(&self.cookie);
b[8..12].copy_from_slice(&self.features.to_be_bytes());
b[12..16].copy_from_slice(&self.format_version.to_be_bytes());
b[16..24].copy_from_slice(&self.data_offset.to_be_bytes());
b[24..28].copy_from_slice(&self.timestamp.to_be_bytes());
b[28..32].copy_from_slice(&self.creator_app);
b[32..36].copy_from_slice(&self.creator_version.to_be_bytes());
b[36..40].copy_from_slice(&self.creator_host_os.to_be_bytes());
b[40..48].copy_from_slice(&self.original_size.to_be_bytes());
b[48..56].copy_from_slice(&self.current_size.to_be_bytes());
b[56..58].copy_from_slice(&self.cylinders.to_be_bytes());
b[58] = self.heads;
b[59] = self.sectors_per_track;
b[60..64].copy_from_slice(&self.disk_type.to_be_bytes());
// 64..68 = checksum, left zero while the sum is taken.
b[68..84].copy_from_slice(&self.unique_id);
b[84] = self.saved_state;
let ck = vhd_checksum(&b);
b[64..68].copy_from_slice(&ck.to_be_bytes());
b
}
/// Parse **and VALIDATE** 512 footer bytes.
///
/// This is the gate, not a decoder: it refuses a bad cookie
/// ([`ERR_VHD_COOKIE`]), a checksum that does not verify
/// ([`ERR_VHD_CHECKSUM`]), a non-fixed disk type ([`ERR_VHD_NOT_FIXED`]) and
/// a virtual size that is not MiB-aligned ([`ERR_VHD_NOT_MIB`]) — the four
/// things that make Azure reject an upload it has already taken the bytes
/// of.
pub fn parse(bytes: &[u8]) -> Result<VhdFooter> {
if bytes.len() < VHD_FOOTER_LEN {
bail!("{ERR_VHD_SHORT}: got {} bytes, need {VHD_FOOTER_LEN}", bytes.len());
}
let b: &[u8; VHD_FOOTER_LEN] =
bytes[..VHD_FOOTER_LEN].try_into().expect("length checked above");
let mut cookie = [0u8; 8];
cookie.copy_from_slice(&b[0..8]);
if cookie != VHD_COOKIE {
bail!("{ERR_VHD_COOKIE}: got {:?}", String::from_utf8_lossy(&cookie));
}
let stored = u32::from_be_bytes(b[64..68].try_into().unwrap());
let computed = vhd_checksum(b);
if stored != computed {
bail!("{ERR_VHD_CHECKSUM}: stored 0x{stored:08x}, computed 0x{computed:08x}");
}
let disk_type = u32::from_be_bytes(b[60..64].try_into().unwrap());
if disk_type != VHD_TYPE_FIXED {
bail!("{ERR_VHD_NOT_FIXED}: disk type is {disk_type}");
}
let current_size = u64::from_be_bytes(b[48..56].try_into().unwrap());
if current_size % MIB != 0 {
bail!("{ERR_VHD_NOT_MIB}: current size {current_size} bytes");
}
let mut creator_app = [0u8; 4];
creator_app.copy_from_slice(&b[28..32]);
let mut unique_id = [0u8; 16];
unique_id.copy_from_slice(&b[68..84]);
Ok(VhdFooter {
cookie,
features: u32::from_be_bytes(b[8..12].try_into().unwrap()),
format_version: u32::from_be_bytes(b[12..16].try_into().unwrap()),
data_offset: u64::from_be_bytes(b[16..24].try_into().unwrap()),
timestamp: u32::from_be_bytes(b[24..28].try_into().unwrap()),
creator_app,
creator_version: u32::from_be_bytes(b[32..36].try_into().unwrap()),
creator_host_os: u32::from_be_bytes(b[36..40].try_into().unwrap()),
original_size: u64::from_be_bytes(b[40..48].try_into().unwrap()),
current_size,
cylinders: u16::from_be_bytes(b[56..58].try_into().unwrap()),
heads: b[58],
sectors_per_track: b[59],
disk_type,
checksum: stored,
unique_id,
saved_state: b[84],
})
}
/// The size the CHS geometry can address, in bytes. Reported so a caller
/// sees the spec's clamp rather than being surprised by it.
pub fn chs_bytes(&self) -> u64 {
self.cylinders as u64 * self.heads as u64 * self.sectors_per_track as u64 * SECTOR
}
}
/// Read the trailing footer off a `.vhd` and validate it — the way back in.
///
/// # Errors
/// [`ERR_VHD_SHORT`] for a file that cannot hold a footer, plus everything
/// [`VhdFooter::parse`] refuses.
pub fn read_vhd_footer(path: &Path) -> Result<VhdFooter> {
let mut f = File::open(path).with_context(|| format!("open {}", path.display()))?;
let len = f.metadata()?.len();
if len < VHD_FOOTER_LEN as u64 {
bail!("{ERR_VHD_SHORT}: {} is {len} bytes", path.display());
}
f.seek(SeekFrom::End(-(VHD_FOOTER_LEN as i64)))?;
let mut buf = [0u8; VHD_FOOTER_LEN];
f.read_exact(&mut buf)?;
VhdFooter::parse(&buf)
}
/// **Azure's medium: a fixed-size VHD** — the raw disk, zero-padded up to the
/// next whole MiB, followed by the 512-byte `conectix` footer.
///
/// The MiB rounding is the point. Azure validates it server-side, *after* the
/// upload has finished, so a 700 MiB + 3 sector image costs the whole transfer
/// before it is refused. Here the padding is written and both numbers are
/// recorded ([`MediumProduct::virtual_size`], [`MediumProduct::padded`]).
pub fn write_vhd(src: &Path, dst: &Path) -> Result<MediumProduct> {
write_vhd_with(src, dst, &VhdOptions::default())
}
/// [`write_vhd`] with explicit [`VhdOptions`].
pub fn write_vhd_with(src: &Path, dst: &Path, opts: &VhdOptions) -> Result<MediumProduct> {
let source_size = validate_source(src)?;
let virtual_size = round_up(source_size, MIB);
let mut out =
BufWriter::new(File::create(dst).with_context(|| format!("create {}", dst.display()))?);
let content_sha256 = stream_padded(src, &mut out, virtual_size)?;
// The unique id is DERIVED from the content unless overridden: same install
// then same VHD, byte for byte. The RFC-4122 version/variant nibbles are
// stamped so the value is a well-formed UUID and not merely 16 bytes.
let unique_id = opts.unique_id.unwrap_or_else(|| {
let mut id = [0u8; 16];
id.copy_from_slice(&content_sha256[..16]);
id[6] = (id[6] & 0x0F) | 0x40;
id[8] = (id[8] & 0x3F) | 0x80;
id
});
let footer = VhdFooter::fixed(virtual_size, unique_id, opts)?;
out.write_all(&footer.to_bytes()).context("write the VHD footer")?;
out.flush().context("flush the VHD")?;
drop(out);
Ok(MediumProduct {
medium: CloudMedium::Vhd,
path: dst.to_path_buf(),
source_size,
virtual_size,
padded: virtual_size - source_size,
file_size: std::fs::metadata(dst)?.len(),
content_sha256,
gib_aligned: virtual_size % GIB == 0,
})
}
// ─────────────────────────────────────────────────────────────────────────────
// 3. disk.raw in a gzipped tar — GCE
// ─────────────────────────────────────────────────────────────────────────────
/// **GCE's medium: `disk.raw` inside a gzipped tar.**
///
/// The tarball holds exactly ONE member and it is named [`GCE_MEMBER_NAME`].
/// That single name is the whole contract.
///
/// Pure Rust: `tar` writes the USTAR header and `flate2` (miniz_oxide backend)
/// the gzip stream — as libraries, in-process. Nothing shells to `tar` or
/// `gzip`. Deterministic: mode 0644, uid/gid 0, mtime 0, and gzip's own header
/// carries no filename or mtime, so the same install produces the same tarball
/// every time.
pub fn write_gce_tar_gz(src: &Path, dst: &Path) -> Result<MediumProduct> {
let source_size = validate_source(src)?;
let out = File::create(dst).with_context(|| format!("create {}", dst.display()))?;
let gz = flate2::write::GzEncoder::new(BufWriter::new(out), flate2::Compression::default());
let mut builder = tar::Builder::new(gz);
let mut header = tar::Header::new_ustar();
header
.set_path(GCE_MEMBER_NAME)
.with_context(|| format!("{ERR_GCE_MEMBER}: cannot name the member {GCE_MEMBER_NAME}"))?;
header.set_size(source_size);
header.set_mode(0o644);
header.set_uid(0);
header.set_gid(0);
header.set_mtime(0);
header.set_entry_type(tar::EntryType::Regular);
// Last, over every field above: the USTAR checksum covers the whole header.
header.set_cksum();
let mut hasher = Sha256::new();
{
let f = File::open(src).with_context(|| format!("open {}", src.display()))?;
let mut reader =
HashingReader { inner: BufReader::with_capacity(COPY_CHUNK, f), hasher: &mut hasher };
builder
.append(&header, &mut reader)
.with_context(|| format!("append {GCE_MEMBER_NAME} to the tarball"))?;
}
let gz = builder.into_inner().context("finish the tar stream")?;
let mut w = gz.finish().context("finish the gzip stream")?;
w.flush().context("flush the GCE tarball")?;
drop(w);
Ok(MediumProduct {
medium: CloudMedium::GceTarGz,
path: dst.to_path_buf(),
source_size,
virtual_size: source_size,
padded: 0,
file_size: std::fs::metadata(dst)?.len(),
content_sha256: hasher.finalize().into(),
gib_aligned: source_size % GIB == 0,
})
}
/// The member names inside a gzipped tar, in stream order — the READ-BACK that
/// makes [`GCE_MEMBER_NAME`] provable instead of merely asserted.
pub fn tar_gz_member_names(path: &Path) -> Result<Vec<String>> {
let f = File::open(path).with_context(|| format!("open {}", path.display()))?;
let gz = flate2::read::GzDecoder::new(BufReader::new(f));
let mut ar = tar::Archive::new(gz);
let mut names = Vec::new();
for e in ar.entries().context("read the tar stream")? {
let e = e.context("read a tar entry")?;
names.push(e.path().context("decode a tar entry path")?.to_string_lossy().into_owned());
}
Ok(names)
}
/// Unpack a GCE tarball's `disk.raw` to `dst` — what GCE does with it, and what
/// the boot proof does before attaching it to QEMU. Returns the bytes written.
///
/// # Errors
/// [`ERR_GCE_MEMBER`] when the archive does not hold exactly one member named
/// [`GCE_MEMBER_NAME`]. Both halves matter: a second member is as wrong as a
/// misnamed one.
pub fn unpack_gce_tar_gz(path: &Path, dst: &Path) -> Result<u64> {
let f = File::open(path).with_context(|| format!("open {}", path.display()))?;
let gz = flate2::read::GzDecoder::new(BufReader::new(f));
let mut ar = tar::Archive::new(gz);
let mut found: Option<u64> = None;
let mut seen: Vec<String> = Vec::new();
for e in ar.entries().context("read the tar stream")? {
let mut e = e.context("read a tar entry")?;
let name = e.path()?.to_string_lossy().into_owned();
seen.push(name.clone());
if name == GCE_MEMBER_NAME {
let mut out = BufWriter::new(
File::create(dst).with_context(|| format!("create {}", dst.display()))?,
);
let n = std::io::copy(&mut e, &mut out).context("unpack disk.raw")?;
out.flush()?;
found = Some(n);
}
}
match found {
Some(n) if seen.len() == 1 => Ok(n),
_ => bail!(
"{ERR_GCE_MEMBER}: archive holds {seen:?}, expected exactly [\"{GCE_MEMBER_NAME}\"]"
),
}
}
// ─────────────────────────────────────────────────────────────────────────────
// 4. qcow2 v3 — OVH / Scaleway / OpenStack glance
// ─────────────────────────────────────────────────────────────────────────────
/// The qcow2 magic: `QFI` followed by `0xfb`.
pub const QCOW2_MAGIC: [u8; 4] = [b'Q', b'F', b'I', 0xfb];
/// The only version this writer emits and this reader accepts.
pub const QCOW2_VERSION: u32 = 3;
/// 64 KiB clusters — qemu's own default, and what every cloud's importer has
/// been reading for a decade.
pub const QCOW2_CLUSTER_BITS: u32 = 16;
/// `refcount_order` 4 gives 16-bit refcounts (qemu's default).
pub const QCOW2_REFCOUNT_ORDER: u32 = 4;
/// v3 header length, through `header_length` itself.
pub const QCOW2_HEADER_LENGTH: u32 = 112;
/// L1/L2 entry bit 63: "refcount is exactly 1, this cluster may be written in
/// place". qemu warns loudly about an allocated cluster without it.
const QCOW2_OFLAG_COPIED: u64 = 1 << 63;
/// L1/L2 entry bits 9..55 hold the host offset.
const QCOW2_OFFSET_MASK: u64 = 0x00ff_ffff_ffff_fe00;
/// A qcow2 v3 header, decoded.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Qcow2Header {
pub version: u32,
pub backing_file_offset: u64,
pub cluster_bits: u32,
/// The GUEST disk size in bytes.
pub size: u64,
pub crypt_method: u32,
pub l1_size: u32,
pub l1_table_offset: u64,
pub refcount_table_offset: u64,
pub refcount_table_clusters: u32,
pub nb_snapshots: u32,
pub incompatible_features: u64,
pub compatible_features: u64,
pub autoclear_features: u64,
pub refcount_order: u32,
pub header_length: u32,
}
impl Qcow2Header {
/// Cluster size in bytes.
pub fn cluster_size(&self) -> u64 {
1u64 << self.cluster_bits
}
/// L2 entries per cluster.
pub fn l2_entries(&self) -> u64 {
self.cluster_size() / 8
}
/// Decode and VALIDATE the first 112 bytes.
///
/// Refuses a bad magic ([`ERR_QCOW2_MAGIC`]), a version that is not 3
/// ([`ERR_QCOW2_VERSION`]) and ANY `incompatible_features` bit
/// ([`ERR_QCOW2_FEATURES`]). The last one is the honest refusal: this reader
/// implements no compression, no external data file and no extended L2, so
/// an image declaring them must not be half-read in silence.
pub fn parse(b: &[u8]) -> Result<Qcow2Header> {
if b.len() < QCOW2_HEADER_LENGTH as usize {
bail!("{ERR_QCOW2_MAGIC}: only {} bytes", b.len());
}
if b[0..4] != QCOW2_MAGIC {
bail!("{ERR_QCOW2_MAGIC}: got {:02x?}", &b[0..4]);
}
let g32 = |o: usize| u32::from_be_bytes(b[o..o + 4].try_into().unwrap());
let g64 = |o: usize| u64::from_be_bytes(b[o..o + 8].try_into().unwrap());
let version = g32(4);
if version != QCOW2_VERSION {
bail!("{ERR_QCOW2_VERSION}: got {version}");
}
let incompatible_features = g64(72);
if incompatible_features != 0 {
bail!("{ERR_QCOW2_FEATURES}: incompatible_features = 0x{incompatible_features:016x}");
}
Ok(Qcow2Header {
version,
backing_file_offset: g64(8),
cluster_bits: g32(20),
size: g64(24),
crypt_method: g32(32),
l1_size: g32(36),
l1_table_offset: g64(40),
refcount_table_offset: g64(48),
refcount_table_clusters: g32(56),
nb_snapshots: g32(60),
incompatible_features,
compatible_features: g64(80),
autoclear_features: g64(88),
refcount_order: g32(96),
header_length: g32(100),
})
}
}
/// **OVH / Scaleway / OpenStack's medium: qcow2 v3** — uncompressed, unbacked,
/// and **sparse**: a guest cluster that is all zeros gets no L2 entry at all and
/// reads back as zeros, which is why the artifact is normally a small fraction
/// of the raw disk.
///
/// The layout is laid in ONE forward pass, with the refcount sizing solved to a
/// fixpoint first (refcount blocks are themselves clusters that must be
/// refcounted, so their count depends on itself):
///
/// ```text
/// cluster 0 header (+ the end-of-extensions marker)
/// next rt_clusters refcount TABLE (u64 offsets of the blocks)
/// next rb_clusters refcount BLOCKS (u16 counts, one per cluster in the file)
/// next l1_clusters L1 table (u64 offsets of the L2 tables)
/// next l2_count L2 tables (one per 512 MiB of guest that has data)
/// the rest data clusters, in guest order
/// ```
///
/// Explicitly NOT implemented: compression, a backing file, encryption, extended
/// L2, snapshots. Each is a real qcow2 feature and each is absent ON PURPOSE —
/// an import image needs none of them, and half of one is worse than none.
/// [`Qcow2Header::parse`] refuses an image that declares them rather than
/// pretending to read it.
pub fn write_qcow2(src: &Path, dst: &Path) -> Result<MediumProduct> {
let source_size = validate_source(src)?;
let cs = 1u64 << QCOW2_CLUSTER_BITS;
let l2_entries = cs / 8;
// 16-bit refcounts: cs bytes gives cs/2 entries per refcount block.
let rb_entries = cs / 2;
let rt_entries = cs / 8;
// The guest size. A disk is whole sectors; the last cluster may run past it
// and is simply zero there.
let virtual_size = round_up(source_size, SECTOR);
let guest_clusters = virtual_size.div_ceil(cs);
// ── pass 1: which guest clusters hold anything but zeros ──
let mut allocated = vec![false; guest_clusters as usize];
{
let mut f =
BufReader::new(File::open(src).with_context(|| format!("open {}", src.display()))?);
let mut buf = vec![0u8; cs as usize];
for slot in allocated.iter_mut() {
let mut got = 0usize;
while got < buf.len() {
let n = f.read(&mut buf[got..])?;
if n == 0 {
break;
}
got += n;
}
if got == 0 {
break;
}
*slot = buf[..got].iter().any(|&b| b != 0);
}
}
let n_data: u64 = allocated.iter().filter(|a| **a).count() as u64;
// Which L1 slots need an L2 table.
let l1_size = guest_clusters.div_ceil(l2_entries).max(1);
let mut l2_needed = vec![false; l1_size as usize];
for (i, a) in allocated.iter().enumerate() {
if *a {
l2_needed[i / l2_entries as usize] = true;
}
}
let l2_count: u64 = l2_needed.iter().filter(|n| **n).count() as u64;
let l1_clusters = (l1_size * 8).div_ceil(cs).max(1);
// ── the refcount fixpoint: the blocks must refcount themselves ──
let (mut rb_count, mut rt_clusters) = (1u64, 1u64);
for _ in 0..8 {
let total = 1 + rt_clusters + rb_count + l1_clusters + l2_count + n_data;
let need_rb = total.div_ceil(rb_entries).max(1);
let need_rt = need_rb.div_ceil(rt_entries).max(1);
if need_rb == rb_count && need_rt == rt_clusters {
break;
}
rb_count = need_rb;
rt_clusters = need_rt;
}
let total_clusters = 1 + rt_clusters + rb_count + l1_clusters + l2_count + n_data;
// ── the layout ──
let rt_off = cs; // cluster 1
let rb_off = rt_off + rt_clusters * cs;
let l1_off = rb_off + rb_count * cs;
let l2_off = l1_off + l1_clusters * cs;
let data_off = l2_off + l2_count * cs;
let mut out = BufWriter::with_capacity(
COPY_CHUNK,
File::create(dst).with_context(|| format!("create {}", dst.display()))?,
);
// ── header ──
let mut hdr = vec![0u8; cs as usize];
hdr[0..4].copy_from_slice(&QCOW2_MAGIC);
hdr[4..8].copy_from_slice(&QCOW2_VERSION.to_be_bytes());
// 8..16 backing_file_offset = 0, 16..20 backing_file_size = 0
hdr[20..24].copy_from_slice(&QCOW2_CLUSTER_BITS.to_be_bytes());
hdr[24..32].copy_from_slice(&virtual_size.to_be_bytes());
// 32..36 crypt_method = 0
hdr[36..40].copy_from_slice(&(l1_size as u32).to_be_bytes());
hdr[40..48].copy_from_slice(&l1_off.to_be_bytes());
hdr[48..56].copy_from_slice(&rt_off.to_be_bytes());
hdr[56..60].copy_from_slice(&(rt_clusters as u32).to_be_bytes());
// 60..64 nb_snapshots = 0, 64..72 snapshots_offset = 0
// 72..80 incompatible, 80..88 compatible, 88..96 autoclear — all zero
hdr[96..100].copy_from_slice(&QCOW2_REFCOUNT_ORDER.to_be_bytes());
hdr[100..104].copy_from_slice(&QCOW2_HEADER_LENGTH.to_be_bytes());
// The header-extension area starts at `header_length`; eight zero bytes are
// the end-of-extensions marker (type 0, length 0), and they already are.
out.write_all(&hdr).context("write the qcow2 header")?;
// ── refcount table ──
let mut rt = vec![0u8; (rt_clusters * cs) as usize];
for i in 0..rb_count {
let off = rb_off + i * cs;
rt[(i as usize) * 8..(i as usize) * 8 + 8].copy_from_slice(&off.to_be_bytes());
}
out.write_all(&rt).context("write the qcow2 refcount table")?;
// ── refcount blocks: every cluster in the file has refcount 1 ──
let mut rb = vec![0u8; (rb_count * cs) as usize];
for c in 0..total_clusters {
let i = (c as usize) * 2;
rb[i..i + 2].copy_from_slice(&1u16.to_be_bytes());
}
out.write_all(&rb).context("write the qcow2 refcount blocks")?;
// ── L1 table ──
let mut l1 = vec![0u8; (l1_clusters * cs) as usize];
{
let mut next = l2_off;
for (k, needed) in l2_needed.iter().enumerate() {
if *needed {
l1[k * 8..k * 8 + 8].copy_from_slice(&(next | QCOW2_OFLAG_COPIED).to_be_bytes());
next += cs;
}
}
}
out.write_all(&l1).context("write the qcow2 L1 table")?;
// ── L2 tables ──
{
let mut host = data_off;
let mut l2 = vec![0u8; cs as usize];
for (k, needed) in l2_needed.iter().enumerate() {
if !*needed {
continue;
}
l2.fill(0);
let base = k * l2_entries as usize;
for j in 0..l2_entries as usize {
let g = base + j;
if g < allocated.len() && allocated[g] {
l2[j * 8..j * 8 + 8]
.copy_from_slice(&(host | QCOW2_OFLAG_COPIED).to_be_bytes());
host += cs;
}
}
out.write_all(&l2).context("write a qcow2 L2 table")?;
}
}
// ── data clusters, in guest order, hashing the WHOLE virtual image ──
let content_sha256 = {
let mut f =
BufReader::new(File::open(src).with_context(|| format!("open {}", src.display()))?);
let mut hasher = Sha256::new();
let mut buf = vec![0u8; cs as usize];
for (i, a) in allocated.iter().enumerate() {
buf.fill(0);
let mut got = 0usize;
while got < buf.len() {
let n = f.read(&mut buf[got..])?;
if n == 0 {
break;
}
got += n;
}
// The hash covers the VIRTUAL image, so the tail cluster counts only
// up to `virtual_size` — otherwise this medium's content id would
// differ from the other three for the same install, and the
// cross-medium identity assertion would be a lie.
let span = ((i as u64 + 1) * cs).min(virtual_size).saturating_sub(i as u64 * cs);
hasher.update(&buf[..span as usize]);
if *a {
out.write_all(&buf).context("write a qcow2 data cluster")?;
}
}
let d: [u8; 32] = hasher.finalize().into();
d
};
out.flush().context("flush the qcow2 image")?;
drop(out);
Ok(MediumProduct {
medium: CloudMedium::Qcow2,
path: dst.to_path_buf(),
source_size,
virtual_size,
padded: virtual_size - source_size,
file_size: std::fs::metadata(dst)?.len(),
content_sha256,
gib_aligned: virtual_size % GIB == 0,
})
}
/// A read-only qcow2 v3 reader — the READ-BACK half, so the writer is proven by
/// reconstructing the raw disk out of what it wrote, rather than by asserting
/// the header fields it has just set (a state round-trip, which proves nothing).
#[derive(Debug)]
pub struct Qcow2Reader {
file: File,
header: Qcow2Header,
l1: Vec<u64>,
}
impl Qcow2Reader {
/// Open and validate. See [`Qcow2Header::parse`] for the refusals.
pub fn open(path: &Path) -> Result<Qcow2Reader> {
let mut file = File::open(path).with_context(|| format!("open {}", path.display()))?;
let mut head = vec![0u8; QCOW2_HEADER_LENGTH as usize];
file.read_exact(&mut head).context("read the qcow2 header")?;
let header = Qcow2Header::parse(&head)?;
let mut l1raw = vec![0u8; header.l1_size as usize * 8];
file.seek(SeekFrom::Start(header.l1_table_offset))?;
file.read_exact(&mut l1raw).context("read the qcow2 L1 table")?;
let l1 =
l1raw.chunks_exact(8).map(|c| u64::from_be_bytes(c.try_into().unwrap())).collect();
Ok(Qcow2Reader { file, header, l1 })
}
/// The decoded header.
pub fn header(&self) -> &Qcow2Header {
&self.header
}
/// Read one guest cluster into `buf` (which must be `cluster_size` long). An
/// unallocated cluster reads as zeros — the sparse contract.
fn read_cluster(&mut self, guest: u64, buf: &mut [u8]) -> Result<()> {
buf.fill(0);
let per = self.header.l2_entries();
let l1i = (guest / per) as usize;
let Some(&l1e) = self.l1.get(l1i) else { return Ok(()) };
let l2_off = l1e & QCOW2_OFFSET_MASK;
if l2_off == 0 {
return Ok(());
}
let j = guest % per;
self.file.seek(SeekFrom::Start(l2_off + j * 8))?;
let mut e = [0u8; 8];
self.file.read_exact(&mut e)?;
let host = u64::from_be_bytes(e) & QCOW2_OFFSET_MASK;
if host == 0 {
return Ok(());
}
self.file.seek(SeekFrom::Start(host))?;
self.file.read_exact(buf)?;
Ok(())
}
/// Reconstruct the whole guest disk to `dst`. Returns the bytes written,
/// which is exactly the header's `size`.
pub fn to_raw(&mut self, dst: &Path) -> Result<u64> {
let cs = self.header.cluster_size();
let n = self.header.size.div_ceil(cs);
let mut out = BufWriter::with_capacity(
COPY_CHUNK,
File::create(dst).with_context(|| format!("create {}", dst.display()))?,
);
let mut buf = vec![0u8; cs as usize];
let mut left = self.header.size;
for g in 0..n {
self.read_cluster(g, &mut buf)?;
let take = cs.min(left) as usize;
out.write_all(&buf[..take])?;
left -= take as u64;
}
out.flush()?;
Ok(self.header.size)
}
/// SHA-256 over the reconstructed guest image — the number that must equal
/// [`MediumProduct::content_sha256`], and the cheapest honest proof that the
/// L1/L2/refcount structure round-trips.
pub fn content_sha256(&mut self) -> Result<[u8; 32]> {
let cs = self.header.cluster_size();
let n = self.header.size.div_ceil(cs);
let mut hasher = Sha256::new();
let mut buf = vec![0u8; cs as usize];
let mut left = self.header.size;
for g in 0..n {
self.read_cluster(g, &mut buf)?;
let take = cs.min(left) as usize;
hasher.update(&buf[..take]);
left -= take as u64;
}
Ok(hasher.finalize().into())
}
}