.nornir/read-path-guide.md · nordisk/skade

The read path — guide

Class: guide · Function: read-path. The what is in read-path-design.md; the why in read-path-reasoning.md.

Pick the lightest read that answers your question

The caches are transparent — you never call them directly — but which catalog method you call decides how much work a warm hit does:

you need	call	warm hit does
"does this table exist?"	`table_exists(&ident)`	one L1 mirror lookup, no lock, no redb
the current metadata location	`resolve_metadata(&ident)` then `.metadata_location` path	L1 mirror
schema / snapshots / props	`resolve_metadata(&ident)`	L1 → L0 (`Arc` clone)
a full `Table` to scan	`load_table(&ident)`	L1 → L1.5 (`Table` clone)

Rule of thumb: don't call load_table if resolve_metadata answers you. Building a Table constructs iceberg's per-Table object cache; the handle cache amortises it, but resolving metadata skips it entirely.

Sizing the caches

Set budgets at build time (builder-guide):

RedbCatalogBuilder::default()
    .metadata_cache_bytes(256 * 1024 * 1024)   // L0: 256 MiB of parsed metadata
    .table_handle_cache_bytes(512 * 1024 * 1024) // L1.5: byte budget for resident Table handles
    // …

L0 (metadata_cache_bytes) is a soft byte budget over parsed TableMetadata; snapshots + schemas dominate the weight. Raise it if you have many large tables and see repeated metadata parses.
L1.5 (table_handle_cache_bytes) is a byte budget, NOT a handle count. Entries are NOT cheap: a handle pins its parsed metadata AND iceberg's own per-Table ObjectCache, which is why a count cap of 512 could pin ~16 GiB. Each handle's object cache is pinned to TABLE_OBJECT_CACHE_BYTES (4 MiB) and charged at that ceiling, so the budget is a real upper bound.
0 on either disables that layer — every read then falls through to the layer below (parse, or build). The L1 pointer mirror is always on.

What you never have to worry about

There is no cache invalidation to call, no TTL to tune, no staleness window. Because each entry is keyed by its immutable metadata location, a commit that changes a table writes a new file at a new key — the old entry is simply never looked up again and eventually evicted. Reads after a commit see the new pointer immediately via the write-through mirror.