osm-katana — crate card
Reverse-engineered (from source — verify, don't trust the name)
- One-line pitch: A fast OSM (XML/PBF) → GeoParquet/PBF converter with parallel decompression, both a library and CLI.
- Does: Reads OpenStreetMap data (planet-scale XML or PBF), decodes it with parallel codecs, resolves node coordinates and admin-boundary relations, converts geometry to Arrow/WKB, and writes GeoParquet or re-encoded PBF. Public surface includes
xml_to_pbf,geo2arrow,optimize/OptimizeOptions,load_admin_boundaries_pbf/PbfAdminRelation,PbfWayRecord,side_outputs, and aparmodule. Big internals:xml_reader(2466 LoC),node_store(1145, coord store),writer(1122, parquet),xml_to_pbf(1034),optimize(859). - Key deps: intra-workspace
znippy-zoomies(vtd/stree),lgz(gzip),lbzip2(bzip2),zstd-sys-rs(zstd); arrow + parquet 58, flatbuffers, protobuf (+ protobuf-codegen build dep), clap CLI, indicatif progress, flate2, ahash, memchr, memmap2. Optionallibrqbit+tokio behindplanet-fixture/devto torrent the OSM planet fixture. - Shape: lib + 3 bins (
osm-katana,vtd_bench,planet_bench); ~12k LoC. Features:search-index,planet-fixture/dev. (pmtileswas declared and gated zerocfgsites; deleted 2026-08-22.) Uses a build.rs (protobuf codegen into OUT_DIR). - Observed role in the workspace: The top-level application/consumer of the workspace; it is the only crate that pulls in all the codec crates plus znippy-zoomies. Re-exports znippy-zoomies modules under legacy names (
xml_vtd,stree64) for back-compat with priorosm2geoparquetcode.
Fill in (one line each — only the human knows these)
- Primary consumer(s):
- Why it exists (vs the obvious alternative):
- Hard constraint (airgap / no-C-deps / no-FFI / pure-Rust / etc.):
- Release cadence & independence (rides a parent? own repo someday?):
- Status (production / prototype / experimental / stubbed):
- Must-NOT-change invariant:
- Biggest known gap or TODO:
- Anything a newcomer would get wrong about it: