crates/gunnar-delta/src/lib.rs
//! **Git delta compression, the encoder half.**
//!
//! gitoxide can *decode* a delta and it can *copy* one, but it cannot *compute*
//! one. `gix_pack::data::output::entry::iter_from_counts` has exactly one
//! `Mode`, `PackCopyAndBaseObjects`: it reuses the deltas already sitting in a
//! pack and treats everything else as a base object. `gix_pack::data::delta`
//! contains `apply` and `decode_header_size` and no encode counterpart, and
//! `iter_from_counts::Options` has no window or depth field, only a `TODO`
//! asking whether one of its numbers *could* become the window size. There is no
//! delta encoder in gitoxide at all. This crate is that missing half.
//!
//! # Shape
//!
//! Zero dependencies on the project this was written for: only `gix-hash`,
//! `gix-object` and `gix-features`. `Options` carries `thread_limit:
//! Option<usize>`, progress goes through `gix_utils::progress`, cancellation
//! is a `&AtomicBool` named `should_interrupt`, errors are `thiserror`, licence
//! is `MIT OR Apache-2.0`. It is meant to read as gitoxide's own, and to be
//! liftable out of its tree with `cp -r`.
//!
//! # Use
//!
//! One pair at a time:
//!
//! ```
//! let base = b"the quick brown fox jumps over the lazy dog, repeatedly".repeat(4);
//! let mut target = base.clone();
//! target.extend_from_slice(b" and then stops");
//! let delta = gunnar_delta::delta(&base, &target)?;
//! assert!(delta.len() < target.len() / 2);
//! # Ok::<_, gunnar_delta::Error>(())
//! ```
//!
//! A whole object set, which is what a repack wants:
//!
//! ```no_run
//! use std::sync::atomic::AtomicBool;
//! use gunnar_delta::{deltify, Object, Options};
//!
//! # fn main() -> Result<(), gunnar_delta::Error> {
//! # let objects: Vec<Object<'_>> = Vec::new();
//! let mut progress = gix_utils::progress::Discard;
//! let out = deltify(&objects, &Options::default(), &mut progress, &AtomicBool::default())?;
//! for entry in &out {
//! match &entry.data {
//! Some(delta) => println!("delta of {} bytes at depth {}", delta.len(), entry.depth),
//! None => println!("base object"),
//! }
//! }
//! # Ok(()) }
//! ```
//!
//! Entries come back in the order a pack writer wants them: every base appears
//! before the deltas that reference it, and `Deltified::base` indexes into the
//! returned vector.
//!
//! # The sort key is worth more than the encoder
//!
//! [`Object::name_hash`] is the primary key after the object kind, and it
//! decides which objects ever share a window — which is to say, which bases can
//! be found at all. Leaving it `0` is legal and is the size-ordered search this
//! crate started with. Filling it in from [`name_hash::v2`] is what two
//! independent published measurements of `git repack -adf` credit with taking
//! microsoft/fluentui from 439 MiB to 160 MiB (and 96 s to 34 s), with nothing
//! else changed. See [`name_hash`].
//!
//! # Correctness, and why it is stated before performance
//!
//! A wrong delta produces a pack that does not reconstruct, and the failure
//! surfaces as a corrupt repository at some later clone, arbitrarily far from
//! the cause. Three things push that failure back to where it can be seen:
//!
//! * the emitter counts the bytes its own instructions will produce and refuses
//! to return a delta whose total is not exactly the target length
//! ([`Error::Inconsistent`]);
//! * every delta in the test suite is reconstructed by **gix's own decoder**,
//! through a real packfile opened with `gix_pack::data::File`, never by an
//! applier belonging to this crate, of which there is deliberately none;
//! * stock `git index-pack --strict` and `git fsck --full --strict` accept a
//! pack built from these deltas, with every object byte-compared back through
//! `git cat-file`, and the negative controls assert that a corrupted delta is
//! *rejected* and that fsck actually saw the objects rather than walking an
//! empty store.
//!
//! # Refusal is a feature
//!
//! A sliding window over a large object set is the exact shape in which a
//! plausible parameter becomes an out-of-memory kill: `window × object size ×
//! threads` is a product and every factor is caller-supplied. [`Limits`] bounds
//! each one, [`Limits::validate`] runs before the first allocation, and an
//! out-of-bounds request comes back as [`Error::Refused`] naming the field, the
//! value and the cap. Nothing is silently clamped.
#![deny(missing_docs, rust_2018_idioms, unsafe_op_in_unsafe_fn)]
#![forbid(clippy::undocumented_unsafe_blocks)]
mod encode;
mod error;
mod index;
mod limits;
pub mod name_hash;
pub mod scan;
mod window;
pub use encode::{delta, delta_into, Encode};
pub use error::{Error, Result};
pub use limits::Limits;
pub use window::{deltify, Deltified, Object, Options};