crates/gunnar-client/src/attributes.rs
//! **A `gunnar` checkout that holds the raw blob where a `.gitattributes`
//! content filter is declared must not pretend it was filtered.**
//!
//! # The wrong checkout this exists to stop
//!
//! A repository that uses Git-LFS carries, in its tree, a 131-byte text file
//! per large asset:
//!
//! ```text
//! version https://git-lfs.github.com/spec/v1
//! oid sha256:6287e9e54d555d802e714aee1a0362e5467d51b99cc07f8627cd4b9a1df1e98a
//! size 200000
//! ```
//!
//! and a `.gitattributes` line naming the driver that turns it back into the
//! asset on the way out of the object database:
//!
//! ```text
//! *.bin filter=lfs diff=lfs merge=lfs -text
//! ```
//!
//! `gunnar clone` over SSH materialises the worktree with
//! `nornir_git::gitio::materialize_worktree`. Until nornir-git 0.1.11 that
//! wrote each index blob's bytes and nothing else — no `smudge`, no eol
//! conversion, no `ident` — so a user who cloned an LFS repository with
//! `gunnar` got 131-byte text files where the binaries should be, and until
//! this module existed **nothing said so**. A build then links the pointer.
//!
//! Since nornir-git 0.1.11 (edda 142c637) the writer is gix's own checkout and
//! runs the configured smudge driver, as git does. The pointer still lands
//! wherever no driver is configured, or the driver hands the stored blob back
//! (git-lfs under `GIT_LFS_SKIP_SMUDGE=1`) — and that checkout is exactly as
//! wrong for a build, so the guard below stays on both clone paths.
//!
//! # Measured, 2026-08-05, git 2.53.0 + git-lfs 3.7.1, both object formats
//!
//! One fixture repository per hash — `*.bin filter=lfs`, a 200 000-byte asset
//! stored as a 131-byte pointer blob — checked out four ways:
//!
//! | writer | sha1 | sha256 |
//! |---|---|---|
//! | `git clone` (the judge) | 200000 | 200000 |
//! | `gix` `main_worktree`, which `nornir-git` uses for a **local** URL | 200000 | 200000 |
//! | `materialize_worktree`, which `gunnar clone` uses over **SSH** | **131** | **131** |
//! | `git clone` with the driver absent and `filter.lfs.required=true` | *refused*, no file at all | *refused* |
//!
//! Two things follow, and both shape this module.
//!
//! **First: the declaration is not the defect.** gix's own checkout speaks the
//! `filter-process` protocol and produces the right bytes, so a guard that fired
//! on "this path declares `filter=lfs`" would refuse a checkout that is
//! perfectly correct. What is being detected is *"gunnar wrote the raw blob"*,
//! and the only honest test for that is the applied output: **the file on disk
//! is byte-identical to the blob in the index**. That is what
//! [`unfiltered_paths`] compares.
//!
//! **Second: git's verdict is already written down.** `filter.lfs.required=true`
//! is what `git lfs install` puts in `/etc/gitconfig`, and with it git refuses
//! to write unfiltered content at all:
//!
//! ```text
//! fatal: big.bin: smudge filter lfs failed
//! warning: Clone succeeded, but checkout failed.
//! ```
//!
//! exit 128, and the clone directory holds the objects, the refs and no files.
//!
//! # What gunnar does, and why this and not the other two
//!
//! Three answers were possible.
//!
//! * *Run the filters.* Not ruled out, and the table above says it is closer
//! than it looks: `gix::Repository::filter_pipeline` already speaks the
//! protocol. The missing piece is that the SSH path's worktree writer lives in
//! `nornir-git`, and adding a second writer here is what one implementation
//! forbids. That is a `nornir-git` change, not a `gunnar-client` one.
//! * *Write pointers silently.* The one answer that is not acceptable.
//! * *Refuse, in git's contract* — what this module does.
//!
//! So: **a required driver is refused**, with git's own sentence, and the
//! unfiltered files are removed, because a 131-byte file a build will happily
//! open is not made safe by an error message printed next to it. **A declared
//! driver that is not required is a loud `warn!`** naming every path — git is
//! silent in that case, and this is the deliberate difference.
//!
//! There is no new flag: the escape hatch is git's own knob, read from the same
//! config git reads.
//!
//! ```text
//! git config --global filter.lfs.required false # accept pointers, loudly
//! git -C <dest> restore --source=HEAD :/ # let git finish the job
//! ```
//!
//! The second line is git's own suggested recovery, and it was verified to turn
//! the 131 bytes back into 200 000.
//!
//! # Where gunnar stops short of git
//!
//! git abandons the **whole** checkout: `plain.txt` is missing from its clone
//! too, not only `big.bin` (measured in the same run). gunnar removes only the
//! paths it could not filter and keeps the rest. A deliberate divergence, stated
//! rather than hidden — the verdict is git's, the blast radius is smaller.
use std::collections::BTreeSet;
use std::path::Path;
use crate::{Error, Result};
/// One worktree path that declares a content filter **and holds the raw blob**,
/// i.e. one path where no filter ran.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct FilteredPath {
/// The path, relative to the worktree root, in git's slash form.
pub path: String,
/// The driver named by the `filter` attribute — `lfs`, in practice.
pub driver: String,
/// `filter.<driver>.required`, read from the repository's **effective** git
/// configuration (system, global and local, in git's own precedence). True
/// is what `git lfs install` writes into `/etc/gitconfig`.
pub required: bool,
}
/// Every path in a checkout that gunnar wrote unfiltered.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct FilterReport {
/// Sorted, one entry per affected path.
pub paths: Vec<FilteredPath>,
}
impl FilterReport {
/// Nothing in the checkout was written unfiltered.
#[must_use]
pub fn is_empty(&self) -> bool {
self.paths.is_empty()
}
/// The paths whose driver is `required`, i.e. the ones git would refuse to
/// write at all.
pub fn required(&self) -> impl Iterator<Item = &FilteredPath> {
self.paths.iter().filter(|p| p.required)
}
/// The distinct driver names, so a message can name them rather than count
/// them.
#[must_use]
pub fn drivers(&self) -> BTreeSet<&str> {
self.paths.iter().map(|p| p.driver.as_str()).collect()
}
/// The one sentence both the refusal and the warning are built from, so the
/// two cannot drift apart (one implementation).
///
/// `dest` appears in it because the recovery command needs a path, and a
/// recovery command a user cannot paste is not a recovery command.
#[must_use]
pub fn message(&self, dest: &Path) -> String {
let drivers: Vec<&str> = self.drivers().into_iter().collect();
let shown: Vec<&str> = self
.paths
.iter()
.take(10)
.map(|p| p.path.as_str())
.collect();
let more = self.paths.len().saturating_sub(shown.len());
let tail = if more == 0 {
String::new()
} else {
format!(" (and {more} more)")
};
let n = self.paths.len();
format!(
"{n} path{} in this checkout hold{} the raw blob rather than the content: \
filter={} on {}{} — the driver is not configured here, or it handed the stored \
blob back (git-lfs does under GIT_LFS_SKIP_SMUDGE). \
Stock git materialises them with `git -C {} restore --source=HEAD :/`.",
if n == 1 { "" } else { "s" },
if n == 1 { "s" } else { "" },
drivers.join(","),
shown.join(", "),
tail,
dest.display()
)
}
}
/// Every path in `repo`'s checkout whose attributes name a `filter` driver
/// **and** whose bytes on disk are identical to the blob — the applied-output
/// test for "no filter ran here".
///
/// The attribute lookup is gix's own stack — `.gitattributes` at every level,
/// `info/attributes`, `core.attributesFile` — rather than a second
/// `.gitattributes` parser written here (one implementation). `required` comes
/// from the same config snapshot git reads.
///
/// # The one false negative, named
///
/// A filter whose output happens to equal its input (a `text` file already in
/// the right line endings, an `ident` blob with no `$Id$`) is indistinguishable
/// from one that did not run, and is reported as clean. That is the right call:
/// the bytes on disk are the bytes git would have written, which is the whole
/// question. For LFS it cannot arise — a pointer is never its own asset.
pub fn unfiltered_paths(repo: &gix::Repository, dest: &Path) -> Result<FilterReport> {
use gix::attrs::StateRef;
use gix::index::entry::Mode;
let index = repo
.index_or_empty()
.map_err(|e| Error::repo("open the index to read .gitattributes", e))?;
let mut stack = repo
.attributes_only(
&index,
gix::worktree::stack::state::attributes::Source::WorktreeThenIdMapping,
)
.map_err(|e| Error::repo("assemble the attribute stack", e))?;
let collection = stack.attributes_collection().clone();
let mut outcome = gix::attrs::search::Outcome::default();
outcome.initialize_with_selection(&collection, ["filter"]);
let config = repo.config_snapshot();
let mut paths = Vec::new();
for entry in index.entries() {
// A gitlink has no blob to filter; a directory entry is not a file.
if entry.mode.is_submodule() || entry.mode == Mode::DIR {
continue;
}
let rela = entry.path(&index);
let driver = {
let platform = stack
.at_entry(rela, Some(entry.mode))
.map_err(|e| Error::repo(format!("attributes for {rela}"), e))?;
outcome.reset();
platform.matching_attributes(&mut outcome);
outcome
.iter_selected()
.find_map(|m| match m.assignment.state {
// `filter` unset, `-filter` or `!filter`: no driver to run.
StateRef::Value(v) if !v.as_bstr().is_empty() => Some(v.as_bstr().to_string()),
_ => None,
})
};
let Some(driver) = driver else { continue };
// The applied output. A path git never wrote (its checkout was refused,
// or the file was deleted) is not an unfiltered path.
// gix 0.90: a repository path that is no path on this host is fallible.
let Ok(rel_path) = gix::path::from_bstr(rela) else {
continue;
};
let on_disk = match std::fs::read(dest.join(rel_path.as_ref())) {
Ok(bytes) => bytes,
Err(_) => continue,
};
let blob = repo
.find_object(entry.id)
.map_err(|e| Error::repo(format!("read the blob for {rela}"), e))?;
if blob.data != on_disk {
// A filter ran: the worktree bytes are not the object's bytes.
continue;
}
let required = config
.boolean(format!("filter.{driver}.required").as_str())
.unwrap_or(false);
paths.push(FilteredPath {
path: rela.to_string(),
driver,
required,
});
}
paths.sort();
paths.dedup();
Ok(FilterReport { paths })
}
/// The policy, written once: refuse a required filter, warn about the rest.
///
/// Called from **both** gunnar code paths that materialise a worktree over SSH
/// ([`crate::fetch::clone_or_fetch_with`] and [`crate::clone::clone_with_io`]),
/// so no verb can decide differently. On refusal the affected paths are
/// **removed** from `dest` first — see the module docs for why an error message
/// beside a 131-byte pointer is not enough.
///
/// It takes a path and opens the repository itself, deliberately. A caller that
/// handed in the `gix::Repository` it had been holding since before the checkout
/// would hand in a **cached, empty index**, and this guard would then walk no
/// entries and report nothing — green, and blind. Reading the freshly written
/// index is not optional here, so it is not left to the caller.
///
/// Returns the report on success, so a caller that wants to say more can, and so
/// a test can assert on what was found rather than on the absence of a panic.
pub fn guard_checkout(dest: &Path) -> Result<FilterReport> {
let repo = gix::open(dest)
.map_err(|e| Error::repo(format!("reopen {} after checkout", dest.display()), e))?;
let report = unfiltered_paths(&repo, dest)?;
if report.is_empty() {
return Ok(report);
}
let message = report.message(dest);
let Some(first) = report.required().next() else {
tracing::warn!("{message}");
return Ok(report);
};
let driver = first.driver.clone();
for p in report.required() {
// Best effort by design: failing to remove one is not a reason to
// swallow the refusal, and git's own failed checkout leaves nothing
// there either.
let _ = std::fs::remove_file(dest.join(&p.path));
}
Err(Error::repo_msg(format!(
"{message} filter.{driver}.required is true, so those files were removed rather than \
left as raw blobs — the same contract git enforces with `fatal: smudge filter \
{driver} failed`. Set `filter.{driver}.required=false` to accept them unfiltered."
)))
}
#[cfg(test)]
mod tests {
use super::*;
/// The message must name the driver and the path, because a count is not a
/// diagnosis. Asserted on the rendered string, which is what a user reads.
///
/// Seen RED by returning `self.paths.len().to_string()` from `message`:
/// *"the message names neither the driver nor the path: 1"*.
#[test]
fn the_message_names_the_driver_and_the_path() {
let report = FilterReport {
paths: vec![FilteredPath {
path: "assets/big.bin".into(),
driver: "lfs".into(),
required: true,
}],
};
let m = report.message(Path::new("/tmp/x"));
assert!(
m.contains("lfs") && m.contains("assets/big.bin"),
"the message names neither the driver nor the path: {m}"
);
assert!(
m.contains("restore --source=HEAD"),
"the message gives no recovery command: {m}"
);
assert!(
m.contains("/tmp/x"),
"the recovery command cannot be pasted, it names no directory: {m}"
);
}
/// Eleven paths must not print eleven paths, and must not silently print ten
/// either — the count is in the sentence and the overflow is named.
#[test]
fn a_long_list_is_truncated_and_says_so() {
let paths = (0..11)
.map(|i| FilteredPath {
path: format!("a{i}.bin"),
driver: "lfs".into(),
required: true,
})
.collect();
let m = FilterReport { paths }.message(Path::new("/tmp/x"));
assert!(m.contains("11 paths"), "the count went missing: {m}");
assert!(m.contains("and 1 more"), "the overflow is unnamed: {m}");
}
}