gix-pack: add a benchmark for pack generation from loose objects

[Amey Pawar (ameyypawar)]

What

For #2611. Adds the first gix-pack benchmark — a harness for pack generation from many loose objects — so the performance and memory questions raised there can be grounded in numbers, as Axel Ibarrondo (@cachel2) and I discussed in the thread. This is the measurement step before any code changes, not a fix for the underlying performance itself.

The harness

gix-pack/benches/pack_generation.rs separates the three phases of a gix pack create-style generation and reports, per phase, both timing (criterion) and the peak heap allocation, plus the resulting pack size:

• discover — walk the odb (odb.iter()) to enumerate loose object ids.
• count — count::objects turns those ids into a Vec<output::Count> (loads each object's header).
• write — iter_from_counts → FromEntriesIter resolves locations, sorts, and encodes the counts into a pack byte stream.

Note on phase boundaries: the counts sort and pack-location resolve happen inside iter_from_counts — i.e. in the write phase, not in count.

Wiring: a [[bench]] target, criterion/tempfile dev-deps, and the parallel feature on the dev-dep (iter_from_counts requires a Send object-database handle). No new crates enter Cargo.lock — both deps are already used elsewhere in the workspace.

What it shows (example run; flat, unique blobs)

objects	discover	count	write	pack
1k	101 KiB	271 KiB	4.0 MiB	61 KiB
10k	843 KiB	2.8 MiB	6.4 MiB	623 KiB
50k	3.2 MiB	11.0 MiB	13.2 MiB	3.1 MiB

Discovery and count memory are O(N) but modest (~64 B and ~225 B per object); write peak grows sub-linearly with object count for this fixture; the pack is the un-deltified baseline.

Honest limits (also in the bench's doc comment)

• the peak figures are peak heap allocation — a lower bound on RSS (excludes thread stacks, memory-mapped pack pages, allocator retention);
• a global tracking allocator adds a small constant overhead to the timings, and write is multi-threaded (wall-clock varies with cores);
• the fixture is flat near-identical blobs written to a discarding sink (no writer back-pressure), and AsIs input with no pre-existing pack never attempts delta compression — so the pack size is an un-deltified baseline; quantifying the delta gap against git would need a git-packed baseline of the same objects.

Verification

• Compiles and runs; cargo clippy -p gix-pack --benches and cargo fmt --check are clean.
• CI compile-checks it via cargo clippy --workspace --all-targets.
• It's the first benchmark in gix-pack, so the object counts are kept modest to stay runnable; raise OBJECT_COUNTS locally to probe larger, more degenerate repositories.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0be0f4ef45

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "Codex (@codex) address that feedback".

[chatgpt-codex-connector]

Delegate realloc in the tracking allocator

For benchmark phases dominated by Vec growth (discover/count collect object lists and write builds chunks), this GlobalAlloc impl changes the allocator behavior because the trait's default realloc allocates a new block, copies, and frees the old block instead of using System.realloc. That inflates both timings and the recorded peak live bytes the benchmark is meant to measure, so the allocator should delegate realloc (and adjust the counters for the size delta) rather than relying on the default.

Useful? React with 👍 / 👎.