Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

initial merge-base support #1557

Merged
merged 14 commits into from
Aug 26, 2024
Merged

initial merge-base support #1557

merged 14 commits into from
Aug 26, 2024

Conversation

Byron
Copy link
Member

@Byron Byron commented Aug 25, 2024 •
edited
Loading

Translate tests from the Git test suite and assure that we produce the same results.

Tasks

  • Git baseline tests
  • implementation that can pass the baseline tests (needs handling of redundant commits)
  • borrowed cache for performance in multi-queries
  • more baseline tests
  • permutation tests to really hit it
  • Repository::merge_base()
  • gix merge-base for real-world scenarious
    • Test git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e on the linux kernel

FollowUp

  • Optimize graph usage so that multiple merge-base runs can re-use each others graphs and thus save a lot of time for performance improvements.

Research

The commit-graph greatly accelerates the operation.

❯ hyperfine -w2 'git -c core.commitgraph=false merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e'
Benchmark 1: git -c core.commitgraph=false merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):     953.6 ms ±  10.8 ms    [User: 903.0 ms, System: 45.9 ms]
  Range (min … max):   942.5 ms … 980.0 ms    10 runs

Benchmark 2: git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):      21.8 ms ±   1.4 ms    [User: 7.3 ms, System: 12.3 ms]
  Range (min … max):    19.5 ms …  27.7 ms    119 runs

Summary
  git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e ran
   43.68 ± 2.85 times faster than git -c core.commitgraph=false merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e

Performance Comparison

gix without commit-graph is exactly as fast as Git. however, with commit-graph Git is 1.6x faster.

❯ hyperfine -w2 'git -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'git merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'gix -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'gix merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e'
Benchmark 1: git -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):     641.8 ms ±   6.4 ms    [User: 600.6 ms, System: 36.9 ms]
  Range (min … max):   631.4 ms … 651.6 ms    10 runs

Benchmark 2: git merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):      62.4 ms ±   3.2 ms    [User: 45.2 ms, System: 14.7 ms]
  Range (min … max):    58.0 ms …  74.1 ms    49 runs

Benchmark 3: gix -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):     643.7 ms ±   6.1 ms    [User: 600.6 ms, System: 39.3 ms]
  Range (min … max):   636.8 ms … 657.7 ms    10 runs

Benchmark 4: gix merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):      99.2 ms ±   1.4 ms    [User: 83.2 ms, System: 13.1 ms]
  Range (min … max):    96.0 ms … 101.9 ms    30 runs

Summary
  git merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e ran
    1.59 ± 0.08 times faster than gix merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
   10.28 ± 0.54 times faster than git -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
   10.31 ± 0.54 times faster than gix -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e

@Byron Byron force-pushed the merge-base branch 4 times, most recently from bdfc717 to 4fc6af8 Compare August 26, 2024 09:04
@Byron
Allow empty-docs
beba720
@Byron
Add first baseline tests for merge-base support
3718a67 
Translate first tests from the Git test suite and set up expectations.
@Byron
feat: add Graph::insert_commit() to easily set flags and get a call…
87c98cd 
…back with looked up commit data.
@Byron
feat: add Graph::insert_parents_with_lookup() and `Graph::insert_da…
638ff28 
…ta()`.

It's another way to insert parents, and tuned for the needs of merge-base.
`insert_data()` makes it easy to insert data for a single commit.
@Byron
feat: first basic implementation of merge_base().
cf83349
@Byron
feat!: Graph now borrows the commitgraph cache.
dfedf6a 
That way, operations that need a commit-graph (graph data structure)
can be run more efficiently as they won't consume the graph, but
can reuse it.
@Byron Byron force-pushed the merge-base branch 2 times, most recently from a6b4db3 to 5ae0985 Compare August 26, 2024 11:20
@Byron Byron marked this pull request as ready for review August 26, 2024 13:57
@Byron
feat: add Repository::merge_base()
3abf043 
A simple method to obtain the merge-base between two commits.
@Byron
feat: gix merge-base for the CLI
7249291 
For now it only supports the standard merge-base, but more derivatives
can be added easily on demand.
@Byron
feat: add graph::LazyCommit::generation_and_timestamp()
60f0e00 
That way, the cost for creating intermediate commit objects are
cut in half.
@Byron Byron force-pushed the merge-base branch 4 times, most recently from df06b24 to 3d57f82 Compare August 26, 2024 16:00
@Byron
feat: add Graph::len() and Graph::is_empty()
6d8612c 
Get an idea about he size of the graph.
@Byron
improve performance of commit-graph variant ever-so-slightly
18c2fc4 
In practice, what really matters is to reuse as much of the
previous computation as possible.

Re-using the graph (or its menory) doesn't do much, but reusing
the parsed commits would be a huge benefit when running many
queries after another.
@Byron Byron enabled auto-merge August 26, 2024 17:26
@Byron Byron merged commit 649f588 into main Aug 26, 2024
15 checks passed
@Byron Byron deleted the merge-base branch August 26, 2024 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Byron