Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance on repositories with a lot of tags #3585

Merged

Conversation

AntonMTolmachev
Copy link
Contributor

@AntonMTolmachev AntonMTolmachev commented Jun 9, 2023

Description

Added a tag SHA cache into IncrementStrategyFinder to prevent read of all tags from repository each time when FindCommitMessageIncrement is called.

Related Issue

Fixes #3212 Fixes #2881 Fixes #2885

Motivation and Context

GitVersion runs slowly on repositories with a lot of tags when there is no gitversion_cache available.
Each tag produces a base version which goes through IncrementStrategyFinder, which in turn reads all tags from repository. This causes N^2 tags to be read from repository in total.

How Has This Been Tested?

Manually checked performance improvement via profiler on a repository with 200 commits & tags. See screenshot attached below.
A test that recreates a repository with a similar structure is provided in pull request.

Screenshots (if appropriate):

image
Performance before (right) and after (left).

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@arturcic
Copy link
Member

arturcic commented Jun 9, 2023

@AntonMTolmachev that is an impressive improvement, let's see the tests pass, then we can continue

@AntonMTolmachev AntonMTolmachev marked this pull request as ready for review June 9, 2023 18:10
@arturcic arturcic merged commit c338f48 into GitTools:main Jun 10, 2023
@mergify
Copy link
Contributor

mergify bot commented Jun 10, 2023

Thank you @AntonMTolmachev for your contribution!

@AntonMTolmachev AntonMTolmachev deleted the cache-tags-in-increment-strategy-finder branch June 20, 2023 11:36
@sreenivas-ps
Copy link

@AntonMTolmachev : I've the same problem with one of my repo that has lots of tags, what exactly I should do to prevent reading 200+ tags? At the moment, I'm using this:

      - name: Install GitVersion
        uses: gittools/actions/gitversion/[email protected]
        with:
          versionSpec: '5.x'
      - name: Determine Version
        id: gitversion
        uses: gittools/actions/gitversion/[email protected]
        with:
          useConfigFile: true
          targetPath: ${{ inputs.targetPath }}
          configFilePath: ${{ inputs.configFilePath }}

@arturcic
Copy link
Member

@sreenivas-ps this should be fixed in 6.0.0-beta3

      - name: Install GitVersion
        uses: gittools/actions/gitversion/[email protected]
        with:
          versionSpec: '6.0.0-beta.3'
          includePrerelease: true

@sreenivas-ps
Copy link

@arturcic : thanks. I've tried using that version before but it is even taking long than 5.x. For each branch in the repo, it is cycling 236 tags ( to find merge base between) the PR and branch and then moves to next branch to find the same...

e.g..

End: Finding merge base between 'pull/1723/merge' and 'develop'. (Took: 109,096.17ms)
End: Finding merge base between 'pull/1723/merge' and 'feature/DATAENG-0000_Quick_fix_backward_compatibility'. (Took: 109,958.75ms)
End: Finding merge base between 'pull/1723/merge' and 'feature/DATAENG_4662_Experian_Files_2023_06_09_05_56_32'. (Took: 114,400.10ms)

I'm not sure if i'm doing anything wrong. Here is my Gitversion.yaml file and new to this gitversion.yaml

assembly-versioning-scheme: MajorMinorPatch
mode: Mainline
branches: {}
ignore:
  sha: []
merge-message-formats: {}

My repo has about 236 tags.
With 5.x, it was taking around 9 to 12 mins but with 6.0.0-beta3, it is running with 35mins and still going on.

@AntonMTolmachev
Copy link
Contributor Author

@sreenivas-ps I think this is a different problem from the one that appeared on my repository.

The log above mentions slow performance while finding merge base between branches (Finding merge base between ...), which comes from MergeBaseFinder.FindMergeBaseOf function.

I optimized the call highlighted in the left part (IncrementStrategyFinder), and it looks totally unrelated to MergeBaseFinder.FindMergeBaseOf call.

image

I could only suggest to create a new issue with reproduction steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants