Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support git references with commits that are not on a branch but are fetchable nevertheless #8817

Merged
merged 4 commits into from
Sep 1, 2020

Conversation

sbidoul
Copy link
Member

@sbidoul sbidoul commented Aug 28, 2020

We attempt a git fetch when the requested revision looks like a commit and it is not available locally.

closes #8815

@pradyunsg pradyunsg closed this Aug 28, 2020
@pradyunsg pradyunsg reopened this Aug 28, 2020
@pradyunsg
Copy link
Member

Close-reopened for CI.

@sbidoul sbidoul force-pushed the improve-git-checkout branch from 1dda515 to 3a48731 Compare August 30, 2020 09:48
Before we were fetching only revisions starting with refs/.
Now we also fetch revisions that look like commit that we
don't have locally.
@sbidoul sbidoul force-pushed the improve-git-checkout branch from 3a48731 to 5797a08 Compare August 30, 2020 09:52
@sbidoul
Copy link
Member Author

sbidoul commented Aug 30, 2020

Thanks for the review @uranusjr and @pradyunsg. This version should now read better.

tests/functional/test_vcs_git.py Outdated Show resolved Hide resolved
Co-authored-by: Pradyun Gedam <[email protected]>
@pradyunsg pradyunsg merged commit a991904 into pypa:master Sep 1, 2020
@sbidoul sbidoul deleted the improve-git-checkout branch September 1, 2020 15:47
aalexanderr added a commit to aalexanderr/fetchcode that referenced this pull request Aug 23, 2021
WARNING: fetchcode's vcs will not work on this commit.
This is result of separating pip's code from changes made in scope of
this repository.

This should make it easier to track potentially replicated issues from
pip when taking their vcs pkg.

It also made cleaning up easier, due to some maintenance activities done
in pip:
- dropping Python 2 & 3.5 support
  pypa/pip#9189
- modernized code after above - partially done, tracked in:
  pypa/pip#8802
- added py3.9 support
- updated vendored libraries (e.g. fixing CVE-2021-28363)
  multiple PRs

pip._internal.vcs (and related code) changes:
- Fetch resources that are missing locally:
  pypa/pip#8817
- Improve SVN version parser (Windows)
  pypa/pip#8665
- Always close stderr after subprocess completion:
  pypa/pip#9156
- Remove vcs export feature:
  pypa/pip#9713
- Remove support for git+ssh@ scheme in favour of git+ssh://
  pypa/pip#9436
- Security fix in git tags parsing (CVE-2021-3572):
  pypa/pip#9827
- Reimplement Git version parsing:
  pypa/pip#10117

In next commits, most of pip's internals will be removed from fetchcode,
leaving only vcs module with supporting code (like utils functions,
tests (which will be added & submitted with this change))

This will allow for changes such as ability to add return codes
(probably via futures) from long running downloads and other features.

Switching to having own vcs module might also be a good call due to
pip._internal.vcs close integration with pip's cli in vcs module (some
pip code has been commented out in commit mentioned below)

While generally copy-pasting code without history is bad idea, this
commit follows precedence set in this repo by:
8046215
with exception that all changes to pip's code will be submitted as separate
commits.

It has been agreed with @pombredanne & @TG1999 that history from pip
will be rebased on fetchcode by @pombredanne (thanks!). It will be done
only for the files that are of concern for fetchcode to limit noise in
git history.

I'm leaving this commit without SoB intentionally, as this is not my
work, but that of the many pip's authors:
https://github.com/pypa/pip/blob/21.2.4/AUTHORS.txt
License of pip: MIT (https://pypi.org/project/pip/)
aalexanderr added a commit to aalexanderr/fetchcode that referenced this pull request Oct 1, 2021
Please note that fetchcode's vcs might not work on this branch.
Initially pip was commited without it's history and with few changes
applied. This update approaches this differently- by commiting pip code
in a single commit & applying changes on top of it in separate commits.
While much of pip's code will be stripped from this repository,
the goal of this is to make it easier to take changes from upstream,
even after the code will be modified.
While git-subtree could be used it brings it's own set of issues.

Update should make it easier to track potentially replicated issues from
pip when taking their vcs pkg.

It also made cleaning up easier, due to some maintenance activities done
in pip:
- dropping Python 2 & 3.5 support
  pypa/pip#9189
- modernized code after above - partially done, tracked in:
  pypa/pip#8802
- added py3.9 support
- updated vendored libraries (e.g. fixing CVE-2021-28363)
  multiple PRs

pip._internal.vcs (and related code) changes between
20.1.1 and 21.2.4
- Fetch resources that are missing locally:
  pypa/pip#8817
- Improve SVN version parser (Windows)
  pypa/pip#8665
- Always close stderr after subprocess completion:
  pypa/pip#9156
- Remove vcs export feature:
  pypa/pip#9713
- Remove support for git+ssh@ scheme in favour of git+ssh://
  pypa/pip#9436
- Security fix in git tags parsing (CVE-2021-3572):
  pypa/pip#9827
- Reimplement Git version parsing:
  pypa/pip#10117

In next commits, most of pip's internals will be removed from fetchcode,
leaving only vcs module with supporting code (like utils functions,
tests (which will be submitted alongside this change))

This will allow for changes such as ability to add return codes
(probably via futures) from long running downloads and other features.

Switching to having own vcs module might also be a good call due to
pip._internal.vcs integration with pip's cli in vcs module (some
pip code has been commented out in commit mentioned below)

While generally copy-pasting code (rather than using
submodules/subtrees etc) makes it harder to track, my git-foo is not great
enough for me to attempt regrafting subset of pips history that is of
note from fetchcode perspective.
It has been agreed with @pombredanne & @TG1999 that history from pip
will be regrafted on fetchcode by @pombredanne (thanks!). It will be done
only for the files that are of concern for fetchcode to limit noise in
git history.

The code submitted in scope of this commit is work of many pip's authors
that can bee seen here:
https://github.com/pypa/pip/blob/21.2.4/AUTHORS.txt

Pip is licensed under MIT (https://pypi.org/project/pip/)

Signed-off-by: Alexander Mazuruk <[email protected]>
aalexanderr added a commit to aalexanderr/fetchcode that referenced this pull request Oct 4, 2021
Please note that fetchcode's vcs might not work on this branch.
Initially pip was commited without it's history and with few changes
applied. This update approaches this differently- by commiting pip code
in a single commit & applying changes on top of it in separate commits.
While much of pip's code will be stripped from this repository,
the goal of this is to make it easier to take changes from upstream,
even after the code will be modified.
While git-subtree could be used it brings it's own set of issues.

Update should make it easier to track potentially replicated issues from
pip when taking their vcs pkg.

It also made cleaning up easier, due to some maintenance activities done
in pip:
- dropping Python 2 & 3.5 support
  pypa/pip#9189
- modernized code after above - partially done, tracked in:
  pypa/pip#8802
- added py3.9 support
- updated vendored libraries (e.g. fixing CVE-2021-28363)
  multiple PRs

pip._internal.vcs (and related code) changes between
20.1.1 and 21.2.4
- Fetch resources that are missing locally:
  pypa/pip#8817
- Improve SVN version parser (Windows)
  pypa/pip#8665
- Always close stderr after subprocess completion:
  pypa/pip#9156
- Remove vcs export feature:
  pypa/pip#9713
- Remove support for git+ssh@ scheme in favour of git+ssh://
  pypa/pip#9436
- Security fix in git tags parsing (CVE-2021-3572):
  pypa/pip#9827
- Reimplement Git version parsing:
  pypa/pip#10117

In next commits, most of pip's internals will be removed from fetchcode,
leaving only vcs module with supporting code (like utils functions,
tests (which will be submitted alongside this change))

This will allow for changes such as ability to add return codes
(probably via futures) from long running downloads and other features.

Switching to having own vcs module might also be a good call due to
pip._internal.vcs integration with pip's cli in vcs module (some
pip code has been commented out in commit mentioned below)

While generally copy-pasting code (rather than using
submodules/subtrees etc) makes it harder to track, my git-foo is not great
enough for me to attempt regrafting subset of pips history that is of
note from fetchcode perspective.
It has been agreed with @pombredanne & @TG1999 that history from pip
will be regrafted on fetchcode by @pombredanne (thanks!). It will be done
only for the files that are of concern for fetchcode to limit noise in
git history.

The code submitted in scope of this commit is work of many pip's authors
that can bee seen here:
https://github.com/pypa/pip/blob/21.2.4/AUTHORS.txt

Pip is licensed under MIT (https://pypi.org/project/pip/)

Signed-off-by: Alexander Mazuruk <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Git URL with a sha that is unreachable via tag or branch is not installable
3 participants