Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update vendored deps & prevent comparison of editable specifiers #3750

Merged
merged 36 commits into from
May 20, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
ff4c8b0
Update vendored deps
techalchemy Apr 18, 2019
1006b50
Fix marker handling
techalchemy Apr 18, 2019
90c2c66
Update pythonfinder, requirementslib and vistir
techalchemy May 11, 2019
289eac3
Update vendored dependencies
techalchemy May 15, 2019
916c53e
Update all vendored dependencies
techalchemy May 15, 2019
aa2b1d2
Update piptools patch and re-lock
techalchemy May 16, 2019
e3bf42e
Merge remote-tracking branch 'origin/master' into update-vendored-deps
techalchemy May 16, 2019
204f689
Refactor resolver into resolver file
techalchemy Apr 18, 2019
e93ce98
Update vendored deps
techalchemy Apr 19, 2019
37a1981
Update echo calls, exception formats, dependencies
techalchemy Apr 19, 2019
8960250
Fix pep517 usage on failure
techalchemy Apr 30, 2019
4118587
Fix pip version parsing
techalchemy Apr 30, 2019
4664925
Update lockfile
techalchemy Apr 30, 2019
4336db9
Fix ``--no-use-pep517`` fallback and environment
techalchemy May 1, 2019
01b0496
Update path
techalchemy May 6, 2019
5310fbd
update python version in pipfile
techalchemy May 6, 2019
06fedde
Update lockfile
techalchemy May 6, 2019
0b600ae
Fix requirementslib marker / installers
techalchemy May 7, 2019
b7de7c1
Fix marker formatting and inclusion
techalchemy May 11, 2019
c6979fd
Update exception handlers
techalchemy May 15, 2019
102b682
Fix keep_outdated markers
techalchemy May 16, 2019
2104959
Fix marker formatting in keep outdated test
techalchemy May 16, 2019
8e1e825
Update vendored pip licenses
techalchemy May 16, 2019
fd91d96
Fix namedtuple implementation
techalchemy May 16, 2019
27c529a
fix namedtuple syntax...
techalchemy May 16, 2019
65751d3
Update dependencies and fix azure pipeline definitions
techalchemy May 16, 2019
cf6305e
Fix piptools patch
techalchemy May 16, 2019
adc8d13
Add dotenv patch to guard typing imports
techalchemy May 16, 2019
d8e66c7
Add future imports to tests
techalchemy May 16, 2019
7e394c3
Fix license download script
techalchemy May 16, 2019
cd4d8f2
Minor requirementslib update
techalchemy May 16, 2019
c23e57b
Fix test and pipeline definitions
techalchemy May 16, 2019
05651cb
Add black config and remove unused azure yaml
techalchemy May 19, 2019
86192bb
make sure we publish test results from test runs
techalchemy May 19, 2019
43d61d8
Update requirementslib with retry for failed wheels
techalchemy May 19, 2019
b06c07c
Update vendoring script
techalchemy May 19, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .azure-pipelines/jobs/run-manifest-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ steps:
export GIT_SSL_CAINFO=$(python -m certifi)
export LANG=C.UTF-8
python -m pip install --upgrade setuptools twine readme_renderer[md]
python setup.py sdist
python setup.py sdist bdist_wheel
twine check dist/*
11 changes: 9 additions & 2 deletions .azure-pipelines/jobs/run-tests-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ steps:

- template: ../steps/install-dependencies.yml

- template: ../steps/create-virtualenv.yml
- template: ../steps/create-virtualenv-windows.yml

- template: ../steps/run-tests.yml
- template: ../steps/run-tests-windows.yml

- task: PublishTestResults@2
displayName: Publish Test Results
inputs:
testResultsFiles: '**/test-results.xml'
testRunTitle: 'Python $(python.version)'
condition: succeededOrFailed()
11 changes: 3 additions & 8 deletions .azure-pipelines/jobs/run-vendor-scripts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,9 @@ jobs:
strategy:
maxParallel: 4
matrix:
${{ if eq(parameters.vmImage, 'vs2017-win2016') }}:
Python37:
python.version: '>= 3.7.2'
python.architecture: x64
${{ if ne(parameters.vmImage, 'vs2017-win2016' )}}:
Python37:
python.version: '>= 3.7'
python.architecture: x64
Python37:
python.version: '>= 3.7'
python.architecture: x64
steps:
- task: UsePythonVersion@0
displayName: Use Python $(python.version)
Expand Down
34 changes: 6 additions & 28 deletions .azure-pipelines/jobs/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,15 @@ jobs:
Python27:
python.version: '2.7'
python.architecture: x64
${{ if eq(parameters.vmImage, 'vs2017-win2016') }}:
# TODO remove once vs2017-win2016 has Python 3.7
Python37:
python.version: '>= 3.7.2'
python.architecture: x64
${{ if ne(parameters.vmImage, 'vs2017-win2016' )}}:
Python37:
python.version: '>= 3.7'
python.architecture: x64
steps:
- ${{ if eq(parameters.vmImage, 'vs2017-win2016') }}:
- template: ./run-tests-windows.yml

- ${{ if ne(parameters.vmImage, 'vs2017-win2016') }}:
- template: ./run-tests.yml

- job: Test_Secondary
displayName: Test python3.6
# Run after Test_Primary so we don't devour time and jobs if tests are going to fail
# dependsOn: Test_Primary
pool:
vmImage: ${{ parameters.vmImage }}
strategy:
maxParallel: 4
matrix:
Python37:
python.version: '>= 3.7'
python.architecture: x64
Python36:
python.version: '3.6'
python.version: '>= 3.6'
python.architecture: x64
steps:
- ${{ if eq(parameters.vmImage, 'vs2017-win2016') }}:
- ${{ if eq(parameters.vmImage, 'windows-2019') }}:
- template: ./run-tests-windows.yml

- ${{ if ne(parameters.vmImage, 'vs2017-win2016') }}:
- ${{ if ne(parameters.vmImage, 'windows-2019') }}:
- template: ./run-tests.yml
4 changes: 3 additions & 1 deletion .azure-pipelines/steps/create-virtualenv-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ steps:
echo "Path $PATH"
echo "Installing Pipenv…"
pipenv install --deploy --dev
pipenv run pip install -e "$(pwd)[test]" --upgrade
pipenv run pip install -e "$(pwd)[tests]" --upgrade
echo pipenv --venv && echo pipenv --py && echo pipenv run python --version
displayName: Make Virtualenv
env:
PYTHONWARNINGS: 'ignore:DEPRECATION'
26 changes: 26 additions & 0 deletions .azure-pipelines/steps/create-virtualenv-windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
steps:

- script: |
echo "##vso[task.setvariable variable=LANG]C.UTF-8"
echo "##vso[task.setvariable variable=PIP_PROCESS_DEPENDENCY_LINKS]1"
displayName: Set Environment Variables

- powershell: |
pip install certifi
$env:PYTHON_PATH=$(python -c "import sys; print(sys.executable)")
$env:CERTIFI_CONTENT=$(python -m certifi)
echo "##vso[task.setvariable variable=GIT_SSL_CAINFO]$env:CERTIFI_CONTENT"
echo "##vso[task.setvariable variable=PY_EXE]$env:PYTHON_PATH"
displayName: Set Python Path
env:
PYTHONWARNINGS: 'ignore:DEPRECATION'

- script: |
echo "Python path: $(PY_EXE)"
echo "GIT_SSL_CAINFO: $(GIT_SSL_CAINFO)"
$(PY_EXE) -m pipenv install --deploy --dev
env:
PIPENV_DEFAULT_PYTHON_VERSION: '$(PIPENV_DEFAULT_PYTHON_VERSION)'
PYTHONWARNINGS: 'ignore:DEPRECATION'
PIPENV_NOSPIN: '1'
displayName: Make Virtualenv
62 changes: 34 additions & 28 deletions .azure-pipelines/steps/create-virtualenv.yml
Original file line number Diff line number Diff line change
@@ -1,31 +1,37 @@
steps:

- powershell: |
$env:PY_EXE=$(python -c "import sys; print(sys.executable)")
if (!$env:PY_EXE) {
$env:PY_EXE="python"
}
Write-Host "##vso[task.setvariable variable=PY_EXE]$env:PY_EXE"
Write-Host "Found Python: $env:PY_EXE"
Invoke-Expression "& '$env:PY_EXE' -m virtualenv D:\.venv"
Write-Host "##vso[task.setvariable variable=VIRTUAL_ENV]D:\.venv"
Invoke-Expression "& 'D:\.venv\Scripts\activate.ps1'"
$env:VIRTUAL_ENV="D:\.venv"
Write-Host "Installing local package..."
Invoke-Expression "& '$env:PY_EXE' -m pip install -e .[test] --upgrade"
Write-Host "upgrading local package in virtual env"
$venv_scripts = Join-Path -path D:\.venv -childpath Scripts
$venv_py = Join-Path -path $venv_scripts -childpath python.exe
Write-Host "##vso[task.setvariable variable=VIRTUAL_ENV_PY]$venv_py"
Invoke-Expression "& '$venv_py' -m pip install -e .[test] --upgrade"
Write-Host "Installing pipenv development packages"
Invoke-Expression "& '$venv_py' -m pipenv install --dev"
Write-Host "Installing local package in pipenv environment"
Invoke-Expression "& '$venv_py' -m pipenv run pip install -e .[test]"
Write-Host "Printing metadata"
Write-Host $(Invoke-Expression "& '$venv_py' -m pipenv --venv")
Write-Host $(Invoke-Expression "& '$venv_py' -m pipenv --py")
Write-Host $(Invoke-Expression "& '$venv_py' -m pipenv run python --version")
displayName: Make Virtualenv
- script: |
echo "##vso[task.setvariable variable=LANG]C.UTF-8"
echo "##vso[task.setvariable variable=PIP_PROCESS_DEPENDENCY_LINKS]1"
displayName: Set Environment Variables

- ${{ if eq(parameters.vmImage, 'windows-2019') }}:
- powershell: |
pip install certifi
$env:PYTHON_PATH=$(python -c "import sys; print(sys.executable)")
$env:CERTIFI_CONTENT=$(python -m certifi)
echo "##vso[task.setvariable variable=GIT_SSL_CAINFO]$env:CERTIFI_CONTENT"
echo "##vso[task.setvariable variable=PY_EXE]$env:PYTHON_PATH"
displayName: Set Python Path
env:
PYTHONWARNINGS: 'ignore:DEPRECATION'
- ${{ if ne(parameters.vmImage, 'windows-2019') }}:
- bash: |
pip install certifi
PYTHON_PATH=$(python -c 'import sys; print(sys.executable)')
CERTIFI_CONTENT=$(python -m certifi)
echo "##vso[task.setvariable variable=GIT_SSL_CAINFO]$CERTIFI_CONTENT"
echo "##vso[task.setvariable variable=PY_EXE]$PYTHON_PATH"
displayName: Set Python Path
env:
PYTHONWARNINGS: 'ignore:DEPRECATION'

- script: |
echo "Python path: $(PY_EXE)"
echo "GIT_SSL_CAINFO: $(GIT_SSL_CAINFO)"
$(PY_EXE) -m pipenv install --deploy --dev
env:
PIPENV_DEFAULT_PYTHON_VERSION: $(PIPENV_DEFAULT_PYTHON_VERSION)
PIPENV_DEFAULT_PYTHON_VERSION: '$(PIPENV_DEFAULT_PYTHON_VERSION)'
PYTHONWARNINGS: 'ignore:DEPRECATION'
PIPENV_NOSPIN: '1'
displayName: Make Virtualenv
4 changes: 3 additions & 1 deletion .azure-pipelines/steps/install-dependencies.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
steps:
- script: 'python -m pip install --upgrade pip setuptools wheel && python -m pip install -e .[test] --upgrade'
- script: 'python -m pip install --upgrade pip setuptools wheel && python -m pip install -e .[tests] --upgrade'
displayName: Upgrade Pip & Install Pipenv
env:
PYTHONWARNINGS: 'ignore:DEPRECATION'
14 changes: 14 additions & 0 deletions .azure-pipelines/steps/run-tests-linux.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
- script: |
# Fix Git SSL errors
git submodule sync && git submodule update --init --recursive
pipenv run pytest --junitxml=test-results.xml
displayName: Run integration tests
env:
PYTHONWARNINGS: 'ignore:DEPRECATION'
PY_EXE: $(PY_EXE)
GIT_SSL_CAINFO: $(GIT_SSL_CAINFO)
LANG: $(LANG)
PIP_PROCESS_DEPENDENCY_LINKS: $(PIP_PROCESS_DEPENDENCY_LINKS)
PIPENV_DEFAULT_PYTHON_VERSION: $(PIPENV_DEFAULT_PYTHON_VERSION)
PYTHONWARNINGS: ignore:DEPRECATION
PIPENV_NOSPIN: '1'
15 changes: 15 additions & 0 deletions .azure-pipelines/steps/run-tests-windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
steps:
- powershell: |
subst T: "$env:TEMP"
Write-Host "##vso[task.setvariable variable=TEMP]T:\"
Write-Host "##vso[task.setvariable variable=TMP]T:\"
displayName: Fix Temp Variable

- script: |
git submodule sync && git submodule update --init --recursive
pipenv run pytest -ra --ignore=pipenv\patched --ignore=pipenv\vendor --junitxml=test-results.xml tests
displayName: Run integration tests
env:
PIPENV_DEFAULT_PYTHON_VERSION: $(PIPENV_DEFAULT_PYTHON_VERSION)
PYTHONWARNINGS: 'ignore:DEPRECATION'
PIPENV_NOSPIN: '1'
38 changes: 19 additions & 19 deletions .azure-pipelines/steps/run-tests.yml
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
steps:
- powershell: |
# Fix Git SSL errors
Invoke-Expression "& '$env:VIRTUAL_ENV_PY' -m pip install certifi"
Invoke-Expression "& '$env:VIRTUAL_ENV_PY' -m certifi > cacert.txt"
Write-Host "##vso[task.setvariable variable=GIT_SSL_CAINFO]$(Get-Content cacert.txt)"
$env:GIT_SSL_CAINFO="$(Get-Content cacert.txt)"
# Shorten paths to get under MAX_PATH or else integration tests will fail
# https://bugs.python.org/issue18199
subst T: "$env:TEMP"
Write-Host "##vso[task.setvariable variable=TEMP]T:\"
$env:TEMP='T:\'
Write-Host "##vso[task.setvariable variable=TMP]T:\"
$env:TMP='T:\'
git submodule sync
git submodule update --init --recursive
Invoke-Expression "& '$env:VIRTUAL_ENV_PY' -m pipenv run pytest -ra --ignore=pipenv\patched --ignore=pipenv\vendor --junitxml=test-results.xml tests"
displayName: Run integration tests
- task: UsePythonVersion@0
inputs:
versionSpec: $(python.version)
architecture: '$(python.architecture)'
addToPath: true
displayName: Use Python $(python.version)

- template: install-dependencies.yml

- script: |
echo '##vso[task.setvariable variable=PIPENV_DEFAULT_PYTHON_VERSION]$(PYTHON_VERSION)'
env:
VIRTUAL_ENV: $(VIRTUAL_ENV)
VIRTUAL_ENV_PY: $(VIRTUAL_ENV_PY)
PYTHON_VERSION: $(python.version)

- template: create-virtualenv.yml

- ${{ if eq(parameters.vmImage, 'windows-2019') }}:
- template: run-tests-windows.yml
- ${{ if ne(parameters.vmImage, 'windows-2019') }}:
- template: run-tests-linux.yml

- task: PublishTestResults@2
displayName: Publish Test Results
Expand Down
2 changes: 1 addition & 1 deletion .azure-pipelines/windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ trigger:
jobs:
- template: jobs/test.yml
parameters:
vmImage: vs2017-win2016
vmImage: windows-2019
13 changes: 2 additions & 11 deletions Pipfile
Original file line number Diff line number Diff line change
@@ -1,22 +1,13 @@
[dev-packages]
pipenv = {path = ".", editable = true, extras = ["test"]}
"flake8" = ">=3.3.0,<4"
sphinx = "<=1.5.5"
twine = "*"
pipenv = {path = ".", editable = true, extras = ["tests", "dev"]}
sphinx-click = "*"
click = "*"
pytest_pypi = {path = "./tests/pytest-pypi", editable = true}
stdeb = {version="*", markers="sys_platform == 'linux'"}
black = {version="*", markers="python_version >= '3.6'"}
pytz = "*"
towncrier = {git = "https://github.com/hawkowl/towncrier.git", editable = true, ref = "master"}
parver = "*"
invoke = "*"
jedi = "*"
isort = "*"
rope = "*"
passa = {editable = true, git = "https://github.com/sarugaku/passa.git"}
bs4 = "*"
passa = {git = "https://github.com/sarugaku/passa.git"}

[packages]

Expand Down
Loading