Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dependencies] update 3rd party dependencies #407

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
9225f3a
[setup.py] remove 'pip' from requires, unsafe
apalala Sep 8, 2021
05ed6c1
[dependencies] pip-compile --upgrade setup.py and tests/requirements.in
apalala Sep 8, 2021
25a5446
[dependencies] pip compile --upgrade setup.py
apalala Sep 8, 2021
f4a9ffc
[dependencies] remove requirements.in as it duplicates setup.py
apalala Sep 8, 2021
450b73d
[dependencies] enable TravisCI on branch
apalala Sep 8, 2021
402dcf4
[cli][bug] fix incorrect parameter passed for `click.prompt(default=)`
apalala Sep 8, 2021
878a967
[travisci] use Python 3.9 + apply to all branches
apalala Sep 8, 2021
3dd95a7
[setup.py] unpin click
apalala Sep 8, 2021
9342d08
[ci] add github pipeline/action for builds
apalala Sep 8, 2021
4df40f0
[dependencies] pin click==7.1.2 for Python 2.7 support
apalala Sep 8, 2021
8df9044
[travisci] restore active branches
apalala Sep 8, 2021
81c3fc8
[github] install other Python versions in action
apalala Sep 8, 2021
d6577e5
[github] bugfix in actions
apalala Sep 8, 2021
02489bf
[tox] drop py36
apalala Sep 8, 2021
0f4558f
[tox] drop py27
apalala Sep 8, 2021
8e316dd
[github] try to install other Python versions
apalala Sep 8, 2021
02bc076
Bump version: 2.13.0 → 2.14.0
apalala Sep 8, 2021
f4f98ba
update from incorrect commit in master
apalala Sep 8, 2021
109ee19
Merge branch 'master' into update_dependencies
apalala Sep 8, 2021
6e8caa9
[tests][cli] skip tests involving `click.invoke(input=)`
apalala Sep 8, 2021
616cdd7
[dependencies] unpin dependencies in setup.py
apalala Sep 8, 2021
8c4b490
[setup.py] drop support for 2.7, 3.5, 3.6
apalala Sep 8, 2021
911cd58
[versioning] revert version changes
apalala Sep 9, 2021
db8f39a
[ci] remove experimental Github Actions
apalala Sep 9, 2021
abf7150
[setup.py] revert version change + unpin requirements
apalala Sep 9, 2021
7b83121
[versioning] restore version
apalala Sep 9, 2021
015be38
[test] include py310
apalala Oct 20, 2021
3213221
[requirements] update + force updated python-scrapinghub from branch
apalala Oct 20, 2021
3811292
Bump version: 2.13.0 → 3.0.0
apalala Oct 20, 2021
55e4131
[setup.py] patch for branch version of python-scrapinghub
apalala Oct 20, 2021
d99ed0f
[setup.py] revert latest change
apalala Oct 20, 2021
227cc64
[setup.py] pin click to >=7.0
apalala Oct 23, 2021
b47afd0
[tox] test on py310
apalala Oct 23, 2021
fc441d4
Merge remote-tracking branch 'refs/remotes/origin/update_dependencies…
apalala Oct 23, 2021
c586b58
[test] clean up tox.ini
apalala Oct 23, 2021
b3f158a
[setup.py] fix syntax error
apalala Oct 23, 2021
cddc023
ignore PyCharm configuration dir .idea/
apalala Oct 23, 2021
cdeaef0
[test] refactor requirments files for library version consistency
apalala Oct 23, 2021
3c98c78
[tests] resolve deprecation in pyyaml.yaml.load()
apalala Oct 23, 2021
fddef8e
[test] resolve self.assertRaisesRegex deprecation
apalala Oct 23, 2021
99f5670
[utils] solve SafeconfigParser deprecation
apalala Oct 23, 2021
b28c03f
[requirements] use the branch version of python-scrapinghub
apalala Oct 23, 2021
962ac7f
[make] requirements-test.txt depends on requirements-test.in
apalala Oct 23, 2021
e1482ac
[tests] disable test_can_deploy_an_egg_from_pypi()
apalala Oct 23, 2021
22d0ad1
[requirements] force branch version of python-scrapinghub
apalala Oct 23, 2021
90886f3
[tests] resolve deprecation warnings
apalala Oct 23, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 2.13.0
current_version = 2.14.0
Gallaecio marked this conversation as resolved.
Show resolved Hide resolved
commit = True
tag = True
tag_name = v{new_version}
Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: shub

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.7
python-version: 3.8
python-version: 3.9
apalala marked this conversation as resolved.
Show resolved Hide resolved
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest tox
pip install -r tests/requirements.txt
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use tox instead of using flake8 directly, and keep this within the tox configuration so that running tox -e flake8 has the same effect, and users can run that locally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Github Actions were an experiment. We don't need them now, so I'll remove them.

- name: Test with tox
run: |
tox

2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
language: python
python: 3.6
python: 3.9
apalala marked this conversation as resolved.
Show resolved Hide resolved
sudo: false
env:
- TOX_ENV=py27
Expand Down
14 changes: 0 additions & 14 deletions requirements.in

This file was deleted.

47 changes: 23 additions & 24 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,43 +1,42 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# pip-compile
# pip-compile setup.py
#
certifi==2020.6.20
certifi==2021.5.30
# via requests
chardet==3.0.4
charset-normalizer==2.0.4
# via requests
click==7.0
# via -r requirements.in
docker==4.2.2
# via -r requirements.in
idna==2.10
click==8.0.1
# via shub (setup.py)
docker==5.0.2
# via shub (setup.py)
idna==3.2
# via requests
pyyaml==5.4
# via -r requirements.in
requests==2.24.0
pyyaml==5.4.1
# via shub (setup.py)
requests==2.26.0
# via
# -r requirements.in
# docker
# scrapinghub
# shub (setup.py)
retrying==1.3.3
# via
# -r requirements.in
# scrapinghub
# shub (setup.py)
scrapinghub==2.3.1
# via -r requirements.in
six==1.15.0
# via shub (setup.py)
six==1.16.0
# via
# -r requirements.in
# docker
# retrying
# scrapinghub
toml==0.10.1
# via -r requirements.in
tqdm==4.55.1
# via -r requirements.in
urllib3==1.25.9
# shub (setup.py)
toml==0.10.2
# via shub (setup.py)
tqdm==4.62.2
# via shub (setup.py)
urllib3==1.26.6
# via requests
websocket-client==0.57.0
websocket-client==1.2.1
# via docker
apalala marked this conversation as resolved.
Show resolved Hide resolved
12 changes: 4 additions & 8 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

setup(
name='shub',
version='2.13.0',
version='2.14.0',
packages=find_packages(exclude=('tests', 'tests.*')),
url=about['DOCS_LINK'],
description='Scrapinghub Command Line Client',
Expand All @@ -29,15 +29,14 @@
include_package_data=True,
zip_safe=False,
install_requires=[
'click==7.0',
'click>=7.1.2',
apalala marked this conversation as resolved.
Show resolved Hide resolved
'docker',
'pip',
'PyYAML',
'pyyaml',
'retrying',
'requests',
'scrapinghub>=2.3.1',
'six>=1.7.0',
'tqdm==4.55.1',
'tqdm>=4.55.1',
'toml',
],
classifiers=[
Expand All @@ -46,9 +45,6 @@
'Natural Language :: English',
'License :: OSI Approved :: BSD License',
'Programming Language :: Python',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
apalala marked this conversation as resolved.
Show resolved Hide resolved
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
Expand Down
2 changes: 1 addition & 1 deletion shub/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = '2.13.0'
__version__ = '2.14.0'


# Links to documentation to use over the project sources
Expand Down
84 changes: 60 additions & 24 deletions tests/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,31 +1,67 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# pip-compile
# pip-compile requirements.in
#
appdirs==1.4.4 # via virtualenv
attrs==19.3.0 # via pytest
coverage==5.1 # via pytest-cov
distlib==0.3.0 # via virtualenv
filelock==3.0.12 # via virtualenv
flake8==3.8.3 # via -r requirements.in
mccabe==0.6.1 # via flake8
mock==3.0.5 # via -r requirements.in
more-itertools==5.0.0 # via pytest
packaging==20.4 # via pytest
pipenv==2020.6.2 # via -r requirements.in
pluggy==0.13.1 # via pytest
py==1.10.0 # via pytest
pycodestyle==2.6.0 # via flake8
pyflakes==2.2.0 # via flake8
pyparsing==2.4.7 # via packaging
pytest-cov==2.9.0 # via -r requirements.in
pytest==5.4.3 # via -r requirements.in, pytest-cov
python-dateutil==2.8.1 # via -r requirements.in
virtualenv-clone==0.5.4 # via pipenv
virtualenv==20.0.21 # via pipenv
wcwidth==0.2.4 # via pytest
attrs==21.2.0
# via pytest
backports.entry-points-selectable==1.1.0
# via virtualenv
certifi==2021.5.30
# via pipenv
coverage==5.5
# via pytest-cov
distlib==0.3.2
# via virtualenv
filelock==3.0.12
# via virtualenv
flake8==3.9.2
# via -r requirements.in
iniconfig==1.1.1
# via pytest
mccabe==0.6.1
# via flake8
mock==4.0.3
# via -r requirements.in
packaging==21.0
# via pytest
pipenv==2021.5.29
# via -r requirements.in
platformdirs==2.3.0
# via virtualenv
pluggy==1.0.0
# via pytest
py==1.10.0
# via
# -r requirements.in
# pytest
pycodestyle==2.7.0
# via flake8
pyflakes==2.3.1
# via flake8
pyparsing==2.4.7
# via packaging
pytest==6.2.5
# via
# -r requirements.in
# pytest-cov
pytest-cov==2.12.1
# via -r requirements.in
python-dateutil==2.8.2
# via -r requirements.in
six==1.16.0
# via
# python-dateutil
# virtualenv
toml==0.10.2
# via
# pytest
# pytest-cov
virtualenv==20.7.2
# via pipenv
virtualenv-clone==0.5.7
# via pipenv

# The following packages are considered to be unsafe in a requirements file:
# pip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It probably makes sense to remove this file in favor of tox.ini.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed this on Slack.

requirements.txt and requirements-test.txt serve to document the versions of the libraries that passed the tests when the branch was merged.

Expand Down
1 change: 1 addition & 0 deletions tests/test_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -505,6 +505,7 @@ def get_project_dir():
os.path.join(basepath, 'a', 'b'))


@unittest.skip('broken by changes in `click.invoke(input=)`')
apalala marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we should either address this in code or limit the upper version of click to a version we support in setup.py.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. This test is complicated yet unimportant, so not enough reason to downgrade click.

Also, there should be only minor changes on this release.

I'll leave this comment open to remind us to search or post an issue against click.

class OnboardingWizardTestCase(unittest.TestCase):

def setUp(self):
Expand Down
2 changes: 1 addition & 1 deletion tox.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[tox]
envlist = py27,py36,py37,py38,py39
envlist = py37,py38,py39

[testenv]
setenv =
Expand Down