Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dependencies] update 3rd party dependencies #407

Open
wants to merge 46 commits into
base: master
Choose a base branch
from

Conversation

apalala
Copy link
Contributor

@apalala apalala commented Sep 8, 2021

Remove pins to very old versions that produce conflicts with other tools used alongside Scrapy (#404).

Remove requirements.in and rely on setup.py.install_requires.

Remove pip from dependencies as mentioning it is deemed unsafe.

Upgrade all libraries through pip-compile --upgrade.

Hopefully solve security warnings reported by dependabot.

This PR drops support for Python 2.7, 3.5, and 3.6.

fixes #406
fixes #412

@apalala apalala added enhancement dependencies Pull requests that update a dependency file labels Sep 8, 2021
@apalala apalala self-assigned this Sep 8, 2021
@apalala apalala requested a review from Gallaecio September 8, 2021 16:24
.travis.yml Outdated Show resolved Hide resolved
setup.py Outdated Show resolved Hide resolved
shub/utils.py Outdated Show resolved Hide resolved
@apalala apalala added the WIP label Sep 8, 2021
.bumpversion.cfg Outdated Show resolved Hide resolved
.github/workflows/python-app.yml Outdated Show resolved Hide resolved
Comment on lines 32 to 35
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use tox instead of using flake8 directly, and keep this within the tox configuration so that running tox -e flake8 has the same effect, and users can run that locally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Github Actions were an experiment. We don't need them now, so I'll remove them.

.travis.yml Show resolved Hide resolved
setup.py Outdated Show resolved Hide resolved
setup.py Show resolved Hide resolved
tests/test_utils.py Show resolved Hide resolved
@hugovk
Copy link

hugovk commented Sep 16, 2021

Tip: add python_requires=">=3.6", to setup.py to help pip know which library version to install for end users.

.bumpversion.cfg Show resolved Hide resolved
.travis.yml Show resolved Hide resolved
requirements.txt Show resolved Hide resolved
setup.py Outdated Show resolved Hide resolved
Comment on lines -40 to +39
'tqdm==4.55.1',
'tqdm',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably keep some limit, assuming we do not support just any version.

setup.py Show resolved Hide resolved
virtualenv==20.7.2
# via pipenv
virtualenv-clone==0.5.7
# via pipenv

# The following packages are considered to be unsafe in a requirements file:
# pip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It probably makes sense to remove this file in favor of tox.ini.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed this on Slack.

requirements.txt and requirements-test.txt serve to document the versions of the libraries that passed the tests when the branch was merged.

@@ -505,6 +505,7 @@ def get_project_dir():
os.path.join(basepath, 'a', 'b'))


@unittest.skip('broken by changes in `click.invoke(input=)`')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we should either address this in code or limit the upper version of click to a version we support in setup.py.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. This test is complicated yet unimportant, so not enough reason to downgrade click.

Also, there should be only minor changes on this release.

I'll leave this comment open to remind us to search or post an issue against click.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file enhancement WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

drop support for Python 2.7, 3.5, 3.6 Add support for Python 3.10
3 participants