time to update python-dateutil again #1872
Comments
This is what we normally have, but this was change we had to make today (#1871) due to a recent dateutil release. This should be a temporary fix until we can fix our installers and revert back to a This would only be an issue if you're installing something else that is pulling in the latest version of dateutils. If I just use Here's the background on this if you're curious: There was a changed merge into dateutils a while back that moved constants out from setup.py to setup.cfg (dateutil/dateutil#880). The consequence of this is that you need a modern version of setuptools for this to work properly. Our bundled installer is using a version of setuptools that doesn't support this and as a result won't install correctly (the linked botocore PR has shows the specific error). In order to unblock today's release, we set an upper bound of |
|
Thanks! Appreciate the background, I was a bit curious. I'm also a bit curious about your use of py2.7 (lol) given it's EOL in less than 2 months (Jan 2020 is gonna be a good month, py2 & win7 FINALLY!). I'm concerned about how you could be using such an old version of setuptools. It has handled cfg files since 30.3.0 (released 8 Dec 2016). That's about 3 years old right now is I hope all I need to say about that. So I guess the solution for now is that one has to manually add For those reading, here's the commit showing how laziness has ruined your day: e87e7a7 |
|
watching this, capping is bad, especially on 'core' libraries. Blocking all update on boto til this is fixed :( https://review.opendev.org/692951 |
boto is capping deps agian boto/botocore#1872 Change-Id: Ica0cc41508895f5126b70533212c809f2c828c8a Signed-off-by: Matthew Thode <[email protected]>
* Update requirements from branch 'master'
- Updated from generate-constraints
boto is capping deps agian boto/botocore#1872
Change-Id: Ica0cc41508895f5126b70533212c809f2c828c8a
Signed-off-by: Matthew Thode <[email protected]>
|
@jamesls were you able to come up with an ETA? |
|
FWIW, 1.13.14 still has this issue (which I got pulled in from the latest boto3 update): |
|
This is preventing me from installing erpnext now, when installing erpnext I am running into this and it's failing. I tried to install an old version of python-dateutil before installing erpnext but it still runs into this error: ERROR: botocore 1.13.14 has requirement python-dateutil<2.8.1,>=2.1; python_version >= "2.7", but you'll have python-dateutil 2.8.1 which is incompatible. Looks like a fix is needed sooner rather than later as people are affected by this. |
swetashre
added
the
enhancement
* Work around issue #9 * This workaround is super slow!!! * The root cause is boto/botocore#1872
|
Even botocore 1.13.23 has this bug. Please fix it asap. ERROR: botocore 1.13.23 has requirement python-dateutil<2.8.1,>=2.1; python_version >= "2.7", but you'll have python-dateutil 2.8.1 which is incompatible. |
to address issue with current release of botocore See boto/botocore#1872
Pins python-dateutil to a version <= 2.8.0. python-dateutil's latest version is 2.8.1, but our botocore dependency is incompatible with that version (see boto/botocore#1872 for more details). We can remove this once the botocore dependency has been fixed. - I also added a `lib/Pipfile.locks/README.md`, to help avoid future confusion about the purpose of that directory - And I removed `lib/Pipfile.locks/python-3.7.3`, because we're no longer testing against that version of Python in CircleCI. Fixes #644
|
@jamesls, you wrote:
This is causing more and more problems and conflicts as other packages update to the newer
|
|
Is this all because OSX's native Python 2.7 (which eventually won't even be included with OSX at all) is failing to run setup? Per the link @Tecktron shared earlier, that seems to be at least part of the problem. Beyond 2020-01-01, there should be no reason to hold this back for the sake of a distribution that will be EOL and soon thereafter defunct as a default package on OSX. |
|
If OSX is an officially supported package, I'd expect it to have at least build job on Travis to propose PRs with fixes. |
|
As far as I can tell the issue didn't appear in Travis - it appeared somewhere else. My guess is it's OS X, but without more information I can't be sure what broke that led to them pinning this in the first place (let alone what they think would solve it). |
|
I am trying to tell that Travis supports OS X and build job for OS X installers where could help pinpoint the problem. If sources for OS X installers is here, of course. |
|
I understand. I want to hear from the person who pinned this dependency what installers they plan to upgrade to fix this properly - and when. |
Remove nonexistent content from package_data entry. Bound py-dateutil as <2.8.1 because of issues with botocore as mentioned in boto/botocore#1872
|
I'm working on this today. I'll update this with more details once I get a fix implemented and tested, but I plan to have an update by the end of the day. |
|
Awesome! This will make a lot of people quite happy. :-) |
|
EOD update: So I have a fix that I've been experimenting with that I think fixes our issues. I've verified the changes on py26/py27 but there seems to be issues on py3. I'm trying to track down if these are related to my changes or not. Once I can confirm it's not due to my changes I'll go ahead with the PR. Should finish all this testing tomorrow. I realize this is affecting quite a few people so this is my main priority right now. |
- Updating boto3 (1.9.128 -> 1.10.37)
https://github.com/boto/boto3/blob/1.10.37/CHANGELOG.rst
- Updating botocore (1.12.130 -> 1.13.37)
https://github.com/boto/botocore/blob/1.13.37/CHANGELOG.rst
- Updating docutils (0.12 -> 0.15.2)
http://docutils.sourceforge.net/RELEASE-NOTES.html
- Updating python-dateutil (2.7.3 -> 2.8.0)
https://github.com/dateutil/dateutil/blob/2.8.1/NEWS
- Updating s3transfer (0.2.0 -> 0.2.1)
https://github.com/boto/s3transfer/blob/0.2.1/CHANGELOG.rst
- Updating six (1.11.0 -> 1.13.0)
https://github.com/benjaminp/six/blob/1.13.0/CHANGES
---
Boto3 1.9.128's dependency tree:
boto3 1.9.128 The AWS SDK for Python
|-- botocore >=1.12.128,<1.13.0
| |-- docutils >=0.10
| |-- jmespath >=0.7.1,<1.0.0
| |-- python-dateutil >=2.1,<3.0.0
| | `-- six >=1.5
| `-- urllib3 >=1.20,<1.25
|-- jmespath >=0.7.1,<1.0.0
`-- s3transfer >=0.2.0,<0.3.0
`-- botocore >=1.12.36,<2.0.0
|-- docutils >=0.10
|-- jmespath >=0.7.1,<1.0.0
|-- python-dateutil >=2.1,<3.0.0
| `-- six >=1.5
`-- urllib3 >=1.20,<1.25
The boto3 (1.9 -> 1.10) and botocore (1.12 -> 1.13) minor version
numbers were incremented due to Boto removing its vendored copy of
requests. Otherwise no significant changes to behavior.
urllib3 was not updated from 1.24.3, despite 1.25.7 being available,
because our pinned version of requests wants urllib3 (>=1.21.1,<1.25).
Similarly, python-dateutil is now stuck at 2.8.0 because botocore wants
python-dateutil (>=2.1,<2.8.1). Relaxing that upper bound is tracked at
boto/botocore#1872.
All of the changelogs look benign, and Boto is fairly self-contained.
However, python-dateutil is also used by elasticsearch-dsl, and six is
used in a few other dependencies.
- Updating boto3 (1.9.128 -> 1.10.37)
https://github.com/boto/boto3/blob/1.10.37/CHANGELOG.rst
- Updating botocore (1.12.130 -> 1.13.37)
https://github.com/boto/botocore/blob/1.13.37/CHANGELOG.rst
- Updating docutils (0.12 -> 0.15.2)
http://docutils.sourceforge.net/RELEASE-NOTES.html
- Updating python-dateutil (2.7.3 -> 2.8.0)
https://github.com/dateutil/dateutil/blob/2.8.1/NEWS
- Updating s3transfer (0.2.0 -> 0.2.1)
https://github.com/boto/s3transfer/blob/0.2.1/CHANGELOG.rst
- Updating six (1.11.0 -> 1.13.0)
https://github.com/benjaminp/six/blob/1.13.0/CHANGES
---
Boto3 1.9.128's dependency tree:
boto3 1.9.128 The AWS SDK for Python
|-- botocore >=1.12.128,<1.13.0
| |-- docutils >=0.10
| |-- jmespath >=0.7.1,<1.0.0
| |-- python-dateutil >=2.1,<3.0.0
| | `-- six >=1.5
| `-- urllib3 >=1.20,<1.25
|-- jmespath >=0.7.1,<1.0.0
`-- s3transfer >=0.2.0,<0.3.0
`-- botocore >=1.12.36,<2.0.0
|-- docutils >=0.10
|-- jmespath >=0.7.1,<1.0.0
|-- python-dateutil >=2.1,<3.0.0
| `-- six >=1.5
`-- urllib3 >=1.20,<1.25
The boto3 (1.9 -> 1.10) and botocore (1.12 -> 1.13) minor version
numbers were incremented due to Boto removing its vendored copy of
requests. Otherwise no significant changes to behavior.
urllib3 was not updated from 1.24.3, despite 1.25.7 being available,
because our pinned version of requests wants urllib3 (>=1.21.1,<1.25).
Similarly, python-dateutil is now stuck at 2.8.0 because botocore wants
python-dateutil (>=2.1,<2.8.1). Relaxing that upper bound is tracked at
boto/botocore#1872.
All of the changelogs look benign, and Boto is fairly self-contained.
However, python-dateutil is also used by elasticsearch-dsl, and six is
used in a few other dependencies.
- Updating boto3 (1.9.128 -> 1.10.37)
https://github.com/boto/boto3/blob/1.10.37/CHANGELOG.rst
- Updating botocore (1.12.130 -> 1.13.37)
https://github.com/boto/botocore/blob/1.13.37/CHANGELOG.rst
- Updating docutils (0.12 -> 0.15.2)
http://docutils.sourceforge.net/RELEASE-NOTES.html
- Updating python-dateutil (2.7.3 -> 2.8.0)
https://github.com/dateutil/dateutil/blob/2.8.1/NEWS
- Updating s3transfer (0.2.0 -> 0.2.1)
https://github.com/boto/s3transfer/blob/0.2.1/CHANGELOG.rst
- Updating six (1.11.0 -> 1.13.0)
https://github.com/benjaminp/six/blob/1.13.0/CHANGES
---
Boto3 1.9.128's dependency tree:
boto3 1.9.128 The AWS SDK for Python
|-- botocore >=1.12.128,<1.13.0
| |-- docutils >=0.10
| |-- jmespath >=0.7.1,<1.0.0
| |-- python-dateutil >=2.1,<3.0.0
| | `-- six >=1.5
| `-- urllib3 >=1.20,<1.25
|-- jmespath >=0.7.1,<1.0.0
`-- s3transfer >=0.2.0,<0.3.0
`-- botocore >=1.12.36,<2.0.0
|-- docutils >=0.10
|-- jmespath >=0.7.1,<1.0.0
|-- python-dateutil >=2.1,<3.0.0
| `-- six >=1.5
`-- urllib3 >=1.20,<1.25
The boto3 (1.9 -> 1.10) and botocore (1.12 -> 1.13) minor version
numbers were incremented due to Boto removing its vendored copy of
requests. Otherwise no significant changes to behavior.
urllib3 was not updated from 1.24.3, despite 1.25.7 being available,
because our pinned version of requests wants urllib3 (>=1.21.1,<1.25).
Similarly, python-dateutil is now stuck at 2.8.0 because botocore wants
python-dateutil (>=2.1,<2.8.1). Relaxing that upper bound is tracked at
boto/botocore#1872.
All of the changelogs look benign, and Boto is fairly self-contained.
However, python-dateutil is also used by elasticsearch-dsl, and six is
used in a few other dependencies.
* poetry update
Updates colorama, a Pytest dependency for supporting ANSI escape
sequences on Windows.
pytest 3.1.3 pytest: simple powerful testing with Python
|-- colorama *
|-- py >=1.4.33
`-- setuptools *
We do not use colorama directly in Kuma.
There are no actual code changes between colorama 0.4.1 and 0.4.3; they
were metadata-only updates:
tartley/colorama#243 (comment)
* Audit constraints
Recategorized 11 dependencies that were mistakenly marked as
constraints, but actually used directly in Kuma.
Removed a duplicate entry for `urlwait`, which was leftover from our
previous system for tracking dependencies.
* Update pytz
- Updating pytz (2018.9 -> 2019.3)
https://github.com/stub42/pytz/blob/release_2019.3/tz/NEWS
No meaningful code changes, but keeps our timezone info current.
Speaking of, I highly recommend reading the zoneinfo database some time,
super interesting historical perspective. More relevantly, Brazil
cancelled Daylight Savings Time since we last updated our copy of pytz.
* Unpin and update Celery's dependencies
- Updating amqp (2.4.2 -> 2.5.2)
https://github.com/celery/py-amqp/blob/2.5.2/Changelog
- Updating billiard (3.6.0.0 -> 3.6.1.0)
https://github.com/celery/billiard/blob/3.6.1/CHANGES.txt
- Updating kombu (4.5.0 -> 4.6.7)
https://github.com/celery/kombu/blob/4.6.7/Changelog.rst
We're already on the latest version of Celery itself.
---
Celery 4.3.0's dependency tree:
celery 4.3.0 Distributed Task Queue.
|-- billiard >=3.6.0,<4.0
|-- kombu >=4.4.0,<5.0
| `-- amqp >=2.5.2,<2.6
| `-- vine >=1.1.3,<5.0.0a1
|-- pytz >0.0-dev
`-- vine >=1.3.0
No red flags in the changelogs, and with the exception of pytz, none of
Celery's dependencies appear elsewhere in our dependency tree.
* Unpin and update Boto-related constraints
- Updating boto3 (1.9.128 -> 1.10.37)
https://github.com/boto/boto3/blob/1.10.37/CHANGELOG.rst
- Updating botocore (1.12.130 -> 1.13.37)
https://github.com/boto/botocore/blob/1.13.37/CHANGELOG.rst
- Updating docutils (0.12 -> 0.15.2)
http://docutils.sourceforge.net/RELEASE-NOTES.html
- Updating python-dateutil (2.7.3 -> 2.8.0)
https://github.com/dateutil/dateutil/blob/2.8.1/NEWS
- Updating s3transfer (0.2.0 -> 0.2.1)
https://github.com/boto/s3transfer/blob/0.2.1/CHANGELOG.rst
- Updating six (1.11.0 -> 1.13.0)
https://github.com/benjaminp/six/blob/1.13.0/CHANGES
---
Boto3 1.9.128's dependency tree:
boto3 1.9.128 The AWS SDK for Python
|-- botocore >=1.12.128,<1.13.0
| |-- docutils >=0.10
| |-- jmespath >=0.7.1,<1.0.0
| |-- python-dateutil >=2.1,<3.0.0
| | `-- six >=1.5
| `-- urllib3 >=1.20,<1.25
|-- jmespath >=0.7.1,<1.0.0
`-- s3transfer >=0.2.0,<0.3.0
`-- botocore >=1.12.36,<2.0.0
|-- docutils >=0.10
|-- jmespath >=0.7.1,<1.0.0
|-- python-dateutil >=2.1,<3.0.0
| `-- six >=1.5
`-- urllib3 >=1.20,<1.25
The boto3 (1.9 -> 1.10) and botocore (1.12 -> 1.13) minor version
numbers were incremented due to Boto removing its vendored copy of
requests. Otherwise no significant changes to behavior.
urllib3 was not updated from 1.24.3, despite 1.25.7 being available,
because our pinned version of requests wants urllib3 (>=1.21.1,<1.25).
Similarly, python-dateutil is now stuck at 2.8.0 because botocore wants
python-dateutil (>=2.1,<2.8.1). Relaxing that upper bound is tracked at
boto/botocore#1872.
All of the changelogs look benign, and Boto is fairly self-contained.
However, python-dateutil is also used by elasticsearch-dsl, and six is
used in a few other dependencies.
* Update requests and unpin its dependencies
- Updating certifi (2018.10.15 -> 2019.11.28)
certifi/python-certifi@2019.03.09...2019.11.28
- Updating idna (2.7 -> 2.8)
https://github.com/kjd/idna/blob/v2.8/HISTORY.rst
- Updating requests (2.20.1 -> 2.22.0)
https://github.com/psf/requests/blob/v2.22.0/HISTORY.md
- Updating urllib3 (1.24.3 -> 1.25.7)
https://github.com/urllib3/urllib3/blob/1.25.7/CHANGES.rst
---
Requests 2.20.1's dependency tree:
requests 2.20.1 Python HTTP for Humans.
|-- certifi >=2017.4.17
|-- chardet >=3.0.2,<3.1.0
|-- idna >=2.5,<2.8
`-- urllib3 >=1.21.1,<1.25
Updating requests also allowed urllib3 to update; it was unpinned in the previous commit.
All of the changes look reasonably minimal.
* Unpin and update click
- Updating click (6.7 -> 7.0)
https://github.com/pallets/click/blob/7.0/CHANGES.rst
Only used by dennis, which we use for linting locale files
* Update pyquery
- Updating pyquery (1.4.0 -> 1.4.1)
https://github.com/gawel/pyquery/blob/1.4.1/CHANGES.rst
* Update cssselect
- Updating cssselect (1.0.3 -> 1.1.0)
https://github.com/scrapy/cssselect/blob/v1.1.0/CHANGES
* Update lxml
- Updating lxml (4.2.5 -> 4.4.2)
https://github.com/lxml/lxml/blob/lxml-4.4.2/CHANGES.txt
As a bonus, lxml >= 4.4.1 provides wheels for CPython 3.8 on Linux,
which should speed up our Docker image builds.
* Unpin and update google-api-python-client
- Installing cachetools (3.1.1)
https://github.com/tkem/cachetools
- Updating google-api-python-client (1.5.1 -> 1.7.11)
https://github.com/googleapis/google-api-python-client/blob/v1.7.11/CHANGELOG
- Installing google-auth (1.8.2)
https://github.com/googleapis/google-auth-library-python
- Installing google-auth-httplib2 (0.0.3)
https://github.com/GoogleCloudPlatform/google-auth-library-python-httplib2
- Updating httplib2 (0.9.2 -> 0.14.0)
https://github.com/httplib2/httplib2/blob/v0.14.0/CHANGELOG
- Updating oauth2client (3.0.0 -> 4.1.3)
https://github.com/googleapis/oauth2client/blob/v4.1.3/CHANGELOG.md
- Updating pyasn1 (0.1.9 -> 0.4.8)
https://github.com/etingof/pyasn1/blob/v0.4.8/CHANGES.rst
- Updating pyasn1-modules (0.0.8 -> 0.2.7)
https://github.com/etingof/pyasn1-modules/blob/v0.2.8/CHANGES.txt
- Updating rsa (3.4.2 -> 4.0)
https://github.com/sybrenstuvel/python-rsa/blob/version-4.0/CHANGELOG.txt
- Updating uritemplate (0.6 -> 3.0.0)
https://github.com/python-hyper/uritemplate/blob/3.0.0/HISTORY.rst
---
google-api-python-client 1.5.1's dependency tree:
google-api-python-client 1.5.1 Google API Client Library for Python
|-- httplib2 >=0.8,<1
|-- oauth2client *
| |-- httplib2 >=0.9.1
| |-- pyasn1 >=0.1.7
| |-- pyasn1-modules >=0.0.5
| | `-- pyasn1 >=0.1.8 (circular dependency aborted here)
| |-- rsa >=3.1.4
| | `-- pyasn1 >=0.1.3 (circular dependency aborted here)
| `-- six >=1.6.1
|-- six >=1.6.1,<2
`-- uritemplate >=0.6,<1
google-api-python-client 1.7.11's dependency tree:
google-api-python-client 1.7.11 Google API Client Library for Python
|-- google-auth >=1.4.1
| |-- cachetools >=2.0.0,<3.2
| |-- pyasn1-modules >=0.2.1
| | `-- pyasn1 >=0.4.6,<0.5.0
| |-- rsa >=3.1.4,<4.1
| | `-- pyasn1 >=0.1.3 (circular dependency aborted here)
| |-- setuptools >=40.3.0
| `-- six >=1.9.0
|-- google-auth-httplib2 >=0.0.3
| |-- google-auth *
| | |-- cachetools >=2.0.0,<3.2
| | |-- pyasn1-modules >=0.2.1
| | | `-- pyasn1 >=0.4.6,<0.5.0
| | |-- rsa >=3.1.4,<4.1
| | | `-- pyasn1 >=0.1.3 (circular dependency aborted here)
| | |-- setuptools >=40.3.0
| | `-- six >=1.9.0
| `-- httplib2 >=0.9.1
|-- httplib2 >=0.9.2,<1dev
|-- six >=1.6.1,<2dev
`-- uritemplate >=3.0.0,<4dev
Note the removal of oauth2client and forking of httplib2.
Both are explicitly discouraged.
__oauth2client:__
> The long-term replacement for `oauth2client` is this library,
> `google-auth`. This library addresses the major issues with oauthclient
>
> https://google-auth.readthedocs.io/en/latest/oauth2client-deprecation.html
__httplib:__
> `httplib` has lots of problems such as lack of threadsafety and
> insecure usage of TLS. Using it is highly discouraged. This library is
> intended to help existing users of `oauth2client` migrate to
> `google-auth`.
>
> https://github.com/GoogleCloudPlatform/google-auth-library-python-httplib2
We do not directly use pyasn1, pyasn1-modules, rsa, or uritemplate, so
allowing those to freely update according to google-api-python-client's
constraints.
* Revert "Update pyquery"
This reverts commit c30a60e.
Pyquery 1.4.1 is preserving leading whitespace in a way that breaks our
tests. Revert for now and deal with in a separate PR.
Any update? (I see the PR above (and the one it's connected to) but it's not clear if these are pending review or test.) |
* check() fails unless a patch is added to python-botocore for upstream issue boto/botocore#1872 git-svn-id: file:///srv/repos/svn-community/svn@527084 9fca08f4-af9d-4005-b8df-a31f2cc04f65
Looks like it's that time to update again.
ERROR: botocore 1.13.9 has requirement python-dateutil<2.8.1,>=2.1; python_version >= "2.7", but you'll have python-dateutil 2.8.1 which is incompatible.Why not pin the upper bound to a major version instead of minor patch version?
The text was updated successfully, but these errors were encountered: