Check and update dependencies

[edmorley]

Continuing from:
90cdde8#commitcomment-25074883

I copied out the version lists from setup.py, trimmed the quotes and commas, which gives roughly:

# tests_require
nose==1.3.7
nose-exclude==0.5.0
httmock==1.2.6
rednose==1.2.1
mock==1.0.1
setuptools-lint==0.3
flake8==2.5.0
psutil==2.1.3
hypothesis==3.6.1
pgpy>=0.4.0,<0.5
tox==2.3.2
coverage==4.1b2
python-dateutil==2.6.0

# install_requires
requests>=2.4.3,<3
mohawk>=0.3.4,<0.4
slugid>=1.0.7,<2
six>=1.10.0,<1.11

# python 2 tests_require addition
subprocess32==3.2.6

# python 3 install_requires addition
aiohttp>=2.0.0,<3
async_timeout>=1.2.0,<2

And then pasted that in https://pyup.io/tools/requirements-checker/ , which shows updates for a few things.

It turns out there is an issue open for adding read-only setup.py support to pyup.io:
pyupio/pyup#137

And there's in fact a workaround suggested there for the moment:
pyupio/pyup#137 (comment)

[jhford]

but if we did that, wouldn't we be forcing installation of subprocess32 on py3 and async_timeout/aiohttp on py2?

[edmorley]

The above isn't a proposed list of dependencies to use, it's what I extracted from setup.py for easy pasting in the pyup online checker tool.

[jhford]

ahhhh, i was thinking the idea was to commit that as a requirements.txt file, which we can't do because of the different requirements for python 2 and 3.

Thanks!

[escapewindow]

Per this comment, we're thinking that:

• since taskcluster-client.py infrequently updates its deps, and
• since a number of downstream users use pyup and similar to regularly update dependencies,

it may be beneficial to stop pinning deps in taskcluster-client.py, and push that responsibility down to the users. An alternate solution could involve regularly using pyup or similar to bump dependencies for taskcluster-client.py, run the tests, and land the PR+release tc.py if the tests pass.

[edmorley]

Unpinning works for me :-)

[jhford]

The problem is right now that there's a new version of slugid which is already not compatible with the current version specs of the client. If we just remove the pinning, instead of the knowledge that you need to use slugid <2 to avoid breaking this library being kept in this library, every single caller needs to keep that up to date.

I'm happy to have Pyup keep track of dependencies for this repository. @escapewindow @edmorley are either of you able to help me set up pyup for this repository?

[escapewindow]

The problem is right now that there's a new version of slugid which is already not compatible with the current version specs of the client. If we just remove the pinning, instead of the knowledge that you need to use slugid <2 to avoid breaking this library being kept in this library, every single caller needs to keep that up to date.

Hm. I was in the loop for the slugid breaking change, and I use slugid in scriptworker, so I would possibly have noticed this sooner. I do think that if tc.py didn't pin, I'd blame the slugid breaking change rather than tc.py.

However, we also update the client apis.json and associated methods on release. We'd benefit from regular tc.py releases anyway to keep the apis up to date.

I'm happy to have Pyup keep track of dependencies for this repository. @escapewindow @edmorley are either of you able to help me set up pyup for this repository?

Sure. I can add that to my list.

[danlabici]

To chip in, CiDuty can help with PyUp PRs once they are setup.
We already handle them for:

• build-puppet
• mozilla-version
• treescript
• addonscript

So adding a new PyUp PR to handle will not be an issue for us, as we usually are the first to know when deps/requirements fail to update.

[escapewindow]

My current thoughts are in #130.

[escapewindow]

Unpinned in #130.