Allow a different install_command for (non-)Jenkins environment

[adamantike]

Hi!

This is intended as a feature request, unless there's something I'm missing, and is possible to do at the moment.

Currently, our environments need a local script that calls pip in our behalf, but checking if devpi is running locally, and adding the corresponding pip options if that's the case. As devpi is optional in our DEV environments, it can't be always configured. Let's call this script proxied-pip.sh.

In Jenkins, however, devpi is not used and everything is already configured, so running python -m pip install ... is enough to install project dependencies.

At the moment, the solution we are using is adding another script to the project repository:

#!/bin/bash
if [[ -z "${JENKINS_URL}" ]]
then
    proxied-pip.sh "$@"
else
    python -m pip install "$@"
fi

However, this needs to be replicated in each repository, or added to Jenkins, something we wouldn't like to do; we think there should be a way to simplify this configuration within Tox.

The main issue is that the install_command is set at the testenv level, instead of being a global setting, so we can't use something like:

[tox]
install_command = proxied-pip.sh {opts} {packages}

[tox:jenkins]
install_command = python -m pip install {opts} {packages}

If that could be done, I think that would be the simplest solution, and would allow people to define global install_command settings, both for Jenkins and non-Jenkins environments.

Of course, we should also consider what happens with the whitelist_externals option. Should that be configured generally as...?

[testenv]
whitelist_externals =
    proxied-pip.sh

Opinions and/or suggestions are welcome!
I have the time to work on this, if it's accepted. Of course, I'll need help to reply the previous questions.

[obestwalter]

Hi @adamantike, I doubt that changing the scope of install_command from testenv to global is gonna happen. The tendency is rather the other way around - to have most settings on a testenv level.

About the whitelisting: you do not have to whitelist whatever you use as install command - that check only happens for commands to make sure that you do not accidentally use a tool that happens to be installed int the parent interpreter (although one could argue that we should do the same for all commands ... hasn't come up yet as a problem though yet).

Also just for reference there is also a discussion around the future of install_command over at #715.

[adamantike]

Thanks for your response @obestwalter!

I completely agree with preferring testenv settings over global ones. However, that's indeed the issue here: tox only provides a global section for Jenkins settings ([tox:jenkins]), so as we try to move away from global to testenv, I think we would need a way to configure Jenkins preferences at the testenv level. Otherwise, the current tox:jenkins section will be less and less useful over time.

In my particular scenario, I could simply fix my issue by using environment variable substitution:

install_command = {env:TOX_INSTALL_COMMAND:python -m pip install} {opts} {packages}

And setting the TOX_INSTALL_COMMAND variable just on non-Jenkins environments, to avoid adding extra scripts that must be packaged and sent to the CI, just for Tox to run everywhere.

However, the issue could still be valid, as many testenv options can't be configured just for Jenkins/CI environments.

[obestwalter]

I think this Jenkins section is a bit of a dinosaur anyway - IMO it should be more generic to accommodate other CI systems that might need settings different from local settings. To also get that onto a testenv level it might be worth thinking along the lines of doing things differently depending on the existence/setting of an env var, but this is just a vague thought for now.

[gaborbernat]

I think this could be nicely done by having a plugin to configure this. The plugins in Jenkins can be different than locally. Maybe use the proxied by default locally, and then without remote?