Allow default name="pypi" source URL to be configured.

[kalebmckale]

Is your feature request related to a problem?

This is addressing the larger issue of pipenv-generated Pipfiles for users with private indices.

Specifically, it's addressing the issue that users may not have access to or wish to access pypi.org in their sources by default. Currently, this requires pipenv to fail, manually removing the source from the generated Pipfile, and re-running pipenv.

I thought that maybe the "PIPENV_PYPI_MIRROR" logic might address this, but the mirror URL is never added to the Pipfile. I can see cases where one wants to temporarily use a mirror and still retain the pypi.org source, but in the case when pypi.org isn't accessible and using a mirror is necessary for sub-dependencies, especially during the locking phase, pipenv will fail.

Describe the solution you'd like

Users should be able to customize the default URL before a Pipfile is generated. I believe this option partially solves using the ~/.pip/pip.conf file situation. I actually like the index-specific installs, especially since some packages in private index share names with those in pypi.org, which has led to problems when letting ~/.pip/pip.conf check both the pypi mirror and private index. So, preferably, I'd rather have a way to just replace the pypi.org default URL with the mirror URL.

Describe alternatives you've considered

I've considered customizing pipenv and then pushing the package to a private repo, but the maintainability of such a solution is not ideal.

Additional context

I'm willing to look into this issue and develop some options, but I think the method by which one changes the default, currently hard-coded, value can vary significantly based on user preferences. My overall goal is to make pipenv easier to use out-of-the-box with little repeated customization by users, which right now, is not possible.

[kalebmckale]

I should note that implementing #5573 could theoretically provide a mechanism for replacing the default source, e.g.

--index="pypi=https://internal.myorg.com/mirror/pypi/simple"

or

--index=pypi:https://internal.myorg.com/mirror/pypi/simple

[kalebmckale]

While doing research, I discovered #2281, which indicates the use of --pypi-mirror is supposed to override the references to pypi.org; however, this only appears to be the case for the specific dependencies in the Pipfile. When it attempts to install dependencies of those or dependencies of those installed from the private index, it uses all sources listed in the Pipfile, including the inaccessible pypi.org and throws an error.

I am asking for consideration of being able to replace the default source in the Pipfile because the goal is to create an easy-to-use process for an entire organization. The current process to get a Pipfile requires, as I indicated above, having pipenv fail, then manually editing the Pipfile to remove or replace the default pypi.org source, and then running pipenv again with the updated Pipfile.

I suppose if pipenv was able to use the ~/.pip/pip.conf file instead, this would also help, but that circumvents the index-dependency security aspect I like about pipenv. Those using pip without pipenv need to have both the internal PyPI mirror and private index in their ~/.pip/pip.conf files. Otherwise, it would be as easy as only specifying the internal PyPI mirror in the configuration file and adding the private index source to Pipfile.

Since there are use cases for using pip outside pipenv, or rather it would be a tough sell to insist everyone always uses pipenv for every installation, the only solution I can come up with is to replace the default source. It would be nice to be able to do this on the initial run of pipenv or have some way to configure pipenv to set the default values that are currently hardcoded.

[matteius]

@kalebmckale I think if I can get back around to finishing up this PR, which would use the default pip.conf index for the default index but I think it would still supply the index-dependency restrictions for all packages. #5297

[kalebmckale]

@matteius Hey! If you can make it work, I'm very happy! The one thing I've learned is that pipenv is quite a beautiful and complex piece of machinery. So, anyone who has mastered its internal moving components is much more qualified than me. 🙂

[matteius]

I possibly like the idea of being able to specify an alternative default index in another way too--I have been really busy with my primary job but I should get back to that PR soon. Thanks for your support!

[matteius]

@kalebmckale I just updated the pipenv branch pip_conf_index-url and I would love if you could give it a whirl and see if it solves all or part of your use case! Please let us know, I've requested it be reviewed for merge.

[kalebmckale]

@matteius One additional observation I had while testing your branch but don't know if this is intentional or a bug is how the pip.conf file is used outside the changes you just made. At first, I thought not at all, but when I swapped the index-url and extra-index-url (so I could see if the trusted-host changes worked), I noticed it was using the extra-index-url during the installation steps but not during locking steps or necessarily the other indices specified in Pipfile (I need to double-check this for certainty but I'm pretty sure the index was already in the Pipfile) but does seem to use the index-url during both. So, when I swapped them back for the same test (PyPI mirror as index-url and private repo for other), it worked for both stages. Seems a bit hard to describe, and it was outside of the scope for my testing at the time so I didn't do extensive testing. However, I wanted to put it on your radar in case you already know something about this, and I'll do some more targeted testing.

[matteius]

@kalebmckale Its likely that the pip.conf is being used during install to supply extra urls. This would definitely be true if you have install_search_all_sources = true in your Pipfile -- if not its a bit more of a mystery to me, but not an issue per-se. The resolver index restriction logic uses the specified indexes when resolving what sha256 packages hashes are allowed, and since the install phase is verifying those hashes, it theoretically is not a security risk because it would not be possible to fake those package hashes and the validation of them during install.