[Question] Is there a way to store private pypi credentials not in the Pipfile?

[ecs-jnguyen]

Hi I configured my pip.conf + keyring so that pip will use it to get the credentials before installing a package.

Is there a way to store my credentials on my local machine somewhere like how pip uses pip.conf? I would rather not store my credentials in my Pipfile since that will be checked into github. Also I don't want to set my credentials in the environment variables either since that would need to be configured in terminal and various IDE's.

$HOME/.config/pip/pip.conf

[global]
index-url = http://<username-here>@localhost:8080/simple/

[uranusjr]

https://pipenv.pypa.io/en/latest/advanced/#injecting-credentials-into-pipfiles-via-environment-variables

[marob]

@uranusjr How does the doc answers the question? I don't find any way to configure pipenv to get the password from the keyring, only a way to configure the password as env var, which is not what @ecs-jnguyen wanted (nor me).

I've tried everything and can't find a way to configure pipenv similarly to pip on this topic.
On pip, I have the following configuration file:

index = https://[email protected]/repository/pypi-all/pypi
index-url = https://[email protected]/repository/pypi-all/simple

Then, the password is configured with the command line keyring set example.com myUser.
Finally, pip knows it has to search the password in the keyring as I filled a username in the URL but no password and it works well.

Similarly, on pipenv, I configured:

[[source]]
url = "https://[email protected]/repository/pypi-all/simple"
...

But pipenv doesn't seem to query the keyring for the password and ends with:

WARNING: 401 Error, Credentials not correct for https://example.com/repository/pypi-all/simple/keyring/
ERROR: Could not find a version that satisfies the requirement xxxx (from versions: none)
ERROR: No matching distribution found for xxxx

On the server side, I see 2 401, both with only the username in the basic auth while it should have added the password for the second request.

If I remove the myUser in the configuration, it's running indefinitely after receiving the first 401 (no Authorization header) and never triggers a second request.

The solution using an env var is not what we expect as it needs some extra configuration that shouldn't be required.

Is there something I missed?

[rehevkor5]

+1 it would be great to be able to provide repo credentials to a builder Docker container via RUN --mount=type=secret https://docs.docker.com/engine/reference/builder/#run---mounttypesecret In particular, without needing to run extra commands to set env vars. If pipenv could obey ~/.pip/pip.conf, that would be ideal because it would require no extra effort.

Also, URL-encoding credentials is unusual and adds additional friction. If you have a central team managing your repo credentials & providing them as secret values to your CI/CD pipelines (Jenkins etc.), they are not typically going to URL-encode those first. So that means you're stuck having to add more logic into your build just to do that yourself (eg. in a shell script... gross). However, you most likely first had to figure out that was the problem in the first place: if you're unlucky enough to have those characters in your credentials, then you'd just get a mysterious access denied problem even though your credentials look fine.

[matteius]

@rehevkor5 I have spent some time on this issue and have a PR out that you might find helpful: #5297

The problem though is you will still want to specify the credentials for the index as environment variables and reference them from the pip.conf as such, example: index-url = https://{$user}:{$password}@example.com/repository/pypi-all/simple
Otherwise, the credentials will be added to the Pipfile and Pipfile.lock in plaintext. There is really not going to be a good way to decouple and not have the sources in the Pipfile, but we can at least prefer the default index specified by pip.conf.

[matteius]

Would anyone be willing to test if this PR which is up to date with main and passing, which reads the pip.conf or pip.ini -- if it solves the private pypi credentials not in Pipfile issue? I still need to write documentation for it but I could use some help validating that it does what is expected. #5297

[matteius]

pipenv==2023.2.18 has been released with support for reading the index information from the pip.conf or pip.ini. I am closing this for now, but let me know if we need to re-open to further enhance how its handling credentials for this case.

[marob]

Hi @matteius. Thanks for your work, but I think it's not completely solving the problem.
As you said, it's only "reading" the information from pip.conf, but doesn't read the password from the keyring. So we still don't have a solution to our problem (no password in clear text and no variable environment).
Would it be possible to add this feature to definitely solve the issue?

Also, note that your solution is copying the pip.conf data in the local directory Pipfile, setting the url (and some other attributes), so it's not really reading the pip.conf: modifications in the pip.conf after creating the pipenv environment won't be taken into account. That means any URL change in the reprository (think about a company proxy like Nexus) will need to be applied on every project instead of only in pip.conf file.