Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to upload package when FIPS is enabled on workstation #775

Closed
cjbland opened this issue Jul 14, 2021 · 2 comments · Fixed by #776
Closed

Unable to upload package when FIPS is enabled on workstation #775

cjbland opened this issue Jul 14, 2021 · 2 comments · Fixed by #776

Comments

@cjbland
Copy link

cjbland commented Jul 14, 2021

Your Environment

Thank you for taking the time to report an issue.

To more efficiently resolve this issue, we'd like to know some basic information about your system and setup.

  1. Your operating system: CentOS 7.9

  2. Version of python you are running: Python 3.6.8

  3. How did you install twine? Did you use your operating system's package manager or pip or something else? Installed via pip in a virtualenv

  4. Version of twine you have installed (include complete output of): twine version 3.4.1 (importlib_metadata: 4.6.1, pkginfo: 1.7.1, requests: 2.25.1, requests-toolbelt: 0.9.1, tqdm: 4.61.2)

  5. Which package repository are you targeting? Private Nexus PyPi repository

The Issue

Our development workstations were recently "upgraded" to enable FIPS. twine successfully creates the package, however the package fails to upload because a blank md5_digest is being sent in the HTTP payload. When this value is left in, our Nexus repo is responding with something like:

HTTPError: 400 Bad Request from https://internal.server/repository/repo/
Digests do not match, found <md5_hash>, expected:

The part after "expected" is the empty md5_digest from the HTTP payload.

I manually edited package.py, removed md5_digest from the metadata_dictionary return value, and the file uploaded successfully.

Steps to Reproduce

Unfortunately I'm not able to test this using the "test" PyPi server as the workstation I'm using is on a private lab/LAN. However, I have to assume this would function the same when trying to publish to there.

  1. You'll need a workstation with FIPS enabled
  2. You'll need a simple "hello world" package
  3. The command we are running is twine upload --repository-url=https://internal.server/repository/repo/ dist/*
@sigmavirus24
Copy link
Member

That's interesting as we have a test for FIPS

twine/tests/test_package.py

Lines 241 to 250 in b4a196e

def test_fips_hash_manager_md5(monkeypatch):
"""Generate hexdigest without MD5 when hashlib is using FIPS mode."""
replaced_md5 = pretend.raiser(ValueError("fipsmode"))
monkeypatch.setattr(package_file.hashlib, "md5", replaced_md5)
filename = "tests/fixtures/twine-1.5.0-py2.py3-none-any.whl"
hasher = package_file.HashManager(filename)
hasher.hash()
hashes = TWINE_1_5_0_WHEEL_HEXDIGEST._replace(md5=None)
assert hasher.hexdigest() == hashes
which thinks this is the correct behaviour.

That said, this is an interaction with Nexus where that test was almost certainly written against PyPI. We can't test against Nexus and PyPI would seem to accept this without issue. We could exclude md5_digest if it is None but I really abhor these 3rd party repositories that claim to support a language and don't appear to put the necessary effort in.

sigmavirus24 added a commit that referenced this issue Jul 15, 2021
Once again, we have to deal with third-party package repositories doing
their own thing and users needing this to just work. Those repositories
combined with FIPS mean that we will need to send the information we
actually have and only that when it comes to digests.

Closes gh-775
sigmavirus24 added a commit that referenced this issue Jul 15, 2021
Once again, we have to deal with third-party package repositories doing
their own thing and users needing this to just work. Those repositories
combined with FIPS mean that we will need to send the information we
actually have and only that when it comes to digests.

Closes gh-775
bhrutledge added a commit that referenced this issue Jul 16, 2021
* Send only the digests we have from PackageFile

Once again, we have to deal with third-party package repositories doing
their own thing and users needing this to just work. Those repositories
combined with FIPS mean that we will need to send the information we
actually have and only that when it comes to digests.

Closes gh-775

* Update function and comments for clarity

Let's be clearer about some of the bizarre logic we use to support 
repositories other than PyPI and distributions using FIPS

Co-authored-by: Brian Rutledge <[email protected]>

Co-authored-by: Brian Rutledge <[email protected]>
@bhrutledge
Copy link
Contributor

bhrutledge commented Jul 20, 2021

Released in https://pypi.org/project/twine/3.4.2/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants