"499 Client Error: Client Disconnected for url: https://upload.pypi.org/legacy/"

[s-m-e]

Synopsis: I am trying to upload a package to pypi. The upload works as expected for testpypi but fails for the actual production pypi. In fails with "499 Client Error".

Environment

My .pypirc file looks like this:

[distutils]
index-servers =
  pypi
  pypitest

[pypi]
username=username
password=password

[pypitest]
repository=https://test.pypi.org/legacy/
username=username
password=password

I am running Python 3.6.1 on Linux (x86_64). The relevant packages (hopefully complete?) in my virtual environment are:

• certifi (2017.4.17)
• pip (9.0.1)
• pkginfo (1.4.1)
• pytz (2017.2)
• requests (2.18.2)
• requests-toolbelt (0.8.0)
• setuptools (36.2.2)
• six (1.10.0)
• tqdm (4.15.0)
• twine (1.9.1)
• urllib3 (1.22)
• virtualenv (15.1.0)
• wheel (0.29.0)

Steps for reproducing the behavior: My upload command uploads two files, a wheel and a source tarball, with detached signatures each, as follows:

twine upload $filename $filename.asc

Expected result: The upload works just like with pypitest.

Actual result: The files are 60 kBytes and 32 kBytes in size (really small). The upload for the first file starts very slow, takes a few seconds to go up to about 10%, stops. Then, after a while, the progress bar disappears (replaced by an empty line in the shell). Sometimes it appears again. After another while, the upload errors and (triggered by a script of mine) the upload of the second file begins. Same pattern here.

The output in all its glory:

Uploading distributions to https://upload.pypi.org/legacy/
Uploading project-1.2.3-py3-none-any.whl
HTTPError: 499 Client Error: Client Disconnected for url: https://upload.pypi.org/legacy/                                       
Uploading distributions to https://upload.pypi.org/legacy/
Uploading project-1.2.3.tar.gz
HTTPError: 499 Client Error: Client Disconnected for url: https://upload.pypi.org/legacy/                                       

After a lot of googling I could not find an explanation or documentation for this error in the context of twine. As far as I can tell, HTTP 499 is an error code specific to nginx indicating that the client "just" disconnected. Is this a timeout issue due to server load on pypi or did I misconfigure twine in some sort of way (masquerading a log-in issue for instance) or is this a bug?

EDIT (1): Throwing a plain wrong password into .pypirc produces the exact same behavior. It even shows me an upload progressbar with progress up to 20%. That's interesting ...

EDIT (2): HTTP error 499 also occurs for me when using setup.py ... upload, which indicated that the issue is more related to pypi-legacy. I have also filed a bug against it. After studying twine's sources, the behavior of the progressbar can likely be explained as a representation of how much data has been read from disk rather than how much data was actually uploaded. That's irritating.

[stur86]

Has there been any progress on this? I still experience it with twine 1.9.1 on open SUSE.

[sigmavirus24]

Hi there, it doesn't make sense to have this bug filed against twine because the error you're seeing wouldn't be seen if Twine was disconnecting from NGinx. This appears to be a bug between NGinx (a client) talking to PyPI.org's python server (uwsgi, gunicorn, or whatever they're using right now). There's a bit more explanation of that here.

This doesn't appear to be something we can fix in twine. As you've discovered, this also happens when you use python setup.py upload.

After studying twine's sources, the behavior of the progressbar can likely be explained as a representation of how much data has been read from disk rather than how much data was actually uploaded. That's irritating.

Sadly it's the limitation of the underlying libraries. We can only know when data has been read by httplib/http.request to send to the server, not when that data has been received by the server correctly. Progress bars aren't always completely true.