Broken pipe uploading

[viric]

Using "ia upload" I get a Broken Pipe error at the 2nd megabyte transferred. With tcpdump I see that the remote S3 host actually sends a TCP reset; ia retries with a 2nd connection, but it's also reset, and then the python exception is dumped to stderr. The "-R" (max num of retries) seems to have little effect

I've tried regenerating the AWS keys, I ensured that the process has the env vars set with the key/secret, and --status-check reports all fine. Nevertheless, my uploads always get Broken Pipe.

This may be a transient error from S3, but it happened to me several times also in the past; anyone has an idea on how to solve this? My solution until now has always been to use the web upload.

I'm susrprised because the connection is dropped after almost 2MB tranferred.

[saper]

Which version of requests do you have? Are you using cython ? It might be urllib3/urllib3#535 which breaks HTTP pipelining on a compressed stream. Also, did you try running bundled tests? For me it failed with python2.7 -m pytest -v tests/test_api.py (connection reset)

[johtso]

I'm having this same issue. I tend to only get the broken pipe with a specific file/item, and it then repeatedly gives me the broken pipe at roughly the same stage of the upload.

versions:

❯ python
Python 2.7.6 (default, Feb  3 2014, 08:57:48)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin

❯ pip freeze
Cython==0.18
PyYAML==3.10
args==0.1.0
clint==0.3.3
docopt==0.6.1
gevent==1.0
greenlet==0.4.6
internetarchive==0.8.1
jsonpatch==0.4
py==1.4.27
pytest==2.3.4
requests==2.3.0
six==1.4.1
ujson==1.33
wsgiref==0.1.2

[johtso]

Tried upgrading to latest requests (2.7.0) and still experience the same problem.

[saper]

What does the tcpdump say? Are you having Connection: keep-alive set in the headers? I have had the same issue you have and requests upgrade fixed it for me. You might need to apply #82