Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken pipe on upload #176

Closed
Asparagirl opened this issue Jan 4, 2017 · 3 comments
Closed

Broken pipe on upload #176

Asparagirl opened this issue Jan 4, 2017 · 3 comments

Comments

@Asparagirl
Copy link

Asparagirl commented Jan 4, 2017

I'm getting broken pipe errors on various items that I'm trying to upload (all through the bulk upload via CSV method). I have the latest version of the ia command line tool. I'm on a Mac running Sierra 10.12.2, if that helps.

File "/usr/local/bin/ia", line 11, in <module>
    sys.exit(main())
  File "/Library/Python/2.7/site-packages/internetarchive/cli/ia.py", line 155, in main
    sys.exit(ia_module.main(argv, session))
  File "/Library/Python/2.7/site-packages/internetarchive/cli/ia_upload.py", line 236, in main
    session)
  File "/Library/Python/2.7/site-packages/internetarchive/cli/ia_upload.py", line 78, in _upload_files
    response = item.upload(files, **upload_kwargs)
  File "/Library/Python/2.7/site-packages/internetarchive/item.py", line 751, in upload
    request_kwargs=request_kwargs)
  File "/Library/Python/2.7/site-packages/internetarchive/item.py", line 613, in upload_file
    **request_kwargs)
  File "/Library/Python/2.7/site-packages/internetarchive/session.py", line 343, in send
    r = super(ArchiveSession, self).send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))
 uploading Reclaim_The_Records_-_NYC_Marriage_Index_-_Queens_-_1931_images.zip: [################################] 42/389 - 00:00:30

Note that it's dropping out at around 42/389, sometimes 40/389. The total file size is about 407 MB.

I saw there was a closed issue similar to this one from a while back. Not sure if this is related.

@jjjake
Copy link
Owner

jjjake commented Jan 5, 2017

Thanks for the report @Asparagirl. #63 is unrelated, but I have been getting more and more reports about connection issues. Usually issues like this are transient, but that does not seem to be the case here?

I'll update you here once I find more out.

@Asparagirl
Copy link
Author

#80 is the closed issue I saw that seemed to be related -- especially how it always happens a few MB into the upload, but not immediately.

(The non-ASCII-formatted CSV thing in #63 is unrelated, but it bugs me. :-) )

Thanks!

@ghost
Copy link

ghost commented May 29, 2017

I'm getting the same with requests==2.16.5 on Python 2.7.12 and requests==2.9.1 on Python 3.5.2. Both running on "new" Ubuntu 16.04 AWS boxes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants