Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix upload with multiple blocks to allow more than 60mb #72

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

sharevb
Copy link
Contributor

@sharevb sharevb commented Oct 25, 2023

Fix #64, #70

@iamleot iamleot self-assigned this Oct 25, 2023
@iamleot iamleot self-requested a review October 25, 2023 07:48
@iamleot iamleot added bug Something isn't working enhancement New feature or request labels Oct 25, 2023
@antoinedelplace
Copy link

antoinedelplace commented Nov 28, 2023

It seems there is a limit to the number of blocks you can send.

Checking for no duplicate filenames
Preparing to upload
Could not find any csrf-token
From storm_upload_token WETRANSFER_STORM_PREFLIGHT URL is: https://storm-us-east-1.w
etransfer.net/api/v2/batch/preflight
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetra
nsfer.net/api/v2/blocks
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetra
nsfer.net/api/v2/batch
Get transfer id ef0332c130***5e81f20231128101843
Doing preflight storm
Preparing storm block upload

Traceback (most recent call last):
  File "/upload_wetransfer.py", line 20, in upload_wetransfer
    url = upload([input])
          ^^^^^^^^^^^^^^^
  File "/transferwee/transferwee.py", line 597, in upload
    blocks = prepare_data["blocks"]["data"]["blocks"]
             ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'data'
>>> print(prepare_data["blocks"])
>>> {'ok': False, 'error': {'message': 'You can announce at most 128 at once'}}

The file I want to send is a unique zip file of size 890M.

@sharevb
Copy link
Contributor Author

sharevb commented Nov 29, 2023

Right dont work with big big files, will update with your chunked solution @antoinedelplace, merci

As of issue iamleot#76, PR iamleot#77 and to accomodate issue iamleot#64, thus case of "many small" or "really big" files, chunk big files in many blocks and blocks request also in chunks if more than 100 blocks need for the transfer of all files
@sharevb sharevb force-pushed the upload60mb branch 4 times, most recently from 9a0c34b to e193115 Compare November 29, 2023 21:43
@sharevb
Copy link
Contributor Author

sharevb commented Nov 29, 2023

So right now, @antoinedelplace , should be ok either with 600 files or a big 800MB zipped

@sharevb
Copy link
Contributor Author

sharevb commented Nov 29, 2023

@iamleot, could you please merge this PR, will solve many things, thanks

@sharevb sharevb mentioned this pull request Nov 29, 2023
@sharevb
Copy link
Contributor Author

sharevb commented Dec 3, 2023

Just updated to accomodate with #78

@pszemus
Copy link

pszemus commented Feb 6, 2024

Works for my 143 MB file. Thanks!

@sharevb
Copy link
Contributor Author

sharevb commented Feb 11, 2024

@iamleot, could you please merge this PR, will solve many things, thanks

@raduhazsda
Copy link

Changed the file contents based on this Fix and I am happy to say that the upload is now working, no more errors. Great work.

@sharevb
Copy link
Contributor Author

sharevb commented Apr 10, 2024

@iamleot, could you please merge this PR, will solve many things, thanks

@anandtripathi5
Copy link

It worked for me for 500mb files, now using the forked version of @sharevb , @iamleot please merge this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Could not upload files bigger than 60MB
6 participants