Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(uploader): Make sure every request is added to the job queue #1326

Merged
merged 1 commit into from
Aug 14, 2024

Conversation

susnux
Copy link
Contributor

@susnux susnux commented Aug 13, 2024

Summary

Remove the hack with the concurrency of the job queue, instead refactor the directory creation:

  1. Add it to the upload queue (to track progress)
  2. Add the request to the job queue
  3. Wait for parent to be created -> continue with children

Screen shots

Note that for demonstration purpose I limited the concurrency to 3 and throttled the network.

Bildschirmaufnahme_20240813_165415.webm

Sorry for the big diff, but some refactoring needed to get rid of some of my hacks 😅

@susnux susnux added bug Something isn't working 3. to review Waiting for reviews labels Aug 13, 2024
Copy link

codecov bot commented Aug 13, 2024

Bundle Report

Changes will increase total bundle size by 16.4kB ⬆️

Bundle name Size Change
@nextcloud/upload-esm 340.73kB 8.06kB ⬆️
@nextcloud/upload-esm-cjs 343.81kB 8.34kB ⬆️

Remove the hack with the concurrency of the job queue,
instead refactor the directory creation:
1. Add it to the upload queue (to track progress)
2. Add the request to the job queue
3. Wait for parent to be created -> continue with children

Signed-off-by: Ferdinand Thiessen <[email protected]>
@skjnldsv skjnldsv merged commit a4a27b6 into main Aug 14, 2024
17 checks passed
@skjnldsv skjnldsv deleted the fix/batch-upload branch August 14, 2024 12:28
@susnux susnux mentioned this pull request Aug 16, 2024
@skjnldsv
Copy link
Contributor

skjnldsv commented Oct 16, 2024

I'm 99% sure this broke the conflict upload. Investigating
nextcloud/server#48270

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Folder upload can overload server with requests
2 participants