Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

I want rclone tested and evaluated #14

Open
mblomdahl opened this issue Apr 26, 2024 · 0 comments
Open

I want rclone tested and evaluated #14

mblomdahl opened this issue Apr 26, 2024 · 0 comments
Assignees
@mblomdahl
Labels
documentation Improvements or additions to documentation

Comments

@mblomdahl
Copy link
Collaborator

I am struggling with mc mirror to sync multi-TB buckets from S3 to MinIO storage, with AWS as source it mostly seem to work but with B2 the sync progress in staggeringly slow (barely usable). The main problem seem to be server-side interruptions to transfers, yielding "http: ContentLength=x with Body length x-n [bytes received]" errors. Installing rclone and configuring it to do the heavy lifting, it might be a way forward.
@mblomdahl mblomdahl added the documentation Improvements or additions to documentation label Apr 26, 2024
@mblomdahl mblomdahl self-assigned this Apr 26, 2024
mblomdahl added a commit to mkdevops-se/hq.mkdevops.se that referenced this issue May 3, 2024
@mblomdahl
Evaluate rclone for minio.owntube.tv MinIO
78f3765 
This `rclone` installation is set up to run a few big sync jobs from Backblaze and AWS in `tmux`, let's see if it performs better than `mc mirror` does... Relates to OwnTube-tv/minio-microk8s-ansible#14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mblomdahl mblomdahl

Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mblomdahl