dagshub upload: variable batch size.

[yairl]

1. Uploads are now grouped into the minimum of 100 files or 100MB-s each (not configurable).
2. All uploaded files are discovered before upload begins, to make the global progress bar more useful.
3. Minor restructuring to make the code cleaner.

Notes:

• On my machine (Debian 12), the progress bar is printed many times and not just updated in place. This is annoying, wondering whether you'd like that changed as well.
• In process prints such as "Upload finished successfully!" are confusing and polluting the screen when it's one part of a bigger upload.
• With this change, you may want to change the file batch size from 100 to a larger number, or remove completely.

Resolves #429.

[dagshub]

Join the discussion on DagsHub!

[yairl]

@kbolashev sorry for the "batchs" mistake.
I hate these :)

[kbolashev]

Thanks a lot for doing this, this is absolutely better now!!!

There's probably still things that can be done, for example not bounding batches by folders, but I honestly don't remember if our uploading API supports that.

I took a liberty of changing a couple variable names to be more descriptive, hope that's ok.

I also liked the suggestion about not cluttering the output that much on dir upload, so I hide those now when a directory is being uploaded. Pull the changes and see if you like it :)