Propagator: Balance concurrent up/down-load

[guruz]

When synchronizing, it makes sense to try to keep at least 1 PUT and 1 GET running at the same time so we don't actually waste time by not trying to use the connection both upstream and downstream at its fullest.

(MOVE, MKCOL, DELETE do not matter in this issue I'd say)

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/1455057-propagator-balance-concurrent-up-down-load?utm_campaign=plugin&utm_content=tracker%2F216457&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F216457&utm_medium=issues&utm_source=github).

[guruz]

Status: With the new job scheduler that @ogoffart did for 1.8, this might be easier to implement now.
Instead of picking just the "next" job in the list, the scheduler could pick a next one based on the kind of current one

[mrow4a]

Awesome, so the concept exists. Nice! Will do that surely.

[mrow4a]

@guruz @ogoffart One question, why do we actually keep a list of all finished jobs in PropagateDirectory::scheduleNextJob(), instead of just destroying them?

[ogoffart]

why do we actually keep a list of all finished jobs

We probably should delete them indeed.

[mrow4a]

Ok. will see what we can do about it.

@ogoffart @guruz @DeepDiver1975 @cdamken
I actualy have also anotheridea about the file scheduling. This could be done both inside the folder and outside folders.

During the loop over all files to synchronise, we structurise our directories so that they are being synced in order of insertion, does not matter is specific tree contains 100GB of data to sync, and another 1MB of very important document, but added to sync list later.

We could do it a little bit smarter. After we have all the "directories" object, we can give the folder ordered sync instructions, with which files/folders to start within that specific folder - according to their total size.

Imagine you have following folder structure to sync:

In current implementation, these files will be synced as they are met in the structure below:

ownCloud Root
├── less-important-folder-10MB/
│ └── less-important-file
├── Home&Work-10GB/
│ └── folder with nested 10GB of photos to be sync
│ └── very-very-important folder with single documents to sync
│ └── less-important-document - 1MB
├── important-folder-12MB/
├── less-important-folder - 5MB/
│ └── less-important-document
└── important-file-1MB

However, post-loop could quickly restructurise it as follows, looking of size of files within to sync (of course not by priority, because ownCloud does not support sth like FOLDER PRIORITIES:

ownCloud Root
└── important-file-1MB
├── less-important-folder-5MB/
├── less-important-folder-10MB/
│ └── less-important-file
├── important-folder-12MB/
└── less-important-document
├── Home&Work-10GB/
└── less-important-document - 1MB
└── very-very-important folder with single documents to sync - 2MB
└── folder with nested 10GB of photos to be sync