Parallel upload of chunks

[ogoffart]

The owncloud client 1.6 and 1.7 do parallel upload of files, but the chunks of the same file are uploaded sequencially.

I was experimenting trying to upload the chunks in parallel, but it seems there is race condition in the server. I am getting error 500 with exception Could not rename part file assembled from chunks

I suspect $chunk_handler->isComplete() returns true in lib/private/connector/sabre/file.php if all the chunks have been started, but some of them might not be finished. (And it is likely that the last chunk ends before the second to last, as it is smaller)

@PVince81

[PVince81]

Sounds like a great idea.
Like you said we need to make sure the server side can handle that.

CC @MTRichards @craigpg

[DeepDiver1975]

@ogoffart @MTRichards @karlitschek what is the timeframe for 'parallel chunk uploads' in the client? We need some time ahead to 'fix' this in core. THX

[ogoffart]

It is only optionally enabled, and disabled by default. It's just as an experiment if it would make the transfer faster.
Anyway, the current implementation work around the problem by never sending the last chunk in parallel.

[guruz]

(it will probably be in 1.8.. but this bug is not urgent because of the workaround.. but nice to have)

[DeepDiver1975]

Well - as soon as you enable this feature it will break with any installation out there.

Keeping in mind that we do server patch releases once a month at most we better fix this now - right?

[DeepDiver1975]

oc8.1

[guruz]

Related: owncloud/client#2724 (comment)

[PVince81]

From what I heard this has already got better ? What to do with this ticket ?

[ogoffart]

The client (1.8) work around the problem by never sending what it thinks is the last chunk in parallel.
But the race condition is still there.

[PVince81]

This should be fixed by this PR already: #14113
Did you guys try it out with the disabled workaround ?

[PVince81]

(fixed in 8.1)

[ogoffart]

Ah, no, i did not try against newer version yet.

[karlitschek]

So we can close? Right? @ogoffart Can you confirm?

[DeepDiver1975]

So we can close?

yes I'd say - please reopen if issues continue to exist