Checksums to Verify Up- and Download

[dragotin]

Some storage backends support hash sums of files. These could be used to ensure the files integrity when up- or downloaded between sync client and server. This adds another level of security.

The basic idea is to add a hashsum as HTTP header to GET or PUT and compare on the other communication side. If the numbers do not match, the transmission has to be an error.

@moscicki has drafted a specification: https://github.com/cernbox/smashbox/blob/master/protocol/checksum.md as discussed on a meeting with @DeepDiver1975 and @dragotin.

Tracked in core repo owncloud/core#12417

[guruz]

Another issue where this might help: owncloud/core#13044

[guruz]

Related #3018

[dragotin]

There is a client branch already: https://github.com/owncloud/client/tree/checksum_verify

[nekoal]

What about perfomance? Checksum calculation need very long time for large files. I tried earlier with using crc32 function in zlib.dll 1.2.3 for my project. Walking directory tree time became 3-4 times longer when checksum enabled for all files include files with size larger than 1 MiB. And I expect md5 hash need more time than crc32. For simple integrity check may be using more fast function as adler32 enough.

[moscicki]

@nekoal: let's see in practice what is the impact. For now I'd say that the choice is yours: screw performance or screw file contents ;-)

[guruz]

Shall we have this for 2.1 ?
Something to discuss on the contributor meetup..

CC @MTRichards

[moscicki]

@guruz: I assume this functionality is available (but must be explicitly activated in the configuration/branding) in all releases starting with 1.8.0, correct?

[guruz]

@moscicki If things didn't change yes. I just wanted to emphasize that i think the non-CernCloud should have this.. :)
Are you coming to the contributor summit?

[moscicki]

Cool. Yes!

[MTRichards]

@guruz Ok, to discuss. In general, I like.

However, I think we should take a look at real tested performance impact...and then depending on those results, make a more informed decision.

[dragotin]

This is outdated, client version 2.1.0 and later have checksums. See #3735