Skip to content
This repository has been archived by the owner on Jul 25, 2021. It is now read-only.

Support for stream #6

Closed
patrickpissurno opened this issue Sep 1, 2020 · 4 comments
Closed

Support for stream #6

patrickpissurno opened this issue Sep 1, 2020 · 4 comments
Assignees

Comments

@patrickpissurno
Copy link

I was wondering if you could add support for FileStream (or Stream in general)?

Since we usually use files for computing the CRC, I find it annoying to load the whole file in memory in order to get all the bytes. When dealing with big files, it's not really good practice. Also, if the files are > 2GB in size it can fail to do so.

So, that's what I can do right now:

var bytes = File.ReadAllBytes(path);
var crc32 = Crc32CAlgorithm.Compute(bytes);

What would be nice:

using (var stream = new FileStream(path, FileMode.Open))
{
    var crc32 = Crc32CAlgorithm.Compute(stream);
}

This is an issue similar to this one, except that I can't use that library as I can't change my project to use Crc32 (non-C).

Thanks

@robertvazan robertvazan self-assigned this Sep 1, 2020
@patrickpissurno
Copy link
Author

For future readers, there is another way of handling big files. I suggest adding this to the README, as it wasn't clear to me I was supposed to be doing it like this:

Instead of doing this:

var crc = Crc32CAlgorithm.Compute(File.ReadAllBytes("big_file.mp4"));

Do this:

uint crc = 0;
using (var stream = File.OpenRead("big_file.mp4"))
{
    var buffer = new byte[4096]; //the buffer size can be tweeked
    int bytesRead;
    while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
    {
        crc = Crc32CAlgorithm.Append(crc, buffer, 0, bytesRead);
    }
}

Keep in mind that "big_file.mp4" is just an example. You should replace it with the filename of your choice.

Both of them result in the very same crc. And during my tests the second one was much quicker, even when dealing with 100MB-ish files. So, I suppose it is the way to go.

It'd be very simple to write some code to wrap the above one and convert it to work with Streams, but I don't think it's really necessary anymore. I do think, though, that the documentation should mention how to use the Crc32CAlgorithm.Append method.

I can submit a PR (to update the README) if you'd like it. Just let me know.

Also, thanks for the library!

@robertvazan
Copy link
Owner

How about this?

using (var stream = File.OpenRead("big_file.mp4"))
{
    uint crc = BitConverter.ToUInt32(new Crc32CAlgorithm().ComputeHash(stream));
    // ...
}

I didn't test it, but it should work according to docs. Anyone has time to try it and confirm it works?

@Artemis-chan
Copy link

Artemis-chan commented Feb 6, 2021

BitConverter.ToUInt32 needs a starting position

using (var stream = File.OpenRead("big_file.mp4"))
{
    uint crc = BitConverter.ToUInt32(new Crc32CAlgorithm().ComputeHash(stream), 0);
    Console.WriteLine(crc);
}

this works

@robertvazan
Copy link
Owner

This library is no longer maintained. Feed free to adopt it if you find it useful.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants