Faster serialization / deserialization?

[cspoerri]

Hi Alex,
I was wondering how difficult it would be to implement a faster way to serialize / deserialize a graph? Was looking at using the package https://github.com/neuecc/MessagePack-CSharp.

Do you have any recommendations, tips or other points I should look for, if I would like to use this library for the serialization work?

The reason is: I'm working with a graph that has 100+K of nodes, and it takes sometimes a couple minutes to read a graph from a file.

Thanks for your help, Chris

[KeRNeLith]

Hello!

Sorry for the delay :(

The current binary serialization offered by the library rely on standard binary serialization offered by the .NET framework. As you may already know this implementation is not really safe in term of security and so Microsoft drop support with .NET 5.
Here is a post in Discussions where I sum up a bit ideas about this.
So what's sure is that binary serialization implementation must be updated, and if we can gain in speed in the same time it's a good point. Now choice is not really stricly determined.

At first I wanted an implementation that will not be too intrusive in the code of the Core library in order to avoid changes as much as possible when we only deal with serialization feature. The best would be something that is fast, lightweight, that may provide backward compatibility if in the future structures evolve, and offer a wide compatibility/easy usage with what is commonly used.

[aviatrix]

Should Serialization be part of the core project?
Can we make an interface (or use) and create sub packages that implement different types of serialization & export functionality (eg. pdf, svg, html,png) ?

[KeRNeLith]

Ideally I would say yes we can split serialization out of the core package. It's basically the idea I had when creating QuikGraph.Serialization. Keeping core structures and algorithms (Note that I also thought about QuikGraph.Algorithms but was unsure about that) in the core library and all exports to QuikGraph.Serialization or dedicated assembly like QuikGraph.Graphviz.

For the binary serialization, the main point is to find an efficient, widely used or acceptable library that ideally allow to do serialization without being too intrusive in the core structures.
Note that recently an issue occurred on the svg export because the web service originally used has been closed. Library was relying on some web available stuff that I think would be better to get rid of. I managed for some graphviz stuff to spot issue when Graphviz website was down but that's not always that easy to find replacement while also trying to not include too big dependencies.

[aviatrix]

IMO, relying on web services for this is kind of a no-go, i didn't even realize that was the case! Might be an err on my side not reading it, but i can see this being a no-go for other people as well, esp if used in corporate env.

[KeRNeLith]

I'm totally agree with you for the web service aspect not being really reliable solution. I kept the original code for that part because I did not have an acceptable replacement at the moment.
For sure it's something that must be changed at some point, BTW I did not anticipate the web service down!