Handling migrations of the internal data model

[joepio]

Sometimes, for whatever reason, I might have to change the internal data model of the persistence layer. This could (and has often) lead to a corrupt database. For example, if I add another datatype, the binary serialization changes, and older databases will not work.

There are a few ways in which we can deal with this.

Add migrations in the code - save (all / many) versions of serialized structs / enums

• Maintain a new version parameter, specifically for the database
• Store a version value in the DB
• When loading a DB, check this value.
• If the value is older than the current value, execute a function that iterates over all items and converts the previous struct / enum into the new item. This means that I'll need to write conversion functions and store all deprecated structs / enums in some 'migrations' folder. Make new sled::Tree during this process to make sure it does not fail.
• alternative to last step: serialize the store to json-ad, remove the database, load the serialized content. Elegant! Export & import server data to json-ad #121

Let user export in previous version / import in new one

• In changelogs and readmes, let users know they'll have to create an export
• Add export and import command to the server app (add clap, I guess?)

Switch to a more change-resistant / predictable data model - non binary based

I currently use binser to serialize all models, which means that the in-memory model is pretty much converted 1:1 to disk storage. This has performance benefits (no unnecessary serialization / deserialization logic), but also means that any change in the model that is stored (including adding new datatypes) will result in having to do migrations.

Switch to storing strings

I could opt for a string serialization, and maybe store datatypes as u8's, and the actual values as strings. This would eliminate most migration needs. However, this would mean that every time a Resource is being constructed from the DB, all strings have to be parsed - a costly endeavor, so less optimal performance. Since most requests will probably be 'give me the json-ad representation of some resource, we could theoretically skip this step, but not for _all_ resources. Things like collections` are constructed dynamically and require properly parsed propvals.

Store values with a new in-between limited datatype enum

Our current Value enum has one enum option for every existing datatype in atomic data. However, some of these actually contain the same Rust literal types. Slug, Markdown and String for example all use Strings. So we could store all these as one string(String) enum option, and store the datatype (e.g. Slug) in a separate value, e.g. as a u8 with some mapping table (because storing entire strings seems like a bad idea).

Don't do anything, let the users figure it out and destroy their DB every once in a while

Very tempting, and basically what is happening right now. I'll add a notice in the readme.