[Feature] the ROM Management Datafile aka dat files

[nickl-]

Wow for only three weeks since first commit the progress here is nothing short of impressive, well done!

I hate to be the one to add yet another feature request, but this is more of a requirement. You need to be informed about this (if you don't already know), sooner rather than later.

tl;dr dat files you must start from here

The ROM Management Datafile, is a fairly basic specification, which provides you with everything you need to identify a ROM, like file names, sizes and hashes. The definition also include a name, description and usually also language, region, etc. all the information you need which is difficult or impossible to find from the file name alone. It is also what is traditionally expected from a ROM Manager, by your target audience and competition.

The tool you use to organize your ROMs, the ROM Manager. To rebuild a working ROM set from other collections or unknown files, while ensuring completeness, no corruption, modification, or manipulation. Which is also why it is championed by the preservation community, wanting to ensure every bit and original byte is accounted for. But it also enables you to compile a collection of known working ROMs, in the structure and layout as expected by different emulators.

MAME is particularly tricky, and as time progress more and more systems and their ROMs are being encapsulated by the project. It has already become the de-facto set for PSX ROMs only in a matter months. The problem is not only that their naming convention is obscure, using only a few lower case characters and numbers as file names, with no spaces, but they have three different types of ROMs for the same game(s). The ROMs can be zipped as split, full merged, or non-merged files, which may include one game, or multiple games of separate releases of the same game, or different games of the same type (hardware speaking). They also may or may not include the required BIOS dumps necessary to play the game which means these also need to be managed.

Not only are the dat files necessary to identify the actual game, but it is also required from a ROM Manager to be able to unzip and compile one type from another, as may be needed, to mitigate having to keep duplicates. I have been praying for a web manager, all the others are desktop clients which makes it difficult to combine into any sensible workflow automation, and they are singular in purpose. There is a disconnect between organizing files, and the identifying data contained therein, which is required to source the additional meta data needed to actually manage a library of media content.

These two worlds of organization vs manageability has remained in disconnect, which impairs the archivability of retro games into a singular source library by any definition. Hopefully RomM is here to finally make sense of all the chaos.

Sources

Logiqx actually came up with the dat file specification which most projects are using, there is of course always that one, no-intro has a slightly different schema but the content is similar and also well documented. Since it's all XML the data is self explaining anyway. This is not an exhaustive list...

http://www.logiqx.com/Dats/ - legacy sets
https://pleasuredome.github.io/pleasuredome/ - mame sets and others
https://github.com/danmons/datfiles - Mister set

The preservationists each with their own focus and peculiarities but each in their own right indispensable for their work cataloging our history.
http://redump.org - optical discs
https://no-intro.org - cartridges
https://www.tosecdev.org/downloads/category/22-datfiles
There is another one, trurip being rebranded as emuarc, but I am still unable to find their actual dat files, moving on...

Should probably be considered out of scope, more applicable to a lister project for sourcing roms, but for interest sake. The first, catalogs the scene releases of split archive files, the other has defined a specification called srr which catalogs the metadata required to "rescene" the extracted content back to identical split archives again.
https://dats.site
https://www.srrdb.com/

Documentation

...and other useful technical information to assist with the implementation of dat files.
https://pleasuredome.miraheze.org/wiki/DAT_File
http://www.logiqx.com/DatFAQs/DatCreation.php
https://wiki.no-intro.org/index.php?title=DAT-o-MATIC_Guide
https://www.oreilly.com/library/view/gaming-hacks/0596007140/ch01s13.html

The missing link which ties ROM files to their meta data, countless hours by a vast community capturing and cataloging this information, a saving grace unavailable to any other type of media.

[zurdi15]

Thank you for all this info, I didn't know anything about it. I will try to take a look and implement it before RomM gets much bigger, as it seems to be a very useful standard

[XargonWan]

Maybe we can work on this together when the time comes as I will definitely need it even on RetroMan.