Implement describe-file-collection

[mih]

The work over at https://github.com/psychoinformatics-de/datalad-concepts has reached a state that makes it worthwhile to start implementing the first pieces for a dataset serializer. I am thinking that a good first step would be to provide the analog of the directory listing compiler that is used in the context of https://github.com/datalad/datalad-catalog. I am thinking that an implementation could simply wrap ls-file-collection. Internally, it would more or less translate the data classes ls-file-collection uses already. But I believe it would be sensible to not pollute ls-file-collection API with this functionality.

I am further thinking that describe-file-collection would be a good name, and it would support any file collection that ls-file-collection supports: directory, tarfile, zipfile, gittree, gitworktree, annexworktree.

The implementation would (need to) be done in a way that feature additions to ls-file-collection (i.e. new collection types) automatically become available for describe-file-collection too.

In terms of metadata types, a file collection maps to a https://concepts.datalad.org/s/distribution/unreleased/Distribution

[mslw]

provide the analog of the directory listing compiler that is used in the context of https://github.com/datalad/datalad-catalog

FTR, my directory listing compilers:

• https://github.com/sfb1451/tabby-utils/blob/main/list_files.py was meant as general, but was really tuned for annex repos
• https://github.com/sfb1451/tabby-utils/blob/main/list_directory.py for regular directories

these were really quick drafts, see commit messages for explanations why they ended up the way they are.

[mih]

https://concepts.datalad.org/ now provides JSON-LD context, LinkML YAML and OWL for every schema.

A sensible behavior for describe-file-collection appears to be to produce JSON-LD that is plain JSON with an @context field that points to the relevant schema and version of it. Template:

  https://concepts.datalad.org/s/<schema>/<version>.jsonld

I see no real need to implement the complexity required for being able to report in multiple "languages" (schemas and versions). We can simply refer to the one that is used. https://concepts.datalad.org/ is already set up to continue to serve past schema versions.

[mih]

After making the first code steps with this, I believe this is not suitable for datalad-next. Too little s understood with confidence. I will pursue this development over at https://github.com/mih/datalad-mihextras until a bigger picture emerges.