Register annex key on dataverse as file-metadata for lookup

[mih]

I have asked the following things in dataverse chat. I think this would be the best way to make EXPORT mode work reliably, and not having to depend on path matching

Hey, I am looking at the possibilities for file-metadata. My use case is adding a non-dataverse file identifier as an additional metadata field in order to enable unambiguous file-lookups (because we need to mangle file/path name info on upload due to name restrictions). Looking at the docs for adding files or updating file metadata I see possibilities for the fields "description", "provFreeform", "categories", "restrict".

Q1: Is there a way to add a custom file and set that? I am specifically asking for the possibilities of any stock dataverse deployment, not for potential customizations that could be done to a particular instance.

Q2: If Q1 is negative, it seems prov-metadata could fit (we essentially derive the dataverse representation of a file from its original (name)). Looking at the docs for prov-upload, I get the impression that provstore terminology and structure is desired. However, the endpoint is called "freeform". Does that mean that no particular structure is mandatory?

Maybe one more clarification: Even if prov is the preferred possibility to express such things, ultimately it is about lookup. The use case requires to be able to determine whether a particular file identity is present in a dataverse dataset or not, and if so, to be able to download its content.

[mih]

We could deposit side-car files with metadata on dataverse: https://guides.dataverse.org/en/5.13/developers/aux-file-support.html

[mih]

TODO here is to check if provFreeform shows up in the listing returned by https://pydataverse.readthedocs.io/en/latest/reference.html#pyDataverse.api.NativeApi.get_dataset and report back to dataverse chat.

[mih]

I am closing this, together with #201, because after some explorations and back-and-forth with upstream no superior alternative materialized.

The present implementation continues to store dataverse fileIds in the local annex key STATE and will attempt to obtain such an ID based on matching a (mangled) remote file path against a listing of files in the latest (or across all) dataverse dataset versions.