ENH: Add "container metadata extractor"

[asmacdo]

datalad-metalad extension should include an extractor to retrieve metadata from a container image.

In scope: singularity
out of scope (for now) OCI containers

Desired metadata ideas:

• base image
• base image build date
• operating system
• installed software & version information (reproman retrace?)
• Vulnerability report (CVEs)
• Image build date
• Image location (ie hub.docker.com, quay.io)

[asmacdo]

file level, not dataset level. we will extract for.all singularity images in the repo.

[yarikoptic]

do develop that extractor within datalad-container extension, not in metalad. I will transfer this issue

[yarikoptic]

target use case: https://github.com/ReproNim/containers

extractor for starters should just look at registered within dataset containers i.e. the ones listed by containers-list command, which is also available via python API:

❯ pwd
/home/yoh/proj/repronim/containers
❯ python -c 'import json,datalad.api as dl; print(json.dumps(dl.containers_list(result_renderer="disabled")))' | jq '.[].path' | head
"/home/yoh/proj/repronim/containers/scripts/tests/arg-test.simg"
"/home/yoh/proj/repronim/containers/images/bids/bids-aa--0.2.0.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-afni-proc--0.0.2.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-antscorticalthickness--2.2.0-1.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-baracus--1.1.4.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-brainiak-srm--latest.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-broccoli--1.0.1.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-cpac--1.1.0_14.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-dparsf--4.3.12.sing"
"/home/yoh/proj/repronim/containers/images/bids/bids-example--0.0.7.sing"
...

[yarikoptic]

singularity inspect is a nice one:

$> singularity inspect images/neurodesk/neurodesk-vina--1.2.3.simg
{
    "org.opencontainers.image.revision": "e50c6323cd703e0c2df7aa56eb18e05757df2402",
    "maintainer": "Anaconda, Inc",
    "org.opencontainers.image.document": "https://github.com/Metaphorme/AutoDock-Vina-Docker",
    "GITHUB_SHA": "da8afcba3a56737deebcc847da204c0247a139f5",
    "org.opencontainers.image.created": "2022-12-23T15:47:51.857Z",
    "org.opencontainers.image.licenses": "MIT",
    "GITHUB_REPOSITORY": "NeuroDesk/neurocontainers",
    "org.opencontainers.image.description": "Build from release",
    "org.opencontainers.image.name": "vina-all",
    "org.opencontainers.image.version": "22.11.1",
    "org.opencontainers.image.source": "https://github.com/ContinuumIO/docker-images",
    "org.opencontainers.image.authors": "Metaphorme",
    "org.opencontainers.image.url": "https://github.com/ContinuumIO/docker-images",
    "org.opencontainers.image.title": "docker-images"
}

may be we should also provide a singularity-inspect metadata extractor which would just do that -- channel output from that command for the file. @mslw mentioned some already existing helper to do such extractors which rely on external command output, might be the easiest

[mslw]

You can capture the output of external programs by using metalad_external_file extractor (see this docs section). But given that we are within a DataLad extension already, I think it would be more logical to make the singularity-inspect extractor a proper MetaLad extractor derived from datalad_metalad.extractors.base.FileMetadataExtractor.

This docs page explains how the derived class needs to be constructed.

[yarikoptic]

I thought you have mastered one @asmacdo as a PR somewhere -- please reference this issue so we get them linked.