HCS acquisition metadata

[will-moore]

Currently, acquisitions are implemented in ome-zarr as a hierarchy group:

plate/acquisition/row/column/field

e.g. see ome/omero-cli-zarr#43

However, this hierarchy doesn't exist in OMERO: Acquisitions are just an orthoganal grouping of fields, but the webclient etc show it as a hierarchy.
We should consider alternatives for storing acquisitions in ome-zarr.

Option 1:
Store acquisition metadata at the plate level (with an ID), and each Well can refer to the acquisition it is linked to via ID:
eg.

"plate": {
  "acquisitions": [
    {
      "id": 1,
      "name": "run 1",
      "description": "foo",
      "starttime": <timestamp>
      "endtime": <timestamp>
      "maximumfieldcount": <integer>
    }
  ],
...}

Then in each 'well', the images list could link to acquisition (optional):

"well": {
    "images": [
        {"path": "0", "acquisition": 1},
        {"path": "1", "acquisition": 2}
    ]
}

Alternative is to put the 'acquisition' in the 'image' metadata, but it is probably more useful to have this together at the Well level, and not 'contaminate' the image metadata with this.

This Option would make it easier to view ALL the fields for a Well, across the different acquisitions.
But makes it harder to mimic the current OMERO client behaviour of viewing a whole Plate with the fields from a particular acquisition.

Other Options or feature requests to consider?

[sbesson]

"plate": {
"id": 1,
"name": "run 1",
"description": "foo",
"starttime":
"endtime":
"maximumfieldcount":
}

Are we putting this information at the plate level or is there still an intermediate acquisitions list of JSON objects describing each acquisition/run?

"plate": {
  "acquisitions": [
    {
      "id": 1,
      "name": "run 1",
      "description": "foo",
      "starttime": <timestamp>
      "endtime": <timestamp>
      "maximumfieldcount": <integer>
   }, {
   ...
}

[will-moore]

Ooops - sorry - I'll fix it above.

[sbesson]

The proposal above makes sense as a starting point from the current in-progress version of the HCS.
As mentioned during today's call, this might lead us to revisiting the requirement for the extra acquisition hierarchy sooner rather than later.

[joshmoore]

👍 from me as well on the metadata layout (and dropping the extra hierarchy layer as well 😉)

[will-moore]

There is no requirement for the extra acquisition hierarchy with this spec change, so I'll remove it.

[will-moore]

So the acquisition ID should simply be unique within this plate (just use index as id) or we could use the OMERO acquisition ID?
And I guess acquisitions list should be ordered by starttime (or endtime), to save clients having to do that. Don't know if that's a MUST but certainly nicer.

Do we need maximumfieldcount? I guess if OMERO has this then easy to add, but maybe harder for other tools so I don't want to make the spec too onerous initially.
What about 'description'?

[joshmoore]

So the acquisition ID should simply be unique within this plate (just use index as id) or we could use the OMERO acquisition ID?

It'd use the OMERO ID until we've unified the "id" concept. The index is the position in the outer list.

[will-moore]

OK, I'll open a PR for omero-cli-zarr to support this, then start exporting idr0001 in this format...

cc @melissalinkert - This is a new proposal for how to represent acquisitions in ome-zarr.

[joshmoore]

It'd use the OMERO ID until we've unified the "id" concept. The index is the position in the outer list.

Based on the recent json-ld discussions, I wonder if that means we should use omero:id for this field to differentiate from, e.g. @id.

[will-moore]

I think we can use @id since we're just using it to reference the acquisition from the wells. The fact that it happens to come from OMERO doesn't mean it's invalid to use as an @id.
I could do more work on the omero-cli-zarr export to create a new @id for each acquisition (and a lookup from omero ID -> @id so that for each Well I can assign the correct @id).

If we actually want to store OMERO IDs and for them to be recognised as such, then we can use omero:id, but if we do that then we can do so for all 'image', 'plate', 'well', 'acquisition', 'label' objects as a separate work from this.

[joshmoore]

I think we can use @id since we're just using it to reference the acquisition from the wells.

@id needs to be something globally unique (at least to the zarr) so that other statements can be made about the object elsewhere.

if we do that then we can do so for all 'image', 'plate', 'well', 'acquisition', 'label' objects as a separate work from this.

Yeah, I don't think it's a rush. Certainly an open question to what extent it's useful as just an ID. Likely need the related server.

---

Thinking about the two points above, using the web address as the @id might cover both issues.

[sbesson]

Thinking about the two points above, using the web address as the @id might cover both issues.

heh, I was certainly about to write that @id: https://idr.openmicroscopy.org/api/v0/m/plates/7051/ might be an option.
Although long term, I'd err on the side of saying this should be a separate entity indicating the provenance and @id is a separate concept.