Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Well metadata #43

Merged
merged 7 commits into from
Nov 10, 2020
Merged

Well metadata #43

merged 7 commits into from
Nov 10, 2020

Conversation

will-moore
Copy link
Member

See https://github.com/ome/omero-ms-zarr/issues/76#issuecomment-721068552

omero zarr export Plate:1 now exports 'well' metadata like

"well": {
    "images": [
        {"path": "0"},
        {"path": "1"}
    ]
}

and 'plate' metadata like:

"plate": {
        "columns": [
            {"name": "1"}, {"name": "2"}, {"name": "3"}
        ],
        "rows": [
            {"name": "A"}, {"name": "B"}
        ],
        "wells": [
            {"path": "0/A/3"},
            {"path": "0/B/2"},
            {"path": "0/A/1"},
            {"path": "0/B/3"},
            {"path": "0/A/2"},
            {"path": "0/B/1"}
        ],
        "field_count": 1,
        "name": "plate1_1_013",
        "acquisitions": [
            {"path": "0"}
        ],
    }

@will-moore
Copy link
Member Author

will-moore commented Nov 4, 2020

@will-moore
Copy link
Member Author

will-moore commented Nov 5, 2020

@manics
Copy link
Member

manics commented Nov 5, 2020

Unrelated to this PR, but note that on https://minio-dev.openmicroscopy.org/idr/idr0094-ellinger-sarscov2/screenB/7825.zarr/.zattrs
acquisitions mentioned in this PR description is named plateAcquisitions instead:

        "field_count": 9,
        "name": "ESP0025722",
        "plateAcquisitions": [
            {
                "path": "0"
            }
        ],

which means we have a mix of snake_case and camelCase

@sbesson
Copy link
Member

sbesson commented Nov 5, 2020

Thanks @manics, I vote for doing plateAcquisitions -> acquisitions as part of this PR

@will-moore
Copy link
Member Author

Thanks @manics for catching that. Fix pushed.

@will-moore
Copy link
Member Author

I think I'll try fixing that by manually editing the exported plate.zattrs above at the same time as updating vizarr to keep everything in sync.
I could try to update the column names to str and fix the version commit too, but those aren't breaking changes.

@will-moore will-moore mentioned this pull request Nov 5, 2020
@will-moore
Copy link
Member Author

Plate metadata exported with current state of this PR:
https://minio-dev.openmicroscopy.org/idr/idr0002-heriche-condensation/plate1_1_013/pr_43/422.zarr/.zattrs
With lists truncated:

{
    "_creator": {
        "name": "omero-zarr",
        "version": "0.1.dev149+g37c48d6"
    },
    "plate": {
        "acquisitions": [
            {
                "path": "Run 422"
            }
        ],
        "columns": [
            {
                "name": "1"
            },
            {
                "name": "2"
            },
        ],
        "field_count": 1,
        "name": "plate1_1_013",
        "rows": [
            {
                "name": "A"
            },
            {
                "name": "B"
            },
        ],
        "wells": [
            {
                "path": "Run 422/B/4/"
            },
            {
                "path": "Run 422/D/2/"
            },
        ]
    }
}

@will-moore
Copy link
Member Author

will-moore commented Nov 6, 2020

Export failed for Plate:4287 after about 20 Wells. Don't know which Well/Image it failed on
(partial plate exported at
https://minio-dev.openmicroscopy.org/idr/idr0012-fuchs-cellmorph/HT01/pr_43/4287.zarr/)
with:

File "/tmp/venv/lib64/python3.6/site-packages/omero/gateway/__init__.py", line 4793, in __call__
    return self.f(*args, **kwargs)
  File "/tmp/venv/lib64/python3.6/site-packages/omero_api_RawPixelsStore_ice.py", line 1199, in getPlane
    return _M_omero.api.RawPixelsStore._op_getPlane.invoke(self, ((z, c, t), _ctx))
omero.ApiUsageException: exception ::omero::ApiUsageException
{   
    serverStackTrace = ome.conditions.ApiUsageException: Z '1' greater than sizeZ '1'.
        at ome.services.RawPixelsBean.handleException(RawPixelsBean.java:836)
        at ome.services.RawPixelsBean.getPlane(RawPixelsBean.java:399)

Loading in napari unexpected array size for one of these Wells? All C Wells have only 1 channel.

LOADING tile... 0/C/24/0/4
Failed to load 0/C/23/0/4
Failed to load 0/C/22/0/4
LOADING tile... 0/D/1/0/4
LOADING tile... 0/D/2/0/4
Failed to load 0/C/24/0/4
LOADING tile... 0/D/3/0/4
...data.shape (1, 1, 3, 64, 84)

Others are ...data.shape (1, 3, 1, 64, 84)

@will-moore
Copy link
Member Author

@dominikl dominikl mentioned this pull request Nov 6, 2020
@will-moore
Copy link
Member Author

Current status demo: https://www.youtube.com/watch?v=i27bF2j03IU

Copy link
Member

@joshmoore joshmoore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks fine. If there are no objections, I'd vote for getting this in with the other linked PRs. Any changes to the metadata can kick off a round of N other PRs.

Copy link
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I am concerned this could be tagged immediately to capture the well metadata layout that was extensively tested over the last week but we can also get #44 in the tag assuming it's soon ready.

@sbesson
Copy link
Member

sbesson commented Nov 10, 2020

No objection raised during today's NGFF call. Merging to make progress on the two next concepts:

  • plate + labels
  • plate + acquisitions

@sbesson sbesson merged commit 1d4e6ac into ome:master Nov 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants