Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force reshape tiff data by the adapter #797

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

genematx
Copy link
Contributor

@genematx genematx commented Oct 16, 2024

Context

Inconsistency between the expected (by the client) and the actual data shapes is a common cause of server-side errors. Barring the cases of actually erroneous metadata, the shape of the array can often be corrected by presenting a reshaped numpy view, i.e. without any modification or copying of the underlying data.

For example, multichannel images may have the dimensionality of n_channels x height x width, which for monochrome images reduced to 1 x height x width; the extra singleton dimension is specified in the Bluesky descriptor document is expected by the client. However, the data may be read from the external file by the I/O library as a 2D array, height x width, therefore resulting in a shape mismatch. Currently this raises a 500 Server Error.

While the descriptors can be modified for any acquisition in the future, changing the existing documents in the database is undesired.

Solution

This PR adds shape validation to the corresponding Adapters on the server side, which have the information about the expected shape in their ._structure().shape attribute (so far, only for TiffSequenceAdapter, but it can be extended to other classes too). If the expected and actual shape differ only in missing or extra singleton dimensions, the returned array is reshaped accordingly. In future, this can be also modified to handle cases of swapped or transposed dimensions, but these will require more rigorous checks.

Reference

This modification is necessitated by the ongoing migration of existing Bluesky documents from Mongo storage to an SQL catalog.

Checklist

  • Add a Changelog entry
  • Add the ticket number which this PR closes to the comment section

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant