Force reshape tiff data by the adapter #797
Open
+145
−34
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Inconsistency between the expected (by the client) and the actual data shapes is a common cause of server-side errors. Barring the cases of actually erroneous metadata, the shape of the array can often be corrected by presenting a reshaped numpy view, i.e. without any modification or copying of the underlying data.
For example, multichannel images may have the dimensionality of
n_channels x height x width
, which for monochrome images reduced to1 x height x width
; the extra singleton dimension is specified in the Bluesky descriptor document is expected by the client. However, the data may be read from the external file by the I/O library as a 2D array,height x width
, therefore resulting in a shape mismatch. Currently this raises a 500 Server Error.While the descriptors can be modified for any acquisition in the future, changing the existing documents in the database is undesired.
Solution
This PR adds shape validation to the corresponding Adapters on the server side, which have the information about the expected shape in their
._structure().shape
attribute (so far, only forTiffSequenceAdapter
, but it can be extended to other classes too). If the expected and actual shape differ only in missing or extra singleton dimensions, the returned array is reshaped accordingly. In future, this can be also modified to handle cases of swapped or transposed dimensions, but these will require more rigorous checks.Reference
This modification is necessitated by the ongoing migration of existing Bluesky documents from Mongo storage to an SQL catalog.
Checklist
Add the ticket number which this PR closes to the comment section