Add new property descr., improve formulations #2256

pwalczysko · 2022-06-29T16:22:16Z

As discussed, adding the improvements cf. ome/omero-model#81 (comment).
Got inspiration from https://github.com/ome/omero-model/blob/abbc62ef981322e6959cc6c75df7c3096c246d88/src/main/resources/omero-model.properties#L203

cc @sbesson

sbesson

In addition to the individual comments, we might want to review and unify the vocabulary to talk about float and double images.

As per thef formal definition of the PixelType in the OME schema:

float == single-precision floating point
double == double-precision floating point

With this in mind, I understand "floating point data types" refers to the superset of all these pixel types. Maybe floating-point data types (single or double precision) would be a better semantics to avoid the confusion float vs floating-point while clarifying there are several types?

omero/sysadmins/limitations.rst

sbesson · 2022-06-30T08:06:45Z

omero/sysadmins/limitations.rst

-generated for images with floating-point pixel data, meaning the imported
-image will be scrambled if it is over the size threshold mentioned above.
+generated for images with floating-point or double pixel data types, meaning the viewing of the imported
+images will be compromised if it is over the size threshold mentioned above.


Main caveat with the usage of compromised is that it might be understood with other implications esp. in terms of security.

Pyramids of sub-resolution image tiles are not generated for images with floating point pixels. However, in some cases, these images can be viewed in OMERO clients at full resolution (if the images are not too large).

This behaviour is configured by omero.pixeldata.max_plane_float_override. By default this is True; OMERO overrides the requirement for floating-point images above the max_plane size to have pyramids, which allows them to be treated as regular images and possibly viewed in the clients.

However, this also allows OMERO to attempt the calculation of min/max pixel intensity for these images (normally disabled for large images because it is resource intensive to read every pixel value).

When the property omero.pixeldata.max_plane_float_override is set to False on your server, OMERO will not attempt to treat large floating-point images as if they are smaller images, so any large images without pre-generated pyramids will not be viewable. However, this will protect the server from expensive attempts to calculate min/max pixel values.

This can replace the last paragraph below

omero/sysadmins/limitations.rst

sbesson · 2022-06-30T08:07:30Z

omero/sysadmins/limitations.rst

@@ -79,6 +80,14 @@ This primarily affects the following file formats:
 This issue can be avoided by pre-generating pyramidal OME-TIFF images as
 described above.

+Requests sent from clients to OMERO.server  when displaying large images
+with floating-point pixel data type which have no pre-generated pyramids
+can cause limitation or even a complete loss of service, necessitating a server restart.


where limitations == performance issues?

pwalczysko · 2022-06-30T10:03:00Z

In addition to the individual comments, we might want to review and unify the vocabulary to talk about float and double images.

As per thef formal definition of the PixelType in the OME schema:
* `float` == single-precision floating point

* `double` == double-precision floating point
With this in mind, I understand "floating point data types" refers to the superset of all these pixel types. Maybe floating-point data types (single or double precision) would be a better semantics to avoid the confusion float vs floating-point while clarifying there are several types?

Yes, definitely, thank you. I was confused about it myself

Co-authored-by: Sébastien Besson <[email protected]>

will-moore · 2022-06-30T12:06:20Z

omero/sysadmins/limitations.rst

@@ -53,6 +53,7 @@ images if it doesn't already exist in the file. The threshold size is configurab
 default. However, this process can be very resource-intensive, depending on the size of the
 image as well as the image format and any data compression used, for example see
 `PixelData threads and pyramid generation issues <https://forum.image.sc/t/pixeldata-threads-and-pyramid-generation-issues/49794>`_.
+Further, OMERO never generates pyramids for floating-point or double pixel data types. Working with large floating-point pixel data type images without pre-generated pyramids in OMERO can cause problems (see below).


Maybe "Working with large floating-point images..." is an acceptable abbreviation? Or even "large floating-point pixel type images" is slightly less verbose

pwalczysko · 2022-06-30T12:47:30Z

@will-moore @sbesson in f642440 I have included all the suggestions from both of you, as well as included a new chapter and new anchor

will-moore · 2022-06-30T13:10:13Z

omero/sysadmins/limitations.rst

+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+:model_doc:`Pyramids <omero-pyramid/>` of sub-resolution image tiles are not generated for images with floating point pixels. 
+However, in some cases, these images can be viewed in OMERO clients at full resolution (if the images are not too large).


Maybe remove that heading above (since we are still on the subject of Large images with floating-point pixel data types. And remove the 1st sentence of this paragraph, since it repeats what's above.
Then: "However, in some cases, large floating-point images without pyramids can be viewed..."

will-moore

I think this is good-enough for this release. It's hard to know if it makes sense to a first-time reader, but at least it makes sense to me now!

sbesson

A few suggested changes to reflow the text and end on a recommendation for large floating-point data. Agreed with @will-moore we are getting close to an acceptable first version that we can improve with time.

omero/sysadmins/limitations.rst

Co-authored-by: Sébastien Besson <[email protected]>

sbesson

Thanks all, I feel comfortable linking to this section for recommendation on how to configure one server from the release announcement.

pwalczysko added 3 commits June 29, 2022 17:16

Add new property descr., improve formulations

e739f03

fix typo

2f95b7c

Parse paragraph

16e84ab

pwalczysko requested a review from sbesson June 29, 2022 16:32

sbesson requested changes Jun 30, 2022

View reviewed changes

pwalczysko and others added 3 commits June 30, 2022 11:31

Update omero/sysadmins/limitations.rst

378f2ea

Co-authored-by: Sébastien Besson <[email protected]>

Update omero/sysadmins/limitations.rst

eddfc83

Co-authored-by: Sébastien Besson <[email protected]>

Update omero/sysadmins/limitations.rst

57ab9e8

Co-authored-by: Sébastien Besson <[email protected]>

will-moore reviewed Jun 30, 2022

View reviewed changes

Include Will's suggestions, adjust the text min/max

f642440

pwalczysko requested review from sbesson and will-moore June 30, 2022 12:47

will-moore reviewed Jun 30, 2022

View reviewed changes

pwalczysko added 2 commits June 30, 2022 14:13

Merge two paragraphs and rename and move the anchor

6737173

Remove the "not smooth" sentence

729a230

pwalczysko requested a review from will-moore June 30, 2022 13:17

will-moore approved these changes Jun 30, 2022

View reviewed changes

Improve formulation

6524e5b

sbesson requested changes Jun 30, 2022

View reviewed changes

pwalczysko and others added 6 commits June 30, 2022 14:29

Update omero/sysadmins/limitations.rst

73eae90

Co-authored-by: Sébastien Besson <[email protected]>

Update omero/sysadmins/limitations.rst

8f8c58a

Co-authored-by: Sébastien Besson <[email protected]>

Update omero/sysadmins/limitations.rst

bcdf59a

Co-authored-by: Sébastien Besson <[email protected]>

Rearrange the order and make a recommendation

682e4cc

Improve formatting

776a07d

Reformulate the sentence to avoid suspicion

4456511

pwalczysko requested a review from sbesson June 30, 2022 13:40

sbesson approved these changes Jun 30, 2022

View reviewed changes

sbesson merged commit 36cced7 into ome:develop Jun 30, 2022

pwalczysko deleted the develop branch June 30, 2022 13:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new property descr., improve formulations #2256

Add new property descr., improve formulations #2256

pwalczysko commented Jun 29, 2022

sbesson left a comment

sbesson Jun 30, 2022

will-moore Jun 30, 2022

will-moore Jun 30, 2022

sbesson Jun 30, 2022

pwalczysko commented Jun 30, 2022

will-moore Jun 30, 2022

pwalczysko commented Jun 30, 2022

will-moore Jun 30, 2022

will-moore left a comment

sbesson left a comment

sbesson left a comment

Add new property descr., improve formulations #2256

Add new property descr., improve formulations #2256

Conversation

pwalczysko commented Jun 29, 2022

sbesson left a comment

Choose a reason for hiding this comment

sbesson Jun 30, 2022

Choose a reason for hiding this comment

will-moore Jun 30, 2022

Choose a reason for hiding this comment

will-moore Jun 30, 2022

Choose a reason for hiding this comment

sbesson Jun 30, 2022

Choose a reason for hiding this comment

pwalczysko commented Jun 30, 2022

will-moore Jun 30, 2022

Choose a reason for hiding this comment

pwalczysko commented Jun 30, 2022

will-moore Jun 30, 2022

Choose a reason for hiding this comment

will-moore left a comment

Choose a reason for hiding this comment

sbesson left a comment

Choose a reason for hiding this comment

sbesson left a comment

Choose a reason for hiding this comment