feat: high resolution screenshot from viewer

[seankmartin]

x1, x2, and x4 screenshot functionality in the neuroglancer webviewer

Overall idea of files/classes:

1. The python integration ScreenshotHandler is what actually interacts with the data to try and grab screenshot data, and repeatedly grab statistics while this is loading.
2. The neuroglancer ScreenshotManager interacts with the display context to start the ScreenshotHandler process. It also receives information from the ScreenshotHandler and passes this on to the screenshot UI menu when appropriate. It saves the file when complete, and stops screenshots automatically if they hang.
3. The ScreenshotDialog handles the user input to trigger actions in the ScreenshotManager, and shows updates from the ScreenshotManager to the user.

[jbms]

Overall this is great.

One issue I noticed is that before you click "Take screenshot", you are essentially "previewing" the screenshot, but "Keep slice FOV fixed with scale change" has strange behavior. In particular, when "keep FOV" is checked and you increase from 1x to 2x, the preview FOV decreases by 2x; when "keep FOV" is unchecked, changing from 1x to 2x has no effect on the preview FOV. It would be better if the preview matched the FOV of the actual screenshot.

[seankmartin]

Good point, thanks!

The reason they behave like this is to try and give more accurate stats in the panel resolution preview and the layer resolution preview. But it's definitely true that this has the unfortunate side effect of visually giving the user the impression that these settings do the exact opposite in the preview than those settings will do in the actual screenshot.

Example:

From here.

x1 resolution stats

Screenshot Size	1546 x 853 px
Panel Type	Pixel Resolution
Slice view (2D)	771 x 853 px
Perspective projection (3D)	771 x 853 px

Layer Name	Data Type	Physical Voxel Resolution
fafb_v14_clahe	Image slice (2D)	x 256 nm, y 256 nm, z 320 nm
fafb-ffn1-20190805	Segmentation slice (2D)	x 16 nm, y 16 nm, z 40 nm

x4 resolution stats, but the FOV will shrink in the screenshot (keep FOV unchecked)

Screenshot Size	6184 x 3412 px
Panel Type	Pixel Resolution
Slice view (2D)	3084 x 3412 px
Perspective projection (3D)	3084 x 3412 px

Layer Name	Data Type	Physical Voxel Resolution
fafb_v14_clahe	Image slice (2D)	x 256 nm, y 256 nm, z 320 nm
fafb-ffn1-20190805	Segmentation slice (2D)	x 16 nm, y 16 nm, z 40 nm

x4 resolution stats, but the FOV will stay the same despite the increase in pixels (keep FOV checked)

Screenshot Size	6184 x 3412 px
Panel Type	Pixel Resolution
Slice view (2D)	3084 x 3412 px
Perspective projection (3D)	3084 x 3412 px

Layer Name	Data Type	Physical Voxel Resolution
fafb_v14_clahe	Image slice (2D)	x 64 nm, y 64 nm, z 80 nm
fafb-ffn1-20190805	Segmentation slice (2D)	x 8 nm, y 8 nm, z 40 nm

Options for improving this

Change the pixels in the canvas

The best option that comes to mind is to actually change the number of pixels in the canvas while the user is interacting with the screenshot. At the moment this is avoided until the screenshot is initiated. This is because, for example picking an x4 resolution could result in a high burden on the user. It's possible though that will be ok. And if it causes issues, that could be overcome with a "confirm setting the resolution higher than the recommended 4k x 4k"

This would also have the added benefit that we could start downloading the required data for an x4 screenshot while in the preview. Previously this data outside of the viewport would start downloading when the screenshot was initiated not during preview.

I've just added a commit that makes this change. Please let me know what you think, and very open to other suggestions or further improvements. Can also do some further testing to see if the canvas pixel change causes issues for users during the preview.

Screencast.from.2024-11-20.12-18-24.mp4

[jbms]

Thanks --- I think changing the canvas size while the user is configuring the settings is okay --- after all it will be happening anyway when the screenshot is actually generated.

I noticed strange behavior when I choose 4x --- it looks like from your screencast that it doesn't happen for you, though. See video below.

cast.webm

[seankmartin]

Ah good spot, thank you. Turns out we didn't have enough pixels on the screen to come across this

I set a higher resolution in dev-tools (see screenshot) and I can replicate the problem when I do that.

I'll look into what might be causing the problem over a certain pixel threshold, but very open to thoughts if any come to mind on this.

[seankmartin]

I dove into the code and I couldn't find anything wrong with the calculations around the canvas size in the display context or on the screenshot side.

The only thing I could guess is that this is a side effect of the warning message that screenshots could fail if they became too large. I just pushed a commit that hard caps the scale factor to remain within these limits.

The exact value cap is 5100 x 5100 pixels. This number is a little bit uncertain, but the screenshot seems to work well coming from a 1920 x 1080 monitor. Such a monitor resolution is fairly common, so seemed a decent base point. A 1920x1080 monitor tends to have about a 1850 x 878 px rendered area in neuroglancer, taking into account borders around viewport canvases and the top bar in neuroglancer etc. The square root of 1850 x 878 * 16 (for x4 scaling in the screenshot menu) is 5097. So a value of 5100 x 5100 should mean that the cap is never hit on a 1920 x 1080 monitor. While bigger monitors may hit the cap depending on their exact setup in neuroglancer.

If this value needs further tuning it is just one constant to change and it would get passed through from that.