Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setSLMImage() with np.arrays #88

Merged
merged 4 commits into from
Oct 18, 2023

Conversation

hinderling
Copy link
Contributor

@hinderling hinderling commented Oct 10, 2023

This code should fix issues with setSLMImage(). Until now, images had to be converted to char * which lead to issues with values > '\x7f' (127) because they don't match to valid ASCII characters (see discussion). For the genericSLMDevice, this meant that gray is the brightest color that could be achieved. Allowing for np.arrays removes one data conversion step for the user, and matches the behaviour of pycro-manager.

Accepted inputs with this PR:

  • 8bit grayscale [h,w] of type uint8
  • 8bit RGB [h,w,c] of type uint8, color order [RGB]
  • As previously: char * with either w*h bytes or w*h*4 bytes (for imgRGB32), so that existing scripts are not broken.

Input formats I chose not to accept:

  • 32bit grayscale np.arrays, as there are currently no DMDs supporting it, and it leads to unexpected behaviour with genericSLMDevice
  • 2bit binary np.arrays, as different DMDs use different values to designate an on-pixel (1 or 255) and there is currently no way to extract it.

Thoughts on further improvement:

  • Find a way to report actual bit depth of DMDs (e.g. Mightex Polygon1000 reports 8bit but is only 2bit). This would allow us to use 2bit np.arrays that behave the same on different hardware setups. This is however complicated by the fact that some DMDs support multiple bit depths (e.g. genericSLMDevice reports 4byte but also accepts 8bit).
  • Testing on other DMD hardware

Caught errors:

  • Wrong width/height of np.array:
    RuntimeError: Image dimensions are wrong for this SLM. Expected (1140, 912), but received (1141, 912)
  • Wrong numpy type (e.g. float):
    RuntimeError: Pixel array type is wrong. Expected uint8.
  • Sending RGB image to DMD that reports lower bit depth:
    RuntimeError: Number of bytes per pixel in pixels is greater than expected. Received: 3, Expected: 1
  • Everything else:
    RuntimeError: Pixels must be a 2D numpy array [h,w] of uint8s , or a 3D numpy array [h,w,c] of uint8s with 3 color channels [R,G,B]

I chose not to highlight that also char * is supported because of the above mentioned issues.

Tested on Windows with genericSLMDevice, Andor Mosaic 3, Mightex Polygon1000.

Copy link
Contributor

@tlambert03 tlambert03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @hinderling!
couple quick notes on the stubs

pymmcore/__init__.pyi Show resolved Hide resolved
pymmcore/__init__.pyi Outdated Show resolved Hide resolved
pymmcore/__init__.pyi Outdated Show resolved Hide resolved
pymmcore/__init__.pyi Outdated Show resolved Hide resolved
pymmcore/__init__.pyi Outdated Show resolved Hide resolved
Copy link
Contributor Author

@hinderling hinderling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!!

@marktsuchida marktsuchida merged commit 3c6f84d into micro-manager:main Oct 18, 2023
17 checks passed
@marktsuchida
Copy link
Member

@hinderling Many thanks for the patch and for the meticulous description, too!
The remaining issues (regarding bit depth differences etc.) need to be solved first in MMDevice and MMCore (and each device); I've added a note to micro-manager/mmCoreAndDevices#34 but feel free to add any thoughts there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants