Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MicroManager: use PositionName key/value for storing the ImageName #3089

Merged
merged 3 commits into from
Apr 11, 2018

Conversation

sbesson
Copy link
Member

@sbesson sbesson commented Mar 1, 2018

See https://trello.com/c/5KT3Bvie/1-idr0040-aymoz-singlecell

Currently the Micro-Manager reader uses one of the TIFF filename for populating the ImageName metadata. For typical replica experiments across multiple positions (Pos0, Pos1,...), this results in duplicate image names. This commit now parses the PositionName key/value in the metadata JSON and use it if available to fill the ImageName.

This PR will probably affect the metadata of existing filesets, require a configuration change in which case it might be more suitable for a minor release increment.

/cc @dominikl

sbesson added 2 commits March 1, 2018 07:49
Currently the Micro-Manager reader uses one of the TIFF filename for populating
the ImageName metadata. For typical replica experiments across multiple
positions (Pos0, Pos1,...), this results in duplicate image names.
This commit now parses the PositionName key/value in the metadata JSON and use
it if available to fill the ImageName.
@sbesson
Copy link
Member Author

sbesson commented Mar 1, 2018

With the last commit, https://ci.openmicroscopy.org/job/BIOFORMATS-DEV-merge-repository-subset/552/ indicates the following changes

sbesson@ls30630:tmp $ grep FAILED consoleText 
   [testng] [2018-03-01 08:12:55,441] [pool-1-thread-3] 	ImageNames: FAILED (Series 0 (got 'Pos1', expected 'img_000000000_00-BF0_000.tif'))
   [testng] [2018-03-01 08:12:55,520] [pool-1-thread-4] 	ImageNames: FAILED (Series 0 (got 'Pos0', expected 'img_000000000_00-BF0_000.tif'))
   [testng] [2018-03-01 08:13:44,663] [pool-1-thread-2] 	ImageNames: FAILED (Series 0 (got 'Pos0', expected 'Test_Dataset_MultiPosition_Stack_1_MMStack_Pos0_metadata.txt'))
   [testng] [2018-03-01 08:14:05,609] [pool-1-thread-1] 	ImageNames: FAILED (Series 0 (got '1-Pos_000_001', expected 'hyper2.0_1_MMStack_1-Pos_000_001_metadata.txt'))
   [testng] [2018-03-01 08:14:32,532] [pool-1-thread-4] 	ImageNames: FAILED (Series 0 (got 'Pos3', expected 'img_000000000_00-BF0_000.tif'))
   [testng] [2018-03-01 08:14:51,017] [pool-1-thread-3] 	ImageNames: FAILED (Series 0 (got '1-Pos_001_001', expected 'hyper2.0_1_MMStack_1-Pos_001_001_metadata.txt'))
   [testng] [2018-03-01 08:14:51,888] [pool-1-thread-3] 	ImageNames: FAILED (Series 0 (got '1-Pos_001_000', expected 'hyper2.0_1_MMStack_1-Pos_001_000_metadata.txt'))
   [testng] [2018-03-01 08:15:07,042] [pool-1-thread-4] 	ImageNames: FAILED (Series 0 (got 'Pos5', expected 'img_000000000_00-BF0_000.tif'))
   [testng] [2018-03-01 08:15:20,726] [pool-1-thread-4] 	ImageNames: FAILED (Series 0 (got 'Pos4', expected 'img_000000000_00-BF0_000.tif'))
   [testng] [2018-03-01 08:15:33,044] [pool-1-thread-4] 	ImageNames: FAILED (Series 0 (got 'Pos2', expected 'img_000000000_00-BF0_000.tif'))
   [testng] [2018-03-01 08:15:45,439] [pool-1-thread-4] 	ImageNames: FAILED (Series 0 (got 'Pos0', expected 'MMStack_Pos0_metadata.txt'))
   [testng] [2018-03-01 08:15:53,740] [pool-1-thread-1] 	ImageNames: FAILED (Series 0 (got '1-Pos_000_000', expected 'hyper2.0_1_MMStack_1-Pos_000_000_metadata.txt'))
   [testng] [2018-03-01 08:15:54,668] [pool-1-thread-1] 	ImageNames: FAILED (Series 0 (got 'Pos0', expected 'img_000000000__000.tif'))
BUILD FAILED

Excluding for now to reduce the impact on other Bio-Formats work and will discuss with @dominikl later today. Leaving @dgault and @melissalinkert to comment on whether they would think such a change would be acceptable on the mainline Bio-Formats to minimize potential further divergences with the IDR branch.

@sbesson
Copy link
Member Author

sbesson commented Mar 1, 2018

--rebased-to #3092

@melissalinkert
Copy link
Member

This seems fine to me, and the configuration changes make sense. Only question would be if adding the name field to Position affects memo files, but I guess this will become obvious via #3092?

@sbesson
Copy link
Member Author

sbesson commented Mar 1, 2018

Thanks @melissalinkert, local testing suggested the memo files were invalidated by the new field addition. bbd2fb8 should reduce the impact by making the name transient.

@dgault dgault added this to the 5.8.2 milestone Mar 23, 2018
@dgault
Copy link
Member

dgault commented Apr 11, 2018

With bbd2fb8 the existing memo files are unaffected and this is suitable for a patch release.

Tested manually using sample datasets which are affected based on https://github.com/openmicroscopy/data_repo_config/pull/274/files. The files continue to be read and displayed correctly, the only metadata change seen is to the image name which now matches the position name when available. This makes more sense than having duplicate image names using the filename.

Builds and tests are green with both this and the associated configuration PR included.

@dgault dgault merged commit 1b827e6 into ome:develop Apr 11, 2018
@sbesson sbesson deleted the micromanager_position branch April 16, 2018 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants