Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ROI tables, dimension separator & build_omero_channel_metadata changes #32

Merged
merged 42 commits into from
Sep 14, 2023
Merged

Add ROI tables, dimension separator & build_omero_channel_metadata changes #32

Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
f316614
Add write_roi_table function
jluethi May 24, 2023
4233cb2
Add ROI table calculation for 2D planes
jluethi May 24, 2023
8afc267
Add basic scripts to process some test data (prototype for Fractal ta…
jluethi May 24, 2023
a12a6c6
Cleanup MetaSeriesUtils
jluethi May 24, 2023
a719be3
Add ROI table generation to montage_stage_pos_image_YX and update tests
jluethi May 24, 2023
5fac806
Update Zarr tests to also expect roi_tables
jluethi May 24, 2023
1e816b3
Add check to build_omero_channel_metadata for start & end values, cha…
jluethi May 25, 2023
cf3cdb4
Improve 3D saving in Fractal tasks
jluethi Jun 2, 2023
d36951d
Set the dimension_separator to / to make it work in napari
jluethi Jun 2, 2023
76ebb7b
Change Fractal tasks to use mode instead of is_2D flag
jluethi Jun 2, 2023
f30dfa4
Improve table metadata updates
jluethi Jun 2, 2023
0d21214
Change create ome zarr into a Fractal task
jluethi Jun 2, 2023
0dff70b
Change fractal task filenames, organize them as actual task functions
jluethi Jun 2, 2023
c917103
Add main functions, update dependencies, add Fractal manifest
jluethi Jun 2, 2023
dfb04d0
Add pydantic dependency for fractal-tasks extra
jluethi Jun 5, 2023
1cd635f
Add tasks manually to Fractal
jluethi Jun 5, 2023
e9211f0
Make Fractal tasks collectable by Fractal server
jluethi Jun 8, 2023
d534795
Fix bug: Channels have different z len => pick largest for ROI table
jluethi Jun 8, 2023
57ba33c
Add tests to check ROI table content
jluethi Jun 8, 2023
726eb5f
Update examples with roi tables
jluethi Jun 8, 2023
ecb62bf
Update Fractal example to have relative paths
jluethi Jun 8, 2023
8487322
Add tests for Fractal tasks
jluethi Jun 8, 2023
c26fa4c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 8, 2023
8bcd1e3
pre-commit cleanup
jluethi Jun 8, 2023
7c8a262
Remove all Fractal task parts from faim-hcs
jluethi Jun 13, 2023
0c2103b
Update anndata dependency & import from io, not experimental
jluethi Jun 14, 2023
0c3084e
Change Z step calculation
jluethi Jun 22, 2023
c6e3214
Update expected Z length in ROI table in MetaSeries tests
jluethi Jun 22, 2023
3b107ec
Parse the dimension_separator through all levels
jluethi Jun 22, 2023
f0e7780
Add lowest_res_target to _compute_chunk_size_cyx
jluethi Jun 22, 2023
356f96b
Expose max_levels, max_size, lowest_res_target in top-level functions…
jluethi Jun 22, 2023
5409d19
Simplify write_roi_table
jluethi Jun 22, 2023
0ca98ef
Simplify montage_grid_image_YX
jluethi Jun 22, 2023
4acb8a9
Merge branch 'main' into fractal-roi-tables
jluethi Jun 23, 2023
1e4fa68
Refactor ROI table functions to have column names in a central place
jluethi Jun 23, 2023
0dee855
Update src/faim_hcs/MetaSeriesUtils.py
jluethi Jul 4, 2023
f57be3f
Improve docstrings
jluethi Jul 11, 2023
c471dd1
Use column names as fixture in tests
jluethi Jul 11, 2023
45cae48
Improve montage_stage_pos_image_YX docstring
jluethi Sep 3, 2023
1e076c3
Switch to kwargs for passing args for _compute_chunk_size_cyx
jluethi Sep 3, 2023
25bdf0d
Fix fsspec & anndata dependencies
jluethi Sep 4, 2023
723208d
Increase compatible anndata version
jluethi Sep 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 68 additions & 62 deletions examples/Create ome-zarr from Single-Plane Multi-Field Acquisition.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
"outputs": [],
"source": [
"from faim_hcs.io.MolecularDevicesImageXpress import parse_single_plane_multi_fields\n",
"from faim_hcs.Zarr import build_zarr_scaffold, write_cyx_image_to_well, PlateLayout\n",
"from faim_hcs.Zarr import build_zarr_scaffold, write_cyx_image_to_well, PlateLayout, write_roi_table\n",
"from faim_hcs.MetaSeriesUtils import get_well_image_CYX, montage_grid_image_YX\n",
"from faim_hcs.UIntHistogram import UIntHistogram\n",
"import shutil\n",
Expand Down Expand Up @@ -131,8 +131,8 @@
" <td>Projection-Mix</td>\n",
" <td>E08</td>\n",
" <td>s2</td>\n",
" <td>w2</td>\n",
" <td>66923EBB-9960-4952-8955-D1721D112EE2</td>\n",
" <td>w1</td>\n",
" <td>B38C01F5-0D36-4A29-9F5A-BE62B6F7F73F</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -141,10 +141,10 @@
" <td>2023-02-21</td>\n",
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E07</td>\n",
" <td>s2</td>\n",
" <td>w1</td>\n",
" <td>DCFD1526-D063-4F8B-9E51-F1BD2EBD9F1A</td>\n",
" <td>E08</td>\n",
" <td>s1</td>\n",
" <td>w2</td>\n",
" <td>81928711-999D-41F6-B88C-999513D4C092</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -153,10 +153,10 @@
" <td>2023-02-21</td>\n",
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E07</td>\n",
" <td>s1</td>\n",
" <td>w1</td>\n",
" <td>E94C24BD-45E4-450A-9919-257C714278F7</td>\n",
" <td>E08</td>\n",
" <td>s2</td>\n",
" <td>w3</td>\n",
" <td>CCE83D85-0912-429E-9F18-716A085BB5BC</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -166,9 +166,9 @@
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E07</td>\n",
" <td>s1</td>\n",
" <td>w2</td>\n",
" <td>B14915F6-0679-4494-82D1-F80894B32A66</td>\n",
" <td>s2</td>\n",
" <td>w3</td>\n",
" <td>B0A47337-5945-4B26-9F5F-4EBA468CDBA9</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -177,10 +177,10 @@
" <td>2023-02-21</td>\n",
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E08</td>\n",
" <td>E07</td>\n",
" <td>s1</td>\n",
" <td>w3</td>\n",
" <td>DD77D22D-07CB-4529-A1F5-DCC5473786FA</td>\n",
" <td>w2</td>\n",
" <td>B14915F6-0679-4494-82D1-F80894B32A66</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -189,10 +189,10 @@
" <td>2023-02-21</td>\n",
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E08</td>\n",
" <td>E07</td>\n",
" <td>s2</td>\n",
" <td>w3</td>\n",
" <td>CCE83D85-0912-429E-9F18-716A085BB5BC</td>\n",
" <td>w2</td>\n",
" <td>607EE13F-AB5E-4E8C-BC4B-52E1118E7723</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -201,10 +201,10 @@
" <td>2023-02-21</td>\n",
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E07</td>\n",
" <td>E08</td>\n",
" <td>s1</td>\n",
" <td>w3</td>\n",
" <td>BB87F860-FC67-4B3A-A740-A9EACF8A8F5F</td>\n",
" <td>DD77D22D-07CB-4529-A1F5-DCC5473786FA</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -226,9 +226,9 @@
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E07</td>\n",
" <td>s2</td>\n",
" <td>w3</td>\n",
" <td>B0A47337-5945-4B26-9F5F-4EBA468CDBA9</td>\n",
" <td>s1</td>\n",
" <td>w1</td>\n",
" <td>E94C24BD-45E4-450A-9919-257C714278F7</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -239,8 +239,8 @@
" <td>Projection-Mix</td>\n",
" <td>E07</td>\n",
" <td>s2</td>\n",
" <td>w2</td>\n",
" <td>607EE13F-AB5E-4E8C-BC4B-52E1118E7723</td>\n",
" <td>w1</td>\n",
" <td>DCFD1526-D063-4F8B-9E51-F1BD2EBD9F1A</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -250,9 +250,9 @@
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E08</td>\n",
" <td>s1</td>\n",
" <td>s2</td>\n",
" <td>w2</td>\n",
" <td>81928711-999D-41F6-B88C-999513D4C092</td>\n",
" <td>66923EBB-9960-4952-8955-D1721D112EE2</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -261,10 +261,10 @@
" <td>2023-02-21</td>\n",
" <td>1334</td>\n",
" <td>Projection-Mix</td>\n",
" <td>E08</td>\n",
" <td>s2</td>\n",
" <td>w1</td>\n",
" <td>B38C01F5-0D36-4A29-9F5A-BE62B6F7F73F</td>\n",
" <td>E07</td>\n",
" <td>s1</td>\n",
" <td>w3</td>\n",
" <td>BB87F860-FC67-4B3A-A740-A9EACF8A8F5F</td>\n",
" <td>.tif</td>\n",
" <td>../resources/Projection-Mix/2023-02-21/1334/Pr...</td>\n",
" </tr>\n",
Expand All @@ -274,32 +274,32 @@
],
"text/plain": [
" date acq_id name well field channel \\\n",
"0 2023-02-21 1334 Projection-Mix E08 s2 w2 \n",
"1 2023-02-21 1334 Projection-Mix E07 s2 w1 \n",
"2 2023-02-21 1334 Projection-Mix E07 s1 w1 \n",
"3 2023-02-21 1334 Projection-Mix E07 s1 w2 \n",
"4 2023-02-21 1334 Projection-Mix E08 s1 w3 \n",
"5 2023-02-21 1334 Projection-Mix E08 s2 w3 \n",
"6 2023-02-21 1334 Projection-Mix E07 s1 w3 \n",
"0 2023-02-21 1334 Projection-Mix E08 s2 w1 \n",
"1 2023-02-21 1334 Projection-Mix E08 s1 w2 \n",
"2 2023-02-21 1334 Projection-Mix E08 s2 w3 \n",
"3 2023-02-21 1334 Projection-Mix E07 s2 w3 \n",
"4 2023-02-21 1334 Projection-Mix E07 s1 w2 \n",
"5 2023-02-21 1334 Projection-Mix E07 s2 w2 \n",
"6 2023-02-21 1334 Projection-Mix E08 s1 w3 \n",
"7 2023-02-21 1334 Projection-Mix E08 s1 w1 \n",
"8 2023-02-21 1334 Projection-Mix E07 s2 w3 \n",
"9 2023-02-21 1334 Projection-Mix E07 s2 w2 \n",
"10 2023-02-21 1334 Projection-Mix E08 s1 w2 \n",
"11 2023-02-21 1334 Projection-Mix E08 s2 w1 \n",
"8 2023-02-21 1334 Projection-Mix E07 s1 w1 \n",
"9 2023-02-21 1334 Projection-Mix E07 s2 w1 \n",
"10 2023-02-21 1334 Projection-Mix E08 s2 w2 \n",
"11 2023-02-21 1334 Projection-Mix E07 s1 w3 \n",
"\n",
" md_id ext \\\n",
"0 66923EBB-9960-4952-8955-D1721D112EE2 .tif \n",
"1 DCFD1526-D063-4F8B-9E51-F1BD2EBD9F1A .tif \n",
"2 E94C24BD-45E4-450A-9919-257C714278F7 .tif \n",
"3 B14915F6-0679-4494-82D1-F80894B32A66 .tif \n",
"4 DD77D22D-07CB-4529-A1F5-DCC5473786FA .tif \n",
"5 CCE83D85-0912-429E-9F18-716A085BB5BC .tif \n",
"6 BB87F860-FC67-4B3A-A740-A9EACF8A8F5F .tif \n",
"0 B38C01F5-0D36-4A29-9F5A-BE62B6F7F73F .tif \n",
"1 81928711-999D-41F6-B88C-999513D4C092 .tif \n",
"2 CCE83D85-0912-429E-9F18-716A085BB5BC .tif \n",
"3 B0A47337-5945-4B26-9F5F-4EBA468CDBA9 .tif \n",
"4 B14915F6-0679-4494-82D1-F80894B32A66 .tif \n",
"5 607EE13F-AB5E-4E8C-BC4B-52E1118E7723 .tif \n",
"6 DD77D22D-07CB-4529-A1F5-DCC5473786FA .tif \n",
"7 17654C10-92F1-4DFD-98AA-6A01BBD77557 .tif \n",
"8 B0A47337-5945-4B26-9F5F-4EBA468CDBA9 .tif \n",
"9 607EE13F-AB5E-4E8C-BC4B-52E1118E7723 .tif \n",
"10 81928711-999D-41F6-B88C-999513D4C092 .tif \n",
"11 B38C01F5-0D36-4A29-9F5A-BE62B6F7F73F .tif \n",
"8 E94C24BD-45E4-450A-9919-257C714278F7 .tif \n",
"9 DCFD1526-D063-4F8B-9E51-F1BD2EBD9F1A .tif \n",
"10 66923EBB-9960-4952-8955-D1721D112EE2 .tif \n",
"11 BB87F860-FC67-4B3A-A740-A9EACF8A8F5F .tif \n",
"\n",
" path \n",
"0 ../resources/Projection-Mix/2023-02-21/1334/Pr... \n",
Expand Down Expand Up @@ -327,7 +327,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 8,
"id": "948904be",
"metadata": {},
"outputs": [],
Expand All @@ -347,14 +347,14 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 9,
"id": "37f65e50",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d39b97703a4f4214b3c1618843c8f481",
"model_id": "cca2a3f86a72410ca69527e3a9b46e26",
"version_major": 2,
"version_minor": 0
},
Expand All @@ -372,20 +372,26 @@
"for well in tqdm(files['well'].unique()):\n",
" well_files = files[files['well'] == well]\n",
" \n",
" img, hists, ch_metadata, metadta = get_well_image_CYX(\n",
" img, hists, ch_metadata, metadata, roi_tables = get_well_image_CYX(\n",
" well_files=well_files,\n",
" channels=channels,\n",
" assemble_fn=montage_grid_image_YX,\n",
" )\n",
" \n",
" well_group = plate[well[0]][str(int(well[1:]))][0]\n",
" write_cyx_image_to_well(img, hists, ch_metadata, metadta, well_group)"
" write_cyx_image_to_well(img, hists, ch_metadata, metadata, well_group)\n",
" \n",
" # Write all ROI tables\n",
" for roi_table in roi_tables:\n",
" write_roi_table(roi_tables[roi_table], roi_table, well_group)"
]
},
{
"cell_type": "markdown",
"id": "7f5a8f42",
"metadata": {},
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"# Inspect ome-zarr plate\n",
"The data can be opened with the [ome-zarr Napari plugin](https://www.napari-hub.org/plugins/napari-ome-zarr)."
Expand All @@ -408,7 +414,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.0"
"version": "3.9.16"
}
},
"nbformat": 4,
Expand Down
Loading