Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OME-Zarr NGFF v0.5 dataset exploration #8122

Merged
merged 12 commits into from
Oct 23, 2024
Merged

OME-Zarr NGFF v0.5 dataset exploration #8122

merged 12 commits into from
Oct 23, 2024

Conversation

frcroth
Copy link
Member

@frcroth frcroth commented Oct 14, 2024

URL of deployed dev instance (used for testing):

  • https://___.webknossos.xyz

Since on GitHub the diff is very big: I renamed the NgffExplorer to NgffV0_4Explorer, created NgffV0_5Explorer, extracted common behavior to the NgffExplorationUtils.

Steps to test:

TODOs:

  • ...

Issues:


(Please delete unneeded items, merge only when none are left open)

@frcroth frcroth requested a review from fm3 October 14, 2024 15:01
@frcroth frcroth marked this pull request as ready for review October 14, 2024 15:01
@frcroth frcroth changed the title Implement exploration for ngff v0.5 OME-Zarr NGFF v0.5 dataset exploration Oct 14, 2024
Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Works for me, and code mostly looks good :) I added just a few comments

Vec3Double(xFactors.product, yFactors.product, zFactors.product)
}

protected def getShape(dataset: NgffDataset, path: VaultPath): Fox[Array[Int]] = ???
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can the = ??? be removed? Then the compiler should check that all classes implementing the trait implement this.

nameFromPath = guessNameFromPath(remotePath)
name = multiscale.name.getOrElse(nameFromPath)
layerTuples <- Fox.serialCombined((0 until channelCount).toList)({ channelIndex: Int =>
for {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of these functions have gotten pretty big. Maybe it would make sense to extract some parts into separate named functions? Maybe the nested for comprehensions are a good candidate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create createLayers

additionalAxes = Some(additionalAxes),
defaultViewConfiguration = Some(viewConfig),
)
} yield (layer, VoxelSize(voxelSizeFactor, unifiedAxisUnit))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like there is still quite a bit of duplication between 0_5 and 0_4. I wonder if some more code could be extracted into functions to be moved into the new trait? But I might have overlooked some subtle differences.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zarrMagFromNgffDataset needs to remain duplicated because it uses getZarrHeader, which returns differnt headers (case classes, so I don't think we can use a trait here).

But I moved a liitle more.

@frcroth frcroth requested a review from fm3 October 16, 2024 09:29
Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, much cleaner :) I also re-tested. Just one last little suggestion

fileAsString <- tryo(new String(fileBytes, StandardCharsets.UTF_8)).toFox ?~> "dataset.explore.failed.readFile"
parsed <- JsonHelper.parseAndValidateJson[T](fileAsString)
} yield parsed
path.parseAsJson[T]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this can be inlined now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function is a single line now, so I would suggest deleting the function and replacing all usages of parseJsonFromPath(path) directly by path.parseAsJson

@frcroth frcroth requested a review from fm3 October 21, 2024 11:34
@frcroth frcroth merged commit bedb16e into master Oct 23, 2024
2 checks passed
@frcroth frcroth deleted the explore-ngff-v0.5 branch October 23, 2024 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Zarr3 / OME NGFF V0.5 Exploration
2 participants