-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ZarrReader: Update getUsedFiles to use the noPixels flag #41
Conversation
@dgault - Just discussed this PR at IDR meeting: - I realise that this isn't going to work when importing NGFF data into OMERO because the server will not be able to find the chunks after import. Currently, on import you will get a symlink created under ManagedRepo to e.g. What we currently do manually (after in-place import) is to replace e.g. But currently we have no logic to create symlinks to directories during in-place import, only symlinks to files. Also, if we want this to work with non in-place import then we'd still want to copy over all the chunks to the |
Closing under the assumption that @will-moore's not currently using this in his workflow. It's also involved in current conflicts. If testing shows it's needed, I'll re-open. |
So with #57, I now have omero-mkngff working. However, if I omit the chunks (which is desired to not overwhelm the database), then on rendering I see what I assume you were running into, @will-moore:
Perhaps, what we actually need is to have
cc: @sbesson |
.forEach(path -> usedFiles.add(path.toFile().getAbsolutePath())); | ||
paths.filter(Files::isRegularFile) | ||
.forEach(path -> {if (!noPixels || | ||
(noPixels && (path.endsWith(".zgroup") || path.endsWith(".zattrs") || path.endsWith(".xml")))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: for mkngff, the logic I used was "don't enter directories that don't have a .zgroup or .zarray". That might be safer if others start using the "add your own random file" strategy.
This is an update to the getUsedFiles method to use the
noPixels
parameter, which before now had been ignored like in many other readers. With this PR calling getUsedFiles with noPixels set to true should return the metadata files for the dataset. For the ZarrReader this will be the zattrs and zgroup files along with any xml files that may also be present.