Loading datasets without copying them

[DimitriosBellos]

Hi, I am currently using SuRVoS with a big dataset and I was wondering if there is a way to prepare the dataset (name it data.h5, place it in a dedicated folder, name the internal dataset '/data') and on the SuRVoS menu select load the dataset instead of opening it.

I would like to do that as my dataset is 57GB and loading it to memory (RAM) in order to make a copy is not an option. I know SuRVoS make a copy of the input dataset because it normalizes it. However, I don't want to normalize my dataset or at least I would like to have the option not to.

I know SuRVoS was not designed for big datasets, but I am planning to process my dataset region by region and my only obstacle is that SuRVoS has to load it as a whole at the beginning. I hope this change is easily applicable as it would allow people to use SuRVoS for big datasets.

Aha! Link: https://dls1.aha.io/features/D-28

[srikanthnagella]

I am not sure how this can be achieved. I will check the code and will come back to you.

[srikanthnagella]

How about adding a data subset selection tool? Here along with the dataset path you select the start, end and stride in each dimension.

[DimitriosBellos]

I agree. The addition of a feature like that would be truly helpful.