Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove duplicate DataFolder type #102

Merged
merged 3 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions docs/examples/sans2d.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -394,6 +394,100 @@
"source": [
"pp.plot(sc.collapse(result.hist(), keep='Q'), norm='log')"
]
},
{
"cell_type": "markdown",
"id": "0977a3dc-eb5f-4067-80c5-cc19137cc915",
"metadata": {},
"source": [
"## Loading local files\n",
"\n",
"The data files used above are hosted on an external server, and downloaded on-the-fly (and cached) when computing the result.\n",
"\n",
"It is also possible to load local files from your hard drive, by using the `DataFolder` parameter.\n",
"We also need to insert the `isis.io.to_path` provider which supplies the file paths to the files in the folder.\n",
"\n",
"As an example, we will save our current direct beam to disk, and then re-load it using a pipeline that reads local files.\n",
"\n",
"**Note** that is it not currently possible to mix local and cloud files in the same pipeline with the present setup."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "871743bf-32d0-4df1-a285-ba722a4198ef",
"metadata": {},
"outputs": [],
"source": [
"# Direct beam computation currently uses the `get_path` provider which\n",
"# fetches files from the remote server\n",
"direct_beam = pipeline.get(DirectBeam)\n",
"direct_beam.visualize()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f35f4d14-58f2-437f-826c-b9ac74570e52",
"metadata": {},
"outputs": [],
"source": [
"# Save the direct beam to disk\n",
"db_filename = 'my_local_direct_beam_file.h5'\n",
"direct_beam.compute().save_hdf5(db_filename)"
]
},
{
"cell_type": "markdown",
"id": "8f712a5d-cab9-4878-9b74-5a6c9751e403",
"metadata": {},
"source": [
"We now modify our pipeline by setting the `DataFolder` parameter,\n",
"as well as our new direct beam filename. Finally, we insert the local file provider `to_path`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "27432974-0f06-4549-8ea5-4f75ff42fbc5",
"metadata": {},
"outputs": [],
"source": [
"pipeline[DataFolder] = '.'\n",
"pipeline[DirectBeamFilename] = db_filename\n",
"\n",
"# Insert provider for local files\n",
"pipeline.insert(isis.io.to_path)"
]
},
{
"cell_type": "markdown",
"id": "37be7d9a-92ec-468c-a36e-1c73682b1bda",
"metadata": {},
"source": [
"We can now see that `to_path` uses both the file name and the local folder to create a file path:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a5647adf-5606-4efa-9e18-2c2943d8250d",
"metadata": {},
"outputs": [],
"source": [
"db_local = pipeline.get(DirectBeam)\n",
"db_local.visualize()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "446ef2a3-bf7d-404b-b493-0c65f71c1a03",
"metadata": {},
"outputs": [],
"source": [
"db_local.compute().plot()"
]
}
],
"metadata": {
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/zoom.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@
" cfg = ConfigService.Instance()\n",
" cfg.setLogLevel(3) # Silence verbose load via Mantid\n",
"\n",
" pipeline[sans.isis.DataFolder] = 'zoom_data'\n",
" pipeline[DataFolder] = 'zoom_data'\n",
" for provider in sans.isis.mantidio.providers:\n",
" pipeline.insert(provider)\n",
"except ImportError:\n",
Expand Down
5 changes: 2 additions & 3 deletions src/esssans/loki/io.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Loading and merging of LoKI data.
"""
from functools import reduce
from typing import NewType, Optional
from typing import Optional

import sciline
import scipp as sc
Expand All @@ -13,6 +13,7 @@
from ..common import gravity_vector
from ..types import (
BackgroundRun,
DataFolder,
Filename,
FilenameType,
FilePath,
Expand All @@ -38,8 +39,6 @@
)
}

DataFolder = NewType('DataFolder', str)


def _patch_data(
da: sc.DataArray, sample_position: sc.Variable, source_position: sc.Variable
Expand Down
Loading