Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking datasets used for this repo #48

Open
2 tasks done
jbusecke opened this issue Apr 10, 2024 · 2 comments
Open
2 tasks done

Tracking datasets used for this repo #48

jbusecke opened this issue Apr 10, 2024 · 2 comments

Comments

@jbusecke
Copy link
Collaborator

jbusecke commented Apr 10, 2024

We should have a place to track all datasets that are ingested specifically for this project (e.g. #43 )

We can then:

  • consolidate all data on the LEAP-Hub
    • Make a list of all datasets that are currently used and how they are brought into the notebook (wget, ...)
    • Come up with a consistent strategy how to ingest future datasets (some thoughts from the LEAP-Pangeo technical documentation)
  • somewhere down the line we can then move all the data to a public target and make everything reproducible everywhere!
@jbusecke
Copy link
Collaborator Author

  • Alternatively we can re-ingest once we actually have a fully public storage location. This depends a bit on priorities.

@suryadheeshjith
Copy link
Collaborator

suryadheeshjith commented Apr 23, 2024

In our current notebooks, we use the following data sources:

  1. Introduction to Xarray: Data downloaded from
    - https://github.com/pangeo-data/tutorial-data/raw/master/sst/NOAA_NCDC_ERSST_v3b_SST-1960.nc
  2. Introduction to XGCM: Data downloaded from
    - https://zenodo.org/record/4421428/files/mitgcm_example_dataset_v2.nc
  3. Basic Plotting with MOM6: Locally stored data present at
    - data/thetao_ePBL_data_timemean_1978-2002.nc
  4. OM4 Animation: Data stored on Leap Pangeo
    - gs://leap-persistent/jbusecke/OM4_m2lines/daily_combined.zarr
  5. Pycnocline Depth using MOM6: Locally stored data present at
    - data/thetao_ePBL_data_timemean_1978-2002.nc
    - data/so_ePBL_data_timemean_1978-2002.nc
    - data/N2_ePBL_data_timemean_1978-2002.nc
  6. Ocean Emulation Animations with U-Net: Data stored on Leap Pangeo
    - gs://leap-persistent/m2lines-data-gallery/grids.nc
    - gs://leap-persistent/m2lines-data-gallery/wet_nan.npy
    - gs://leap-persistent/m2lines-data-gallery/ground_truth.npy
    - gs://leap-persistent/m2lines-data-gallery/model_pred_unet.npy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants