Merge pull request #8 from nasa/dev

2i2c JupyterHub Updates

.gitignore

@@ -3,6 +3,7 @@
 *__pycache__*
 
 # local development
+*.tif*
 *.img*
 *.envi*
 *.hdr*

README.md

@@ -1,3 +1,7 @@
+> **_Fall 2023 ECOSTRESS Science and Application Team Meeting:_**  This workshop is hosted by NASA’s Land Processes Distributed Activate Archive [LP DAAC](https://lpdaac.usgs.gov/) and NASA Jet Propulsion Laboratory [JPL](https://www.jpl.nasa.gov/) with support from the NASA [Openscapes](https://nasa-openscapes.github.io/) project. Hands-on exercises will be executed from a [Jupyter Hub on the 2i2c cloud instance](https://openscapes.2i2c.cloud/). Your GitHub username is used to enable access to the cloud instance during the workshop. Please pass along your Github Username to get access if you have not already.
+
+> To open the JupyterHub and clone the VITALS repository automatically you can use the following link: [https://tinyurl.com/yckery74](https://tinyurl.com/yckery74).
+
 # VITALS
 
 Welcome to the **VSWIR Imaging and Thermal Applications, Learning, and Science** Repository! This repository provides Python Jupyter notebooks to help users work with visibile to short-wave infrared imaging spectroscopy data, thermal infrared data, and other products from the Earth Surface Mineral Dust Source Investigation (EMIT) and ECOsystem Spaceborne Thermal Radiometer Experiment on Space Station (ECOSTRESS) missions. The instruments overlapping fields of view provide an unprecedented opportunity to demonstrate the compounded benefits of working with both datasets. In the interest of open science this repository has been made public but is still under active development. Make sure to consult the CHANGE_LOG.md for the most recent changes to the repository. Contributions from all parties are welcome.

python/01_Finding_Concurrent_Data.ipynb

@@ -4,17 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "> If running this notebook locally, you will find instructions to set up a compatible environment in the `setup` folder. If running on the Openscapes 2i2c Cloud Instance, please uncomment and run the cell below to install the `scikit-image` python library that needs to be added to the default 2i2c environment. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# # Uncomment and run this cell if on the Openscapes 2i2c Cloud Instance to install the scikit-image library\n",
-    "# !pip install scikit-image"
+    "> If running this notebook locally, you will find instructions to set up a compatible environment in the `setup` folder. If running on the Openscapes 2i2c Cloud Instance for a Workshop, no additional setup is required."
    ]
   },
   {
@@ -105,17 +95,7 @@
    "source": [
     "### 1.2 NASA Earthdata Login Credentials\n",
     "\n",
-    "To download or stream NASA data you will need an Earthdata account, you can create one [here](https://urs.earthdata.nasa.gov/home). We will use the `login` function from the `earthaccess` library for authentication. Adding the `persist=True` argument will create a local `.netrc` file if it doesn't exist, or add your login info to an existing `.netrc` file. If no Earthdata Login credentials are found in the `.netrc` you'll be prompted for them. This step is not necessary to conduct searches, but is needed to download or stream data."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Authenticate with Earthdata Login\n",
-    "earthaccess.login(persist=True)"
+    "To download or stream NASA data you will need an Earthdata account, you can create one [here](https://urs.earthdata.nasa.gov/home). We will use the `login` function from the `earthaccess` library for authentication before downloading at the end of the notebook. This function can also be used to create a local `.netrc` file if it doesn't exist, or add your login info to an existing `.netrc` file. If no Earthdata Login credentials are found in the `.netrc` you'll be prompted for them. This step is not necessary to conduct searches, but is needed to download or stream data."
    ]
   },
   {
@@ -345,7 +325,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "#results"
+    "len(results)"
    ]
   },
   {
@@ -531,7 +511,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Our first step toward filtering the datasets will be to add a a column with a `datetime`. You may have noticed that the date format is similar for ECOSTRESS and EMIT, but the ECOSTRESS data has an additional fractional seconds. To account for this discrepancy, pass the `format='ISO8601'`argument to the `to_datetime` function."
+    "Our first step toward filtering the datasets will be to add a column with a `datetime`. \n",
+    "\n",
+    "> **You may have noticed that the date format is similar for ECOSTRESS and EMIT, but the ECOSTRESS data has an additional fractional seconds. If using the recommended `lpdaac_vitals` Windows environment, you will need to pass the `format='ISO8601'`argument to the `to_datetime` function, like in the commented out line.**"
    ]
   },
   {
@@ -540,7 +522,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gdf['datetime_obj'] = pd.to_datetime(gdf['start_datetime'], format='ISO8601')"
+    "gdf['datetime_obj'] = pd.to_datetime(gdf['start_datetime'])\n",
+    "# gdf['datetime_obj'] = pd.to_datetime(gdf['start_datetime'], format='ISO8601')"
    ]
   },
   {
@@ -902,7 +885,25 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now we can download all of the associated assets, or retrieve the URLS and further filter them to specifically what we want."
+    "Now we can download all of the associated assets, or retrieve the URLS and further filter them to specifically what we want. \n",
+    "\n",
+    "First, log into Earthdata using the `login` function from the `earthaccess` library. The `persist=True` argument will create a local `.netrc` file if it doesn't exist, or add your login info to an existing `.netrc` file. If no Earthdata Login credentials are found in the `.netrc` you'll be prompted for them. As mentioned in section 1.2, this step is not necessary to conduct searches, but is needed to download or stream data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "earthaccess.login(persist=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we can download all assets using the following cell."
    ]
   },
   {
@@ -912,7 +913,6 @@
    "outputs": [],
    "source": [
     "# # Download All Assets for Granules in Filtered Results\n",
-    "# earthaccess.login(persist=True)\n",
     "# earthaccess.download(filtered_results, '../data/')"
    ]
   },
@@ -973,7 +973,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Uncomment the cell below (select all, then ctrl+/) and download the data that we've filtered. If you get an authentication error, be sure you ran the cell in Section 1.2 above."
+    "Uncomment the cell below (select all, then ctrl+/) and download the data that we've filtered. "
    ]
   },
   {
@@ -982,16 +982,19 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# # Get Https Session using Earthdata Login Info\n",
-    "# fs = earthaccess.get_fsspec_https_session()\n",
+    "# # Get requests https Session using Earthdata Login Info\n",
+    "# fs = earthaccess.get_requests_https_session()\n",
     "# # Retrieve granule asset ID from URL (to maintain existing naming convention)\n",
     "# for url in filtered_asset_links:\n",
     "#     granule_asset_id = url.split('/')[-1]\n",
     "#     # Define Local Filepath\n",
     "#     fp = f'../data/{granule_asset_id}'\n",
     "#     # Download the Granule Asset if it doesn't exist\n",
     "#     if not os.path.isfile(fp):\n",
-    "#         fs.download(url, fp)"
+    "#         with fs.get(url,stream=True) as src:\n",
+    "#             with open(fp,'wb') as dst:\n",
+    "#                 for chunk in src.iter_content(chunk_size=64*1024*1024):\n",
+    "#                     dst.write(chunk)"
    ]
   },
   {
@@ -1013,7 +1016,7 @@
     "Voice: +1-866-573-3222  \n",
     "Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹  \n",
     "Website: <https://lpdaac.usgs.gov/>  \n",
-    "Date last modified: 09-27-2023  \n",
+    "Date last modified: 10-12-2023  \n",
     "\n",
     "¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I. "
    ]

setup/setup_instructions.md

@@ -41,8 +41,7 @@ If you're having trouble creating a compatible Python Environment or having an i
 For Windows:
 
 ```cmd
-mamba create -n lpdaac_vitals -c conda-forge --yes python=3.11 fiona=1.8.22 gdal hvplot geoviews rioxarray rasterio jupyter geopandas earthaccess jupyter_bokeh h5py 
-h5netcdf spectral scikit-image
+mamba create -n lpdaac_vitals -c conda-forge --yes python=3.11 fiona=1.8.22 gdal hvplot geoviews rioxarray rasterio jupyter geopandas earthaccess jupyter_bokeh h5py h5netcdf spectral scikit-image
 ```
 
 For MacOSX: