Add an example of ERA5 and GRIB data & visualization to the gallery #3199
Conversation
Codecov Report
@@ Coverage Diff @@
## scipy19-docs #3199 +/- ##
================================================
- Coverage 95.76% 95.41% -0.36%
================================================
Files 64 64
Lines 13188 13188
================================================
- Hits 12630 12583 -47
- Misses 558 605 +47 |
|
Hi @zbruick - thanks for this and sorry for not following up sooner. Do you feel this example is ready for review? |
|
In its current form, I'd say that your example is missing some text (markdown cells) which provide a narrative. See this example - http://xarray.pydata.org/en/scipy19-docs/examples/ROMS_ocean_model.html - for what I am talking about. |
|
Happy to add some markdown to this - that was an oversight. I'll work on that and push it up shortly. |
|
It looks like the doc build is failing with this error It appears that you are loading the data from a local file, rather than from the tutorial datasets api. This won't work, because the local file is not available in the build environment. That's why we added it to the tutorial repo in pydata/xarray-data#14. Can you try something like ds = xr.tutorial.load_dataset('era5-2mt-2019-03-uk.grib', engine='cfgrib') |
|
I just fixed a file naming issue in pydata/xarray-data#18 and can confirm that my code above works for me. |
|
Now we are getting https://dev.azure.com/xarray/xarray/_build/results?buildId=780 |
|
Thanks for pushing that commit and updating the data import call. I forgot I had to switch that from my local file. Let me know if this issue is solvable on my side of things, but that seems like that's an issue with the data file? |
|
Can someone with the proper credentials (e.g. @shoyer) restart the azure build? I can't reproduce the md5 error locally. |
|
I am really stumped here. When I run xr.tutorial.load_dataset('era5-2mt-2019-03-uk.grib', engine='cfgrib')on the scipy19-docs branch, it works fine. |
|
I'm not sure if this helps you, but I noticed two issues in |
|
@zbruick Can you merge in scipy19-docs? That might fix it. |
zbruick
force-pushed
the
grib_example
branch
from
86e8701 to
1c07b35
Compare
|
Rebased on scipy19-docs. Hope that helps! |
|
Oops I think you lost ryan's commit with |
|
Ah, oops. Updated to use that method now. |
|
Still failing. Instead of it should be |
|
unless something else is wrong, either modifying the md5 file (remove the newline) or adding |
|
I removed the newline upstream. |
|
that seems to have worked, the failures should be unrelated. |
|
🤦♂️ finally! |
|
Thanks @keewis |
|
We got it! Thanks everyone! |
|
Unfortunately the RTD build failed! https://readthedocs.org/projects/xray/builds/9700942/ 😭😭😭 This is really trying my patience! |
|
|
|
(roughly) following the instructions in the build log, I can reproduce it raising a 404: $ conda env create --quiet --name scipy19-docs --file doc/environment.yml
$ conda install --yes --quiet --name scipy19-docs mock pillow sphinx sphinx_rtd_theme
(scipy19-docs)$ python -m pip install -U recommonmark readthedocs-sphinx-ext
(scipy19-docs)$ python setup.py install --force
(scipy19-docs)$ cd doc; make htmlI didn't check yet, but maybe the second |
|
Unfortunately, I don't think we can remove the second conda install. RTD
insists on it to ensure that it has the build environment it needs. We
could certainly ask on their GitHub support page, though.
…On Mon, Sep 23, 2019 at 5:59 PM keewis ***@***.***> wrote:
(roughly) following the instructions in the build log, I can reproduce it
raising a 404:
$ conda env create --quiet --name scipy19-docs --file doc/environment.yml
$ conda install --yes --quiet --name scipy19-docs mock pillow sphinx sphinx_rtd_theme
(scipy19-docs)$ python -m pip install -U recommonmark readthedocs-sphinx-ext
(scipy19-docs)$ python setup.py install --force
(scipy19-docs)$ cd doc; make html
I didn't check yet, but maybe the second conda install should not happen,
as all those get installed on env creation?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3199?email_source=notifications&email_token=AAJJFVQVW3X5GAUBKXSBT7DQLFQ7LA5CNFSM4IKVN6A2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7MWVIQ#issuecomment-534342306>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAJJFVVKECGIALOX4DMWUSLQLFQ7LANCNFSM4IKVN6AQ>
.
|
|
err, well, this is somewhat confusing: I'm not sure what pulls it, but it seems something depends on xarray (installed in the creation of the environment). The version that is conda-installed is 0.13.0, so as master does not have the new version of |
|
Thanks @keewis . I just merged master and pushed. hopefully this works |
|
still fails with 404: https://readthedocs.org/projects/xray/builds/9704981/ |
|
The problem is still the same: while the output of the environment printed by sphinx lists the new version as Something that comes to mind is waiting for a new release with the new version of |
|
After some experimentation, it seems this is only the case if $ python setup.py install --forcewith $ python -m pip install .makes this work, even without the merge commit. The docs on the settings mention |
|
👍 Let's see if it works: https://readthedocs.org/projects/xray/builds/9705929/ |
|
OK that almost worked. Now it looks like we need to bump pandas? or dask? |
|
it seems all of these problems boil down to that single command. What gets executed is (essentially) $ python -m pip install --upgrade --upgrade-strategy eager .which upgrades pandas to 0.25.1, but leaves dask as is because it is an optional dependency. This, however, defeats the purpose of version pinning in the conda file (or a requirements file). I can't find an option to tell readthedocs to install without the eager upgrade (normal upgrade is fine). Maybe ask support? The addition of the upgrade strategy happened in readthedocs/readthedocs.org#5635 |
|
Yikes, OK. Yes, it would be good to open an issue on the RTD GitHub page to
discuss our options. --upgrade-strategy eager seems like a bad idea for us
-- it really undermines the value of dependency pinning.
…On Tue, Sep 24, 2019 at 1:31 PM keewis ***@***.***> wrote:
it seems all of these problems boil down to that single command. What gets
executed is (essentially)
$ python -m pip install --upgrade --upgrade-strategy eager .
which upgrades pandas to 0.25.1, but leaves dask as is because it is an
optional dependency. This, however, defeats the purpose of version pinning
in the conda file (or a requirements file).
I can't find an option to tell readthedocs to install without the eager
upgrade (normal upgrade is fine). Maybe ask support? The addition of the
upgrade strategy happened in readthedocs/readthedocs.org#5635
<readthedocs/readthedocs.org#5635>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3199?email_source=notifications&email_token=AAJJFVULDVAUPID5WWAZAF3QLJ2JXA5CNFSM4IKVN6A2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7PWUAY#issuecomment-534735363>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAJJFVRI6S3TML3E27VEYNTQLJ2JXANCNFSM4IKVN6AQ>
.
|
|
Is someone planning to open this RTD issue? Or is there another workaround? I would love to get this scipy19-docs branch merged to master, but this seems to be standing in the way. |
|
I'm currently not planning to because I will probably be offline for the next few days Edit: the only workaround I can imagine is building the docs without installing (which is mentioned in the docs), but I neither know why the current configuration is the way it is or how building the docs without installing works with RTD |
|
looking at this once again, there are a lot of projects that don't install (eg aiohttp (build), but I'm not sure how this is set up or which parts of their configuration comes from somewhere other than their Also, it seems we are using the old version of And finally, it seems strange to me that the As I don't have access to readthedocs I can't really experiment on these suggestions (making this informed guessing at best), so I would like to leave the tracking down and fixing to someone else. |
|
Hi, check out the work I did on the RTD requirements file here, it may help: #3358 |
…ydata#3199) * Adds an example of ERA5 and GRIB data to the gallery * Add markdown narrative cells to GRIB example * Update load method to use xr.tutorial * Fix load method
* Switch doc examples to use nbsphinx (#3105) * switching out examples to use nbsphinx * added jupyter_client to doc env * added ipykernel to doc env * Replace sphinx_gallery with notebook (#3106) * switching out examples to use nbsphinx * added jupyter_client to doc env * moved gallery to notebook * Allow other tutorial filename extensions (#3121) * switching out examples to use nbsphinx * added jupyter_client to doc env * allow non netcdf tutorial files * Added ROMS ocean model example notebook (#3116) * change name of test env to xarray-tests (#3110) * ROMS_ocean_model example added * Allow other tutorial filename extensions (#3121) * switching out examples to use nbsphinx * added jupyter_client to doc env * allow non netcdf tutorial files * Changed load to xr.tutorial.open_dataset(), and added some extra documentation. * change name of test env to xarray-tests (#3110) * ROMS_ocean_model example added * Changed load to xr.tutorial.open_dataset(), and added some extra documentation. * fixed colormap issues leftover from cmocean import * Added intro paragraph to ROMS example notebook, removed comments, and added citation in whats-new. * Add an example of ERA5 and GRIB data & visualization to the gallery (#3199) * Adds an example of ERA5 and GRIB data to the gallery * Add markdown narrative cells to GRIB example * Update load method to use xr.tutorial * Fix load method * require nbsphinx for the documentation builds * add more nbsphinx dependencies * install cfgrib using pip * add the eccodes library to the dependencies * remove the dependency on sphinx-gallery * add the ERA5 GRIB example to the list * update the documentation links Missing: section links in visualization_gallery.ipynb don't work yet, also the one in io.rst (it has a unicode char). * Fix leap year condition in monthly means example (#3464) * Typo correction in docs (#3387) * Update terminology.rst (#3455) Fixed broken link * Error in leap year? I've tried this script; however, it adds +1 to all months of the leap years. It sounds like an error, or I am wrong? So I wrote the condition "and month == 2" line 86 so that only the month of February gets +1. * Fix leap year (#3464) * Update doc/whats-new.rst Co-Authored-By: Deepak Cherian <[email protected]> * fix the reference to the rasterio geocoordinates docs * update whats-new.rst
This was from the SciPy sprint, but failed to get submitted as a PR yet. This was developed by myself and @StephanSiemen to demonstrate visualizing ERA5 GRIB data. The data was already added in pydata/xarray-data#14.