Automate documentation testing with a PR

[duvivier]

We want to integrate test building of documentation for any PR that has modifications to the /doc/ directory (or just any pull request)? Possible ways of doing this include possibly using a probot or incorporating it into the Travis CI tests.

[phil-blain]

This is a planned feature of readthedocs, apparently coming this year !
readthedocs/readthedocs.org#1340

[duvivier]

This is great news. It might just be worth the wait of holding out to see what they come up with instead of inventing the wheel ourselves.

[phil-blain]

See also the project board https://github.com/orgs/readthedocs/projects/8

Apparently the feature is ready to beta test: https://blog.readthedocs.com/building-docs-for-pull-requests/ @duvivier if you want to try it out.

[apcraig]

We just received early access to the PR capability with readthedocs and I have turned on the webhook for readthedocs in both Icepack and CICE to "pull request" as directed by readthedocs folks. We'll have to see what happens. My hope is with the next pull request, we get a new link in the PR pointing to the readthedocs documentation build. We'll see.

[apcraig]

This seems to be working now in both Icepack and CICE. I will close this for now. If we need to reopen or need to open a new issue if something comes up, we can.

[phil-blain]

Just to document, what was missing to make it work ?

[apcraig]

This is a good question. The final piece of the puzzle was to go into my personal settings, under applications, authorized OAuth apps, click on readthedocs, then make sure readthedocs was authorized in the CICE-Consortium.

The permissions thing is something I still don't totally understand. I have to do things in my personal settings under github and readthedocs to allow the CICE-Consortium Project to interact between github and readthedocs. I guess this is because I have some admin permissions, but it would make more sense if I set something in the CICE-Consortium settings in github and readthedocs as an admin to set the permissions.

The other thing that had to be done was in the github CICE-Consortium Icepack repo settings, make sure the webhook for readthedocs had Pull Requests turned on.

Finally, something was done on the readthedocs side outside my control because this was a beta feature. So it had to be turned on for the cice-consortium. I assume once this is formally supported in readthedocs, it will be "on" for all projects.

As I said, it's working and I understand what I did. But I'm still fuzzy how it all works.

[duvivier]

@apcraig Thanks so much for ironing out these details. Do you happen to know if anyone with the admin settings for the consortium (Elizabeth for example) could also make the changes to allow this? Or did it have to be you?

[apcraig]

I'm pretty sure anyone with the same permissions as I can do the same thing.