CI: Add link checker & refactor two files into one

.github/scripts/muffet.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+
+# This script is used in Github Actions to do link checking on commits to branches and master, so
+# please edit with care!  See ../workflows/ci.yml for specific usage.
+#
+# If you're doing local development and want to run link checkers locally, please use this script!
+# You'll need:
+# - install go - https://golang.org/doc/install
+# - install muffet - https://github.com/raviqqe/muffet
+# - make sure muffet is in your PATH
+# - run from root of this repo: ./.github/scripts/muffet.sh
+#
+# If we have errors from this script being too aggressive (see how large downloads on
+# download.docker.com are excluded below) you can add additional sites as needed. Please
+# use extremely granular URLs so as not to exclude more than we need to.
+
+muffet http://localhost:1313 \
+  --buffer-size 50000 \
+  --timeout 35 \
+  --concurrency 2 \
+  --ignore-fragments \
+  --exclude ".*demo\.app\.medicmobile\.org.*" \
+  --exclude ".*download\.docker\.com.*" \
+  --exclude ".*www\.npmjs\.com/org/medic/team/developers" \
+  --exclude "https://github\.com/medic/cht-docs/issues/new.*" \
+  --exclude "https://github\.com/medic/cht-core/issues/new.*" \
+  --exclude "https://github\.com/medic/cht-docs/commit.*" \
+  --exclude "https://github\.com/medic/cht-core/commit.*" \
+  --exclude "https://github\.com/medic/cht-docs/edit/master/.*" \
+  --exclude ".*localhost:5984.*"

.github/workflows/ci.yml

@@ -1,40 +1,72 @@
-name: CHT Documentation Site Build
+# Notes:
+#  - All software installed below is pinned so they don't get upgraded and break w/o us knowing why.
+#  - If you bump one version up to latest, please bump them all so we stay current!
+#  - Hugo builds the site twice: once for link checking and once minified for pushing live
+#  - Bash script for muffet is intentionally stand alone so devs/content editors can easily run it locally
 
-on: 
+on:
   push:
     branches:
-      - master
+      - '*' # branch checking is below so all this is one CI file
 
+name: Test site and push live if we're on Master
 jobs:
-  build:
-
+  BuildLinkCheckPushLive:
+    name:
     runs-on: ubuntu-18.04
-
     steps:
-      - name: Git checkout
+
+      - name: All Branches - Git checkout, including recursive call for Docsy theme
         uses: actions/checkout@v2
         with:
           submodules: recursive  # Fetch the Docsy theme
           fetch-depth: 0         # Fetch all history for .GitInfo and .Lastmod
 
-      - name: Install Node depencies
-        uses: actions/setup-node@v1
-        with:
-          node-version: '12.x'
-      - run: npm install
-      - run: npm install -g postcss-cli
-      - run: npm i -D autoprefixer
-
-      - name: Setup hugo
+      - name: All Branches - Install hugo extended v0.71.1
         uses: peaceiris/actions-hugo@v2
         with:
           hugo-version: "0.71.1"
           extended: true
 
-      - name: Build
+      # Pinning to current version as of Aug 3, 2020 for stability
+      - name: All Branches - Install Go v1.14
+        uses: actions/setup-go@v2
+        with:
+          go-version: '1.14'
+
+      # Pinning to current version as of Aug 3, 2020 for stability
+      - name: All Branches - Install Muffet v1.3.3
+        run: env GO111MODULE="on" go get github.com/raviqqe/[email protected]
+
+      # run in background so CI doesn't hang waiting for "ctrl + c".  Sleep so hugo server is ready
+      - name: All Branches - Serve Hugo site
+        run: |
+          hugo server&
+          sleep 5
+
+      - name: All Branches - Run Muffet link checker
+        shell: bash
+        run: ./.github/scripts/muffet.sh
+
+      - name: Master Branch Only - Install Node 12.x
+        if: ${{ github.ref == 'refs/heads/master' }}
+        uses: actions/setup-node@v1
+        with:
+          node-version: '12.x'
+
+      - name: Master Branch Only - Install npm modules for hugo build
+        if: ${{github.ref == 'refs/heads/master' }}
+        run: |
+          npm install
+          npm install -g postcss-cli
+          npm i -D autoprefixer
+
+      - name: Master Branch Only - Build for prod with minify
+        if: github.ref == 'refs/heads/master'
         run: env HUGO_ENV="production" hugo --minify
 
-      - name: Deploy
+      - name: Master Branch Only - Deploy to GH pages
+        if: ${{ github.ref == 'refs/heads/master' }}
         uses: peaceiris/actions-gh-pages@v3
         with:
           personal_token: ${{ secrets.DEPOLY_TO_SITE}}
@@ -44,16 +76,14 @@ jobs:
           user_email: medic-ci@github
           publish_branch: master
 
-      - name: Failed Deploy
+      - name: Master Branch Only - Report errors to Slack, if any
+        if: ${{ github.ref == 'refs/heads/master' && failure() }}
         uses: rtCamp/[email protected]
-        if: ${{ failure() }}
         env:
           SLACK_WEBHOOK: '${{ secrets.SLACK_WEB_HOOK }}'
           SLACK_CHANNEL: '#cht-squad-docs'
           SLACK_COLOR: '#3278BD'
           SLACK_ICON: ":flappingphone:"
           SLACK_MESSAGE: 'Deploy Failed'
-          SLACK_TITLE: CHT DOCS  DEPLOY
-          SLACK_USERNAME: Medic-Bot
-
-
+          SLACK_TITLE: CHT DOCS DEPLOY
+          SLACK_USERNAME: Medic-Bot

README.md

@@ -1,10 +1,12 @@
-# Documentation for the Community Health Tookit
+# Documentation for the Community Health Toolkit
+
+[![CHT Documentation Site Build](https://github.com/medic/cht-docs/workflows/CHT%20Documentation%20Site%20Build/badge.svg)](https://github.com/medic/cht-docs/actions)
 
 ## About
 
 This repo contains documentation for the Community Health Toolkit (CHT), and how to build digital health applications with [CHT Core](https://github.com/medic/cht-core).
 
-The documentation is built using Markdown pages, which can be converted into a navigatable website using a static-site-generator. The Hugo static-site-generator is being used with the Docsy theme. To maintain portability content should be written in plain Markdown with limited use of HTML, custom shortcodes, and modifications to the theme. Contributions should align with the [documentation style guide](https://docs.communityhealthtoolkit.org/contribute/docs/style-guide/).
+The documentation is built using Markdown pages, which can be converted into a navigable website using a static-site-generator. The Hugo static-site-generator is being used with the [Docsy theme](https://themes.gohugo.io/docsy/). To maintain portability, content should be written in plain Markdown with limited use of HTML, custom shortcodes, and modifications to the theme. Contributions should align with the [documentation style guide](https://docs.communityhealthtoolkit.org/contribute/docs/style-guide/).
 
 ## Installing Hugo
 
@@ -16,15 +18,25 @@ To install, follow the [installation instructions for your Operating System](htt
 
 1. Get local copies of the project submodules so you can build and run your site locally:
    - `git submodule update --init --recursive`
-
-2. Build your site:
+1. Build your site:
    - `hugo server`
    - If there are any errors, check that you have the latest version of Hugo, and are using the extended version.
+1. Preview your site in your browser at: http://localhost:1313/
+1. As you make changes to the site, your browser will automatically reload your local dev site so you can easily and quickly see your changes. 
 
-3. Preview your site in your browser at: http://localhost:1313/
+## Optional Link Checking
 
-## Continuous Deployment
+We validate that all links on the docs site work (do not 404) using a tool called [Muffet](https://github.com/raviqqe/muffet) along with  [Actions](https://github.com/features/actions).  If you're creating a lot of new links, or editing a lot of existing links, you may optionally run Muffet locally before pushing your commits.  This will save a lot of time of pushing a build, waiting for the Action to run, finding you have a broken link, fixing it, pushing a new change etc.
 
-All changes to `master` branch run a [GitHub action](.github/workflows/ci.yml) to deploy the documentation site: [docs.communityhealthtoolkit.org](https://docs.communityhealthtoolkit.org)
+  - Install [Go](https://golang.org/doc/install) as a prerequisite 
+  - Install Muffet: `go get -u github.com/raviqqe/muffet`
+  - Ensure you've run `hugo server` so your local docs instance is reachable at http://localhost:1313/
+  - Test the links with the [`muffet.sh`](https://github.com/medic/cht-docs/blob/master/.github/scripts/muffet.sh) script.  If you're in the root of this repo, that'd be: `./.github/scripts/muffet.sh` 
+
+It should take about 60 seconds depending on your Internet connection. If Muffet returns no output, you have no broken links, congrats! 
+
+_Note_: The `muffet.sh` script here is the identical script we run on GitHub. If you simply run `muffet http://localhost:1313` you will hit GitHub's rate limiting and get lots of [429's](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429). Our script intentionally reduces concurrency and excludes some repetitive GitHub URLs.
+
+## Continuous Deployment
 
-![CHT Documentation Site Build](https://github.com/medic/cht-docs/workflows/CHT%20Documentation%20Site%20Build/badge.svg)
+All changes to `master` branch run a [GitHub action](.github/workflows/ci.yml) to first check for any broken links (per above) and then deploy to the documentation site: [docs.communityhealthtoolkit.org](https://docs.communityhealthtoolkit.org)