docs: splitting readme into pebble docs with the diataxis method (#419)

Splitting the README documentation into smaller chunks into the docs folder, using the Diataxis method.

docs/custom_conf.py

@@ -19,7 +19,7 @@
 ############################################################
 
 # Product name
-project = 'Documentation starter pack'
+project = 'Pebble'
 author = 'Canonical Group Ltd'
 
 # The title you want to display for the documentation in the sidebar.
@@ -67,7 +67,7 @@
     # For example: "ubuntu.com/lxd" or "microcloud.is"
     # If there is no product website, edit the header template to remove the
     # link (see the readme for instructions).
-    'product_page': 'documentation.ubuntu.com',
+    'product_page': 'github.com/canonical/pebble',
 
     # Add your product tag (the orange part of your logo, will be used in the
     # header) to ".sphinx/_static" and change the path here (start with "_static")
@@ -77,34 +77,34 @@
     # Change to the discourse instance you want to be able to link to
     # using the :discourse: metadata at the top of a file
     # (use an empty value if you don't want to link)
-    'discourse': 'https://discourse.ubuntu.com',
+    'discourse': 'https://discourse.charmhub.io',
 
     # Change to the Mattermost channel you want to link to
     # (use an empty value if you don't want to link)
-    'mattermost': 'https://chat.canonical.com/canonical/channels/documentation',
+    'mattermost': '',
 
     # Change to the Matrix channel you want to link to
     # (use an empty value if you don't want to link)
-    'matrix': 'https://matrix.to/#/#documentation:ubuntu.com',
+    'matrix': 'https://matrix.to/#/#charmhub-charmdev:ubuntu.com',
 
     # Change to the GitHub URL for your project
     # This is used, for example, to link to the source files and allow creating GitHub issues directly from the documentation.
-    'github_url': 'https://github.com/canonical/sphinx-docs-starter-pack',
+    'github_url': 'https://github.com/canonical/pebble',
 
     # Change to the branch for this version of the documentation
-    'github_version': 'main',
+    'github_version': 'master',
 
     # Change to the folder that contains the documentation
     # (usually "/" or "/docs/")
-    'github_folder': '/',
+    'github_folder': '/docs/',
 
     # Change to an empty value if your GitHub repo doesn't have issues enabled.
     # This will disable the feedback button and the issue link in the footer.
     'github_issues': 'enabled',
 
     # Controls the existence of Previous / Next buttons at the bottom of pages
     # Valid options: none, prev, next, both
-    'sequential_nav': "none", 
+    'sequential_nav': "both", 
 
     # Controls if to display the contributors of a file or not
     "display_contributors": True,
@@ -165,7 +165,8 @@
     'canonical.related-links',
     'canonical.custom-rst-roles',
     'canonical.terminal-output',
-    'notfound.extension'
+    'notfound.extension',
+    'canonical.filtered-toc'
     ]
 
 # Add custom required Python modules that must be added to the

docs/explanation/api-and-clients.md

@@ -0,0 +1,11 @@
+# API and clients
+
+The Pebble daemon exposes an API (HTTP over a unix socket) to allow remote clients to interact with the daemon. It can start and stop services, add configuration layers the plan, and so on.
+
+There is currently no official documentation for the API at the HTTP level (apart from the [code itself](https://github.com/canonical/pebble/blob/master/internals/daemon/api.go)!); most users will interact with it via the Pebble command line interface or by using the Go or Python clients.
+
+The Go client is used primarily by the CLI, but is importable and can be used by other tools too. See the [reference documentation and examples](https://pkg.go.dev/github.com/canonical/pebble/client) at pkg.go.dev.
+
+We try to never change the underlying HTTP API in a backwards-incompatible way, however, in rare cases we may change the Go client in a backwards-incompatible way.
+
+In addition to the Go client, there's also a [Python client](https://github.com/canonical/operator/blob/master/ops/pebble.py) for the Pebble API that's part of the [`ops` library](https://github.com/canonical/operator) used by Juju charms ([documentation here](https://juju.is/docs/sdk/interact-with-pebble)).

docs/explanation/general-model.md

@@ -0,0 +1,5 @@
+# General model
+
+Pebble is organised as a single binary that works as a daemon and also as a client to itself. When the daemon runs it loads its own configuration from the `$PEBBLE` directory, as defined in the environment, and also records in that same directory its state and unix sockets for communication. If that variable is not defined, Pebble will attempt to look for its configuration from a default system-level setup at `/var/lib/pebble/default`. Using that directory is encouraged for whole-system setup such as when using Pebble to control services in a container.
+
+The `$PEBBLE` directory must contain a `layers/` subdirectory that holds a stack of configuration files with names similar to `001-base-layer.yaml`, where the digits define the order of the layer and the following label uniquely identifies it. Each layer in the stack sits above the former one, and has the chance to improve or redefine the service configuration as desired.

docs/explanation/index.md

@@ -0,0 +1,10 @@
+# Explanation
+
+```{toctree}
+:hidden:
+:titlesonly:
+:maxdepth: 1
+
+General model <general-model>
+API and clients <api-and-clients>
+```

docs/how-to/changes-and-tasks.md

@@ -0,0 +1,22 @@
+# How to use changes and tasks
+
+When Pebble performs a (potentially invasive or long-running) operation such as starting or stopping a service, it records a "change" object with one or more "tasks" in it. The daemon records this state in a JSON file on disk at `$PEBBLE/.pebble.state`.
+
+To see recent changes, for this or previous server runs, use `pebble changes`. You might see something like this:
+
+```
+$ pebble changes
+ID  Status  Spawn                Ready                Summary
+1   Done    today at 14:33 NZDT  today at 14:33 NZDT  Autostart service "srv1"
+2   Done    today at 15:26 NZDT  today at 15:26 NZDT  Start service "srv2"
+3   Done    today at 15:26 NZDT  today at 15:26 NZDT  Stop service "srv1" and 1 more
+```
+
+To drill down and see the tasks that make up a change, use `pebble tasks <change-id>`:
+
+```
+$ pebble tasks 3
+Status  Spawn                Ready                Summary
+Done    today at 15:26 NZDT  today at 15:26 NZDT  Stop service "srv1"
+Done    today at 15:26 NZDT  today at 15:26 NZDT  Stop service "srv2"
+```

docs/how-to/configure-layers.md

@@ -0,0 +1,74 @@
+# How to configure layers
+
+Below is an example of the current configuration format. For full details of all fields, see the [complete layer specification](../reference/layer-specification).
+
+```yaml
+summary: Simple layer
+
+description: |
+    A better description for a simple layer.
+
+services:
+    srv1:
+        override: replace
+        summary: Service summary
+        command: cmd arg1 "arg2a arg2b"
+        startup: enabled
+        after:
+            - srv2
+        before:
+            - srv3
+        requires:
+            - srv2
+            - srv3
+        environment:
+            VAR1: val1
+            VAR2: val2
+            VAR3: val3
+
+    srv2:
+        override: replace
+        startup: enabled
+        command: cmd
+        before:
+            - srv3
+
+    srv3:
+        override: replace
+        command: cmd
+```
+
+The `override` field (which is required) defines whether this entry _overrides_ the previous service of the same name (if any), or merges with it. See the [full layer specification](../reference/layer-specification) for more details.
+
+## Layer override example
+
+Any of the fields can be replaced individually in a merged service configuration. To illustrate, here is a sample override layer that might sit on top of the one above:
+
+```yaml
+summary: Simple override layer
+
+services:
+    srv1:
+        override: merge
+        environment:
+            VAR3: val3
+        after:
+            - srv4
+        before:
+            - srv5
+
+    srv2:
+        override: replace
+        summary: Replaced service
+        startup: disabled
+        command: cmd
+
+    srv4:
+        override: replace
+        command: cmd
+        startup: enabled
+
+    srv5:
+        override: replace
+        command: cmd
+```

docs/how-to/health-checks.md

@@ -0,0 +1,79 @@
+# How to use health checks
+
+Separate from the service manager, Pebble implements custom "health checks" that can be configured to restart services when they fail.
+
+Each check can be one of three types. The types and their success criteria are:
+
+* `http`: an HTTP `GET` request to the URL specified must return an HTTP 2xx status code
+* `tcp`: opening the given TCP port must be successful
+* `exec`: executing the specified command must yield a zero exit code
+
+Checks are configured in the layer configuration using the top-level field `checks`. Full details are given in the [layer specification](../reference/layer-specification), but below is an example layer showing the three different types of checks:
+
+```
+checks:
+    up:
+        override: replace
+        level: alive
+        period: 30s
+        threshold: 1  # an aggressive threshold
+        exec:
+            command: service nginx status
+
+    online:
+        override: replace
+        level: ready
+        tcp:
+            port: 8080
+
+    test:
+        override: replace
+        http:
+            url: http://localhost:8080/test
+```
+
+Each check is performed with the specified `period` (the default is 10 seconds apart), and is considered an error if a timeout happens before the check responds -- for example, before the HTTP request is complete or before the command finishes executing.
+
+A check is considered healthy until it's had `threshold` errors in a row (the default is 3). At that point, the check is considered "down", and any associated `on-check-failure` actions will be triggered. When the check succeeds again, the failure count is reset to 0.
+
+To enable Pebble auto-restart behavior based on a check, use the `on-check-failure` map in the service configuration (this is what ties together services and checks). For example, to restart the "server" service when the "test" check fails, use the following:
+
+```
+services:
+    server:
+        override: merge
+        on-check-failure:
+            # can also be "shutdown", "success-shutdown", or "ignore" (the default)
+            test: restart
+```
+
+You can view check status using the `pebble checks` command. This reports the checks along with their status (`up` or `down`) and number of failures. For example:
+
+```
+$ pebble checks
+Check   Level  Status  Failures  Change
+up      alive  up      0/1       10
+online  ready  down    1/3       13 (dial tcp 127.0.0.1:8000: connect: connection refused)
+test    -      down    42/3      14 (Get "http://localhost:8080/": dial t... run "pebble tasks 14" for more)
+```
+
+The "Failures" column shows the current number of failures since the check started failing, a slash, and the configured threshold.
+
+The "Change" column shows the change ID of the [change](#changes-and-tasks) driving the check, along with a (possibly-truncated) error message from the last error. Running `pebble tasks <change-id>` will show the change's task, including the last 10 error messages in the task log.
+
+Health checks are implemented using two change kinds:
+
+* `perform-check`: drives the check while it's "up". The change finishes when the number of failures hits the threshold, at which point the change switches to Error status and a `recover-check` change is spawned. Each check failure records a task log.
+* `recover-check`: drives the check while it's "down". The change finishes when the check starts succeeding again, at which point the change switches to Done status and a new `perform-check` change is spawned. Again, each check failure records a task log.
+
+## Health endpoint
+
+If the `--http` option was given when starting `pebble run`, Pebble exposes a `/v1/health` HTTP endpoint that allows a user to query the health of configured checks, optionally filtered by check level with the query string `?level=<level>` This endpoint returns an HTTP 200 status if the checks are healthy, HTTP 502 otherwise.
+
+Each check can specify a `level` of "alive" or "ready". These have semantic meaning: "alive" means the check or the service it's connected to is up and running; "ready" means it's properly accepting network traffic. These correspond to [Kubernetes "liveness" and "readiness" probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).
+
+The tool running the Pebble server can make use of this, for example, under Kubernetes you could initialize its liveness and readiness probes to hit Pebble's `/v1/health` endpoint with `?level=alive` and `?level=ready` filters, respectively.
+
+Ready implies alive, and not-alive implies not-ready. If you've configured an "alive" check but no "ready" check, and the "alive" check is unhealthy, `/v1/health?level=ready` will report unhealthy as well, and the Kubernetes readiness probe will act on that.
+
+If there are no checks configured, the `/v1/health` endpoint returns HTTP 200 so the liveness and readiness probes are successful by default. To use this feature, you must explicitly create checks with `level: alive` or `level: ready` in the layer configuration.

docs/how-to/index.md

@@ -0,0 +1,21 @@
+# How-to guides
+
+```{toctree}
+:hidden:
+:titlesonly:
+:maxdepth: 1
+
+Install Pebble <install-pebble>
+Run the daemon <run-the-daemon>
+View, start, stop services <view-start-stop-services>
+Update and restart services <update-restart-services>
+Manage service dependencies <service-dependencies>
+Configure service auto-restart <service-auto-restart>
+Use health checks <health-checks>
+Use changes and tasks <changes-and-tasks>
+Get logs <logs>
+Use log forwarding <log-forwarding>
+Use notices <notices>
+Configure layers <configure-layers>
+Use Pebble in containers <pebble-in-containers>
+```

docs/how-to/install-pebble.md

@@ -0,0 +1,15 @@
+# How to install Pebble
+
+To install the latest version of Pebble, run the following command (we don't currently ship binaries, so you must first [install Go](https://go.dev/doc/install)):
+
+```
+go install github.com/canonical/pebble/cmd/pebble@latest
+```
+
+Pebble is invoked using `pebble <command>`. To get more information:
+
+* To see a help summary, type `pebble -h`.
+* To see a short description of all commands, type `pebble help --all`.
+* To see details for one command, type `pebble help <command>` or `pebble <command> -h`.
+
+A few of the commands that need more explanation are detailed below.

docs/how-to/log-forwarding.md

@@ -0,0 +1,81 @@
+# How to use log forwarding
+
+Pebble supports forwarding its services' logs to a remote Loki server. In the `log-targets` section of the plan, you can specify destinations for log forwarding, for example:
+```yaml
+log-targets:
+    staging-logs:
+        override: merge
+        type: loki
+        location: http://10.1.77.205:3100/loki/api/v1/push
+        services: [all]
+    production-logs:
+        override: merge
+        type: loki
+        location: http://my.loki.server.com/loki/api/v1/push
+        services: [svc1, svc2]
+```
+
+## Specifying services
+
+For each log target, use the `services` key to specify a list of services to collect logs from. In the above example, the `production-logs` target will collect logs from `svc1` and `svc2`.
+
+Use the special keyword `all` to match all services, including services that might be added in future layers. In the above example, `staging-logs` will collect logs from all services.
+
+To remove a service from a log target when merging, prefix the service name with a minus `-`. For example, if we have a base layer with
+```yaml
+my-target:
+    services: [svc1, svc2]
+```
+and override layer with
+```yaml
+my-target:
+    services: [-svc1]
+    override: merge
+```
+then in the merged layer, the `services` list will be merged to `[svc1, svc2, -svc1]`, which evaluates left to right as simply `[svc2]`. So `my-target` will collect logs from only `svc2`.
+
+You can also use `-all` to remove all services from the list. For example, adding an override layer with
+```yaml
+my-target:
+    services: [-all]
+    override: merge
+```
+would remove all services from `my-target`, effectively disabling `my-target`. Meanwhile, adding an override layer with
+```yaml
+my-target:
+    services: [-all, svc1]
+    override: merge
+```
+would remove all services and then add `svc1`, so `my-target` would receive logs from only `svc1`.
+
+## Labels
+
+In the `labels` section, you can specify custom labels to be added to any outgoing logs. These labels may contain `$ENVIRONMENT_VARIABLES` - these will be interpreted in the environment of the corresponding service. Pebble may also add its own default labels (depending on the protocol). For example, given the following plan:
+```yaml
+services:
+  svc1:
+    environment:
+      OWNER: 'alice'
+  svc2:
+    environment:
+      OWNER: 'bob'
+
+log-targets:
+  tgt1:
+    type: loki
+    labels:
+      product: 'juju'
+      owner: 'user-$OWNER'
+```
+the logs from `svc1` will be sent with the following labels:
+```yaml
+product: juju
+owner: user-alice     # env var $OWNER substituted
+pebble_service: svc1  # default label for Loki
+```
+and for svc2, the labels will be
+```yaml
+product: juju
+owner: user-bob       # env var $OWNER substituted
+pebble_service: svc2  # default label for Loki
+```