Review comments

docs/Server/API/README.md

@@ -9,8 +9,12 @@ or just "datasets".
 
 The [V1 API](V1/README.md) provides a REST-like functional interface.
 
-The Pbench Server APIs use a combination of parameters embedded in the URI path
-along with query parameters and serialized JSON parameters (mimetype
-`application/json`) for requests. A few exceptions use raw byte streams
-(`application/octet-stream`) to allow uploading new datasets and to access
-individual files from a dataset.
+The Pbench Server APIs accept parameters from a variety of sources. See the
+individual API documentation for details.
+1. Some parameters, especially "resource ids", are embedded in the URI, such as
+`/api/v1/datasets/<resource_id>`;
+2. Some parameters are passed as query parameters, such as
+`/api/v1/datasets?name:fio`;
+3. For `PUT` and `POST` APIs, parameters may also be passed as a JSON
+(`application/json` content type) request payload, such as
+`{"metadata": {"dataset.name": "new name"}}`

docs/Server/API/V1/README.md

@@ -132,20 +132,3 @@ through the `directories` list of each
 The [inventory](inventory.md) API returns the raw byte stream of any regular
 file within the directory hierarchy, including log files, postprocessed JSON
 files, and benchmark result text files.
-
-### Example
-
-```
-    def directory(request, url: str, name: str = "/", level: int = 0):
-        ls = request.get(url).get_json()
-        print(f"{'  '*level}{name}")
-        for d in ls.directories:
-            directory(request, level + 1, d.name, d.url)
-        for f in ls.files:
-            print(f"{'  '*(level+1)}{f.name})
-            bytes = request.get(f.url)
-            # display byte stream:
-            # inline on terminal doesn't really make sense
-
-    directory(request, "http://host.example.com/api/v1/datasets/<dataset>/contents/")
-```

docs/Server/API/V1/relay.md

@@ -1,37 +1,39 @@
 # `POST /api/v1/relay/<uri>`
 
-This API creates a dataset resource by reading data from a Relay server in two
-steps. The provided URI represents a "manifest file" stored on the Relay server,
-a JSON file (`application/json` MIME format) defining the original tarball name,
-the MD5 hash value, optional metadata key/value pairs for the dataset, and a
-URI from which the Pbench Server expects to `GET` an `application/octet-stream`
-of a file created by `tar` and compressed with the `xz` program.
-
-Primarily this is expected to be a native Pbench Agent tarball with a specific
-structure; however with the `server.archiveonly` metadata key the Pbench Server
-can be used to archive and manage metadata for any tarball passed through a
-Relay server.
+This API creates a dataset resource by reading data from a Relay server. There
+are two distinct steps involved:
+
+1. A `GET` on the provided URI returns a "manifest file" stored on the Relay
+server. This is a JSON file (`application/json` MIME format) defining the
+original tarball filename, the tarball's MD5 hash value, the URI of the
+performance data tarball file, and optionally metadata key/value pairs to be
+applied to the new dataset.
+2. A `GET` on the Relay manifest file's `uri` field value returns an
+`application/octet-stream` payload for a file created by `tar` and compressed
+with the `xz` program, which will be stored by the Pbench Server as a
+dataset.
 
 ## URI parameters
 
 `<uri>` string \
 The Relay server URI of the tarball's manifest `application/json` file. This
-must provide the following JSON keys:
+JSON object must provide a set of parameter keys as defined below in
+[Manifest file keys](#manifest-file-keys).
+
+## Manifest file keys
 
 For example,
 
 ```json
 {
     "uri": "https://relay.example.com/52adfdd3dbf2a87ed6c1c41a1ce278290064b0455f585149b3dadbe5a0b62f44",
-    "md5": "22a4bc5748b920c6ce271eb68f08d91c".
-    "name": "fio_rw_2018.02.01T22.40.57.tar.xz".
+    "md5": "22a4bc5748b920c6ce271eb68f08d91c",
+    "name": "fio_rw_2018.02.01T22.40.57.tar.xz",
     "access": "private",
     "metadata": ["server.origin:myrelay", "global.agent:cloud1"]
 }
 ```
 
-## Manifest file keys
-
 `access`: [ `private` | `public` ] \
 The desired initial access scope of the dataset. Select `public` to make the
 dataset accessible to all clients, or `private` to make the dataset accessible
@@ -56,8 +58,6 @@ In particular the client can set any of:
 * `server.archiveonly`: [suppress indexing](../metadata.md#serverarchiveonly)
 * `server.deletion`: [default dataset expiration time](../metadata.md#serverdeletion).
 
-For example, `?metadata=server.archiveonly:true,global.project:oidc`
-
 `name`: The original tarball file name \
 The string value must represent a legal filename with the compound type of
 `.tar.xz` representing a `tar` archive compressed with the `xz` program.
@@ -86,7 +86,8 @@ The return is a serialized JSON object with status information.
 `200`   **OK** \
 Successful request. The dataset MD5 hash is identical to that of a dataset
 previously uploaded to the Pbench Server. This is assumed to be an identical
-tarball.
+tarball, and the secondary URI (the `uri` field in the Relay manifest file)
+has not been consumed.
 
 `201`   **CREATED** \
 The tarball was successfully uploaded and the dataset has been created.
@@ -112,8 +113,10 @@ a message, and optional JSON data provided by the system administrator.
 
 ## Response body
 
-The `application/json` response body consists of a JSON object giving a detailed
-message on success or failure:
+The `application/json` response body consists of a JSON object containing a
+`message` field. On failure this will describe the nature of the problem and
+in some cases an `errors` array will provide details for cases where multiple
+problems can occur.
 
 ```json
 {

docs/Server/API/V1/upload.md

@@ -82,8 +82,10 @@ a message, and optional JSON data provided by the system administrator.
 
 ## Response body
 
-The `application/json` response body consists of a JSON object giving a detailed
-message on success or failure:
+The `application/json` response body consists of a JSON object containing a
+`message` field. On failure this will describe the nature of the problem and
+in some cases an `errors` array will provide details for cases where multiple
+problems can occur.
 
 ```json
 {