V7 (clay#581)

* stashing small update

* stashing changes;

* stashing changes before beginning testing refactor

* testing for non-api tests

* fixing api tests

* updating db tests

* 100%

* eslint

* moving packages over to dev dependencies

* more updates

* lots of changes

* metadata for the layouts

* forgot file

* more layouts/meta work

* getting things in order

* New DB API: Storage (clay#554)

* stashing small update

* stashing changes;

* stashing changes before beginning testing refactor

* testing for non-api tests

* fixing api tests

* updating db tests

* 100%

* eslint

* moving packages over to dev dependencies

* Meta Patch Requests (clay#569)

* adding patch support

* code comments

* Publish data meta (clay#571)

* adding patch support

* code comments

* bringing in meta updating

* cleanup

* cleaning up some publish/unpublish logic and keeping page history

* passing bus publish method to plugins

* using meta object and raw query for db

* small pruning and comments

* using amphora-fs for file service

* updating for amphora-search

* lists api

* removing cache control

* moving plugins

* some testing work

* components & composer services

* stashing tests

* more tests

* middle of tests

* stashing test changes

* initial fixes done, moving to api tests

* most testing work done

* mostly done

* fixing model call

* making a doc of changing things

* linting and test changes

* writing plugin docs

* writing more docs

* adding to table of contents

* pass site to attachRoutes

* do not  need that package

* fixing issue

* newline

* no actual changes

* Validate site routes (clay#598)

* no actual changes

* attach amphora api routes to the router first to avoid misleading errors caused by less specific site-level routes

* set reservedRoutes in a function + pr feedback

* making long stacktraces an env var

* stashing changes

* fixing up some tests

* actually fixing tests

* V7 meta changes (clay#605)

* no actual changes

* some fixes discovered when working with kiln

* unwrap list routes in amphora-storage-postgres

* normalizing fields between page and layout metadata and removing unused fields

* use iso string timestamps

* 7.0.0-0

* Redirect From Published Meta (clay#607)

* adding in redirect and fixing eslint comment

* moving to responses

* name change

* look for custom url and dynamic prop before site pub chain

* removing comment

* package-lock

.eslintrc

@@ -22,7 +22,7 @@
     // best practices
     "complexity": [2, 8],
     "default-case": 2,
-    "guard-for-in": 2,
+    "guard-for-in": 0,
     "no-alert": 1,
     "no-floating-decimal": 1,
     "no-self-compare": 2,

docs/SUMMARY.md

@@ -11,11 +11,11 @@
     * [Building Custom Renderers](basics/renderers/custom-renderers.md)
   * [Event Bus](basics/event-bus.md)
 * [Advanced](advanced/README.md)
+  * [Storage API](advanced/storage.md)
   * [Data Versioning \(Upgrades\)](advanced/upgrade.md)
   * [Renderer Models](advanced/renderer-models.md)
 * [Plugins](plugins/README.md)
   * [Hooks](plugins/hooks.md)
   * [Writing A Plugin](plugins/writing-a-plugin.md)
   * [Plugin vs. Renderer](plugins/plugin-vs-renderer.md)
 * [Glossary](glossary.md)
-

docs/advanced/storage.md

@@ -0,0 +1,152 @@
+# Storage
+
+The Storage API for Amphora is meant to provide a bridge between Amphora's composition layer and how data for sites in Clay is persisted. At its core, Amphora maintains six top-level data structures (Components, Layouts, Pages, Users, Uris, Lists) and it's the responsibility of the storage module to store and compose the data.
+
+Currently it is possible to write any storage midule that best fits the needs of your Clay instance, but the platform officially supports a Postgres storage module that [can be found here](https://github.com/clay/amphora-storage-postgres).
+
+For more information in writing your own storage module you can refer to this module as an example.
+
+---
+
+## Setup
+
+The function in which the storage module should connect to a database as well as perform any other instantiation operations, such as setting up required tables/collections.
+
+**Returns:** `Promise`
+
+---
+
+## Put
+
+This function is a simple write to the DB for any of the Clay data structures. It should be able to handle writing one individual component/layout/page/list/uri/user. The return value should
+
+**Arguments:** `key` (String), `value` (Object)
+
+**Returns:** `Promise<Object|String>`
+
+---
+
+## Get
+
+This function should return one individual instance of a any of the data types.
+
+**Arguments:** `key` (String)
+
+**Returns:** `Promise<Object|String>`
+
+---
+
+## Del
+
+This function should delete one individual instance of any of the data types. The value of the item that was just deleted should be returned.
+
+**Arguments:** `key` (String)
+
+**Returns:** `Promise<Object|String>`
+
+---
+
+## Batch
+
+A function which accepts an array of objects (operations), with each object representing one "save" action for any of the supported data structures. An operation is an object with the following structure:
+
+- `key` (String): the id of the individual component/layout/page/uri/list/user instance
+- `value` (Object): the value to be saved for the key
+
+```javascript
+// Example operations array
+[{
+  key: 'domain.com/_components/foo/instances/bar',
+  value: {
+    foobar: 'baz'
+  }
+}, {
+  key: 'domain.com/_pages/foobarbaz',
+  value: {
+    layout: 'domain.com/_layouts/layout/instances/default',
+    main: [
+      'domain.com/_components/foo/instances/bar'
+    ]
+  }
+}]
+```
+
+**Arguments:** `ops` (Array)
+
+**Returns:** `Promise`
+
+---
+
+## Get Meta
+
+Retrieves the metadata for a page/layout. The function will always be called with the raw page/layout uri, not a uri ending in `/meta`.
+
+For example, Amphora will respond to a request for `domain.com/_pages/foo/meta` by making the following invoking the `getMeta` function of the storage module with a key of `domain.com/_pages/foo` and then returning the data to client. If no data is returned, Amphora will return an empty object.
+
+**Arguments:** `key` (String)
+
+**Returns:** `Promise<Object>`
+
+---
+
+## Put Meta
+
+Saves the metadata for a page/layout. The function will always be called with the raw page/layout uri, not a uri ending in `/meta`.
+
+**Arguments:** `key` (String), `value` (Object)
+
+**Returns:** `Promise<Object>`
+
+---
+
+## Patch Meta
+
+Updates properties on the metadata object with the values passed into the request. This method should _never_ update the entire metadata object with what's passed to the function or metadata will be destroyed when users edit data.
+
+If the storage solution does not support partial updates to the data then the function will need to request the full object from the database, merge the old data with the new data and then save the merged object.
+
+**Arguments:** `key` (String), `value` (Object)
+
+**Returns:** `Promise<Object>`
+
+---
+
+## Create Read Stream
+
+Amphora responds to certain requests with a Stream of data, such as a request to `domain.com/_components/foo/instances`. In this case, Amphora will read all the instances of the `foo` component from the database and send back an array of component uris. To handle this in the most memory efficient way, Amphora processes the data from the DB as a Stream, and the `createReadStream` function should return a Read Stream for Amphora to act on.
+
+The function will receive an object of options with three properties:
+
+- `prefix` (String): the string which should prefix all of the keys that Amphora needs to display data for
+- `keys` (Boolean): if `true`, the stream should return the uri of all data matcing the prefix
+- `values` (Boolean): if `true`, the stream should return the values of all uris matcing the prefix
+
+Using the example from earlier, if a request comes in for `domain.com/_components/foo/instances`, Amphora will pass an `options` object to the `createReadStream` function like so:
+
+```javascript
+// Example `options` object for the createReadStream function
+{
+  prefix: 'domain.com/_components/foo/instances',
+  keys: true,
+  values: false
+}
+```
+
+Amphora would expect a Read Stream (in object mode) where each item in the stream is an object with the following signature:
+
+- `key` (String)[Optional]: the uri of the component/layout/page/uri/user/list
+- `value` (Object)[Optional]: the object/string value of the uri
+
+Amphora will then take this data and make sure it's a properly formatted response to the client that initiated the request.
+
+---
+
+## Raw
+
+The `raw` function is the most ambiguous function of the API, but is the crux of plugins. To ensure that plugins can add in their own data to the database that your Clay instance is using, the db client is passed to each plugin included in your instance.
+
+This puts a lot of responsibility on plugins to manage their own data properly, but allows developers to write their own plugins to accomplish anything with Clay.
+
+This function should simply be a passthrough to the client that connects to the data store being used and should return a Promise for any executed action.
+
+**Returns:** `Promise`

docs/basics/plugins.md

@@ -0,0 +1,42 @@
+# Plugins
+
+Plugins allow for you to extend the functionality of your site by allowing you to attach routes to each site's router in your instance. While this may not seem any different than [defining routes for your site](/docs/basics/routes.md), the basic site router will assign routes that only respond to `GET` requests and will run through Amphora's composition/rendering functionality. Plugins allow you to assign routes that respond to any [Express supported request method](https://expressjs.com/en/4x/api.html#app.METHOD) with your own custom handlers.
+
+## Anatomy of a Plugin
+
+A plugin should be a function and will receive the following arguments:
+
+- `router`: the router for the site. Attach listeners and handlers as you would to any Express router.
+- `db`: an instance of Amphora's internal database connector.
+- `bus`: a passthrough of Amphora's internal event bus publish method so that a plugin can publish to the event bus on its own.
+- `sites`: the internal `sites` service, used for discovering which site in your instance a uri belongs to.
+
+An example plugin might look like the following:
+
+```javascript
+module.exports = (router, db, bus, sites) => {
+  // Attach a route that responds to PUT requests
+  router.put('/_coolroute', (req, res) => {
+    return db.put(req.uri, req.body)
+      .then(() => {
+        // Pub to the event bus that the route was called
+        bus('coolRouteCalled', { body: req.body });
+        // Respond to the request
+        res.json({ status: 200 })
+      });
+  });
+
+  // Note: we're not explicity returning anything, but if we were
+  // performing some async action we could return a Promise
+}
+```
+
+You aren't required to explicitly `return` any value from your plugin, but you can return a `Promise` if needed.
+
+## Lifecycle of Plugin
+
+A plugin is called...
+
+- once for every site in your Clay instance
+- _after_ the storage module is instantiated
+- _before_ bootstrapping happens

docs/basics/routes.md

@@ -48,4 +48,3 @@ module.exports.routes = [
 ```
 
 By adding this path object into your `routes` object you'll be able to create one page to handle all the requests to the `/archive/*` route. **Make sure your** [**dynamic page is published**](publishing.md#dynamic-pages--publishing) **or else this won't work!**
-