Docs update (#199)

.github/styles/config/vocabularies/fabric-vocab/accept.txt

@@ -8,3 +8,5 @@ LLM
 Splunk
 Cyber
 CTI
+ASCII
+stdout

cmd/render.go

@@ -36,7 +36,7 @@ func init() {
 var renderCmd = &cobra.Command{
 	Use:   "render TARGET",
 	Short: "Render the document",
-	Long:  `Render the specified document into Markdown and output it either to stdout or to a file`,
+	Long:  `Render the specified document and either publish it or output it to stdout.`,
 	Args:  cobra.ExactArgs(1),
 	RunE: func(cmd *cobra.Command, args []string) (err error) {
 		target := strings.TrimSpace(args[0])

docs/_index.md

@@ -22,6 +22,7 @@ Together, Fabric CLI tool and FCL offer a comprehensive solution for organizatio
 - [Fabric CLI]({{< ref "cli.md" >}}) — Fabric CLI usage details.
 - [Tutorial]({{< ref "tutorial.md" >}}) — an extensive tutorial for starting with Fabric.
 - [Language]({{< ref "language.md" >}}) — Fabric Configuration Language details and main concepts.
-- [Data Sources]({{< ref "data-sources.md" >}}) — the data sources supported.
-- [Content Providers]({{< ref "content-providers.md" >}}) — the content providers supported.
 - [Plugins]({{< ref "plugins.md" >}}) — the documentation for the plugins Fabric supports.
+- [Data Sources]({{< ref "plugins/data-sources.md" >}}) — the data sources supported.
+- [Content Providers]({{< ref "plugins/content-providers.md" >}}) — the content providers supported.
+- [Publishers]({{< ref "plugins/publishers.md" >}}) — the outgoing integrations for rendered content.

docs/cli.md

@@ -18,7 +18,7 @@ The core Fabric commands are:
 To get more details, run `fabric --help`:
 
 ```text
-$ ./fabric --help
+$ fabric --help
 Usage:
   fabric [command]
 

docs/language/configs.md

@@ -9,20 +9,35 @@ weight: 20
 
 ## Global configuration
 
-The `fabric` configuration block serves as the global configuration for Fabric, offering a centralized space for defining essential aspects such as plugin dependencies and local directory paths.
+The `fabric` configuration block serves as the global configuration for Fabric. Use it to define
+configuration properties, including default behavior, plugin dependencies, and local paths.
 
 ```hcl
 fabric {
-  ...
+  plugin_versions = {
+    "blackstork/elastic" = ">0.4.1"
+    "blackstork/openai" = ">0.4.1"
+  }
 }
 ```
 
 Within the codebase, only one `fabric` block can be defined.
 
 ### Supported arguments
 
-- `plugin_versions`: (required) a map that aligns namespaced plugin names with version constraints in SemVer (refer to Terraform [version constraint syntax](https://developer.hashicorp.com/terraform/language/expressions/version-constraints#version-constraint-syntax)).
-- `cache_dir`: (optional) a path to a directory on the local file system. The default value is `.fabric` directory in the current folder. If the directory doesn't exist, Fabric creates it upon the first run.
+- `plugin_versions`: (optional) a map of plugin dependencies. The version constraints are defined in
+  SemVer (refer to Terraform [version constraint
+  syntax](https://developer.hashicorp.com/terraform/language/expressions/version-constraints#version-constraint-syntax)).
+- `expose_env_vars_with_pattern`: (optional) a glob pattern for filtering the environment variables
+  exposed in the templates by name. Default value is `FABRIC_*`. To expose all environment
+  variables, set the value to `*`. See [Environment variables]({{< ref
+  "configs.md#environment-variables">}}) for the details.
+- `cache_dir`: (optional) a path to a directory on the local file system. The default value is
+  `.fabric` - a directory in the current folder. If the directory doesn't exist, Fabric will create
+  it during the first run.
+
+To install all dependencies defined in `plugin_versions`, run `fabric install` command (see
+[Installing plugins]({{< ref "install.md#installing-plugins" >}}) for more details)
 
 ### Supported nested blocks
 
@@ -56,99 +71,151 @@ fabric {
 }
 ```
 
-## Data source configuration
+## Block configuration
 
-A Fabric plugin can include one or more data sources. For example, `blackstork/github` plugin includes `github_issues` data source.
+The data sources, content provides and publishers can be configured using `config` blocks.
 
-A data source within Fabric serves the purpose of loading data from either a local or an external data store, platform, or service.
+`config` block arguments are specific to a data source, content provider or a publisher the block if
+configuring.
 
-Configuration for data sources is accomplished through the use of the `config` block:
+`config` block must be defined on a root level of Fabric file, outside `document` block. The
+signature of `config` block consists of a block type selector (`content`, `data` or `publish`), a
+data source / content provider / publisher name, and a block name:
 
 ```hcl
-config data <data-source-name> "<name>" {
+config <block-type> <source/provider/publisher-name> "<name>" {
   ...
 }
 ```
 
-If the block is named (`<name>` is provided), the `config` block can be referenced in a `config` argument inside `data` blocks. This is helpful if there is a need to have more than one configuration for the same data source.
+If `<name>` isn't provided, the configuration acts as a default configuration for a specified data
+source / content provider / publisher.
 
-If `<name>` isn't provided, the configuration acts as a default configuration for a specified data source.
+If the block has a name (`<name>` is specified), `config` block can be referenced in a `config` argument.
+This is helpful if there is a need to have more than one configuration available.
 
 ### Supported arguments
 
-The arguments allowed in the configuration block depend on the data source. See [Plugins]({{< ref "plugins.md" >}}) for the details on the configuration parameters supported.
+The arguments allowed in the configuration block depend on the data source / content provider / publisher. See the documentation for [Data Sources]({{< ref data-sources.md >}}), [Content Providers]({{< ref content-providers.md >}}), and [Publishers]({{< ref publishers.md >}}) for the details on the configuration parameters supported.
 
 ### Supported nested blocks
 
-Nested blocks aren't supported inside the `config` blocks.
+- `meta`: (optional) a block containing metadata for the block. See [Metadata]({{< ref "configs.md#metadata" >}}) for details.
 
 ### Example
 
 ```hcl
-
 config data csv {
   delimiter = ";"
 }
 
-data csv "events_a" {
-  path = "/tmp/events-a.csv"
+config content openai_text {
+  api_key = env.FABRIC_OPENAI_API_KEY
+  system_prompt = "You are the best at saying Hi!"
 }
 
 document "test-document" {
 
-  data ref {
-    base = data.csv.events_a
+  data csv "events_a" {
+    path = "/tmp/events-a.csv"
   }
 
   data csv "events_b" {
+    # Overriding the default configuration for CSV data source
     config {
-      delimiter = ",";
+      delimiter = ","
     }
 
     path = "/tmp/events-b.csv"
   }
+
+  content openai_text {
+    prompt = "Say hi!"
+  }
 }
 ```
 
-## Content provider configuration
+## Environment variables
 
-A Fabric plugin can include one or more content providers. For example, `blackstork/openai` plugin includes `openai_text` content provider.
-Content providers generate Markdown content that Fabric will include into the rendered document. The provider can either render content locally or use an external API (like OpenAI API).
+Fabric templates can be configured with environment variables, either set in the shell or provided
+in `.env` file.
 
-Content providers can be configured using `config` block:
+The values of environment variables are available through global `env` namespace and inside the context under `.env` root.
+
+For example:
 
 ```hcl
-config content <content-provider-name> "<name>" {
-  ...
+config data elasticsearch {
+  basic_auth_username = "elastic"
+  basic_auth_password = env.ELASTICSEARCH_PASSWORD
+}
+
+content text {
+  local_var = query_jq(".env.FOOBAR | split(\",\") | length")
+  value = "There are {{ .vars.local }} elements in `FOOBAR` env var"
 }
 ```
 
-If the block is named (`<name>` is provided), the `config` block can be explicitly referenced in a `config` argument inside `content`. This is helpful if there is a need to have more than one configuration for the same content provider.
+{{< hint warning >}}
+Fabric limits which environment variables are exposed inside the templates. By default, only the
+variables that match `FABRIC_*` name glob pattern are available under `env` namespace and in the
+evaluation context. The glob pattern is configured using `expose_env_vars_with_pattern` argument
+in [Global configuration]({{< ref "configs.md#global-configuration">}}).
 
-If `<name>` isn't provided, the configuration acts as a default configuration for a specified content provider.
+Name-based filtering minimizes the risk of unintentionally leaking sensitive environment variables to
+untrustworthy plugins. It's recommended to use restrictive glob patterns as much as possible.
+{{< /hint >}}
 
-### Supported arguments
+## Metadata
 
-The arguments allowed in the configuration block depend on the content provider. See [Plugins]({{< ref "plugins.md" >}}) for the details on the configuration parameters supported.
+Define the metadata for a template or a block using the `meta` block. The metadata includes the
+template name, tags, authors' names, license, version, and other relevant information.
 
-### Supported nested blocks
+`meta` block supports following arguments:
 
-Nested blocks aren't supported inside the `config` blocks.
+- `name` — name of the template or the block
+- `description` — description of the template or the block
+- `url` — a string with a URL
+- `license` — the license that applies to the template or the block
+- `authors` — a list of the authors
+- `tags` — a list of tags
+- `updated_at` — ISO8601-formatted date with time
+- `version` — version of the template or the block
 
-### Example
+For example:
 
 ```hcl
-config content openai_text {
-  api_key = 'some-openai-api-key'
+# Document template with metadata
+document "mitre_ctid_campaign_report" {
 
-  system_prompt = 'You are the best at saying Hi!'
-}
+  meta {
+    name = "MITRE CTID Campaign Report Template"
 
-document "test-document" {
+    description = <<-EOT
+      The Campaign Report is designed to highlight new information related to a threat actor or
+      capabilities. This should focus on new information and highlight how it poses a changed risk
+      to your organization. This should not be an exhaustive product cataloguing all information
+      about the topic, but rather a succinct report designed to convey a change in the status quo to
+      the intended recipient.
+    EOT
 
-  content openai_text {
-    prompt = "Say hi!"
+    url = "https://github.com/center-for-threat-informed-defense/cti-blueprints"
+
+    license = "Apache License 2.0"
+    tags = ["mitre", "campaign"]
+
+    updated_at = "2024-01-22T10:00:01+01:00"
   }
+
+}
+
+content text "disclaimer" {
+  meta {
+    name = "Disclaimer text"
+    tags = ["foo", "bar"]
+  }
+
+  value = "Some disclaimer text"
 }
 ```
 

docs/language/content-blocks.md

@@ -4,10 +4,11 @@ description: Learn how to use Fabric content blocks efficiently for building mod
 type: docs
 weight: 60
 ---
-
 # Content blocks
 
-`content` blocks define document segments: text paragraphs, tables, graphs, lists, etc. The block signature includes the name of the content provider that will execute the content block. Fabric supports many [content pproviders]({{< ref "content-providers.md" >}}) through its [plugin ecosystem]({{< ref "plugins.md" >}}).
+`content` blocks define document segments: text paragraphs, tables, graphs, lists, etc.
+
+The block signature includes the name of the content provider that will execute the content block.
 
 ```hcl
 # Root-level definition of a content block
@@ -40,72 +41,62 @@ A content block is rendered by a specified content provider. See [Content Provid
 
 ## Supported arguments
 
-The arguments provided in the block are either generic arguments or plugin-specific input parameters.
+The arguments provided in the block are either generic arguments or provider-specific arguments.
 
 ### Generic arguments
 
-- `config`: (optional) a reference to a named configuration block for the data source or a content provider. If provided, it takes precedence over the default configuration. See [Content provider configuration]({{< ref "configs.md#content-provider-configuration" >}}) for the details.
-- `query`: (optional) a [JQ](https://jqlang.github.io/jq/manual/) query to be executed against the context object. The results of the query will be placed under `query_result` field in the context. See [Context](#context) object for the details.
-- `render_if_no_query_result`: (optional) a boolean flag that determines if the content block should be rendered when `query` returned no data. Defaults to `true`.
-- `text_when_no_query_result`: (optional) provides the text to be rendered instead of calling the plugin when `render_if_no_query_result` is `true`.
-- `query_input_required`: (optional) a boolean flag that specifies if `query_input` must be explicitly provided when the content block is referenced. `false` by default. See [Query Input Requirement]({{< ref "references.md#query-input-requirement" >}}) for the details.
-- `query_input`: (optional) a JQ query to be applied to the context object. The results of the query are stored under `query_input` field in the context. See [Query Input Requirement]({{< ref "references.md#query-input-requirement" >}}) for the details.
+- `config`: (optional) a reference to a named configuration block for the content provider. If
+  provided, it takes precedence over the default configuration. See content provider
+  [configuration details]({{< ref "configs.md#block-configuration" >}}) for more information.
+- `local_var`: (optional) a shortcut for specifying a local variable. See [Variables]({{< ref
+  "context.md#variables" >}}) for the details.
 
 ### Content provider arguments
 
-Content provider specific are defined by a plugin specification. See [Plugins]({{< ref "plugins.md" >}}) for the details on the supported arguments per plugin.
-
-Content provider arguments differ per content provider. See the documentation for a specific content provider (find it in [supported content providers]({{< ref "content-providers.md" >}})) for the details on the supported arguments.
+Content provider arguments differ per content provider. See the documentation for a specific content provider (find it in [Content Providers]({{< ref "content-providers.md" >}}) documentation) for the details on the supported arguments.
 
 ## Supported nested blocks
 
-- `meta`: (optional) a block containing metadata for the block.
+- `meta`: (optional) a block containing metadata for the block. See [Metadata]({{< ref "configs.md#metadata" >}}) for details.
 - `config`: (optional) an inline configuration for the block. If provided, it takes precedence over the `config` argument and default configuration for the content provider.
-
-## Context
-
-When Fabric renders a content block, a corresponding content provider is executed. Along with the [configuration]({{< ref "configs/#content-provider-configuration" >}}) and the input parameters, the provider receives the context object with the data available for the provider.
-
-The context object is a JSON dictionary with pre-defined root-level fields:
-
-- `data` points to a map of all resolved data definitions for the document. The JSON path to a specific data point follows the data block signature: `data.<data-source>.<block-name>`.
-- `query_result` points to a result of the execution of a JQ query provided in `query` argument.
+- `vars`: (optional) a block with variable definitions. See [Variables]({{< ref
+  "context.md#variables" >}}) for the details.
 
 ## References
 
 See [References]({{< ref references.md >}}) for the details about referencing content blocks.
 
 ## Example
 
-The template `document.test-doc` with one `data.inline` block, one `content.text` block and one `content.openai_text` block:
-
+FIXME: TORUN
 ```hcl
 config content openai_text "test_account" {
-  api_key = "<OPENAI-KEY>"
+  # Reading a key from an environment variable
+  api_key = env.FABRIC_OPENAI_KEY
 }
 
 document "test-doc" {
 
-  data inline "foo" {
+  vars {
     items = ["aaa", "bbb", "ccc"]
   }
 
   content text {
     # Query contains a JQ query executed against the context
-    query = ".data.inline.foo.items | length"
+    local_var = ".vars.items | length"
 
-    # The result of the query is stored in the `query_result` field in the context.
-    # The context is available for the templating engine inside the `content.text` plugin.
-    value = "There are {{ .query_result }} items"
+    # The context can be accessed in Go templates
+    value = "There are {{ .vars.local }} items: {{ .vars.items | toPrettyJson }}"
   }
 
   content openai_text {
     config = config.content.openai_text.test_account
-    query = ".data.inline.foo"
 
     prompt = <<-EOT
        Write a short story, just a paragraph, about space exploration
-       using the values from the provided items list as character names.
+       using the values from the provided items list as character names:
+
+       {{ .vars.items | toPrettyJson }}
     EOT
   }
 }