Skip to content

Commit

Permalink
Docs updates
Browse files Browse the repository at this point in the history
  • Loading branch information
traut committed Jun 14, 2024
1 parent 9e14cf1 commit bf82d52
Show file tree
Hide file tree
Showing 13 changed files with 683 additions and 124 deletions.
59 changes: 56 additions & 3 deletions docs/language/configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,12 @@ fabric {
}
```

## Data source, content provider and publisher configurations
## Block configuration

The data sources, content provides and publishers can be configured using `config` blocks.

`config` blocks expose configuration properties supported by data sources, content providers and
publishers.
`config` block arguments are specific to a data source, content provider or a publisher the block if
configuring.

`config` block must be defined on a root level of Fabric file, outside `document` block. The
signature of `config` block consists of a block type selector (`content`, `data` or `publish`), a
Expand Down Expand Up @@ -120,6 +120,59 @@ document "test-document" {
}
```

## Metadata

The metadata for the template or the block can be defined with `meta` block. The metadata includes the name of the template, the tags, the names of the authors, license, version, etc.
block license, etc.

`meta` block supports following arguments:

- `name` — name of the template or the block
- `description` — description of the template or the block
- `url` — a string with a URL
- `license` — the license that applies to the template or the block where `meta` is defined
- `authors` — a list of the authors
- `tags` — a list of tags
- `updated_at` — ISO8601-formatted date with time
- `version` — version of the template or the block

For example:

```hcl
# Document template with metadata
document "mitre_ctid_campaign_report" {
meta {
name = "MITRE CTID Campaign Report Template"
description = <<-EOT
The Campaign Report is designed to highlight new information related to a threat actor or
capabilities. This should focus on new information and highlight how it poses a changed risk
to your organization. This should not be an exhaustive product cataloguing all information
about the topic, but rather a succinct report designed to convey a change in the status quo to
the intended recipient.
EOT
url = "https://github.com/center-for-threat-informed-defense/cti-blueprints"
license = "Apache License 2.0"
tags = ["mitre", "campaign"]
updated_at = "2024-01-22T10:00:01+01:00"
}
}
content text "disclaimer" {
meta {
name = "Disclaimer text"
tags = ["foo", "bar"]
}
value = "Some disclaimer text"
}
```

## Next steps

See [Documents]({{< ref "documents.md" >}}) to learn how to build document templates in Fabric configuration language.
47 changes: 21 additions & 26 deletions docs/language/content-blocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@ description: Learn how to use Fabric content blocks efficiently for building mod
type: docs
weight: 60
---

# Content blocks

`content` blocks define document segments: text paragraphs, tables, graphs, lists, etc. The block signature includes the name of the content provider that will execute the content block. Fabric supports many [content pproviders]({{< ref "content-providers.md" >}}) through its [plugin ecosystem]({{< ref "plugins.md" >}}).
`content` blocks define document segments: text paragraphs, tables, graphs, lists, etc.

The block signature includes the name of the content provider that will execute the content block.

```hcl
# Root-level definition of a content block
Expand Down Expand Up @@ -40,66 +41,60 @@ A content block is rendered by a specified content provider. See [Content Provid

## Supported arguments

The arguments provided in the block are either generic arguments or provider-specific input parameters.
The arguments provided in the block are either generic arguments or provider-specific arguments.

### Generic arguments

- `config`: (optional) a reference to a named configuration block for the content provider. If provided, it takes precedence over the default configuration. See [Content provider configuration]({{< ref "configs.md#content-provider-configuration" >}}) for the details.
- `query`: (optional) a [JQ](https://jqlang.github.io/jq/manual/) query to be executed against the context object. The results of the query will be placed under `query_result` field in the context. See [Context](#context) object for the details.
- `config`: (optional) a reference to a named configuration block for the content provider. If
provided, it takes precedence over the default configuration. See content provider
[configuration details]({{< ref "configs.md#block-configuration" >}}) for more information.
- `local_var`: (optional) a shortcut for specifying a local variable. See [Variables]({{< ref
"context.md#variables" >}}) for the details.

### Content provider arguments

Content provider arguments differ per content provider. See the documentation for a specific content provider (find it in [supported content providers]({{< ref "content-providers.md" >}})) for the details on the supported arguments.
Content provider arguments differ per content provider. See the documentation for a specific content provider (find it in [Content Providers]({{< ref "content-providers.md" >}}) documentation) for the details on the supported arguments.

## Supported nested blocks

- `meta`: (optional) a block containing metadata for the block.
- `meta`: (optional) a block containing metadata for the block. See [Metadata]({{< ref "configs.md#metadata" >}}) for details.
- `config`: (optional) an inline configuration for the block. If provided, it takes precedence over the `config` argument and default configuration for the content provider.

## Context

When Fabric renders a content block, a corresponding content provider is executed. Along with the [configuration]({{< ref "configs/#content-provider-configuration" >}}) and the input parameters, the provider receives the context object with the data available for the provider.

The context object is a JSON dictionary with pre-defined root-level fields:

- `data` points to a map of all resolved data definitions for the document. The JSON path to a specific data point follows the data block signature: `data.<data-source>.<block-name>`.
- `query_result` points to a result of the execution of a JQ query provided in `query` argument.

## References

See [References]({{< ref references.md >}}) for the details about referencing content blocks.

## Example

The template `document.test-doc` with one `data.inline` block, one `content.text` block and one `content.openai_text` block:

FIXME: TORUN
```hcl
config content openai_text "test_account" {
api_key = "<OPENAI-KEY>"
# Reading a key from an environment variable
api_key = env.FABRIC_OPENAI_KEY
}
document "test-doc" {
data inline "foo" {
vars {
items = ["aaa", "bbb", "ccc"]
}
content text {
# Query contains a JQ query executed against the context
query = ".data.inline.foo.items | length"
local_var = ".vars.items | length"
# The result of the query is stored in the `query_result` field in the context.
# The context is available for the templating engine inside the `content.text` plugin.
value = "There are {{ .query_result }} items"
# The context can be accessed in Go templates
value = "There are {{ .vars.local }} items: {{ .vars.items | toPrettyJson }}"
}
content openai_text {
config = config.content.openai_text.test_account
query = ".data.inline.foo"
prompt = <<-EOT
Write a short story, just a paragraph, about space exploration
using the values from the provided items list as character names.
using the values from the provided items list as character names:
{{ .vars.items | toPrettyJson }}
EOT
}
}
Expand Down
139 changes: 139 additions & 0 deletions docs/language/context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
---
title: Evaluation Context
description: Learn about the evaluation context, the shared data structure that holds data available for the blocks the Fabric configuration language files.
type: docs
weight: 45
---

# Evaluation context

The evaluation context keeps all available data during the template evaluation. This includes the
results of `data` block executions, the template metadata, and defined variables.

The context data can be queried with `query_jq()` function and is available in Go templates (if
supported by the plugin).

## Keys

The context is a dictionary. The values can be accessed using JSON paths, and the data can be
filtered or mutated using JQ queries.

Using JQ-style JSON path syntax, the root keys are:

- `.data` — stores the results of the `data` blocks evaluation under
`.data.<data-source-name>.<block-name>` keys
- `.document` — contains the data about the current document template. For example, `.document.meta`
has values from the `meta` block defined on the root level of the template
- `.vars` — holds the values of variables defined in the current or parent blocks.

The context also can contain local properties:
- `.section` — holds the data about the current section, when called inside one. Similar to
`.document.meta`, `.section.meta` contains the data from `meta` block defined inside the section.
- `.content` — holds the data about the current content block, when called inside one. Similar to
`.document.meta` and `.section.meta`, `.content.meta` stores the data from `meta` block defined
inside the content block.

## Variables

Define variables inside the template using the `vars` block:

```hcl
vars {
foo = 1
bar = {
x = "a"
y = env.CUSTOM_ENV_VAR
}
# The variables are evaluated in the order of definition
baz = query_jq(".vars.foo")
}
```

The `vars` block can be defined inside `document`, `section`, and `content` blocks.

The variable values can be static (like `foo` and `bar` in the example above) or results of data mutations
(like `baz`, which contains the result of a JQ query applied to the context).

When evaluated, the variable becomes available in the context under the `.vars` root keyword.
For example, `.vars.foo` and `.vars.bar` refer to the `foo` and `bar` variables from the snippet
above.

### Local variable

A shortcut for defining a single local variable is to use `local_var` argument, which sets
`.vars.local` variable.

For example:

```hcl
content text {
local_var = "World"
value = "Hello, {{ .vars.local }}!"
}
```

will render `Hello, World!`.

### Inheritance

Variables defined in a parent block (such as `document` or `section`) are available for use in nested
blocks. Nested blocks can redefine a variable in their context, overwriting the parent variable's
value.

For example:

```hcl
section {
vars {
foo = 11
bar = 22
}
section {
vars {
foo = 33
baz = 44
}
content text {
# Renders: `Variable values: foo=33, bar=22, baz=44`
value = "Variable values: foo={{ .vars.foo }}, bar={{ .vars.bar }}, baz={{ .vars.baz }}"
}
}
}
```

### `query_jq()` function

To filter and mutate the data in the context, use [JQ queries](https://jqlang.github.io/jq/manual/). The queries specified as calls of `query_jq()` function execute against the context and return the results into the variable.

For example:

```hcl
section {
vars {
items = ["a", "b", "c"]
}
section {
vars {
items_count = query_jq(".vars.items | length")
items_uppercase = query_jq(
<<-JQ
.vars.items[1] | ascii_upcase
JQ
)
}
content text {
# Renders: `Items count: 3; Uppercase items: A:B:C`
value = "Items count: {{ .vars.items_count }}; Uppercase items: {{ .vars.items_uppercase }}"
}
}
}
```

## Next steps

See [Data Blocks]({{< ref "data-blocks.md" >}}) documentation to learn how to define data requirements in the templates.
29 changes: 19 additions & 10 deletions docs/language/data-blocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ description: Learn how to use Fabric data blocks for defining data requirements
type: docs
weight: 50
---

# Data blocks

`data` blocks define data requirements for the template. The block signature includes the name of the data source that will execute the data block and load the data from external source. Fabric supports many [data sources]({{< ref "data-sources.md" >}}) through its [plugin ecosystem]({{< ref "plugins.md" >}}).
`data` blocks define data requirements for the template. The data block represents a call to an
integration with provided parameters.

The block signature includes the name of the data source that will execute the data block.

```hcl
# Root-level definition of a data block
Expand All @@ -25,28 +27,35 @@ document "foobar" {
}
```

Both a data source name and a block name are required. The pair makes an unique identifier for the block.
Both a data source name and a block name are required, making an unique identifier for the block.

The data blocks must be placed either on a root-level of the file or on a root-level of the document.
The data blocks must be placed either on a root-level of the file or on a root-level of the
document.

When Fabric starts rendering the template, the data sources for the data blocks are executed and the results are placed under the block names in the context (see [Context]({{< ref "content-blocks.md#context" >}}) for more details), available for queries.
When Fabric renders the template, the data blocks are executed and the results are stored in the
context (see [Context]({{< ref context.md >}}) for more details), available for other blocks to use.

## Supported arguments

The arguments provided in the block are either generic arguments or data-source-specific input parameters.
The arguments provided in the block are either generic arguments or data-source-specific arguments.

### Generic arguments

- `config`: (optional) a reference to a named configuration block for the data source or a content provider. If provided, it takes precedence over the default configuration. See [Data source configuration]({{< ref "configs.md#data-source-configuration" >}}) for the details.
- `config`: (optional) a reference to a named configuration block for the data source. If provided,
it takes precedence over the default configuration. See data source [configuration details]({{<
ref "configs.md#block-configuration" >}}) for more information.

### Data source arguments

Data source arguments differ per data source. See the documentation for a specific data source (find it in [supported data sources]({{< ref "data-sources.md" >}})) for the details on the supported arguments.
Data source arguments differ per data source. See the documentation for a specific data source (find
it in [supported data sources]({{< ref "data-sources.md" >}})) for the details on the supported
arguments.

## Supported nested blocks

- `meta`: (optional) a block containing metadata for the block.
- `config`: (optional) an inline configuration for the block. If provided, it takes precedence over the `config` argument and default configuration for the content provider.
- `config`: (optional) an inline configuration for the block. If provided, it takes precedence over
the `config` argument and default configuration for the data source.

## References

Expand All @@ -71,7 +80,7 @@ document "test-document" {
data csv "events_b" {
config {
delimiter = ",";
delimiter = ","
}
path = "/tmp/events-b.csv"
Expand Down
11 changes: 6 additions & 5 deletions docs/language/documents.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,12 @@ The `document` block is a structure that groups the data definitions, the sectio

## Supported nested blocks

- `meta`: (optional) a block containing metadata for the document.
- `data`: see [Data Blocks]({{< ref data-blocks.md >}}) for the details.
- `content`: see [Content Blocks]({{< ref content-blocks.md >}}) for the details.
- `section`: see [Section Blocks]({{< ref section-blocks.md >}}) for the details.
- `publish`: see [Publish Blocks]({{< ref publish-blocks.md >}}) for the details.
- `meta`: see [Metadata]({{< ref "configs.md/#metadata" >}})
- `data`: see [Data Blocks]({{< ref data-blocks.md >}})
- `vars`: see [Variables]({{< ref "context.md/#variables" >}})
- `content`: see [Content Blocks]({{< ref content-blocks.md >}})
- `section`: see [Section Blocks]({{< ref section-blocks.md >}})
- `publish`: see [Publish Blocks]({{< ref publish-blocks.md >}})

## Next steps

Expand Down
Loading

0 comments on commit bf82d52

Please sign in to comment.