From 7d8609cc724ccab8fdcb8d5d26c645b5913a8b90 Mon Sep 17 00:00:00 2001 From: Andrew Wilkins Date: Tue, 21 Nov 2023 09:32:03 +0800 Subject: [PATCH 1/6] pkg/ottl: add section about design principles Drawing inspiration from https://github.com/bazelbuild/starlark#design-principles and https://github.com/google/cel-spec/blob/master/doc/langdef.md#overview, add a section about design principles. The aim of this is to ensure OTTL is and remains, by design, safe for execution of untrusted programs in multi-tenant systems where a tenant can provide their own OTTL programs. --- pkg/ottl/LANGUAGE.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/pkg/ottl/LANGUAGE.md b/pkg/ottl/LANGUAGE.md index b346f2a4ef6c..7e45aa33f12e 100644 --- a/pkg/ottl/LANGUAGE.md +++ b/pkg/ottl/LANGUAGE.md @@ -2,6 +2,12 @@ The OTTL grammar includes function invocations, Values and Boolean Expressions. These parts all fit into a Statement, which is the basis of execution in the OTTL. +### Design principles + +The OTTL and built-in [ottlfuncs](//github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs) are designed according to these principles: +- OTTL programs operating in separate contexts cannot influence one another -- an OTTL program may have side-effects only within its own execution [context](#contexts). +- OTTL programs cannot loop forever, except through the use of non built-in functions. + ### Editors Editors are functions that transform the underlying telemetry payload. They may return a value, but typically do not. There must be a single Editor Invocation in each OTTL statement. @@ -288,4 +294,4 @@ It is possible to update the Value in a telemetry field using a Setter. For read ## Logging inside a OTTL function -To emit logs inside a OTTL function, add a parameter of type [`component.TelemetrySettings`](https://pkg.go.dev/go.opentelemetry.io/collector/component#TelemetrySettings) to the function signature. The OTTL will then inject the TelemetrySettings that were passed to `NewParser` into the function. TelemetrySettings can be used to emit logs. \ No newline at end of file +To emit logs inside a OTTL function, add a parameter of type [`component.TelemetrySettings`](https://pkg.go.dev/go.opentelemetry.io/collector/component#TelemetrySettings) to the function signature. The OTTL will then inject the TelemetrySettings that were passed to `NewParser` into the function. TelemetrySettings can be used to emit logs. From c58ee351bf0df08dc89e58a96c0392d61fd1007d Mon Sep 17 00:00:00 2001 From: Andrew Wilkins Date: Tue, 21 Nov 2023 10:23:03 +0800 Subject: [PATCH 2/6] Fix link --- pkg/ottl/LANGUAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkg/ottl/LANGUAGE.md b/pkg/ottl/LANGUAGE.md index 7e45aa33f12e..6aff2e9630f4 100644 --- a/pkg/ottl/LANGUAGE.md +++ b/pkg/ottl/LANGUAGE.md @@ -4,7 +4,7 @@ The OTTL grammar includes function invocations, Values and Boolean Expressions. ### Design principles -The OTTL and built-in [ottlfuncs](//github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs) are designed according to these principles: +The OTTL and built-in [ottlfuncs](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs) are designed according to these principles: - OTTL programs operating in separate contexts cannot influence one another -- an OTTL program may have side-effects only within its own execution [context](#contexts). - OTTL programs cannot loop forever, except through the use of non built-in functions. From 4f7a2d99db70854a1914ec79e12209b767545197 Mon Sep 17 00:00:00 2001 From: Andrew Wilkins Date: Tue, 28 Nov 2023 15:08:26 +0800 Subject: [PATCH 3/6] Clarifications --- pkg/ottl/LANGUAGE.md | 9 ++++++--- pkg/ottl/ottlfuncs/README.md | 11 +++++++++++ 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/pkg/ottl/LANGUAGE.md b/pkg/ottl/LANGUAGE.md index 6aff2e9630f4..4cfb9800ad6e 100644 --- a/pkg/ottl/LANGUAGE.md +++ b/pkg/ottl/LANGUAGE.md @@ -4,9 +4,12 @@ The OTTL grammar includes function invocations, Values and Boolean Expressions. ### Design principles -The OTTL and built-in [ottlfuncs](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs) are designed according to these principles: -- OTTL programs operating in separate contexts cannot influence one another -- an OTTL program may have side-effects only within its own execution [context](#contexts). -- OTTL programs cannot loop forever, except through the use of non built-in functions. +The OTTL is designed according to these principles (excluding user-defined functions): + +- *Terminating*: OTTL statements cannot loop forever. +- *Static*: OTTL functions and statements cannot be dynamically loaded or executed. + +These principles do not apply to user-defined functions. ### Editors diff --git a/pkg/ottl/ottlfuncs/README.md b/pkg/ottl/ottlfuncs/README.md index acf2ed70658a..f5e775a20b44 100644 --- a/pkg/ottl/ottlfuncs/README.md +++ b/pkg/ottl/ottlfuncs/README.md @@ -8,6 +8,17 @@ This document contains documentation for both types of OTTL functions: - [Functions](#functions) that transform telemetry. - [Converters](#converters) that provide utilities for transforming telemetry. +## Design principles + +For the standard OTTL functions described in this document, we specify design principles to ensure they are always +secure and safe for use: + +- Built-in OTTL functions may not access the file system, network, or any other I/O devices. +- Built-in OTTL functions may share information only through their parameters and results. + +OTTL functions are implemented in Go, and so are fundamentally limited to what can be implemented by a Go program. +User-defined OTTL functions may therefore not adhere the above principles. + ## Working with functions Functions generally expect specific types to be returned by `Paths`. From b50deca18a4b0f950dc013c436a5835ff1095a76 Mon Sep 17 00:00:00 2001 From: Andrew Wilkins Date: Mon, 4 Dec 2023 16:03:04 +0800 Subject: [PATCH 4/6] Address review comments - Remove bit about dynamic loading, it's not yet decided - Move termination to ottlfuncs - Add a brief explanation of intent behind OTTL - Reword about (~limitless) constraints imposed by Go --- pkg/ottl/LANGUAGE.md | 10 +++++----- pkg/ottl/ottlfuncs/README.md | 3 ++- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/pkg/ottl/LANGUAGE.md b/pkg/ottl/LANGUAGE.md index 4cfb9800ad6e..707ac8a99e6d 100644 --- a/pkg/ottl/LANGUAGE.md +++ b/pkg/ottl/LANGUAGE.md @@ -4,12 +4,12 @@ The OTTL grammar includes function invocations, Values and Boolean Expressions. ### Design principles -The OTTL is designed according to these principles (excluding user-defined functions): +The OTTL is intended as a domain-specific language (DSL) for telemetry mutation and generation, +and is not intended to be used as a general-purpose programming language. -- *Terminating*: OTTL statements cannot loop forever. -- *Static*: OTTL functions and statements cannot be dynamically loaded or executed. - -These principles do not apply to user-defined functions. +The OTTL has been designed to work directly with [pdata](https://github.com/open-telemetry/opentelemetry-collector/tree/main/pdata). +While the OTTL can work with external data formats due to being implemented directly in Go instead of on top of a custom VM, +it is not expressly intended to perform operations that don't touch `pdata` in some way. ### Editors diff --git a/pkg/ottl/ottlfuncs/README.md b/pkg/ottl/ottlfuncs/README.md index f5e775a20b44..581fd35d1adb 100644 --- a/pkg/ottl/ottlfuncs/README.md +++ b/pkg/ottl/ottlfuncs/README.md @@ -15,8 +15,9 @@ secure and safe for use: - Built-in OTTL functions may not access the file system, network, or any other I/O devices. - Built-in OTTL functions may share information only through their parameters and results. +- Built-in OTTL functions must be terminating; they must not loop forever. -OTTL functions are implemented in Go, and so are fundamentally limited to what can be implemented by a Go program. +OTTL functions are implemented in Go, and so are only limited by what can be implemented by a Go program. User-defined OTTL functions may therefore not adhere the above principles. ## Working with functions From 35e437a0066a1588b36203698fa0e2e65b58c744 Mon Sep 17 00:00:00 2001 From: Andrew Wilkins Date: Tue, 5 Dec 2023 07:54:43 +0800 Subject: [PATCH 5/6] Apply suggestions from code review Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com> --- pkg/ottl/LANGUAGE.md | 8 ++++---- pkg/ottl/ottlfuncs/README.md | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/pkg/ottl/LANGUAGE.md b/pkg/ottl/LANGUAGE.md index 707ac8a99e6d..ef1f18461a8f 100644 --- a/pkg/ottl/LANGUAGE.md +++ b/pkg/ottl/LANGUAGE.md @@ -4,12 +4,12 @@ The OTTL grammar includes function invocations, Values and Boolean Expressions. ### Design principles -The OTTL is intended as a domain-specific language (DSL) for telemetry mutation and generation, +OTTL is intended as a domain-specific language (DSL) for telemetry mutation and generation, and is not intended to be used as a general-purpose programming language. -The OTTL has been designed to work directly with [pdata](https://github.com/open-telemetry/opentelemetry-collector/tree/main/pdata). -While the OTTL can work with external data formats due to being implemented directly in Go instead of on top of a custom VM, -it is not expressly intended to perform operations that don't touch `pdata` in some way. +OTTL has been designed to work directly with [pdata](https://github.com/open-telemetry/opentelemetry-collector/tree/main/pdata). +While OTTL can work with external data formats due to being implemented directly in Go instead of on top of a custom VM, +it is not expressly intended to perform operations that don't touch pdata in some way. ### Editors diff --git a/pkg/ottl/ottlfuncs/README.md b/pkg/ottl/ottlfuncs/README.md index 581fd35d1adb..c48d992f47b7 100644 --- a/pkg/ottl/ottlfuncs/README.md +++ b/pkg/ottl/ottlfuncs/README.md @@ -17,7 +17,7 @@ secure and safe for use: - Built-in OTTL functions may share information only through their parameters and results. - Built-in OTTL functions must be terminating; they must not loop forever. -OTTL functions are implemented in Go, and so are only limited by what can be implemented by a Go program. +OTTL functions are implemented in Go, and so are only limited by what can be implemented in a Go program. User-defined OTTL functions may therefore not adhere the above principles. ## Working with functions From be2cd7fdf2c168f8d014ca146cd01834655c61ee Mon Sep 17 00:00:00 2001 From: Andrew Wilkins Date: Thu, 7 Dec 2023 07:43:25 +0800 Subject: [PATCH 6/6] Update pkg/ottl/LANGUAGE.md Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com> --- pkg/ottl/LANGUAGE.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/pkg/ottl/LANGUAGE.md b/pkg/ottl/LANGUAGE.md index ef1f18461a8f..05df67e0ffef 100644 --- a/pkg/ottl/LANGUAGE.md +++ b/pkg/ottl/LANGUAGE.md @@ -7,9 +7,7 @@ The OTTL grammar includes function invocations, Values and Boolean Expressions. OTTL is intended as a domain-specific language (DSL) for telemetry mutation and generation, and is not intended to be used as a general-purpose programming language. -OTTL has been designed to work directly with [pdata](https://github.com/open-telemetry/opentelemetry-collector/tree/main/pdata). -While OTTL can work with external data formats due to being implemented directly in Go instead of on top of a custom VM, -it is not expressly intended to perform operations that don't touch pdata in some way. +OTTL has been designed to work directly with [pdata](https://github.com/open-telemetry/opentelemetry-collector/tree/main/pdata), but can operate on other data formats. ### Editors