[confmap/provider] feat: Add confmap.Provider that supports AES decry…

…ption of values (#35549)

**Description:**


This package provides a `confmap.Provider` implementation for symmetric
AES encryption of credentials (and other sensitive values) in
configurations. It relies on the environment variable
`OTEL_CREDENTIAL_PROVIDER` with the value of the AES key, base64
encoded. 16, 24, or 32 byte keys are supported, selecting AES-128,
AES-192, or AES-256 respectively.

An AES 32-byte (AES-256) key can be generated using the following
command:

```shell
openssl rand -base64 32
```

Configurations can now use placeholders with the following pattern
`${credential:<encrypted & base64-encoded value>}`. The value will be
decrypted using the AES key provided in the environment variable
`OTEL_CREDENTIAL_PROVIDER`

> For example:
> 
> ```shell
> export
OTEL_CREDENTIAL_PROVIDER="GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac="
> ```
> 
> ```yaml
> password: ${aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=}
> ```
> 
> will resolve to:
> ```yaml
> password: '1'
> ```

Since AES is a symmetric encryption algorithm, the same key must be used
to encrypt and decrypt the values. If the key is exchanged between the
collector and a server, it should be done over a secure connection.

When the collector persists its configuration to disk, storing the key
in the environment prevents compromising secrets in the configuration.
It still presents a vulnerability if the attacker has access to the
collector's memory or the environment's configuration, but increases
security over plaintext configurations.


**Testing:**

Unit tests with 93.0% coverage, built agent and configured with
`${credential:<encrypted value>}` values.

**Documentation:**

`README.md` reflecting this PR description.

**Issue:**


#35550

.chloggen/aes-credential-provider.yaml

@@ -0,0 +1,16 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: new_component
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: confmap/aesprovider
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Initial aes encryption provider. Allows configurations to decrypt secrets using AES encryption.
+
+# One or more tracking issues related to the change
+issues: [35550]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:

.github/CODEOWNERS

@@ -23,6 +23,7 @@ cmd/telemetrygen/                                                   @open-teleme
 
 confmap/provider/s3provider/                                        @open-telemetry/collector-contrib-approvers @Aneurysm9
 confmap/provider/secretsmanagerprovider/                            @open-telemetry/collector-contrib-approvers @driverpt @atoulme
+confmap/provider/aesprovider/                                       @open-telemetry/collector-contrib-approvers @djaglowski @shazlehu
 
 connector/countconnector/                                           @open-telemetry/collector-contrib-approvers @djaglowski @jpkrohling
 connector/datadogconnector/                                         @open-telemetry/collector-contrib-approvers @mx-psi @dineshg13 @ankitpatel96

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -23,6 +23,7 @@ body:
       - cmd/otelcontribcol
       - cmd/oteltestbedcol
       - cmd/telemetrygen
+      - confmap/provider/aesprovider
       - confmap/provider/s3provider
       - confmap/provider/secretsmanagerprovider
       - connector/count

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -17,6 +17,7 @@ body:
       - cmd/otelcontribcol
       - cmd/oteltestbedcol
       - cmd/telemetrygen
+      - confmap/provider/aesprovider
       - confmap/provider/s3provider
       - confmap/provider/secretsmanagerprovider
       - connector/count

.github/ISSUE_TEMPLATE/other.yaml

@@ -17,6 +17,7 @@ body:
       - cmd/otelcontribcol
       - cmd/oteltestbedcol
       - cmd/telemetrygen
+      - confmap/provider/aesprovider
       - confmap/provider/s3provider
       - confmap/provider/secretsmanagerprovider
       - connector/count

.github/ISSUE_TEMPLATE/unmaintained.yaml

@@ -22,6 +22,7 @@ body:
       - cmd/otelcontribcol
       - cmd/oteltestbedcol
       - cmd/telemetrygen
+      - confmap/provider/aesprovider
       - confmap/provider/s3provider
       - confmap/provider/secretsmanagerprovider
       - connector/count

confmap/provider/aesprovider/Makefile

@@ -0,0 +1 @@
+include ../../../Makefile.Common

confmap/provider/aesprovider/README.md

@@ -0,0 +1,33 @@
+## Summary
+
+This package provides a `confmap.Provider` implementation for symmetric AES encryption of credentials (and other sensitive values) in configurations. It relies on the environment variable `OTEL_AES_CREDENTIAL_PROVIDER` set to the value of the AES key, base64 encoded. 16, 24, or 32 byte keys are supported, selecting AES-128, AES-192, or AES-256 respectively.
+
+An AES 32-byte (AES-256) key can be generated using the following command:
+
+```shell
+openssl rand -base64 32
+```
+
+## How it works
+ Use placeholders with the following pattern `${aes:<encrypted & base64-encoded value>}` in a configuration. The value will be decrypted using the AES key provided in the environment variable `OTEL_AES_CREDENTIAL_PROVIDER`
+
+> For example:
+> 
+> ```shell
+> export OTEL_AES_CREDENTIAL_PROVIDER="GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac="
+> ```
+> 
+> ```yaml
+> password: ${aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=}
+> ```
+> 
+> will resolve to:
+> ```yaml
+> password: '1'
+> ```
+
+## Caveats
+
+Since AES is a symmetric encryption algorithm, the same key must be used to encrypt and decrypt the values. If the key needs to be exchanged between the collector and a server, it should be done over a secure connection.
+
+When the collector persists its configuration to disk, storing the key in the environment prevents compromising secrets in the configuration. It still presents a vulnerability if the attacker has access to the collector's memory or the environment's configuration, but increases security over plaintext configurations.

confmap/provider/aesprovider/go.mod

@@ -0,0 +1,23 @@
+module github.com/open-telemetry/opentelemetry-collector-contrib/confmap/provider/aesprovider
+
+go 1.22.0
+
+require (
+	github.com/stretchr/testify v1.9.0
+	go.opentelemetry.io/collector/confmap v1.17.1-0.20241008154146-ea48c09c31ae
+	go.uber.org/zap v1.27.0
+
+)
+
+require (
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
+	github.com/knadh/koanf/maps v0.1.1 // indirect
+	github.com/knadh/koanf/providers/confmap v0.1.0 // indirect
+	github.com/knadh/koanf/v2 v2.1.1 // indirect
+	github.com/mitchellh/copystructure v1.2.0 // indirect
+	github.com/mitchellh/reflectwalk v1.0.2 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)

confmap/provider/aesprovider/metadata.yaml

@@ -0,0 +1,3 @@
+status:
+  codeowners:
+    active: [djaglowski, shazlehu]

confmap/provider/aesprovider/provider.go

@@ -0,0 +1,108 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package aesprovider // import "github.com/open-telemetry/opentelemetry-collector-contrib/confmap/provider/aesprovider"
+
+import (
+	"context"
+	"crypto/aes"
+	"crypto/cipher"
+	"encoding/base64"
+	"fmt"
+	"os"
+	"strings"
+
+	"go.opentelemetry.io/collector/confmap"
+	"go.uber.org/zap"
+)
+
+const (
+	schemaName = "aes"
+	// This environment variable holds a base64-encoded AES key, either 16, 24, or 32 bytes to select AES-128, AES-192, or AES-256.
+	keyEnvVar = "OTEL_AES_CREDENTIAL_PROVIDER"
+)
+
+type provider struct {
+	logger *zap.Logger
+	key    []byte
+}
+
+// NewFactory creates a new provider factory
+func NewFactory() confmap.ProviderFactory {
+	return confmap.NewProviderFactory(
+		func(settings confmap.ProviderSettings) confmap.Provider {
+			return &provider{
+				logger: settings.Logger,
+			}
+		})
+}
+
+func (*provider) Scheme() string {
+	return schemaName
+}
+
+func (*provider) Shutdown(context.Context) error {
+	return nil
+}
+
+func (p *provider) Retrieve(_ context.Context, uri string, _ confmap.WatcherFunc) (*confmap.Retrieved, error) {
+
+	if !strings.HasPrefix(uri, schemaName+":") {
+		return nil, fmt.Errorf("%q uri is not supported by %q provider", uri, schemaName)
+	}
+
+	if p.key == nil {
+		// base64 decode env var
+		base64Key, ok := os.LookupEnv(keyEnvVar)
+		if !ok {
+			return nil, fmt.Errorf("env var %q not set, required for %q provider", keyEnvVar, schemaName)
+		}
+		key, err := base64.StdEncoding.DecodeString(base64Key)
+		if err != nil {
+			return nil, fmt.Errorf("%q provider uri failed to base64 decode key: %w", schemaName, err)
+		}
+		p.key = key
+	}
+
+	// Remove schemaName
+	cipherText := strings.Replace(uri, schemaName+":", "", 1)
+
+	clearText, err := p.decrypt(cipherText)
+	if err != nil {
+		return nil, fmt.Errorf("%q provider failed to decrypt value: %w", schemaName, err)
+	}
+
+	return confmap.NewRetrieved(clearText)
+}
+
+func (p *provider) decrypt(cipherText string) (string, error) {
+
+	cipherBytes, err := base64.StdEncoding.DecodeString(cipherText)
+	if err != nil {
+		return "", err
+	}
+
+	block, err := aes.NewCipher(p.key)
+	if err != nil {
+		return "", err
+	}
+
+	aesGCM, err := cipher.NewGCM(block)
+	if err != nil {
+		return "", err
+	}
+
+	nonceSize := aesGCM.NonceSize()
+	if len(cipherBytes) < nonceSize {
+		return "", fmt.Errorf("ciphertext too short")
+	}
+
+	nonce, cipherBytes := cipherBytes[:nonceSize], cipherBytes[nonceSize:]
+
+	clearBytes, err := aesGCM.Open(nil, nonce, cipherBytes, nil)
+	if err != nil {
+		return "", err
+	}
+
+	return string(clearBytes), nil
+}

confmap/provider/aesprovider/provider_test.go

@@ -0,0 +1,121 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package aesprovider
+
+import (
+	"context"
+	"os"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+	"go.opentelemetry.io/collector/confmap"
+)
+
+func TestAESCredentialProvider(t *testing.T) {
+
+	tests := []struct {
+		name          string
+		configValue   string
+		expectedValue string
+		expectedError string
+		envVars       map[string]string
+	}{
+		{
+			name:          "Valid type, key, JSON value",
+			configValue:   `aes:RsEf6cTWrssi8tls+5Tlk3H8AQBVT2uVDcJ88zj3Sph1iSN8z97yGlxNrWf7oUHSyS4lISEcuqrQq2sPsvcl480rEajM0DIWNVoPi2ApKhADCygGoRxbykm4Tuce+rx0aWIPj1zDkuBM6NIYI1E9H2gxrH+P89Hlwmwq26S3HkqIvtW/BFAHJ8Z08iIg95NFwdbZBu4bMwY93r0KWhL0Y4xk9PGmCZhHk5jaaCgd5YREhJCH8ahenZ/t71yGdu71gSIVbg1xwGZNDKf9wRpOCW4oVQ+OWORDrgKX+58QZMOu0zjRxIkeFQV66xMIEziLYERW0xnT/GthH2+FdO/OlrUlzTBnQBbgqpp86cW1xKPFE1XeFJAIHXca7sYIjl/bz6csiUGSXKVyzApsz6fQEhaFwPGw7YKg/hN4+AEu7zYwHlpiPZyQZVE8xPEgwAy1ZKKk7nJ391ujXohXEsmc7u40bOdQmvktyjemf3KcYSCSoVqvdm49K8/ZfKgG1LMCSnHMk0HWdVGyO8jjBj3RnhpT1/dQ/aMudwPMsPFE+85GYEPTe9Sjq8erkjLRfGomDYWJhSpMESMovVRKpyjv9W+3xS0fHracj1AIx8iRQ9KNq7YKSX9n+wtE7LXMr6CUxwDs0wm1FCdVpc/JfH7BghbAh9qNSZg7qeNWQO9BG9vVa31EpRTDHOOogzGOQU2APtjc0qU65we6lpShBl2HU6S/5SgjB5m9ZnCsJSqlCQTf/e/Riuvx8l5LBlv1JNbHnLD6LO7xarpEKzR2Nc2N2+6pP86SvVB/ZqxGug06SUckjQbrmVrjU5X0RFWQAb4ZdPUobxk2xOXGhxUxEB/pDv5DcuDaEry97XsYBgzYpCtVZr8uQc5kd5jPcMsVgIYo78t+v+2yvCdYtRSHOrAcrOyBbrXCo1yI4UA9qAmfBE1PWC7km9xdhtlIAA5Szei+2oRxCwSvVO0TeYCwByDmYDolL0Tv5jtdgsPbcgnZsL/b9KRBAUU4wXKVm55mzw3AiOehX/bms84XLnRWZaxN06tJ/DiMbMcatTQP0pxk4zoemVD66wo7dA8U0nrnfP8AMfQmFQ==`,
+			expectedValue: `{   "type": "service_account",   "project_id": "my-test-project-12345",   "private_key_id": "abcdef1234567890abcdef1234567890abcdef12",   "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCAmMwggJfAgEAAoGBALKlO+j9ALhlg5po\nfakePrivateKeyValue+/abc/def/123/fakeData==\n-----END PRIVATE KEY-----\n",   "client_email": "test-service-account@my-test-project-12345.iam.gserviceaccount.com",   "client_id": "123456789012345678901",   "auth_uri": "https://accounts.google.com/o/oauth2/auth",   "token_uri": "https://oauth2.googleapis.com/token",   "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",   "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test-service-account%40my-test-project-12345.iam.gserviceaccount.com" }`,
+
+			envVars: map[string]string{
+				"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
+			},
+		},
+
+		{
+			name:          "Invalid base64 key",
+			configValue:   `aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=`,
+			expectedError: `"aes" provider uri failed to base64 decode key: illegal base64 data at input byte 25`,
+
+			envVars: map[string]string{
+				"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8lN8bVU2k0TAKtzwJzac=",
+			},
+		},
+		{
+			name:          "Invalid AES key",
+			configValue:   `aes:RsEf6cTWrssi8tls+5Tlk3H8AQBVT2uVDcJ88zj3Sph1iSN8z97yGlxNrWf7oUHSyS4lISEcuqrQq2sPsvcl480rEajM0DIWNVoPi2ApKhADCygGoRxbykm4Tuce+rx0aWIPj1zDkuBM6NIYI1E9H2gxrH+P89Hlwmwq26S3HkqIvtW/BFAHJ8Z08iIg95NFwdbZBu4bMwY93r0KWhL0Y4xk9PGmCZhHk5jaaCgd5YREhJCH8ahenZ/t71yGdu71gSIVbg1xwGZNDKf9wRpOCW4oVQ+OWORDrgKX+58QZMOu0zjRxIkeFQV66xMIEziLYERW0xnT/GthH2+FdO/OlrUlzTBnQBbgqpp86cW1xKPFE1XeFJAIHXca7sYIjl/bz6csiUGSXKVyzApsz6fQEhaFwPGw7YKg/hN4+AEu7zYwHlpiPZyQZVE8xPEgwAy1ZKKk7nJ391ujXohXEsmc7u40bOdQmvktyjemf3KcYSCSoVqvdm49K8/ZfKgG1LMCSnHMk0HWdVGyO8jjBj3RnhpT1/dQ/aMudwPMsPFE+85GYEPTe9Sjq8erkjLRfGomDYWJhSpMESMovVRKpyjv9W+3xS0fHracj1AIx8iRQ9KNq7YKSX9n+wtE7LXMr6CUxwDs0wm1FCdVpc/JfH7BghbAh9qNSZg7qeNWQO9BG9vVa31EpRTDHOOogzGOQU2APtjc0qU65we6lpShBl2HU6S/5SgjB5m9ZnCsJSqlCQTf/e/Riuvx8l5LBlv1JNbHnLD6LO7xarpEKzR2Nc2N2+6pP86SvVB/ZqxGug06SUckjQbrmVrjU5X0RFWQAb4ZdPUobxk2xOXGhxUxEB/pDv5DcuDaEry97XsYBgzYpCtVZr8uQc5kd5jPcMsVgIYo78t+v+2yvCdYtRSHOrAcrOyBbrXCo1yI4UA9qAmfBE1PWC7km9xdhtlIAA5Szei+2oRxCwSvVO0TeYCwByDmYDolL0Tv5jtdgsPbcgnZsL/b9KRBAUU4wXKVm55mzw3AiOehX/bms84XLnRWZaxN06tJ/DiMbMcatTQP0pxk4zoemVD66wo7dA8U0nrnfP8AMfQmFQ==`,
+			expectedError: `"aes" provider failed to decrypt value: crypto/aes: invalid key size 1`,
+
+			envVars: map[string]string{
+				"OTEL_AES_CREDENTIAL_PROVIDER": "MQ==",
+			},
+		},
+
+		{
+			name:          "simple message",
+			configValue:   "aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=",
+			expectedValue: "1",
+			envVars: map[string]string{
+				"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
+			},
+		},
+		{
+			name:          "empty message",
+			configValue:   "aes:RsEf6cTWrssi8tlsZ4V68iFEJRFI8o71+QoYYw==",
+			expectedValue: "",
+			envVars: map[string]string{
+				"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
+			},
+		},
+		{
+			name:          "Truncated base64 value",
+			configValue:   "aes:R=",
+			expectedError: `"aes" provider failed to decrypt value: illegal base64 data at input byte 1`,
+			envVars: map[string]string{
+				"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
+			},
+		},
+		{
+			name:          "Truncated encryted text",
+			configValue:   "aes:MQ==",
+			expectedError: `"aes" provider failed to decrypt value: ciphertext too short`,
+			envVars: map[string]string{
+				"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
+			},
+		},
+		{
+			name:          "Wrong schema",
+			configValue:   "foo:MQ==",
+			expectedError: `"foo:MQ==" uri is not supported by "aes" provider`,
+		},
+		{
+			name:          "No env vars",
+			configValue:   "aes:MQ==",
+			expectedError: `env var "OTEL_AES_CREDENTIAL_PROVIDER" not set, required for "aes" provider`,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			os.Clearenv()
+			for k, v := range tt.envVars {
+				if err := os.Setenv(k, v); err != nil {
+					t.Fatalf("Failed to set env var %s: %v", k, err)
+				}
+			}
+
+			p := NewFactory().Create(confmap.ProviderSettings{})
+			retrieved, err := p.Retrieve(context.Background(), tt.configValue, nil)
+			if tt.expectedError == "" {
+				require.NoError(t, err)
+			} else {
+				require.Error(t, err)
+				require.Equal(t, tt.expectedError, err.Error())
+				return
+			}
+			require.NotNil(t, retrieved)
+			stringValue, err := retrieved.AsString()
+			require.NoError(t, err)
+			require.Equal(t, tt.expectedValue, stringValue)
+		})
+	}
+}