Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TransformOptions Configuration Block to SecretProviderClass #963

Closed
wants to merge 12 commits into from
Closed

Add TransformOptions Configuration Block to SecretProviderClass #963

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
2599999
json path work in progress
manedurphy May 27, 2022
5021a7d
implemented json path templating
manedurphy May 27, 2022
ed2fcbe
removed "todo" comments
manedurphy May 27, 2022
0f94d4c
small bug fix and code formatting
manedurphy May 27, 2022
70f8e3e
cleanup
manedurphy Aug 17, 2022
28f2fc0
regenerate charts
manedurphy Aug 17, 2022
06e579f
add unit tests
manedurphy Aug 17, 2022
f10e694
wip
manedurphy Aug 17, 2022
0b53f7c
added comments back
manedurphy Aug 17, 2022
d302f22
validate secret type
manedurphy Aug 17, 2022
3d42b90
added unit test for transformOptions with json path
manedurphy Aug 17, 2022
a4c59fe
more error handling
manedurphy Aug 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions apis/v1/secretproviderclass_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,32 @@ type SecretObject struct {
Data []*SecretObjectData `json:"data,omitempty"`
}

type Secret struct {
// name of the object to sync
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't follow this very well in a lot of the code but per https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences comments should be full sentences. Don't try to fix the other stuff in this PR but would be good to try to apply to the comments added here.

ObjectName string `json:"objectName,omitempty"`
// expected format of the secret received from the provider (e.g. plaintext, json)
Format string `json:"format,omitempty"`
// JSON path to target for a secret received in JSON format
JsonPath string `json:"jsonPath,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSONPath instead of JsonPath per https://github.com/golang/go/wiki/CodeReviewComments#initialisms

}

type TransformOptions struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we actually want the ProviderFilePath or InFilePath here to note the input data for the translation. I could see a single SecretProviderClass that has a number of different secrets that you want to split out in different ways.

// expected format of the secret received from the provider (e.g. plaintext, json)
Format string `json:"format,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can actually omit this just determine behavior based on whether JsonPath is populated.

// JSON path to target for a secret received in JSON format
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice from the implementation that it expects the JsonPath to resolve to string key's that become filenames. I think this comment should be explicit about that behavior.

Additionally I think it might be nice to support extracting a single value if the path resolves to a string instead of a json object. It may be a bit more verbose but I think more explicit to support resolving to a json value and adding an OutFilePath parameter. Consider a response from the provider that says file my_secret has value {"username":"admin", "password": "hunter2"}

You could then write an SPC like:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: app-secrets
spec:
  provider: vault
  parameters:
    objects: |
      - objectName: "my_secret"
        secretPath: "secret/data/my-kv"
        secretKey: "blob"
  transforms:
    - inFilePath: "my_secret"
      jsonPath: "$.username"
      outFilePath: "username"
    - inFilePath: "my_secret"
      jsonPath: "$.password"
      outFilePath: "password"

Later we could add a transform like the this to resolve #948

  transforms:
    - staticValue: "admin"
      outFilePath: "username"

Where we could generate a file based just on SPC data for non-secret static value admin to file username.

JsonPath string `json:"jsonPath,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSONPath

Secrets []Secret `json:"secrets,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand what this represents and it doesn't seem like the test cases cover where TransformOptions.Secrets is populated?

I see the example in #963 (comment) but I'm not clear why the secret: part there is needed.

}

// SecretProviderClassSpec defines the desired state of SecretProviderClass
type SecretProviderClassSpec struct {
// Configuration for provider name
Provider Provider `json:"provider,omitempty"`
// Configuration for specific provider
Parameters map[string]string `json:"parameters,omitempty"`
SecretObjects []*SecretObject `json:"secretObjects,omitempty"`
// Configuration for secret transformation
TransformOptions TransformOptions `json:"transformOptions,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want this to be a list like []*TransformOptions so that multiple transforms can be done on the secrets before presenting in the pod filesystem.

}

// ByPodStatus defines the state of SecretProviderClass as seen by
Expand Down
36 changes: 36 additions & 0 deletions apis/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions charts/secrets-store-csi-driver/crds/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,34 @@ spec:
type: string
type: object
type: array
transformOptions:
description: Configuration for secret transformation
properties:
format:
description: expected format of the secret received from the provider
(e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in JSON
format
type: string
secrets:
items:
properties:
format:
description: expected format of the secret received from
the provider (e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in
JSON format
type: string
objectName:
description: name of the object to sync
type: string
type: object
type: array
type: object
type: object
status:
description: SecretProviderClassStatus defines the observed state of SecretProviderClass
Expand Down
28 changes: 28 additions & 0 deletions config/crd/bases/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,34 @@ spec:
type: string
type: object
type: array
transformOptions:
description: Configuration for secret transformation
properties:
format:
description: expected format of the secret received from the provider
(e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in JSON
format
type: string
secrets:
items:
properties:
format:
description: expected format of the secret received from
the provider (e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in
JSON format
type: string
objectName:
description: name of the object to sync
type: string
type: object
type: array
type: object
type: object
status:
description: SecretProviderClassStatus defines the observed state of SecretProviderClass
Expand Down
28 changes: 28 additions & 0 deletions deploy/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,34 @@ spec:
type: string
type: object
type: array
transformOptions:
description: Configuration for secret transformation
properties:
format:
description: expected format of the secret received from the provider
(e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in JSON
format
type: string
secrets:
items:
properties:
format:
description: expected format of the secret received from
the provider (e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in
JSON format
type: string
objectName:
description: name of the object to sync
type: string
type: object
type: array
type: object
type: object
status:
description: SecretProviderClassStatus defines the observed state of SecretProviderClass
Expand Down
28 changes: 28 additions & 0 deletions ...harts/secrets-store-csi-driver/crds/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,34 @@ spec:
type: string
type: object
type: array
transformOptions:
description: Configuration for secret transformation
properties:
format:
description: expected format of the secret received from the provider
(e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in JSON
format
type: string
secrets:
items:
properties:
format:
description: expected format of the secret received from
the provider (e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in
JSON format
type: string
objectName:
description: name of the object to sync
type: string
type: object
type: array
type: object
type: object
status:
description: SecretProviderClassStatus defines the observed state of SecretProviderClass
Expand Down
28 changes: 28 additions & 0 deletions manifest_staging/deploy/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,34 @@ spec:
type: string
type: object
type: array
transformOptions:
description: Configuration for secret transformation
properties:
format:
description: expected format of the secret received from the provider
(e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in JSON
format
type: string
secrets:
items:
properties:
format:
description: expected format of the secret received from
the provider (e.g. plaintext, json)
type: string
jsonPath:
description: JSON path to target for a secret received in
JSON format
type: string
objectName:
description: name of the object to sync
type: string
type: object
type: array
type: object
type: object
status:
description: SecretProviderClassStatus defines the observed state of SecretProviderClass
Expand Down
6 changes: 6 additions & 0 deletions pkg/errors/errors.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,10 @@ const (
PodVolumeNotFound = "PodVolumeNotFound"
// FileWriteError error
FileWriteError = "FileWriteError"
// MarhsalError error
MarshalError = "MarshalError"
// UnmarhsalError error
UnmarshalError = "UnmarshalError"
// InvalidJSONPathError error
InvalidJSONPathError = "InvalidJSONPathError"
)
2 changes: 1 addition & 1 deletion pkg/rotation/reconciler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@ func (r *Reconciler) reconcile(ctx context.Context, spcps *secretsstorev1.Secret
r.generateEvent(pod, corev1.EventTypeWarning, mountRotationFailedReason, fmt.Sprintf("failed to lookup provider client: %q", providerName))
return fmt.Errorf("failed to lookup provider client: %q", providerName)
}
newObjectVersions, errorReason, err := secretsstore.MountContent(ctx, providerClient, string(paramsJSON), string(secretsJSON), spcps.Status.TargetPath, string(permissionJSON), oldObjectVersions)
newObjectVersions, errorReason, err := secretsstore.MountContent(ctx, providerClient, string(paramsJSON), string(secretsJSON), spcps.Status.TargetPath, string(permissionJSON), oldObjectVersions, &spc.Spec.TransformOptions)
if err != nil {
r.generateEvent(pod, corev1.EventTypeWarning, mountRotationFailedReason, fmt.Sprintf("provider mount err: %+v", err))
return fmt.Errorf("failed to rotate objects for pod %s/%s, err: %w", spcps.Namespace, spcps.Status.PodName, err)
Expand Down
7 changes: 4 additions & 3 deletions pkg/secrets-store/nodeserver.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"path/filepath"
"time"

v1 "sigs.k8s.io/secrets-store-csi-driver/apis/v1"
internalerrors "sigs.k8s.io/secrets-store-csi-driver/pkg/errors"
"sigs.k8s.io/secrets-store-csi-driver/pkg/k8s"
"sigs.k8s.io/secrets-store-csi-driver/pkg/util/fileutil"
Expand Down Expand Up @@ -238,7 +239,7 @@ func (ns *nodeServer) NodePublishVolume(ctx context.Context, req *csi.NodePublis
}
mounted = true
var objectVersions map[string]string
if objectVersions, errorReason, err = ns.mountSecretsStoreObjectContent(ctx, providerName, string(parametersStr), string(secretStr), targetPath, string(permissionStr), podName); err != nil {
if objectVersions, errorReason, err = ns.mountSecretsStoreObjectContent(ctx, providerName, string(parametersStr), string(secretStr), targetPath, string(permissionStr), podName, &spc.Spec.TransformOptions); err != nil {
klog.ErrorS(err, "failed to mount secrets store object content", "pod", klog.ObjectRef{Namespace: podNamespace, Name: podName})
return nil, fmt.Errorf("failed to mount secrets store objects for pod %s/%s, err: %w", podNamespace, podName, err)
}
Expand Down Expand Up @@ -335,7 +336,7 @@ func (ns *nodeServer) NodeUnstageVolume(ctx context.Context, req *csi.NodeUnstag
return &csi.NodeUnstageVolumeResponse{}, nil
}

func (ns *nodeServer) mountSecretsStoreObjectContent(ctx context.Context, providerName, attributes, secrets, targetPath, permission, podName string) (map[string]string, string, error) {
func (ns *nodeServer) mountSecretsStoreObjectContent(ctx context.Context, providerName, attributes, secrets, targetPath, permission, podName string, transformOptions *v1.TransformOptions) (map[string]string, string, error) {
if len(attributes) == 0 {
return nil, "", errors.New("missing attributes")
}
Expand All @@ -358,7 +359,7 @@ func (ns *nodeServer) mountSecretsStoreObjectContent(ctx context.Context, provid

klog.InfoS("Using gRPC client", "provider", providerName, "pod", podName)

return MountContent(ctx, client, attributes, secrets, targetPath, permission, nil)
return MountContent(ctx, client, attributes, secrets, targetPath, permission, nil, transformOptions)
}

func (ns *nodeServer) NodeGetInfo(ctx context.Context, req *csi.NodeGetInfoRequest) (*csi.NodeGetInfoResponse, error) {
Expand Down
2 changes: 1 addition & 1 deletion pkg/secrets-store/nodeserver_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -405,7 +405,7 @@ func TestMountSecretsStoreObjectContent(t *testing.T) {
if err != nil {
t.Fatalf("expected error to be nil, got: %+v", err)
}
_, errorReason, err := ns.mountSecretsStoreObjectContent(context.TODO(), "provider1", test.attributes, test.secrets, test.targetPath, test.permission, "pod")
_, errorReason, err := ns.mountSecretsStoreObjectContent(context.TODO(), "provider1", test.attributes, test.secrets, test.targetPath, test.permission, "pod", nil)
if errorReason != test.expectedErrorReason {
t.Fatalf("expected error reason to be %s, got: %s", test.expectedErrorReason, errorReason)
}
Expand Down
Loading