[Fleet] Exception when trying to save CSPM integration with a Secret

[criamico]

Kibana version: Found in 8.12

Describe the bug:

Steps to reproduce:

• Install cloud_security_posture-1.8.0-preview02 (note that a licence is needed to install CSPM)
• Select Setup access: manual and Preferred manual method: Direct access keys
• Add some test values a secrets and try to install

TypeError: Cannot read properties of undefined (reading 'id')
    at reduce (secrets.ts:790:66)
    at Array.reduce (<anonymous>)
    at forEach (secrets.ts:782:21)
    at Array.forEach (<anonymous>)
    at getPolicyWithSecretReferences (secrets.ts:781:15)
    at extractAndWriteSecrets (secrets.ts:244:32)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at PackagePolicyClientWithAuthz.create (package_policy.ts:253:28)
    at createPackagePolicyHandler (handlers.ts:270:27)
    at core_versioned_route.ts:179:22
    at Router.handle (router.ts:228:30)
    at handler (router.ts:162:13)
    at exports.Manager.execute (/Users/**/elastic/kibana/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
    at Object.internals.handler (/Users/**/elastic/kibana/node_modules/@hapi/hapi/lib/handler.js:46:20)
    at exports.execute (/Users/**/elastic/kibana/node_modules/@hapi/hapi/lib/handler.js:31:20)
    at Request._lifecycle (/Users/**/elastic/kibana/node_modules/@hapi/hapi/lib/request.js:371:32)
    at Request._execute (/Users/**/elastic/kibana/node_modules/@hapi/hapi/lib/request.js:281:9)

Expected behavior:
The policy should install as expected and show encrypted secrets

** Additional context**
The line that is breaking is this one:

kibana/x-pack/plugins/fleet/server/services/secrets.ts

Line 790 in fbf9fe4

acc[val].value = toVarSecretRef(secrets[secretPathIndex].id);

In the reduce function we are trying to access an index in "Secrets" variable that doesn't exists. There is no check for the length of that variable, or it's simply assumed that has the same length as secretPaths, which in this case is not true.

This function isn't tested at all so we should make sure to add regression tests to avoid breaking existing functionality.

I had a PR where I was simply checking for existence of that index, but while testing I found that it wasn't encrypting correctly the policy in the editor, so I closed it. When fixing this bug, make sure that the final policy correctly hides the secrets both in the editor and in the "view policy" panel.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[jen-huang]

@criamico, could you take a look at fixing this bug for 8.12?

[kpollich]

Summarizing some of my findings here.

Following the steps to reproduce, the arguments received by getPolicyWithSecretReferences are as follows:

{
  "secretPaths": [
    {
      "path": [
        "inputs",
        "1",
        "streams",
        "0",
        "vars",
        "secret_access_key"
      ],
      "value": {
        "type": "text"
      }
    },
    {
      "path": [
        "inputs",
        "2",
        "streams",
        "0",
        "vars",
        "secret_access_key"
      ],
      "value": {
        "value": "changeme"
      }
    },
    {
      "path": [
        "inputs",
        "4",
        "streams",
        "0",
        "vars",
        "azure.credentials.client_secret"
      ],
      "value": {
        "type": "text"
      }
    },
    {
      "path": [
        "inputs",
        "4",
        "streams",
        "0",
        "vars",
        "azure.credentials.client_password"
      ],
      "value": {
        "type": "text"
      }
    },
    {
      "path": [
        "inputs",
        "4",
        "streams",
        "0",
        "vars",
        "azure.credentials.client_certificate_password"
      ],
      "value": {
        "type": "text"
      }
    }
  ],
  "secrets": [
    {
      "id": "iGGa1YwBli8js5OpznzB"
    }
  ],
  "packagePolicy": {
    "name": "cspm-2",
    "namespace": "default",
    "description": "",
    "package": {
      "name": "cloud_security_posture",
      "title": "Security Posture Management",
      "version": "1.8.0-preview02"
    },
    "enabled": true,
    "policy_id": "4bfb3e36-b027-4751-bf9b-5346d46a24c0",
    "inputs": [
      {
        "type": "cloudbeat/cis_k8s",
        "policy_template": "kspm",
        "enabled": false,
        "streams": [
          {
            "enabled": false,
            "data_stream": {
              "type": "logs",
              "dataset": "cloud_security_posture.findings"
            },
            "id": "cloudbeat/cis_k8s-cloud_security_posture.findings-1123651f-2e59-4fc7-a6d1-a0ed649b1e4b"
          }
        ]
      },
      {
        "type": "cloudbeat/cis_eks",
        "policy_template": "kspm",
        "enabled": false,
        "streams": [
          {
            "enabled": false,
            "data_stream": {
              "type": "logs",
              "dataset": "cloud_security_posture.findings"
            },
            "vars": {
              "access_key_id": {
                "type": "text"
              },
              "secret_access_key": {
                "type": "text"
              },
              "session_token": {
                "type": "text"
              },
              "shared_credential_file": {
                "type": "text"
              },
              "credential_profile_name": {
                "type": "text"
              },
              "role_arn": {
                "type": "text"
              },
              "aws.credentials.type": {
                "type": "text"
              }
            },
            "id": "cloudbeat/cis_eks-cloud_security_posture.findings-1123651f-2e59-4fc7-a6d1-a0ed649b1e4b"
          }
        ]
      },
      {
        "type": "cloudbeat/cis_aws",
        "policy_template": "cspm",
        "enabled": true,
        "streams": [
          {
            "enabled": true,
            "data_stream": {
              "type": "logs",
              "dataset": "cloud_security_posture.findings"
            },
            "vars": {
              "access_key_id": {
                "value": "elastic"
              },
              "secret_access_key": {
                "value": "changeme"
              },
              "session_token": {
                "type": "text"
              },
              "shared_credential_file": {
                "type": "text"
              },
              "credential_profile_name": {
                "type": "text"
              },
              "role_arn": {
                "type": "text"
              },
              "aws.credentials.type": {
                "value": "direct_access_keys"
              },
              "aws.account_type": {
                "value": "organization-account",
                "type": "text"
              }
            },
            "id": "cloudbeat/cis_aws-cloud_security_posture.findings-1123651f-2e59-4fc7-a6d1-a0ed649b1e4b"
          }
        ],
        "config": {
          "cloud_formation_template_url": {}
        }
      },
      {
        "type": "cloudbeat/cis_gcp",
        "policy_template": "cspm",
        "enabled": false,
        "streams": [
          {
            "enabled": false,
            "data_stream": {
              "type": "logs",
              "dataset": "cloud_security_posture.findings"
            },
            "vars": {
              "gcp.account_type": {
                "type": "text"
              },
              "gcp.organization_id": {
                "type": "text"
              },
              "gcp.project_id": {
                "type": "text"
              },
              "gcp.credentials.type": {
                "type": "text"
              },
              "gcp.credentials.file": {
                "type": "text"
              },
              "gcp.credentials.json": {
                "type": "text"
              }
            },
            "id": "cloudbeat/cis_gcp-cloud_security_posture.findings-1123651f-2e59-4fc7-a6d1-a0ed649b1e4b"
          }
        ]
      },
      {
        "type": "cloudbeat/cis_azure",
        "policy_template": "cspm",
        "enabled": false,
        "streams": [
          {
            "enabled": false,
            "data_stream": {
              "type": "logs",
              "dataset": "cloud_security_posture.findings"
            },
            "vars": {
              "azure.account_type": {
                "type": "text"
              },
              "azure.credentials.type": {
                "type": "text"
              },
              "azure.credentials.client_id": {
                "type": "text"
              },
              "azure.credentials.tenant_id": {
                "type": "text"
              },
              "azure.credentials.client_secret": {
                "type": "text"
              },
              "azure.credentials.client_username": {
                "type": "text"
              },
              "azure.credentials.client_password": {
                "type": "text"
              },
              "azure.credentials.client_certificate_path": {
                "type": "text"
              },
              "azure.credentials.client_certificate_password": {
                "type": "text"
              }
            },
            "id": "cloudbeat/cis_azure-cloud_security_posture.findings-1123651f-2e59-4fc7-a6d1-a0ed649b1e4b"
          }
        ]
      },
      {
        "type": "cloudbeat/vuln_mgmt_aws",
        "policy_template": "vuln_mgmt",
        "enabled": false,
        "streams": [
          {
            "enabled": false,
            "data_stream": {
              "type": "logs",
              "dataset": "cloud_security_posture.vulnerabilities"
            },
            "id": "cloudbeat/vuln_mgmt_aws-cloud_security_posture.vulnerabilities-1123651f-2e59-4fc7-a6d1-a0ed649b1e4b"
          }
        ],
        "config": {
          "cloud_formation_template_url": {
            "value": "https://console.aws.amazon.com/cloudformation/home#/stacks/quickcreate?templateURL=https://elastic-cspm-cft.s3.eu-central-1.amazonaws.com/cloudformation-cnvm-8.12.0.yml&stackName=Elastic-Vulnerability-Management&param_EnrollmentToken=FLEET_ENROLLMENT_TOKEN&param_FleetUrl=FLEET_URL&param_ElasticAgentVersion=KIBANA_VERSION&param_ElasticArtifactServer=https://artifacts.elastic.co/downloads/beats/elastic-agent"
          }
        }
      }
    ],
    "vars": {
      "posture": {
        "value": "cspm",
        "type": "text"
      },
      "deployment": {
        "value": "aws",
        "type": "text"
      }
    }
  }
}

Here's the raw API request value that's sent when making a POST to create the CSPM integration policy as well:

{
  "name": "cspm-2",
  "description": "",
  "namespace": "default",
  "policy_id": "9755128b-f9e6-403f-9c00-ca9c9b213f64",
  "enabled": true,
  "inputs": [
    {
      "type": "cloudbeat/cis_k8s",
      "policy_template": "kspm",
      "enabled": false,
      "streams": [
        {
          "enabled": false,
          "data_stream": {
            "type": "logs",
            "dataset": "cloud_security_posture.findings"
          }
        }
      ]
    },
    {
      "type": "cloudbeat/cis_eks",
      "policy_template": "kspm",
      "enabled": false,
      "streams": [
        {
          "enabled": false,
          "data_stream": {
            "type": "logs",
            "dataset": "cloud_security_posture.findings"
          },
          "vars": {
            "access_key_id": {
              "type": "text"
            },
            "secret_access_key": {
              "type": "text"
            },
            "session_token": {
              "type": "text"
            },
            "shared_credential_file": {
              "type": "text"
            },
            "credential_profile_name": {
              "type": "text"
            },
            "role_arn": {
              "type": "text"
            },
            "aws.credentials.type": {
              "type": "text"
            }
          }
        }
      ]
    },
    {
      "type": "cloudbeat/cis_aws",
      "policy_template": "cspm",
      "enabled": true,
      "streams": [
        {
          "enabled": true,
          "data_stream": {
            "type": "logs",
            "dataset": "cloud_security_posture.findings"
          },
          "vars": {
            "access_key_id": {
              "value": "elastic"
            },
            "secret_access_key": {
              "value": "changeme"
            },
            "session_token": {
              "type": "text"
            },
            "shared_credential_file": {
              "type": "text"
            },
            "credential_profile_name": {
              "type": "text"
            },
            "role_arn": {
              "type": "text"
            },
            "aws.credentials.type": {
              "value": "direct_access_keys"
            },
            "aws.account_type": {
              "value": "organization-account",
              "type": "text"
            }
          }
        }
      ],
      "config": {
        "cloud_formation_template_url": {}
      }
    },
    {
      "type": "cloudbeat/cis_gcp",
      "policy_template": "cspm",
      "enabled": false,
      "streams": [
        {
          "enabled": false,
          "data_stream": {
            "type": "logs",
            "dataset": "cloud_security_posture.findings"
          },
          "vars": {
            "gcp.account_type": {
              "type": "text"
            },
            "gcp.organization_id": {
              "type": "text"
            },
            "gcp.project_id": {
              "type": "text"
            },
            "gcp.credentials.type": {
              "type": "text"
            },
            "gcp.credentials.file": {
              "type": "text"
            },
            "gcp.credentials.json": {
              "type": "text"
            }
          }
        }
      ]
    },
    {
      "type": "cloudbeat/cis_azure",
      "policy_template": "cspm",
      "enabled": false,
      "streams": [
        {
          "enabled": false,
          "data_stream": {
            "type": "logs",
            "dataset": "cloud_security_posture.findings"
          },
          "vars": {
            "azure.account_type": {
              "type": "text"
            },
            "azure.credentials.type": {
              "type": "text"
            },
            "azure.credentials.client_id": {
              "type": "text"
            },
            "azure.credentials.tenant_id": {
              "type": "text"
            },
            "azure.credentials.client_secret": {
              "type": "text"
            },
            "azure.credentials.client_username": {
              "type": "text"
            },
            "azure.credentials.client_password": {
              "type": "text"
            },
            "azure.credentials.client_certificate_path": {
              "type": "text"
            },
            "azure.credentials.client_certificate_password": {
              "type": "text"
            }
          }
        }
      ]
    },
    {
      "type": "cloudbeat/vuln_mgmt_aws",
      "policy_template": "vuln_mgmt",
      "enabled": false,
      "streams": [
        {
          "enabled": false,
          "data_stream": {
            "type": "logs",
            "dataset": "cloud_security_posture.vulnerabilities"
          }
        }
      ],
      "config": {
        "cloud_formation_template_url": {
          "value": "https://console.aws.amazon.com/cloudformation/home#/stacks/quickcreate?templateURL=https://elastic-cspm-cft.s3.eu-central-1.amazonaws.com/cloudformation-cnvm-8.12.0.yml&stackName=Elastic-Vulnerability-Management&param_EnrollmentToken=FLEET_ENROLLMENT_TOKEN&param_FleetUrl=FLEET_URL&param_ElasticAgentVersion=KIBANA_VERSION&param_ElasticArtifactServer=https://artifacts.elastic.co/downloads/beats/elastic-agent"
        }
      }
    }
  ],
  "package": {
    "name": "cloud_security_posture",
    "title": "Security Posture Management",
    "version": "1.8.0-preview02"
  },
  "vars": {
    "posture": {
      "value": "cspm",
      "type": "text"
    },
    "deployment": {
      "value": "aws",
      "type": "text"
    }
  },
  "force": false
}

Comparing the above to a known successful test case:

kibana/x-pack/plugins/fleet/server/services/secrets.test.ts

Lines 1017 to 1037 in 33f8368

	describe('when only required secret value is provided', () => {
	it('returns single secret reference for required secret', async () => {
	const mockPackagePolicy = {
	vars: {
	'pkg-secret-1': {
	value: 'pkg-secret-1-val',
	},
	},
	inputs: [],
	} as unknown as NewPackagePolicy;
	const result = await extractAndWriteSecrets({
	packagePolicy: mockPackagePolicy,
	packageInfo: mockIntegrationPackage,
	esClient: esClientMock,
	});
	expect(esClientMock.transport.request).toHaveBeenCalledTimes(1);
	expect(result.secretReferences).toHaveLength(1);
	});
	});

{
  "secretPaths": [
    {
      "value": {
        "value": "pkg-secret-2-val"
      },
      "path": [
        "vars",
        "pkg-secret-2"
      ]
    }
  ],
  "secrets": [
    {
      "id": "e4bc6689-d1b5-433a-9ffa-fdccb72f4f25"
    }
  ],
  "packagePolicy": {
    "vars": {
      "pkg-secret-2": {
        "value": "pkg-secret-2-val"
      }
    },
    "inputs": []
  }
}

This test case should be a similar case to what's happening in CSPM, as we have multiple defined secrets in the package manifest, but only a single secret is provided in the request.

They key difference I see here is that with the CSPM request, we're receiving secretPaths even for secrets that are not set. So we have a mismatch of the counts for secretPaths and secrets, which shouldn't be the case. It seems like the getPolicyWithSecretReferences expects these arrays to be the same size today.

I was able to come up with a better test case by updating a known passing test to pass all secret paths instead of just those with values. I'll use that test case to figure out next steps here and post another followup.

[kpollich]

Update: this wound up being a one line fix. We were passing the unfiltered secrets array to the function instead of the filtered secretsToCreate

https://github.com/elastic/kibana/pull/174264/files#diff-da5efbb23b0aa4bda0850e130206b226217827821bbf0dcc0a6b343ab23618fbR245

I've updated the test cases related to this change to as mentioned above.