Skip to content
This repository has been archived by the owner on Jul 26, 2022. It is now read-only.

Commit

Permalink
feat(ibm): add spec option keyByName to support the use of a name, in…
Browse files Browse the repository at this point in the history
…stead of id, as the key (#850)

* Look up secrets based on names when keyByName is true.
* Allow unit tests for the backend running without mocking in development.
* Add missing unit tests for generic secrets and IAM credentials.
* Add keyByName property to CRD.
* Document keyByName in README.
  • Loading branch information
davesteinberg authored Nov 17, 2021
1 parent ca549f5 commit 20496ab
Show file tree
Hide file tree
Showing 5 changed files with 227 additions and 38 deletions.
36 changes: 29 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -847,20 +847,22 @@ The secrets will persist even if the helm installation is removed, although they

### IBM Cloud Secrets Manager

kubernetes-external-secrets supports fetching secrets from [IBM Cloud Secrets Manager](https://cloud.ibm.com/catalog/services/secrets-manager)
kubernetes-external-secrets supports fetching secrets from [IBM Cloud Secrets Manager](https://cloud.ibm.com/catalog/services/secrets-manager).

create username_password secret by using the [ui, cli or API](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-user-credentials).
The cli option is illustrated below:
Create username_password secret by using the [UI, CLI or API](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-user-credentials).
The CLI option is illustrated below:

```bash
# you need to configure ibm cloud cli with a valid endpoint
# You need to configure ibm cloud cli with a valid endpoint.
# If you're using plug-in version 0.0.8 or later, export the following variable.
export SECRETS_MANAGER_URL=https://{instanceid}.{region}.secrets-manager.appdomain.cloud
# If you're using plug-in version 0.0.6 or earlier, export the following variable.
export IBM_CLOUD_SECRETS_MANAGER_API_URL=https://{instance_ID}.{region}.secrets-manager.appdomain.cloud
ibmcloud secrets-manager secret-create --secret-type username_password \
--metadata '{"collection_type": "application/vnd.ibm.secrets-manager.secret+json", "collection_total": 1}' \
--resources '[{"name": "example-username-password-secret","description": "Extended description for my secret.","username": "user123","password": "cloudy-rainy-coffee-book"}]'
--metadata '{"collection_type": "application/vnd.ibm.secrets-manager.secret+json", "collection_total": 1}' \
--resources '[{"name": "example-username-password-secret","description": "Extended description for my secret.","username": "user123","password": "cloudy-rainy-coffee-book"}]'
```

You will need to set these env vars in the deployment of kubernetes-external-secrets:
Expand All @@ -880,7 +882,27 @@ spec:
# The guid id of the secret
- key: <guid>
name: username
property: username
property: username
secretType: username_password
```


Alternately, you can use `keyByName` on the spec to interpret keys as secret names, instead of IDs.
Using names is slightly less efficient than using IDs, but it makes your ExternalSecrets more robust, as they are not tied to a particular instance of a secret in a particular instance of Secrets Manager:

```yml
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: ibmcloud-secrets-manager-example
spec:
backendType: ibmcloudSecretsManager
keyByName: true
data:
# The name of the secret
- key: my-creds
name: username
property: username
secretType: username_password
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,11 @@ spec:
type: string
description: >-
Used by: gcpSecretsManager
keyByName:
type: boolean
description: >-
Whether to interpret the key as a secret name (if true) or ID (the default).
Used by: ibmcloudSecretsManager
oneOf:
- properties:
backendType:
Expand Down
6 changes: 4 additions & 2 deletions examples/ibmcloud-secrets-manager.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@ metadata:
name: ibmcloud-secrets-manager
spec:
backendType: ibmcloudSecretsManager
# optional: true to key secrets by name instead of by ID
keyByName: true
data:
# The guid id of the secret
- key: guid
- key: my-creds
name: username_password
# Secret Manager secret type: username_password, arbitrary, or iam_credentials
secretType: username_password
25 changes: 20 additions & 5 deletions lib/backends/ibmcloud-secrets-manager-backend.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class IbmCloudSecretsManagerBackend extends KVBackend {

_secretsManagerClient () {
let authenticator
if (process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE && process.env.IBM_CLOUD_SECRETS_MANAGER_API_APIKEY) {
if (process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE) {
authenticator = getAuthenticatorFromEnvironment('IBM_CLOUD_SECRETS_MANAGER_API')
} else {
authenticator = new IamAuthenticator({
Expand All @@ -36,16 +36,31 @@ class IbmCloudSecretsManagerBackend extends KVBackend {
/**
* Get secret_data property value from IBM Cloud Secrets Manager
* @param {string} key - Key used to store secret property value.
* @param {object} specOptions - Options for this external secret, eg role
* @param {object} specOptions.keyByName - Interpret key as secret names if true, as id otherwise
* @param {string} keyOptions.secretType - Type of secret - one of username_password, iam_credentials or arbitrary
* @returns {Promise} Promise object representing secret property value.
*/
async _get ({ key, keyOptions: { secretType } }) {
async _get ({ key, specOptions: { keyByName }, keyOptions: { secretType } }) {
const client = this._secretsManagerClient()
this._logger.info(`fetching secret ${key} from IBM Cloud Secrets Manager ${this._credential.endpoint}`)
let id = key
keyByName = keyByName === true
this._logger.info(`fetching ${secretType} secret ${id}${keyByName ? ' by name' : ''} from IBM Cloud Secrets Manager ${this._credential.endpoint}`)

if (keyByName) {
const secrets = await client.listAllSecrets({ search: key })
const filtered = secrets.result.resources.filter((s) => (s.name === key && s.secret_type === secretType))
if (filtered.length === 1) {
id = filtered[0].id
} else if (filtered.length === 0) {
throw new Error(`No ${secretType} secret named ${key}`)
} else {
throw new Error(`Multiple ${secretType} secrets named ${key}`)
}
}

const secret = await client.getSecret({
secretType: secretType,
id: key
id
})
if (secretType === 'iam_credentials') {
return JSON.stringify(secret.result.resources[0].api_key)
Expand Down
193 changes: 169 additions & 24 deletions lib/backends/ibmcloud-secrets-manager-backend.test.js
Original file line number Diff line number Diff line change
@@ -1,48 +1,193 @@
/* eslint-env mocha */
'use strict'

process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE = 'noauth'
process.env.IBM_CLOUD_SECRETS_MANAGER_API_APIKEY = 'iamkey'

const { expect } = require('chai')
const sinon = require('sinon')

const IbmCloudSecretsManagerBackend = require('./ibmcloud-secrets-manager-backend')

// In the unit test suite, these tests mock calls to IBM Secrets Manager, but mocking can be disabled during development to validate actual operation.
// To diable mocking and enable real calls to an instance of Secrets Manager:
//
// 1. Set the three credential environment variables:
// SECRETS_MANAGER_API_AUTH_TYPE=iam
// SECRETS_MANAGER_API_ENDPOINT=https://{instance-id}.{region}.secrets-manager.appdomain.cloud
// SECRETS_MANAGER_API_APIKEY={API key with Read+ReadSecrets access to the instance}
//
// 2. Add the three secrets described in the data object below to Secrets Manager.
// When you add the IAM secret, be sure that "Reuse IAM credentials until lease expires" is checked.
//
// 3. Set the following three environment variables to the IDs of those secrets:
// IBM_CLOUD_SECRETS_MANAGER_TEST_CREDS_ID
// IBM_CLOUD_SECRETS_MANAGER_TEST_SECRET_ID
// IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_ID
//
// 4. Set the following environment variable to the API key generated as part of the IAM credential:
// IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_APIKEY
//
// Note: In the Secrets Manager UI, you can select "Show snippet" from the secret's overflow menu to show a curl command that will retrieve the value.
// Or you can use the "ibmcloud sm secret" CLI command to handle authentication for you.
//
// You can switch back to mocking simply by unsetting SECRETS_MANAGER_API_AUTH_TYPE.
// This makes it easy to switch back and forth between the two modes when writing new tests.

const endpoint = process.env.IBM_CLOUD_SECRETS_MANAGER_API_ENDPOINT || 'https://fake.secrets-manager.appdomain.cloud'

const data = {
creds: {
id: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_CREDS_ID || 'id1',
name: 'test-creds',
secretType: 'username_password',
username: 'johndoe',
password: 'p@ssw0rd'
},
secret: {
id: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_SECRET_ID || 'id2',
name: 'test-secret',
secretType: 'arbitrary',
payload: 's3cr3t'
},
iam: {
id: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_ID || 'id3',
name: 'test-iam',
secretType: 'iam_credentials',
apiKey: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_APIKEY || 'key'
}
}

describe('IbmCloudSecretsManagerBackend', () => {
const mock = !process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE
let loggerMock
let clientMock
let ibmCloudSecretsManagerBackend

const username = 'fakeUserName'
const password = 'fakeSecretPropertyValue'
const secret = { result: { resources: [{ secret_data: { password: password, username: username } }] } }
const returnsecret = JSON.stringify({ password: password, username: username })
const key = 'username_password'

beforeEach(() => {
loggerMock = sinon.mock()
loggerMock.info = sinon.stub()
clientMock = sinon.mock()
clientMock.getSecret = sinon.stub().returns(secret)
if (mock) {
process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE = 'noauth'
}

loggerMock = {
info: sinon.stub()
}

ibmCloudSecretsManagerBackend = new IbmCloudSecretsManagerBackend({
credential: { endpoint: 'https//sampleendpoint' },
credential: { endpoint },
logger: loggerMock
})
ibmCloudSecretsManagerBackend._secretsManagerClient = sinon.stub().returns(clientMock)
})

afterEach(() => {
if (mock) {
delete process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE
ibmCloudSecretsManagerBackend._secretsManagerClient.restore()
}
})

function mockClient ({ list = [], get = {} }) {
if (mock) {
const client = {
listAllSecrets: sinon.stub().resolves({ result: { resources: list } }),
getSecret: sinon.stub().resolves({ result: { resources: [get] } })
}
sinon.stub(ibmCloudSecretsManagerBackend, '_secretsManagerClient').returns(client)
}
}

describe('_get', () => {
it('returns secret property value', async () => {
const specOptions = {}
const keyOptions = { secretType: 'password' }
const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
key: key,
specOptions,
keyOptions
describe('with default spec options', () => {
it('returns a username_password secret', async () => {
const { id, secretType, username, password } = data.creds
mockClient({ get: { secret_data: { password, username } } })

const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
key: id,
specOptions: {},
keyOptions: { secretType }
})
expect(secretPropertyValue).equals('{"password":"p@ssw0rd","username":"johndoe"}')
})

it('returns an arbitrary secret', async () => {
const { id, secretType, payload } = data.secret
mockClient({ get: { secret_data: { payload } } })

const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
key: id,
specOptions: {},
keyOptions: { secretType }
})
expect(secretPropertyValue).equals('{"payload":"s3cr3t"}')
})
expect(secretPropertyValue).equals(returnsecret)

it('returns an API key from an iam_credentials secret', async () => {
const { id, secretType, apiKey } = data.iam
mockClient({ get: { api_key: apiKey } })

const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
key: id,
specOptions: {},
keyOptions: { secretType }
})
expect(secretPropertyValue).equals(`"${apiKey}"`)
})
})

describe('with key by name enabled', () => {
it('returns a secret that matches the given name and type', async () => {
const { name, secretType, username, password } = data.creds
const list = [
{ name, secret_type: 'arbitrary' },
{ name, secret_type: secretType },
{ name: 'test-creds2', secret_type: secretType }
]
mockClient({ list, get: { secret_data: { password, username } } })

const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
key: name,
specOptions: { keyByName: true },
keyOptions: { secretType }
})
expect(secretPropertyValue).equals('{"password":"p@ssw0rd","username":"johndoe"}')
})

it('throws if there is no secret with the given name and type', async () => {
mockClient({ list: [] })

try {
await ibmCloudSecretsManagerBackend._get({
key: 'test-missing',
specOptions: { keyByName: true },
keyOptions: { secretType: 'username_password' }
})
} catch (error) {
expect(error).to.have.property('message').that.includes('No username_password secret')
return
}
expect.fail('expected to throw an error')
})

// Defensive test: this condition does not appear to be possible currently with a real Secrets Manager instance.
if (mock) {
it('throws if there are multiple secrets with the given name and type', async () => {
const { name, secretType, username, password } = data.creds
const list = [
{ name, secret_type: secretType },
{ name, secret_type: secretType }
]
mockClient({ list, get: { secret_data: { password, username } } })

try {
await ibmCloudSecretsManagerBackend._get({
key: name,
specOptions: { keyByName: true },
keyOptions: { secretType }
})
} catch (error) {
expect(error).to.have.property('message').that.includes('Multiple username_password secrets')
return
}
expect.fail('expected to throw an error')
})
}
})
})
})

0 comments on commit 20496ab

Please sign in to comment.