From b0724f2a46cfc496d6f930a4af58b4962441c42c Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Thu, 22 Feb 2024 15:44:24 -0600 Subject: [PATCH 01/22] Start import docs --- .../content/docs/commands/operator/import.mdx | 81 +++++++++++ website/content/docs/import/gcpsm.mdx | 8 + website/content/docs/import/index.mdx | 137 ++++++++++++++++++ website/data/docs-nav-data.json | 27 ++++ 4 files changed, 253 insertions(+) create mode 100644 website/content/docs/commands/operator/import.mdx create mode 100644 website/content/docs/import/gcpsm.mdx create mode 100644 website/content/docs/import/index.mdx diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx new file mode 100644 index 000000000000..8be6baecef35 --- /dev/null +++ b/website/content/docs/commands/operator/import.mdx @@ -0,0 +1,81 @@ +--- +layout: docs +page_title: operator import - Command +description: >- + The "operator import" command imports secrets from external systems + in to Vault. +--- + +# operator import + +The `operator import` command imports secrets from external systems in to Vault. +Secrets with the same name at the same storage path will be overwritten upon import. + +This is potentially a long-running process as the importer can be configured to read +as many secrets as the import plan specified and from multiple external systems. + +## Examples + +Execute an import config file named import.hcl to generate an import plan: + +```shell-session +$ vault operator import -config import.hcl plan + +----------- +Import plan +----------- +The following namespaces are missing: + * ns-1/ + +The following mounts are missing: + * ns1/mount-1 + +Secrets to be imported from the source "my-dest-1": + * secret1 + * secret2 +... +``` + +## Configuration + +The `operator import` command uses a dedicated configuration file to specify the source, +destination, and mapping rules. + +```hcl +source_gcp { + name = "my-gcp-source-1" + credentials = "@/path/to/service-account-key.json" +} + +destination_vault { + name = "my-dest-1" + address = "http://127.0.0.1:8200/" + token = "root" + namespace = "ns-1" + mount = "mount-1" +} + +mapping_regex { + name = "my-map-1" + source = "my-gcp-source-1" + destination = "my-dest-1" + priority = 1 + expression = "^database/.*$" +} +``` + +## Usage + +The following flags are available for the `operator import` command. + +- `-config` `(string: )` - Path to the import configuration HCL file. + +- `auto-approve` `(bool: )` - Automatically skips the user-input requirement of "yes" when + running the "apply" command. + +- `auto-create` `(bool: )` - Automatically creates any missing namespaces and mounts when + "running the "apply" command. + +- `-log-level` ((#\_log_level)) `(string: "info")` - Log verbosity level. Supported values (in + order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`. This can + also be specified via the `VAULT_LOG_LEVEL` environment variable. diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx new file mode 100644 index 000000000000..18bc7a91f9ad --- /dev/null +++ b/website/content/docs/import/gcpsm.mdx @@ -0,0 +1,8 @@ +--- +layout: docs +page_title: Google Cloud Platform Secret Manager - Secrets Import Source +description: The Google Cloud Platform Secret Manager source imports secrets from GCP to Vault. +--- + +# Google Cloud Platform Secret Manager + diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx new file mode 100644 index 000000000000..e99ec5fe3c7d --- /dev/null +++ b/website/content/docs/import/index.mdx @@ -0,0 +1,137 @@ +--- +layout: docs +page_title: Secrets import +description: Secrets import allows you to safely onboard secrets from external sources into Vault KV for management. +--- + + +# Secrets sync + + + +@include 'alerts/beta.mdx' + +Having collections of secrets across various external systems can present several challenges to an organization, such as +the increased operational overhead required to safely maintain multiple secrets management tools. While Vault can offer +a single source of truth (SSOT) solution to managing these secrets for increased security and reduced overhead, the effort +required to migrate all your secrets from multiple external sources into Vault can be steep. To aid in this onboarding +process, secrets import provides a mechanism to automate this process. + +A secret that is imported from an external system is stored in Vault's KVv2 Secrets Engine. The import process is defined +an import plan, which is an HCL file. Using HCL enables users to declaratively configure which of their secrets will be read +from an external source and where exactly they will be stored in Vault. + +## Sources + +Secrets can be imported from various external systems, called sources. The supported destinations are: +* [GCP Secret Manager](/vault/docs/import/gcpsm) + +A source is defined by an HCL block in the format of `source_`, for example `source_gcp`. The source is +provided with the credentials it needs to access the external system's secrets to be able to import them into Vault. See +the [HCL syntax section](#hcl-syntax) for more information about this block. + +## Vault destinations + +Secrets that get imported are stored into a Vault KVv2 Secrets Engine mount. The HCL block `destination_vault` is provided +with a KVv2 mount path and/or namespace to specify exactly where the secret gets stored. See the +[HCL syntax section](#hcl-syntax) for more information about this block. + +## Mappings + +Mappings blocks are what glue together a source block to a destination block. The supported mapping types are: +* [mapping_passthrough](/vault/docs/import/mappings#passthrough) +* [mapping_metadata](/vault/docs/import/mappings#metadata) +* [mapping_regex](/vault/docs/import/mappings#regex) + +A mapping is where filtering logic is defined to control which secrets in the external system will be imported and which will +be ignored. Mappings are defined by an HCL block in the format of `mapping_`, for example `mapping_passthrough`. +See the [HCL syntax section](#hcl-syntax) for more information about this block. + +## HCL syntax + +There are three main types of HCL blocks used to configure a secrets import plan. They are source blocks, the Vault destination +block, and mapping blocks. + +### Source blocks + +#### Argument reference + +All types of source blocks, no matter the external system, have one shared argument: + +* `name` - (Required) The name of the source block used to internally reference this source inside of a mapping. + +Refer to a specific import source's documentation for details on how to define all remaining arguments of a source block for +that external system type. + +#### Example + +A `source_` block may look like the following, using GCP as the example: + +```hcl +source_gcp { + name = "my-gcp-source-1" + credentials = "@/path/to/service-account-key.json" +} +``` + +### Vault destination blocks + +#### Argument reference + +The following arguments are supported: + +* `name` - (Required) The name of the Vault destination block used to internally reference this destination inside of a mapping. + +* `address` - (Optional) The + +* `token` - (Optional) The + +* `namespace` - (Optional) The + +* `mount` - (Optional) The + +#### Example + +An example `destination_vault` block may look like the following: + +```hcl +destination_vault { + name = "my-dest-1" + address = "http://127.0.0.1:8200/" + token = "root" + namespace = "ns-1" + mount = "mount-1" +} +``` + +### Mapping blocks + +#### Argument reference + +All types of mapping blocks, no matter the type, have four shared arguments: + +* `name` - (Required) The + +* `source` - (Required) The + +* `destination` - (Required) The + +* `priority` - (Required) The + +#### Example + +A `mapping_` block may look like the following, using regex as the example: + +```hcl +mapping_regex { + name = "my-map-1" + source = "my-gcp-source-1" + destination = "my-dest-1" + priority = 1 + expression = "^database/.*$" +} +``` + +## API + +Please see the [secrets import API](/vault/api-docs/system/secrets-import) for more details. diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 8c36ee89aaa6..37cce47003f6 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -751,6 +751,10 @@ "title": "generate-root", "path": "commands/operator/generate-root" }, + { + "title": "import", + "path": "commands/operator/import" + }, { "title": "init", "path": "commands/operator/init" @@ -1603,6 +1607,29 @@ } ] }, + { + "title": "Secrets Import", + "badge": { + "text": "ENTERPRISE", + "type": "outlined", + "color": "neutral" + }, + "routes": [ + { + "title": "Overview", + "path": "import", + "badge": { + "text": "BETA", + "type": "outlined", + "color": "highlight" + } + }, + { + "title": "GCP Secret Manager", + "path": "import/gcpsm" + } + ] + }, { "title": "Auth Methods", "routes": [ From d7933576419c051831fddffb3455c4834ae91d61 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Thu, 22 Feb 2024 16:01:47 -0600 Subject: [PATCH 02/22] Use hideClipboard block on output --- .../content/docs/commands/operator/import.mdx | 35 +++++++++++-------- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx index 8be6baecef35..c9dd92b76cf4 100644 --- a/website/content/docs/commands/operator/import.mdx +++ b/website/content/docs/commands/operator/import.mdx @@ -20,21 +20,27 @@ Execute an import config file named import.hcl to generate an import plan: ```shell-session $ vault operator import -config import.hcl plan +``` ------------ -Import plan ------------ -The following namespaces are missing: - * ns-1/ +Output: -The following mounts are missing: - * ns1/mount-1 + -Secrets to be imported from the source "my-dest-1": - * secret1 - * secret2 -... -``` + ----------- + Import plan + ----------- + The following namespaces are missing: + * ns-1/ + + The following mounts are missing: + * ns1/mount-1 + + Secrets to be imported from the source "my-dest-1": + * secret1 + * secret2 + ... + + ## Configuration @@ -55,12 +61,11 @@ destination_vault { mount = "mount-1" } -mapping_regex { +mapping_passthrough { name = "my-map-1" - source = "my-gcp-source-1" + source = "my-gcp-1" destination = "my-dest-1" priority = 1 - expression = "^database/.*$" } ``` From eaf2a9ee2d2126050f5fb87880c25a6e44036a38 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Thu, 22 Feb 2024 16:57:21 -0600 Subject: [PATCH 03/22] Fix missed line --- website/content/docs/import/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index e99ec5fe3c7d..3a9821bdaa05 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -5,7 +5,7 @@ description: Secrets import allows you to safely onboard secrets from external s --- -# Secrets sync +# Secrets import From 51c92680dcff2a34a769ab5e2033574ad75298f1 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Mon, 26 Feb 2024 05:13:13 -0600 Subject: [PATCH 04/22] Reorganize mappings and source docs, fill out more of the info --- .../content/docs/commands/operator/import.mdx | 15 ++- .../content/docs/commands/operator/index.mdx | 1 + website/content/docs/import/gcpsm.mdx | 38 +++++- website/content/docs/import/index.mdx | 123 ++++++++---------- website/content/docs/import/mappings.mdx | 119 +++++++++++++++++ website/content/docs/sync/gcpsm.mdx | 12 +- website/data/docs-nav-data.json | 4 + 7 files changed, 227 insertions(+), 85 deletions(-) create mode 100644 website/content/docs/import/mappings.mdx diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx index c9dd92b76cf4..5ec71ba81463 100644 --- a/website/content/docs/commands/operator/import.mdx +++ b/website/content/docs/commands/operator/import.mdx @@ -11,8 +11,9 @@ description: >- The `operator import` command imports secrets from external systems in to Vault. Secrets with the same name at the same storage path will be overwritten upon import. -This is potentially a long-running process as the importer can be configured to read -as many secrets as the import plan specified and from multiple external systems. +~> **NOTE:** This has the potential to be a long-running process. Import plans can be +written to read from as many sources as desired and each source may migrate as many +secrets as requested based on filters. ## Examples @@ -33,12 +34,11 @@ Output: * ns-1/ The following mounts are missing: - * ns1/mount-1 + * ns-1/mount-1 Secrets to be imported from the source "my-dest-1": - * secret1 - * secret2 - ... + * secret-1 + * secret-2 @@ -73,7 +73,8 @@ mapping_passthrough { The following flags are available for the `operator import` command. -- `-config` `(string: )` - Path to the import configuration HCL file. +- `-config` `(string: "")` - Path to the import configuration HCL file. If unspecified, +a file named import.hcl will be automatically looked for. - `auto-approve` `(bool: )` - Automatically skips the user-input requirement of "yes" when running the "apply" command. diff --git a/website/content/docs/commands/operator/index.mdx b/website/content/docs/commands/operator/index.mdx index e268da6ef4d3..f5e29c04bbbc 100644 --- a/website/content/docs/commands/operator/index.mdx +++ b/website/content/docs/commands/operator/index.mdx @@ -54,6 +54,7 @@ Usage: vault operator [options] [args] Subcommands: generate-root Generates a new root token + import Import secrets from external systems into Vault init Initializes a server key-status Provides information about the active encryption key rekey Generates new unseal keys diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index 18bc7a91f9ad..25dcabfa7efe 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -4,5 +4,41 @@ page_title: Google Cloud Platform Secret Manager - Secrets Import Source description: The Google Cloud Platform Secret Manager source imports secrets from GCP to Vault. --- -# Google Cloud Platform Secret Manager +# GCP import source +The GCP source enables users to import secret data from GCP Secret Manager into their Vault instance. To be able +to import secret data from here into your Vault destination, the following permissions are required: + +```shell-session +"secretmanager.secrets.list", +"secretmanager.versions.access", +``` + +## Argument reference + +* `name` - (Required) The name of the source block used to internally reference this source inside of a mapping. + +* `credentials` - (Optional) The path to the service account key credentials file for the service account with the ncessary +permissions. If `credentials` is set, then `vault_mount_path` and `vault_role_name` must be unset. + +* `vault_mount_path` - (Optional) The Vault mount path to a preconfigured GCP secrets engine that is used to generate a +dynamic credential for the importer to use. If `vault_mount_path` or `vault_role_name` are set, then `credentials` must be unset. + +* `vault_role_name` - (Optional) The Vault role's name that exists in the preconfigured GCP secrets engine mount that is +used to generate a dynamic credential for the importer to use. If `vault_role_name` or `vault_mount_path` are set, then +`credentials` must be unset. + +Refer to a [specific import source's documentation](#sources) for details on how to define all remaining arguments of a source +block for that external system type. + +## Example + +In this example, a GCP source is defined and configured to use + +```hcl +source_gcp { + name = "my-gcp-source-1" + secrets_engine_mount = "gcp" + secrets_engine_role = "my-gcp-role-1" +} +``` diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index 3a9821bdaa05..25191ac36769 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -12,69 +12,25 @@ description: Secrets import allows you to safely onboard secrets from external s @include 'alerts/beta.mdx' Having collections of secrets across various external systems can present several challenges to an organization, such as -the increased operational overhead required to safely maintain multiple secrets management tools. While Vault can offer -a single source of truth (SSOT) solution to managing these secrets for increased security and reduced overhead, the effort -required to migrate all your secrets from multiple external sources into Vault can be steep. To aid in this onboarding -process, secrets import provides a mechanism to automate this process. +increased operational overhead, a greater risk of secrets sprawl, and more. Vault can offer a single source of truth (SSOT) +solution to managing these secrets, providing increased security and reducing that overhead, yet the effort required +to migrate all your secrets from multiple external sources into Vault can be steep. To aid in this onboarding process, +secrets import provides a mechanism to automate it. A secret that is imported from an external system is stored in Vault's KVv2 Secrets Engine. The import process is defined an import plan, which is an HCL file. Using HCL enables users to declaratively configure which of their secrets will be read from an external source and where exactly they will be stored in Vault. -## Sources - -Secrets can be imported from various external systems, called sources. The supported destinations are: -* [GCP Secret Manager](/vault/docs/import/gcpsm) - -A source is defined by an HCL block in the format of `source_`, for example `source_gcp`. The source is -provided with the credentials it needs to access the external system's secrets to be able to import them into Vault. See -the [HCL syntax section](#hcl-syntax) for more information about this block. - -## Vault destinations - -Secrets that get imported are stored into a Vault KVv2 Secrets Engine mount. The HCL block `destination_vault` is provided -with a KVv2 mount path and/or namespace to specify exactly where the secret gets stored. See the -[HCL syntax section](#hcl-syntax) for more information about this block. - -## Mappings - -Mappings blocks are what glue together a source block to a destination block. The supported mapping types are: -* [mapping_passthrough](/vault/docs/import/mappings#passthrough) -* [mapping_metadata](/vault/docs/import/mappings#metadata) -* [mapping_regex](/vault/docs/import/mappings#regex) - -A mapping is where filtering logic is defined to control which secrets in the external system will be imported and which will -be ignored. Mappings are defined by an HCL block in the format of `mapping_`, for example `mapping_passthrough`. -See the [HCL syntax section](#hcl-syntax) for more information about this block. - -## HCL syntax - There are three main types of HCL blocks used to configure a secrets import plan. They are source blocks, the Vault destination block, and mapping blocks. -### Source blocks - -#### Argument reference - -All types of source blocks, no matter the external system, have one shared argument: - -* `name` - (Required) The name of the source block used to internally reference this source inside of a mapping. - -Refer to a specific import source's documentation for details on how to define all remaining arguments of a source block for -that external system type. - -#### Example - -A `source_` block may look like the following, using GCP as the example: +## Vault destinations -```hcl -source_gcp { - name = "my-gcp-source-1" - credentials = "@/path/to/service-account-key.json" -} -``` +Secrets that get imported are stored into a Vault KVv2 Secrets Engine mount. The destination will accept the mount path of +the desired KVv2 mount and optionally a namespace. This specifies the exact location to store the secrets in. A destination +is defined by an HCL block in the format of `destination_vault`. -### Vault destination blocks +### HCL syntax #### Argument reference @@ -82,13 +38,13 @@ The following arguments are supported: * `name` - (Required) The name of the Vault destination block used to internally reference this destination inside of a mapping. -* `address` - (Optional) The +* `address` - (Optional) The address of the Vault server with the KVv2 Secrets Engine for which the secrets will be imported into. -* `token` - (Optional) The +* `token` - (Optional) A token for authentication to the Vault server located at the specified address. -* `namespace` - (Optional) The +* `namespace` - (Optional) The name of the Vault namespace containing the specified KVv2 mount. -* `mount` - (Optional) The +* `mount` - (Optional) The name of the KVv2 mount in Vault for which the secrets will be imported into. #### Example @@ -104,23 +60,56 @@ destination_vault { } ``` -### Mapping blocks +## Sources -#### Argument reference +Secrets can be imported from various external systems, called sources. The supported destinations are: +* [GCP Secret Manager](/vault/docs/import/gcpsm) + +A source requires the credentials necessary to read the external system's secrets and their values so that it may import +them into Vault. Either credentials to the system can be directly provided, or you can leverage Vault's capabilities to +geneerate a dynamic secret for the importer to use if you have the corresponding secrets engine configured. A source is +defined by an HCL block in the format of `source_`. For example, `source_gcp` defines a source block +for GCP Secret Manager. + +### HCL syntax + +Refer to a [specific source's documentation](#sources) for details on how to define all remaining arguments of a source +block for that external system type. + +#### Example + +A `source_` block may look like the following, using GCP as an example: + +```hcl +source_gcp { + name = "my-gcp-source-1" + credentials = "@/path/to/service-account-key.json" +} +``` -All types of mapping blocks, no matter the type, have four shared arguments: +## Mappings -* `name` - (Required) The +Mappings blocks are what glue together a source block to a destination block. The supported mapping types are: +* [mapping_passthrough](/vault/docs/import/mappings#passthrough) +* [mapping_metadata](/vault/docs/import/mappings#metadata) +* [mapping_regex](/vault/docs/import/mappings#regex) -* `source` - (Required) The +A mapping is where filtering logic is defined to control which secrets in the external system will be imported and which will +be ignored. The definition of the rule depends on the type of mapping block that is being configured. Mapping blocks are +applied in order of their given priority. -* `destination` - (Required) The +Once a secret being imported matches a single mapping's rule, it cannot match to another mapping again. A mapping is defined +by an HCL block in the format of `mapping_`. For example, `mapping_regex` defines a mapping block to pass +all secrets through whose secret name passes the given regular expression. -* `priority` - (Required) The +### HCL syntax + +See the [mapping documentation](/vault/docs/import/mappings) to see more info regarding specific mapping types and +their arguments. #### Example -A `mapping_` block may look like the following, using regex as the example: +A `mapping_` block may look like the following, using regex as an example: ```hcl mapping_regex { @@ -131,7 +120,3 @@ mapping_regex { expression = "^database/.*$" } ``` - -## API - -Please see the [secrets import API](/vault/api-docs/system/secrets-import) for more details. diff --git a/website/content/docs/import/mappings.mdx b/website/content/docs/import/mappings.mdx new file mode 100644 index 000000000000..f101a0ce931e --- /dev/null +++ b/website/content/docs/import/mappings.mdx @@ -0,0 +1,119 @@ +--- +layout: docs +page_title: Secrets import mappings +description: Mappings lets users apply various filtering methods to secrets being imported in to Vault. +--- + +# Import mappings + +There are multiple types of mapping blocks that can be defined in the import plan. Each type enables a +different mechanism for filtering out secrets from an external system during the execution of an import. + +## Argument reference + +All types of mapping blocks, no matter the type, have four shared arguments: + +* `name` - (Required) The name of the mapping block used internally to track secrets imported from a source to a destination. + +* `source` - (Required) The name of the source block for which this mapping's filter logic will be applied to. + +* `destination` - (Required) The name of the Vault destination block for which this mapping will place the matching secrets into. + +* `priority` - (Required) Controls the order in which multiple mapping blocks will be applied to the secrets being imported. The +lower the value specified, the higher the priority will be during filtering. + +## Passthrough + +Passthrough mapping blocks `mapping_passthrough` allow all secrets through from the specified source to the +specified destination. For example, one use case is using it as a base-case for imported secrets. By assigning +it the lowest priority in the import plan, all other mapping blocks will be applied first. Secrets that fail +to match any of the previous mappings will fall through to the passthrough block and be collected in a single +KVv2 bucket. + +### Additional arguments + +There are no extra arguments to specify in a `mapping_passthrough` block. + +### Example + +In this example, every single secret that `my-gcp-source-1` is reads from GCP Secret Manager will be imported +to the KVv2 Secrets Engine mount defined in `my-dest-1`. + +```hcl +mapping_passthrough { + name = "my-map-1" + source = "my-gcp-source-1" + destination = "my-dest-1" + priority = 1 +} +``` + +## Metadata + +Metadata mapping blocks `mapping_metadata` allow secrets through from the specified source to the specified +destination if they contain matching metadata key-value pairs. Metadata is not supported in all external secret +manageement systems, and ones that do may use different terminology for metadata. For example, AWS allows tags +on secrets while [GCP](/vault/docs/import/gcpsm) allows labels. + +### Additional arguments + +* `tags` - (Required) A set of key-value pairs to match on secrets from the external system. All of the specified +keys must be found on a secret and all of the values must be exact matches. A key with an empty value, i.e. `""`, +must also match an empty value from the external system. + +### Example + +In this example, there are two metadata mappings. The first, `my-map-1`, has a priority of 1. This will only import +the secrets into the destination `my-dest-1` that contain both tag keys `database` and `importable`. Each of these +keys' values must also match to `users` and `true` respectively to be imported. The second, `my-map-2`, has a +priority of 2. Even though all the secrets in the first map would also qualify for the second map's filtering rule, +those secrets will only be imported into `my-dest-1` because of the lower priority set in the second mapping. All +remaining secrets that have the tag `importable` with a value of `true` will be imported into `my-dest-2`. + +```hcl +mapping_metadata { + name = "my-map-1" + source = "my-gcp-source-1" + destination = "my-dest-1" + priority = 1 + tags = { + "database" = "users" + "importable" = "true" + } +} + +mapping_metadata { + name = "my-map-2" + source = "my-gcp-source-1" + destination = "my-dest-2" + priority = 2 + tags = { + "importable" = "true" + } +} +``` + +## Regex + +Regex mapping blocks `mapping_regex` allow secrets through from the specified source to the specified +destination if their secret name passes a regular expression check. + +### Additional arguments + +* `expression` - (Required) A regular expression to match secrets names from the external system. All of +the specified keys must be found on a secret and all of the values must be exact matches. A key with an empty +value, e.g. `""`, must also match an empty value from the external system. + +### Example + +In this example, any secret in the GCP source whose name begins with `database/` will be imported into Vault. + +```hcl +mapping_regex { + name = "my-map-1" + source = "my-gcp-source-1" + destination = "my-dest-1" + priority = 1 + expression = "^database/.*$" +} +``` diff --git a/website/content/docs/sync/gcpsm.mdx b/website/content/docs/sync/gcpsm.mdx index 589bbcf8bedf..c5782a22f16f 100644 --- a/website/content/docs/sync/gcpsm.mdx +++ b/website/content/docs/sync/gcpsm.mdx @@ -124,14 +124,9 @@ The credentials given to Vault must have the following permissions to synchroniz ```shell-session secretmanager.secrets.create secretmanager.secrets.delete -secretmanager.secrets.get -secretmanager.secrets.list secretmanager.secrets.update -secretmanager.versions.access secretmanager.versions.add -secretmanager.versions.destroy -secretmanager.versions.get -secretmanager.versions.list +secretmanager.versions.destroy ``` ## Provision service account @@ -160,8 +155,9 @@ to provision a GCP Service Account with the appropriate policies. // GOOGLE_REGION (optional) // GOOGLE_PROJECT // GOOGLE_CREDENTIALS (The path to a service account key file with the - // "Service Account Admin", "Service Account Key - // Admin", and "Security Admin" roles attached) + // "Service Account Admin", "Service Account Key Admin", + // "Secret Manager Admin", and "Project IAM Admin" roles + // attached) } data "google_client_config" "config" {} diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 37cce47003f6..dc2a38b8af90 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -1627,6 +1627,10 @@ { "title": "GCP Secret Manager", "path": "import/gcpsm" + }, + { + "title": "Mappings", + "path": "import/mappings" } ] }, From ce6f30c93d499ac6e349150ecaeaa5026c0ec2fa Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Wed, 28 Feb 2024 12:40:47 -0600 Subject: [PATCH 05/22] Remove copied section --- website/content/docs/import/gcpsm.mdx | 3 --- 1 file changed, 3 deletions(-) diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index 25dcabfa7efe..6686ab86998f 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -28,9 +28,6 @@ dynamic credential for the importer to use. If `vault_mount_path` or `vault_role used to generate a dynamic credential for the importer to use. If `vault_role_name` or `vault_mount_path` are set, then `credentials` must be unset. -Refer to a [specific import source's documentation](#sources) for details on how to define all remaining arguments of a source -block for that external system type. - ## Example In this example, a GCP source is defined and configured to use From 22c9a1e5c5bac62763ac674a1d2e879cedb27d02 Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 02:39:37 -0500 Subject: [PATCH 06/22] Apply suggestions from code review Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- .../content/docs/commands/operator/import.mdx | 24 ++++++++++--------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx index 5ec71ba81463..70d8c4a44c8c 100644 --- a/website/content/docs/commands/operator/import.mdx +++ b/website/content/docs/commands/operator/import.mdx @@ -11,13 +11,18 @@ description: >- The `operator import` command imports secrets from external systems in to Vault. Secrets with the same name at the same storage path will be overwritten upon import. -~> **NOTE:** This has the potential to be a long-running process. Import plans can be -written to read from as many sources as desired and each source may migrate as many -secrets as requested based on filters. + + +You can write import plans that read from as many sources as you want. The +amount of data migrated for each source depends on the filters applied and the +dataset available. Be mindful of the time needed to read from each source, +apply any filters, and store the data in Vault. + + ## Examples -Execute an import config file named import.hcl to generate an import plan: +Read the config file `import.hcl` to generate a new import plan: ```shell-session $ vault operator import -config import.hcl plan @@ -71,17 +76,14 @@ mapping_passthrough { ## Usage -The following flags are available for the `operator import` command. +The `operator import` command accepts the following flags: -- `-config` `(string: "")` - Path to the import configuration HCL file. If unspecified, -a file named import.hcl will be automatically looked for. +- `-config` `(string: "")` - Path to the import configuration HCL file. The default path is `import.hcl`. -- `auto-approve` `(bool: )` - Automatically skips the user-input requirement of "yes" when - running the "apply" command. +- `auto-approve` `(bool: )` - Automatically response "yes" to all user-input prompts for the `apply` command. - `auto-create` `(bool: )` - Automatically creates any missing namespaces and mounts when "running the "apply" command. - `-log-level` ((#\_log_level)) `(string: "info")` - Log verbosity level. Supported values (in - order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`. This can - also be specified via the `VAULT_LOG_LEVEL` environment variable. + order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`. You can also set log-level with the `VAULT_LOG_LEVEL` environment variable. From 194a6215acf7bf2077992844c8c42460f3845701 Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 03:16:37 -0500 Subject: [PATCH 07/22] Apply suggestions from code review Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- website/content/docs/import/index.mdx | 60 ++++++++++++++------------- 1 file changed, 31 insertions(+), 29 deletions(-) diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index 25191ac36769..e95a69d66d41 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -11,30 +11,44 @@ description: Secrets import allows you to safely onboard secrets from external s @include 'alerts/beta.mdx' -Having collections of secrets across various external systems can present several challenges to an organization, such as -increased operational overhead, a greater risk of secrets sprawl, and more. Vault can offer a single source of truth (SSOT) -solution to managing these secrets, providing increased security and reducing that overhead, yet the effort required -to migrate all your secrets from multiple external sources into Vault can be steep. To aid in this onboarding process, -secrets import provides a mechanism to automate it. +Distributing sensitive information across multiple external systems creates +several challenges, including: -A secret that is imported from an external system is stored in Vault's KVv2 Secrets Engine. The import process is defined -an import plan, which is an HCL file. Using HCL enables users to declaratively configure which of their secrets will be read -from an external source and where exactly they will be stored in Vault. +- increased operational overhead. +- increased exposure risk from data sprawl. +- increased risk of outdated and out-of-sync information. -There are three main types of HCL blocks used to configure a secrets import plan. They are source blocks, the Vault destination -block, and mapping blocks. +Using Vault as a single source of truth (SSOT) for sensitive data increases +security and reduces management overhead, but migrating data from multiple and +varied sources can be complex and costly. -## Vault destinations +The secrets import process helps you automate and streamline your data +migration with codified import plans as HCL files. Import plans tell Vault +which data to read from a given source and where to store the data in the KVv2 +secrets engine: -Secrets that get imported are stored into a Vault KVv2 Secrets Engine mount. The destination will accept the mount path of -the desired KVv2 mount and optionally a namespace. This specifies the exact location to store the secrets in. A destination -is defined by an HCL block in the format of `destination_vault`. +- the `source` block provides credentials for connecting to the source system. +- the `destination` block defines target mounts in KVv2. +- the `mapping` block define how Vault should transform the imported data before + writing the information to KVv2. + +## Data migration destinations + +Vault stores imported secrets in a Vault KVv2 secrets engine mount. Destination +blocks start with `destination_vault` and define the desired KVv2 mount path and +an optional namespace. The combination of a path and namespace represent the exact location in your +Vault instance you want the information stored. ### HCL syntax -#### Argument reference -The following arguments are supported: +```hcl +destination_vault { + name = "my-dest-1" + namespace = "ns-1" + mount = "mount-1" +} +``` * `name` - (Required) The name of the Vault destination block used to internally reference this destination inside of a mapping. @@ -46,21 +60,9 @@ The following arguments are supported: * `mount` - (Optional) The name of the KVv2 mount in Vault for which the secrets will be imported into. -#### Example - -An example `destination_vault` block may look like the following: -```hcl -destination_vault { - name = "my-dest-1" - address = "http://127.0.0.1:8200/" - token = "root" - namespace = "ns-1" - mount = "mount-1" -} -``` -## Sources +## Data migration sources Secrets can be imported from various external systems, called sources. The supported destinations are: * [GCP Secret Manager](/vault/docs/import/gcpsm) From da4323ca51c7c2dbcf390fc51a1c6deb4f09b99c Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 03:45:41 -0500 Subject: [PATCH 08/22] Apply suggestions from code review Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- website/content/docs/commands/operator/import.mdx | 2 +- website/content/docs/import/gcpsm.mdx | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx index 70d8c4a44c8c..9a44f0dd4846 100644 --- a/website/content/docs/commands/operator/import.mdx +++ b/website/content/docs/commands/operator/import.mdx @@ -83,7 +83,7 @@ The `operator import` command accepts the following flags: - `auto-approve` `(bool: )` - Automatically response "yes" to all user-input prompts for the `apply` command. - `auto-create` `(bool: )` - Automatically creates any missing namespaces and mounts when - "running the "apply" command. + running the `apply` command. - `-log-level` ((#\_log_level)) `(string: "info")` - Log verbosity level. Supported values (in order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`. You can also set log-level with the `VAULT_LOG_LEVEL` environment variable. diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index 6686ab86998f..0f9831c575fb 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -6,15 +6,15 @@ description: The Google Cloud Platform Secret Manager source imports secrets fro # GCP import source -The GCP source enables users to import secret data from GCP Secret Manager into their Vault instance. To be able -to import secret data from here into your Vault destination, the following permissions are required: + +Use the GCP source to import secret data from the GCP secret manager into your Vault instance. To use a GCP source, you must grant GCP the following permissions: ```shell-session "secretmanager.secrets.list", "secretmanager.versions.access", ``` -## Argument reference +## GCP source parameters * `name` - (Required) The name of the source block used to internally reference this source inside of a mapping. @@ -30,7 +30,7 @@ used to generate a dynamic credential for the importer to use. If `vault_role_na ## Example -In this example, a GCP source is defined and configured to use +Define and configure the `my-gcp-source-1` GCP source: ```hcl source_gcp { From 34a5f71f46a91e9c051f168746806ccf42f50404 Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 04:54:06 -0500 Subject: [PATCH 09/22] Apply suggestions from code review Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- website/content/docs/import/index.mdx | 101 ++++++++++++++++---------- 1 file changed, 64 insertions(+), 37 deletions(-) diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index e95a69d66d41..52385c051146 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -50,37 +50,35 @@ destination_vault { } ``` -* `name` - (Required) The name of the Vault destination block used to internally reference this destination inside of a mapping. +- `name (string: )` - A unique name for the destination block that can + be referenced in subsequent mapping blocks. -* `address` - (Optional) The address of the Vault server with the KVv2 Secrets Engine for which the secrets will be imported into. +- `address (string)` - Optional network address of the Vault server with the + KVv2 secrets engine enabled. * `token` - (Optional) A token for authentication to the Vault server located at the specified address. -* `namespace` - (Optional) The name of the Vault namespace containing the specified KVv2 mount. +- `namespace (string)` - Optional namespace path containing the specified KVv2 + mount. By default, Vault looks for the KVv2 mount under the root namespace. -* `mount` - (Optional) The name of the KVv2 mount in Vault for which the secrets will be imported into. +- `mount (string)` - Optional mount path for the target KVv2 instance. ## Data migration sources -Secrets can be imported from various external systems, called sources. The supported destinations are: -* [GCP Secret Manager](/vault/docs/import/gcpsm) +Vault can import secrets from the following sources: +- [GCP Secret Manager](/vault/docs/import/gcpsm) -A source requires the credentials necessary to read the external system's secrets and their values so that it may import -them into Vault. Either credentials to the system can be directly provided, or you can leverage Vault's capabilities to -geneerate a dynamic secret for the importer to use if you have the corresponding secrets engine configured. A source is -defined by an HCL block in the format of `source_`. For example, `source_gcp` defines a source block -for GCP Secret Manager. +To pull data from a source during import, Vault needs read credentials for the +external system. You can provide credentials directly as part of the import +plan, or use Vault to generate dynamic credentials for the importer if you already +have dynamic credentials configured for the corresponding secrets engine. ### HCL syntax -Refer to a [specific source's documentation](#sources) for details on how to define all remaining arguments of a source -block for that external system type. - -#### Example - -A `source_` block may look like the following, using GCP as an example: +Source blocks start with `source_` and include any connection +information required by the target system or engine. For example: ```hcl source_gcp { @@ -89,36 +87,65 @@ source_gcp { } ``` -## Mappings +- `name (string: )` - A unique name for the source block that can be + referenced in subsequent mapping blocks. -Mappings blocks are what glue together a source block to a destination block. The supported mapping types are: -* [mapping_passthrough](/vault/docs/import/mappings#passthrough) -* [mapping_metadata](/vault/docs/import/mappings#metadata) -* [mapping_regex](/vault/docs/import/mappings#regex) +- `credentials (string: )` - Path to a credential file or token with + read permissions for the target system. -A mapping is where filtering logic is defined to control which secrets in the external system will be imported and which will -be ignored. The definition of the rule depends on the type of mapping block that is being configured. Mapping blocks are -applied in order of their given priority. +Depending on the source system, additional information may be required. Refer to +the connection documentation for your source system to determine the full set of +required fields for that system type. -Once a secret being imported matches a single mapping's rule, it cannot match to another mapping again. A mapping is defined -by an HCL block in the format of `mapping_`. For example, `mapping_regex` defines a mapping block to pass -all secrets through whose secret name passes the given regular expression. -### HCL syntax -See the [mapping documentation](/vault/docs/import/mappings) to see more info regarding specific mapping types and -their arguments. +## Data migration mappings -#### Example +Mappings glue the source and destination together and filter the migrated data +to control what is imported and what is ignored. Vault currently supports the +following mapping methods: -A `mapping_` block may look like the following, using regex as an example: +- [mapping_passthrough](/vault/docs/import/mappings#passthrough) +- [mapping_metadata](/vault/docs/import/mappings#metadata) +- [mapping_regex](/vault/docs/import/mappings#regex) + +### HCL syntax + +Mapping blocks start with `mapping_` and include a source name, +destination name, and any transformations or filters that apply. For example: ```hcl mapping_regex { - name = "my-map-1" - source = "my-gcp-source-1" + name = "my-map-1" + source = "my-gcp-source-1" destination = "my-dest-1" - priority = 1 - expression = "^database/.*$" + priority = 1 + expression = "^database/.*$" } ``` + +- `name (string: )` - A unique name for the mapping block. + +- `source (string: )` - The name of a previously defined source block + **from** which the data should be read. + +- `destination (string: )` - The name of a previously defined + destination block **to** which the data should be written. + +- `priority (integer: )` - The order in which Vault should apply the + mapping block during the import process. The lower the number, the higher the + priority. For example, a mapping with priority 1 executes before a mapping + with priority 2. + +Depending on the filter type, additional fields may be required/possible. Refer +to the filter documentation for the different mappings for a list of +mapping-specific fields. + + + + Vault applies mapping definitions in priority order and a given secret only + matches to the first mapping that applies. Once Vault imports a secret with a + particular mapping, subsequent reads from the same source will ignore that + secret. + + From 302831ce4fcb0c48b121223ce4b8e3bf6e5cc1d6 Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 04:57:56 -0500 Subject: [PATCH 10/22] Apply suggestions from code review Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- website/content/docs/import/mappings.mdx | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/website/content/docs/import/mappings.mdx b/website/content/docs/import/mappings.mdx index f101a0ce931e..4f24f53db78f 100644 --- a/website/content/docs/import/mappings.mdx +++ b/website/content/docs/import/mappings.mdx @@ -6,8 +6,9 @@ description: Mappings lets users apply various filtering methods to secrets bein # Import mappings -There are multiple types of mapping blocks that can be defined in the import plan. Each type enables a -different mechanism for filtering out secrets from an external system during the execution of an import. +Vault supports multiple filter types for mapping blocks. You can use the +different types to filter the information Vault reads in during the import +process. ## Argument reference @@ -22,7 +23,7 @@ All types of mapping blocks, no matter the type, have four shared arguments: * `priority` - (Required) Controls the order in which multiple mapping blocks will be applied to the secrets being imported. The lower the value specified, the higher the priority will be during filtering. -## Passthrough +## Passthrough mapping filters Passthrough mapping blocks `mapping_passthrough` allow all secrets through from the specified source to the specified destination. For example, one use case is using it as a base-case for imported secrets. By assigning From 2435ae0af3dd5b07cae23aa9d7d4abe43d19a81d Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 04:58:49 -0500 Subject: [PATCH 11/22] Fix typo --- website/content/docs/import/gcpsm.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index 0f9831c575fb..ae57797f21e5 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -18,7 +18,7 @@ Use the GCP source to import secret data from the GCP secret manager into your V * `name` - (Required) The name of the source block used to internally reference this source inside of a mapping. -* `credentials` - (Optional) The path to the service account key credentials file for the service account with the ncessary +* `credentials` - (Optional) The path to the service account key credentials file for the service account with the necessary permissions. If `credentials` is set, then `vault_mount_path` and `vault_role_name` must be unset. * `vault_mount_path` - (Optional) The Vault mount path to a preconfigured GCP secrets engine that is used to generate a From 9aa1b51e6a3e72e469691c066db9bbc2aee7cae2 Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 12:56:01 -0500 Subject: [PATCH 12/22] Update website/content/docs/import/index.mdx Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- website/content/docs/import/index.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index 52385c051146..d4c7039cfa20 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -56,7 +56,8 @@ destination_vault { - `address (string)` - Optional network address of the Vault server with the KVv2 secrets engine enabled. -* `token` - (Optional) A token for authentication to the Vault server located at the specified address. +- `token (string)` - Optional authentication token for the Vault server at the + specified address. - `namespace (string)` - Optional namespace path containing the specified KVv2 mount. By default, Vault looks for the KVv2 mount under the root namespace. From 7af65e868611e4bda69038d1d4f348fc110f8a91 Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 15:13:12 -0500 Subject: [PATCH 13/22] Apply suggestions from code review Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- website/content/docs/import/gcpsm.mdx | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index ae57797f21e5..a6bae09ac95c 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -16,17 +16,20 @@ Use the GCP source to import secret data from the GCP secret manager into your V ## GCP source parameters -* `name` - (Required) The name of the source block used to internally reference this source inside of a mapping. +- `name` - (Required) The unique name used by Vault to reference the GCP source elsewhere (e.g., in a mapping). * `credentials` - (Optional) The path to the service account key credentials file for the service account with the necessary permissions. If `credentials` is set, then `vault_mount_path` and `vault_role_name` must be unset. -* `vault_mount_path` - (Optional) The Vault mount path to a preconfigured GCP secrets engine that is used to generate a -dynamic credential for the importer to use. If `vault_mount_path` or `vault_role_name` are set, then `credentials` must be unset. +- `vault_mount_path` - (Optional) The Vault mount path to a pre-configured GCP + secrets engine used to generate dynamic credentials for the importer. If + `vault_mount_path` or `vault_role_name` are set, then `credentials` must be + unset. -* `vault_role_name` - (Optional) The Vault role's name that exists in the preconfigured GCP secrets engine mount that is -used to generate a dynamic credential for the importer to use. If `vault_role_name` or `vault_mount_path` are set, then -`credentials` must be unset. +- `vault_role_name` - (Optional) The Vault role used to generate a dynamic + credential for the importer. The role name must exist in the pre-configured + GCP secrets engine mount. If `vault_role_name` or `vault_mount_path` are set, + then `credentials` must be unset. ## Example From 11c8ae85e34f74cea8b06c3186e21fd1b1c8f429 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 16:18:48 -0500 Subject: [PATCH 14/22] Add experimental partial, apply more suggestions that can't be autoapplied, fix some typos/etc --- .../content/docs/commands/operator/import.mdx | 36 +++++---- website/content/docs/import/gcpsm.mdx | 26 ++++--- website/content/docs/import/index.mdx | 44 +++++------ website/content/docs/import/mappings.mdx | 73 ++++++++++--------- website/content/docs/sync/gcpsm.mdx | 8 +- .../content/partials/alerts/experimental.mdx | 7 ++ website/data/docs-nav-data.json | 2 +- 7 files changed, 112 insertions(+), 84 deletions(-) create mode 100644 website/content/partials/alerts/experimental.mdx diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx index 9a44f0dd4846..683dd6b4a067 100644 --- a/website/content/docs/commands/operator/import.mdx +++ b/website/content/docs/commands/operator/import.mdx @@ -41,7 +41,7 @@ Output: The following mounts are missing: * ns-1/mount-1 - Secrets to be imported from the source "my-dest-1": + Secrets to be imported to the destination "my-dest-1": * secret-1 * secret-2 @@ -50,39 +50,49 @@ Output: ## Configuration The `operator import` command uses a dedicated configuration file to specify the source, -destination, and mapping rules. +destination, and mapping rules. To learn more about these types and secrets importing, +refer to [the Secrets Import documentation](/vault/docs/import). ```hcl source_gcp { - name = "my-gcp-source-1" + name = "my-gcp-source-1" credentials = "@/path/to/service-account-key.json" } destination_vault { - name = "my-dest-1" - address = "http://127.0.0.1:8200/" - token = "root" + name = "my-dest-1" + address = "http://127.0.0.1:8200/" + token = "root" namespace = "ns-1" - mount = "mount-1" + mount = "mount-1" } mapping_passthrough { - name = "my-map-1" - source = "my-gcp-1" + name = "my-map-1" + source = "my-gcp-1" destination = "my-dest-1" - priority = 1 + priority = 1 } ``` ## Usage +### Arguments + +- `plan` - Executes a read-only operation to let operators preview the secrets to import based on the configuration file. + +- `apply` - Executes the secrets import operations to bring the specified secrets from the source into Vault. Apply first + executes a plan, then asks the user to approve the results before performing the actual import. + +### Flags + The `operator import` command accepts the following flags: -- `-config` `(string: "")` - Path to the import configuration HCL file. The default path is `import.hcl`. +- `-config` `(string: "import.hcl")` - Path to the import configuration HCL file. The default path is `import.hcl`. -- `auto-approve` `(bool: )` - Automatically response "yes" to all user-input prompts for the `apply` command. +- `-auto-approve` `(bool: )` - Automatically response "yes" to all user-input prompts for the `apply` command. -- `auto-create` `(bool: )` - Automatically creates any missing namespaces and mounts when +- `-auto-create` `(bool: )` - Automatically creates any missing namespaces and mounts when running the `apply` command. - `-log-level` ((#\_log_level)) `(string: "info")` - Log verbosity level. Supported values (in diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index a6bae09ac95c..ff2a403b4934 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -7,19 +7,16 @@ description: The Google Cloud Platform Secret Manager source imports secrets fro # GCP import source -Use the GCP source to import secret data from the GCP secret manager into your Vault instance. To use a GCP source, you must grant GCP the following permissions: - -```shell-session -"secretmanager.secrets.list", -"secretmanager.versions.access", -``` +Use the GCP source to import secret data from the GCP Secret Manager into your Vault instance. To use dynamic +credentials for this source, ensure that the [GCP secrets engine](/vault/docs/secrets/gcp) has +already been configured. ## GCP source parameters - `name` - (Required) The unique name used by Vault to reference the GCP source elsewhere (e.g., in a mapping). -* `credentials` - (Optional) The path to the service account key credentials file for the service account with the necessary -permissions. If `credentials` is set, then `vault_mount_path` and `vault_role_name` must be unset. +- `credentials` - (Optional) The path to the service account key credentials file for the service account with + the necessary permissions. If `credentials` is set, then `vault_mount_path` and `vault_role_name` must be unset. - `vault_mount_path` - (Optional) The Vault mount path to a pre-configured GCP secrets engine used to generate dynamic credentials for the importer. If @@ -37,8 +34,17 @@ Define and configure the `my-gcp-source-1` GCP source: ```hcl source_gcp { - name = "my-gcp-source-1" + name = "my-gcp-source-1" secrets_engine_mount = "gcp" - secrets_engine_role = "my-gcp-role-1" + secrets_engine_role = "my-gcp-role-1" } ``` + +## Permissions + +To use a GCP source, you must grant the GCP identity being used to read secrets the following permissions: + +```shell-session +"secretmanager.secrets.list", +"secretmanager.versions.access", +``` \ No newline at end of file diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index d4c7039cfa20..6e214b70bbe9 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -7,16 +7,16 @@ description: Secrets import allows you to safely onboard secrets from external s # Secrets import - +@include 'alerts/enterprise-only.mdx' -@include 'alerts/beta.mdx' +@include 'alerts/experimental.mdx' Distributing sensitive information across multiple external systems creates several challenges, including: -- increased operational overhead. -- increased exposure risk from data sprawl. -- increased risk of outdated and out-of-sync information. +- Increased operational overhead. +- Increased exposure risk from data sprawl. +- Increased risk of outdated and out-of-sync information. Using Vault as a single source of truth (SSOT) for sensitive data increases security and reduces management overhead, but migrating data from multiple and @@ -27,9 +27,9 @@ migration with codified import plans as HCL files. Import plans tell Vault which data to read from a given source and where to store the data in the KVv2 secrets engine: -- the `source` block provides credentials for connecting to the source system. -- the `destination` block defines target mounts in KVv2. -- the `mapping` block define how Vault should transform the imported data before +- The `source` block provides credentials for connecting to the source system. +- The `destination` block defines target mounts in KVv2. +- The `mapping` block define how Vault should transform the imported data before writing the information to KVv2. ## Data migration destinations @@ -50,19 +50,19 @@ destination_vault { } ``` -- `name (string: )` - A unique name for the destination block that can +- `name` `(string: )` - A unique name for the destination block that can be referenced in subsequent mapping blocks. -- `address (string)` - Optional network address of the Vault server with the - KVv2 secrets engine enabled. +- `mount` `(string: )` - The mount path for the target KVv2 instance. -- `token (string)` - Optional authentication token for the Vault server at the - specified address. +- `address` `(string)` - Optional network address of the Vault server with the + KVv2 secrets engine enabled. By default, the Vault client's address will be used. -- `namespace (string)` - Optional namespace path containing the specified KVv2 - mount. By default, Vault looks for the KVv2 mount under the root namespace. +- `token` `(string)` - Optional authentication token for the Vault server at the + specified address. By default, the Vault client's token will be used. -- `mount (string)` - Optional mount path for the target KVv2 instance. +- `namespace` `(string)` - Optional namespace path containing the specified KVv2 + mount. By default, Vault looks for the KVv2 mount under the root namespace. @@ -88,10 +88,10 @@ source_gcp { } ``` -- `name (string: )` - A unique name for the source block that can be +- `name` `(string: )` - A unique name for the source block that can be referenced in subsequent mapping blocks. -- `credentials (string: )` - Path to a credential file or token with +- `credentials` `(string: )` - Path to a credential file or token with read permissions for the target system. Depending on the source system, additional information may be required. Refer to @@ -125,15 +125,15 @@ mapping_regex { } ``` -- `name (string: )` - A unique name for the mapping block. +- `name` `(string: )` - A unique name for the mapping block. -- `source (string: )` - The name of a previously defined source block +- `source` `(string: )` - The name of a previously-defined source block **from** which the data should be read. -- `destination (string: )` - The name of a previously defined +- `destination` `(string: )` - The name of a previously defined destination block **to** which the data should be written. -- `priority (integer: )` - The order in which Vault should apply the +- `priority` `(integer: )` - The order in which Vault should apply the mapping block during the import process. The lower the number, the higher the priority. For example, a mapping with priority 1 executes before a mapping with priority 2. diff --git a/website/content/docs/import/mappings.mdx b/website/content/docs/import/mappings.mdx index 4f24f53db78f..583bc95bace4 100644 --- a/website/content/docs/import/mappings.mdx +++ b/website/content/docs/import/mappings.mdx @@ -10,19 +10,6 @@ Vault supports multiple filter types for mapping blocks. You can use the different types to filter the information Vault reads in during the import process. -## Argument reference - -All types of mapping blocks, no matter the type, have four shared arguments: - -* `name` - (Required) The name of the mapping block used internally to track secrets imported from a source to a destination. - -* `source` - (Required) The name of the source block for which this mapping's filter logic will be applied to. - -* `destination` - (Required) The name of the Vault destination block for which this mapping will place the matching secrets into. - -* `priority` - (Required) Controls the order in which multiple mapping blocks will be applied to the secrets being imported. The -lower the value specified, the higher the priority will be during filtering. - ## Passthrough mapping filters Passthrough mapping blocks `mapping_passthrough` allow all secrets through from the specified source to the @@ -60,16 +47,12 @@ on secrets while [GCP](/vault/docs/import/gcpsm) allows labels. * `tags` - (Required) A set of key-value pairs to match on secrets from the external system. All of the specified keys must be found on a secret and all of the values must be exact matches. A key with an empty value, i.e. `""`, -must also match an empty value from the external system. +acts as a wildcard match to the external system's values. ### Example -In this example, there are two metadata mappings. The first, `my-map-1`, has a priority of 1. This will only import -the secrets into the destination `my-dest-1` that contain both tag keys `database` and `importable`. Each of these -keys' values must also match to `users` and `true` respectively to be imported. The second, `my-map-2`, has a -priority of 2. Even though all the secrets in the first map would also qualify for the second map's filtering rule, -those secrets will only be imported into `my-dest-1` because of the lower priority set in the second mapping. All -remaining secrets that have the tag `importable` with a value of `true` will be imported into `my-dest-2`. +In this example, `my-map-1` will only import the secrets into the destination `my-dest-1` that contain a tag with +a key named `importable` and its value set to `true`. ```hcl mapping_metadata { @@ -77,17 +60,6 @@ mapping_metadata { source = "my-gcp-source-1" destination = "my-dest-1" priority = 1 - tags = { - "database" = "users" - "importable" = "true" - } -} - -mapping_metadata { - name = "my-map-2" - source = "my-gcp-source-1" - destination = "my-dest-2" - priority = 2 tags = { "importable" = "true" } @@ -101,9 +73,7 @@ destination if their secret name passes a regular expression check. ### Additional arguments -* `expression` - (Required) A regular expression to match secrets names from the external system. All of -the specified keys must be found on a secret and all of the values must be exact matches. A key with an empty -value, e.g. `""`, must also match an empty value from the external system. +* `expression` - (Required) A regular expression to match the secrets names from the external system. ### Example @@ -118,3 +88,38 @@ mapping_regex { expression = "^database/.*$" } ``` + +## Priority + +Priority works in a "first match" fashion where lower values are higher priority. To explain in more detail, +consider the above metadata example with a second additional mapping. + +Below are two metadata mappings. The first, `my-map-1`, has a priority of 1. This will only import the secrets +into the destination `my-dest-1` that contain both tag keys `database` and `importable`. Each of these keys' values +must also match to `users` and `true` respectively. The second, `my-map-2`, has a priority of 2. Even though all +the secrets in the first mapping would also qualify for the second mapping's filtering rule, those secrets will only +be imported into `my-dest-1` because of `my-map-2`'s lower priority. All remaining secrets that have the tag +`importable` with a value of `true` will be imported into `my-dest-2`. + +```hcl +mapping_metadata { + name = "my-map-1" + source = "my-gcp-source-1" + destination = "my-dest-1" + priority = 1 + tags = { + "database" = "users" + "importable" = "true" + } +} + +mapping_metadata { + name = "my-map-2" + source = "my-gcp-source-1" + destination = "my-dest-2" + priority = 2 + tags = { + "importable" = "true" + } +} +``` \ No newline at end of file diff --git a/website/content/docs/sync/gcpsm.mdx b/website/content/docs/sync/gcpsm.mdx index c5782a22f16f..5b46b1e537db 100644 --- a/website/content/docs/sync/gcpsm.mdx +++ b/website/content/docs/sync/gcpsm.mdx @@ -154,10 +154,10 @@ to provision a GCP Service Account with the appropriate policies. // Environment Variables // GOOGLE_REGION (optional) // GOOGLE_PROJECT - // GOOGLE_CREDENTIALS (The path to a service account key file with the - // "Service Account Admin", "Service Account Key Admin", - // "Secret Manager Admin", and "Project IAM Admin" roles - // attached) + // GOOGLE_CREDENTIALS (The path to a service account key file with the + // "Service Account Admin", "Service Account Key Admin", + // "Secret Manager Admin", and "Project IAM Admin" roles + // attached) } data "google_client_config" "config" {} diff --git a/website/content/partials/alerts/experimental.mdx b/website/content/partials/alerts/experimental.mdx new file mode 100644 index 000000000000..120519e098df --- /dev/null +++ b/website/content/partials/alerts/experimental.mdx @@ -0,0 +1,7 @@ + + + Experimental features are features in an alpha state or available early in development to + provide as a tech demo experience and are subject to change. + **We strongly discourage using experimental features in production deployments of Vault.** + + \ No newline at end of file diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index dc2a38b8af90..2a6903417e65 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -1619,7 +1619,7 @@ "title": "Overview", "path": "import", "badge": { - "text": "BETA", + "text": "EXPERIMENTAL", "type": "outlined", "color": "highlight" } From f8232dc801f431fa044682773d05ac617bcd5434 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 16:21:48 -0500 Subject: [PATCH 15/22] Align remaining HCL blocks --- website/content/docs/import/index.mdx | 2 +- website/content/docs/import/mappings.mdx | 39 +++++++++++++----------- 2 files changed, 22 insertions(+), 19 deletions(-) diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index 6e214b70bbe9..e3c65c41c931 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -83,7 +83,7 @@ information required by the target system or engine. For example: ```hcl source_gcp { - name = "my-gcp-source-1" + name = "my-gcp-source-1" credentials = "@/path/to/service-account-key.json" } ``` diff --git a/website/content/docs/import/mappings.mdx b/website/content/docs/import/mappings.mdx index 583bc95bace4..0559d58b7cd0 100644 --- a/website/content/docs/import/mappings.mdx +++ b/website/content/docs/import/mappings.mdx @@ -29,10 +29,10 @@ to the KVv2 Secrets Engine mount defined in `my-dest-1`. ```hcl mapping_passthrough { - name = "my-map-1" - source = "my-gcp-source-1" + name = "my-map-1" + source = "my-gcp-source-1" destination = "my-dest-1" - priority = 1 + priority = 1 } ``` @@ -56,10 +56,11 @@ a key named `importable` and its value set to `true`. ```hcl mapping_metadata { - name = "my-map-1" - source = "my-gcp-source-1" + name = "my-map-1" + source = "my-gcp-source-1" destination = "my-dest-1" - priority = 1 + priority = 1 + tags = { "importable" = "true" } @@ -81,11 +82,11 @@ In this example, any secret in the GCP source whose name begins with `database/` ```hcl mapping_regex { - name = "my-map-1" - source = "my-gcp-source-1" + name = "my-map-1" + source = "my-gcp-source-1" destination = "my-dest-1" - priority = 1 - expression = "^database/.*$" + priority = 1 + expression = "^database/.*$" } ``` @@ -103,23 +104,25 @@ be imported into `my-dest-1` because of `my-map-2`'s lower priority. All remaini ```hcl mapping_metadata { - name = "my-map-1" - source = "my-gcp-source-1" + name = "my-map-1" + source = "my-gcp-source-1" destination = "my-dest-1" - priority = 1 + priority = 1 + tags = { - "database" = "users" + "database" = "users" "importable" = "true" } } mapping_metadata { - name = "my-map-2" - source = "my-gcp-source-1" + name = "my-map-2" + source = "my-gcp-source-1" destination = "my-dest-2" - priority = 2 + priority = 2 + tags = { "importable" = "true" } } -``` \ No newline at end of file +``` From 9735bc767a006ff4b1075737ce71f923d252e4bf Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 17:00:28 -0500 Subject: [PATCH 16/22] Fix typos, wording, add links, fill in small bits of info --- .../content/docs/commands/operator/import.mdx | 14 +++--- website/content/docs/import/gcpsm.mdx | 7 ++- website/content/docs/import/index.mdx | 49 ++++++++++--------- 3 files changed, 35 insertions(+), 35 deletions(-) diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx index 683dd6b4a067..3f905c9686d5 100644 --- a/website/content/docs/commands/operator/import.mdx +++ b/website/content/docs/commands/operator/import.mdx @@ -14,7 +14,7 @@ Secrets with the same name at the same storage path will be overwritten upon imp You can write import plans that read from as many sources as you want. The -amount of data migrated for each source depends on the filters applied and the +amount of data migrated from each source depends on the filters applied and the dataset available. Be mindful of the time needed to read from each source, apply any filters, and store the data in Vault. @@ -50,8 +50,8 @@ Output: ## Configuration The `operator import` command uses a dedicated configuration file to specify the source, -destination, and mapping rules. To learn more about these types and secrets importing, -refer to [the Secrets Import documentation](/vault/docs/import). +destination, and mapping rules. To learn more about these types and secrets importing in +general, refer to the [Secrets Import documentation](/vault/docs/import). ```hcl source_gcp { @@ -81,8 +81,8 @@ mapping_passthrough { - `plan` - Executes a read-only operation to let operators preview the secrets to import based on the configuration file. -- `apply` - Executes the secrets import operations to bring the specified secrets from the source into Vault. Apply first - executes a plan, then asks the user to approve the results before performing the actual import. +- `apply` - Executes the import operations to read the specified secrets from the source and write them into Vault. + Apply first executes a plan, then asks the user to approve the results before performing the actual import. ### Flags @@ -90,9 +90,9 @@ The `operator import` command accepts the following flags: - `-config` `(string: "import.hcl")` - Path to the import configuration HCL file. The default path is `import.hcl`. -- `-auto-approve` `(bool: )` - Automatically response "yes" to all user-input prompts for the `apply` command. +- `-auto-approve` `(bool: )` - Automatically responds "yes" to all user-input prompts for the `apply` command. -- `-auto-create` `(bool: )` - Automatically creates any missing namespaces and mounts when +- `-auto-create` `(bool: )` - Automatically creates any missing namespaces and KVv2 mounts when running the `apply` command. - `-log-level` ((#\_log_level)) `(string: "info")` - Log verbosity level. Supported values (in diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index ff2a403b4934..bd21d35bc3d0 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -11,12 +11,11 @@ Use the GCP source to import secret data from the GCP Secret Manager into your V credentials for this source, ensure that the [GCP secrets engine](/vault/docs/secrets/gcp) has already been configured. -## GCP source parameters - -- `name` - (Required) The unique name used by Vault to reference the GCP source elsewhere (e.g., in a mapping). +## Additional arguments - `credentials` - (Optional) The path to the service account key credentials file for the service account with - the necessary permissions. If `credentials` is set, then `vault_mount_path` and `vault_role_name` must be unset. + the [necessary permissions](#permissions). If `credentials` is set, then `vault_mount_path` and + `vault_role_name` must be unset. - `vault_mount_path` - (Optional) The Vault mount path to a pre-configured GCP secrets engine used to generate dynamic credentials for the importer. If diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index e3c65c41c931..2291d9f56793 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -19,24 +19,24 @@ several challenges, including: - Increased risk of outdated and out-of-sync information. Using Vault as a single source of truth (SSOT) for sensitive data increases -security and reduces management overhead, but migrating data from multiple and -varied sources can be complex and costly. +security and reduces management overhead, but migrating preexisting data from multiple +and/or varied sources can be complex and costly. -The secrets import process helps you automate and streamline your data -migration with codified import plans as HCL files. Import plans tell Vault -which data to read from a given source and where to store the data in the KVv2 -secrets engine: +The secrets import process helps you automate and streamline your sensitive data +migration with codified import plans as HCL files. Import plans tell Vault which KVv2 secrets +engine instance to store the expected secret data in, the source system for which data will be +read from, and how to filter this data. Three HCL blocks make this possible: -- The `source` block provides credentials for connecting to the source system. -- The `destination` block defines target mounts in KVv2. -- The `mapping` block define how Vault should transform the imported data before +- The `destination` block defines target KVv2 mounts. +- The `source` block provides credentials for connecting to the external system. +- The `mapping` block defines how Vault should decide which data gets imported before writing the information to KVv2. -## Data migration destinations +## Destinations Vault stores imported secrets in a Vault KVv2 secrets engine mount. Destination blocks start with `destination_vault` and define the desired KVv2 mount path and -an optional namespace. The combination of a path and namespace represent the exact location in your +an optional namespace. The combination of these represent the exact location in your Vault instance you want the information stored. ### HCL syntax @@ -66,20 +66,20 @@ destination_vault { -## Data migration sources +## Sources Vault can import secrets from the following sources: - [GCP Secret Manager](/vault/docs/import/gcpsm) To pull data from a source during import, Vault needs read credentials for the external system. You can provide credentials directly as part of the import -plan, or use Vault to generate dynamic credentials for the importer if you already -have dynamic credentials configured for the corresponding secrets engine. +plan, or use Vault to automatically generate dynamic credentials if you already +have the corresponding secrets engine configured. ### HCL syntax Source blocks start with `source_` and include any connection -information required by the target system or engine. For example: +information required by the target system or the secrets engine to leverage. For example: ```hcl source_gcp { @@ -100,10 +100,10 @@ required fields for that system type. -## Data migration mappings +## Mappings -Mappings glue the source and destination together and filter the migrated data -to control what is imported and what is ignored. Vault currently supports the +Mappings glue the source and destination together and filter the migrated data, +to determine what is imported and what is ignored. Vault currently supports the following mapping methods: - [mapping_passthrough](/vault/docs/import/mappings#passthrough) @@ -112,8 +112,9 @@ following mapping methods: ### HCL syntax -Mapping blocks start with `mapping_` and include a source name, -destination name, and any transformations or filters that apply. For example: +Mapping blocks start with `mapping_` and require a source name, +destination name, an execution priority, and any corresponding transformations +or filters that apply for each mapping type. For example: ```hcl mapping_regex { @@ -138,15 +139,15 @@ mapping_regex { priority. For example, a mapping with priority 1 executes before a mapping with priority 2. -Depending on the filter type, additional fields may be required/possible. Refer -to the filter documentation for the different mappings for a list of -mapping-specific fields. +Depending on the filter type, additional fields may be required or possible. Refer +to the [import mappings documentation](/vault/docs/import/mappings) for the available +supported options and for a list of each mapping's specific fields. Vault applies mapping definitions in priority order and a given secret only matches to the first mapping that applies. Once Vault imports a secret with a particular mapping, subsequent reads from the same source will ignore that - secret. + secret. See the [priority section](/vault/docs/import/mappings#priority) for an example. From 38fb9d7fa35602494509c6eec71be0dc848b5b66 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Mon, 25 Mar 2024 18:17:45 -0500 Subject: [PATCH 17/22] Update wording and formatting for gcp source and mappings --- website/content/docs/import/gcpsm.mdx | 18 ++++++++----- website/content/docs/import/mappings.mdx | 34 +++++++++++++----------- 2 files changed, 30 insertions(+), 22 deletions(-) diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index bd21d35bc3d0..3ae85d7cba01 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -7,22 +7,26 @@ description: The Google Cloud Platform Secret Manager source imports secrets fro # GCP import source -Use the GCP source to import secret data from the GCP Secret Manager into your Vault instance. To use dynamic +Use the GCP source to import secret data from GCP Secret Manager into your Vault instance. To use dynamic credentials for this source, ensure that the [GCP secrets engine](/vault/docs/secrets/gcp) has already been configured. +## Argument reference + +Refer to the [HCL syntax](/vault/docs/import#hcl-syntax-1) for arguments common to all source types. + ## Additional arguments -- `credentials` - (Optional) The path to the service account key credentials file for the service account with - the [necessary permissions](#permissions). If `credentials` is set, then `vault_mount_path` and +- `credentials` `(string: "")` - The path to the service account key credentials file for the service account + with the [necessary permissions](#permissions). If `credentials` is set, then `vault_mount_path` and `vault_role_name` must be unset. -- `vault_mount_path` - (Optional) The Vault mount path to a pre-configured GCP +- `vault_mount_path` `(string: "")` - The Vault mount path to a pre-configured GCP secrets engine used to generate dynamic credentials for the importer. If `vault_mount_path` or `vault_role_name` are set, then `credentials` must be unset. -- `vault_role_name` - (Optional) The Vault role used to generate a dynamic +- `vault_role_name` `(string: "")` - The Vault role used to generate a dynamic credential for the importer. The role name must exist in the pre-configured GCP secrets engine mount. If `vault_role_name` or `vault_mount_path` are set, then `credentials` must be unset. @@ -41,9 +45,9 @@ source_gcp { ## Permissions -To use a GCP source, you must grant the GCP identity being used to read secrets the following permissions: +To use this source type, you must grant the GCP identity being used to read secrets the following permissions: ```shell-session "secretmanager.secrets.list", "secretmanager.versions.access", -``` \ No newline at end of file +``` diff --git a/website/content/docs/import/mappings.mdx b/website/content/docs/import/mappings.mdx index 0559d58b7cd0..310e9e65710c 100644 --- a/website/content/docs/import/mappings.mdx +++ b/website/content/docs/import/mappings.mdx @@ -6,17 +6,21 @@ description: Mappings lets users apply various filtering methods to secrets bein # Import mappings -Vault supports multiple filter types for mapping blocks. You can use the -different types to filter the information Vault reads in during the import -process. +Vault supports multiple filter types for mapping blocks. Each of the types provides a different mechanism +used to filter the scanned secrets and determine which will be imported in to Vault. + + +## Argument reference + +Refer to the [HCL syntax](/vault/docs/import#hcl-syntax-2) for arguments common to all mapping types. ## Passthrough mapping filters -Passthrough mapping blocks `mapping_passthrough` allow all secrets through from the specified source to the -specified destination. For example, one use case is using it as a base-case for imported secrets. By assigning +The passthrough mapping block `mapping_passthrough` allows all secrets through from the specified source to the +specified destination. For example, one use case is setting it as a base-case for imported secrets. By assigning it the lowest priority in the import plan, all other mapping blocks will be applied first. Secrets that fail to match any of the previous mappings will fall through to the passthrough block and be collected in a single -KVv2 bucket. +KVv2 location. ### Additional arguments @@ -24,8 +28,8 @@ There are no extra arguments to specify in a `mapping_passthrough` block. ### Example -In this example, every single secret that `my-gcp-source-1` is reads from GCP Secret Manager will be imported -to the KVv2 Secrets Engine mount defined in `my-dest-1`. +In this example, every single secret that `my-gcp-source-1` scans from GCP Secret Manager will be imported +to the KVv2 secrets engine mount defined in `my-dest-1`. ```hcl mapping_passthrough { @@ -38,16 +42,16 @@ mapping_passthrough { ## Metadata -Metadata mapping blocks `mapping_metadata` allow secrets through from the specified source to the specified +The metadata mapping block `mapping_metadata` allows secrets through from the specified source to the specified destination if they contain matching metadata key-value pairs. Metadata is not supported in all external secret -manageement systems, and ones that do may use different terminology for metadata. For example, AWS allows tags +management systems, and ones that do may use different terminology for metadata. For example, AWS allows tags on secrets while [GCP](/vault/docs/import/gcpsm) allows labels. ### Additional arguments -* `tags` - (Required) A set of key-value pairs to match on secrets from the external system. All of the specified -keys must be found on a secret and all of the values must be exact matches. A key with an empty value, i.e. `""`, -acts as a wildcard match to the external system's values. +* `tags` `(string: )` - A set of key-value pairs to match on secrets from the external system. All of the specified +keys must be found on a secret and all of the values must be exact matches. Specifying a key in this mapping with +an empty value, i.e. `""`, acts as a wildcard match to the external system's key's value. ### Example @@ -69,12 +73,12 @@ mapping_metadata { ## Regex -Regex mapping blocks `mapping_regex` allow secrets through from the specified source to the specified +The regex mapping block `mapping_regex` allows secrets through from the specified source to the specified destination if their secret name passes a regular expression check. ### Additional arguments -* `expression` - (Required) A regular expression to match the secrets names from the external system. +* `expression` `(string: )` - The regular expression used to match secrets' names from the external system. ### Example From 4e6f242a6fc8133b5bc04c768e20c4721c1bec33 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Tue, 26 Mar 2024 13:00:25 -0500 Subject: [PATCH 18/22] Change experimental to alpha --- website/content/docs/commands/operator/import.mdx | 4 ++++ website/content/docs/import/index.mdx | 2 +- website/content/partials/alerts/alpha.mdx | 8 ++++++++ 3 files changed, 13 insertions(+), 1 deletion(-) create mode 100644 website/content/partials/alerts/alpha.mdx diff --git a/website/content/docs/commands/operator/import.mdx b/website/content/docs/commands/operator/import.mdx index 3f905c9686d5..8144b1c92713 100644 --- a/website/content/docs/commands/operator/import.mdx +++ b/website/content/docs/commands/operator/import.mdx @@ -8,6 +8,10 @@ description: >- # operator import +@include 'alerts/enterprise-only.mdx' + +@include 'alerts/alpha.mdx' + The `operator import` command imports secrets from external systems in to Vault. Secrets with the same name at the same storage path will be overwritten upon import. diff --git a/website/content/docs/import/index.mdx b/website/content/docs/import/index.mdx index 2291d9f56793..2303b248a85e 100644 --- a/website/content/docs/import/index.mdx +++ b/website/content/docs/import/index.mdx @@ -9,7 +9,7 @@ description: Secrets import allows you to safely onboard secrets from external s @include 'alerts/enterprise-only.mdx' -@include 'alerts/experimental.mdx' +@include 'alerts/alpha.mdx' Distributing sensitive information across multiple external systems creates several challenges, including: diff --git a/website/content/partials/alerts/alpha.mdx b/website/content/partials/alerts/alpha.mdx new file mode 100644 index 000000000000..62755be50883 --- /dev/null +++ b/website/content/partials/alerts/alpha.mdx @@ -0,0 +1,8 @@ +<<<<<<< HEAD + + + Alpha features are features in an active-development state or available early in development + to provide as a tech demo experience and are subject to change. + **We strongly discourage using alpha features in production deployments of Vault.** + + \ No newline at end of file From d14f0e1f838aa95bd0a42763d7db975fb3fb2d23 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Tue, 26 Mar 2024 13:00:53 -0500 Subject: [PATCH 19/22] Change list tag to alpha --- website/data/docs-nav-data.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index a8438e677d98..2616e4967697 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -1679,7 +1679,7 @@ "title": "Overview", "path": "import", "badge": { - "text": "EXPERIMENTAL", + "text": "ALPHA", "type": "outlined", "color": "highlight" } From 2fa73295856e7be262af92e4e44d6ac1acfa8941 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Tue, 26 Mar 2024 13:02:02 -0500 Subject: [PATCH 20/22] Fix file --- website/content/partials/alerts/alpha.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/website/content/partials/alerts/alpha.mdx b/website/content/partials/alerts/alpha.mdx index 62755be50883..5bacd871eec2 100644 --- a/website/content/partials/alerts/alpha.mdx +++ b/website/content/partials/alerts/alpha.mdx @@ -1,4 +1,3 @@ -<<<<<<< HEAD Alpha features are features in an active-development state or available early in development From 908c8580bb3c6c55973e04abcc99fef03f178a85 Mon Sep 17 00:00:00 2001 From: Robert <17119716+robmonte@users.noreply.github.com> Date: Tue, 26 Mar 2024 14:39:06 -0500 Subject: [PATCH 21/22] Apply suggestions from code review Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com> --- website/content/docs/import/gcpsm.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index 3ae85d7cba01..b085481b0e38 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -1,6 +1,6 @@ --- layout: docs -page_title: Google Cloud Platform Secret Manager - Secrets Import Source +page_title: GCP secret import source description: The Google Cloud Platform Secret Manager source imports secrets from GCP to Vault. --- @@ -8,8 +8,8 @@ description: The Google Cloud Platform Secret Manager source imports secrets fro Use the GCP source to import secret data from GCP Secret Manager into your Vault instance. To use dynamic -credentials for this source, ensure that the [GCP secrets engine](/vault/docs/secrets/gcp) has -already been configured. +credentials with GCP import, ensure the [GCP secrets engine](/vault/docs/secrets/gcp) is +already configured. ## Argument reference @@ -45,7 +45,7 @@ source_gcp { ## Permissions -To use this source type, you must grant the GCP identity being used to read secrets the following permissions: +To use GCP import, you must grant the associated GCP identity permission to read secrets: ```shell-session "secretmanager.secrets.list", From 5359662a645465d0410a00b240e767d554a88980 Mon Sep 17 00:00:00 2001 From: robmonte <17119716+robmonte@users.noreply.github.com> Date: Tue, 26 Mar 2024 23:56:28 -0500 Subject: [PATCH 22/22] Remove extra line, trigger CI rerun --- website/content/docs/import/gcpsm.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/website/content/docs/import/gcpsm.mdx b/website/content/docs/import/gcpsm.mdx index b085481b0e38..7a9767a1d42c 100644 --- a/website/content/docs/import/gcpsm.mdx +++ b/website/content/docs/import/gcpsm.mdx @@ -6,7 +6,6 @@ description: The Google Cloud Platform Secret Manager source imports secrets fro # GCP import source - Use the GCP source to import secret data from GCP Secret Manager into your Vault instance. To use dynamic credentials with GCP import, ensure the [GCP secrets engine](/vault/docs/secrets/gcp) is already configured.