From 41d9ab2e5ea89d7e261ac4da3206c572d6d710a3 Mon Sep 17 00:00:00 2001 From: Victor Rodriguez Date: Fri, 15 Oct 2021 14:51:53 -0400 Subject: [PATCH] Update docs with Transform FPE advanced I/O handling features (#12744) --- website/content/api-docs/secret/transform.mdx | 153 ++++++++++++------ .../content/docs/secrets/transform/index.mdx | 21 ++- 2 files changed, 126 insertions(+), 48 deletions(-) diff --git a/website/content/api-docs/secret/transform.mdx b/website/content/api-docs/secret/transform.mdx index c5b9aa53f59b..0fd21cfbfd49 100644 --- a/website/content/api-docs/secret/transform.mdx +++ b/website/content/api-docs/secret/transform.mdx @@ -455,8 +455,8 @@ template exists, it will be updated with the new attributes. request URL. - `type` `(string: )` - - Specifies the type of pattern matching to perform. The ony type currently supported - by this backend is `regex`. + Specifies the type of pattern matching to perform. The only type currently + supported by this backend is `regex`. - `pattern` `(string: )` - Specifies the pattern used to match a particular value. For regex type @@ -468,13 +468,28 @@ template exists, it will be updated with the new attributes. Specifies the name of the alphabet to use when this template is used for FPE encoding and decoding operations. +- `encode_format` `(string)` - + The regular expression template to use to format encoded values. This can be + used to normalize the encoded output. This is only used during FPE + transformations. + +- `decode_formats` `(key-value-map: {})` - + An optional map of regular expression templates that can be used to customize + decoded output. For example, this can be used to decode only the last four + digits of a credit card number. This is only used during FPE transformations. + ### Sample Payload ```json { "type": "regex", - "pattern": "(\\d{9})", - "alphabet": "builtin/numeric" + "alphabet": "builtin/numeric", + "pattern": "(\\d{3})[-/](\\d{2})[-/](\\d{4})", + "encode_format": "$1-$2-$3", + "decode_formats": { + "first-three": "$1", + "last-four": "$3" + } } ``` @@ -513,7 +528,12 @@ $ curl \ { "data": { "alphabet": "builtin/numeric", - "pattern": "(\\d{9})", + "decode_formats": { + "first-three": "$1", + "last-four": "$3" + }, + "encode_format": "$1-$2-$3", + "pattern": "(\\d{3})[-/](\\d{2})[-/](\\d{4})", "type": "regex" } } @@ -708,9 +728,6 @@ The database user configured here should only have permission to `SELECT`, Specifies the database driver to use, and thus which SQL database type. Currently the supported options are `postgres` or `mysql` -- `supported_transformations: `(list: ["tokenization"])` The types of transformations this store can host. Currently only`tokenization` - is supported. - - `connection_string` `(string: )` - A database connection string with template slots for username and password that Vault will use for locating and connecting to a database. Each @@ -724,6 +741,9 @@ The database user configured here should only have permission to `SELECT`, - `password`: `(string: )` - The password value to use when connecting to the database. +- `supported_transformations: `(list: ["tokenization"])` The types of transformations this store can host. Currently only`tokenization` + is supported. + - `schema`: `(string: "public")` - The schema within the database to expect tokenization state tables. @@ -734,7 +754,7 @@ The database user configured here should only have permission to `SELECT`, The maximum number of idle connections to the database at any given time. - `max_connection_lifetime` `(duration: 0)` - - The maximum amount of time a connection can be open before closing it. + The maximum amount of time a connection can be open before closing it. 0 means no limit. ### Sample Payloads @@ -787,15 +807,15 @@ operation. Specifies the name of the store to create or update. This is part of the request URL. -- `transformation_type`: `(string: "tokenization")` - - The transformation type. Currently only tokenization is supported. - - `username`: `(string: )` - The username value to use when connecting to the database. - `password`: `(string: )` - The password value to use when connecting to the database. +- `transformation_type`: `(string: "tokenization")` - + The transformation type. Currently only `tokenization` is supported. + ### Sample Payload ```json @@ -934,16 +954,17 @@ This endpoint encodes the provided value using a named role. transformations with `supplied` as the tweak source. The tweak must be a 7-byte value that is then base64 encoded. -- `reference` `(string: "")` - - Only valid on batch requests using 'batch_input' below. A user supplied - string that will be present on the corresponding item in the batch - response, to assist in correlating inputs with outputs. +- `reference` `(string: "")` - + A user-supplied string that will be present in the `reference` field on the + corresponding `batch_results` item in the response, to assist in understanding + which result corresponds to a particular input. Only valid on batch requests + when using ‘batch_input’ below. - `batch_input` `(array: nil)` - Specifies a list of items to be encoded in a single batch. When this parameter is set, the 'value', 'transformation', 'ttl', 'tweak' and - 'reference' parameters are ignored. Instead, the aforementioned parameters - should be provided within each object in the list. + 'reference' parameters are ignored. Instead, the aforementioned parameters + should be provided within each object in the list. ```json [ @@ -1048,9 +1069,9 @@ $ curl \ This endpoint decodes the provided value using a named role. -| Method | Path | -| :----- | :----------------------------- | -| `POST` | `/transform/decode/:role_name` | +| Method | Path | +| :----- | :---------------------------------------------- | +| `POST` | `/transform/decode/:role_name(/:decode_format)` | ### Parameters @@ -1061,6 +1082,13 @@ This endpoint decodes the provided value using a named role. - `value` `(string: )` – Specifies the value to be decoded. +- `decode_format` `(string)` - + The name of the decode format to use for decoding. These are defined in + `decode_formats` when creating the transformation's template, and can be used + to selectively decode or format the output. If one is not defined or + specified, the template's pattern will be used. Only applicable for FPE + transformations. + - `transformation` `(string)` – Specifies the transformation within the role that should be used for this decode operation. If a single transformation exists for role, this parameter @@ -1071,16 +1099,17 @@ This endpoint decodes the provided value using a named role. applicable for FPE transformations with `supplied` or `generated` as the tweak source. The tweak must be a 7-byte value that is then base64 encoded. -- `reference` `(string: "")` - - Only valid on batch requests using 'batch_input' below. A user supplied - string that will be present on the corresponding item in the batch - response, to assist in correlating inputs with outputs. +- `reference` `(string: "")` - + A user-supplied string that will be present in the `reference` field on the + corresponding `batch_results` item in the response, to assist in understanding + which result corresponds to a particular input. Only valid on batch requests + when using ‘batch_input’ below. - `batch_input` `(array: nil)` - - Specifies a list of items to be encoded in a single batch. When this + Specifies a list of items to be decoded in a single batch. When this parameter is set, the 'value', 'transformation', 'tweak' and - 'reference' parameters are ignored. Instead, the aforementioned parameters - should be provided within each object in the list. + 'reference' parameters are ignored. Instead, the aforementioned parameters + should be provided within each object in the list. ```json [ @@ -1093,10 +1122,39 @@ This endpoint decodes the provided value using a named role. ### Sample Payload + ```json +{ + "value": "418-56-4374", + "transformation": "example-transformation" +} +``` + +### Sample Request + +```shell-session +$ curl \ + --header "X-Vault-Token: ..." \ + --request POST \ + --data @payload.json \ + http://127.0.0.1:8200/v1/transform/decode/example-role +``` + +### Sample Response + ```json { - "value": "5682-4613-6822-8064", - "transformation": "ccn-fpe" + "data": { + "decoded_value": "111-22-3333" + } +} +``` + +### Sample Payload + + ```json +{ + "value": "418-56-4374", + "transformation": "example-transformation" } ``` @@ -1107,7 +1165,7 @@ $ curl \ --header "X-Vault-Token: ..." \ --request POST \ --data @payload.json \ - http://127.0.0.1:8200/v1/transform/encode/example-role + http://127.0.0.1:8200/v1/transform/decode/example-role/last-four ``` ### Sample Response @@ -1115,7 +1173,7 @@ $ curl \ ```json { "data": { - "decoded_value": "1111-2222-3333-4444" + "decoded_value": "4444" } } ``` @@ -1190,16 +1248,17 @@ Only valid for tokenization transformations. may be skipped and will be inferred. If multiple transformations exist, one must be specified. -- `reference` `(string: "")` - - Only valid on batch requests using 'batch_input' below. A user supplied - string that will be present on the corresponding item in the batch - response, to assist in correlating inputs with outputs. +- `reference` `(string: "")` - + A user-supplied string that will be present in the `reference` field on the + corresponding `batch_results` item in the response, to assist in understanding + which result corresponds to a particular input. Only valid on batch requests + when using ‘batch_input’ below. - `batch_input` `(array: nil)` - Specifies a list of items to be validated in a single batch. When this parameter is set, the 'value', 'transformation' and - 'reference' parameters are ignored. Instead, the aforementioned parameters - should be provided within each object in the list. + 'reference' parameters are ignored. Instead, the aforementioned parameters + should be provided within each object in the list. ```json [ @@ -1265,10 +1324,11 @@ transformations. may be skipped and will be inferred. If multiple transformations exist, one must be specified. -- `reference` `(string: "")` - - Only valid on batch requests using 'batch_input' below. A user supplied - string that will be present on the corresponding item in the batch - response, to assist in correlating inputs with outputs. +- `reference` `(string: "")` - + A user-supplied string that will be present in the `reference` field on the + corresponding `batch_results` item in the response, to assist in understanding + which result corresponds to a particular input. Only valid on batch requests + when using ‘batch_input’ below. - `batch_input` `(array: nil)` - Specifies a list of items to be decoded in a single batch. When this @@ -1339,10 +1399,11 @@ Only valid for tokenization transformations. may be skipped and will be inferred. If multiple transformations exist, one must be specified. -- `reference` `(string: "")` - - Only valid on batch requests using 'batch_input' below. A user supplied - string that will be present on the corresponding item in the batch - response, to assist in correlating inputs with outputs. +- `reference` `(string: "")` - + A user-supplied string that will be present in the `reference` field on the + corresponding `batch_results` item in the response, to assist in understanding + which result corresponds to a particular input. Only valid on batch requests + when using ‘batch_input’ below. - `batch_input` `(array: nil)` - Specifies a list of items to be decoded in a single batch. When this diff --git a/website/content/docs/secrets/transform/index.mdx b/website/content/docs/secrets/transform/index.mdx index 7841744a42af..2ee9d7b2c58d 100644 --- a/website/content/docs/secrets/transform/index.mdx +++ b/website/content/docs/secrets/transform/index.mdx @@ -61,7 +61,9 @@ management tool. ```text $ vault write transform/template/ccn \ type=regex \ - pattern='(\d{4})-(\d{4})-(\d{4})-(\d{4})' \ + pattern='(\d{4})[- ](\d{4})[- ](\d{4})[- ](\d{4})' \ + encode_format='$1-$2-$3-$4' \ + decode_formats=last-four='$4' \ alphabet=numerics Success! Data written to: transform/template/ccn ``` @@ -106,6 +108,21 @@ values. transformation. A tweak must be provided if the tweak source for the transformation is "supplied" or "generated". +1. Decode some input value using the `/decode` endpoint with a named role and decode format: + + ```text + $ vault write transform/decode/payments/last-four value=9300-3376-4943-8903 + Key Value + --- ----- + decoded_value 4444 + ``` + + A transformation must be provided if the role contains more than one + transformation. A tweak must be provided if the tweak source for the + transformation is "supplied" or "generated". A decode format can optionally + be provided. If one isn't provided, the decoded output will be formatted to + match the template's pattern as in the previous example. + ## Roles, Transformations, Templates, and Alphabets The Transform secrets engine contains several types of resources that @@ -250,7 +267,7 @@ for external storage. #### Mapping Modes [Tokenization](transform/tokenization) stores the results of an encode operation -in storage using a cryptographic construct that enhances the safety of its values. +in storage using a cryptographic construct that enhances the safety of its values. In the `default` mapping mode, the token itself is transformed via a one way function involving the transform key and elements of the token. As Vault does not store the token, the values in Vault storage themselves cannot be used to