Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs with Transform FPE advanced I/O handling features #12744

Merged
merged 4 commits into from
Oct 15, 2021
Merged

Update docs with Transform FPE advanced I/O handling features #12744

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c00d898
Update docs with Trasnform FPE advanced I/O handling features.
victorr Oct 5, 2021
5cb8ee1
FIXUP: PR review fixes:
victorr Oct 6, 2021
24e80a0
FIXUP: PR fixes #2: examples for encode/decode formats, etc.
victorr Oct 8, 2021
14efaaa
FIXUP: PR review fixes #3: final documentation tweaks.
victorr Oct 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 81 additions & 30 deletions website/content/api-docs/secret/transform.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ configuration endpoints, and will be removed in a future release.
```json
{
"type": "fpe",
"template": "builtin/creditcardnumber",
"template": "example-template",
"tweak_source": "internal",
"allowed_roles": ["example-role"]
}
Expand Down Expand Up @@ -237,7 +237,7 @@ transformation exists, it will be updated with the new attributes.

```json
{
"template": "builtin/creditcardnumber",
"template": "example-template",
"tweak_source": "internal",
"allowed_roles": ["example-role"]
}
Expand Down Expand Up @@ -381,7 +381,7 @@ $ curl \
{
"data": {
"allowed_roles": ["example-role"],
"templates": ["builtin/creditcardnumber"],
"templates": ["example-template"],
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious— why remove the reference to the builtin template?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That change does look unwarranted. I'll undo it along with the other two other cases where I did the same.

"tweak_source": "internal",
"type": "fpe"
}
Expand Down Expand Up @@ -468,13 +468,26 @@ template exists, it will be updated with the new attributes.
Specifies the name of the alphabet to use when this template is used for FPE
encoding and decoding operations.

- `encode_format` `(string)` -
The regular expression template to use to format encoded values. This is only
used during FPE transformations.

- `decode_formats` `(key-value-map: {})` -
An optional map of regular expression templates that can be used to customize
decoded output. This is only used during FPE transformations.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a quick use-case-based example here, such as "customize decoded output. For example, this can be used to decode only the last four digits of a credit card number. This is only used during FPE transformations."

Copy link
Contributor Author

@victorr victorr Oct 7, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent idea, I'll add it.
Update: added.


### Sample Payload

```json
{
"type": "regex",
"pattern": "(\\d{9})",
"alphabet": "builtin/numeric"
"alphabet": "builtin/numeric",
"pattern": "(\\d{3})[-/](\\d{2})[-/](\\d{4})",
"encode_format": "$1-$2-$3",
"decode_formats": {
"first-three": "$1",
"last-four": "$3"
}
}
```

Expand Down Expand Up @@ -513,7 +526,12 @@ $ curl \
{
"data": {
"alphabet": "builtin/numeric",
"pattern": "(\\d{9})",
"decode_formats": {
"first-three": "$1",
"last-four": "$3"
},
"encode_format": "$1-$2-$3",
"pattern": "(\\d{3})[-/](\\d{2})[-/](\\d{4})",
"type": "regex"
}
}
Expand Down Expand Up @@ -734,7 +752,7 @@ The database user configured here should only have permission to `SELECT`,
The maximum number of idle connections to the database at any given time.

- `max_connection_lifetime` `(duration: 0)` -
The maximum amount of time a connection can be open before closing it.
The maximum amount of time a connection can be open before closing it.
0 means no limit.

### Sample Payloads
Expand Down Expand Up @@ -934,16 +952,16 @@ This endpoint encodes the provided value using a named role.
transformations with `supplied` as the tweak source. The tweak must be a
7-byte value that is then base64 encoded.

- `reference` `(string: "")` -
- `reference` `(string: "")` -
Only valid on batch requests using 'batch_input' below. A user supplied
string that will be present on the corresponding item in the batch
string that will be present on the corresponding item in the batch
response, to assist in correlating inputs with outputs.

- `batch_input` `(array<object>: nil)` -
Specifies a list of items to be encoded in a single batch. When this
parameter is set, the 'value', 'transformation', 'ttl', 'tweak' and
'reference' parameters are ignored. Instead, the aforementioned parameters
should be provided within each object in the list.
'reference' parameters are ignored. Instead, the aforementioned parameters
should be provided within each object in the list.

```json
[
Expand Down Expand Up @@ -1048,16 +1066,20 @@ $ curl \

This endpoint decodes the provided value using a named role.

| Method | Path |
| :----- | :----------------------------- |
| `POST` | `/transform/decode/:role_name` |
| Method | Path |
| :----- | :---------------------------------------------- |
| `POST` | `/transform/decode/:role_name(/:decode_format)` |

### Parameters

- `role_name` `(string: <required>)` –
Specifies the role name to use for this operation. This is specified as part
of the URL.

- `decode_format` `(string)` -
victorr marked this conversation as resolved.
Show resolved Hide resolved
The name of the decode format to use for decoding. If one is not specified,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add some extra information here if you want? For example:

"The name of the decode format to use for decoding. These is defined in decode_formats when creating the transformation, and can be used to selectively decode or format the output. If one is not defined or specified, the template's pattern will be used. Only applicable for FPE transformations."

Just a suggestion, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great suggestion. I will use the text with one modification: "... when creating the transformation's template".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

the template's pattern will be used. Only applicable for FPE transformations.

- `value` `(string: <required>)` –
Specifies the value to be decoded.

Expand All @@ -1071,16 +1093,16 @@ This endpoint decodes the provided value using a named role.
applicable for FPE transformations with `supplied` or `generated` as the tweak
source. The tweak must be a 7-byte value that is then base64 encoded.

- `reference` `(string: "")` -
- `reference` `(string: "")` -
Only valid on batch requests using 'batch_input' below. A user supplied
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: user supplied should be user-supplied

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

string that will be present on the corresponding item in the batch
string that will be present on the corresponding item in the batch
response, to assist in correlating inputs with outputs.

- `batch_input` `(array<object>: nil)` -
Specifies a list of items to be encoded in a single batch. When this
victorr marked this conversation as resolved.
Show resolved Hide resolved
parameter is set, the 'value', 'transformation', 'tweak' and
'reference' parameters are ignored. Instead, the aforementioned parameters
should be provided within each object in the list.
'reference' parameters are ignored. Instead, the aforementioned parameters
should be provided within each object in the list.

```json
[
Expand All @@ -1093,10 +1115,39 @@ This endpoint decodes the provided value using a named role.

### Sample Payload

```json
{
"value": "418-56-4374",
"transformation": "example-transformation"
}
```

### Sample Request

```shell-session
$ curl \
--header "X-Vault-Token: ..." \
--request POST \
--data @payload.json \
http://127.0.0.1:8200/v1/transform/decode/example-role
```

### Sample Response

```json
{
"value": "5682-4613-6822-8064",
"transformation": "ccn-fpe"
"data": {
"decoded_value": "111-22-3333"
}
}
```

### Sample Payload

```json
{
"value": "418-56-4374",
"transformation": "example-transformation"
}
```

Expand All @@ -1107,15 +1158,15 @@ $ curl \
--header "X-Vault-Token: ..." \
--request POST \
--data @payload.json \
http://127.0.0.1:8200/v1/transform/encode/example-role
http://127.0.0.1:8200/v1/transform/decode/example-role/last-four
```

### Sample Response

```json
{
"data": {
"decoded_value": "1111-2222-3333-4444"
"decoded_value": "4444"
}
}
```
Expand Down Expand Up @@ -1190,16 +1241,16 @@ Only valid for tokenization transformations.
may be skipped and will be inferred. If multiple transformations exist, one
must be specified.

- `reference` `(string: "")` -
- `reference` `(string: "")` -
Only valid on batch requests using 'batch_input' below. A user supplied
string that will be present on the corresponding item in the batch
string that will be present on the corresponding item in the batch
response, to assist in correlating inputs with outputs.

- `batch_input` `(array<object>: nil)` -
Specifies a list of items to be validated in a single batch. When this
parameter is set, the 'value', 'transformation' and
'reference' parameters are ignored. Instead, the aforementioned parameters
should be provided within each object in the list.
'reference' parameters are ignored. Instead, the aforementioned parameters
should be provided within each object in the list.

```json
[
Expand Down Expand Up @@ -1265,9 +1316,9 @@ transformations.
may be skipped and will be inferred. If multiple transformations exist, one
must be specified.

- `reference` `(string: "")` -
- `reference` `(string: "")` -
Only valid on batch requests using 'batch_input' below. A user supplied
string that will be present on the corresponding item in the batch
string that will be present on the corresponding item in the batch
response, to assist in correlating inputs with outputs.

- `batch_input` `(array<object>: nil)` -
Expand Down Expand Up @@ -1339,9 +1390,9 @@ Only valid for tokenization transformations.
may be skipped and will be inferred. If multiple transformations exist, one
must be specified.

- `reference` `(string: "")` -
- `reference` `(string: "")` -
Only valid on batch requests using 'batch_input' below. A user supplied
string that will be present on the corresponding item in the batch
string that will be present on the corresponding item in the batch
response, to assist in correlating inputs with outputs.

- `batch_input` `(array<object>: nil)` -
Expand Down
20 changes: 18 additions & 2 deletions website/content/docs/secrets/transform/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ management tool.
```text
$ vault write transform/template/ccn \
type=regex \
pattern='(\d{4})-(\d{4})-(\d{4})-(\d{4})' \
pattern='(\d{4})[- ](\d{4})[- ](\d{4})[- ](\d{4})' \
encode_format='$1-$2-$3-$4' \
decode_formats=last-four='$4' \
alphabet=numerics
Success! Data written to: transform/template/ccn
```
Expand Down Expand Up @@ -106,6 +108,20 @@ values.
transformation. A tweak must be provided if the tweak source for the
transformation is "supplied" or "generated".

1. Decode some input value using the `/decode` endpoint with a named role and decode format:

```text
$ vault write transform/decode/payments/last-four value=9300-3376-4943-8903
Key Value
--- -----
decoded_value 4444
```

A transformation must be provided if the role contains more than one
transformation. A tweak must be provided if the tweak source for the
transformation is "supplied" or "generated". A decode format can optionally
be provided.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth adding here as well what happens if no decode format is provided.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a sentence explaining it.


## Roles, Transformations, Templates, and Alphabets

The Transform secrets engine contains several types of resources that
Expand Down Expand Up @@ -250,7 +266,7 @@ for external storage.
#### Mapping Modes

[Tokenization](transform/tokenization) stores the results of an encode operation
in storage using a cryptographic construct that enhances the safety of its values.
in storage using a cryptographic construct that enhances the safety of its values.
In the `default` mapping mode, the token itself is transformed via a one way
function involving the transform key and elements of the token. As Vault does
not store the token, the values in Vault storage themselves cannot be used to
Expand Down