From 2c81bb6d218d40c6cd1a6fc37bfbb2e50483ec81 Mon Sep 17 00:00:00 2001 From: gaobinlong Date: Mon, 15 Apr 2024 16:00:47 +0800 Subject: [PATCH 1/7] Add rename ingest processor Signed-off-by: gaobinlong --- .../processors/index-processors.md | 1 + _ingest-pipelines/processors/rename.md | 141 ++++++++++++++++++ 2 files changed, 142 insertions(+) create mode 100644 _ingest-pipelines/processors/rename.md diff --git a/_ingest-pipelines/processors/index-processors.md b/_ingest-pipelines/processors/index-processors.md index 60fcac82e2..4f462e4dde 100644 --- a/_ingest-pipelines/processors/index-processors.md +++ b/_ingest-pipelines/processors/index-processors.md @@ -54,6 +54,7 @@ Processor type | Description `pipeline` | Runs an inner pipeline. `remove` | Removes fields from a document. `remove_by_pattern` | Removes fields from a document by field pattern. +`rename` | Rename an existing field. `script` | Runs an inline or stored script on incoming documents. `set` | Sets the value of a field to a specified value. `sort` | Sorts the elements of an array in ascending or descending order. diff --git a/_ingest-pipelines/processors/rename.md b/_ingest-pipelines/processors/rename.md new file mode 100644 index 0000000000..386e8b527c --- /dev/null +++ b/_ingest-pipelines/processors/rename.md @@ -0,0 +1,141 @@ +--- +layout: default +title: Rename +parent: Ingest processors +nav_order: 230 +redirect_from: + - /api-reference/ingest-apis/processors/rename/ +--- + +# Rename processor + +The `rename` processor is used to rename an existing field, which can also be used to move a field from one object to other object or the root level. + +## Syntax + +The following is the syntax for the `rename` processor: + +```json +{ + "rename": { + "field": "field_name", + "target_field" : "target_field_name" + } +} +``` +{% include copy-curl.html %} + +## Configuration parameters + +The following table lists the required and optional parameters for the `rename` processor. + +| Parameter | Required/Optional | Description | +|---|---|---| +`field` | Required | The field name containing the data to be removed. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | +`target_field` | Required | The new name of the field. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | +`ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not contain the specified `field`. If set to `true`, the processor does not modify the document if the `field` does not exist. Default is `false`. | +`description` | Optional | A brief description of the processor. | +`if` | Optional | A condition for running the processor. | +`ignore_failure` | Optional | Specifies whether the processor continues execution even if it encounters errors. If set to `true`, failures are ignored. Default is `false`. | +`on_failure` | Optional | A list of processors to run if the processor fails. | +`tag` | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. | + +## Using the processor + +Follow these steps to use the processor in a pipeline. + +**Step 1: Create a pipeline** + +The following query creates a pipeline, named `rename_field`, that moves a field in a object to the root level: + +```json +PUT /_ingest/pipeline/rename_field +{ + "description": "Pipeline that moves a field to the root level.", + "processors": [ + { + "rename": { + "field": "message.content", + "target_field": "content" + } + } + ] +} +``` +{% include copy-curl.html %} + +**Step 2 (Optional): Test the pipeline** + +It is recommended that you test your pipeline before you ingest documents. +{: .tip} + +To test the pipeline, run the following query: + +```json +POST _ingest/pipeline/rename_field/_simulate +{ + "docs": [ + { + "_index": "testindex1", + "_id": "1", + "_source":{ + "message": { + "type": "nginx", + "content": "192.168.1.10 - - [03/Nov/2023:15:20:45 +0000] \"POST /login HTTP/1.1\" 200 3456" + } + } + } + ] +} +``` +{% include copy-curl.html %} + +**Response** + +The following example response confirms that the pipeline is working as expected: + +```json +{ + "docs": [ + { + "doc": { + "_index": "testindex1", + "_id": "1", + "_source": { + "message": { + "type": "nginx", + }, + "content": """192.168.1.10 - - [03/Nov/2023:15:20:45 +0000] "POST /login HTTP/1.1" 200 3456""" + }, + "_ingest": { + "timestamp": "2024-04-15T07:54:16.010447Z" + } + } + } + ] +} +``` + +**Step 3: Ingest a document** + +The following query ingests a document into an index named `testindex1`: + +```json +PUT testindex1/_doc/1?pipeline=rename_field +{ + "message": { + "type": "nginx", + "content": "192.168.1.10 - - [03/Nov/2023:15:20:45 +0000] \"POST /login HTTP/1.1\" 200 3456" + } +} +``` +{% include copy-curl.html %} + +**Step 4 (Optional): Retrieve the document** + +To retrieve the document, run the following query: + +```json +GET testindex1/_doc/1 +``` +{% include copy-curl.html %} From 79b105a2a46d09c22d31ab72e6e783eb1587e520 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 16 Apr 2024 12:08:04 -0600 Subject: [PATCH 2/7] Update rename.md Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/rename.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/rename.md b/_ingest-pipelines/processors/rename.md index 386e8b527c..b089c98cd5 100644 --- a/_ingest-pipelines/processors/rename.md +++ b/_ingest-pipelines/processors/rename.md @@ -9,7 +9,7 @@ redirect_from: # Rename processor -The `rename` processor is used to rename an existing field, which can also be used to move a field from one object to other object or the root level. +The `rename` processor is used to rename an existing field, which can also be used to move a field from one object to another object or the root level. ## Syntax From dacebb110fcba256d1dee28df5394ccc5c891bbf Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 16 Apr 2024 12:23:37 -0600 Subject: [PATCH 3/7] Update index-processors.md Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/index-processors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/index-processors.md b/_ingest-pipelines/processors/index-processors.md index 4f462e4dde..0c6f2ca4a8 100644 --- a/_ingest-pipelines/processors/index-processors.md +++ b/_ingest-pipelines/processors/index-processors.md @@ -54,7 +54,7 @@ Processor type | Description `pipeline` | Runs an inner pipeline. `remove` | Removes fields from a document. `remove_by_pattern` | Removes fields from a document by field pattern. -`rename` | Rename an existing field. +`rename` | Renames an existing field. `script` | Runs an inline or stored script on incoming documents. `set` | Sets the value of a field to a specified value. `sort` | Sorts the elements of an array in ascending or descending order. From 0c69d1d9ac0eff1517f9765ed6d4839584e3c762 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 16 Apr 2024 12:33:43 -0600 Subject: [PATCH 4/7] Update rename.md Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/rename.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_ingest-pipelines/processors/rename.md b/_ingest-pipelines/processors/rename.md index b089c98cd5..fd3488ccbd 100644 --- a/_ingest-pipelines/processors/rename.md +++ b/_ingest-pipelines/processors/rename.md @@ -46,7 +46,7 @@ Follow these steps to use the processor in a pipeline. **Step 1: Create a pipeline** -The following query creates a pipeline, named `rename_field`, that moves a field in a object to the root level: +The following query creates a pipeline named `rename_field` that moves a field in a object to the root level: ```json PUT /_ingest/pipeline/rename_field @@ -66,7 +66,7 @@ PUT /_ingest/pipeline/rename_field **Step 2 (Optional): Test the pipeline** -It is recommended that you test your pipeline before you ingest documents. +It is recommended that you test your pipeline before ingesting documents. {: .tip} To test the pipeline, run the following query: From cf0b8ac119918a74ec8f346a3787a192debafda6 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 17 Apr 2024 09:27:25 -0600 Subject: [PATCH 5/7] Update _ingest-pipelines/processors/rename.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/rename.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/rename.md b/_ingest-pipelines/processors/rename.md index fd3488ccbd..be55757794 100644 --- a/_ingest-pipelines/processors/rename.md +++ b/_ingest-pipelines/processors/rename.md @@ -9,7 +9,7 @@ redirect_from: # Rename processor -The `rename` processor is used to rename an existing field, which can also be used to move a field from one object to another object or the root level. +The `rename` processor is used to rename an existing field, which can also be used to move a field from one object to another object or to the root level. ## Syntax From 14c3277ce60aad67fa1cf5b37432a8e2ed681e00 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 17 Apr 2024 09:27:32 -0600 Subject: [PATCH 6/7] Update _ingest-pipelines/processors/rename.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/rename.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/rename.md b/_ingest-pipelines/processors/rename.md index be55757794..534ba1d8a1 100644 --- a/_ingest-pipelines/processors/rename.md +++ b/_ingest-pipelines/processors/rename.md @@ -36,7 +36,7 @@ The following table lists the required and optional parameters for the `rename` `ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not contain the specified `field`. If set to `true`, the processor does not modify the document if the `field` does not exist. Default is `false`. | `description` | Optional | A brief description of the processor. | `if` | Optional | A condition for running the processor. | -`ignore_failure` | Optional | Specifies whether the processor continues execution even if it encounters errors. If set to `true`, failures are ignored. Default is `false`. | +`ignore_failure` | Optional | Specifies whether the processor continues execution even if it encounters an error. If set to `true`, failures are ignored. Default is `false`. | `on_failure` | Optional | A list of processors to run if the processor fails. | `tag` | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. | From 1680cab0858d1276f27053af08857b7e6ac37584 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 17 Apr 2024 09:27:40 -0600 Subject: [PATCH 7/7] Update _ingest-pipelines/processors/rename.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/rename.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/rename.md b/_ingest-pipelines/processors/rename.md index 534ba1d8a1..d839c1a213 100644 --- a/_ingest-pipelines/processors/rename.md +++ b/_ingest-pipelines/processors/rename.md @@ -46,7 +46,7 @@ Follow these steps to use the processor in a pipeline. **Step 1: Create a pipeline** -The following query creates a pipeline named `rename_field` that moves a field in a object to the root level: +The following query creates a pipeline named `rename_field` that moves a field in an object to the root level: ```json PUT /_ingest/pipeline/rename_field