From 887050120fa0ad5874ba12f08c443d84878b1788 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Fri, 22 Dec 2023 13:23:15 -0700 Subject: [PATCH 01/13] Add JSON processor documentation Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 86 ++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) create mode 100644 _ingest-pipelines/processors/json.md diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md new file mode 100644 index 0000000000..558f2bf33e --- /dev/null +++ b/_ingest-pipelines/processors/json.md @@ -0,0 +1,86 @@ +--- +layout: default +title: JSON +parent: Ingest processors +nav_order: 170 +--- + +# JSON processor + +The `json` processor is used to . + +The following is the syntax for the `json` processor: + +```json + +``` +{% include copy-curl.html %} + +## Configuration parameters + +The following table lists the required and optional parameters for the `json` processor. + +Parameter | Required/Optional | Description | +|-----------|-----------|-----------| + + +## Using the processor + +Follow these steps to use the processor in a pipeline. + +### Step 1: Create a pipeline + +The following query creates a pipeline, named , that uses the `json` processor to : + +```json + +``` +{% include copy-curl.html %} + +### Step 2 (Optional): Test the pipeline + +It is recommended that you test your pipeline before you ingest documents. +{: .tip} + +To test the pipeline, run the following query: + +```json + +``` +{% include copy-curl.html %} + +#### Response + +The following example response confirms that the pipeline is working as expected: + +```json + +``` + +### Step 3: Ingest a document + +The following query ingests a document into an index named `testindex1`: + +```json + +``` +{% include copy-curl.html %} + +#### Response + +The request indexes the document into the index and will index all documents with . + +```json + +``` + +### Step 4 (Optional): Retrieve the document + +To retrieve the document, run the following query: + +```json + +``` +{% include copy-curl.html %} + + \ No newline at end of file From 0d189f6faf9e45fcbc5f0df4b5a0b0161666038d Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 22 May 2024 12:11:09 -0600 Subject: [PATCH 02/13] Add pipeline examples Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 107 ++++++++++++++++++++++++--- 1 file changed, 96 insertions(+), 11 deletions(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index 558f2bf33e..de4bf3d827 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -7,12 +7,22 @@ nav_order: 170 # JSON processor -The `json` processor is used to . +The `json` processor is used to parse and extract data from JSON-formatted documents. It can be used to flatten nested JSON structures, rename fields, and perform other transformations on the JSON data. The following is the syntax for the `json` processor: ```json - +{ + "description": "...", + "processors": [ + { + "json": { + "field": "message", + "add_to_root": true + } + } + ] +} ``` {% include copy-curl.html %} @@ -30,10 +40,21 @@ Follow these steps to use the processor in a pipeline. ### Step 1: Create a pipeline -The following query creates a pipeline, named , that uses the `json` processor to : +The following query creates a pipeline named `json-pipeline` that uses the `json` processor to flatten a nested JSON structure in the message field:: ```json - +PUT _ingest/pipeline/json-pipeline +{ + "description": "Flattens nested JSON data in the message field", + "processors": [ + { + "json": { + "field": "message", + "add_to_root": true + } + } + ] +} ``` {% include copy-curl.html %} @@ -45,7 +66,16 @@ It is recommended that you test your pipeline before you ingest documents. To test the pipeline, run the following query: ```json - +POST _ingest/pipeline/json-pipeline/_simulate +{ + "docs": [ + { + "_source": { + "message": "{\"user\":{\"name\":\"John Doe\",\"age\":32}}" + } + } + ] +} ``` {% include copy-curl.html %} @@ -54,33 +84,88 @@ To test the pipeline, run the following query: The following example response confirms that the pipeline is working as expected: ```json - +{ + "docs": [ + { + "doc": { + "_index": "_index", + "_id": "_id", + "_source": { + "message": """{"user":{"name":"John Doe","age":32}}""", + "user": { + "name": "John Doe", + "age": 32 + } + }, + "_ingest": { + "timestamp": "2024-05-22T18:07:27.269027084Z" + } + } + } + ] +} ``` +{% include copy-curl.html %} ### Step 3: Ingest a document The following query ingests a document into an index named `testindex1`: ```json - +PUT testindex1/_doc/1?pipeline=json-pipeline +{ + "message": "{\"user\":{\"name\":\"Jane Smith\",\"age\":28}}" +} ``` {% include copy-curl.html %} #### Response -The request indexes the document into the index and will index all documents with . +The request indexes the document into the index and will index all documents with the flattened JSON data from the message field.. ```json - +{ + "_index": "testindex1", + "_id": "1", + "_version": 1, + "result": "created", + "_shards": { + "total": 2, + "successful": 1, + "failed": 0 + }, + "_seq_no": 0, + "_primary_term": 1 +} ``` +{% include copy-curl.html %} ### Step 4 (Optional): Retrieve the document To retrieve the document, run the following query: ```json - +GET testindex1/_doc/1 ``` {% include copy-curl.html %} - \ No newline at end of file +#### Response + +```json +{ + "_index": "testindex1", + "_id": "1", + "_version": 1, + "_seq_no": 0, + "_primary_term": 1, + "found": true, + "_source": { + "message": """{"user":{"name":"Jane Smith","age":28}}""", + "user": { + "name": "Jane Smith", + "age": 28 + } + } +} +``` +{% include copy-curl.html %} From bb5c62feb2b9440e2bb24c7929aa9c2b734b2a12 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 22 May 2024 13:28:13 -0600 Subject: [PATCH 03/13] Add parameters Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index de4bf3d827..6a89db632c 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -32,7 +32,17 @@ The following table lists the required and optional parameters for the `json` pr Parameter | Required/Optional | Description | |-----------|-----------|-----------| - +`field` | Required | The field containing the JSON data to be parsed. +`target_field` | Optional | The field where the parsed JSON data should be stored. If not specified, the data is stored in the `field` specified. Any existing data in the `target_field` is overwritten. +`add_to_root` | Optional | If `true`, the parsed JSON data is added to the root of the document. +`add_to_root_conflict_strategy` | Optional | Specifies how to handle conflicts when adding parsed JSON data to the root of the document. If set to `replace` (default), root fields that conflict with fields from the parsed JSON are overwritten. If set to `merge`, conflicting fields are merged. This parameter is only applicable when `add_to_root` is set to `true`. +`allow_duplicate_keys` | Optional | If `true`, the JSON parser does not fail if the JSON contains duplicate keys. Instead, the parser handles the duplicate keys by keeping the last encountered value for any given key. +`strict_json_parsing` | Optional | Controls the behavior of the JSON parser when processing the field value. When set to `true` (the default), the JSON parser strictly parses the `field` value, which means that the parser enforces strict JSON syntax rules and throws an error if the `field` value does not conform to the expected JSON format. +`description` | Optional | A brief description of the processor. +`if` | Optional | A condition for running the processor. +`ignore_failure` | Optional | Specifies whether the processor continues execution even if it encounters errors. If set to `true`, failures are ignored. Default is `false`. +`on_failure`| Optional | A list of processors to run if the processor fails. +`tag` | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. ## Using the processor From 3fc0a4b48a8ad7440c8ed6e54fb84dd58361e8a0 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 28 May 2024 13:14:29 -0600 Subject: [PATCH 04/13] Update parameters Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index 6a89db632c..e40d134d91 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -35,9 +35,6 @@ Parameter | Required/Optional | Description | `field` | Required | The field containing the JSON data to be parsed. `target_field` | Optional | The field where the parsed JSON data should be stored. If not specified, the data is stored in the `field` specified. Any existing data in the `target_field` is overwritten. `add_to_root` | Optional | If `true`, the parsed JSON data is added to the root of the document. -`add_to_root_conflict_strategy` | Optional | Specifies how to handle conflicts when adding parsed JSON data to the root of the document. If set to `replace` (default), root fields that conflict with fields from the parsed JSON are overwritten. If set to `merge`, conflicting fields are merged. This parameter is only applicable when `add_to_root` is set to `true`. -`allow_duplicate_keys` | Optional | If `true`, the JSON parser does not fail if the JSON contains duplicate keys. Instead, the parser handles the duplicate keys by keeping the last encountered value for any given key. -`strict_json_parsing` | Optional | Controls the behavior of the JSON parser when processing the field value. When set to `true` (the default), the JSON parser strictly parses the `field` value, which means that the parser enforces strict JSON syntax rules and throws an error if the `field` value does not conform to the expected JSON format. `description` | Optional | A brief description of the processor. `if` | Optional | A condition for running the processor. `ignore_failure` | Optional | Specifies whether the processor continues execution even if it encounters errors. If set to `true`, failures are ignored. Default is `false`. From 6bbcb6795e8fd468bfe4cf0a3bd5898dfcaf3dea Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Thu, 30 May 2024 09:14:01 -0600 Subject: [PATCH 05/13] Update json.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 30 +++++++++++++--------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index e40d134d91..42dd869a6c 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -7,21 +7,19 @@ nav_order: 170 # JSON processor -The `json` processor is used to parse and extract data from JSON-formatted documents. It can be used to flatten nested JSON structures, rename fields, and perform other transformations on the JSON data. +The `json` processor serializes a string-valued field into a map of maps, which can be useful for various data processing and enrichment tasks. The following is the syntax for the `json` processor: ```json { - "description": "...", - "processors": [ - { - "json": { - "field": "message", - "add_to_root": true - } + "processor": { + "json": { + "field": "", + "target_field": "", + "add_to_root": } - ] + } } ``` {% include copy-curl.html %} @@ -32,13 +30,13 @@ The following table lists the required and optional parameters for the `json` pr Parameter | Required/Optional | Description | |-----------|-----------|-----------| -`field` | Required | The field containing the JSON data to be parsed. -`target_field` | Optional | The field where the parsed JSON data should be stored. If not specified, the data is stored in the `field` specified. Any existing data in the `target_field` is overwritten. -`add_to_root` | Optional | If `true`, the parsed JSON data is added to the root of the document. -`description` | Optional | A brief description of the processor. -`if` | Optional | A condition for running the processor. -`ignore_failure` | Optional | Specifies whether the processor continues execution even if it encounters errors. If set to `true`, failures are ignored. Default is `false`. -`on_failure`| Optional | A list of processors to run if the processor fails. +`field` | Required | The name of the field containing the JSON-formatted string to be deserialized. +`target_field` | Optional | The name of the field where the deserialized JSON data will be stored. If this parameter is not provided, the deserialized data is added to the root of the document. +`add_to_root` | Optional | A boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). Default value is `false`. +`description` | Optional | A description of the processor's purpose or configuration. +`if` | Optional | Specifies to conditionally execute the processor. +`ignore_failure` | Optional | Specifies to ignore failures for the processor. See [Handling pipeline failures]({{site.url}}{{site.baseurl}}/ingest-pipelines/pipeline-failures/). +`on_failure`| Optional | Specifies a list of processors to run if the processor fails during execution. These processors are executed in the order they are specified. `tag` | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. ## Using the processor From bbbd4bba46273ad5434733bc3506f894c514e2f3 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Thu, 30 May 2024 10:06:21 -0600 Subject: [PATCH 06/13] Update json.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 108 ++++++++++++++++----------- 1 file changed, 66 insertions(+), 42 deletions(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index 42dd869a6c..530a2bcb0a 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -45,17 +45,37 @@ Follow these steps to use the processor in a pipeline. ### Step 1: Create a pipeline -The following query creates a pipeline named `json-pipeline` that uses the `json` processor to flatten a nested JSON structure in the message field:: +The following query creates a pipeline named `my-json-pipeline` that uses the `json` processor to process JSON data and enrich the documents with additional information: ```json -PUT _ingest/pipeline/json-pipeline +PUT _ingest/pipeline/my-json-pipeline { - "description": "Flattens nested JSON data in the message field", + "description": "Example pipeline using the JsonProcessor", "processors": [ { "json": { - "field": "message", - "add_to_root": true + "field": "raw_data", + "target_field": "parsed_data", + "add_to_root": false, + "on_failure": [ + { + "set": { + "field": "error_message", + "value": "Failed to parse JSON data" + } + }, + { + "fail": { + "message": "Failed to process JSON data" + } + } + ] + } + }, + { + "set": { + "field": "processed_timestamp", + "value": "{{_ingest.timestamp}}" } } ] @@ -71,12 +91,17 @@ It is recommended that you test your pipeline before you ingest documents. To test the pipeline, run the following query: ```json -POST _ingest/pipeline/json-pipeline/_simulate +POST _ingest/pipeline/my-json-pipeline/_simulate { "docs": [ { "_source": { - "message": "{\"user\":{\"name\":\"John Doe\",\"age\":32}}" + "raw_data": "{\"name\":\"John\",\"age\":30,\"city\":\"New York\"}" + } + }, + { + "_source": { + "raw_data": "{\"name\":\"Jane\",\"age\":25,\"city\":\"Los Angeles\"}" } } ] @@ -96,14 +121,34 @@ The following example response confirms that the pipeline is working as expected "_index": "_index", "_id": "_id", "_source": { - "message": """{"user":{"name":"John Doe","age":32}}""", - "user": { - "name": "John Doe", - "age": 32 + "processed_timestamp": "2024-05-30T15:24:48.064472090Z", + "raw_data": """{"name":"John","age":30,"city":"New York"}""", + "parsed_data": { + "name": "John", + "city": "New York", + "age": 30 } }, "_ingest": { - "timestamp": "2024-05-22T18:07:27.269027084Z" + "timestamp": "2024-05-30T15:24:48.06447209Z" + } + } + }, + { + "doc": { + "_index": "_index", + "_id": "_id", + "_source": { + "processed_timestamp": "2024-05-30T15:24:48.064543006Z", + "raw_data": """{"name":"Jane","age":25,"city":"Los Angeles"}""", + "parsed_data": { + "name": "Jane", + "city": "Los Angeles", + "age": 25 + } + }, + "_ingest": { + "timestamp": "2024-05-30T15:24:48.064543006Z" } } } @@ -114,24 +159,24 @@ The following example response confirms that the pipeline is working as expected ### Step 3: Ingest a document -The following query ingests a document into an index named `testindex1`: +The following query ingests a document into an index named `my-index`: ```json -PUT testindex1/_doc/1?pipeline=json-pipeline +POST my-index/_doc?pipeline=my-json-pipeline { - "message": "{\"user\":{\"name\":\"Jane Smith\",\"age\":28}}" + "raw_data": "{\"name\":\"John\",\"age\":30,\"city\":\"New York\"}" } ``` {% include copy-curl.html %} #### Response -The request indexes the document into the index and will index all documents with the flattened JSON data from the message field.. +This response confirms that the document with the JSON data from the `raw_data` field was successfully indexed: ```json { - "_index": "testindex1", - "_id": "1", + "_index": "my-index", + "_id": "mo8yyo8BwFahnwl9WpxG", "_version": 1, "result": "created", "_shards": { @@ -139,8 +184,8 @@ The request indexes the document into the index and will index all "successful": 1, "failed": 0 }, - "_seq_no": 0, - "_primary_term": 1 + "_seq_no": 3, + "_primary_term": 2 } ``` {% include copy-curl.html %} @@ -150,27 +195,6 @@ The request indexes the document into the index and will index all To retrieve the document, run the following query: ```json -GET testindex1/_doc/1 -``` -{% include copy-curl.html %} - -#### Response - -```json -{ - "_index": "testindex1", - "_id": "1", - "_version": 1, - "_seq_no": 0, - "_primary_term": 1, - "found": true, - "_source": { - "message": """{"user":{"name":"Jane Smith","age":28}}""", - "user": { - "name": "Jane Smith", - "age": 28 - } - } -} +GET my-index/_doc/1 ``` {% include copy-curl.html %} From ceb27320f46e4fc165758e20c44183a63b29e4dd Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 4 Jun 2024 16:13:18 -0600 Subject: [PATCH 07/13] Address tech review feedback Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index 530a2bcb0a..a3be5aca29 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -31,8 +31,8 @@ The following table lists the required and optional parameters for the `json` pr Parameter | Required/Optional | Description | |-----------|-----------|-----------| `field` | Required | The name of the field containing the JSON-formatted string to be deserialized. -`target_field` | Optional | The name of the field where the deserialized JSON data will be stored. If this parameter is not provided, the deserialized data is added to the root of the document. -`add_to_root` | Optional | A boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). Default value is `false`. +`target_field` | Optional | The name of the field where the deserialized JSON data will be stored. If `target_field` is unspecified, the value is stored in field. If `target_field` exists, it is overwritten. +`add_to_root` | Optional | A boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). If `add_to_root` is `true`, then `target-field` is invalid. Default value is `false`. `description` | Optional | A description of the processor's purpose or configuration. `if` | Optional | Specifies to conditionally execute the processor. `ignore_failure` | Optional | Specifies to ignore failures for the processor. See [Handling pipeline failures]({{site.url}}{{site.baseurl}}/ingest-pipelines/pipeline-failures/). @@ -55,8 +55,7 @@ PUT _ingest/pipeline/my-json-pipeline { "json": { "field": "raw_data", - "target_field": "parsed_data", - "add_to_root": false, + "target_field": "parsed_data" "on_failure": [ { "set": { From 058fb48da95593b9646baea71ce41358c0a0d19d Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 5 Jun 2024 08:38:12 -0600 Subject: [PATCH 08/13] Update _ingest-pipelines/processors/json.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index a3be5aca29..7981929101 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -31,7 +31,7 @@ The following table lists the required and optional parameters for the `json` pr Parameter | Required/Optional | Description | |-----------|-----------|-----------| `field` | Required | The name of the field containing the JSON-formatted string to be deserialized. -`target_field` | Optional | The name of the field where the deserialized JSON data will be stored. If `target_field` is unspecified, the value is stored in field. If `target_field` exists, it is overwritten. +`target_field` | Optional | The name of the field in which the deserialized JSON data will be stored. If `target_field` is unspecified, then the value is stored in field. If `target_field` exists, it is overwritten. `add_to_root` | Optional | A boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). If `add_to_root` is `true`, then `target-field` is invalid. Default value is `false`. `description` | Optional | A description of the processor's purpose or configuration. `if` | Optional | Specifies to conditionally execute the processor. From f735d4e8a78ac8f42a0ac4bdd0113ce359f4c805 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 5 Jun 2024 08:38:21 -0600 Subject: [PATCH 09/13] Update _ingest-pipelines/processors/json.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index 7981929101..cb7900fb0d 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -32,7 +32,7 @@ Parameter | Required/Optional | Description | |-----------|-----------|-----------| `field` | Required | The name of the field containing the JSON-formatted string to be deserialized. `target_field` | Optional | The name of the field in which the deserialized JSON data will be stored. If `target_field` is unspecified, then the value is stored in field. If `target_field` exists, it is overwritten. -`add_to_root` | Optional | A boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). If `add_to_root` is `true`, then `target-field` is invalid. Default value is `false`. +`add_to_root` | Optional | A Boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). If `add_to_root` is `true`, then `target-field` is invalid. Default value is `false`. `description` | Optional | A description of the processor's purpose or configuration. `if` | Optional | Specifies to conditionally execute the processor. `ignore_failure` | Optional | Specifies to ignore failures for the processor. See [Handling pipeline failures]({{site.url}}{{site.baseurl}}/ingest-pipelines/pipeline-failures/). From 61ba96e6777bad3573bf9a549cc84a7d7994daf7 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 5 Jun 2024 08:38:28 -0600 Subject: [PATCH 10/13] Update _ingest-pipelines/processors/json.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index cb7900fb0d..ca01396d76 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -35,7 +35,7 @@ Parameter | Required/Optional | Description | `add_to_root` | Optional | A Boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). If `add_to_root` is `true`, then `target-field` is invalid. Default value is `false`. `description` | Optional | A description of the processor's purpose or configuration. `if` | Optional | Specifies to conditionally execute the processor. -`ignore_failure` | Optional | Specifies to ignore failures for the processor. See [Handling pipeline failures]({{site.url}}{{site.baseurl}}/ingest-pipelines/pipeline-failures/). +`ignore_failure` | Optional | Specifies to ignore processor failures. See [Handling pipeline failures]({{site.url}}{{site.baseurl}}/ingest-pipelines/pipeline-failures/). `on_failure`| Optional | Specifies a list of processors to run if the processor fails during execution. These processors are executed in the order they are specified. `tag` | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. From 608dfe96ba492c699ee3748af1a15d37938f149e Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 5 Jun 2024 08:38:34 -0600 Subject: [PATCH 11/13] Update _ingest-pipelines/processors/json.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index ca01396d76..a6b693dc2a 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -170,7 +170,7 @@ POST my-index/_doc?pipeline=my-json-pipeline #### Response -This response confirms that the document with the JSON data from the `raw_data` field was successfully indexed: +The response confirms that the document containing the JSON data from the `raw_data` field was successfully indexed: ```json { From ffc3b6919a64819b82b09d363d4b9e92642d564d Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 5 Jun 2024 08:46:30 -0600 Subject: [PATCH 12/13] Update _ingest-pipelines/processors/json.md Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index a6b693dc2a..e1ef63e764 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -31,7 +31,7 @@ The following table lists the required and optional parameters for the `json` pr Parameter | Required/Optional | Description | |-----------|-----------|-----------| `field` | Required | The name of the field containing the JSON-formatted string to be deserialized. -`target_field` | Optional | The name of the field in which the deserialized JSON data will be stored. If `target_field` is unspecified, then the value is stored in field. If `target_field` exists, it is overwritten. +`target_field` | Optional | The name of the field in which the deserialized JSON data is stored. When not provided, the data is stored in the `field` field. If `target_field` exists, its existing value is overwritten with the new JSON data. `add_to_root` | Optional | A Boolean flag that determines whether the deserialized JSON data should be added to the root of the document (`true`) or stored in the target_field (`false`). If `add_to_root` is `true`, then `target-field` is invalid. Default value is `false`. `description` | Optional | A description of the processor's purpose or configuration. `if` | Optional | Specifies to conditionally execute the processor. From 36ee8ccb7b74cac5ee1e05c3995642c06ba36a50 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Wed, 5 Jun 2024 08:46:37 -0600 Subject: [PATCH 13/13] Update _ingest-pipelines/processors/json.md Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/json.md b/_ingest-pipelines/processors/json.md index e1ef63e764..d251533e27 100644 --- a/_ingest-pipelines/processors/json.md +++ b/_ingest-pipelines/processors/json.md @@ -7,7 +7,7 @@ nav_order: 170 # JSON processor -The `json` processor serializes a string-valued field into a map of maps, which can be useful for various data processing and enrichment tasks. +The `json` processor serializes a string value field into a map of maps, which can be useful for various data processing and enrichment tasks. The following is the syntax for the `json` processor: