Skip to content

Commit

Permalink
[DOC] Add ingest error metadata (#75653) (#75657)
Browse files Browse the repository at this point in the history
These docs were accidentally removed in #70253.

Co-authored-by: Dan Hermann <[email protected]>
  • Loading branch information
jrodewig and danhermann authored Jul 23, 2021
1 parent 533a5af commit b4a9831
Showing 1 changed file with 30 additions and 4 deletions.
34 changes: 30 additions & 4 deletions docs/reference/ingest.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ template's index pattern should match `logs-<dataset-name>-*`.
--
You can create this template using {kib}'s <<manage-index-templates,**Index
Management**>> feature or the <<indices-put-template,create index template
API>>.
API>>.

For example, the following request creates a template matching `logs-my_app-*`.
The template uses a component template that contains the
Expand Down Expand Up @@ -552,7 +552,7 @@ PUT _ingest/pipeline/my-pipeline
"description": "Use geo_point dynamic template for address field",
"field": "_dynamic_templates",
"value": {
"address": "geo_point"
"address": "geo_point"
}
}
}
Expand All @@ -562,8 +562,8 @@ PUT _ingest/pipeline/my-pipeline

The set processor above tells ES to use the dynamic template named `geo_point`
for the field `address` if this field is not defined in the mapping of the index
yet. This processor overrides the dynamic template for the field `address` if
already defined in the bulk request, but has no effect on other dynamic
yet. This processor overrides the dynamic template for the field `address` if
already defined in the bulk request, but has no effect on other dynamic
templates defined in the bulk request.

WARNING: If you <<create-document-ids-automatically,automatically generate>>
Expand Down Expand Up @@ -719,6 +719,32 @@ PUT _ingest/pipeline/my-pipeline
----
// TEST[s/\.\.\./{"lowercase": {"field":"my-keyword-field"}}/]

Additional information about the pipeline failure may be available in the
document metadata fields `on_failure_message`, `on_failure_processor_type`,
`on_failure_processor_tag`, and `on_failure_pipeline`. These fields are
accessible only from within an `on_failure` block.

The following example uses the error metadata fields to provide additional
information on the document about the failure.

[source,console]
----
PUT _ingest/pipeline/my-pipeline
{
"processors": [ ... ],
"on_failure": [
{
"set": {
"description": "Record error information",
"field": "error_information",
"value": "Processor {{ _ingest.on_failure_processor_type }} with tag {{ _ingest.on_failure_processor_tag }} in pipeline {{ _ingest.on_failure_pipeline }} failed with message {{ _ingest.on_failure_message }}"
}
}
]
}
----
// TEST[s/\.\.\./{"lowercase": {"field":"my-keyword-field"}}/]

[discrete]
[[conditionally-run-processor]]
=== Conditionally run a processor
Expand Down

0 comments on commit b4a9831

Please sign in to comment.