Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON processor documentation #5982

Merged
merged 19 commits into from
Jun 5, 2024
Merged

Add JSON processor documentation #5982

merged 19 commits into from
Jun 5, 2024

Conversation

vagimeli
Copy link
Contributor

Description

Closes content gap

Issues Resolved

#4647

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@vagimeli vagimeli added Needs SME Waiting on input from subject matter expert Content gap labels Dec 22, 2023
@vagimeli vagimeli self-assigned this Dec 22, 2023
@vagimeli
Copy link
Contributor Author

@hdhalter We need a dev to take the first pass at drafting this content. I can support them in refining the content and getting it through the documentation process. The ingest processor template is provided with this PR, so it should help the dev get started. Thanks!

@vagimeli vagimeli marked this pull request as draft February 15, 2024 21:14
@vagimeli vagimeli changed the title [DOC] Add JSON processor documentation Add JSON processor documentation May 22, 2024
@vagimeli vagimeli added the backport 2.14 PR: Backport label for 2.14 label May 22, 2024
@vagimeli vagimeli marked this pull request as ready for review May 22, 2024 16:42
@vagimeli vagimeli removed the Needs SME Waiting on input from subject matter expert label May 29, 2024
@vagimeli vagimeli added the 3 - Tech review PR: Tech review in progress label May 29, 2024
@vagimeli
Copy link
Contributor Author

@gaobinlong This PR is ready for your technical review. Thank you.

vagimeli added 2 commits May 30, 2024 08:57
Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>
_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
"json": {
"field": "raw_data",
"target_field": "parsed_data",
"add_to_root": false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add_to_root is not needed here because it defaults to false.

Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vagimeli Please see my comments/changes and tag me for approval on lines 10 and 34. Thanks!


# JSON processor

The `json` processor serializes a string-valued field into a map of maps, which can be useful for various data processing and enrichment tasks.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "string-valued" the right phrasing here, or should it just be "string value field"?

_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
Parameter | Required/Optional | Description |
|-----------|-----------|-----------|
`field` | Required | The name of the field containing the JSON-formatted string to be deserialized.
`target_field` | Optional | The name of the field where the deserialized JSON data will be stored. If `target_field` is unspecified, the value is stored in field. If `target_field` exists, it is overwritten.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last two sentences here have me a bit confused. Can we check for clarity/accuracy?

_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
vagimeli and others added 4 commits June 5, 2024 08:38
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
_ingest-pipelines/processors/json.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vagimeli LGTM!

@vagimeli vagimeli removed the 3 - Tech review PR: Tech review in progress label Jun 5, 2024
@vagimeli vagimeli merged commit 4b99619 into main Jun 5, 2024
5 checks passed
@vagimeli vagimeli deleted the geo-json branch June 5, 2024 14:50
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 5, 2024
* Add JSON processor documentation

Signed-off-by: Melissa Vagi <[email protected]>

* Add pipeline examples

Signed-off-by: Melissa Vagi <[email protected]>

* Add parameters

Signed-off-by: Melissa Vagi <[email protected]>

* Update parameters

Signed-off-by: Melissa Vagi <[email protected]>

* Update json.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update json.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Address tech review feedback

Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/json.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/json.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/json.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/json.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/json.md

Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/json.md

Signed-off-by: Melissa Vagi <[email protected]>

---------

Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
(cherry picked from commit 4b99619)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.14 PR: Backport label for 2.14 Content gap
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants