Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mutability of rest of document in Ingest-ForeachProcessor sub processors #19592

Closed
talevy opened this issue Jul 26, 2016 · 0 comments
Closed
Assignees
Labels
>bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP

Comments

@talevy
Copy link
Contributor

talevy commented Jul 26, 2016

Spoke with @martijnvg and @BigFunger about some strange behavior around mutated state within sub-processors within ForEach. Realization was that we expose top-level document fields, but silently ignore mutations to them. This is a really confusing behavior.

Currently, the foreach processor makes it possible to read all other fields in the ingest document besides _value. It does not enable mutability of these fields.

This test passes that attempts to append to a top-level field passes. The append does not actually do anything since it operates on a copy. This should be fixed so that the original documents reflects modifications outside of _value.

    public void testExecute() throws Exception {
        List<Object> values = new ArrayList<>();
        values.add("string");
        values.add(1);
        values.add(null);
        IngestDocument ingestDocument = new IngestDocument(
            "_index", "_type", "_id", null, null, null, null, Collections.singletonMap("values", values)
        );

        TemplateService ts = TestTemplateService.instance();

        ForEachProcessor processor = new ForEachProcessor(
            "_tag", "values", new CompoundProcessor(false,
            Collections.singletonList(new UppercaseProcessor("_tag_upper", "_value")),
            Collections.singletonList(new AppendProcessor("_tag", ts.compile("errors"), (model) -> (Collections.singletonList("added"))))
        ));
        processor.execute(ingestDocument);

        List<String> result = ingestDocument.getFieldValue("values", List.class);
        assertThat(result.get(0), equalTo("STRING"));
        assertThat(result.get(1), equalTo(1));
        assertThat(result.get(2), equalTo(null));
    }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP
Projects
None yet
Development

No branches or pull requests

2 participants