Add append option to add_entries processor

[oeyh]

Description

Adds an append_if_key_exists option to add_entries processor as described in #4129

Issues Resolved

Resolves #4129

Check List

• New functionality includes testing.
• New functionality has a documentation issue. Please link to it in this PR.
  • New functionality has javadoc added
• Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[graytaylor0]

Would it make sense to instead have something like this

action_when_key_exists: append //  can be one of `append`, `overwrite`, `skip`

Not a blocking comment just something to consider

[oeyh]

action_when_key_exists: append // can be one of append, overwrite, skip

@graytaylor0 I like it, making the config more concise. I can do the change if other reviewers think the same.

[dlvenable]

overwrite_if_key_exists and append_if_key_exists can not be set at the same time.

Please change to:

overwrite_if_key_exists and append_if_key_exists can not be set to true at the same time.

They can be "set" together.

[dlvenable]

Consider combining these with the use of Java 8 lambdas.

Here is some starting point for one way to do it.

private void mergeValueToEvent(final Event recordEvent, final String key, final Object value)
  mergeValue(recordEvent, () -> recordEvent.getMetadata().getAttribute(key), newValue -> recordEvent.put(key, newValue);
}

mergeValue(Supplier<Object> getter, Consumer<Object> setter) {
          final Object currentValue = getter.get();
        final List<Object> mergedValue = new ArrayList<>();
        if (currentValue instanceof List) {
            mergedValue.addAll((List<Object>) currentValue);
        } else {
            mergedValue.add(currentValue);
        }

        mergedValue.add(value);
        setter.accept(mergedValue);
}

[oeyh]

Thanks for the suggestion!

[dlvenable]

Would it make sense to instead have something like this

action_when_key_exists: append //  can be one of `append`, `overwrite`, `skip`

Not a blocking comment just something to consider

@graytaylor0 , I like this idea. But, I also think we should aim for consistency. Right now grok has overwrite_when_key_exists. Do you think grok could also move toward using this approach? I'd really like to see all the processors use similar behavior. And I think this enum could facilitate that.

[oeyh]

These processors have similar overwrite_if_key_exists options:

• copy_values
• rename_keys
• parse_json
• parse_ion
• key_value

grok processor has a keys_to_overwrite option so that you can determine to overwrite or append (default) for each grok generated field.

I will open a follow-up issue for these and we can discuss further there.