add copy_from parameter for set ingest processor

[gaobinlong]

Relates to #55682 and #51046, and refractor the code of #56985.

The main point of this PR is adding a copy_from parameter for set ingest processor to support copy the value of one field to another. We can copy boolean, number, array, object and other data types by the parameter copy_from instead of using template snippet which only produces string value.

Why adding a new parameter to implement the copy function but not modifying the template snippet so that it can produce object or array value ? I think we can make it compatible with older versions, users can still use template snippet to copy the produced string value to other field, but also can use copy_from to copy object or array value from one field to another.

The main changes are:

1. Add copy_from parameter for set ingest processor.
2. Add unit test and yml test for the change.
3. Update the set processor's document.

[danhermann]

Why adding a new parameter to implement the copy function but not modifying the template snippet so that it can produce object or array value ? I think we can make it compatible with older versions, users can still use template snippet to copy the produced string value to other field, but also can use copy_from to copy object or array value from one field to another.

Would it be possible to modify the template snippet code to produce object and array values? Our team discussed this and couldn't think of any good reason for producing string values. We would release this feature in the next major version of ES so maintaining compatibility with older versions is not required.

[gaobinlong]

@danhermann, the usage of template snippet is flexible, for example, "{{field_a}} {{field_b}}" is used to concatenate the value of field_a and field_b, but if field_a and field_b are not string, what's the result after executing the template snippet if template snippet can produce object or array value? I'm not clear about that. Another example is that {{.}} is valid currently which will output the whole document. So maybe specifying a field by the parameter copy_from can simply copy the value from one filed to another without handing some complex situation. In addition, maybe it's not easy to modify the template snippet code to produce object and array values, from the source code: MustacheScriptEngine, I found that the MustacheScriptEngine only produces string value. Expect your opinion on this.

[danhermann]

@danhermann, the usage of template snippet is flexible, for example, "{{field_a}} {{field_b}}" is used to concatenate the value of field_a and field_b, but if field_a and field_b are not string, what's the result after executing the template snippet if template snippet can produce object or array value? I'm not clear about that. Another example is that {{.}} is valid currently which will output the whole document. So maybe specifying a field by the parameter copy_from can simply copy the value from one filed to another without handing some complex situation. In addition, maybe it's not easy to modify the template snippet code to produce object and array values, from the source code: MustacheScriptEngine, I found that the MustacheScriptEngine only produces string value. Expect your opinion on this.

Thanks, @gaobinlong. Your explanation makes sense and we'd like to move forward with the copy_from parameter as you've implemented it. I'll review this shortly and we'll work on getting it merged.

[danhermann]

@elasticmachine update branch

[danhermann]

@elasticmachine ok to test

[danhermann]

@gaobinlong, from my testing, this looks good and works well. I left some minor wording changes below as well as a request to split up one of the unit tests. I think we can get it merged after that.

[gaobinlong]

@elasticmachine test this please.

[danhermann]

Thanks, @gaobinlong. This looks good. The last failing test is due to the fact that line 120 in SetProcessor.java is now longer than the limit of 140 characters. If you can split the line to keep it under the length limit, I believe that all tests will pass.

[danhermann]

@elasticmachine update branch

[danhermann]

Thanks again for this contribution, @gaobinlong! I'll get it merged and backported.

[elasticmachine]

Pinging @elastic/es-core-features (:Core/Features/Ingest)