Emulate "fields" option on older versions

[cbuescher]

We introduced the new "fields" option in search with version 7.10. With this
change we are trying to do a best-effort attempt at emulating this new behaviour
in mixed cluster or CCS scenarios where search requests using the "fields"
option also target older nodes or clusters. In that case, currently we don't
return anything in the "fields" section of the response.

This change tries to emulate the fields behaviour by modifying the request to
include the respective "_source" fields and then parsing them back into the
"fields" section on return. This will not be fully equivalent to the post-7.10 "fields"
functionality but at least try to include whatever we find in "_source" in earlier
versions.

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[cbuescher]

I think this is ready for a first general view. Known caveats: currently for mixed clusters only the default query_then_fetch search type is supported. While dfs_query_then_fetch is probably possible to implement in a somewhat similar way, it seems a bit more complex and we talked about query_then_fetch being a common enough default to support.
Tests probably need extension too, but at least the basic mixed cluster scenario and a remote CCS cluster on an older version are covered.
Also missing error handling in the case that the original search request contains both "fields" option and a "source" parameter with includes/excludes. I think we should error in this case but left this open for discussion so far.

[cbuescher]

One general open question here is if we should put this bwc emulation behaviour behind a flag. I, for example, would prefer something like an "enable_fields_emulation" option on the "fields" parameter itself that's off by default and would only exist on 7.x branches going forwards (we don't need this emulation behaviour on 8+). That way clients interested in this best-effort emulation would need to consciously opt in to it with all the known limitations (only fetching from source, no formatting, no dfs-query-then-fetch).

[cbuescher]

I added another commit that adds an "enable_fields_emulation" request parameter that switches the bwc behaviour off by default but can be enabled explicitely. We can always revert that change if we decide we don't want to do that.

[cbuescher]

@jimczi thanks for the first round of reviews, I pushed some small changes and left questions regarding you other comments.

[cbuescher]

Just to double check, when "_source" is disabled in a remote target index mapping, we will get an exception. This is the same as for non-CCS searches with source filtering or if the "fields" option is used on indices with disabled source, so I think we don't need any extra treatment here.

[cbuescher]

@jimczi while adressing your review comments I found several ways to imporve and simplify the adapter class. In previous revisions the adapter and the internal field fetcher were created too often. Now the adapter only holds the basic information from the original request (like the fact that we need fields emulation at all and references that are needed to create the field fetcher later), and only creates more expensive resources like the field fetcher only once and when needed.
I didn't remove the second "minimize_roundtrip" path just yet, the simplifications make this aditional path look much less complex so it might be worth keeping it for completeness.

[cbuescher]

I didn't remove the second "minimize_roundtrip" path just yet, the simplifications make this aditional path look much less complex so it might be worth keeping it for completeness.

I pushed a commit now that disables "minimize_roundtrip" for all requests that have the "fields emulation" enablement flag set and a non-empty fetch-fields parameter. This means that a potential "minimize_roundtrip=true" setting will be disregarded in that case. I don't know if that is a restriction we can live with and if that was the intention of the previous comment here.

[jimczi]

The change looks good to me. Thanks @cbuescher !

[jimczi]

I pushed a commit now that disables "minimize_roundtrip" for all requests that have the "fields emulation" enablement flag set and a non-empty fetch-fields parameter.

The intent is to use this option in async_search where minimize roundtrip is not implemented so that's okish ;).

[cbuescher]

To summarize the usage of the emulation layer:

• clients need to set the enable_fields_emulation flag on the search request
• the emulation behaviour will be limited to searches using the default query_then_fetch search type
• if both the “fields” option and source filtering are mixed in the same search request, we will throw an error
• 'minimize_roundtrip' settings is ignored when the emulation is effective

[cbuescher]

Reverted since the serialization logic needs adapting, otherwise we break bwc for tranport on master