x-opaque-id header breaks CCS in mixed version cluster

[Dosant]

Elasticsearch version (bin/elasticsearch --version): 7.16.0 bc1 (also potentially 7.15.2)

Description of the problem including expected versus actual behavior:

Presence of x-opaque-id header changes cross cluster search behavior in mixed version cluster

When header is missing cross cluster search works as expected,
When header is present cross cluster search doesn't work

Steps to reproduce:

1. Assume 7.16 has a 7.15 as a remote cluster named 7_15
2. Assume 7_15 has data in my-index index
3. Run against 7.16:

GET /_resolve/index/7_15:* HTTP/1.1
Host: localhost:9200

Observe correct result with return indices from 7.15 cluster resolved

4. Now run against 7.16 adding x-opaque-id header:

GET /_resolve/index/7_15:* HTTP/1.1
Host: localhost:9200
x-opaque-id: test

Observe empty results

Additional context:

1. This popped up in Kibana: Unable to create Index pattern when 7.16 is parent build and 7.15 is child build in CCS setup. kibana#116399, where it can't resolve CCS indices via index pattern creation UI, but can using Dev tools. I found that the difference in the requests that causing this is x-opaque-id header: Unable to create Index pattern when 7.16 is parent build and 7.15 is child build in CCS setup. kibana#116399 (comment)

2. I didn't reproduce with 7.15.1 -> 7.14, but I did reproduce with 7.15 snapshot -> 7.14. Likely the bug is introduced in 7.15.2

3. Possibly related pull request: Preserve request headers in a mixed version cluster #79412

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[ywangd]

Thanks a lot for reporting the issue. I raised #79973 for the fix and will backport all the way to 7.15.2

[ywangd]

I added the >non-issue label and removed >bug because the buggy code has not been released yet.