Configurable RUM response headers

[graphaelli]

All intake responses currently include a X-Content-Type-Options "nosniff" header. Users would like some additional headers, often for compliance. Examples include:

• X-Frame-Options "SAMEORIGIN"
• X-Xss-Protection "1; mode=block"
• Content-Security-Policy "default-src https: data: 'unsafe-inline' 'unsafe-eval'"

for these endpoints:

/config/v1/rum/agents
/intake/v2/rum/events
/intake/v3/rum/events

@jahtalab @vigneshshanmugam any concerns about adding these for all RUM intake responses?

Following @joshbressers' suggestion, this issue is for a new configuration option apm-server.rum.response_headers as a dictionary, matching kibana's definition:

 server.customResponseHeaders:
    Default: {} Header names and values to send on all responses to the client from the Kibana server. 

as well as our own output.elasticsearch.headers config. While multiple headers with the same key are normal in responses, they are also permitted to be combined so there shouldn't be an issue with using a dictionary.

This should not be alllowlist on cloud without confirming validation rules first.

[vigneshshanmugam]

I see no harm in adding these headers to the server endpoint. But to be honest these are totally unnecessary headers for the server endpoint as we are not serving/loading arbitrary content from the server.

[graphaelli]

Thanks for the input @vigneshshanmugam. Understood about the necessity of these headers, I'm happy to add them for cases where it makes adoption easier for our users that need a box checked requiring these headers.

[joshbressers]

We should be careful adding these specific headers. Every customer security team has their own special headers, if we set these, a different customer will request something else.

It would make sense to go the Kibana route where users can specify any arbitrary headers they want via a configuration setting (see server.customResponseHeaders)

[hmdhk]

@joshbressers suggestion seems reasonable to me. Even though these headers will probably have no effect in most cases, supporting a config option would cover a lot of different use-cases without having to hardcode any specific header.

[graphaelli]

We discussed an option like Kibana's customResponseHeaders initially but argued that just doing the right thing by default would probably serve users best. In discussion about this again today, @axw suggested adding a custom response headers with good default values listed, even commented out.

I'll update the description to reflect that. Implementer should follow up on what validation is done for server.customResponseHeaders, if that's allowed there.