Allow custom user-agent via --extra-headers

[benschwarz]

Chrome requires custom user agents to be set using Network.setUserAgentOverride rather than accepting a user-agent to be set as additional HTTP headers.

This is confusing to users and a weird quirk of the devtools API that should not be exposed to users of Lighthouse.

My proposal is that we add a check to see if there are any headers set, detect the User-agent header if present and apply it using Network.setUserAgentOverride.

As an example:

  if (extraHeaders) {
    const headerUA = extraHeaders.find(header =>
      header.name.match(new RegExp('User-Agent', 'i'))
    )

    if (headerUA) {
      logger.info(
        '[Config]: Setting custom User-Agent based on test headers',
        headerUA.value
      )
      await Network.setUserAgentOverride({ userAgent: headerUA.value })
    }
  }

💁‍♂️ No longer will people need to set user-agent as a chrome flag.

[wildlyinaccurate]

Would this work for setting a custom UA under mobile emulation as well? AFAIK even with --chrome-flags="--user-agent='...'" the Nexus 5 emulation overrides the UA.

[benschwarz]

@wildlyinaccurate I mean, it's all up to implementation… but my pitch would be to make whatever User-agent header is set to be respected.

It should override emulation, for sure. 👍

[patrickhulce]

It should override emulation, for sure.

I'm not 100% sold on this one. I think in order for us to use the user-agent you should have specified a different or no device emulation. Otherwise you can find yourself in unexpected and inconsistent device setups like a desktop user agent but emulating a nexus 5x...

WDYT about warning as a first step?

[patrickhulce]

We'll cover this as part of #10910

[paulirish]

@benschwarz my plan is to expose a emulatedUserAgent prop to the config. Does that work for you instead of using --extra-headers ?

[benschwarz]

@paulirish, that sounds helpful — we're still using setUserAgentOverride from devtools proto (and blocking LH on certain commands).

_{For brevity, those blocked methods are: Emulation.setVisibleSize, Emulation.setDeviceMetricsOverride, Network.setUserAgentOverride and Emulation.setTouchEmulationEnabled}