Should JavaScript module imports respect referrer policy, and if so, how?

[domenic]

In whatwg/html#1150 some folks decided that given

<!-- x.html -->
<script type="module" src="a.js">

// a.js
import "./b.js";

the referrer when fetching b.js should be a.js, not x.html. This apparently follows CSS, which is nice.

However, we didn't think about referrer policies. It seems like if we were to follow CSS as specified in Referrer Policy, we should be checking for a Referrer-Policy header on a.js, and then falling back to x.html's referrer policy if no such header were present. Does that sound reasonable?

[mikewest]

@domenic: Has this been defined in the intervening ~year?

If not, @jeisinger: opinions? :)

[domenic]

Nope. I tried to rope @domfarolino into doing it, but did not yet succeed :)

[domfarolino]

Ah yes yes, thanks for the ping. I should get on this!

[domenic]

So the OP has an easy situation: x.html -> a.js -> b.js. Here are two harder situations:

Deeper graph

I.e., x.html -> a.js -> b.js -> c.js.

When fetching c.js, which referrer policy header do we use? x's (the HTML), a's (the top-level script), or b's (the referrering script)?

a's seems like a strange choice; either x or b seem more consistent to me.

Multiple referrers

Consider:

<!-- x.html -->
<script type="module" src="a.js">
<script type="module" src="b.js">

// a.js
import "./c.js";

// b.js
import "./c.js";

This will fetch a and b in parallel, and whichever of them finishes being parsed first, will kick off a fetch to c.js.

Which referrer should we use for c.js?

---

In both cases, I wonder what CSS does? (With its @import and background: url() etc. functionality, which per https://w3c.github.io/webappsec-referrer-policy/#integration-with-css supposedly respect the style sheet's referrer policy.)

[annevk]

CSS doesn't do de-duplication of imports as far as I know though perhaps certain caches can make that happen anyway. I doubt that's well-tested also with how many hits a service worker gets, for instance.

If CSS does some de-duplication due to these caches it would be a race.

cc @emilio @noamr

[emilio]

CSS definitely coalesces loads of the same @import uri (and even the same <link> element, as long as it's same-origin actually).

[annevk]

You mean that it's defined by the specification? Or that it's a thing implementations do?

[emilio]

It's a thing implementations do. I don't think the spec defines it.

[domfarolino]

a's seems like a strange choice; either x or b seem more consistent to me.

I agree. I'll amend whatwg/html#9210 to not be influenced by isTopLevel.

CSS doesn't do de-duplication of imports
If CSS does some de-duplication due to these caches it would be a race.

CSS definitely coalesces loads of the same @import uri

What do we mean by "de-duplication" and "coalescing"? Are we talking about destination-specific mechanisms, like whether the CSS "infra" holds onto some map of finished requests that it can use to serve other similar requests (that might only differ by referrer policy), similar to a module script's "module map"? If so, regardless of some mechanism's explicit existence, this sounds like the memory cache which has already been acknowledge to influence requests like this (depending on whether it keys on referrer policy or not).

I think the question that needs to be answered is: do any de-duplication mechanisms key based on referrer/referrer policy? It doesn't look like Chrome's memory cache does, nor HTML's "module map" (keyed by (URL, module type)), and from @emilio's comment, it sounds like CSS is happy to de-duplicate requests with different referrers/RPs, as long as their same-origin.

Since it seems that we have this behavior all the way down, I'd expect all of these duplicate CSS and module script dependencies to be made with referrers and referrer policies that differ from equivalent resources in the memory cache (or destination-specific mechanism), but to be served by them anyways. It sounds like "which one makes it into the cache first" is acceptably racy, so maybe we don't have to do anything special for this case :) Thoughts?

---

Aside:
I suppose another more general question is do we allow the coalescing of in-flight requests (sort of described by this thread) rather than just the de-duplication of complete responses by a cache? I think this question maybe matters more for general "memory cache" discussion so we can probably ignore it for now.

[annevk]

Yeah, it seems like a race is the de facto solution for duplicates in CSS, which seems somewhat acceptable to generalize to module scripts.

[noamr]

It seems related to whatwg/fetch#1400.
The specifications are very lacking in the space of defining how in-flight requests and in-memory caching should work.
IMO though, specifically about this, referrerpolicy should never be a cache key, and people who use different referrerpolicies to retrieve the same resource and expect a different result should be aware that this might be racy.

[naugtur]

Since referrer-policy is a security mechanism, shouldn't it take into account nesting, assuming the intention of the author is described best at the top?
I mean with x.html -> a.js -> b.js -> c.js before sending b.js as referrer to c.js it'd start by looking up the policy on x.html and go down, and only send the referrer if all agreed?
This does get more complicated when we take into account some of the requests might be cross-origin. Then scripts from a different origin should probably form their own subtree in the hierarchy.
Then again, if x.html policy is no-referrer, should it ban other origins from sending referrer? Probably not. So the sequence could start at the top of the tree for the origin in question.

In short, what I mean to say is - shouldn't the policy on the HTML document be enough to ban referrers for good?

[noamr]

In short, what I mean to say is - shouldn't the policy on the HTML document be enough to ban referrers for good?

Currently the relationship everywhere (e.g. between the document and elements with referrerpolicy attributes) is of default->override and not of loose->tighter. @domfarolino might have more context about that but I think that's beyond the scope of decisions specific JS-module imports.