Header to opt out of opaque redirect

[jakearchibald]

Something like Access-Control-Allow-Visible-Redirect: *

This would make a redirect responses visible. If the request was cross-origin, it would still have to pass existing CORS checks, and would be filtered accordingly.

[annevk]

Rough duplicate of #75.

I don't think we should call it Access-Control as it's not really about CORS with redirects as we hide them same-origin as well (for cross-origin redirect resources you'd still need CORS of course).

Can you explain the use case and do you know to what extent Chrome is interested in implementing? And other browsers?

[jakearchibald]

The suggestion came from an internal team who wanted to act conditionally on the type of redirect they received. However, they may not have the server control they need to add this header anyway.

I was just adding it for completeness. Should be considered low priority unless we get demand.

[rajkjag]

In our specific case, the navigation preload request, is being DOS blocked and redirected to a CAPTCHA page. However the preload response resolves to a response with type "opaqueredirect" and no access to the redirect url (sent back in the Location: header).The server which performs the DOS block is not under our control and happens upstream.

While we can work around this specific case, it might be easier if ServiceWorker code on the client, has access to the redirect URL for error handling and recovery.

[annevk]

@rajkjag that would violate https://fetch.spec.whatwg.org/#atomic-http-redirect-handling.

[rajkjag]

The proposal on the table was to have server specify via a response header the list of headers to expose.

[annevk]

Okay, I got confused since you said the server wasn't under your control and it would be easier if service workers had access to the redirect URL.

[rajkjag]

I took a closer look at our setup; While the server that generates the captcha URL is a different server, it is being operated in a slave mode and we seem to have control on the final response and headers. Apologies on the confusion.

1. Primary server gets requests.
2. Contact DOS slave server, gets back indication on whether to block or redirect.
3. Returns a redirect response with the redirect URL.

If we cannot implement a header and have the client respect it and expose the redirect URL, then our fallback option would be to embed the redirect URL inside the body of the redirect response (If and when the server determines it is safe to do so).

[ibnesayeed]

Being able to expose status code, location, and various other headers will be great when redirects are handled manually in a fetch. The name of this redirect mode is confusing and misleading. However, if we can somehow opt-out of the opaqueredirect then the name can be justified.

Our use case is related to web archiving. An important need in web archival replay systems is to avoid live-leakage (we call them zombies). When a web page is archived and replayed (for example in the Internet Archive's Wayback Machine or other web archives), it is very important that all the page requisites are served from the archive and not from the live web. However, depending on how the resource was referenced (i.e., absolute URL, absolute path, relative), it may resolve to an invalid location or the live web. To prevent it from happening, archival replay systems rewrite all the references before serving to the client. However, in some complex situations when resources references are generated using JavaScript, they might fail to fix them.

To prevent it from happening we created Reconstructive which allows intercepting all requests in a Service Worker and rerouting them back to the archive if they are going elsewhere. The goal is to free the server from performing any rewrites and deliver the original archived content which can then be fixed by the SW to ensure proper replay. This generally works, but there are many cases where the archive has captured redirect responses originally from the live pages and replays them back with the original Location header as seen at the time of archiving. If the location is an absolute URI (potentially using the domain of the original site, not the archive) then not being able to rewrite it from the SW before handling it over to the browser throws it out of the scope of the SW (oduwsdl/ipwb#456). Current work around is to make sure such headers are rewritten on the server side, but we would like it to work without the server being smart.

[annevk]

@ibnesayeed does adding a header to all redirects count as the server being smart? Because we cannot just reveal this information as already discussed. It has to be opt-in.

[ibnesayeed]

@annevk opt-in mechanism would work for us.

[felixfbecker]

I'm implementing a REST API for contents in a git repository and would like to represent symlinks through redirects, so that the client has the ability to a) easily follow the (final) destination but also b) inspect the Location header manually if needed. This seems impossible at the moment, and it is very counter-intuitive because you'd expect to be able to handle redirects manually when specifying redirect: 'manual'.
It would work if this was made possible through a new access control header.

[slaneyrw]

This is starting to become a problem with Single page applications and remote authentication providers. A lot of server side frameworks will send back a 302 in reaction to an authentication failure so the user can authenticate. If this is initiated from a fetch request we need to navigate the browser, not have the fetch follow the response.

With xmlHttpRequest we could tell the difference by inspecting the X-Requested-With header and, on the server, switch the response from 302 to another status code that isn't auto followed. There is no "hints" in the standard that we can use stop sending a 302, and the javascript cannot read the intended destination... it's lose-lose

[annevk]

You can still send an X-Requested-With header if you want, fetch() is no different in that respect. But yeah, there's probably enough that we should add something here for the server to opt into.

[garygreen]

We also have a use case for this in service workers - we would like to fetch a request and see if it was redirected and to where, then we clear any cache for where the browser will be redirected to before the service worker fetches that page and potentially returns with a cached / stale content. This is particulary important in a cache-first strategy.

Our particular flow is:

1. POST request
2. Server validates request and responds with a 301 Redirect with any errors / success message flashed to session (and will be rendered on the new page once requested)
3. The Location redirect returned from the server should have it's cached cleared by the service worker, so that when the page is loaded by the server worker it will fetch from the network and won't show stale content.

Maybe I'm approaching this the wrong way though as I'm kinda confused what even is a opaqueredirect and why it's useful.

[annevk]

#601 (comment) has a link that explains why we don't expose contents of redirects. opaqueredirect exists to allow redirect navigations to still work offline.

[nemzes]

Just adding to the list of "interest shown" in this feature.

Our use-case is very much as described in #601 (comment) — we are implementing a generic authentication proxy wrapper (so it's not application-specific), which will occasionally respond with an oauth redirect when the auth token has expired. fetch() obviously can't identify these responses, so we have a conundrum.

[slaneyrw]

Ok, it's over 2 years since this was raised. Surely the stakeholders have had some say by now

[ibnesayeed]

Ok, it's over 2 years since this was raised. Surely the stakeholders have had some say by now

We are still waiting for this to make it into the specs and then be implemented. However, we know standardization is a slow process and things don't always happen the way (and when) we want them to.

[annevk]

@yutakahirano @youennf @ddragana thoughts?

[yutakahirano]

Clarification question:

This is not for general responses, but only for navigation responses available in service workers, right?

[annevk]

This header would be about exposing redirect responses to script in general (if the response is cross-origin it would also have to use CORS). You'd have to use redirect mode "manual" and the response would have to set the header.

[annevk]

What I linked also points to https://fetch.spec.whatwg.org/#atomic-http-redirect-handling which is also discussed earlier in the thread. Basically, the value of a Location header can about as sensitive as an HttpOnly cookie.

[slaneyrw]

It's for service workers to be able to deal with responses to navigation requests (which are handled manually by the navigate algorithm). Explained under https://fetch.spec.whatwg.org/#concept-request-redirect-mode.

Ok, so the spec was updated in June 2020 (after my last comment) to appear to give special right to service workers. It's still not clear how that should work.

If you look at the section on https://fetch.spec.whatwg.org/#concept-filtered-response-opaque-redirect, it mentions the URL List is harmless.

So do we now have conflicting statements whether reading the location to redirect to is available or not ?
Why does service workers get special rights for this ?
Do we now need to use service workers exclusively to make fetch calls ?

Or have I completely misread how this is supposed to work

Just to re-iterate my scenario.

SPA javascript application initiates fetch call back to the SAME domain for some sort of resource.
Server evaluates that an authorization is required and sends back a 301 response with the location the SPA is supposed to navigate to ( i.e. redirect to OpenId/connect authorize endpoint ).
SPA framework navigates browser to intended destination

[annevk]

You have misread it. The URL list of a response doesn't contain the Location header information. In case of an opaque-redirect response it would only contain the request URL, which you already have. This feature also was only ever meant for service workers so they can store an opaque redirect response and replay it (but they won't ever be able to obtain the value from the Location header either).

[slaneyrw]

You have misread it. The URL list of a response doesn't contain the Location header information. In case of an opaque-redirect response it would only contain the request URL, which you already have. This feature also was only ever meant for service workers so they can store an opaque redirect response and replay it (but they won't ever be able to obtain the value from the Location header either).

ok, thank you for the clarification... but still pointless then. The workarounds continue!

[mitar]

Could maybe spec allow accessing Location header if the redirect target is on the same origin? Or even if it is absolute URL (one starting with /, without hostname, etc.)?

[annevk]

Not without opt-in: #601 (comment).

[freewheel70]

Basically, the value of a Location header can about as sensitive as an HttpOnly cookie

Hi @annevk Could you explain more about why Location header is so sensitive ?

In case of 302, the Location header contains the value of the temporary redirection, right ? So I guess its value is not that security-critical.

[slaneyrw]

Basically, the value of a Location header can about as sensitive as an HttpOnly cookie

Hi @annevk Could you explain more about why Location header is so sensitive ?

In case of 302, the Location header contains the value of the temporary redirection, right ? So I guess its value is not that security-critical.

Especially if it's same origin, absolutely not a security hole.

But there are implementations in the wild so if the spec was changed to allow opt-in ( or even just change the default implementation ) we still cannot rely on the behaviour and there is no workaround. I'm afraid that this spec is basically worthless.

It's been over 4 years and this is still not "fixed"... the world has moved on and we work around the problem by sending make other response status codes that do not have this arbitrary limitation.

[annevk]

Sigh. It is a security hole, for the same reason we cannot expose HttpOnly cookies same-origin to script, as already explained.

[slaneyrw]

Sigh. It is a security hole, for the same reason we cannot expose HttpOnly cookies same-origin to script, as already explained.

It's not a security hole for same origin requests. Cookies are a different issue, please don't conflate the two.

As I mentioned in my previous answer, the industry has worked around the problem by changing the response HttpStatus to anything other than 30x, where we can read the header.

[figroc]

All the workarounds are nasty. Maybe we can intoduce a new HTTP response header Location-Policy. We could expose the Location to javascript if it sets to some value.

[slaneyrw]

All the workarounds are nasty. Maybe we can intoduce a new HTTP response header Location-Policy. We could expose the Location to javascript if it sets to some value.

This is what was proposed above... setting the redirect flag to manual.

The problem is that those in control of this spec have a fundamental issue with this approach. To be honest, this issue is dead and buried - even if common sense prevails.

The existing implementations of fetch in the wild are impossible to polyfill, nor detect when necessary to do so. We cannot trust what the browser will do so we have to assume it won't be able to read the header and stop sending back 30x.

When standards bodies put their head in the sand, industry works around the problem, then we ignore the standards - to everyone's detriment.

[annevk]

No, it's not what you proposed above. It's equivalent to what OP proposed, which still would be a viable way forward I think, provided we get implementer interest.

[slaneyrw]

No, it's not what you proposed above. It's equivalent to what OP proposed, which still would be a viable way forward I think, provided we get implementer interest.

What a difference a single missed word can do to change the context, I should have proof read it again!.

This is what was proposed above... WHEN setting the redirect flag to manual.

However, I still stand by my assertion that the proverbial horse has already bolted. The alternate behaviour cannot be patched or polyfilled.

[gschadow]

This amounts to a major bug because it is impossible to implement the Digest Authentication with redirect "follow" as there is no way to increase the nc and use a new cnonce as required by Digest Authentication. So redirect fails anyway.

At least there needs to be some way to supply updated headers for the redirected call.

[SampsonCrowley]

Sigh. It is a security hole, for the same reason we cannot expose HttpOnly cookies same-origin to script, as already explained.

It's not a security hole for same origin requests. Cookies are a different issue, please don't conflate the two.

As I mentioned in my previous answer, the industry has worked around the problem by changing the response HttpStatus to anything other than 30x, where we can read the header.

@slaneyrw Just because you don't understand it doesn't make it not a potential threat. It does need to be opt-in, as there are a lot of redirects in the world of the web that contain sensitive tokens. your JS is not the only js just because it's same origin.

That however, doesn't mean it needs to be a required security feature that can't be opted out of and users have zero control over. and that, the complete lack of option is where the standard has completely dropped the ball

[slaneyrw]

@SampsonCrowley I understand this space initimately, having been building web systems for large financials for over 25 years and I take security very seriously. Please don't make assumptions. We already have plenty of "pre-flight" checks, this is just a post-flight check.

As I said, we worked around the problem by avoiding the 30x response codes. Unfortunately the landscape is getting worse with the most popular JS frameworks coalescing around fetch under the hood.