Allow setting an HTTP request for server side PWA detection

[hrantabelyan]

There is really no way to detect standalone PWA on server side? It makes things very complicated, there is a logic for some ajax requests and content parts that are not needed for PWAs, of course we can use window.navigator.standalone or window.matchMedia() to detect standalone mode, then use CSR to fetch the appropriate content and for ajax requests just add that is_pwa parameter, but it's just too much not needed things for many startups that want to have their PWAs without doing tons of changes to their current code, if just there was a way to define a special HTTP request header in manifest.json to use on all requests.

For now thinking about a very quick and messy solution that can work, use JS to rewrite all internal links (hrefs) and add query parameter to them. So, first time when app is opened with the start_url that has a special query parameter, we will use it on server side to know that we have a PWA user, then on client side we will rewrite all links to include that parameter as well, but it's a very poor solution, it will be easier to replace links on front end, but what about back-end redirections and ajax responses that contain a link?

It's a really great technology and I enjoying to work with it, but lack of this feature is much frustrating :(

P.S. Tried to use cookies for detection, adding is_pwa cookie helped, but not much, because PWA installed app and browser are sharing the same cookies, but we really don't want our mobile view and standalone PWA app to function the same way

[dmurph]

There is a proposal to do this with display mode client hints, would that solve your use case?

[hrantabelyan]

With HTTP headers? it should.
Can you post a link of it?

[dmurph]

I don't see a manifest spec issue / feature request. Maybe this bug turns into that request?

Here is the chromium bug and (very informal) eng exploration I did:
https://bugs.chromium.org/p/chromium/issues/detail?id=1174843
https://docs.google.com/document/d/e/2PACX-1vTx1mTTJB1Hhhgp7IE32ftpqcNq1rYzgyO3ay1_zw6Gj9YEbBdJ52x2RsRZuF2eFE0Nm546pwccuJMo/pub

[hrantabelyan]

Quote - "Client hints are extra headers that the user agent sets on requests. They are only set when the server explicitly requests them in it's response headers. So there needs to be at least one request to the origin before these are sent to the server."

There is still one request should be sent first, what if user opens a link from email with installed PWA (standalone mode) app? in this case even start_url will not be triggered, only if there could be a way to do it without first request.
BUT, this actually can work like this

• server can detect if request header doesn't have display-mode in it and pass a corresponding variable to JS
• browser will check if (1) it supports adding display-mode Client Hint and (2) "display-mode" = "standalone", if true, it will call window.location.reload()
• on reload, the browser will send the display-mode to server
• on all next http requests from same origin, the browser will send display-mode header

Think this can work I think.
One question, browser will detect the display-mode automatically to send to the server, right?

Conclusion, anyway I think for now this feature support is too poor, meantime will use a workaround, almost same thing that above, but Cookies will be used instead of CH, here is how:
Browser checks if "display-mode" = "standalone" and if there is a "pwa_standalone" cookie, if is standalone but there is no cookie, it adds the cookie and refreshes the page, so server knows the display-mode, and if user opens the browser after that let's say, then browser sees that there is a cookie but display-mode isn't standalone, think this can work

[aarongustafson]

We are also investigating providing the installation source as a header, which would be proactive and come on the first browse: w3c/manifest-app-info#19

[dmurph]

I was under the impression that the browser remembers the client hint settings basically indefinitely, so having a single request when the user first visits the site without the hints is fine. But perhaps that isn't acceptable.

[aarongustafson]

@dmurph Interesting. That isn’t how I read it when I saw the spec. I thought it would persist for the session only, but I was a bit distracted while reading it, so I may have missed that. From the browser-install side of things, it’s probably fine as the install would have happened after that initial load. For app catalog-driven installs, however, it would not. I wonder if there’s a good way to auto-enable that flag for the domain upon installation.

FWIW, the discussion in the app info spec also seeks to enable differentiation of installation origin as well.

[tomayac]

As a side remark since the question came up above: there is an Issue open created by @yoavweiss for specifying a way to send hints even with the initial request.

[dmurph]

@aarongustafson : I think I'm just revealing how little I know about client hints.
@yoavweiss do you know how long client hints last? I see accept-ch-lifetime, but nothing else talks about how long these are saved so the hints are sent to the server.

[aarongustafson]

I think I'm just revealing how little I know about client hints.

It’s still more than me 😉

[dmurph]

Looping back here, it looks like we can solve the first request problem with the Critical-CH header:
https://github.com/WICG/client-hints-infrastructure/blob/master/reliability.md#critical-ch

So I think this should solve this?

[hrantabelyan]

@dmurph
WOW sure , that's cool, worth to check once CH support reaches to major Firefox and Safari versions too, for now will use cookie approach mentioned above

[aarongustafson]

Looping back here, it looks like we can solve the first request problem with the Critical-CH header:
https://github.com/WICG/client-hints-infrastructure/blob/master/reliability.md#critical-ch

So I think this should solve this?

An I reading it correctly that the client would get back the Critical-CH header and then pause the loading/rendering prices while it re-requests all of the resources? If so, what is the performance impact on slow networks? It seems like it would cause double downloads of at least the HTML. Does the engine stop all additional asset requests when it gets this back? How does this play with HTTP2 if the server is pushing files into the stream with a request?

[dmurph]

I believe that this would happen in the networking level, so the very first request would be retried, which shouldn't be pausing anything other than the network request, probably on the packet level, and only the first one. This wouldn't be for every request - the user agent only retries if it hasn't populated the CH in the request.

Another cool thing is that this could also be done during the TLS handshake:
https://github.com/WICG/client-hints-infrastructure/blob/master/reliability.md#example

[aarongustafson]

@dmurph I will tuck into this a bit more next week. It looks cool and might solve the issue we’re trying to address for the installation source in the app info spec.

[marcoscaceres]

@hrantabelyan can you help me understand the use case a bit more? What do you do differently in your app when the user is in/out of "PWA mode"? That might help us understand a bit more if this something we should solve at the HTTP level or in JS.

[hrantabelyan]

@marcoscaceres it's about serving different layout/content for PWA standalone mode, for example excluding footer or displaying different content in header, it's not about show/hide something, that's possible on client side, but we also don't want to make DB queries for that layout/content that DB query is not something complex for the case that we have right now, but imagine if it's something with multiple SQL DB joins and/or heavy API calls.

[marcoscaceres]

Ok, I see. Right, so then window.matchMedia() runs too late because at that point the request for content has already been made. Got it. So yeah, a client hint might be good here.

[hrantabelyan]

Ok, I see. Right, so then window.matchMedia() runs too late because at that point the request for content has already been made. Got it. So yeah, a client hint might be good here.

Yes, waiting for better support from browsers

[marcoscaceres]

Ok, we probably want to be getting input from @yoavweiss if we want to proceed with something here... in particular, if we need to move this issue over to the CH repo (or elsewhere).

[yoavweiss]

There's no "CH repo" for features that want to use the infrastructure. If y'all want to define a client hint, you can just do that here, and then add the relevant token and policy controlled feature to the feature registry

[aarongustafson]

Thanks @yoavweiss! I’m may look to define one here for "installed" and a second over in App Info for "installation source." Three questions:

1. Do we need to specify whether a CH should be available in Critical-CH or is that open to all CHs?
2. Do you think it would make sense for these two to follow the "Sec-" approach used for UA-related hints?
3. Do all CHs automatically accrue to the privacy budget work (or similar) or do we need to specify that they should be considered as potential fingerprinting vectors so implementors know to include them in whatever mechanisms they use to control for that?

[dmurph]

Adding @b1tr0t, who was interesting in doing the spec work for display mode client hints :)

[aarongustafson]

@dmurph It would probably make sense to create a section for "Associated Client Hints" or some such. Once I get @yoavweiss’ answers I should be able to get a draft together. Likely this week as this is something we have a good deal of interest about from partners.

[yoavweiss]

I’m may look to define one here for "installed" and a second over in App Info for "installation source."

One rule we kept with Client Hints is to not expose values that are not also exposed through JS (to maintain the parity with them exposing already-exposed data, but not beyond that). Are those values currently web exposed?

1. Do we need to specify whether a CH should be available in Critical-CH or is that open to all CHs?

Any CH can be a critical one AFAIUI. /cc @amtunlimited & @davidben who are leading that Critical CH work.

2. Do you think it would make sense for these two to follow the "Sec-" approach used for UA-related hints?

That's the recommended approach in the RFC.

3. Do all CHs automatically accrue to the privacy budget work (or similar) or do we need to specify that they should be considered as potential fingerprinting vectors so implementors know to include them in whatever mechanisms they use to control for that?

I'm no expert on the privacy budget or other mechanisms employed by browsers to limit/monitor active fingerprinting. I suspect that all active entropy-exposing surfaces contribute to the overall exposed entropy, and hence are monitored/controlled by those efforts.
I would expect the Client Hints you plan to expose to be controlled in exactly the same way that the equivalent DOM APIs are.

[aarongustafson]

One rule we kept with Client Hints is to not expose values that are not also exposed through JS (to maintain the parity with them exposing already-exposed data, but not beyond that). Are those values currently web exposed?

These are not currently web-exposed. We could probably suggest adding them to navigator if you think it’s necessary. Would we need to land that first or could we do this work in parallel?

[dmurph]

we currently expose display mode through css media query:
https://developer.mozilla.org/en-US/docs/Web/CSS/@media/display-mode

[aarongustafson]

we currently expose display mode through css media query:
https://developer.mozilla.org/en-US/docs/Web/CSS/@media/display-mode

Yep. I was talking about the install signal specifically.

[dmurph]

Ah - also, display mode is also changed outside of app context - like 'fullscreen' is set if the web page is set to fullscreen. So not sure if this means the spec definition should be elsewhere.

[aarongustafson]

I took a first pass at adding a Client Hints block to the App Info spec. I tried to model it on the work on the UA CH doc. Would appreciate thoughts: w3c/manifest-app-info#32

I am on the fence as to whether App Info should be the home of both or if the install flag CH should exist in the Manifest, but I’m totally fine with breaking them up and moving the install flag CH here if folx have strong opinions on that.

• @dmurph @marcoscaceres @yoavweiss

I know I need to resolve a few ReSpec issues, but I’ll hit Marcos up with that separately.

[acbr5]

There is really no way to detect standalone PWA on server side? It makes things very complicated, there is a logic for some ajax requests and content parts that are not needed for PWAs, of course we can use window.navigator.standalone or window.matchMedia() to detect standalone mode, then use CSR to fetch the appropriate content and for ajax requests just add that is_pwa parameter, but it's just too much not needed things for many startups that want to have their PWAs without doing tons of changes to their current code, if just there was a way to define a special HTTP request header in manifest.json to use on all requests.

For now thinking about a very quick and messy solution that can work, use JS to rewrite all internal links (hrefs) and add query parameter to them. So, first time when app is opened with the start_url that has a special query parameter, we will use it on server side to know that we have a PWA user, then on client side we will rewrite all links to include that parameter as well, but it's a very poor solution, it will be easier to replace links on front end, but what about back-end redirections and ajax responses that contain a link?

It's a really great technology and I enjoying to work with it, but lack of this feature is much frustrating :(

P.S. Tried to use cookies for detection, adding is_pwa cookie helped, but not much, because PWA installed app and browser are sharing the same cookies, but we really don't want our mobile view and standalone PWA app to function the same way

Hello, the problem solved? I need a solution for server side PWA detection.