Clarify behaviour of a page linking to a manifest that is not within scope of that manifest

[mgiuca]

Coming across from Chrome issue 961525.

Consider the case where a page (e.g., https://stormy-pincushion.glitch.me) links to a manifest whose scope does not enclose the page that originally linked the manifest (e.g., "scope": "/why"). Should this be considered a valid manifest for the page?

The spec doesn't really provide any guidance on this. Since the details in the spec around installation are pretty fuzzy, it's hard to see exactly how to fit this in, but I think we'd like to essentially ignore the manifest entirely in this situation. The manifest does not apply to this page, therefore it shouldn't influence the display of the page (e.g., the theme colour, etc), nor should we consider this page to be able to trigger installation.

Currently, Chrome does allow installation for this page, resulting in a weird situation where as soon as you install the app, you are shown an out-of-scope address bar since the page you were on is not part of the app scope.

I would like to add to the installation process text along these lines:

"If obtaining the manifest succeeds, if the current document URL is not within scope of the manifest, fail."

Also the installation process could generally be cleaned up, since it suggests that the install takes place in parallel to obtaining the manifest, which doesn't make sense because you need to obtain the manifest first before you can even know whether it can be installed.

See https://crbug.com/961525 for some Chrome-team discussion about what should happen. Another possibility is that we explicitly allow this, basically saying that a page can offer a manifest on the same origin but in a different scope for installation. My feeling is that this almost certainly represents an accidentally-too-small-scope so we shouldn't allow it.

We should also change authority of the manifest's metadata.

Note: Original test case was https://harmonious-cosmonaut.glitch.me which has no "scope" (using the implicit scope of the manifest directory); I forked it to https://stormy-pincushion.glitch.me which has an explicit scope to more directly demonstrate the issue.

[mgiuca]

From an offline discussion with @dominickng : I'm pseudo-convinced that we should allow the installation from an off-scope page of a manifest that is on the same origin.

We identified two separate use cases for why you might want to do this as a site owner:

1. A site contains an index of PWAs that exist in sub-paths of the index page. For example, our test site Killer Marmot contains an index page (which is not a PWA) linking to many sub-paths which are individually scoped PWAs. In this case, you have links to many manifests for all of the sub-PWAs, which can be installed from the index page.
2. A site has a separate PWA implementation, because they don't (at least initially) want to move all of their main site traffic over to a service worker, etc, they have a main site at https://example.com/ and a separate PWA site at https://example.com/pwa. They still want to promote installation of their PWA from https://example.com, which would result in the installation of https://example.com/pwa. In this case, you have a single manifest link --- to the /pwa subsite, and the user agent allows it to be installed from the main page.

We determined that number 1 is not a valid use case. It wouldn't even work, since you can't have multiple installable manifests and there's no way for the site UI to trigger installation of a specific one. This is an interesting use case (a "PWA app store") which we should explore in the future, but through a dedicated API for installing a specific manifest, and likely with an associated permission.

However, number 2 seems like a reasonable use case. While we ideally want developers to make their main site the PWA, the reality is that large websites are understandably hesitant to move all of their traffic over to a new architecture, so having a separate "beta PWA" site seems to be pretty normal practice.

Thus, we propose that:

1. We explicitly allow the linking to a manifest from a document that is not in scope of that manifest, and allow installation of that manifest to be triggered.
2. We explicitly ban the manifest from having any other effect on the out-of-scope document (e.g., setting its theme colour, etc), via a new paragraph in the authority of the manifest's metadata section.

[aarongustafson]

This seems reasonable, but let me offer a counter-proposal:

<a href="/pwa" install="/pwa/manifest.json">Install our PWA</a>

We could maintain the same same-domain restriction as a[download] and this could allow the manifest to remain scoped (directory-wise) to the actual PWA. It also has decent backwards compat (at least as much as download) and provides quick reference to the manifest URL so the manifest could be fetched without having to navigate to the PWA to resolve its location.

I’m not wedded to this by any means, just throwing out an alternate way to solve the same use case.

[aarongustafson]

Or you could piggyback on link[rel=alternate] in some way.

[mgiuca]

I think that suggestion is really a separate feature to what's being proposed here. It's what I was referring to above with:

This is an interesting use case ... which we should explore in the future, but through a dedicated API for installing a specific manifest, and likely with an associated permission.

Giving a site a means to explicitly trigger an install through a button (or link). What we're proposing here is that the normal semi-automated install mechanisms (showing an "install" button in the browser UI, automatically triggering the install prompt when enough engagement is reached, etc) would be allowed for a manifest that's out of scope.

[aarongustafson]

So the manifest would remain scoped to "/pwa" though, right? I guess I don’t have a problem with that. I would still probably suggest a link[rel="alternate"] as well though (to create the explicit association of the PWA being the same as the current site, content wise).

[dominickng]

+1 to Matt here.

The other problem is that there's no guarantee that the start_url referenced by the <a> is under a service worker. Ergonomically, it seems like using an <a install=""> should mean that the current page need not have either a manifest or a SW, but then we'd have to load the linked-to page anyway to see if a service worker is going to be present (or alternatively implement the serviceworker member which no engine yet supports).

rel=alternate appears to be interpreted as an internationalisation feature. What specifically were you thinking of there?

[aarongustafson]

rel=alternate is more generally for alternate representations. It’s used for alternate languages, feeds, stylesheets, and more. I could see something like

<link rel="alternate app" href="https://example.com/pwa">

"app" could just as easily be "installable" or some other key.

[christianliebel]

I think that suggestion is really a separate feature to what's being proposed here.

I think here’s the discussion on that: #627

[mgiuca]

Note that @marcoscaceres @dominickng and I agreed that we can proceed with this. Marcos related that this was the original intention behind the spec, it just wasn't explicit.

Specifically, change number 1 (explicitly allow the manifest to be installed from a page outside of scope) is just a clarification and doesn't make a normative change. Change number 2 (explicitly ban the manifest from applying to documents outside of scope) is a normative change, but should be quite uncontroversial since it's pretty obvious that it shouldn't apply.

[marcoscaceres]

Refreshing myself on this discussion, the original proposal that we had discussed still make sense. It would also make sense if, for example, one was at "example.com/about" or just "example.com/" and start_url was in scope:

{
   "start_url": "/pwa/",
   "scope": "/pwa/"
}

so then it's a matter of invalidating the manifest if the computed start_url no longer in scope, right?

[mgiuca]

Ah, I think in that case we wouldn't invalidate the manifest, because it's same origin as the document.

There are two cases.

Case 1. Document is out of scope, but same origin.

Document is https://example.com/about and manifest is:

{
   "start_url": "https://example.com/pwa/",
   "scope": "https://example.com/pwa/"
}

Answer: Allow the document to install this fully-valid manifest. The start_url and scope are unmodified. They just happen not to include the document. (This is case 2 in the above taxonomy: where a non-PWA version of a site wants to install its PWA version on the same origin.) I think we should allow this.

Case 2. Document is on a different origin to the manifest scope.

Document is https://example.com/about and manifest is:

{
   "start_url": "https://mypwa.com/pwa/",
   "scope": "https://mypwa.com/pwa/"
}

Answer: Invalidate the manifest. This document is not allowed to refer to a manifest whose scope is on a different origin. If the user agent allows the document to be installed, it would be treated as if it had no manifest.

[mgiuca]

A counter here is that instead of fully invalidating the manifest, we should just invalidate the URL-based members:

• start_url
• scope
• shortcuts
• share_target (incubation)
• file_handlers (incubation)
• protocol_handlers (incubation)

The rationale is that it's valid to have a manifest with, say, just a title and icon, to add metadata to your site. That manifest would have no start_url or scope, and thus not be subject to purging by the above same-origin check. If you then added a scope to the manifest which disagrees with the document origin, we would then start ignoring its icons. That doesn't seem right.

So it seems better to just do the same-origin check, and if it fails, abandon any members that contain URLs that are required to be in scope. (Icon URLs are not required to be in scope so they would stay.)

[marcoscaceres]

Yeah, that seems like a reasonable compromise. Then the UA is again in a good position to make a determination on the installability of the application.

[mgiuca]

Discussed on the call with @marcoscaceres .

The mechanism described in #834 ("scope-dependent members") would do nicely here. The idea would be that if you try to install a manifest whose scope is on another origin to your document, we invalidate start_url, scope and all scope-dependent members (hence you could still use icons and other metadata, but not install it). But if it's on the same origin, we don't invalidate anything --- we let you install a manifest that does not enclose this document's scope, as long as its start_url is within its scope.

[b1tr0t]

Discussed on the call with @marcoscaceres .

The mechanism described in #834 ("scope-dependent members") would do nicely here. The idea would be that if you try to install a manifest whose scope is on another origin to your document, we invalidate start_url, scope and all scope-dependent members (hence you could still use icons and other metadata, but not install it). But if it's on the same origin, we don't invalidate anything --- we let you install a manifest that does not enclose this document's scope, as long as its start_url is within its scope.

This makes sense to me.

Also to Aaron's suggestions around using rel=alternate, I think this is a great idea and important functionality for enabling PWA discovery/install if we can mitigate some of the potential abuse issues. We're working on a draft proposal for this functionality we should be ready to socialize in the next few weeks.

[marcoscaceres]

Co-pilot has a good example:

To illustrate the problem mentioned in issue #784, here is an example web manifest and the URL at which the app is being installed from:

Web Manifest (manifest.json)

{
  "name": "Example App",
  "short_name": "Example",
  "start_url": "/start.html",
  "scope": "/why",
  "display": "standalone",
  "background_color": "#ffffff",
  "theme_color": "#000000",
  "icons": [
    {
      "src": "icon.png",
      "sizes": "192x192",
      "type": "image/png"
    }
  ]
}

Example Page Linking to the Manifest (https://stormy-pincushion.glitch.me)

<!DOCTYPE html>
<html>
<head>
    <title>Example App</title>
    <link rel="manifest" href="manifest.json">
</head>
<body>
    <h1>Welcome to the Example App</h1>
</body>
</html>

In this example, the page https://stormy-pincushion.glitch.me links to the manifest but the scope defined in the manifest is /why, which does not include the page itself. This situation illustrates the issue where Chrome currently allows installation but results in a page being displayed out of scope when the app is launched.

[marcoscaceres]

The issue of showing some particular UI is an issue of a given user agent.

However, it a valid issue that if the manifest is missing a start URL, and the current URL is out of scope, we end up in a strange (undefined) situation.

[marcoscaceres]

I like this solution:

Explicitly state that if the current page URL is not within the scope of the manifest, the manifest should be ignored for that page (use the default/null manifest instead). This would prevent the page from influencing the display or triggering installation if it is out of scope.

[christianliebel]

I think this is dependent on what happens during installation:

On Chromium-based browsers on desktop and for the standalone display mode, the page linking to the manifest "leaves the tab" after installation. The tab is closed, and the current page is opened in a new window with the manifest applied. In this scenario, the current page can indeed be outside the scope.

However, this is not an issue on Safari, as the application is added to the dock/home screen but not automatically launched with the current page. When the user opens the app, the start_url is loaded. When no start_url was defined, it falls back to the page linking to the manifest. The processed start URL is guaranteed to be in scope, as the scope will otherwise be adjusted to include the start URL.

Our spec does not define what should happen after installation, so I'm unsure if we should specify this at all. Installing apps from an out-of-scope page is currently possible, so changing this may break existing users, and it does not lead to problems in Safari. Chromium could solve this issue by not opening the current page in the window if it is out of scope, showing UI to redirect the user to the start URL of the app, etc.

[benfrancis]

@marcoscaceres wrote:

However, it a valid issue that if the manifest is missing a start URL, and the current URL is out of scope, we end up in a strange (undefined) situation.

Is it undefined? My understanding is that in that situation the start URL would be set to the document URL and the scope would be overridden from the start URL to make it in scope.

Regarding Chrome's behaviour, section 4.2 Launching a web application says that:

"target URL, if given, MUST be within scope of manifest."

If I've understood correctly then what Chrome is currently doing is essentially immediately launching the app after installing it, but at a target URL outside the scope of the app, which arguably doesn't conform with the specification (see also #995).

@marcoscaceres wrote:

I like this solution:

Explicitly state that if the current page URL is not within the scope of the manifest, the manifest should be ignored for that page (use the default/null manifest instead). This would prevent the page from influencing the display or triggering installation if it is out of scope.

What would it mean to "ignore" a manifest? Refuse to install it? What is "the default/null manifest"? As I understand it currently all members of the manifest are optional in the specification so technically even an empty manifest {} (or invalid JSON, which gets replaced with an empty map) is installable, with the processed manifest populated with the bare minimum defaults that can be derived from manifest URL and document URL (though in practice implementations do impose their own minimum requirements).

@christianliebel wrote:

Our spec does not define what should happen after installation

The current specification also doesn't define a process for actually installing an app, only launching an app that has been installed.

It is not ideal that there are various circumstances under which the processed start URL, scope and ID of an app may be different depending on the page from which it is installed. But short of making some members mandatory, defining the process for installing an app (and/or minimum installability criteria), or changing the defaults used for missing members, I think this can only be worked around with UI design and recommendations of best practices.