Reference resources that are not part of the publication

[HadrienGardeur]

This is based on my gap analysis between EPUB 3.2 and WP: #176 (comment)

While the current WP infoset allows to reference in the manifest resources that are part of the publication (through the default reading order and the resource list), we don't have the ability to reference resources that are not part of the publication.

This could be useful in a number of different use cases:

• linking to an external metadata record (Linking to an external metadata record #162)
• linking to a privacy policy (Linking to a privacy policy #163)
• linking to alternate representations (Linking to alternate representations #159)
• discovering various services (such as search Search is a service and an affordance #149, dictionaries or annotations)

[HadrienGardeur]

IMO this should be handled at a WP level but it will be useful for PWP/EPUB4 as well.

Our current draft already contains a WebIDL dictionary for links: https://w3c.github.io/wpub/#link-webidl

The current WebIDL also contains links to handle this use case, but there's no text in our infoset about this concept.

[RachelComerford]

I'm not sure I'm following - From someone still fairly new to this structure, why does the infoset need to link to resources outside of the publication is they are already referenced in the manifest?

[HadrienGardeur]

With the current infoset, we can't reference resources outside of the publication.

I've listed multiple use cases for that in the first comment of this issue, some of them are already part of our infoset, so this feels like a gap in our draft.

[iherman]

@HadrienGardeur I am not sure there is a gap. The general infoset layer should list the kinds of information that we want to provide in general; as you say, e.g., privacy policy is listed as part of the infoset, alternate representations are not (yet?). How, eg, the privacy policy is embodied in the manifest is a different, serialization issue. Answering issues 162, 163, etc, is the main goal, regardless of how it is implemented. If the manifest gives the possibility to add additional links that are not specified in the infoset, that becomes part of the extensibility features.

[HadrienGardeur]

@iherman there is a gap between EPUB 3.2 and the WP.

EPUB 3.2 has the ability to reference any resource outside the scope of the publication using a link. In WP, we only have specialized use cases right now (privacy policy and external metadata record).

[iherman]

Then it is a matter of formulating the extensibility section of the infoset to make this possibility more explicit. But all the examples you cited above are, as far as I am concerned, potential infoset items, regardless on how we access them.

[HadrienGardeur]

You can definitely consider linking as part of an extensibility mechanism, while the issues that I've referenced are all infoset items.

That said, you can't say that having such items in our infoset is enough to cover extensibility as well since in the WAM-style serialization, these infoset items might be implemented as new elements (privacy and metadata for example) rather than using a more extensible mechanism (links).

Purely from an infoset perspective, the biggest gap between EPUB 3.2 and WP is the support of alternate in 3.2 which allows alternate representations (#159).
I think that this is a key item, since it also allows us to bridge the gap with older versions of EPUB and other publication formats.

[iherman]

You can definitely consider linking as part of an extensibility mechanism, while the issues that I've referenced are all infoset items.

There is already a text saying "by the provision of linked metadata records." That could be modified to something like "by the possibility to link to external data like metadata records or other resources".

That said, you can't say that having such items in our infoset is enough to cover extensibility as well

I didn't say that.

biggest gap between EPUB 3.2 and WP is the support of alternate in 3.2 which allows alternate representations (#159).

... which is in a separate issue, ie, it does not belong to this one.

[HadrienGardeur]

There is already a text saying "by the provision of linked metadata records." That could be modified to something like "by the possibility to link to external data like metadata records or other resources".

I don't think that reworking that single sentence is enough: this section needs to be more fully fleshed out in general.
Even without a specific serialization (which I don't believe will be solved anytime soon given the complete lack of progress of the WAM TF), we could agree on extensibility mechanisms such as:

• allowing the manifest to reference any external resource, not just external metadata record (which would be a specific case for linking, not necessarily the main one)
• a vocabulary for rel values (IANA link registry?)
• infoset for our link element (title, support for URI templates, an equivalent to EPUB properties to provide hints about linked resources)
• various requirements (EPUB requires a rel for instance, not sure that this is a good idea)

We'll need to do the same about metadata extensibility as well (need a separate issue).

[iherman]

@HadrienGardeur please do not extend the issue because this will become unmanageable.

This issue relates to one specific question, we should see if that change answers the question; if yes, let us propose it, and see if this can be accepted by the WG and then we can close the issue by making a proposed change.

The other questions you are raising are, though possibly real questions, are not relevant to this very issue.

[HadrienGardeur]

@iherman I'm not extending the scope of this issue, there are other issues to deal with that (#149, #159, #162 and #163).

All the items that I've listed in my previous comment are directly related to a generic mechanism for referencing external resources.

[iherman]

I am sorry @HadrienGardeur, we will have to agree that we disagree. Or that the answer is somewhere in the middle.

For me, the issue of rel-s, IANA, etc, are part of the manifest serialization when we get there, and are not part of the infoset. The only thing the infoset extension section says that there must be clear ways to refer to external resources, including their role.

[HadrienGardeur]

Well let's a look at them one by one.

allowing the manifest to reference any external resource, not just external metadata record

This is consistent with what you've proposed so far.

a vocabulary for rel values (IANA link registry?)

HTTP, HTML and Atom (to take three examples among many) all have different serializations and infoset for their link elements, yet they all use rel values. I don't think that discussing rel values is serialization specific or out of scope.

EPUB does the following:

• require a rel value
• define a registry of rel values

IMO we should do the opposite and not require a rel + align with an existing registry instead of rolling our own.

Even if we select the WAM, it doesn't have any element that's a good fit for what we're discussing, so we'll have to roll our own.

infoset for our link element (title, support for URI templates, an equivalent to EPUB properties to provide hints about linked resources)

That's necessary for any infoset item and we need to go in depth with quite a few of them by the way.

[BigBlueHat]

Even without a specific serialization (which I don't believe will be solved anytime soon given the complete lack of progress of the WAM TF)

Careful how you word things, please.

The progress of the WAM TF has been stated--there's no action to take because we (as a Working Group) don't have specific requests to make of the Web App Manifest specification at this time.

We should also continue to be careful to not change the resource model of the Web (issue forthcoming).

To that end, answering "Reference resources that are not part of the publication" looks like making a link in some HTML or HTTP header (at most). If we're defining our own linking semantics (i.e. the current WebIDL), then we're over-defining and/or redefining things that already exist in the Web rather than building up from what exists.

[HadrienGardeur]

Careful how you word things, please.

Well, I haven't seen a single call planned or any progress on related issues, so this seems pretty accurate.

To that end, answering "Reference resources that are not part of the publication" looks like making a link in some HTML or HTTP header (at most). If we're defining our own linking semantics (i.e. the current WebIDL), then we're over-defining and/or redefining things that already exist in the Web rather than building up from what exists.

We've already discussed this issue multiple times.

A link in any given resource of the publication is very different semantically from a link in the manifest.

The WAM has many links as well (icons, screenshots, application stores) and these are not the same as having the same links in one of the HTML resource of the Web App.

[BigBlueHat]

The WAM has many links as well (icons, screenshots, application stores) and these are not the same as having the same links in one of the HTML resource of the Web App.

The icons are equivalent to what can be expressed in HTML. The store identifiers are for search engines more than the "application context" created. Nothing in the WAM is mandatory for using the Web App, and Web Apps are more than welcome to exist without them.

One only needs to provide a WAM if one wants to provide a more "native" style experience by having a narrowed, scope-limited browsing experience launch-able from an icon on a home screen of some kind.

The WAM is a quantifiably different thing, as you've codified here #127 (comment)

[HadrienGardeur]

@BigBlueHat I'm not suggesting to make references to external resources mandatory in WP either.

But you keep on insisting over and over again on including infoset items that are actually about the publication in one of the resources of the publication instead.

This is not only semantically wrong but also very inefficient whenever we need to process the manifest. This is the top reason why our current process is complex: the requirement to extract the default reading order from HTML somehow.

[iherman]

@BigBlueHat @HadrienGardeur: as an aside, we may want to have a separate issue about the recurrent problem of whether infoset items should/could/may be expressed as HTML meta elements. This comes up in many places (issues 186, 159, for example). Wouldn't it be better to treat it separately?

[BigBlueHat]

@iherman it's less about where it's expressed and more about the resource models in tension between the Web model and the EPUB model of "binding."

The Web is built gradually via links and hypermedia. EPUB is build explicitly through a single OPF file.

As this is a Web spec, it is indeed my assumption that we're working up from that model.

[BigBlueHat]

But you keep on insisting over and over again on including infoset items that are actually about the publication in one of the resources of the publication instead.

And this sums things up perfectly, @HadrienGardeur. Thank you.

Consider it self-referential integrity. 😄

[HadrienGardeur]

@BigBlueHat instead of commenting on this point, I suggest that you open a separate issue as suggested by @iherman.

[BigBlueHat]

We should also continue to be careful to not change the resource model of the Web (issue forthcoming).

I'd already promised to do that above. 😄 So I didn't think it needed re-mentioning.

[HadrienGardeur]

Yet, you've posted three additional comments about this very issue since then.

[TzviyaSiegman]

Yet, you've posted three additional comments about this very issue since then.

Please remember that we observe the W3C Code of Conduct here. Please stick to technical issues.

[iherman]

To put my moneyactions where my mouth is, I have created a separate issue: #193

[HadrienGardeur]

This has been added to the latest WP draft.