Stream a url to a target DOM element.
In the year 2022/5783/Tiger/2076/2014/47, all browsers have become stream capable (🎉), in the context of an already opened HTML page. This opens a huge number of doors as far as new approaches to building applications. However, declarative support is not there yet, so this is one of many (I'm sure) attempts to fill the gap.
<div 📜=https://html.spec.whatwg.org></div>... streams the contents of https://html.spec.whatwg.org into the div (well, see below for one significant caveat).
The syntax above is shorthand for:
<div be-written='{
"from": "https://html.spec.whatwg.org/",
"to": ".",
"reqInit": {},
"beBased": true
}'>If "to" is ".", be-written writes directly to its (shadow) children.
reqInit is the second optional parameter of fetch.
beBased indicates to enable rewriting url's coming from third parties. Having it set to true (the default), does impose something of a performance cost, so set it to false if that works okay.
Security is a particularly thorny issue for this component, and is one of the many slam dunk reasons this functionality really should be built into the browser, with proper security mechanisms in place. In particular, the ability to filter out script tags and other dangerous HTML is nearly impossible with the currently available, cross-browser api's, afaik. So if the stream contains script tags or other such syntax, it will be written with no interference.
In the absence of any signs of mercy from the w3c, we apply security thusly:
- Since import maps require the web page to specify things inside a script tag, and onerror attributes are things which are filtered out from most any DOM purification / sanitizing, we can rely on this to assume that if a path is specified by either an import map or a link preload tag with an onerror attribute, the site has given a green light for content coming from that url.
- Thus, be-written provides rudimentary support for import maps, and for url resolving via link preload tags, as long as the link tags have onerror attributes.
- Not only does be-written provide this rudimentary support, it requires that the path be "endorsed" by one or both of these mechanisms.
So in fact the example shown above will not work.
To make it work, do one of the following:
<head>
<!--doesn't have to be in the head tag, but it's probably where it should go -->
<script type=importmap>
{
"imports": {
"html-spec/": "https://html.spec.whatwg.org/"
}
}
</script>
</head>
...
<div 📜="html-spec/"></div>and/or:
<head>
<link
id="html-spec"
rel=preload
as=fetch
href="https://html.spec.whatwg.org/"
onerror="console.error(href)"
>
</head>
...
<div 📜=html-spec></div>What goes inside the onerror attribute, if anything, is entirely up to each application/developer. But the presence of the onerror attribute is required to unlock the capability of being streamed into the browser.
It seems likely, even with all the advances that HTTP/3 provides, that in cases where most of the users are hit-and-run type visitors, some amount of bundling would be beneficial when it comes time to deploy to production. Or maybe it is a bit difficult to say which is better - bundling or no bundling, so switching back and forth seamlessly is of upmost importance.
The fact that the necessity for security dictates that we can't directly specify the url of what we want to stream directly in the adorned element, actually can be viewed as a blessing in disguise when we consider how to bundle. This is how bundling can work quite easily with be-written (but will require some custom solution for whatever build system you are adopting).
There are two scenarios to consider when bundling -- a page only has one instance where it points to that url:
<div 📜="html-spec/"></div>vs lots of instances.
In the former case, the most effective way to bundle may be to do what this custom enhancement does, but during the build process -- essentially, copy and paste the contents of the resource inside the tag. In that scenario, might as well remove the be-written or 📜 attribute during the build process, so that this enhancement can go fishing for the weekend.
However, because be-written does a bit more than simply blindly paste the full contents of the resource into the tag (such as support snipping the contents and other things described in the rest of this document), some attention should be applied to get an exact duplicate of functionality.
be-written makes the commitment that if the platform decides to embrace all of humanity and fight global warming by endorsing this proposal, be-written, in gratitude, will add a build plugin based on that API, that takes care of all that nuance, in order to achieve that optimal user experience (and I will also forever shut up about their lack of HTML love). I'm sure that's precisely the incentive that will sway them to get cracking.
In the absence of such a plugin, or if the page contains two or more references to the same reference, or if lazy loading is needed, then the instructions below seem to me to be more effective, and I think are probably also acceptable for a single instance as well:
- You must adopt the link preload tag approach mentioned above. Import maps are also fine, and may be more convenient to use during development, but they provide no support for bundling, due to lack of a standard way of specifying metadata. So link preload tags is the least cumbersome approach. Don't forget to add the onerror attribute to the link tag. And remember, if the use of the url won't come into play until well after the page has loaded, use some other value for rel (recommendation: "lazy", or just remove it completely).
- If bundling can be accomplished, either during a build process, or dynamically by the server (again, with the help of an HTMLRewriter, w3c willing), the process that performs the (dynamic) bundling should add attribute "data-imported" to the link tag, which specifies the id of the template. The process should also remove "rel=preload" if applicable.
So basically:
<link id=xtal-side-nav/xtal-side-nav.html
rel=preload as=fetch href=https://cdn.jsdelivr.net/npm/[email protected]/xtal-side-nav.html
onerror=console.error(href)>...becomes, during the build / server rendering process:
<head>
...
<link id=xtal-side-nav/xtal-side-nav.html
data-imported=032c2e8a-36a7-4f9c-96a0-673cba30c142
onerror=console.error(href)
as=fetch
href=https://cdn.jsdelivr.net/npm/[email protected]/root.html>
...
<template id=032c2e8a-36a7-4f9c-96a0-673cba30c142 be-a-beacon=#>
<xtal-side-nav>
<template shadowrootmode="open"><!--begin--><!--begin-->
...
<!--end--><!--end--></template>
</xtal-side-nav>
<script type=module>
if(customElements.get('be-importing') === undefined){
import('be-importing/be-importing.js').catch(err => {
console.debug(err);
import('https://esm.run/[email protected]');
});
}
</script>
</template>
</xtal-side-nav>
</template>
</head>It may even be better to append (some of) the template(s) at the end of the body tag, if there are many many template imports. If they are all front loaded in the head tag, it would mean delays before the user can see above the fold content.
What be-written does is search for the matching template by id. If not found, it waits for document loaded event (if applicable) in case the bundled content was added at the end of the document. If at that time, it cannot locate the template, it logs an error.
But notice the extra attribute: be-a-beacon=#. This causes the template to emit an event that be-written picks up the moment it is added to the DOM tree, so that the inclusion can happen prior to the full document loading, if the template is added outside any shadow DOM. [TODO]
Note
This web component is a member of the be-enhanced family of custom enhancements. As such, it can also become active during template instantiation, though my head spins even thinking about it.
Note
By streaming content into the live DOM Document, it is quite possible the browser will find itself performing multiple page reflows. Be sure to use the Chrome Dev tools (for example) | rendering | web vitals to watch for any performance issues. Various CSS approaches can be employed to minimize this:
- content-visibility
- contain
- overflow - worst case?
Note
be-written tries its best to adjust url's as needed, but mileage may vary, depending on the browser and the time of day (?) as far as avoiding premature downloads (404's).
Note
If you need to modify the HTML as it streams through, be-rewritten [Big-time WIP, not at all ready for prime time], which is (partly) a stop-gap for this key missing primitive. However, it has come to my attention that there is now a browser-compatible implementation that supports streaming. Payload size seems too high for me to embrace it, but I think it's great they provide that option, in case it meets your needs.
Note
For importing HTML optimized for HTML-first web components, see be-importing.
Note
To be HTML5 compliant, use data-enh-be-written for the attribute name instead [Untested].
<details 📜='{
"from": "https://html.spec.whatwg.org/",
"to": "div",
"shadowRootMode": "open"
}'>
<summary>HTML Specs</summary>
<div></div>
</details>A crude filter can be applied to the streamed content:
"between": ["<!--begin-->", "<!--end-->"]It is crude because the way the text streams, it is possible that the sought after string spans across two consecutive chunks. To make the chances of this breaking anything approach nil, repeat the search string twice:
<template shadowrootmode="open"><!--begin--><!--begin-->
...
<!--end--><!--end--></template>As alluded to earlier, the "from" parameter can also be the id of a link tag. If that is the case, the url that is fetched comes from the href property of the link tag. But remember, the link tag requires having an onerror attribute present to ensure it has been given the green light by the site.
Also as mentioned earlier, be-written supports rudimentary url substitution based on import maps:
<script type=importmap>{
"imports":{
"xtal-side-nav/": "https://cdn.jsdelivr.net/npm/[email protected]/"
}
}</script>
<xtal-side-nav 📜=xtal-side-nav/xtal-side-nav.html></xtal-side-nav>Note: The json-in-html vs-code plugin makes editing JSON attributes like this much more pleasant / natural.
By default, be-written adds class "be-written-in-progress" to the element it adorns while the streaming is in progress.
The name of the class can be explicitly set ("inProgressCss": "whatever-you-want").
Because it may be critical to provide custom styling within the shadow DOM (like content-visibility / contains mentioned above), be-written provides the ability to slip in a cloned template into the Shadow DOM before the streaming starts. Likewise, it may be useful to insert some content after - for example providing a link / acknowledgment from where the content came from.
Conceptually, such inserts would look as follows:
<div 📜='{
"from": "blah.html",
"inserts": {
"before": "<div>before</div>",
"after": "<div>after</div>"
}
}'></div>The HTML must pass through the standard sanitizing api that is becoming part of the platform.
When the streaming has finished, the element adorned by the be-written decorator emits event: "enh-by-be-decorated.written.resolved".
For lazy loading, set "defer" to true, and adorn the element with the be-oosoom attribute:
<div be-oosoom 📜='{
"from": "https://html.spec.whatwg.org/",
"defer": true
}'></div>Any web server that can serve static html files will do, but...
- Install git.
- Fork/clone this repo.
- Install node.
- Open command window to folder where you cloned this repo.
-
npm install
-
npm run serve
- Open http://localhost:3030/demo/dev in a modern browser.
import 'be-written/be-written.js';import 'https://esm.run/be-written';