Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTML Modules #645

Open
justinfagnani opened this issue Jun 20, 2017 · 189 comments
Open

HTML Modules #645

justinfagnani opened this issue Jun 20, 2017 · 189 comments

Comments

@justinfagnani
Copy link
Contributor

Now that JavaScript modules are on the verge of widespread browser support, we should think about an HTML module system that plays well with it. HTML Imports doesn't have a mechanism for exporting symbols, nor can they be imported by JavaScript, however its loading behavior is quite compatible with JavaScript modules.

@dglazkov sketched out a proposal for HTML modules here: https://github.com/dglazkov/webcomponents/blob/html-modules/proposals/HTML-Imports-and-ES-Modules.md

The main points are that, using the JavaScript modules plumbing, you can import HTML.

Either in HTML:

<script type="module" url="foo.html">

or JavaScript:

import * as foo from "foo.html";

But within the scope of that sketch there are still several questions about the specifics of exporting symbols from HTML and importing to JavaScript.

@justinfagnani
Copy link
Contributor Author

@TakayoshiKochi made a polyfill (based on the <script type-module> polyfill by @matthewp) that implements one set of choices to the questions, and shows JS and HTML modules working together: https://github.com/TakayoshiKochi/script-type-module/tree/html-module-experiment

@domenic
Copy link
Collaborator

domenic commented Jun 20, 2017

Thanks for starting the discussion, @justinfagnani!

For context for others, I've been involved with @TakayoshiKochi and others trying to firm up what HTML modules means before we present it widely. There's still a variety of possibilities. Here's my high-level overview:


Writing a custom element that authors can include using a single file is a bit tricky. In particular, how do you get your template HTML? (Which is often literally a <template>.)

  • If you want to stay self-contained, and thus friendly to consumers, you're currently forced to inline your HTML into your JavaScript as a template string, then later use innerHTML. This is costly; it requires first parsing it in JavaScript, then later as HTML, and using up extra memory for both representations. It requires all-at-once parsing, instead of streaming, and requires it to be done on the main thread (I think). And it defeats the preload scanner for any subresources.
  • If you want to "use the platform", you'll have to ask your consumers to include the <template> in their index.html, with some well-known ID. Then all of the above disadvantages disappear, but consuming your library is very un-ergonomic.

At a high level, HTML modules wants to solve this problem. I think the case for it being a worthwhile problem would be stronger if we had data on the costs of the HTML-in-a-JS-string solution, but just by inspection it seems pretty bad. Still, I hope we can provide some data.


The next big question then is about developer ergonomics: what should the authoring and consumption experience for web components be?

  • At one extreme, we could just allow importing of inert DocumentFragments via the JS module system. This solves the above problem in a minimal, simple fashion. But, it means you cannot collocate your HTML and JS into a single file. Most modern frameworks seem to aim for a developer experience of collation, from React's JSX, to Vue's .vue files, to Polymer's HTML imports usage, to Angular's HTML-in-a-JS-string approach.
  • At another, we could work to add some opinionated semantics to imported HTML files that make them good as a "container" format for a web component. This is the approach @TakayoshiKochi has gone with, and I think it's a good illustration. It's much less of a primitive, and requires adding new unusual semantics (e.g. executing scripts outside the main document), but that might be OK.

Hope this helps set the stage!

@addyosmani
Copy link

addyosmani commented Jun 20, 2017

Thanks for getting a public discussion going, @justinfagnani!

I see HTML Modules as being a good container format for achieving generic subresource dependency specification. They solve a need for platform-level colocated <template>, <style> and <script>/Module in a similar style to what I would get with something like Vue's .vue files or Polymer's existing HTML Imports. I'd love for them to sufficiently export a JS module - I found this to be a shortcoming of HTML Imports when I last used them.

Some questions:

  • What it is about HTML Modules that makes them specific to Web Components? If a third-party decides to just use HTML + CSS + JS in this container format, that should just work too right? I think we should cater for a few different styles of component to work well with HTML Modules.

  • Are HTML Modules primarily a DX convenience for containment? or do we also foresee trying to bake in a performance carrot (e.g preload scanner benefits) to the design?

  • How do we see HTML Modules fitting with respect to the broader Web Packaging efforts?

@justinfagnani
Copy link
Contributor Author

I also want to point out that HTML modules would not prescribe how component developers package their libraries, or even prescribe Web Components for that matter.

There are a number of styles that components could be written and/or packaged:

  1. Pure JS, using either template strings, DOM builder libraries, JSX, etc. (no HTML modules)
  2. Logic in JS, template/CSS in HTML: import template from 'template.html'; in the component's module. Maybe event import style from 'styles.css'; is we include CSS modules.
  3. Single-file, like Vue and Polymer: JavaScript, HTML and CSS all in one file.

These styles could all be compatible.

@domenic
Copy link
Collaborator

domenic commented Jun 20, 2017

These styles could all be compatible.

That's a good point. Even if we go with an opinionated container format with new capabilities, like @TakayoshiKochi's prototype, it will likely be possible to opt-out of those new capabilities, e.g. by just making your main element a <template> so that everything inside it is inert.

@matthewp
Copy link

matthewp commented Jun 20, 2017

It looks like @TakayoshiKochi's experiment doesn't create a new document for the imported module. Not sure if this is an intentional omission or not, as the @dglazkov sketch does mention there being a new document like in html imports. Here's an example module for those interested.

In the interest of listing problems and not solutions I would say some things I want (in addition to other things mentioned by people above) are:

  1. I want to be able to export JavaScript modules from an HTML module.
  2. I want to be able to query for elements within the HTML module.
  3. I want to be able to "reach in" to an HTML module some how from the outside and use stuff other than the HTML modules within. But maybe if (1) and (2) are satisfied that would mean that an HTML module could make the other parts (like the <template>) accessible?

@TakayoshiKochi
Copy link
Member

Thanks @justinfagnani for starting this thread! Thanks @domenic for the summary.
I was wondering if this is a good place as @addyosmani mentioned if HTML Modules may not be
specifically for web components. But as it is, this is good starting here in public:)

We're open to any questions / problems for HTML modules.
Feel free to pour your thoughts here.

FYI, here's slide deck I presented in BlinkOn back in February:
https://docs.google.com/presentation/d/1ksnC9Qr3c8RwbDyo1G8ZZSVOEfXpnfQsTHhR5ny9Wk4/edit

@TakayoshiKochi
Copy link
Member

Re @matthewp (#645 (comment))

It looks like @TakayoshiKochi's experiment doesn't create a new document for the imported module.

Yes, it was intentional. Imported HTML is treated as DocumentFragment (as opposed to Document), so we don't have to bother about various semantics to be satisfied, although it yields some weirdness (e.g. why/how can <script> inside be executed?). I'm interested if you prefer Document (other than that was described in Dimitri's doc) to DocumentFragment.

If you're interested in history, here's record of Document vs DocumentFramgent for HTML Imports:
https://www.w3.org/Bugs/Public/show_bug.cgi?id=22305

@TakayoshiKochi
Copy link
Member

We are interested in hearing requirements from framework authors, as HTML modules is not (should not be?) specific to web components definition (= container for custom elements and shadow DOM definition).

I admit that my experimental polyfill was biased towards HTML Imports + web components, and would be a good fit for Polymer or Vue use cases, but I would be more excited if the resulting HTML modules would be usable for more than these frameworks or libraries.

In my blink-dev thread, a developer from Scirra introduced me his very inspiring blog post:
https://www.scirra.com/blog/ashley/34/html-imports-are-the-best-web-component
The post is long, but I recommend reading it from top to bottom - my interpretation is that HTML Imports fit their product development requirement quite well, even though they do not use Shadow DOM and do use only little Custom Elements. That exemplifies the need for something that supports scalable development on web platform, and HTML Imports is one key primitive for it.

@rniwa
Copy link
Collaborator

rniwa commented Jun 21, 2017

Here's my take on this. In today's world, developers are putting HTML inside their JS, not JS inside their HTML. The fact that we don't currently preload resources in a string literal, or we end up two string representations of HTML if you used string literal to include HTML in a JS file is a pure implementation detail. There is nothing preventing us from implementing an optimization if there was a well known symbol which was used to tag a HTML in a JS.

Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.

So I'm not all convinced that we need to put JS into HTML as done in the HTML import. In fact, we'd likely oppose to such an approach on the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.

@matthewp
Copy link

@rniwa I find this opinion confusing given your interest in template parts. Are you not interested in that feature either, now? Or are you saying you'd prefer to define your <template> (with parts) as a string literal in JS rather than in HTML. Why?

@web-padawan
Copy link

web-padawan commented Jun 21, 2017

the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.

While not being a framework author nor a browser developer here, just an end-user, I'd like to say that, although it is a widely adopted approach, it still requires tooling and you'll not be able to serve vanilla HTML, CSS and JS unprocessed by stuff like Webpack and so on.

I'd also like to remind you that React was initially built with PHP where including HTML was once widely used. But is such a mixing really considered as a good practice by PHP developers now?

Furthermore, how shall HTML modules work in browsers with disabled JavaScript? If markup stays in HTML, I can imagine them still working somehow. But what if it is placed in string literals?

@AshleyScirra
Copy link

I wrote this blog post. I'm interested in participating in work for HTML modules, and I think our experience is valuable: we have published a commercial PWA using ~300 HTML imports.

As I touched on in the blog, I quite like the idea of being able to import a HTML file and get a Document, much like XHR for type "document". So this:

import * as doc from "./myimport.html"

would be similar to assigning doc the Document resulting from fetching myimport.html. You can then querySelector etc. on doc and use its DOM content in the JS module. (IMHO, this is far better than pasting chunks of markup in to strings in JS)

Then all you need to do is add a HTML tag that can propagate to further dependencies. I suggested changing

<link rel="import" href="import.html">

to be equivalent to

<script type="module">
import * as doc from "./myimport.html"
</script>

It could equally be <script type="module" src="myimport.html"></script> to do the same. This means further scripts/imports/modules are loaded and run, etc.

I have to say style application from imports is a useful feature for us - see my example of a dialog definition for an example. It's possible a framework could work around this (e.g. applyStylesFrom(doc)), but it seems a shame if the dialog definition example can't work natively.

If HTML modules covered that, I'd guess that largely covers everything we use imports for. In our case they're largely just the architectural scaffolding that brings together all the components of our web app, and as I tried to emphasise in my blog, they excel at that, particularly since it covers all three aspects of a web app (markup, style, script, at least until Google roll back style application 😢).

I am strongly against trying to use script for any markup or style content. I know some frameworks are having success with that approach, and so we shouldn't do anything that makes that harder. But if you have 1000 lines of markup, surely nobody wants to paste that in to a script as a giant string? I strongly believe that there should at least be the provision for leaving markup in HTML files, and not orient the spec around doing what one framework does today. Remember frameworks come and go over the years. In our case, our PWA is notable for not using VDOM, has almost no use of markup-in-script, uses plain markup (no Angular-style directives), no templating (like {{this}} in markup), and no two-way data binding. I don't think any other framework on the web today takes this approach, but I believe our PWA proves that you can still build scalable, complex web apps that way. I do worry that people have framework-blindness where they think things should be done like the frameworks do, rather than thinking in terms of providing general-purpose capabilities that frameworks of different designs and approaches can make use of, even in the future when everything likely changes.

Anyway, here's a couple of extra ideas that may be useful to extend the utility of HTML modules as I described them. Perhaps it'd be useful to add a selector to pick out specific elements, e.g.:

import "#elem" as elem from "./myimport.html"

Now elem is the DOM element returned by matching the selector #elem on myimport.html. Not sure how to handle multiple results (e.g. using ".myclass" as a selector) - maybe import ".myclass" as [resultsArray] from ".myimport.html"?

Finally I don't know if it makes sense to export from a HTML module, but perhaps you could have something like:

import myElem from "./myimport.html"

where myimport.html has a special tag like

<export name="myElem">
    <div>This div is an exported element!</div>
</export>

Perhaps that could also be reconfigured to re-export a JS module that the HTML module imports, e.g:

<export name="submodule">
    <script type="module" src="a.js"></script>
</export>

...then allowing import submodule from "./myimport.html", which propagates the exports from a.js through the HTML import and in to another JS module. I feel like I'm shooting in the dark there though!

@domenic
Copy link
Collaborator

domenic commented Jun 21, 2017

Furthermore, how shall HTML modules work in browsers with disabled JavaScript? If markup stays in HTML, I can imagine them still working somehow. But what if it is placed in string literals?

I can answer that, at least: they won't. HTML modules are part of the module system, which is entirely dependent on JavaScript.

Even HTML imports never worked in browsers without JavaScript. (The document would be "imported", but useless, since you need script to access imported documents.)

@justinfagnani
Copy link
Contributor Author

Even HTML imports never worked in browsers without JavaScript. (The document would be "imported", but useless, since you need script to access imported documents.)

Depending on future developments of course. Declarative custom elements (which we'll talk about at TPAC) presumably wouldn't need script enabled.

@domenic
Copy link
Collaborator

domenic commented Jun 21, 2017

Declarative custom elements (which we'll talk about at TPAC) presumably wouldn't need script enabled.

Uh... let's just say I disagree with you very strongly on that, and we can leave that discussion for another time, instead of derailing the thread.

@justinfagnani
Copy link
Contributor Author

@rniwa I wouldn't want to encode only what has become most popular today for two reasons:

  1. Because where we are now is largely a consequence of what capabilities the platform has had. JavaScript developed popular user-land module systems and loaders long before HTML Imports, and is now getting widespread native modules before HTML. Given the lack of support HTML Imports has, Polymer has had to carefully weigh it's options for working in browsers without polyfills. If Polymer had decided to support a pure-JS distribution that would have been be a pragmatic choice only and not an indication of what we'd really like to use.

  2. The assertion (developers are putting HTML inside their JS, not JS inside their HTML) might not be so absolute. Polymer and Vue are current examples of libraries that support single-file HTML components with scripts. There would be much more if it were a native capability.

I'm not saying JS shouldn't also gain pre-loading and multi-module support, of course.

@milieuChris
Copy link

milieuChris commented Jun 21, 2017 via email

@AshleyScirra
Copy link

AshleyScirra commented Jun 21, 2017

The only way I can imagine HTML modules being useful with script disabled is if they basically inserted their full DOM content at the place they were included. So to use HTML imports as an example, <link rel="import" href="import.html"> would be equivalent to copy-pasting the contents of import.html in place of that tag. I guess it would actually be nice to have a native way to have your header and footer sections defined in one place.

I get that some people are annoyed by the obnoxious things JS does on some pages. But if you're building a web app, it's simply inconceivable to develop it without JS. It would be like writing an Android app with Java disabled - can you still produce a useful app? The web is still an amazing resource for static documents which can work with JS disabled, but on the whole I think modules are aimed very much at the app side of development, where it would naturally pair with JS logic for an app. Still, I don't see any reason to exclude the HTML-only case. Maybe an include attribute, like <script type="module" src="import.html" include> , could specify to just paste in the DOM content there?

@milieuChris
Copy link

milieuChris commented Jun 21, 2017 via email

@rniwa
Copy link
Collaborator

rniwa commented Jun 22, 2017

Perhaps we need to figure out what the declarative syntax for shadow DOM / custom elements will look like before we can figure out what HTML import should look like.

It's hard to talk about packaging a thing without knowing what the thing looks like.

@TakayoshiKochi
Copy link
Member

For including HTML into HTML, probably you can imagine there is a <include src="..."> HTML tag that magically does what server-side include did on the client side. It would be nice to have.
However it is a separate problem or use case from HTML Modules - which wants to load HTML as a resource and do something on it with scripting.

@AshleyScirra
Copy link

Yes, I think the include case should be split out in to a separate discussion. Maybe a new issue should be filed for it?

@TakayoshiKochi
Copy link
Member

@AshleyScirra if you want it to happen, whatwg/html would be more appropriate place to file an issue than here.

@rniwa declarative Shadow DOM / Custom Elements would be nice to have, but we don't have to be blocked on it, because assuming it's all written in HTML and no script then it is just a plain HTML. Although I can imagine that declarative syntax would not be such a simple thing because its semantics would involve hooks into HTML parser and DOM tree construction...

Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.

Is this idea presented or discussed anywhere? This is very interesting and I would like to learn more about it.

So I'm not all convinced that we need to put JS into HTML as done in the HTML import. In fact, we'd likely oppose to such an approach on the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.

Real-world web development is constrained by actual implementations of browsers available and used today, and React may be the optimal thing to do modular development on the current web platform, but as we are expanding (relaxing?) the constraints of the platform, we don't have to be trapped in the local optima. (that said, I don't say we should ignore React's popularity)

@addyosmani
Copy link

So I'm not all convinced that we need to put JS into HTML as done in the HTML import. In fact, we'd likely oppose to such an approach on the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.

I would be very curious to hear opinions from some framework authors - e.g @sebmarkbage (React), @developit (Preact), @yyx990803 (Vue) and @IgorMinar (Angular) on the direction of embedding resources like styles and HTML inside JS. I acknowledge this is an increasingly popular pattern, however I'm curious if the availability of HTML Modules where colocated resources (including ES Modules) are a first-class citizen would actually be something they would find useful.

In particular for the final bundles that get generated from their tools/Webpack.

Vue, at least during the dev/iteration workflow appears to have shifted closer to a HTML Imports-type thing while preserving some of the options for authoring in JSX or styles-in-JS if you really want to. It still converts everything to JS during the build cycle.

Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.

Do you see the need for a mechanism offering multiple ES modules in a single JS file being much different to a HTML module being the container enabling that? e.g

bundle.1.html

<script type="module">...</script>
<script type="module">...</script>
<script type="module">...</script>

@treshugart
Copy link

declarative Shadow DOM / Custom Elements would be nice to have

I'd like to emphasise that a lack of declarative shadow DOM is a pretty massive complaint from the community and shouldn't be dismissed as a nice to have.

@justinfagnani
Copy link
Contributor Author

I'd like to emphasise that a lack of declarative shadow DOM is a pretty massive complaint from the community and shouldn't be dismissed as a nice to have.

I don't think it's a dismissal at all. In the context of HTML Modules, the question is whether or not declarative custom elements and shadow roots should block moving forward on modules.

@treshugart
Copy link

👍 just making sure it's being heard is all.

@sebmarkbage
Copy link

sebmarkbage commented Jun 23, 2017

@addyosmani I do think this general architecture is valuable. It can be used to import both somewhat static content but also for loading pieces of a server-rendered page out-of-order and inject dynamically. So, I'm all for it for frameworks that are HTML heavy and centric.

However, for React, and I believe this is a more general trend, we're increasingly looking for ways to avoid passing static, or server responses, as HTML as the serialization format. For these use cases we might use custom byte codes like Glimmer or some other format. There's a lot of value in a richer serialization format that can include more data to pass to JS, cheap branching and conditional logic, immediate access to references to nodes etc. I'm asking myself what value HTML, as the serialization format, offers to a JS library once it's already up and running. I see soft value like familiarity, and concrete value like browser specific caching formats. However, those have to be weighed against other things.

Even if you use what seems to happen is that you end up with two modes in the JS library to deal with nodes created from HTML and nodes created from another mechanism. That adds complexity and code.

@justinfagnani
Copy link
Contributor Author

Can you clarify

you end up with two modes in the JS library to deal with nodes created from HTML and nodes created from another mechanism. That adds complexity and code.

Node in HTML Modules would be standard Nodes in the rest of the DOM, there wouldn't be different types.

@ox-harris
Copy link

Anyone played with some of these ideas in real life?

I come with some good engagement with the usecase and I can say that what the warfront is dictating is a HTML-primary, JS-secodnary type of a module system. It seems to me that the JS approach is all being induced for some reason I can't figure out! It seems to me that important comments here emphasizing a HTML-oriented approach aren't being honoured at all. But truth is, these comments are simply the usecase talking. Even with the passing of time, the usecase today is still all about just being able to reuse HTML all by itself.

Unfortunately, JavaScript keeps hunting down HTML everywhere. It's a feeling we don't want! And if we end up shipping one more thing like this, we would naturally seek peace with some other means to the same end, and "HTML Modules" would sit all by itself!

All I have ever wanted, is being able to reuse HTML content; and without JavaScript! I already have ES6 modules for anything JS! Let me explain properly...

  1. Should HTML modules be JavaScript oriented (using ES6's module infrastructure) or HTML oriented (using HTML's own infrasture)?

    The answer should be obvious when we understand that we are talking about two different domains that each lends itself to be treated differently, with each having its own concept of reusability.

    • When we think of HTML modules, what we are really thinking of is reusable HTML contents - not reusable JavaScript, as this is what ES6 modules are designed for. And when we look, it is for this purpose of reusability in HTML that we have the <template> element. And with the template.content API, JavaScript code is already able to import these reusable HTML elements for use.

      So, is there anything new about reusing HTML contents that we want to drop HTML's perfect infrastructure for the job in favour of ES6 modules? Is it just about that HTML now being in a remote file? We could simply do something that gets these remote contents to the same end - the <template> element!

      <template src="/bundle.html"></template>

      The template element above is simply loading itself from a remote file. This is even how HTML thinks.

      <script>
          // Code
      </script>
      
      or 
      
      <script src="/script.js"></script>
      <style>
          /* Rules */
      </style>
      
      or 
      
      <link href="/styles.css" />
      <svg>
          <!-- Contents -->
      </svg>
      
      or
      
      <img src="/image.svg" />

      So, HTML already gives us a way to either define things in-place or load them from a file. I am only wondering how come the <template> element shipped without an equivalent feature in the first place.

      If reusable HTML is the question anywhere, it seems pretty straight forward to just ask the <template> element. There shouldn't be one system for consuming HTML defined statcally and another separate system for consuming the same HTML defined in a remote file. The word "load" should simply be a means to get the supposed contents to the same end. Othereise, we would be introducing a new sort of engineering for an existing concept without any additional benefit.

  2. How about we rather talk about spicing up reusability in HTML with the <template> element, this time, using the module, import and export paradigm, while maintaining all the benefits of doing HTML with HTML?

    I've engaged with this concept extensively using a polyfill I made. And after a million iterations building a real-word app with it, here are my conclusions.

    1. The <template> element is just perfect as one interface for anything reusable, whether defined statcally or defined remotely. And this is the HTML module! Its contents should simply be taken as exports!

      In this sense of a module:

      1. The <template> element would have a name that's used as the module ID.

        <template name="module1">
            <!-- Contents -->
        </template>
        
        or 
        
        <template name="module1" src="/bundle.html"></template>
      2. The export terminology of a standard module system would go to an <export> element that properly puts contents up for consumption using some export ID.

        <template name="module1">
        
            <export name="export1">
                <div>Part of export1</div>
                <div>Part of export1</div>
            </export>
        
        </template>

        Contents may also be tagged to an export ID using something like an exportgroup attribute.

        <template name="module1">
        
            <div exportgroup="export1">Part of export1</div>
            <div exportgroup="export1">Part of export1</div>
        
        </template>
      3. The import terminology of a standard module system would go to an <import> element that let's us declaratively place any of a module's export on any location in the main document.

        <body>
            <import name="export1" template="module1"></import>
        </body>

        Conincidentally, this import (or include) feature is something we've all asked for under different proposals. What's new here is that instead of being all about including server-side contents, the <import> element just maintains a relationship with the <template> element - the module. So, whether a <template> has its content statically defined or has it loaded, the <import> element's job would be just to import from the <template>.

    2. Many new things become possible with this HTML-oriented HTML module system, all of which would be lost otherwise.

      1. Modules would now be nestable, to let us organize them more meaningfully.

        <template name="module1">
        
            <div exportgroup="export1">Part of export1</div>
            <div exportgroup="export1">Part of export1</div>
        
            <template name="module_nested_remote" src="/bundle.html"></template>
        
        </template>

        So, a module can have other modules, which themselves can have other modules, as long as they each have a module ID.

        The full module ID for a nested module becomes a path like module1/module_nested_remote.

      2. Loading modules from a remote file can go without any render-blocking or defer semantics. They would simply announce a successful load event whenever loading is successful - just the way the <img> element works. Then, <import> elements in the UI that depend on these remote contents are simply resolved as exports become available - just as the <img> element is rendered at whatever time loading is successful.

        <body>
            <!-- resolves when module1/module_nested_remote loads -->
            <import name="export1" template="module1/module_nested_remote"></import>
        </body>
      3. With the event system of HTML modules, the main document/UI is able to maintain a dynamic relationship with its modules.

      Then, an HTML modules API!

      1. With something like a document.templates property, we could simply access modules without having to query the document for <template> elements using CSS selectors.

        let module1 = document.templates.module1;

        And for nested modules, something like a template.templates property would be used.

        let module_nested_remote = module1.templates.module_nested_remote;
      2. With something like a template.exports property, we could access a module's exports without having to query <template> elements using CSS selectors.

        let import1 = module1.exports.export1;
      3. And the document.addEventListener() method could always be used to observe the state of a document's modules - especially where modules are loading remote content or where modules are being programmatically added to the document.

      Then some extended usecases where some exports have some JS?

      <template name="module1">
      
          <export name="export1">
              <div>Part of export1</div>
          </export>
      
          <export name="export1">
              <div>
                  Part of export2
                  <script>
                      // Some JS?
                  </script>
              </div>
          </export>
      
      
      </template>

      Ususally, this kind of JS is solely for the purpose of scoped functionality, not for the purpose of being imported into another script. If I really wanted to import a script from another script, I would do so directly:

      import something from './script.js';

      It doesn't make sense to me that I would first have taken the script into some HTML, then begin debating how to import the script!

      Scoped JavaScript is an interesting possibility that should really be explored for its peculiar usecases. This, I certainly have! And here's what I've settled for:

      <template name="module1">
      
          <export name="export1">
              <div>Part of export1</div>
          </export>
      
          <export name="export2">
              <div id="some-div">
                  Part of export2
                  <script type="subscript">
                      // The this variable is a reference to the immediate host element
                      console.log(this.id); // some-div
                  </script>
              </div>
          </export>
      
      </template>

      And the script remains inert until the export is imported into the main document. It ever runs in the context of its immediate host element. Variables defined within aren't available outside, but variables in the global scope are available inside.

      On being imported, our final document becomes:

      <body>
          <!-- <import name="export2" template="module1"></import> -->
          <div id="some-div">
              Part of export2
              <script type="subscript">
                  // The this variable is a reference to the immediate host element
                  console.log(this.id); // some-div
              </script>
          </div>
      </body>

      And the rest of the world of scoped functionality can go on:

      <body>
          <!-- <import name="export2" template="module1"></import> -->
          <div id="some-div">
              Part of export2
              <script type="subscript">
                  console.log(this.closest('body')); // <body>
                  console.log(document.title); // ...
                  this.addEventListener('click', () => {
                      this.remove();
                  });
              </script>
          </div>
      </body>

I can say convincingly that once you begin working with <template>s, <imports, and <script type="subscript">, everything becomes simple. We've particularly taken an app to production at WebQit with this approach, using the polyfill linked below.

You'll find out however that all of the above features is only a part of what we need together to be able to author modular, reactive UI with just native HTML. Answering this challenge has been my project at WebQit which I have summed up as OOHTML (formally CHTML). OOHTML (Object-Oriented HTML) is a proposed suite of UI features that let's us build modular, reactive UIs natively.

@ndugger
Copy link

ndugger commented Aug 7, 2021

I also come in advocacy of HTML modules, and an HTML-first approach. I'm working on a tool that will compile TypeScript (+ JSX) to HTML Modules. It just takes existing tools, the TS compiler & WBN (for creating web bundles), and spits out HTML modules, either disparately, or as part of a single bundle. I only have a bare-bones prototype working, but I figured it couldn't hurt to add to the discussion here in favor of standardizing HTML modules.

webmake

(Feel free to tell me how bad you think this idea is; I have many ideas, not all of them are worth seeing the light of day, lol. On the off chance that you approve of the idea, I'm looking for help.)

@sashafirsov
Copy link

Html include is a still hot topic. Html module just a sugar on top of concept. HTC is a w3c proposal and does include most of needed but well forgotten.
I recently created 3 oss projects based on remote html loading. Html-demo-component, slotted-element, embed-page.
Those are taking remote content, not just html, and injecting the presentation into page. Depend of use, with or without insulation.

@ox-harris
Copy link

I am pleased to announce our new documentation for OOHTML: https://webqit.io/tooling/oohtml.

It's been over 7 months since I posted about OOHTML - our implementation of a HTML-based module system for HTML.

Anyone played with some of these ideas in real life?

...

The months since then have furnished us a complete picture of this approach and have brought even more stability to our implementation. OOHTML is fully usable today.

Here is a 3-min overview: https://webqit.io/tooling/oohtml/docs/getting-started/overview

@GeorgeTailor
Copy link

@ox-harris the thing that you do is really impressive and I admire that, but I have great doubts about the marketing promises that you make. You mention native a couple of times per paragraph per page, which may sound that this is definitely going to be implemented the way you suggest. But looking at the state of HTML and CSS modules, there are no proposals that reached somewhat mature stage, so your "guess" of how the things should work might turn into a disaster for people using your package, given that you mess a lot with HTML without using data-* attributes or import tag, which officially doesn't exist yet.

@ox-harris
Copy link

@ox-harris the thing that you do is really impressive and I admire that, but I have great doubts about the marketing promises that you make. You mention native a couple of times per paragraph per page, which may sound that this is definitely going to be implemented the way you suggest. But looking at the state of HTML and CSS modules, there are no proposals that reached somewhat mature stage, so your "guess" of how the things should work might turn into a disaster for people using your package, given that you mess a lot with HTML without using data-* attributes or import tag, which officially doesn't exist yet.

It's indeed an important feedback @GeorgeTailor . But somehow, we also have factored this into OOHTML; maybe it isn't very obvious. This page in the docs - https://webqit.io/tooling/oohtml/docs/resources/meta-tag - recommends using the OOHTML meta tag to customise attribute and element names to something like what you've wished. So, the <import> element could read <html-import>, and the id attribute could read data-id - as we have shown in these examples: https://webqit.io/tooling/oohtml/docs/learn/examples. We've left this completely open as at now to encourage experimentation in the community. But OOHTML's default attribute and element names are reading without prefixes because it needs to first give a native-like experience.

We're hoping to see the different ways developers imagine a module system for HTML. And with an implementation that is completely bendable, we hope OOTHML can be a great foundation for that.

I am hoping I can put up a more formal proposal at the relevant for OOHTML. The goal is to get OOHTML on a standards track.

@sashafirsov
Copy link

@ox-harris , could you highlight the difference between oohtml and web components v1?
The use of templates, slots, etc.
It looks like both are covering same goal: declarative web components use. Except of V1 been deprecated, it is still a standard with polyfills available. If I am going to take a look into same direction, well-tested V1 looks a bit more attractive. Please prove me wrong.

@sashafirsov
Copy link

sashafirsov commented Aug 14, 2021

@justinfagnani , @dglazkov, @domenic let take a look into root cause of the need for such html imports, how it was addressed in history and what is really missing in given proposal.

The imports for different content types has been in demand since HTML appearance. Except of initially they been addressed differently in own "tiers" and there was no systematic/unified approach. IFrame, CSS, images in HTML, of course SCRIPT of few kinds, etc.

That combination of not unified load and consuming resources have created the need for unification as the volume of web apps simply could not be supported by such approach. Industry have developed the work around as site builders and later build toolchains which been treated resources in same fashion - from registry/dependency tree to loading.

import './mystyles.scss`
import `@typography/spacing.scss`
import `@typography/roboto.ttf`
import scopedDef from `./myCssMod.scss`
import logo from './app.svg'
import headerTemplate from './headerTemplate.html`
...

images, fonts, images, html templates are been treated alike by WebPack, RollUp, and other build toolchains.
Those have served as loading depend of content type as module location resolution as chain of transformations before delivering into final content:

SASS been compiled into CSS, than converted either to LINK or STYLE. Along the way it also been massaged to add compatibility layers or virtually any programmable functionality.

Similar transformation is done for images and fonts.

If we are going to tackle little problems like HTML imports or import maps in insulation from whole delivery chain, the standard would prevent the concept extension into already existing patterns proven to be in high demand ( build toolchains) in web platform. Rather 'html', 'import maps', etc. we need from browser platform the layers of functionality needed for complex Web 3.0.

The module resolvers capable of working with semver CDNs with integrity in mind: package.json functionality in browser for each loaded module, not the import maps. With ability to override in app the externally defined dependencies( missing in NPM/yarn).

The loader with pre- and post- processing transformation chain driven by the changing along content type.

The container capable of insulation layers from CSS to JS gradually exposing or hiding internals.

And so on, everything which we are currently utilize as app build framework but in browser. The idea is not new: build toolchains running in browser are known more than decade. Question is how to make it right?.

The html import here is just a string loader as I see it. The import as module could treat itself or its parts in several different ways:

  • as html string
    import myTemplate from './myTemplate.html`
    el.innerHTML = `...${ myTemplate } ...`
  • as loaded template DOM
    import myTemplateDom from './myTemplate.html`
    el.append(myTemplateDom)
  • exported parts as template slots
    import pendigAnimation from './loading.smil`
    import myTemplateDom,{ loadingSlot, dataSlot } from './myTemplate.html`
    loadingSlot = pendigAnimation;// would instantiate myTemplateDom and set slot
    el.append(myTemplateDom)
  • exported parts as slot setters for template object ( imported object evaluated to template clone DOM with toString() to HTML and with slot setters )
    import pendigAnimation from './loading.smil`
    import myTemplate,{ loadingSlot, dataSlot } from './myTemplate.html`
    loadingSlot = pendigAnimation;// would instantiate myTemplate and set slot
    el.innerHTML = `...${ myTemplate } ...` // autoconverted to string

and so on.

Taking/picking one or another side of implementation now would mean 100% future deprecation!

Instead, the page and particular module should have ability to control ( or point to external descriptor ) the content-type driven transformation. The app developer would pick or define the way this transformation happen. And consumer JS or HTML would be written according to defined transformation chain.

@ox-harris
Copy link

ox-harris commented Aug 16, 2021

@sashafirsov

I might need direct pointers to the areas of comparison you have in mind. But OOHTML and Web Components aren't competing for the same problems. OOHTML is seeking to address aspects that (we think) do not necessarily fall in the domain of Web Components. Yet, it bases itself off Web Components to offer a higher-level feature set. And in turn, these feature set comes with a simpler way to make Web Components. (Lest the name "OOHTML" sounds like a big thing to ask, think of it as simply a codename for a set of features that solve common HTML-related difficulties.)

For example:

  1. Web Components gives us the <template> element for keeping reusable markup. Using standard APIs like the .querySelector() method, we are able to find these contents and put them to use. But in-between lies additional questions that OOHTML seeks to answer:

    • Since every piece of HTML has to be identified by a CSS selector in order to be accessible by .querySelector(), why don't we have a common naming convention for <template> elements or their contents? You'll realise that this is a component system without a proper naming system and it has all been half the feature. OOHTML introduces this missing naming convention: <template name="module1"></template> and builds on it to come off with something that works like a module system for HTML. For example it introduces nesting of these modules to make for structuring markup bundles:

      <template name="module1">
      
          <div>A reusable DIV</div>
      
          <template name="nested_module1">
              <div>A reusable DIV</div>
          </template>
      
      </template>

      It goes further to introduce identifiers for the reusable contents in a <template> element and calls them module exports:

      <template name="module1">
      
          <div exportgroup="export1">A reusable DIV</div>
      
          <!-- Or to put it another way: -->
          <export name="export1">
              <div>A reusable DIV</div>
          </export>
      
          <template name="nested_module1">
              <!-- And an identifier may not always be necessary too. We'll call the contents without an identifier as collectively the "default export" -->
              <div>A reusable DIV</div>
          </template>
      
      </template>

      And lastly, OOHTML introduces an src attribute for referencing remote content for a <template> element. Why deal with an imperative API - fetch() - for this when we can simply do what we already do for remote stuffs: <script src=""></script>, <img src=""></img>, etc.!

      <template name="module1" src="/bundle.html"></template><!-- a "load" event is fired on load -->
      <!-- -->

      We might not realise it, but these are questions each developer would have to answer for himself; and well, for no extra benefit.

    • How do we take this JavaScript - .querySelector('template[name="module1"] > [div[exportgroup="export1"]') - out of the picture in cases where we simply want to place a piece of markup from a <template> element into document body? OOHTML introduces the <import> element!

      <body>
      
          <!-- Named imports -->
          <import template="module1" name="export1"></import>
          
          <!-- Default exports: exports without an identifier will all be imported below -->
          <import template="module1/nested_module1"></import>
      
      </body>
    • How do we take the CSS selectors - .querySelector('template[name="module1"] > [div[exportgroup="export1"]') - completely out of JavaScript code where we need to programmatically reuse exports? OOHTML offers modules and their exports as objects and properties:

      let module1 = document.templates.module1;
      let export1 = module1.exports.export1;
      let deepdefaultExport = module1.templates.nested_module1.exports.default;
    • How does this enhance Web Components development? One scenario is when we need to use web components (or even regular elements) from remote sources (or even third-party vendors). A proper naming module system makes this seamless and collision-free:

      We would simply include the vendor's HTML bundles and import their elements.

      <head>
          <template name="officialModule" src="/bundle.html"></template>
          <template name="vendor1Module" src="https://vendor1.com/bundle.html"></template>
          <template name="vendor2Module" src="https://vendor2.com/bundle.html"></template>
      </head>
      <body>
          <import template="vendor1Module" name="header-component"></import>
          <import template="vendor2Module/layout" name="hero-component"></import>
      </body>

      And we can seamlessly use these elements within other elements.

      customElements.define('extended-header', class ExtendedHeader extends HTMLElement {
          constructor() {
              super();
              // Get the referenced template element
              let moduleDependency = document.templates.officialModule; // Or document.templates.vendor1Module
              // Clone its "export1" export
              let shadowContent = moduleDependency.exports['header-component'].map(el => el.cloneNode(true));
              // Create Shadow DOM and send in the content
              let shadow =  this.attachShadow({mode: 'open'});
              shadow.append(...shadowContent);
          }
      });

      Also, OOHTML introduces a template attribute that any element can use to declare its dependency on module exports.

      <extended-header template="officialModule"></extended-header>
      <extended-header template="vendor1Module/nested_module1"></extended-header>

      The ExtendedHeader JavaScript would simply work with whatever this declaration resolves to.

      customElements.define('extended-header', class ExtendedHeader extends HTMLElement {
          constructor() {
              super();
              // Get the referenced template element
              let moduleDependency = this.template;
              ...
          }
      });
  2. Web Components lets us define custom elements. And where we need to add some reactivity, we could follow a number of approaches; e.g.:

    Property setters/getters

    customElements.define('extended-header', class ExtendedHeader extends HTMLElement {
        
        // Make this header fixed
        set fixed(trueOrFalse) {}
        // Get the fixed state
        get fixed() {}
    
    });

    Attribute-change callbacks

    customElements.define('extended-header', class ExtendedHeader extends HTMLElement {
        
        attributeChangedCallback() {}
    
    });

    Well, these cover only half of the case for reactivity. The community therefore teams with different Web Component libraries rolling out their own change-detection mechanism / reactivity system. OOHTML introduces the State API and the Observer API that just make this easier! These APIs even bring the concept of state reactivity home to every other element (not just web components), and to the document object itself.

    Observer.observe(document.state, 'titleBar', mutation => {
        document.title = mutation.value;
    });
    var counter = 0;
    setInterval(() => {
        document.state.titleBar = 'Document Title Update #' + (++ counter);
    }, 1000);

    Elements are also able to keep everything state-related to a dedicated, reactive State Object:

    customElements.define('extended-header', class ExtendedHeader extends HTMLElement {
        
        constructor() {
            super();
            Observer.observe(document.state, 'menuItems', mutation => {
                this.renderMenuItems(mutation.value);
            });
            Observer.observe(this.state, 'fixed', mutation => {
                if (mutation.value) {
                    $(this).addClass('fixed');
                } else {
                    $(this).removeClass('fixed');
                }
            });
        }
    
    });

    Other code can observe element states.

    let extendedHeader = document.querySelector('extended-header');
    Observer.observe(extendedHeader.state, 'fixed', mutation => {
        // Do something
    });

    A dedicated State Object exposed as .state should also save us from inadvertently overriding an element's native property or method; e.g.:

    customElements.define('extended-header', class ExtendedHeader extends HTMLElement {
        set style(value) {
        }
        get style() {
        }
    });

    With the state API, that could simply belong in element.state.style not in element.style.

    Deep mutations can be made and observed. And all of this works even at the plain object level:

    Observer.set(extendedHeader.state, 'dataTree', { root: { deep: {} } });
    Observer.observe(extendedHeader.state, mutations => {
        mutations.forEach(mutation => {
            console.log(mutation.path);
        });
    }, { subtree: true });
    Observer.set(extendedHeader.state.dataTree.root.deep, 'prop', 'value');
    // console: [ 'dataTree', 'root', 'deep',  'prop' ]
  3. There are also the Namespaced HTML and Subscript features of OOHTML. These are also trying to address common problems that do not necessarily fall in the domain of Web Components but at the same time are able to simplify how we author Web Components.

Overall, "OOHTML" is simply an umbrella name for a feature-set that saves us of unnecessary JavaScript, help us write better HTML and simplify how we author web components.

@adrianhelvik
Copy link

It would be really nice if somehow we could fix inline event handlers with HTML modules.

@Swivelgames
Copy link

Swivelgames commented May 2, 2022

This may require its own champion and proposal/explainer, but I feel like it falls well enough in line that it might be worth mentioning: Scoping of JavaScript references inside of <template>, potentially to their accompanying inline <script type="module"> tags. This would alleviate under-utilization of inline JavaScript as it is right now. Being able to use on*= attributes, and others, while referencing the included class would be incredibly useful.

To elaborate slightly (before rushing to dinner), the actual mechanism may include waiting to evaluate JS references in attributes for children of a template tag until the ShadowDom is registered. Even if all we do is set the context (this) to the currently relevant instance, that would be a huge win for Web Components.

<template id="myCustomElementTemplate">
    <button onclick="this.#handleClick">Click me!</button>
</template>

<script type="module">
    /* ... */
    class myCustomElement extends HTMLElement {
        constructor() {
            super();
            // ShadowDOM is attached to instance of myCustomElement here
            let shadowRoot = this.attachShadow({ mode: "open" });
            let template = importDoc.getElementById("myCustomElementTemplate");
            // The `this` of any children of a ShadowRoot will be equal to the instance of the
            // element, allowing access to public **and private** methods
            shadowRoot.appendChild(template.content.cloneNode(true));
        }
        #handleClick(e) {
            console.log("Clicked!", e);
        }
    }
    /* ... */
</script>

Again, this may very well need to be in its own proposal

@Swivelgames
Copy link

This could potentially pave the way for other mechanisms, including (one which is certainly out of the scope of this proposal): binding attribute values to instance properties, and so on.

@Jamesernator
Copy link

This would alleviate under-utilization of inline JavaScript as it is right now. Being able to use on*= attributes, and others, while referencing the included class would be incredibly useful.

This sort've concept is interesting, but web specs don't add features that have special powers to be able to do things in other JS scopes. Like in order to resolve this.#handleClick somehow shadowRoot.appendChild(template.content.cloneNode(true)) would have to be able to look into the constructor of this to find private fields, this isn't a capability the language offers.

A general rule of thumb, is that web objects don't really have more power than JS objects, they are just (potentially) implemented in a different language. In this case there would be no way to implement something like what you were describing:

class ShadowRoot extends DocumentFragment {
    appendChild(node) {
        if (node.hasAttribute("onclick")) {
            // No way to access this.#handleClick in this method
            // so web specs can't either
        }
    }
}

The exact reasons for this general rule of thumb are varied, some reasons include the ability to separate the JS engine from the browser, some are to allow the engine to optimize things it otherwise couldn't, some are just to respect the intended design of features (i.e. private means private), some are to allow JS to faithfully mock APIs in tests, etc.

@clshortfuse
Copy link

clshortfuse commented Nov 10, 2022

@Swivelgames

You can already create this monstrosity:

<template id="myCustomElementTemplate">
    <button onclick="this.getRootNode().host.constructor.handleClick.call(this, event)">Click me!</button>
</template>

And yes, it works and is cloneable. Though on* has the browser construct a new function and bind it with a EventListener that cannot be removed from JS. That doesn't scale well for very large DOMs (eg: table grid cells with click).

@Swivelgames
Copy link

@Jamesernator Fair enough for private members. However, I still feel that this may still be viable for public fields, no?

@clshortfuse
Copy link

clshortfuse commented Mar 27, 2023

I've kinda moved to just wanting HTML to parse HTML and then some extra code for the client to consider how to use it. It then moves to the current, live document which decides when and how to adopt the imported HTML to the current, live document.

For example, if an HTML import has a <link> URL. Should it always start fetching that link? If it has imports of other JS scripts, does it happen before it lands on the JS. What about a synchronous, ES5 script? Does it need to run to completion before it's handed back? Could a <script> change its own document and add another <script>. Can it dynamically build a <link>?

It kinda leans to an HTML import should probably be a DocumentFragment + extras. This would be similar to how HTMLTemplateElement has a content property that returns a DocumentFragment. Those extras could be invocation functions/properties that depend on the intended purpose JS side. If it's to be used as a Web Component, maybe it'll be cloned during the constructor phase. That means that if it's never called nothing happens.

If it's to consume the JS inside it, maybe a importedHTML.modules call which can be synchronous for those that want it as a critical for runtime: import { modules } from 'file.html'. Or you can do it asynchronously with await import.

If it's a set of resources, we might want to immediate start fetching those resourced by calling document.adoptNode.

What really gets interested is, should we just lock all the new intended functionality behind an import? For example CSS imports are just constructed CSSStyleSheet. What if, in the interest of minification, I want to bundle that functionality into one file, either in JS, or part of the main HTML file. Do I need to tell the browser to import an HTML file to get the import scoping? That means a call back to the server to return a resource, which is wasted time. Couldn't this functionality be added to HTMLTemplateElement instead, so I can stick that in my index.html and reference it from my bundled index.min.js instead?

Right now, I'm experimenting the use case of splitting a page that has something like 30 different Web Components as critical and non-critical Web Components. As of now, HTML-in-JS requires parsing of the string and that's done synchronously. That means my Web Components are loaded in series. I would like them be loaded in parallel by using await import with an HTML Module. It'll reduce the deployment size since we don't need to bundle non-crtitical HTML as strings. We can already do this with CSS, as the browser will return a parsed CSSStyleSheet done so without interrupting the JS runtime. So it makes sense we should be able to do this with HTML as well.

@jfbrennan
Copy link

jfbrennan commented Jun 23, 2023

Just throwing in my support and desire for HTML Modules.

I like that idea of using them to import what looks to be similar to an HTMLTemplateElement with the same content property:

import htmlTemplateThing from 'foo.html';

const documentFragment = htmlTemplateThing.content.cloneNode(true);

// Do stuff with documentFragment or append it wherever you want...

The alternative today is authoring HTML as JS template literals and being limited to innerHTML.

More than that though, we so badly need this for packaging Web Components.

Authoring and "importing" a complete Web Component is a pretty lame experience. You have to somehow get all your components' template source code, e.g. <template>My Web Component<style>:host {...}</style></template>, into the main document and then include the JS with the Custom Element definitions. Or you use template literals or imperative DOM to construct the template in JS. It's janky and error-prone and inefficient either way. I look at (and use) Riot.js and Vue.js single-file components with envy. I've been experimenting with a Single File Web Components hack out of desperation. Would love for the web platform to make importing this HTML file possible:

<template>
  Single File Web Component!
  <style>
    :host {...}
  </style>
</template>
<script type="module">
  customElements.define('x-sfwc', class extends HTMLElement {
    #template = import.meta.document.querySelector('template');

    constructor() {
      super();
      this.attachShadow({ mode: 'open' }).appendChild(this.#template.content.cloneNode(true));
    }
  })  
</script>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests