Output multiple resources

[dgp1130]

Currently, prerender_page() and prerender_page_bundled() output a single HTML page (and other included resources). However, users may want to prerender multiple pages from a single prerender file. A common example might be a blogging site, which has a number of blog posts which use mostly equivalent HTML/JS/CSS, but are built from different markdown. Rather than having a prerender_page_bundled() for each blog post, there should be a way to generate all of them from a single rule.

This could work in Node by simply having the default export return an Iterable or AsyncIterable of PrerenderResource objects, where PrerenderResource simply correlates a URL path to its contents. It could look something like:

import { promises as fs } from 'fs';
import { render } from 'markdown-it';
import { PrerenderResource } from 'rules_prerender';

export default async function* (): AsyncIterable<string> {
    const mdFiles = await fs.readdir('some/directory/', { withFileTypes: true });
    for (const file of mdFiles) {
        yield PrerenderResource.of(`/posts/${file}`, render(await fs.readFile(file, { encoding: 'utf8' })));
    }
}

In Starlark, we would probably need another macro, since the output is complete different. A corollary to prerender_page() is pretty straightforward, since we don't have to worry about bundling.

load("@npm//rules_prerender:index.bzl", "prerender_multi_page")

prerender_multi_page(
    name = "posts",
    src = "posts.ts",
    lib_deps = [
        "@npm//markdown-it",
        "@npm//rules_prerender",
    ],
)

# Outputs a `web_resources()` rule with all the prerendered files.

prerender_page_bundled() is a little trickier but would look the same to the user. We would need to get Rollup and PostCSS to bundle each page individually and then inject the resources into each one. Alternatively, we could treat all pages as sharing the same resources (which is likely mostly true in practice, but there would certainly be edge cases) and simply do a single bundle step to generate one JavaScript and another CSS file, then inject just those two into every page. That would likely be easier from a tooling perspective, but would also be less optimal, since one page including a large dep would cause that dep to be included in all pages.

[dgp1130]

Was able to make a bit more progress today than expected. I got prerender_multi_page_bundled() mostly working (with the caveat that JS and CSS files are not tree shaken between different generated files). We're able to generate multiple files at once, bundle the JavaScript and CSS, and inject them into the generated pages. //examples/multi_page/... shows off a working example.

I just published [email protected] (changelog) and updated the project README with some basic documentation for this. The doc does lie a bit as the data field is not supported yet, so blog use cases aren't totally supported.

A few things still TODO:

• Add README for multi-page example.
• Add data attribute to prerender_multi_page_bundled() and friends.
• Play with it in the site example.

I'd also like to try simplifying some of the infrastructure. As of now there are a few tools with multiple versions, a basic version which processes a single HTML file, and then a "multi" version with processes multiple. I'm curious if I can drop the basic version and update the BUILD rules to use the "multi" versions with just a single input. This is a bit awkward for a few reasons, but it may be worth doing if we can drop a lot of the extra complexity.

I'll file separate issues for those points, but I think this is good enough of an MVP to consider this issue closed. 🥳

[dgp1130]

Some follow up for today:

• Added a README for the multi-page example in d568e46.
• Filed data attribute #24 for adding a data attribute.
• I'd like to generate pages from markdown when adding this to the site example as that is likely the most common use case for the feature. So further experimenting there is blocked on data attribute #24.
• Filed Remove non-multi versions of tools #25 to look into simplifying some of the implementation of our tools.