Prerendering async data

[chopfitzroy]

Similar to the question asked in #305 I have a collection of posts/entries coming from an API, while developing I am happy for them to be requested real time but ideally I would like to build them statically for production using prerender.

Is there a built in way to do this?

I did take a look at use-content.js from the docs directory but I couldn't quite reason about what it is doing, those pages do appear to be prerendered though.

[rschristian]

The docs site works by patching fetch() to a fs.readFile() implementation, which you can find here:

wmr/docs/public/prerender.js

Lines 7 to 11 in 3c5672e

	const fs = (await eval('u=>import(u)')('fs')).promises;
	globalThis.fetch = async url => {
	const text = () => fs.readFile('dist/' + String(url).replace(/^\//, ''), 'utf-8');
	return { text, json: () => text().then(JSON.parse) };
	};

This has become built-in as of #934 (as it's pretty handy) though, so you won't need to do that manually if that's the behavior you want. Fetching a local file (using a relative specifier, like fetch('/foo.js')) will automatically use fs when prerendering.

However, for external data sources (as it sounds like you have), you can lightly modify useContent to fulfill your needs. For example, let's say I have an API at https://example.com serving JSON like { "html": ... }, we can support this by doing the following:

import { useState } from 'preact/hooks';

const CACHE = new Map();

async function load(url) {
	const res = await fetch(url);
        const { html } = await res.json();
	return { html };
}

export function useContent(url) {
	let update = useState(0)[1];
	let p = CACHE.get(url);
	if (!p) {
		p = load(url);
		CACHE.set(url, p);
		p.then(
			v => update((p.v = v)),
			e => update((p.e = e))
		);
	}
	if (p.v) return p.v;
	if (p.e) throw p.e;
	throw p;
}

Then in our component:

function MyComponent() {
    const { html } = useContent('https://example.com/some-resource');
}

useContent will check to see if the data is cached, and if not, will fetch, cache it, and trigger a rerender whilst returning the data (or throwing, if the promise was rejected).

Now, fetch is not built into Node pre v18 I believe, so if you want to fetch external resources like this, you'll need to be using Node v18+ or patch fetch with a polyfill (globalThis.fetch = myFetchPolyfill) in an initialization method like the one I linked above.

Hope this helps, let me know if you have any issues.

[chopfitzroy]

Okay this is incredible and absolutely works, could I be a pain and check my understanding as to why prerender waits for the data to populate?

Is it because we throw p; if we don't have a response/rejection yet?

The documentation definitely hints at this:

The difference is that it is asynchronous, and waits for any Promises thrown by components during rendering (Suspense-style) to resolve before returning the HTML

I just had no idea this could be extended to hooks as well.

[rschristian]

That's exactly it, yup!

Basically we make the standard renderToString() attempt, but if it throws, and the thrown value is a promise, we'll attempt another render (up to a maximum depth, don't want to lock up your build forever).

wmr/packages/preact-iso/prerender.js

Lines 31 to 39 in 3c5672e

	const render = () => {
	if (++tries > maxDepth) return;
	try {
	return renderToString(vnode);
	} catch (e) {
	if (e && e.then) return e.then(render);
	throw e;
	}
	};

[chopfitzroy]

Thank you so much for the explanation! That has definitely improved my understanding 🙏

[chopfitzroy]

For anyone who comes across this thread in the future here is a generic hook I've created to do this with any promise.

import { useState } from "preact/hooks";

const CACHE = new Map();

export const usePromise = <T = any>(
    key: string,
    handler: (key?: string) => Promise<T>
) => {
    const [_, update] = useState<unknown>(null);

    let value = CACHE.get(key);

    if (value === undefined) {
        value = handler(key);
        CACHE.set(key, value);
        value.then((res: unknown) => update((value.res = res)));
    }

    if (value.res) {
        return value.res as Awaited<ReturnType<typeof handler>>;
    }

    throw value;
};

It can then be used like:

const response = usePromise('some-key', async () => {
    const resp = await fetch('https://...');
    return resp.json();
});

Or even create some additional hooks from that base hook:

const useFetch = (url: string) => usePromise(url, async () => {
    const resp = await fetch(url);
    return resp.json();
});

Haven't run into any issues yet.