Questions on Page Loaders

[davidgasquez]

Love the new Page loaders feature! They made me wonder....

1. Is there a way to customize the entire html layout (e.g: removing all the modulepreload, scripts, and other Framework specific things). Would love to have the customizability of Astro while being able to run Python data and page loaders
2. Would it be possible to also generate the dynamicPaths from a script? With something like this, almost no JS would be required for data folks to jump into Framework.

[mythmon]

For your second question, the observablehq.config.js (or .ts) file is code and is free to run functions, generate the paths, query databases, etc. Notably, the dynamicPaths key can be an async generator function.

[mbostock]

For (1), we’d like to better understand this use case. Why would you want to remove the modulepreload hints? Why would you remove Framework’s scripts, which are essential for the code to run?

You can fully customize the appearance of the page by disabling Framework’s UI by setting the sidebar and toc to false, by specifying a custom (possibly null) header and footer, and by providing a custom stylesheet. See the Response iframe example. We’re also exploring an embedding API #1084 #1637 if you want to use Framework to create embeddable components rather than full-page apps.

We previously explored making the page template a user-specifiable function in #253, but it wasn’t clear what it would be used for, so we opted for higher-level features such as a user-configurable header, footer, and styles.

For (2), as @mythmon said, you can specify the dynamicPaths option as an async function. This is the example in the docs which uses SQL (wrapped in JavaScript):

import postgres from "postgres";

const sql = postgres(); // Note: uses psql environment variables

export default {
  async *dynamicPaths() {
    for await (const {id} of sql`SELECT id FROM products`.cursor()) {
      yield `/products/${id}`;
    }
  }
};

If you wanted to invoke an external script in another language to enumerate the dynamic paths, you could do that in the same way that Framework invokes data loaders using child_process.spawn (or cross-spawn if you want something that works more seamlessly on Windows).

import {spawn} from "node:child_process";
import readline from "node:readline";

export default {
  async *dynamicPaths() {
    const script = spawn("python3", ["script.py"]);
    yield* readline.createInterface({input: script.stdout});
    const code = await new Promise((resolve) => script.on("close", resolve));
    if (code !== 0) throw new Error(`script exited with code ${code}`);
  }
};

Here script.py outputs the dynamic paths:

import sys

def main():
    data = ["/path1", "/path2", "/path3"]
    for line in data:
        print(line)

if __name__ == "__main__":
    main()

If there’s demand for this pattern, we could definitely extend the dynamicPaths option to allow invoking of a data loader. But in the meantime we recommend copying the above snippet.

[davidgasquez]

Why would you want to remove the modulepreload hints? Why would you remove Framework’s scripts, which are essential for the code to run?

My idea was to keep things as minimal as possible. I'm not very familiar with the front end world though. I was imagining being able to import only Plot and customize the scripts that get run as well as the layout.

Thinking something similar to Astro layouts.

Again, I'm not entirely familiar with front end and this might not be the best approach at all!

You can fully customize the appearance of the page by disabling Framework’s UI by setting the sidebar and toc to false, by specifying a custom (possibly null) header and footer, and by providing a custom stylesheet. See the Response iframe example. We’re also exploring an embedding API #1084 #1637 if you want to use Framework to create embeddable components rather than full-page apps.

This is the approach I'm taking for now but the DX wasn't as good as with proper templating support (html strings in the config file vs proper files with syntax you can import and reuse).

If there’s demand for this pattern, we could definitely extend the dynamicPaths option to allow invoking of a data loader. But in the meantime we recommend copying the above snippet.

This would be ideal for folks that prefer to stay within one language but totally understant that might not be a large enough ammount of people to work on this.

---

Thanks so much! Love working with Framework!