Feature Request: Export list of dependencies to trace template includes

[noelforte]

Background

I've added functionality within the Eleventy plugin to do some "smart caching", that is, compare if the template's source code matches what Vento has cached, and remove that template from the cache before compiling if it doesn't match. My implementation looks something like this (source):

// `input` is of type { path: string, source: string }, is set from Eleventy data

if (env.cache.get(input.path)?.source !== input.source) {
  env.cache.delete(input.path);
}

env.runString(...);

This handles almost every case in both one-off builds and development servers with the exception of internal Vento file loading (like {{ include ...}}).

Solution I'd like

It would be great if Vento could export a dependencies Set, array, object, etc. that lists a given templates dependencies in addition to content. If a template changes during development, I could selectively clear the cache on those files as well.

Alternatives I've considered

As per the Vento documentation, I could clear the cache on every file change but I want to be as conservative as possible when it comes to handling Vento's cache, especially if Eleventy's cache and Vento's cache can work in tandem. My current implementation follows Eleventy's default, which only recompiles files where the file path or file contents have changed, but it would be cool to leverage Vento's cache for determining when to recompile too!

Additional context

Implementation of some sort of list would also help out with eleventy --serve --incremental runs, since Eleventy can get a better sense of what needs to be rebuilt during development if it knows what files need other files. See "Registering Dependencies" on the Eleventy docs for more info.

I tried forking this repo and adding this feature myself but was getting type and build errors using Deno 2. Happy to contribute if I can!

[oscarotero]

I'm not sure if this is easy, because what you're proposing only works for static dependencies, but Vento can also import dependencies dynamically. For example:

{{ include `templates/${template_name}.vto` }}

In this case, the same template can have different dependencies, depending on the data passed:

env.run("template.vto", { template_name: "template1" });
env.run("template.vto", { template_name: "template2" });

The only way to have this is by tracking the dependencies for every specific page. For example:

const result = env.run("template.vto", { template_name: "template1" });
console.log(result.dependences); // ["template.vto", "templates/template1.vto"]

env.run("template.vto", { template_name: "template2" });
console.log(result.dependences); // ["template.vto", "templates/template2.vto"]

I could clear the cache on every file change

Does Eleventy has an event or something similar that is triggered when a file changed?
In Lume, for example, the Vento plugin only clear the cache of the changed files, not the entire cache (https://github.com/lumeland/lume/blob/main/plugins/vento.ts#L91).

[noelforte]

Does Eleventy has an event or something similar that is triggered when a file changed?

In short, no. The chokidar instance isn't exposed to the configuration API in any way, and Eleventy plugins are synonymous to nested user config files.

The problem is includes are handled internally by Vento without passing any information back to Eleventy. If an included file changes, Eleventy will run a full rebuild of everything but I don't think I can hook into that behavior in any way from a plugin. When Eleventy runs a template it only has access to the template content and path. Unless there's a way I can retrieve associated files from Vento given a specific template, clearing the cache entirely for every build seems like the only way to guarantee things will update.

Maybe I'm over-simplifying it, but why not have a map that keeps track of what paths have been loaded by specific files, either as __exports.dependencies or on the Template object itself? Then at least there's a way to remove a file and all its associated files from the cache easilly which I can then handle with my own integration-specific env.cache and env.compile calls instead of runString. Does that make sense?

[oscarotero]

Vento only knows the dependencies after rendering a template. I guess it's possible to output all files loaded by a template (when using include, import or layout) in the template result object. I mean:

const result = await env.runString(source, data, file);
// For example, console.log(result.dependencies);

But, as said in my previous comment, templates with dynamic includes can have different dependencies every time they are rendered with different data. If you are okay with this, I think it can be implemented.

Saddly, I won't have time in the next weeks for this change (there's too much in my plate right now), but if you want to work on a PR, I can review it.

[noelforte]

Totally understandable. I know I'm asking for a lot for a specific implementation. I'll take a look and raise a PR if I get something working that meets my needs.

[noelforte]

Somehow I overlooked eleventy.beforeWatch 😅 (late nights haha). I'm going to give that a spin to see if I can replicate what Lume does to clear the cache out on file changes. If that doesn't work then I'll see about raising a PR.

[oscarotero]

Great! I was surprised that Eleventy didn't have it! :D