Improve partial rebuilds

[bep]

There are some problems with how we currently do partial rebuilds:

• In some situations it's too coarse grained (when in doubt, build everything)
• In some situations we don't detect the complete change set, and you end up having to do a server restart.

For the first item we have a "fast render mode", which helps. Most of the issues in the second item can be worked around with "cache buster" configuration, but it's hard to get right.

The goal for the next Hugo version is, for a given change when running the server, to quickly

1. Detect all other resources that is likely to change because of this and clear any related cache/state to allow them to be rebuilt.
2. Have as few false negatives as possible. These should be documented so they could be worked around.

Some examples:

• Edit of layouts/_default/single.html should re-render items¹ using this template².
• Edit of layouts/_default/baseof.html etc. should work as expected.
• Edit of layouts/partials/foo.html should re-render items using this template².
• Edit of a template used in rendering of page content (e.g. .Content) should re-render that content², e.g. render hook and shortcode templates.
• Edit of a given content file (front matter or Markdown) should re-render items using this content.
• Transitive changes to content or other resources should be detected, e.g. page1 -> shortcode -> partial -> page2.Title.
• The transformations for a resource (e.g. {{ $css | fingerprint }}) will be rebuilt if the source changes.
• Edit of a transitive dependency in JavaScript (imports resolved with js.Build) should work.
• Edit of a transitive dependency in CSS when enabling inlineImports in resources.PostCSS should work.
• Edit of a transitive dependency in CSS for Dart Sass should work.
• Edit of a transitive dependency in CSS for LibSass should work.
• Changes to aggregated data (e.g. site.Lastmod) should work.
• Changes to cascade should work.
• Changes should be reflected in all collections, e.g. site.RegularPages, $page.Pages.

Some special considerations:

It would still be recommended to design the site to produce the least amount of changes during development. A typical construct would be to avoid fingerprinting in development:

{{ $js := resources.Get "js/main.js" | js.Build }}
{{ if not hugo.IsDevelopment }}
    {{ $js = $js | fingerprint }}
{{ end }}

The above JS file would typically be included via its RelPermalink into many (maybe all) of the rendered HTML files, and we really don't want to re-render everything on every change of a JS file. So we special case the use of RelPermalink and Permalink, unless these get updated on change (e.g. fingerprint).

I have a working version of the above which I'm in the process of wrapping up, but thought I'd write it up as a proposal to potentially get some feedback.

Footnotes

1. Typically all matching page output formats. ↩

2. Either directly or transitively, e.g. a partial used in a shortcode template. ↩ ↩² ↩³

[jmooring]

I think that this can be closed, addressed with v0.122.0.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.