Source maps

[v4dkou]

Is your feature request related to a problem? Please describe.
It would be useful if we could pipe Eta's output into sourcemap-aware tooling to associate things like rendered HTML linting errors with the template source code.

Describe the solution you'd like
I'd like to be able to produce a sourcemap file along with the rendered template string.

Describe alternatives you've considered
Right now I am working on a project that uses EJS, and I need to figure out a way to have transient error mapping.
Alternatives I've considered:

• Forking EJS
• Developing an Eta plugin (but plugins don't receive enough metadata)
• Fork/reimplement Mustache.JS or any other compatible and embeddable platform (Lua, C + WASM), whatever is easier (but then struggle with porting over existing templates)

Additional context

Similar issue for EJS
mde/ejs#123

[shadowtime2000]

Sorry, I don't know much about source maps or how they work. Could you explain how piping Eta's output into sourcemap-aware tooling would work?

[v4dkou]

Generally, the way source maps work for minified JS, for instance, is that minifiers produce source maps that contain associations between source AST and the resulting AST, which allows you to backtrack from line/column position in the output file back to where it came from.
https://github.com/mozilla/source-map

consumer.originalPositionFor({
      line: 2,
      column: 28
    })

Tools like Sentry use that to allow you to capture errors in minified JS in production and unravel them in your IDE.

For HTML/CSS, I would expect being able to take linter's messages (line/col position) and use source maps from Eta to find the part of the template that resulted in this erroneous string.
https://htmlhint.com/
https://stylelint.io/

For HTML, for instance, that would allow us to quickly catch unclosed tags, invalid doctypes, unnecessary whitespace, etc.

[shadowtime2000]

@v4dkou Sorry for getting back so late, do you know exactly what we would need to implement for this? If this requires another library, we should probably make it a plugin, eta_plugin_sourcemaps.

[v4dkou]

@shadowtime2000 We'd need to extend the compilation sequence with template source line/column numbers, so that running the compiled template could yield the source map along with the rendered string.
https://github.com/eta-dev/eta/blob/master/src/compile-string.ts#L68

Here's a description of a solution for source maps in EJS for reference.
mde/ejs#123 (comment)

Libraries are not related to the concept of source mapping, unless someone would like to have it adhere to some specific format, but as this feature would incur a performance penalty, I agree that creating a plugin is the best course of action.

However, I don't think the current API with plugins: Array<{ processFnString?: Function; processAST?: Function; processTemplate?: Function }> would allow us to add this metainfo cleanly.

Perhaps we should allow extending the generated function string during it's compilation rather than applying custom functions afterwards? If you see a way to extend the function string with metainfo using the current API, I'd like to discuss it and implement a plugin myself.

[shadowtime2000]

Perhaps we should allow extending the generated function string during it's compilation rather than applying custom functions afterwards? If you see a way to extend the function string with metainfo using the current API, I'd like to discuss it and implement a plugin myself.

@v4dkou What do you mean by this? I am a little confused by what you mean.

[v4dkou]

@shadowtime2000 Right now the function that gets compiled from a string, can only be manipulated through a plugin.processFnString method. The way AST is mapped onto the FnString is not extendable.

This prevents me from developing a plugin that adds custom AST handlers (such as Source mapping).

If Eta had a hook between the AstObject buffer and the string compilation, one could add their own handlers inside the compiled function (take a look at compile-string.ts, lines 73-109).

If this hook receives data about the Eta compilation process, current line in the original template string, AstObjects, etc. (metainfo), I could write a plugin that injects a "collect line number associations" function to store source line positions and resulting line positions for text injected into the template.

[shadowtime2000]

@v4dkou Sorry 😅, still a little confused by what you mean by this. Do you mean like allowing just dropping in a new compile to string function?

[v4dkou]

@shadowtime2000 No worries 😄

Dropping in a new compile to string would work, but that's cumbersome to use.
What I mean instead is to be able to extend the existing compile to string function.

So whenever tR+=' + content + '\n' gets added, I could also add something like E.config.hooks.onContentRendered(source, content, sourceLineNumber, contentLineNumber)

[shadowtime2000]

@v4dkou What you are proposing would have some more runtime overhead though, as far as I can see. Any other way that could be done?