Lift Template Literal Restriction on Backslashes

[DanielRosenwasser]

Overview

Everyone loves L^AT_EX so ESNext will likely allow tagged template strings to contain backslashes on any sequence - even ones that look like they might contain an invalid Unicode escapes.

In other words, something like

let x = tag `\u{wat}`;

is now valid, whereas previously it would be considered an error.

Reference: https://tc39.github.io/proposal-template-literal-revision/

As of this writing, the linked proposal is at Stage 3.

Notes

1. This restriction lift doesn't apply to untagged template strings. For example:

   // All of these are still invalid.
   let bad = `bad escape sequence: \unicode`;
   let bracingOurselves = `\u{shouldntDoThis}`;
   let theWorst = `\xtremely bad`;

2. Because there is no appropriate representation for a "cooked" string that contains one of these escapes, the returned value at a position on the top-level array will be undefined, but the raw representation will be available. For example:

   function tag(strs) {
     console.log(strs[0] === undefined);
     console.log(strs.raw[0] === "\\unicode and \\u{55}");
   }

// Prints 'true' twice.
tag\unicode and \u{55}

Proposed Emit

Input:

let x = tag `\u{hello} ${ 100 } \xtraordinary ${ 200 } wonderful ${ 300 } \uworld`;

Output:

var x = (
    _a = [undefined, undefined, " wonderful ", undefined],
    _a.raw = ["\\u{hello} ", " \\xtraordinary ", " wonderful ", " \\uworld"],
    tag(_a, 100, 200, 300)
);

[clavecoder]

This was bumped to stage 4 in 2017-03. It is slated for inclusion in ES2018

https://github.com/tc39/proposals/blob/master/finished-proposals.md

[Kingwl]

Interested in this😉

[DanielRosenwasser]

@Kingwl In implementing it, or using it? 😃

[Kingwl]

related #5291
should we re-implement EscapeSequence in the scanner ?

[DanielRosenwasser]

should we re-implement EscapeSequence in the scanner ?

I don't know why that'd be necessary, I think you just need to potentially thread through some extra context when you're scanning these things (which is what the ECMAScript spec does in the grammar now). But I might've missed something, so feel free to send the PR and we can iterate on it!

[DanielRosenwasser]

By the way, template emit now uses a helper, so the proposed emit is a little different.

[Kingwl]

Should the new grammar be always used or what options are needed?
more than 90 baseline changed

[Kingwl]

@JakeGinnivan

We have just hit this issue upgrading from 2.4.1 to 2.6.1.

glamorous.h3({
'&::after': {
content: "\\00BB"
}
})
Before it output chevrons, after the upgrade it output x000BB.

seems could fixed in this change

[Kingwl]

how could i know that template is tagged in,scanner？ i have no idea😓

[DanielRosenwasser]

I think you'll have to actually add a rescan method on the scanner (e.g. rescanTaggedTemplateHead or something).

[Kingwl]

ahhh, reScan is work, but the error which first time scan are preserved,
how could i remove those incorrect error