[Bug] 2.0.0 Errors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though) #191

NullVoxPopuli · 2023-12-04T15:08:22Z

🐞 Describe the Bug

I recieved a syntax error, even though I was not expecting one.

🔬 Minimal Reproduction

the file

import Component from '@glimmer/component';
import { on } from '@ember/modifier';

import { getSnippetElement, toClipboard, withExtraStyles } from './copy-utils';
import Menu from './menu';

/**
 * This component is injected via the markdown rendering
 */
export default class CopyMenu extends Component {
  copyAsText = (event: Event) => {
    let code = getSnippetElement(event);

    navigator.clipboard.writeText(code.innerText);
  };

  copyAsImage = async (event: Event) => {
    let code = getSnippetElement(event);

    await withExtraStyles(code, () => toClipboard(code));
  };

  <template>
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class="absolute top-3 right-4 z-10" data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on "click" this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on "click" this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  </template>
}

😕 Actual Behavior

the error:

app/components/limber/copy-menu.gts
[error] app/components/limber/copy-menu.gts: SyntaxError: Unexpected token (41:1)
[error]   39 |     </Menu>
[error]   40 |   </template>
[error] > 41 | }
[error]      | ^
[error]   42 |

🤔 Expected Behavior

no error

🌍 Environment

prettier-plugin-ember-template-tag version: - 2.0.0-0
ember-template-imports version (if applicable): - n/a
content-tag version (if applicable): - n/a
eslint-plugin-ember version (if applicable): - n/a

➕ Additional Context

prettier 3.1

The text was updated successfully, but these errors were encountered:

gitKrystan · 2023-12-05T02:00:06Z

I have a reproduction of this and I'm digging into the code. It seems like content-tag might not be giving me the range I expect here:

#=> code
'import Component from '@glimmer/component';
import { on } from '@ember/modifier';

import { getSnippetElement, toClipboard, withExtraStyles } from './copy-utils';
import Menu from './menu';

/**
 * This component is injected via the markdown rendering
 */
export default class CopyMenu extends Component {
  copyAsText = (event: Event) => {
    let code = getSnippetElement(event);

    navigator.clipboard.writeText(code.innerText);
  };

  copyAsImage = async (event: Event) => {
    let code = getSnippetElement(event);

    await withExtraStyles(code, () => toClipboard(code));
  };

  <template>
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class='absolute top-3 right-4 z-10' data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on 'click' this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on 'click' this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  </template>
}
'
#=> templateNode.contents
'
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class='absolute top-3 right-4 z-10' data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on 'click' this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on 'click' this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  '
#=> code.slice(templateNode.range.start, templateNode.range.end)
'<template>
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class='absolute top-3 right-4 z-10' data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on 'click' this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on 'click' this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  </template>
}' <-----🐛 SEEMS LIKE A BUG 🐛

NullVoxPopuli · 2023-12-05T04:22:58Z

With slice, on't you need to -1 with the end index? Or do end-start? Something was length based, yeah?

gitKrystan · 2023-12-05T05:17:43Z

I don't think that's the case. I have lots of tests for similar component code and they all work. If I try - 1, 💣 💥 😵...

Speaking of emojis, here's the minimal repro 😂 😂 😂

import Component from '@glimmer/component';

class MyComponent extends Component {
  <template>
    💩
  </template>
}

I opened embroider-build/content-tag#45 since I think the fix lives there.

NullVoxPopuli · 2023-12-17T19:34:24Z

I think the fix could still be here depending on discussions from the meeting this upcoming Tuesday.

We've since learned that Rust's indices are byte-indices , rather than character-indices

Techn1x · 2024-01-02T10:04:03Z

I hit this bug on prettier-plugin-ember-template-tag v2.0.0, so I suspect this isn't quite fixed

One example;

test('it encodes login for URI', async function (assert) {
  const student = make('student', {
    firstName: 'first_name',
    // login: 'abc다윤6', // THIS FAILS (even if remaining as a comment, any use of <template> after this line and prettier explodes)
    login: 'abc\uB2E4\uC7246', // THIS WORKS
  })

  await render(<template><LoginCard @student={{student}} /></template>)
  ...

Error

app/components/login-card/component-test.gjs
[error] app/components/login-card/component-test.gjs: Error: failed to process all templates, 1 remaining
[error]     at convertAst (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected][email protected]/node_modules/prettier-plugin-ember-template-tag/dist/prettier-plugin-ember-template-tag.js:70060:11)
[error]     at Object.parse (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected][email protected]/node_modules/prettier-plugin-ember-template-tag/dist/prettier-plugin-ember-template-tag.js:70079:5)
[error]     at async parse (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected]/node_modules/prettier/index.mjs:19362:11)
[error]     at async coreFormat (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected]/node_modules/prettier/index.mjs:20683:7)
[error]     at async formatWithCursor (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected]/node_modules/prettier/index.mjs:20885:14)
[error]     at async formatFiles (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected]/node_modules/prettier/internal/cli.mjs:7073:18)
[error]     at async main (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected]/node_modules/prettier/internal/cli.mjs:7279:5)
[error]     at async Module.run (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/[email protected]/node_modules/prettier/internal/cli.mjs:7225:5)

I also have an example of the same bug in a non-test GTS file, using the endash character. This also blows up in the same way

  get rows(): LessonGuideRow[] {
    const guides = {
      [Number]: '1–3',
      [Fractions]: '1',
      [Operations]: '1–6',
      [Geometry]: '1–4',
      [Measurement]: '1–5',
    }
  }
  
  <template>...</template>

Would you like a new issue opened?

gitKrystan · 2024-01-02T18:06:02Z

@Techn1x Thanks for the report. I re-opened this issue. If you can make me a failing test that would be great. Here's an example: #219

Techn1x · 2024-01-03T03:33:10Z

Thanks @gitKrystan I've added a failing test for it here #220

gitKrystan · 2024-01-24T17:45:03Z

At this point I am waiting on either:

an update from content-tag to make this easier to fix.
free time to rewrite the dang thing again.

pulien · 2024-02-02T08:45:00Z

Hi, I have the same problem with certain French characters such as: à - â - ä - é - è - ê - ë - î - ï - ô - ö - ù - û - ü - ÿ - ç and their uppercase versions

gitKrystan · 2024-02-02T17:38:14Z

I'd advise to stay on the latest 1.x for now if you are running into this issue.

BoussonKarel · 2024-03-06T15:55:05Z

I have the same issue using the → character in a file. I solved it by using the unicode notation \u2192

chancancode · 2024-03-12T00:04:00Z

It seems like the problem is here: https://github.com/gitKrystan/prettier-plugin-ember-template-tag/blob/844a082f66b55ebd27f118584ef89ad8e94b90bc/src/parse/index.ts#L48-L55

The t parameter ultimately comes from content tag, so t.range.start etc are byte offsets), while start comes from Babel's node.range.start which is in character offset. This causes the findIndex to fail and leaves the template tag unprocessed.

gitKrystan · 2024-03-15T22:23:30Z

I just released v2.0.1 with @patricklx's fix (omgthankyou):

https://github.com/gitKrystan/prettier-plugin-ember-template-tag/releases/tag/v2.0.1

Please let me know how it goes!

gitKrystan mentioned this issue Dec 5, 2023

Template node has incorrect range if the template contains emoji or other multi-byte characters 💩 embroider-build/content-tag#45

Closed

gitKrystan changed the title ~~[Bug] 2.0.0-0 / pre-release, SyntaxError, Unexpected Tokne~~ [Bug] 2.0.0-0 / pre-release, SyntaxErrors when components contain multi-byte characters (e.g. emoji) 💩 Dec 5, 2023

gitKrystan added a commit that referenced this issue Dec 6, 2023

Add test for #191

5ee814d

gitKrystan added a commit that referenced this issue Dec 27, 2023

Add test for #191

7da0653

gitKrystan added a commit that referenced this issue Dec 27, 2023

Add test for #191

42fdef1

gitKrystan mentioned this issue Dec 28, 2023

Fix formatting issues #191, #192, #193 #194

Merged

gitKrystan closed this as completed in #194 Dec 28, 2023

NullVoxPopuli mentioned this issue Dec 29, 2023

fix rust utf8 ranges vs js utf16 ranges ember-tooling/ember-eslint-parser#20

Merged

gitKrystan reopened this Jan 2, 2024

gitKrystan changed the title ~~[Bug] 2.0.0-0 / pre-release, SyntaxErrors when components contain multi-byte characters (e.g. emoji) 💩~~ [Bug] 2.0.0 SyntaxErrors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though) Jan 2, 2024

gitKrystan changed the title ~~[Bug] 2.0.0 SyntaxErrors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though)~~ [Bug] 2.0.0 Errors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though) Jan 2, 2024

Techn1x mentioned this issue Jan 3, 2024

test: failing test for multi-byte char before template #220

Closed

gitKrystan mentioned this issue Jan 5, 2024

[Bug] Incorrect behavior when template contains a #222

Closed

gitKrystan mentioned this issue Jan 16, 2024

[Bug] An apostrophe in a comment breaks formatting #228

Closed

gitKrystan mentioned this issue Jan 24, 2024

[Bug] Emoji in JS code breaking plugin #238

Closed

patricklx mentioned this issue Mar 15, 2024

Fix errors when components contain multi-byte characters #253

Merged

gitKrystan closed this as completed in #253 Mar 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] 2.0.0 Errors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though) #191

[Bug] 2.0.0 Errors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though) #191

NullVoxPopuli commented Dec 4, 2023 •

edited

Loading

gitKrystan commented Dec 5, 2023

NullVoxPopuli commented Dec 5, 2023

gitKrystan commented Dec 5, 2023

NullVoxPopuli commented Dec 17, 2023

Techn1x commented Jan 2, 2024 •

edited

Loading

gitKrystan commented Jan 2, 2024

Techn1x commented Jan 3, 2024

gitKrystan commented Jan 24, 2024

pulien commented Feb 2, 2024

gitKrystan commented Feb 2, 2024

BoussonKarel commented Mar 6, 2024

chancancode commented Mar 12, 2024

gitKrystan commented Mar 15, 2024

[Bug] 2.0.0 Errors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though) #191

[Bug] 2.0.0 Errors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though) #191

Comments

NullVoxPopuli commented Dec 4, 2023 • edited Loading

🐞 Describe the Bug

🔬 Minimal Reproduction

😕 Actual Behavior

🤔 Expected Behavior

🌍 Environment

➕ Additional Context

gitKrystan commented Dec 5, 2023

NullVoxPopuli commented Dec 5, 2023

gitKrystan commented Dec 5, 2023

NullVoxPopuli commented Dec 17, 2023

Techn1x commented Jan 2, 2024 • edited Loading

gitKrystan commented Jan 2, 2024

Techn1x commented Jan 3, 2024

gitKrystan commented Jan 24, 2024

pulien commented Feb 2, 2024

gitKrystan commented Feb 2, 2024

BoussonKarel commented Mar 6, 2024

chancancode commented Mar 12, 2024

gitKrystan commented Mar 15, 2024

NullVoxPopuli commented Dec 4, 2023 •

edited

Loading

Techn1x commented Jan 2, 2024 •

edited

Loading