Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: safer string truncation #1478

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

Conversation

pauldambra
Copy link
Member

@pauldambra pauldambra commented Oct 16, 2024

We truncate strings by slicing them at a given length both in replay and in general capture property processing

String.prototype.slice() and String.prototype.substring() are not unicode aware

so

const str = 'A 😄 B';
console.log(str.slice(2, 3)); // Outputs broken part of emoji

Major browsers and Node 20 have toWellFormed

Our CI/tests/etc don't use Node 20 so even though we could use toWellFormed in prod it would be a pain to add. Moving to Node20 would probably fix some people's builds and break others. Let's avoid confusion for now.

Luckily the one consistent thing about JS is that it is not consistent.

Array.from(someString) is unicode aware and correctly splits a string into an array of valid characters. So we can then slice that array. To get a string that is sort-of the right length.

Our string truncation in the JS is set arbitrarily so it doesn't matter that this mechanism returns strings that may not have a .length that exactly matches the maxLength passed to truncation

Copy link

vercel bot commented Oct 16, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
posthog-js ✅ Ready (Inspect) Visit Preview Nov 29, 2024 9:40am

@pauldambra
Copy link
Member Author

@daibhin since this would maybe help with your upgrade PR...

I'm considering adding a (lazy loaded?) polyfill for array.from

Copy link

github-actions bot commented Oct 16, 2024

Size Change: +1.66 kB (+0.05%)

Total Size: 3.18 MB

Filename Size Change
dist/array.full.es5.js 258 kB +153 B (+0.06%)
dist/array.full.js 361 kB +147 B (+0.04%)
dist/array.full.no-external.js 360 kB +147 B (+0.04%)
dist/array.js 175 kB +235 B (+0.13%)
dist/array.no-external.js 174 kB +153 B (+0.09%)
dist/main.js 176 kB +235 B (+0.13%)
dist/module.full.js 361 kB +147 B (+0.04%)
dist/module.full.no-external.js 360 kB +147 B (+0.04%)
dist/module.js 176 kB +146 B (+0.08%)
dist/module.no-external.js 174 kB +145 B (+0.08%)
ℹ️ View Unchanged
Filename Size
dist/all-external-dependencies.js 206 kB
dist/customizations.full.js 12.5 kB
dist/dead-clicks-autocapture.js 14.3 kB
dist/exception-autocapture.js 9.37 kB
dist/external-scripts-loader.js 2.29 kB
dist/recorder-v2.js 115 kB
dist/recorder.js 115 kB
dist/surveys-preview.js 57.6 kB
dist/surveys.js 63.2 kB
dist/tracing-headers.js 1.75 kB
dist/web-vitals.js 10.3 kB

compressed-size-action

Copy link
Contributor

@daibhin daibhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interestingly I'm still seeing failures on IE11 in #1276 despite my changes :/ Regardless, this seems good

@pauldambra
Copy link
Member Author

NO, that polyfill definitely does not work 🤣

@pauldambra pauldambra added the bump patch Bump patch version when this PR gets merged label Oct 21, 2024
@pauldambra
Copy link
Member Author

interesting... looks like the array.from polyfill in IE11 bundle makes this timeout 🤯

@posthog-bot
Copy link
Collaborator

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

@posthog-bot
Copy link
Collaborator

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.

@posthog-bot posthog-bot closed this Nov 4, 2024
@pauldambra pauldambra reopened this Nov 6, 2024
@posthog-bot posthog-bot removed the stale label Nov 7, 2024
@posthog-bot
Copy link
Collaborator

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

@posthog-bot
Copy link
Collaborator

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.

@pauldambra pauldambra reopened this Nov 21, 2024
@posthog-bot
Copy link
Collaborator

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

@posthog-bot
Copy link
Collaborator

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bump patch Bump patch version when this PR gets merged stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants