Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HOLD for C+ manual request payment] [$500] Chat - Underscore doesn't work in emoji suggestions (works fine in search) #29081

Closed
6 tasks done
kbecciv opened this issue Oct 9, 2023 · 32 comments
Assignees
Labels
Awaiting Payment Auto-added when associated PR is deployed to production Bug Something is broken. Auto assigns a BugZero manager. Engineering External Added to denote the issue can be worked on by a contributor Weekly KSv2

Comments

@kbecciv
Copy link

kbecciv commented Oct 9, 2023

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!


Action Performed:

  1. Open the app
  2. Open any report
  3. Click on emoji to open emoji picker
  4. Search 'juggling_' and observe that it works fine
  5. Now in composer, type ':juggling_' and observe that it removes suggestion on underscore

Expected Result:

App should display emoji suggestions for text with underscore as many emojis have underscore in them

Actual Result:

App does not display emoji suggestions for text with underscore

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

  • Android / native
  • Android / Chrome
  • iOS / native
  • iOS / Safari
  • MacOS / Chrome / Safari
  • MacOS / Desktop

Version Number: 1.3.79.3
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

android.native.underscore.does.not.work.suggestion.mov
Recording.4897.mp4
mac.chrome.desktop.underscore.not.allowed.in.emoji.suggestion.mov
ios.safari.native.underscore.not.allowed.in.emoji.suggestion.mov
android.chrome.underscore.breaks.suggestions.mp4
windows.chrome.underscore.breaks.suggestions.mp4
android.native.timezone.position.error.1.mov

Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1696680165185589

View all open jobs on GitHub

Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~01f81536e559725a51
  • Upwork Job ID: 1711343562494205952
  • Last Price Increase: 2023-10-09
  • Automatic offers:
    • eh2077 | Contributor | 27243430
    • dhanashree-sawant | Reporter | 27243434
Issue OwnerCurrent Issue Owner: @kadiealexander
@kbecciv kbecciv added External Added to denote the issue can be worked on by a contributor Daily KSv2 Bug Something is broken. Auto assigns a BugZero manager. labels Oct 9, 2023
@melvin-bot melvin-bot bot changed the title Chat - Underscore doesn't work in emoji suggestions (works fine in search) [$500] Chat - Underscore doesn't work in emoji suggestions (works fine in search) Oct 9, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 9, 2023

Job added to Upwork: https://www.upwork.com/jobs/~01f81536e559725a51

@melvin-bot
Copy link

melvin-bot bot commented Oct 9, 2023

Triggered auto assignment to @kadiealexander (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

@melvin-bot melvin-bot bot added the Help Wanted Apply this label when an issue is open to proposals by contributors label Oct 9, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 9, 2023

Bug0 Triage Checklist (Main S/O)

  • This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
  • This bug is not a duplicate report (check E/App issues and #expensify-bugs)
    • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
  • This bug is reproducible using the reproduction steps in the OP. S/O
    • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
    • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
  • This issue is filled out as thoroughly and clearly as possible
    • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
  • I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

@saranshbalyan-1234
Copy link
Contributor

saranshbalyan-1234 commented Oct 9, 2023

Proposal

Please re-state the problem that we are trying to solve in this issue.

underscore doesn't work in emoji suggestions in composer but works fine in emoji picker

What is the root cause of that problem?

App/src/CONST.ts

Lines 1277 to 1279 in 389d7b0

SPECIAL_CHAR_OR_EMOJI:
// eslint-disable-next-line no-misleading-character-class
/[\n\s,/?"{}[\]()&_~^%\\;`$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3/gu,

The issue is with regex that is used to test the input from composer,.
const isEmojiCode = (str, pos) => {
const leftWords = str.slice(0, pos).split(CONST.REGEX.SPECIAL_CHAR_OR_EMOJI);
const leftWord = _.last(leftWords);
return CONST.REGEX.HAS_COLON_ONLY_AT_THE_BEGINNING.test(leftWord) && leftWord.length > 2;
};

the above code is splitting the inputText based upon the regex which also includes special characters including underscore(_) and then select the last from the leftWords array,
after using underscore the last word is next to underscore which causes this behaviour

What changes do you think we should make in order to solve the problem?

First of all change the isEmojiCode function to like this below

const isEmojiCode = (str, pos) => {
    const after_ = str.slice(str.indexOf(':'));
    const leftWords = after_.slice(0, pos).split(CONST.REGEX.SPECIAL_CHAR_OR_EMOJI);
    const leftWord = _.last(leftWords);
    return CONST.REGEX.HAS_COLON_ONLY_AT_THE_BEGINNING.test(leftWord) && leftWord.length > 2;
};

and remove the underscore from the regex and change the regex to this

/[\n\s,/?"{}[\]()&~^%\\;`$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3/gu,

Result

screen-recording-2023-10-07-at-73636-pm_zHolOi5f.mp4

What alternative solutions did you explore? (Optional)

N/A

@melvin-bot
Copy link

melvin-bot bot commented Oct 9, 2023

Triggered auto assignment to Contributor-plus team member for initial proposal review - @mollfpr (External)

@abzokhattab
Copy link
Contributor

Proposal

Please re-state the problem that we are trying to solve in this issue.

Placing underscore in an emoji disables the emoji selection

What is the root cause of that problem?

in the following regex: we unmatch the _ before and after the colon

App/src/CONST.ts

Lines 1277 to 1279 in 389d7b0

SPECIAL_CHAR_OR_EMOJI:
// eslint-disable-next-line no-misleading-character-class
/[\n\s,/?"{}[\]()&_~^%\\;`$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3/gu,

What changes do you think we should make in order to solve the problem?

we should match any underscore after the colon but not before the colon:

/[\n\s,/?"{}[\]()&~^%\\;`$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3|:.+/gu;

also we need to use str.match instead of str.split when matching the regex in here:

https://github.com/abzokhattab/App/blob/c0fe5051041d29b585bd01d9fd9455f5eb2b3a5f/src/pages/home/report/ReportActionCompose/SuggestionEmoji.js#L21

const leftWords = str.slice(0, pos).match(CONST.REGEX.SPECIAL_CHAR_OR_EMOJI);

POC

Screen.Recording.2023-10-08.at.2.46.03.AM.mov

@ZhenjaHorbach
Copy link
Contributor

ZhenjaHorbach commented Oct 9, 2023

Proposal

Please re-state the problem that we are trying to solve in this issue.

Underscore doesn’t work in emoji suggestions (works fine in search)

What is the root cause of that problem?

The main problem is that, unlike search, we have an additional check

const isCurrentlyShowingEmojiSuggestion = isEmojiCode(value, selectionEnd);

As a result, a regular expression is used that does not allow us to use _ and several other characters

App/src/CONST.ts

Lines 1277 to 1280 in 389d7b0

SPECIAL_CHAR_OR_EMOJI:
// eslint-disable-next-line no-misleading-character-class
/[\n\s,/?"{}[\]()&_~^%\\;`$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3/gu,

What changes do you think we should make in order to solve the problem?

We can update this regular like
/[\n\s,/?"{}[\]()&~^%\\;$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3/gu`

What alternative solutions did you explore? (Optional)

We can don’t use isCurrentlyShowingEmojiSuggestion to make the same behavior as in search

@eh2077
Copy link
Contributor

eh2077 commented Oct 9, 2023

Proposal

Please re-state the problem that we are trying to solve in this issue.

Underscore doesn't work in emoji suggestions (works fine in search)

What is the root cause of that problem?

The root cause of this issue is that regex split string by underscore _.

App/src/CONST.ts

Lines 1277 to 1280 in 619aee6

SPECIAL_CHAR_OR_EMOJI:
// eslint-disable-next-line no-misleading-character-class
/[\n\s,/?"{}[\]()&_~^%\\;`$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3/gu,

For example, if we type :joy_ in composer, it'll be split into array [':joy', '']. And method isEmojiCode return false, see

const isEmojiCode = (str, pos) => {
const leftWords = str.slice(0, pos).split(CONST.REGEX.SPECIAL_CHAR_OR_EMOJI);
const leftWord = _.last(leftWords);
return CONST.REGEX.HAS_COLON_ONLY_AT_THE_BEGINNING.test(leftWord) && leftWord.length > 2;
};

So emoji suggestion is not shown, see

const isCurrentlyShowingEmojiSuggestion = isEmojiCode(value, selectionEnd);
const nextState = {
suggestedEmojis: [],
colonIndex,
shouldShowEmojiSuggestionMenu: false,
};
const newSuggestedEmojis = EmojiUtils.suggestEmojis(leftString, preferredLocale);
if (newSuggestedEmojis.length && isCurrentlyShowingEmojiSuggestion) {
nextState.suggestedEmojis = newSuggestedEmojis;
nextState.shouldShowEmojiSuggestionMenu = !_.isEmpty(newSuggestedEmojis);
}

What changes do you think we should make in order to solve the problem?

The character underscore _ is a word character in JavaScript and is used in emoji code. So we should not split the underscore in the emoji code part.

To fix this issue, we can remove _ from the regex and add an alternative (_\b(?!$)) to split underscore followed by a word boundary and it's not the suffix of the string. The new regex to split text can be

/[\n\s,/?"{}[\]()&~^%\\;`$=#<>!*\p{Extended_Pictographic}\u200d\u{1f1e6}-\u{1f1ff}\u{1f3fb}-\u{1f3ff}\u{e0020}-\u{e007f}\u20E3\uFE0F]|[#*0-9]\uFE0F?\u20E3|(_\b(?!$))/gu,

So, text :joy_ will be splited into array [':joy_'] and suggestion is shown as expected. See demo video for various examples

Screen.Recording.2023-10-09.at.9.35.09.PM.mov

What alternative solutions did you explore? (Optional)

N/A

@mollfpr
Copy link
Contributor

mollfpr commented Oct 10, 2023

@saranshbalyan-1234 Your solution does not working on _:joy, it should be open the suggestion.


also we need to use str.match instead of str.split when matching the regex in here:

@abzokhattab Why is that?


@ZhenjaHorbach Your regex doesn't work.

Screenshot 2023-10-11 at 00 58 33


@ZhenjaHorbach
Copy link
Contributor

@mollfpr
Oh sorry
It's not me )))
When I copy text \\ it turns into \

@saranshbalyan-1234
Copy link
Contributor

@saranshbalyan-1234 Your solution does not working on _:joy, it should be open the suggestion.

also we need to use str.match instead of str.split when matching the regex in here:

@abzokhattab Why is that?

@ZhenjaHorbach Your regex doesn't work.

Screenshot 2023-10-11 at 00 58 33

Proposal updated #29081 (comment)
@mollfpr sorry i missed this edge case, could please check again, i have updated the proposal

@saranshbalyan-1234
Copy link
Contributor

all the conditions should now be handled with the updated proposal #29081 (comment)

@abzokhattab
Copy link
Contributor

abzokhattab commented Oct 10, 2023

@abzokhattab Why is that?

changed split to match because we want to extract and capture the matched special characters or emojis from the string, whereas split is used to split the string at those matched characters, effectively removing them from the resulting array. Using match ensures that you keep the matched characters in the array for further processing.

all tests are passing

image

I think we need to take this issue into consideration when reviewing the proposals

@melvin-bot melvin-bot bot added the Overdue label Oct 13, 2023
@mollfpr
Copy link
Contributor

mollfpr commented Oct 15, 2023

Running out of time, I'll get this reviewed in the morning 🙏

@melvin-bot melvin-bot bot removed the Overdue label Oct 15, 2023
@mollfpr
Copy link
Contributor

mollfpr commented Oct 16, 2023

Thank you all for the proposals!

I see a lot of ways to solve this issue, and I would like to have the solution simpler. The proposal from @eh2077 seems to have done the job, and it's not making a lot of change. I don't see any regression from the updated regex.

🎀 👀 🎀 C+ reviewed!

@kadiealexander Could you add the engineering label? Thank you!

@melvin-bot
Copy link

melvin-bot bot commented Oct 17, 2023

Triggered auto assignment to @tgolen (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

@tgolen
Copy link
Contributor

tgolen commented Oct 17, 2023

Thanks @mollfpr I agree with that regex change proposal.

@melvin-bot melvin-bot bot removed the Help Wanted Apply this label when an issue is open to proposals by contributors label Oct 17, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 17, 2023

🎯 ⚡️ Woah @eh2077, great job pushing this forwards! ⚡️

The pull request got merged within 3 working days of assignment, so this job is eligible for a 50% #urgency bonus 🎉

  • when @eh2077 got assigned: 2023-10-17 16:09:16 Z
  • when the PR got merged: 2023-10-17 17:53:58 UTC

On to the next one 🚀

@melvin-bot melvin-bot bot added Weekly KSv2 Awaiting Payment Auto-added when associated PR is deployed to production and removed Weekly KSv2 labels Oct 18, 2023
@melvin-bot melvin-bot bot changed the title [$500] Chat - Underscore doesn't work in emoji suggestions (works fine in search) [HOLD for payment 2023-10-25] [$500] Chat - Underscore doesn't work in emoji suggestions (works fine in search) Oct 18, 2023
@melvin-bot melvin-bot bot removed the Reviewing Has a PR in review label Oct 18, 2023
@melvin-bot
Copy link

melvin-bot bot commented Oct 18, 2023

Reviewing label has been removed, please complete the "BugZero Checklist".

@melvin-bot
Copy link

melvin-bot bot commented Oct 18, 2023

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.86-5 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

If no regressions arise, payment will be issued on 2023-10-25. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

  • External issue reporter
  • Contributor that fixed the issue
  • Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

  • Merged PR within 3 business days of assignment - 50% bonus
  • Merged PR more than 9 business days after assignment - 50% penalty

@melvin-bot
Copy link

melvin-bot bot commented Oct 18, 2023

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

@mollfpr
Copy link
Contributor

mollfpr commented Oct 25, 2023

The PR that introduced the bug has been identified. Link to the PR:

#27984

The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:

https://github.com/Expensify/App/pull/27984/files#r1371846486

A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:

The regression step should be enough.

Determine if we should create a regression test for this bug.
If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.

  1. Go to a chat and add a message, :joy_
  2. Verify emoji suggestion is shown
  3. Type message, :joy
  4. Verify emoji suggestion is also shown
  5. Move the cursor to before the ending underscore
  6. Verify emoji suggestions are shown
  7. 👍 or 👎

@kadiealexander
Copy link
Contributor

kadiealexander commented Oct 27, 2023

Payouts due:

Eligible for 50% #urgency bonus? Yes

Upwork job is here.

@mollfpr
Copy link
Contributor

mollfpr commented Oct 27, 2023

Thanks @kadiealexander for the payment summary. I'll do manual request in NewDot.

@kadiealexander
Copy link
Contributor

I will withdraw the offer I just sent 😂 Thanks @mollfpr

@kadiealexander kadiealexander changed the title [HOLD for payment 2023-10-25] [$500] Chat - Underscore doesn't work in emoji suggestions (works fine in search) [HOLD for C+ manual request payment] [$500] Chat - Underscore doesn't work in emoji suggestions (works fine in search) Oct 27, 2023
@kadiealexander kadiealexander added Weekly KSv2 and removed Daily KSv2 labels Oct 27, 2023
@melvin-bot melvin-bot bot added the Overdue label Oct 30, 2023
@kadiealexander
Copy link
Contributor

Not overdue.

@melvin-bot melvin-bot bot removed the Overdue label Oct 31, 2023
@JmillsExpensify
Copy link

$750 payment approved for @mollfpr based on this summary.

@melvin-bot melvin-bot bot added the Overdue label Nov 8, 2023
@tgolen
Copy link
Contributor

tgolen commented Nov 9, 2023

Can this be closed out now?

@eh2077
Copy link
Contributor

eh2077 commented Nov 9, 2023

I think yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Awaiting Payment Auto-added when associated PR is deployed to production Bug Something is broken. Auto assigns a BugZero manager. Engineering External Added to denote the issue can be worked on by a contributor Weekly KSv2
Projects
None yet
Development

No branches or pull requests

9 participants