Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: fix flaky test Increase Token Allowance increases token spending cap to allow other accounts to transfer tokens @no-mmi #24734

Merged
merged 1 commit into from
May 23, 2024

Conversation

seaona
Copy link
Contributor

@seaona seaona commented May 23, 2024

Description

This PR fixes the flaky test Increase Token Allowance increases token spending cap to allow other accounts to transfer tokens @no-mmi).
The problem with this flaky test is that we are changing the allowance amount (from 0 to 1) and we confirm the transaction before the gas has been updated. This makes that the increase token allowance transaction fails, and the subsequent transaction with the Account 2 cannot be performed as it doesn't have the allowance for doing a transfer from action. So the dialog window cannot be found, as the popup is never triggered (see dapp error).

The fix is to simply await for the expected gas for the increaseAllowance with amount 1.

This can be clearly seen when you check the tx history from Account 1:

  • Here is a successful increaseAllowance with amount 1, and the total gas is 31000 -- correct

Screenshot from 2024-05-23 09-41-59

  • Here is a failed increaseAllowance with amount 1, and the total gas limit is 26776 -- incorrect (this value corresponds to the increaseAllowance with amount 0)

Screenshot from 2024-05-23 09-41-53

Open in GitHub Codespaces

Related issues

Fixes: one of this cases, but not all of them, as the rest of testcases need to be looked separately #24603

Manual testing steps

  1. Run this test locally multiple times
  2. Check ci jobs yarn test:e2e:single test/e2e/tests/tokens/increase-token-allowance.spec.js --browser=firefox --leave-running --retryUntilFailure --retries=10

Screenshots/Recordings

increase-token-allowance-flaky.mp4

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've completed the PR template to the best of my ability
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@seaona seaona self-assigned this May 23, 2024
Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@seaona seaona marked this pull request as ready for review May 23, 2024 07:57
@seaona seaona requested a review from a team as a code owner May 23, 2024 07:57
@@ -272,10 +272,14 @@ describe('Increase Token Allowance', function () {
tag: 'button',
text: 'Next',
});
driver.waitForSelector({
await driver.waitForSelector({
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this await was missing, but it's not the cause of the flakiness

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch.

await driver.waitForSelector({
tag: 'h6',
text: '0.000062 ETH',
});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we await for the correct gas estimate. This way we ensure that the increaseAllowance tx is successful and we can proceed with the transferFrom, from account 2

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(could) add explicit expected maximum timeout in seconds here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for the review @legobeat 🙇‍♀️ I think in this case is safe to rely on the default timeout, since I've observed that the gas update is really a matter of ms, so the default timeout seems more than enough.
But let me know if you think otherwise and we could add it 🙏

async waitForSelector(
    rawLocator,
    { timeout = this.timeout, state = 'visible' } = {}

Screenshot from 2024-05-23 10-11-42

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ after talking with Lego, we've agreed to merge this PR but we could have another task to look into explicit timeouts more generally

@seaona seaona added the needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) label May 23, 2024
@legobeat legobeat requested a review from a team May 23, 2024 08:04
@metamaskbot
Copy link
Collaborator

Builds ready [798cf22]
Page Load Metrics (904 ± 550 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint611691013416
domContentLoaded85017115
load4926619041145550
domInteractive85017115
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

@seaona seaona merged commit c10a4a3 into develop May 23, 2024
86 of 89 checks passed
@seaona seaona deleted the flaky-fix-increase-token-allowance branch May 23, 2024 11:18
@github-actions github-actions bot locked and limited conversation to collaborators May 23, 2024
@github-actions github-actions bot removed the needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) label May 23, 2024
@metamaskbot metamaskbot added release-11.16.3 Issue or pull request that will be included in release 11.16.3 and removed release-11.18.0 labels May 31, 2024
@metamaskbot
Copy link
Collaborator

Missing release label release-11.16.3 on PR. Adding release label release-11.16.3 on PR and removing other release labels(release-11.18.0), as PR was cherry-picked in branch 11.16.3.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
flaky tests release-11.16.3 Issue or pull request that will be included in release 11.16.3 team-extension-platform
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants