Streamline 3DS UI Tests

[scannillo]

Summary of changes

• Remove all 3DS UI tests except the following cases:
  1. Successfully completes challenge flow & returns to app
  2. Cancels 3DS challenge flow & returns to app
• Why?
  • The SDK's test suite is not responsible for serving as E2E integration tests for numerous test card cases. Our UI tests should capture the functionality that our SDK adds to a checkout experience.
  • UI tests are timely and expensive. Read about Mike Cohn's "Testing Pyramid" concept, where UI tests should make up the top, smallest part of the pyramid.

Tests removed:

Total time reduced: 2 min 20 sec (140 seconds)

    ✔ testThreeDSecurePaymentFlowV2_frictionlessFlow_andTransacts (17.442 seconds)
    ✔ testThreeDSecurePaymentFlowV2_noChallenge_andFails (17.517 seconds)
    ✔ testThreeDSecurePaymentFlowV2_acceptsPassword_failsToAuthenticateNonce_dueToCardinalError (28.011 seconds)
    ✔ testThreeDSecurePaymentFlowV2_bypassedAuthentication (19.161 seconds)
    ✔ testThreeDSecurePaymentFlowV2_lookupError (17.653 seconds)
    ✔ testThreeDSecurePaymentFlowV2_timeout (41.804 seconds)

Feedback

If folks agree that these 2 cases are the only ones we need to UI test for, we can make the same update to our Android 3DS UI tests. Keep me honest here and call out any concerns you may have.

Checklist

• Added a changelog entry

Authors

@scannillo

[jaxdesmarais]

This make sense to me. Thanks for sharing that testing pyramid resource, I had not seen that before.

My only potential concern is that the testing pyramid seems to rely heavily on a QA team for end to end testing, which we don't currently have. Would it be worth adding an autofill button or something similar for some of these cases we are removing? That way if we are touching the 3DS flow we can ensure something like error handling is still working as expected? I am not sure we need all of them, but could see something like and error being helpful. What do you think?

[scannillo]

My only potential concern is that the testing pyramid seems to rely heavily on a QA team for end to end testing, which we don't currently have. Would it be worth adding an autofill button or something similar for some of these cases we are removing?

Yeh, that crossed my mind. We don't have a QA team, and we sometimes play that role, even though I don't think it's our responsibility. We can also consider running the UI tests less often as well.

We could add back in the error UI test case? So we would have 3: success w/ challenge, error on challenge, cancel.

[jaxdesmarais]

We could add back in the error UI test case? So we would have 3: success w/ challenge, error on challenge, cancel.

This makes sense to me and I think is a good middle ground.

[agedd]

🚀