Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network mocking for all UI and screenshot tests #11788

Merged
merged 10 commits into from
Jun 11, 2019
Merged

Conversation

jtreanor
Copy link
Contributor

@jtreanor jtreanor commented May 28, 2019

This PR updates all of our UI and screenshot tests to use mocked networking using WireMock. The mocking functionality is shared with WPAndroid from the WordPressMocks repo.

It relies on the following open PRs:

Other related PRs:

To test:

  • Check out this branch. Run rake dependencies as usual.
  • Run rake mocks to start the mock server.
  • Run the UI tests in the WordPressUITests target with Xcode.
  • All tests should pass and there should be no errors/warnings in the terminal when you ran the mock server.

Note: The screenshot tests are broken in develop due to the recent stats changes. I think we should fix that in a separate change.

Update release notes:

  • If there are user facing changes, I have added an item to RELEASE-NOTES.txt.

@jtreanor jtreanor added this to the 12.6 milestone May 28, 2019
@jtreanor jtreanor added Tooling Build, Release, and Validation Tools Testing Unit and UI Tests and Tooling labels May 28, 2019
@jtreanor jtreanor force-pushed the ui-test-mocking branch 2 times, most recently from 979a2df to 192515e Compare May 28, 2019 12:04
@wordpress-mobile wordpress-mobile deleted a comment from houndci-bot May 28, 2019
@jkmassel jkmassel modified the milestones: 12.6, Someday Jun 3, 2019
@jtreanor jtreanor marked this pull request as ready for review June 7, 2019 14:53
@jtreanor jtreanor requested a review from rachelmcr June 10, 2019 08:17
Copy link
Member

@rachelmcr rachelmcr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed the tests in the WordPressUITests target all pass. Thanks for finishing the magic link test with the mocking!

However, I did get this warning in the terminal when I started the mock server:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

And while running the UI tests on an iPad simulator (iPad Pro, 12.9-inch, 3rd Generation), I saw a number of "Request was not matched" errors (although the tests still pass). These didn't show up when running the tests on an iPhone simulator. Here were the requests without a matching stub:

/rest/v1.2/read/sites/53424024?locale=en
/rest/v1.1/sites/158396482/stats//?locale=en&max=10
/v1.1/users/suggest?locale=en&site_id=158396482

Let me know if you need more details or can't reproduce that on your end.

Note: The screenshot tests are broken in develop due to the recent stats changes. I think we should fix that in a separate change.

Agreed, I think that's fine to address separately.

@jtreanor
Copy link
Contributor Author

Thanks @rachelmcr!

However, I did get this warning in the terminal when I started the mock server:

This is expected and fine. It doesn't affect us.

And while running the UI tests on an iPad simulator (iPad Pro, 12.9-inch, 3rd Generation), I saw a number of "Request was not matched" errors (although the tests still pass).

Oh great, thank you! These should all be quick to fix.

@jtreanor
Copy link
Contributor Author

Okay, I've updated wordpress-mobile/WordPressMocks#3 with the missing requests.

@rachelmcr It should all be working now. Once you pull, you will need to run rake dependencies and then rake mocks again.

@rachelmcr
Copy link
Member

Looks great now! :shipit:

@jtreanor
Copy link
Contributor Author

Thanks, @rachelmcr! Could you also take a quick look at wordpress-mobile/WordPressMocks#3? That's where the mocks live and you have already effectively tested them by reviewing this 😄

Once that is merged, we can merge this PR and get these tests running on CI.

@jtreanor
Copy link
Contributor Author

Okay, I have updated to the release version of WordPressMocks. There are no changes, so I will merge when green.

@jtreanor jtreanor merged commit 5e62f57 into develop Jun 11, 2019
@jtreanor jtreanor deleted the ui-test-mocking branch June 11, 2019 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing Unit and UI Tests and Tooling Tooling Build, Release, and Validation Tools
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants