-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add acceptance tests #84
Conversation
.github/workflows/test.yml
Outdated
- name: Run WP Acceptance | ||
if: matrix.php-versions == '7.2' | ||
run: | | ||
if [ -n "${{ secrets.AWS_ACCESS_KEY }}" ]; then ./vendor/bin/wpsnapshots configure 10up --aws_key=${{ secrets.AWS_ACCESS_KEY }} --aws_secret=${{ secrets.SECRET_ACCESS_KEY }} --user_name=Travis [email protected]; fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jeffpaul We'll need AWS_ACCESS_KEY and SECRET_ACCESS_KEY added to the repo secrets for the WP Acceptance snapshot. I adapted this line from Distributor, which uses Travis and passes Travis user credentials, though it should work the same in the GH action.
@dinhtungdu This is ready for review. Took me 1,000 commits to get the WP Acceptance tests to pass 😫 |
composer.json
Outdated
@@ -14,18 +14,28 @@ | |||
"role": "Developer" | |||
} | |||
], | |||
"repositories": [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will create a follow-up issue for this when this PR is merged. We'll need to switch back to the main WP Acceptance repo after Felipe's PR is merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@johnwatkins0 FYI that WPA has been updated from Felipe's PR.
.github/workflows/test.yml
Outdated
@@ -42,3 +43,8 @@ jobs: | |||
run: | | |||
composer run setup-local-tests | |||
composer test | |||
- name: Run WP Acceptance | |||
if: matrix.php-versions == '7.3' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we run tests on 7.2 too?
@jeffpaul @dinhtungdu I'm having a lot of trouble with the GitHub action for this. After getting Acceptance tests to pass, the action is now failing on PHPUnit because mysql doesn't appear to be available in the container as of yesterday. I don't know what changed. I've done some searching in the GH forums and so far no one else is having this problem. I propose: |
@johnwatkins0 last year we had an odd issue on ClassifAI and Travis and had to add this line to get things working again: https://github.com/10up/classifai/blob/978d79c9c6d1573a65bbac2db787366125a6f94f/.travis.yml#L27 It looks like you've got something similar in https://github.com/10up/autoshare-for-twitter/blob/feature/wp-acceptance-tests/.github/workflows/test.yml so maybe something's off in the way we're referencing |
@jeffpaul Yeah, last night I found myself back in that Slack conversation about the missing |
Co-Authored-By: Helen Hou-Sandi <[email protected]>
Co-Authored-By: Helen Hou-Sandi <[email protected]>
Co-Authored-By: Helen Hou-Sandi <[email protected]>
@johnwatkins0 anything else planned on this PR or are we good to merge this in an have some fun new tests? |
@jeffpaul Yes, it's good to merge. I removed a couple of my more complicated acceptance tests in my last commit, specifically:
Both of these tests were failing about 25% of the time. This is a little worrying, but I have not been able to replicate the failures in manual testing. If there is a bug in those scenarios, I think it's one that requires super-fast fingers for a human to replicate. Maybe we could merge this in and create a new issue for WP Acceptance tests for those two scenarios. |
Description of the Change
Resolves #17 by adding and configuring WP Acceptance and setting up a snapshot for testing.
Resolves #19 by adding an initial set of acceptance tests covering post save functionality.
Alternate Designs
Further tests could be added to cover the settings page.
Benefits
Acceptance tests can now be added for new features and bug fixes.
Possible Drawbacks
Verification Process
Tests in the GH Actions CI process should all pass.
Checklist:
Applicable Issues
Changelog Entry