Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update test selectors to improve test flakiness #19679

Closed
liza-mae opened this issue Jun 5, 2018 · 5 comments
Closed

Update test selectors to improve test flakiness #19679

liza-mae opened this issue Jun 5, 2018 · 5 comments

Comments

@liza-mae
Copy link
Contributor

liza-mae commented Jun 5, 2018

I am currently running the Kibana UI tests for cloud instance and I had about 22% failures. A lot had to do with the selectors not working and I noticed the following usage which should be changed:

use of .ng-scope, .ng-binding, .ng-hide and nth child

These selectors make the tests flaky / brittle.

We should move towards using data-test-subj.

To temporarily get the tests on cloud to pass I am going to remove the .ng-scope from the tests.

@liza-mae liza-mae added the test label Jun 5, 2018
@liza-mae liza-mae self-assigned this Jun 5, 2018
@cjcenizal
Copy link
Contributor

I agree we should be moving away from brittle selectors (e.g. those which depend upon DOM structure or CSS) and to test-specific attributes (data-test-subj).

To temporarily get the tests on cloud to pass I am going to remove the .ng-scope from the tests.

@liza-mae Do we know why this fixes the tests? I may not fully understand the difference between running these tests on a cloud instance vs. locally, but this seems unusual.

@liza-mae
Copy link
Contributor Author

liza-mae commented Jun 5, 2018

@cjcenizal I don't know why there is this particular difference between cloud and running locally. I do have failures locally, I just have not had a chance to go through those. Focusing on cloud first. In any case removing the .ng stuff should not break existing tests. It takes 1 and half hours to test the whole suite when trying something, so it takes time for me to get results, but can post once I have them.

@liza-mae
Copy link
Contributor Author

liza-mae commented Jun 5, 2018

Although this one is not causing me specific issues, since it was brought up that it may be causing problems. We should move away from using ng-hide or getting .getAttribute('aria-hidden'), selenium isElementPresent function should take care of this for us.
async isGlobalLoadingIndicatorHidden() {
log.debug('isGlobalLoadingIndicatorHidden');
return await find.byCssSelector('[data-test-subj="globalLoadingIndicator"].ng-hide',
defaultFindTimeout * 10);
}

@liza-mae
Copy link
Contributor Author

liza-mae commented Jun 5, 2018

Applying the fix for this issue reduced pretty mostly all the failures I was seeing.
There are still 6 failures related to this issue, that can't easily be removed, so I will need to investigate.
The remaining 6 or so issues are assertion failures.

@liza-mae
Copy link
Contributor Author

Merged #19688, closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants