Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Enable failed test logging and fix flaky UT #910

Merged
merged 4 commits into from
Dec 10, 2020

Conversation

dai-chen
Copy link
Member

@dai-chen dai-chen commented Dec 10, 2020

Issue #, if available: #909

Description of changes:

  1. Fix UT: Suspect root cause is the use of HashSet instead of LinkedHashSet. Thus fix it and also split original UT into two in case any more flaky behavior.
  2. Enable test logging for failed UT in ppl, sql, core, elasticsearch, protocol, legacy module: reference https://stackoverflow.com/questions/39350802/gradle-stack-trace-on-terminal

Sample output:

Here is an example of failed UT with current changes:

com.amazon.opendistroforelasticsearch.sql.sql.parser.AstBuilderTest > can_build_count_literal() FAILED
    org.opentest4j.AssertionFailedError: expected: <Project(projectList=[Alias(name=COUNT(1), delegated=COUNT(1), alias=null)], argExprList=[], child=[Aggregation(aggExprList=[Alias(name=COUNT(1), delegated=COUNT(1), alias=null)], sortExprList=[], groupExprList=[], argExprList=[], child=[Relation(tableName=test, alias=null)])])> but was: <Project(projectList=[Alias(name=1, delegated=1, alias=null)], argExprList=[], child=[Relation(tableName=test, alias=null)])>
        at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
        at org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
        at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
        at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
        at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1124)
        at com.amazon.opendistroforelasticsearch.sql.sql.parser.AstBuilderTest.can_build_count_literal(AstBuilderTest.java:201)

Previously without current PR changes:

com.amazon.opendistroforelasticsearch.sql.sql.parser.AstBuilderTest > can_build_count_literal() FAILED
    org.opentest4j.AssertionFailedError at AstBuilderTest.java:201

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@dai-chen dai-chen added the infra A change to infrastructure, testing, CI/CD, pipelines, etc. label Dec 10, 2020
@dai-chen dai-chen self-assigned this Dec 10, 2020
@dai-chen dai-chen changed the title Fix flaky ut Enable failed test logging and fix flaky UT Dec 10, 2020
@codecov
Copy link

codecov bot commented Dec 10, 2020

Codecov Report

Merging #910 (ac50522) into develop (2a932d8) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             develop     #910   +/-   ##
==========================================
  Coverage      99.85%   99.85%           
  Complexity      2149     2149           
==========================================
  Files            216      216           
  Lines           4851     4851           
  Branches         323      323           
==========================================
  Hits            4844     4844           
  Misses             5        5           
  Partials           2        2           
Impacted Files Coverage Δ Complexity Δ
...rch/sql/sql/parser/context/QuerySpecification.java 100.00% <100.00%> (ø) 15.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a932d8...ac50522. Read the comment docs.

@dai-chen dai-chen marked this pull request as ready for review December 10, 2020 19:44
Copy link
Member

@chloe-zh chloe-zh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the changes!

@dai-chen dai-chen merged commit d170da3 into opendistro-for-elasticsearch:develop Dec 10, 2020
@dai-chen dai-chen deleted the fix-flaky-ut branch December 10, 2020 21:03
penghuo pushed a commit that referenced this pull request Dec 15, 2020
* Fix UT and add logging in gradle

* Fix UT and add logging in gradle

* Simplify test logging config
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
infra A change to infrastructure, testing, CI/CD, pipelines, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants