[QA][Code Coverage] Drop catchError and use try / catch instead, #69198

wayneseymour · 2020-06-15T19:53:37Z

to fail the build on test run and / or ingestion.

Summary

We no longer wish to be notified for unstable builds. However, if the coverage
ingestion fails, we want the build to fail and slack notify our team.

to fail the build on test run and / or ingestion.

wayneseymour

@dmlemeshko Perhaps we should only wrap the handleIngestion fn with the try / catch? Your thoughts man?

dmlemeshko · 2020-06-15T23:23:12Z

@dmlemeshko Perhaps we should only wrap the handleIngestion fn with the try / catch? Your thoughts man?

I think it won't hurt to keep runTest() as well:

If jest fails, we are not getting coverage reports and no need to ingest, return red build
If functional fails, we proceed unless moving files fails and no chance for ingestion again, return the red build

// Worker for running functional tests. Runs a setup process (e.g. the kibana build) then executes a map of closures in parallel (e.g. one for each ciGroup)
def functional(name, Closure setup, Map processes) {
  return {
    parallelProcesses(name: name, setup: setup, processes: processes, delayBetweenProcesses: 20, size: 'xl')
  }
}

I don't see failure is handled here, so probably we must wrap it up to handle and report failure

dmlemeshko · 2020-06-15T23:25:10Z

@wayneseymour could you push a breaking commit to see it really works for broken ingestion/incorrect vault data?

wayneseymour · 2020-06-16T16:55:57Z

@wayneseymour could you push a breaking commit to see it really works for broken ingestion/incorrect vault data?

Yeah great idea!

…-coverage-ingestion-error

wayneseymour · 2020-06-16T20:03:35Z

Last build failed, now testing: https://kibana-ci.elastic.co/job/elastic+kibana+qa-research/11/console

wayneseymour · 2020-06-16T21:00:40Z

testing again: https://kibana-ci.elastic.co/job/elastic+kibana+qa-research/12/

…-coverage-ingestion-error

wayneseymour · 2020-06-16T23:23:44Z

@dmlemeshko @LeeDr This job keeps breaking with:

15:02:33  repoInfo: [branch:fail-build-on-coverage-ingestion-error, targetBranch:null, commit:a679685bcda1e7bca9ee562b60c82857b07139d9]

I'm not sure what's going on.

From: https://kibana-ci.elastic.co/job/elastic+kibana+qa-research/12/console

wayneseymour · 2020-06-17T19:40:14Z

@dmlemeshko @LeeDr This job keeps breaking with:
15:02:33  repoInfo: [branch:fail-build-on-coverage-ingestion-error, targetBranch:null, commit:a679685bcda1e7bca9ee562b60c82857b07139d9]
I'm not sure what's going on.

From: kibana-ci.elastic.co/job/elastic+kibana+qa-research/12/console

@dmlemeshko ideas?

wayneseymour · 2020-06-29T15:36:25Z

@elasticmachine merge upstream

wayneseymour · 2020-06-29T15:37:46Z

Testing: https://kibana-ci.elastic.co/job/elastic+kibana+qa-research/21

…lastic/kibana into fail-build-on-coverage-ingestion-error

…-coverage-ingestion-error

wayneseymour · 2020-06-29T19:08:02Z

Testing: kibana-ci.elastic.co/job/elastic+kibana+qa-research/23

dmlemeshko

Code LGTM. Interesting to hear Brian opinion :)

wayneseymour · 2020-06-29T19:42:53Z

@dmlemeshko Yeah I think @brianseeders can affirm whether our assumptions of catchError is correct

brianseeders · 2020-06-29T21:09:25Z

catchError { doSomething() }

is basically the same as

try {
  doSomething()
} catch(err) {
  // Set the build status to failure
  // Print the error
  // Maybe other stuff?
}

So, I don't think you need this PR at all, unless I'm not understanding your intentions.

wayneseymour · 2020-06-29T21:54:36Z

@brianseeders ok, well one consideration is that we do not want to be notified for unstable builds, just failed builds.

brianseeders · 2020-06-30T00:27:28Z

@brianseeders ok, well one consideration is that we do not want to be notified for unstable builds, just failed builds.

You could add another condition to handleFail() that checks for UNSTABLE

update handleFail fn to not slack notifiy if the build is unstable, per Brian S.

wayneseymour · 2020-06-30T17:19:29Z

Testing https://kibana-ci.elastic.co/job/elastic+kibana+qa-research/25/

kibanamachine · 2020-06-30T19:10:09Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 3c9c99f

Build metrics

✅ unchanged

History

💚 Build #57288 succeeded 8d70c7d
💚 Build #57200 succeeded 3de6b3a
💚 Build #54432 succeeded a679685
💚 Build #54164 succeeded 448bab6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…stic#69198) Co-authored-by: Elastic Machine <[email protected]>

…-based-rbac * upstream/master: (38 commits) Move logger configuration integration test to jest (elastic#70378) Changes observability plugin codeowner (elastic#70439) update (elastic#70424) [Logs UI] Avoid CCS-incompatible index name resolution (elastic#70179) Enable "Explore underlying data" actions for Lens visualizations (elastic#70047) Initial work on uptime homepage API (elastic#70135) expressions indexPattern function (elastic#70315) [Discover] Deangularization context error message refactoring (elastic#70090) [Lens] Add "no data" popover (elastic#69147) [Lens] Move chart switcher over (elastic#70182) chore: add missing mjs extension (elastic#70326) [Lens] Multiple y axes (elastic#69911) skip flaky suite (elastic#70386) fix bug to add timeline to case (elastic#70343) [QA][Code Coverage] Drop catchError and use try / catch instead, (elastic#69198) [QA] [Code Coverage] Integrate with Team Assignment Pipeline and Add Research and Development Indexes and Cluster (elastic#69348) [Metrics UI] Add context.reason and alertOnNoData to Inventory alerts (elastic#70260) Resolver refactoring (elastic#70312) [Ingest Manager] Fix agent ack after input format change (elastic#70335) [eslint][ts] Enable prefer-ts-expect-error (elastic#70022) ...

kibanamachine · 2020-07-02T23:08:44Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 69198 or prevent reminders by adding the backport:skip label.

kibanamachine · 2020-07-03T23:18:43Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 69198 or prevent reminders by adding the backport:skip label.

kibanamachine · 2020-07-06T23:18:44Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 69198 or prevent reminders by adding the backport:skip label.

Drop catchError and use try / catch instead,

448bab6

to fail the build on test run and / or ingestion.

wayneseymour added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes test-coverage issues & PRs for improving code test coverage labels Jun 15, 2020

wayneseymour requested a review from dmlemeshko June 15, 2020 19:53

wayneseymour self-assigned this Jun 15, 2020

wayneseymour commented Jun 15, 2020

View reviewed changes

wayneseymour linked an issue Jun 15, 2020 that may be closed by this pull request

[QA] [Code Coverage] Fail build on code coverage ingestion error #69168

Closed

wayneseymour added 2 commits June 16, 2020 10:57

Merge branch 'master' of github.com:elastic/kibana into fail-build-on…

cd19c61

…-coverage-ingestion-error

Add fake breaking change per Dima.

a679685

wayneseymour marked this pull request as ready for review June 16, 2020 21:00

Merge branch 'master' of github.com:elastic/kibana into fail-build-on…

4e82409

…-coverage-ingestion-error

wayneseymour changed the title ~~Drop catchError and use try / catch instead,~~ [QA][Code Coverage] Drop catchError and use try / catch instead, Jun 17, 2020

Merge branch 'master' into fail-build-on-coverage-ingestion-error

3de6b3a

wayneseymour requested a review from brianseeders June 29, 2020 18:47

wayneseymour added 3 commits June 29, 2020 13:01

Merge branch 'fail-build-on-coverage-ingestion-error' of github.com:e…

452918d

…lastic/kibana into fail-build-on-coverage-ingestion-error

Merge branch 'master' of github.com:elastic/kibana into fail-build-on…

ee446ed

…-coverage-ingestion-error

Drop fake error

8d70c7d

dmlemeshko approved these changes Jun 29, 2020

View reviewed changes

Drop using try / catch directly and instead,

3c9c99f

update handleFail fn to not slack notifiy if the build is unstable, per Brian S.

wayneseymour requested a review from LeeDr June 30, 2020 23:07

wayneseymour merged commit f428b2d into master Jun 30, 2020

wayneseymour deleted the fail-build-on-coverage-ingestion-error branch June 30, 2020 23:07

Bamieh pushed a commit to Bamieh/kibana that referenced this pull request Jul 1, 2020

[QA][Code Coverage] Drop catchError and use try / catch instead, (ela…

3110868

…stic#69198) Co-authored-by: Elastic Machine <[email protected]>

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jul 2, 2020

LeeDr added the backport:skip This commit does not require backporting label Jul 7, 2020

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jul 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QA][Code Coverage] Drop catchError and use try / catch instead, #69198

[QA][Code Coverage] Drop catchError and use try / catch instead, #69198

wayneseymour commented Jun 15, 2020 •

edited

Loading

wayneseymour left a comment

dmlemeshko commented Jun 15, 2020 •

edited

Loading

dmlemeshko commented Jun 15, 2020 •

edited

Loading

wayneseymour commented Jun 16, 2020

wayneseymour commented Jun 16, 2020

wayneseymour commented Jun 16, 2020

wayneseymour commented Jun 16, 2020 •

edited

Loading

wayneseymour commented Jun 17, 2020

wayneseymour commented Jun 29, 2020

wayneseymour commented Jun 29, 2020

wayneseymour commented Jun 29, 2020

dmlemeshko left a comment

wayneseymour commented Jun 29, 2020

brianseeders commented Jun 29, 2020

wayneseymour commented Jun 29, 2020

brianseeders commented Jun 30, 2020

wayneseymour commented Jun 30, 2020

kibanamachine commented Jun 30, 2020

kibanamachine commented Jul 2, 2020

kibanamachine commented Jul 3, 2020

kibanamachine commented Jul 6, 2020

[QA][Code Coverage] Drop catchError and use try / catch instead, #69198

[QA][Code Coverage] Drop catchError and use try / catch instead, #69198

Conversation

wayneseymour commented Jun 15, 2020 • edited Loading

Summary

wayneseymour left a comment

Choose a reason for hiding this comment

dmlemeshko commented Jun 15, 2020 • edited Loading

dmlemeshko commented Jun 15, 2020 • edited Loading

wayneseymour commented Jun 16, 2020

wayneseymour commented Jun 16, 2020

wayneseymour commented Jun 16, 2020

wayneseymour commented Jun 16, 2020 • edited Loading

wayneseymour commented Jun 17, 2020

wayneseymour commented Jun 29, 2020

wayneseymour commented Jun 29, 2020

wayneseymour commented Jun 29, 2020

dmlemeshko left a comment

Choose a reason for hiding this comment

wayneseymour commented Jun 29, 2020

brianseeders commented Jun 29, 2020

wayneseymour commented Jun 29, 2020

brianseeders commented Jun 30, 2020

wayneseymour commented Jun 30, 2020

kibanamachine commented Jun 30, 2020

💚 Build Succeeded

Build metrics

History

kibanamachine commented Jul 2, 2020

kibanamachine commented Jul 3, 2020

kibanamachine commented Jul 6, 2020

wayneseymour commented Jun 15, 2020 •

edited

Loading

dmlemeshko commented Jun 15, 2020 •

edited

Loading

dmlemeshko commented Jun 15, 2020 •

edited

Loading

wayneseymour commented Jun 16, 2020 •

edited

Loading