[Task Manager] Changed alerts schedule logic to use Task Manager internals #80149

gmmorris · 2020-10-12T12:26:15Z

Summary

Current PR should resolve #46001 and include thew following changes:

Adds support for hour and day intervals to Task Manager to align with Alerting.
Adds support for rescheduling a Task through its TaskRun result. This would allow tasks to return a recurring schedule instead of just a runAt when they return a value. This addresses the update/OCC issues we encountered around this in the past.
Ensures that an update to the Alert schedule during a task run do trickle down into the underlying task after the alert executor has completed.
Adds the ability to throwUnrecoverableError which allows a Task runner to indicate to TM that it would like to fail the task and avoid rescheduling the task in the future.

Addresses the issue identified in Tested the #53650 issue:

Alerts execution that times out will try again 10 minutes later
The 10 minutes comes from the default 5 minute timeout + 5 minute backoff attempt multiple. This doesn't work well for alerts that run every 10 seconds or 1 minute. The expectation would be to simply run at the next interval. (Reason why Convert alerts to use task manager intervals #46001 is higher in priority but needs discussion).

and as a result timeout between executions is 5 minutes + alerts schedule interval (2 sec)

server    log   [20:03:45.649] [warning][plugins][taskManager] Cancelling task alerting:.index-threshold "dfd4eba0-1a49-11eb-ae78-a97473db5ac0" as it expired at 2020-10-30T03:03:42.749Z after running for 05m 02s (with timeout set at 5m).
server    log   [20:08:46.049] [warning][plugins][taskManager] Cancelling task alerting:.index-threshold "dfd4eba0-1a49-11eb-ae78-a97473db5ac0" as it expired at 2020-10-30T03:08:45.649Z after running for 05m 00s (with timeout set at 5m).

Note for docs

Document task manager API changes. Explain how returning a schedule from the runner will update the task's schedule.

Checklist

Delete any items that are not applicable to this PR.

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

…-tm-intervals # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

…-tm-intervals # Conflicts: # x-pack/plugins/task_manager/server/task_runner.ts

…ed right after task was complete previous execution

…-tm-intervals # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

…kibana into alerts/convert-to-tm-intervals * 'alerts/convert-to-tm-intervals' of github.com:gmmorris/kibana: (88 commits) fixed jest APM Experiments settings (elastic#81554) [Resolver] Enable resolver test plugin tests (elastic#81339) Add TS project references for inspector (elastic#81792) Add uri decode to es_ui_shared and fix navigation issues with special characters (elastic#80835) [Fleet] Rename ingestManager translations fleet (elastic#81837) [Logs UI] Transmit and render array field values in log entries (elastic#81385) Audit Logging: use the original url (elastic#81282) [User experience] Fix JS error rate (elastic#81512) [UX] Add median/percentile info in titles (elastic#79824) Support export for SO with circular refs (elastic#81582) Get rid of global types (elastic#81739) [APM] Fix precommit script (elastic#81594) skips overview tests (elastic#81877) [Security Solution][Case] Fix connector's labeling (elastic#81824) Added simple test, which only covers successful case when edit happened right after task was complete previous execution [Maps] Fix EMS test (elastic#81856) [Security Solutions][Detections] - Fix bug, last response not showing for disabled rules (elastic#81783) skip flaky suite (elastic#81853) Fixed type checks and unit tests ...

elasticmachine · 2020-10-30T04:50:09Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

* master: (71 commits) [Chrome] Extension to append an element to the last breadcrumb (elastic#82015) [Monitoring] Thread pool rejections alert (elastic#79433) [Actions] Fix actionType type on registerType function (elastic#82125) [Security Solution] Modal for saving timeline (elastic#81802) add tests for index pattern switching (elastic#81987) TS project references for share plugin (elastic#82051) [Graph] Fix problem with duplicate ids (elastic#82109) skip 'returns a single bucket if array has 1'. related elastic#81460 Add a link to documentation in the alerts and actions management UI (elastic#81909) [Fleet] fix duplicate ingest pipeline refs (elastic#82078) Context menu trigger for URL Drilldown (elastic#81158) SO management: fix legacy import index pattern selection being reset when switching page (elastic#81621) Fixed dead links (elastic#78696) [Search] Add "restore" to session service (elastic#81924) fix Lens heading structure (elastic#81752) [ML] Data Frame Analytics: Fix feature importance cell value and decision path chart (elastic#82011) Remove legacy app arch items from codeowners. (elastic#82084) [TSVB] Renamed 'positive rate' to 'counter rate' (elastic#80939) Expressions/migrations2 (elastic#81281) [Telemetry] [Schema] remove number type and support all es number types (elastic#81774) ...

ymao1

LGTM!

mikecote

Looks great! I might have found an issue that would come back with this PR (see comments).

I want to do some testing on Monday around error handling (similar to what I did here: #53650). I will finalize my review based on the results.

x-pack/plugins/alerts/server/task_runner/task_runner.ts

mikecote · 2020-10-30T19:53:11Z

x-pack/plugins/alerts/server/task_runner/task_runner.ts

-      runAt: resolveErr<Date | undefined, Error>(runAt, (err) => {
-        return isAlertSavedObjectNotFoundError(err, alertId)
+      schedule: resolveErr<IntervalSchedule | undefined, Error>(schedule, (error) => {
+        return isAlertSavedObjectNotFoundError(error, alertId)
          ? undefined


I did some local testing, it seems to break the following fix we did a while ago #63093.

Previously returning undefined as the next runAt would cause task manager to delete the task. When returning undefined to the schedule, I believe task manager no longer deletes these tasks and re-uses previous schedule.

Ah, good catch... that's because we relied on an implicit behaviour there... because the task was not a scheduled task.
Considering we decided that scheduled tasks should keep running forever (from tTM's perspective) even if they fail and get deleted... so, arguably this is still correct.

What is missing, I guess, is a clean up process in the Alerting framework itself. I'll look into it.

Addressed here: 16729cd

(cc @YulNaumenko )

…alerting where needed

* master: Add derivative function (elastic#81178) [Discover] Deangularize context_app.html, part 3 (elastic#81838) [Visualize] Vis listing page breaks on unknown vis type (elastic#82018) Rename `batchSize` parameter to `batch_size` to be consisten with the API namings guidelines. (elastic#82123) Minor edits in Single Metric Viewer (elastic#82159) [Actions] Fix type contract (elastic#82168) Upgrade EUI to v30.1.1 (elastic#81499) Skip failing ES snapshot test (elastic#82207) Skip ES snapshot failing suite (elastic#82206) [Alerting UI] Grouped list of alert types using producers in Types filter of Alerts tab (elastic#81876) [Maps] convert vector style component to typescript round 1 (elastic#81961) Fix link to upgrade assistant (elastic#82138) Rename "service overview" to "service inventory" (elastic#81933) adjust policy test to drop test for server addresses (elastic#82120) Cleanup/codeowners (elastic#82146) [DOCS] Updates add data content (elastic#81093) [DOCS] Remove index mgmt docs (elastic#82099) [Search] fix cancelation related memory leaks (elastic#81996)

mikecote

Changes LGTM!

x-pack/plugins/task_manager/README.md

…rnals (elastic#80149) * spiked intervals in alerts * ensure scheduled tasks dont get wiped * Fixed type checks and unit tests * Added simple test, which only covers successful case when edit happened right after task was complete previous execution * fixed jest * fallback to existing task schedule when possible * added missing test * Added support for day and hour schedule interval values * added docs for new schedule run result * fixed doc * added UnrecoverableError support for task runners nad pluged it into alerting where needed * typo Co-authored-by: Yuliia Naumenko <[email protected]>

…rnals (#80149) (#82330) * spiked intervals in alerts * ensure scheduled tasks dont get wiped * Fixed type checks and unit tests * Added simple test, which only covers successful case when edit happened right after task was complete previous execution * fixed jest * fallback to existing task schedule when possible * added missing test * Added support for day and hour schedule interval values * added docs for new schedule run result * fixed doc * added UnrecoverableError support for task runners nad pluged it into alerting where needed * typo Co-authored-by: Yuliia Naumenko <[email protected]> Co-authored-by: Gidi Meir Morris <[email protected]>

kibanamachine · 2020-12-16T20:32:59Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 583ff0b
Pipeline Steps (look for red circles / failed steps)
Interpreting CI Failures

Failed CI Steps

Test Failures

Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/discover/feature_controls/discover_security·ts.discover feature controls discover feature controls security "before all" hook in "discover feature controls security"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: discover
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: feature controls
[00:00:00]             └-> "before all" hook
[00:00:00]             └-: discover feature controls security
[00:00:00]               └-> "before all" hook
[00:00:00]               └-> "before all" hook
[00:00:00]                 │ info [discover/feature_controls/security] Loading "mappings.json"
[00:00:00]                 │ info [discover/feature_controls/security] Loading "data.json"
[00:00:00]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_1/kLyNGuJ9RlOxtkWFCKj6lg] deleting index
[00:00:00]                 │ info [discover/feature_controls/security] Deleted existing index [".kibana_1"]
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana] creating index, cause [api], templates [], shards [1]/[0]
[00:00:00]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana][0]]])." previous.health="YELLOW" reason="shards started [[.kibana][0]]"
[00:00:00]                 │ info [discover/feature_controls/security] Created index ".kibana"
[00:00:00]                 │ debg [discover/feature_controls/security] ".kibana" settings {"index":{"number_of_shards":"1","auto_expand_replicas":"0-1","number_of_replicas":"0"}}
[00:00:00]                 │ info [discover/feature_controls/security] Indexed 3 docs into ".kibana"
[00:00:00]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana/yHnD8MEASjywPg7e4YQ3GA] update_mapping [_doc]
[00:00:00]                 │ debg Migrating saved objects
[00:00:00]                 │ proc [kibana]   log   [20:32:37.697] [info][savedobjects-service] Creating index .kibana_2.
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1]
[00:00:00]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] updating number_of_replicas to [0] for indices [.kibana_2]
[00:00:00]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana_2][0]]])." previous.health="YELLOW" reason="shards started [[.kibana_2][0]]"
[00:00:00]                 │ proc [kibana]   log   [20:32:37.784] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[00:00:00]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] updating number_of_replicas to [0] for indices [.kibana_1]
[00:00:00]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana_1][0]]])." previous.health="YELLOW" reason="shards started [[.kibana_1][0]]"
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.tasks] creating index, cause [auto(task api)], templates [], shards [1]/[1]
[00:00:00]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] updating number_of_replicas to [0] for indices [.tasks]
[00:00:00]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.tasks][0]]])." previous.health="YELLOW" reason="shards started [[.tasks][0]]"
[00:00:00]                 │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] 1322 finished with response BulkByScrollResponse[took=30.8ms,timed_out=false,sliceId=null,updated=0,created=3,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:00:00]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana/yHnD8MEASjywPg7e4YQ3GA] deleting index
[00:00:00]                 │ proc [kibana]   log   [20:32:38.177] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:00:00]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/aB9yzfDqTTSFKaSu_k21Lw] update_mapping [_doc]
[00:00:01]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/aB9yzfDqTTSFKaSu_k21Lw] update_mapping [_doc]
[00:00:01]                 │ proc [kibana]   log   [20:32:38.359] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:00:01]                 │ proc [kibana]   log   [20:32:38.406] [info][savedobjects-service] Finished in 712ms.
[00:00:01]                 │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:00:02]                 │ info [logstash_functional] Loading "mappings.json"
[00:00:02]                 │ info [logstash_functional] Loading "data.json.gz"
[00:00:02]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] failed on parsing mappings on index creation [logstash-2015.09.22]
[00:00:02]                 │      org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:304) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:277) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:915) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:409) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:637) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:407) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:485) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:370) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:377) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:300) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:59) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:697) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:319) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:214) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:674) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
[00:00:02]                 │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
[00:00:02]                 │      	at java.lang.Thread.run(Thread.java:832) [?:?]
[00:00:02]                 │      Caused by: org.elasticsearch.index.mapper.MapperParsingException: No handler for type [runtime] declared on field [runtime_number]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:328) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:251) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:151) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:93) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:82) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:302) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]                 │      	... 22 more
[00:00:02]                 │ info Taking screenshot "/dev/shm/workspace/parallel/1/kibana/x-pack/test/functional/screenshots/failure/discover feature controls discover feature controls security _before all_ hook.png"
[00:00:02]                 │ info Current URL is: data:/,
[00:00:02]                 │ info Saving page source to: /dev/shm/workspace/parallel/1/kibana/x-pack/test/functional/failure_debug/html/discover feature controls discover feature controls security _before all_ hook.html
[00:00:02]                 └- ✖ fail: discover feature controls discover feature controls security "before all" hook in "discover feature controls security"
[00:00:02]                 │      Error: [mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]
[00:00:02]                 │       at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
[00:00:02]                 │       at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
[00:00:02]                 │       at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
[00:00:02]                 │       at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/lodash/lodash.js:4949:19)
[00:00:02]                 │       at endReadableNT (_stream_readable.js:1145:12)
[00:00:02]                 │       at process._tickCallback (internal/process/next_tick.js:63:19)
[00:00:02]                 │ 
[00:00:02]                 │

Stack Trace

{ Error: [mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]
    at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/lodash/lodash.js:4949:19)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 400,
  displayName: 'BadRequest',
  message:
   '[mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]',
  path: '/logstash-2015.09.22',
  query: {},
  body:
   { error:
      { root_cause: [Array],
        type: 'mapper_parsing_exception',
        reason:
         'Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]',
        caused_by: [Object] },
     status: 400 },
  statusCode: 400,
  response:
   '{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [runtime] declared on field [runtime_number]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [runtime] declared on field [runtime_number]"}},"status":400}',
  toString: [Function],
  toJSON: [Function] }

X-Pack Accessibility Tests.x-pack/test/accessibility/apps/dashboard_edit_panel·ts.Dashboard Edit Panel "before all" hook for " A11y test on dashboard edit panel menu options"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:03:25]         └-: Dashboard Edit Panel
[00:03:25]           └-> "before all" hook
[00:03:25]           └-> "before all" hook
[00:03:25]             │ info [dashboard/drilldowns] Loading "mappings.json"
[00:03:25]             │ info [dashboard/drilldowns] Loading "data.json"
[00:03:25]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_1/uwrKgj_1QbeWE2lPlJDPUA] deleting index
[00:03:25]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/GyRN1Xl8TUW2gNyTOZxFJQ] deleting index
[00:03:25]             │ info [dashboard/drilldowns] Deleted existing index [".kibana_2",".kibana_1"]
[00:03:25]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana] creating index, cause [api], templates [], shards [1]/[1]
[00:03:25]             │ info [dashboard/drilldowns] Created index ".kibana"
[00:03:25]             │ debg [dashboard/drilldowns] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:03:25]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana/O5g39yIdQ7qRM30d7-IxhQ] update_mapping [_doc]
[00:03:25]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana/O5g39yIdQ7qRM30d7-IxhQ] update_mapping [_doc]
[00:03:25]             │ info [dashboard/drilldowns] Indexed 12 docs into ".kibana"
[00:03:25]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana/O5g39yIdQ7qRM30d7-IxhQ] update_mapping [_doc]
[00:03:25]             │ debg Migrating saved objects
[00:03:25]             │ proc [kibana]   log   [20:34:29.657] [info][savedobjects-service] Creating index .kibana_2.
[00:03:25]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1]
[00:03:25]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] updating number_of_replicas to [0] for indices [.kibana_2]
[00:03:25]             │ proc [kibana]   log   [20:34:29.732] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:03:25]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[00:03:25]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] updating number_of_replicas to [0] for indices [.kibana_1]
[00:03:25]             │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] 6477 finished with response BulkByScrollResponse[took=42.1ms,timed_out=false,sliceId=null,updated=0,created=12,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:03:25]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana/O5g39yIdQ7qRM30d7-IxhQ] deleting index
[00:03:25]             │ proc [kibana]   log   [20:34:30.090] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:03:25]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/myllJZMVR2OP7P5arbT97g] update_mapping [_doc]
[00:03:25]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/myllJZMVR2OP7P5arbT97g] update_mapping [_doc]
[00:03:25]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/myllJZMVR2OP7P5arbT97g] update_mapping [_doc]
[00:03:26]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/myllJZMVR2OP7P5arbT97g] update_mapping [_doc]
[00:03:26]             │ proc [kibana]   log   [20:34:30.360] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:03:26]             │ proc [kibana]   log   [20:34:30.401] [info][savedobjects-service] Finished in 745ms.
[00:03:26]             │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:03:26]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] [.kibana_2/myllJZMVR2OP7P5arbT97g] update_mapping [_doc]
[00:03:27]             │ info [logstash_functional] Loading "mappings.json"
[00:03:27]             │ info [logstash_functional] Loading "data.json.gz"
[00:03:27]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] failed on parsing mappings on index creation [logstash-2015.09.22]
[00:03:27]             │      org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]
[00:03:27]             │      	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:304) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:277) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:915) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:409) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:637) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:407) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:485) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:370) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:377) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:300) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:59) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:697) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:319) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:214) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:674) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
[00:03:27]             │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
[00:03:27]             │      	at java.lang.Thread.run(Thread.java:832) [?:?]
[00:03:27]             │      Caused by: org.elasticsearch.index.mapper.MapperParsingException: No handler for type [runtime] declared on field [runtime_number]
[00:03:27]             │      	at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:328) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:251) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.index.mapper.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:151) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:93) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:82) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:302) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:03:27]             │      	... 22 more
[00:03:27]             │ info Taking screenshot "/dev/shm/workspace/parallel/13/kibana/x-pack/test/functional/screenshots/failure/Dashboard Edit Panel _before all_ hook.png"
[00:03:27]             │ info Current URL is: http://localhost:61131/app/management/kibana/settings
[00:03:27]             │ info Saving page source to: /dev/shm/workspace/parallel/13/kibana/x-pack/test/functional/failure_debug/html/Dashboard Edit Panel _before all_ hook.html
[00:03:27]             └- ✖ fail: Dashboard Edit Panel "before all" hook for " A11y test on dashboard edit panel menu options"
[00:03:27]             │      Error: [mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]
[00:03:27]             │       at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
[00:03:27]             │       at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
[00:03:27]             │       at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
[00:03:27]             │       at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/lodash/lodash.js:4949:19)
[00:03:27]             │       at endReadableNT (_stream_readable.js:1145:12)
[00:03:27]             │       at process._tickCallback (internal/process/next_tick.js:63:19)
[00:03:27]             │ 
[00:03:27]             │

Stack Trace

{ Error: [mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]
    at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/lodash/lodash.js:4949:19)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 400,
  displayName: 'BadRequest',
  message:
   '[mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]',
  path: '/logstash-2015.09.22',
  query: {},
  body:
   { error:
      { root_cause: [Array],
        type: 'mapper_parsing_exception',
        reason:
         'Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]',
        caused_by: [Object] },
     status: 400 },
  statusCode: 400,
  response:
   '{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [runtime] declared on field [runtime_number]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [runtime] declared on field [runtime_number]"}},"status":400}',
  toString: [Function],
  toJSON: [Function] }

Firefox XPack UI Functional Tests.x-pack/test/functional/apps/canvas.Canvas app "before all" hook in "Canvas app"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://dryrun

[00:00:00]       │
[00:00:00]         └-: Canvas app
[00:00:00]           └-> "before all" hook
[00:00:00]           └-> "before all" hook
[00:00:00]             │ debg set roles = test_logstash_reader,global_canvas_all
[00:00:00]             │ debg creating user test_user
[00:00:00]             │ info [o.e.x.s.a.u.TransportPutUserAction] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] updated user [test_user]
[00:00:00]             │ debg created user test_user
[00:00:00]             │ debg TestSubjects.exists(kibanaChrome)
[00:00:00]             │ debg Find.existsByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=2500
[00:00:02]             │ info [logstash_functional] Loading "mappings.json"
[00:00:02]             │ info [logstash_functional] Loading "data.json.gz"
[00:00:02]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1608148897856344978] failed on parsing mappings on index creation [logstash-2015.09.22]
[00:00:02]             │      org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]
[00:00:02]             │      	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:304) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:277) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:915) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:409) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:637) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:407) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:485) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:370) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:377) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:300) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:59) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:697) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:319) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:214) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:674) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
[00:00:02]             │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
[00:00:02]             │      	at java.lang.Thread.run(Thread.java:832) [?:?]
[00:00:02]             │      Caused by: org.elasticsearch.index.mapper.MapperParsingException: No handler for type [runtime] declared on field [runtime_number]
[00:00:02]             │      	at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:328) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:251) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.index.mapper.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:151) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:93) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:82) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:302) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:00:02]             │      	... 22 more
[00:00:02]             │ info Taking screenshot "/dev/shm/workspace/parallel/14/kibana/x-pack/test/functional/screenshots/failure/Canvas app _before all_ hook.png"
[00:00:02]             │ info Current URL is: about:blank
[00:00:02]             │ info Saving page source to: /dev/shm/workspace/parallel/14/kibana/x-pack/test/functional/failure_debug/html/Canvas app _before all_ hook.html
[00:00:02]             └- ✖ fail: Canvas app "before all" hook in "Canvas app"
[00:00:02]             │      Error: [mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]
[00:00:02]             │       at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
[00:00:02]             │       at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
[00:00:02]             │       at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
[00:00:02]             │       at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/lodash/lodash.js:4949:19)
[00:00:02]             │       at endReadableNT (_stream_readable.js:1145:12)
[00:00:02]             │       at process._tickCallback (internal/process/next_tick.js:63:19)
[00:00:02]             │ 
[00:00:02]             │

Stack Trace

{ Error: [mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]
    at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/lodash/lodash.js:4949:19)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 400,
  displayName: 'BadRequest',
  message:
   '[mapper_parsing_exception] No handler for type [runtime] declared on field [runtime_number]',
  path: '/logstash-2015.09.22',
  query: {},
  body:
   { error:
      { root_cause: [Array],
        type: 'mapper_parsing_exception',
        reason:
         'Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]',
        caused_by: [Object] },
     status: 400 },
  statusCode: 400,
  response:
   '{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [runtime] declared on field [runtime_number]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping: No handler for type [runtime] declared on field [runtime_number]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [runtime] declared on field [runtime_number]"}},"status":400}',
  toString: [Function],
  toJSON: [Function] }

and 9 more failures, only showing the first 3.

Metrics [docs]

Distributable file count

id	before	after	diff
`default`	48128	48129	+1

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/development-plugin-saved-objects.html#_mappings

id	before	after	diff
`_doc_count`	-	1	+1
`fleet-agents`	25	23	-2
`ingest-outputs`	11	10	-1
`ingest-package-policies`	35	30	-5
`lens`	7	6	-1
total			-8

History

💚 Build #85214 succeeded 583ff0b
💚 Build #85082 succeeded 9108d13
💚 Build #84875 succeeded 1929b5d
💚 Build #84761 succeeded ce07868

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

gmmorris · 2021-02-22T15:31:09Z

Decided to remove the needs_docs label on this as it is documented in the Developer Docs and as an end user can't use Task Manager directly, it doesn't make sense to add them to user docs.

spiked intervals in alerts

a68d300

gmmorris mentioned this pull request Oct 12, 2020

Convert alerts to use task manager intervals #46001

Closed

YulNaumenko and others added 11 commits October 23, 2020 11:40

ensure scheduled tasks dont get wiped

118780a

Merge remote-tracking branch 'upstream/master' into alerts/convert-to…

144caf5

…-tm-intervals # Conflicts: # x-pack/plugins/task_manager/server/task_runner.ts

Fixed type checks and unit tests

8ba34d7

Added simple test, which only covers successful case when edit happen…

653d831

…ed right after task was complete previous execution

fixed jest

533e522

fallback to existing task schedule when possible

cd4f668

added missing test

3c4dfe9

Added support for day and hour schedule interval values

ce07868

YulNaumenko marked this pull request as ready for review October 30, 2020 04:48

YulNaumenko requested a review from a team as a code owner October 30, 2020 04:48

YulNaumenko added Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels Oct 30, 2020

YulNaumenko added release_note:skip Skip the PR/issue when compiling release notes v7.11.0 v8.0.0 labels Oct 30, 2020

YulNaumenko assigned YulNaumenko and gmmorris Oct 30, 2020

YulNaumenko linked an issue Oct 30, 2020 that may be closed by this pull request

Convert alerts to use task manager intervals #46001

Closed

YulNaumenko changed the title ~~spiked intervals in alerts~~ [Task Manager] Changed alerts schedule logic to use Task Manager internal Oct 30, 2020

YulNaumenko changed the title ~~[Task Manager] Changed alerts schedule logic to use Task Manager internal~~ [Task Manager] Changed alerts schedule logic to use Task Manager internals Oct 30, 2020

gmmorris added 2 commits October 30, 2020 14:30

added docs for new schedule run result

ed55572

ymao1 approved these changes Oct 30, 2020

View reviewed changes

mikecote self-requested a review October 30, 2020 19:12

mikecote reviewed Oct 30, 2020

View reviewed changes

YulNaumenko and others added 3 commits October 30, 2020 17:24

fixed doc

6c07f2e

added UnrecoverableError support for task runners nad pluged it into …

16729cd

…alerting where needed

mikecote approved these changes Nov 2, 2020

View reviewed changes

dhurley14 reviewed Nov 2, 2020

View reviewed changes

x-pack/plugins/task_manager/README.md Outdated Show resolved Hide resolved

dhurley14 reviewed Nov 2, 2020

View reviewed changes

x-pack/plugins/task_manager/README.md Outdated Show resolved Hide resolved

typo

583ff0b

YulNaumenko merged commit 51acfb9 into elastic:master Nov 2, 2020

YulNaumenko mentioned this pull request Nov 2, 2020

[7.x] [Task Manager] Changed alerts schedule logic to use Task Manager internals (#80149) #82330

Merged

peluja1012 mentioned this pull request Nov 5, 2020

[Security Solution] Rule execution marked as failed when the rule is modified meanwhile is executed #82320

Closed

mikecote added the needs_docs label Dec 16, 2020

gmmorris removed the needs_docs label Feb 22, 2021

ymao1 mentioned this pull request Nov 3, 2021

[alerting] rule tasks that fail 3 times are never run again, with no indication in the rule #116321

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Task Manager] Changed alerts schedule logic to use Task Manager internals #80149

[Task Manager] Changed alerts schedule logic to use Task Manager internals #80149

gmmorris commented Oct 12, 2020 •

edited by mikecote

Loading

elasticmachine commented Oct 30, 2020

ymao1 left a comment

mikecote left a comment

mikecote Oct 30, 2020

gmmorris Nov 2, 2020

gmmorris Nov 2, 2020 •

edited

Loading

mikecote left a comment

kibanamachine commented Dec 16, 2020 •

edited

Loading

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

gmmorris commented Feb 22, 2021

[Task Manager] Changed alerts schedule logic to use Task Manager internals #80149

[Task Manager] Changed alerts schedule logic to use Task Manager internals #80149

Conversation

gmmorris commented Oct 12, 2020 • edited by mikecote Loading

Summary

Note for docs

Checklist

For maintainers

elasticmachine commented Oct 30, 2020

ymao1 left a comment

Choose a reason for hiding this comment

mikecote left a comment

Choose a reason for hiding this comment

mikecote Oct 30, 2020

Choose a reason for hiding this comment

gmmorris Nov 2, 2020

Choose a reason for hiding this comment

gmmorris Nov 2, 2020 • edited Loading

Choose a reason for hiding this comment

mikecote left a comment

Choose a reason for hiding this comment

kibanamachine commented Dec 16, 2020 • edited Loading

💔 Build Failed

Failed CI Steps

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Metrics [docs]

Distributable file count

Saved Objects .kibana field count

History

gmmorris commented Feb 22, 2021

gmmorris commented Oct 12, 2020 •

edited by mikecote

Loading

gmmorris Nov 2, 2020 •

edited

Loading

kibanamachine commented Dec 16, 2020 •

edited

Loading