Skip to content
This repository has been archived by the owner on Sep 17, 2024. It is now read-only.

chore(ci): convert regular pipeline into MBP #258

Merged
merged 4 commits into from
Sep 4, 2020

Conversation

mdelapenya
Copy link
Contributor

@mdelapenya mdelapenya commented Sep 3, 2020

What does this PR do?

This PR migrates the regular pipelines for the scheduled jobs for each test suite, to multibranch pipelines, filtering the affected branches by name: only master branch and those branches following a 2-digit-plus-x semver name (i.e. 7.9.x, 7.10.x...)

The affected pipelines are:

  • Helm
  • Integrations
  • Fleet

Why is it important?

We are currently running these scheduled pipelines once a day for the master branch, but after appearing maintenance branches, we want to schedule the execution for them too.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have run the Unit tests for the CLI, and they are passing locally
  • I have run the End-2-End tests for the suite I'm working on, and they are passing locally
  • I have noticed new Go dependencies (run make notice in the proper directory)

Author's Checklist

Related issues

Use cases

Scenario Outline: Run scheduled builds for desired branches
  Given the "<branch>" branch
  When a daily schedule is defined
  Then a scheduled job for the "<branch>" branch will be triggered
Examples:
| branch |
| master |
| 7.9.x  |
| 7.10.x |

Follow-ups

We must contact Infra to remove the old jobs for clean-up

@mdelapenya mdelapenya self-assigned this Sep 3, 2020
@mdelapenya mdelapenya requested a review from a team September 3, 2020 07:43
scm:
- github:
branch-discovery: no-pr
head-filter-regex: '(master|\d+\.\d+\.x)'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Master or maintenance branch following 2-digit-plus-x semver name

@elasticmachine
Copy link
Contributor

elasticmachine commented Sep 3, 2020

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #258 updated]

  • Start Time: 2020-09-04T07:47:05.063+0000

  • Duration: 18 min 5 sec

Test stats 🧪

Test Results
Failed 15
Passed 48
Skipped 13
Total 76

Test errors

Expand to view the tests failures

  • Name: Initializing / End-To-End Tests / ingest-manager_stand_alone_mode / Deploying a stand-alone agent – Stand-alone Agent

    • Age: 1
    • Duration: 177.32826
    • Error Details: Step there is new data in the index from agent: Not enough hits in the index yet. Current: 1, Desired: 50
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Deploying the centos agent – Fleet Mode Agent

    • Age: 1
    • Duration: 13.674083
    • Error Details: Step the agent is listed in Fleet as online
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Deploying the debian agent – Fleet Mode Agent

    • Age: 1
    • Duration: 12.839294
    • Error Details: Step the agent is listed in Fleet as online
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Starting the centos agent starts backend processes – Fleet Mode Agent

    • Age: 1
    • Duration: 7.7565565
    • Error Details: Step the "filebeat" process is in the "started" state on the host
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Starting the debian agent starts backend processes – Fleet Mode Agent

    • Age: 1
    • Duration: 7.808148
    • Error Details: Step the "filebeat" process is in the "started" state on the host
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Stopping the centos agent stops backend processes – Fleet Mode Agent

    • Age: 1
    • Duration: 7.9571853
    • Error Details: Step the "elastic-agent" process is "stopped" on the host
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Stopping the debian agent stops backend processes – Fleet Mode Agent

    • Age: 1
    • Duration: 8.750028
    • Error Details: Step the "elastic-agent" process is "stopped" on the host
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Restarting the centos host with persistent agent restarts backend processes – Fleet Mode Agent

    • Age: 1
    • Duration: 9.031495
    • Error Details: Step the host is restarted
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Restarting the debian host with persistent agent restarts backend processes – Fleet Mode Agent

    • Age: 1
    • Duration: 9.996369
    • Error Details: Step the host is restarted
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Un-enrolling the centos agent – Fleet Mode Agent

    • Age: 1
    • Duration: 9.063052
    • Error Details: Step the agent is un-enrolled
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Un-enrolling the debian agent – Fleet Mode Agent

    • Age: 1
    • Duration: 9.958403
    • Error Details: Step the agent is un-enrolled
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Re-enrolling the centos agent – Fleet Mode Agent

    • Age: 1
    • Duration: 8.949715
    • Error Details: Step the agent is un-enrolled
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Re-enrolling the debian agent – Fleet Mode Agent

    • Age: 1
    • Duration: 10.187378
    • Error Details: Step the agent is un-enrolled
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Revoking the enrollment token for the centos agent – Fleet Mode Agent

    • Age: 1
    • Duration: 9.030776
    • Error Details: Step the enrollment token is revoked
  • Name: Initializing / End-To-End Tests / ingest-manager_fleet_mode / Revoking the enrollment token for the debian agent – Fleet Mode Agent

    • Age: 1
    • Duration: 10.062329
    • Error Details: Step the enrollment token is revoked

Steps errors

Expand to view the steps failures

  • Name: Run functional tests for ingest-manager:stand_alone_mode

    • Description:

    • Duration: 8 min 3 sec

    • Start Time: 2020-09-04T07:52:20.286+0000

    • log

  • Name: Error signal

    • Description:

    • Duration: 0 min 0 sec

    • Start Time: 2020-09-04T08:00:23.675+0000

    • log

  • Name: General Build Step

    • Description: [2020-09-04T08:00:24.227Z] Archiving artifacts
      hudson.AbortException: script returned exit code 1

    • Duration: 0 min 0 sec

    • Start Time: 2020-09-04T08:00:24.221+0000

    • log

  • Name: Run functional tests for ingest-manager:fleet_mode

    • Description:

    • Duration: 8 min 9 sec

    • Start Time: 2020-09-04T07:52:20.131+0000

    • log

  • Name: Error signal

    • Description:

    • Duration: 0 min 0 sec

    • Start Time: 2020-09-04T08:00:29.477+0000

    • log

  • Name: General Build Step

    • Description: [2020-09-04T08:00:29.956Z] Archiving artifacts
      hudson.AbortException: script returned exit code 1

    • Duration: 0 min 0 sec

    • Start Time: 2020-09-04T08:00:29.951+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-09-04T08:04:10.343Z] metricbeat_1     | 2020-09-04T08:04:09.468Z	DEBUG	[modules]	beater/metricbeat.go:151	Available modules and metricsets: Register [ModuleFactory:[aws, azure, beat, cloudfoundry, docker, elasticsearch, kibana, linux, logstash, mongodb, mssql, mysql, oracle, postgresql, system, uwsgi], MetricSetFactory:[aerospike/namespace, apache/status, appsearch/stats, aws/cloudwatch, aws/ec2, aws/rds, aws/s3_daily_storage, aws/s3_request, aws/sqs, azure/compute_vm, azure/compute_vm_scaleset, azure/monitor, azure/storage, beat/state, beat/stats, ceph/cluster_disk, ceph/cluster_health, ceph/cluster_status, ceph/mgr_cluster_disk, ceph/mgr_cluster_health, ceph/mgr_osd_perf, ceph/mgr_osd_pool_stats, ceph/mgr_osd_tree, ceph/mgr_pool_disk, ceph/monitor_health, ceph/osd_df, ceph/osd_tree, ceph/pool_disk, cloudfoundry/container, cloudfoundry/counter, cloudfoundry/value, consul/agent, coredns/stats, couchbase/bucket, couchbase/cluster, couchbase/node, couchdb/server, docker/container, docker/cpu, docker/diskio, docker/event, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, dropwizard/collector, elasticsearch/ccr, elasticsearch/cluster_stats, elasticsearch/enrich, elasticsearch/index, elasticsearch/index_recovery, elasticsearch/index_summary, elasticsearch/ml_job, elasticsearch/node, elasticsearch/node_stats, elasticsearch/pending_tasks, elasticsearch/shard, envoyproxy/server, etcd/leader, etcd/metrics, etcd/self, etcd/store, golang/expvar, golang/heap, googlecloud/stackdriver, graphite/server, haproxy/info, haproxy/stat, http/json, http/server, istio/citadel, istio/galley, istio/mesh, istio/mixer, istio/pilot, jolokia/jmx, kafka/consumergroup, kafka/partition, kibana/stats, kibana/status, kubernetes/apiserver, kubernetes/container, kubernetes/controllermanager, kubernetes/event, kubernetes/node, kubernetes/pod, kubernetes/proxy, kubernetes/scheduler, kubernetes/state_container, kubernetes/state_cronjob, kubernetes/state_deployment, kubernetes/state_node, kubernetes/state_persistentvolume, kubernetes/state_persistentvolumeclaim, kubernetes/state_pod, kubernetes/state_replicaset, kubernetes/state_resourcequota, kubernetes/state_service, kubernetes/state_statefulset, kubernetes/state_storageclass, kubernetes/system, kubernetes/volume, kvm/dommemstat, kvm/status, linux/conntrack, linux/ksm, linux/pageinfo, logstash/node, logstash/node_stats, memcached/stats, mongodb/collstats, mongodb/dbstats, mongodb/metrics, mongodb/replstatus, mongodb/status, mssql/performance, mssql/transaction_log, munin/node, mysql/galera_status, mysql/status, nats/connections, nats/routes, nats/stats, nats/subscriptions, nginx/stubstatus, openmetrics/collector, oracle/performance, oracle/tablespace, php_fpm/pool, php_fpm/process, postgresql/activity, postgresql/bgwriter, postgresql/database, postgresql/statement, prometheus/collector, prometheus/query, prometheus/remote_write, rabbitmq/connection, rabbitmq/exchange, rabbitmq/node, rabbitmq/queue, redis/info, redis/key, redis/keyspace, sql/query, stan/channels, stan/stats, stan/subscriptions, statsd/server, system/core, system/cpu, system/diskio, system/entropy, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/network_summary, system/process, system/process_summary, system/raid, system/service, system/socket, system/socket_summary, system/uptime, system/users, traefik/health, uwsgi/status, vsphere/datastore, vsphere/host, vsphere/virtualmachine, zookeeper/connection, zookeeper/mntr, zookeeper/server], LightModules:[LightModules:[iis/webserver, iis/website, aws/ebs, aws/vpn, aws/elb, aws/usage, aws/billing, aws/sns, aws/lambda, aws/dynamodb, aws/transitgateway, aws/natgateway, cockroachdb/status, activemq/queue, activemq/topic, activemq/broker, tomcat/threading, tomcat/memory, tomcat/requests, tomcat/cache, googlecloud/pubsub, googlecloud/loadbalancing, googlecloud/storage, googlecloud/compute, kafka/broker, kafka/producer, kafka/consumer, azure/database_account, azure/container_registry, azure/container_instance, azure/container_service, ibmmq/qmgr, redisenterprise/node, redisenterprise/proxy]]]
[2020-09-04T08:04:10.343Z] metricbeat_1     | 2020-09-04T08:04:09.469Z	INFO	[monitoring]	log/log.go:118	Starting metrics logging every 30s
[2020-09-04T08:04:10.343Z] metricbeat_1     | 2020-09-04T08:04:09.469Z	INFO	instance/beat.go:463	metricbeat start running.
[2020-09-04T08:04:10.343Z] metricbeat_1     | 2020-09-04T08:04:09.470Z	DEBUG	[module]	module/wrapper.go:127	Starting Wrapper[name=mysql, len(metricSetWrappers)=1]
[2020-09-04T08:04:10.343Z] metricbeat_1     | 2020-09-04T08:04:09.470Z	DEBUG	[module]	module/wrapper.go:181	mysql/status will start after 8.240636851s
[2020-09-04T08:04:10.343Z] time="2020-09-04T08:04:10Z" level=debug msg="Docker compose executed." cmd="[logs metricbeat]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/metricbeat/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.8.0 mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.8.0]" profile=metricbeat
[2020-09-04T08:04:10.344Z] time="2020-09-04T08:04:10Z" level=debug msg="Updating state" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:04:10.344Z] time="2020-09-04T08:04:10Z" level=debug msg="State updated" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:04:10.344Z] time="2020-09-04T08:04:10Z" level=debug msg="Waiting 20 seconds" seconds=20
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:30Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:30Z" level=debug msg="Response information" hits=2 status="200 OK" took=2
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:30Z" level=warning msg="Waiting for more hits in the index" currentHits=2 desiredHits=5 elapsedTime=9.557512ms index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=1
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:30Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:30Z" level=debug msg="Response information" hits=2 status="200 OK" took=2
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:30Z" level=warning msg="Waiting for more hits in the index" currentHits=2 desiredHits=5 elapsedTime=606.74308ms index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=2
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:31Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:31Z" level=debug msg="Response information" hits=2 status="200 OK" took=2
[2020-09-04T08:04:32.333Z] time="2020-09-04T08:04:31Z" level=warning msg="Waiting for more hits in the index" currentHits=2 desiredHits=5 elapsedTime=1.35566588s index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=3
[2020-09-04T08:04:33.280Z] time="2020-09-04T08:04:32Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:33.280Z] time="2020-09-04T08:04:32Z" level=debug msg="Response information" hits=2 status="200 OK" took=2
[2020-09-04T08:04:33.280Z] time="2020-09-04T08:04:32Z" level=warning msg="Waiting for more hits in the index" currentHits=2 desiredHits=5 elapsedTime=2.985317512s index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=4
[2020-09-04T08:04:39.879Z] time="2020-09-04T08:04:38Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:39.879Z] time="2020-09-04T08:04:38Z" level=debug msg="Response information" hits=2 status="200 OK" took=2
[2020-09-04T08:04:39.879Z] time="2020-09-04T08:04:38Z" level=warning msg="Waiting for more hits in the index" currentHits=2 desiredHits=5 elapsedTime=8.723862267s index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=5
[2020-09-04T08:04:45.179Z] time="2020-09-04T08:04:44Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:45.179Z] time="2020-09-04T08:04:44Z" level=debug msg="Response information" hits=3 status="200 OK" took=3
[2020-09-04T08:04:45.179Z] time="2020-09-04T08:04:44Z" level=warning msg="Waiting for more hits in the index" currentHits=3 desiredHits=5 elapsedTime=14.940703114s index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=6
[2020-09-04T08:04:51.771Z] time="2020-09-04T08:04:51Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:51.771Z] time="2020-09-04T08:04:51Z" level=debug msg="Response information" hits=4 status="200 OK" took=2
[2020-09-04T08:04:51.771Z] time="2020-09-04T08:04:51Z" level=warning msg="Waiting for more hits in the index" currentHits=4 desiredHits=5 elapsedTime=21.453671394s index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=7
[2020-09-04T08:04:58.363Z] time="2020-09-04T08:04:57Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:04:58.363Z] time="2020-09-04T08:04:57Z" level=debug msg="Response information" hits=4 status="200 OK" took=2
[2020-09-04T08:04:58.363Z] time="2020-09-04T08:04:57Z" level=warning msg="Waiting for more hits in the index" currentHits=4 desiredHits=5 elapsedTime=27.611685489s index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj retry=8
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Response information" hits=5 status="200 OK" took=3
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=info msg="Hits number satisfied" currentHits=5 desiredHits=5 elapsedTime=31.034840133s retries=9
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Elasticsearch query" index=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj query="map[query:map[bool:map[must:[map[match:map[event.module:mysql]]]]]]"
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Response information" hits=5 status="200 OK" took=2
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=info msg="Hits number satisfied" currentHits=5 desiredHits=5 elapsedTime=7.373611ms retries=1
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="After scenario..."
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Removing services to compose" profile=metricbeat services="[metricbeat mysql]"
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml type=profiles
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/metricbeat/docker-compose.yml type=services
[2020-09-04T08:05:01.672Z] time="2020-09-04T08:05:01Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/mysql/docker-compose.yml type=services
[2020-09-04T08:05:01.672Z] Stopping metricbeat_metricbeat_1 ... 
[2020-09-04T08:05:02.260Z] 
Stopping metricbeat_metricbeat_1 ... done
Removing metricbeat_metricbeat_1 ... 
[2020-09-04T08:05:02.260Z] 
Removing metricbeat_metricbeat_1 ... done
Going to remove metricbeat_metricbeat_1
[2020-09-04T08:05:02.523Z] time="2020-09-04T08:05:02Z" level=debug msg="Docker compose executed." cmd="[rm -fvs metricbeat]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/mysql/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.8.0 mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.8.0]" profile=metricbeat
[2020-09-04T08:05:02.523Z] time="2020-09-04T08:05:02Z" level=debug msg="Updating state" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:05:02.523Z] time="2020-09-04T08:05:02Z" level=debug msg="State updated" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:05:02.523Z] time="2020-09-04T08:05:02Z" level=debug msg="Service removed from compose" profile=metricbeat service=metricbeat
[2020-09-04T08:05:02.523Z] time="2020-09-04T08:05:02Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml type=profiles
[2020-09-04T08:05:02.523Z] time="2020-09-04T08:05:02Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/metricbeat/docker-compose.yml type=services
[2020-09-04T08:05:02.523Z] time="2020-09-04T08:05:02Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/mysql/docker-compose.yml type=services
[2020-09-04T08:05:03.095Z] Stopping metricbeat_mysql_1 ... 
[2020-09-04T08:05:05.651Z] 
Stopping metricbeat_mysql_1 ... done
Removing metricbeat_mysql_1 ... 
[2020-09-04T08:05:05.651Z] 
Removing metricbeat_mysql_1 ... done
Going to remove metricbeat_mysql_1
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="Docker compose executed." cmd="[rm -fvs mysql]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/services/mysql/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.8.0 mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.8.0]" profile=metricbeat
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="Updating state" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="State updated" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="Service removed from compose" profile=metricbeat service=mysql
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="Index deleted using Elasticsearch Go client" indexName=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj status="400 Bad Request"
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="Index Alias deleted using Elasticsearch Go client" indexAlias=metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj status="400 Bad Request"
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml type=profiles
[2020-09-04T08:05:05.651Z] time="2020-09-04T08:05:05Z" level=debug msg="Compose file found at workdir" composeFilePath=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml type=profiles
[2020-09-04T08:05:06.225Z] Stopping metricbeat_elasticsearch_1 ... 
[2020-09-04T08:05:07.179Z] 
Stopping metricbeat_elasticsearch_1 ... done
Removing metricbeat_elasticsearch_1 ... 
[2020-09-04T08:05:07.180Z] 
Removing metricbeat_elasticsearch_1 ... done
Removing network metricbeat_default
[2020-09-04T08:05:07.442Z] time="2020-09-04T08:05:07Z" level=debug msg="Docker compose executed." cmd="[down]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.8.0-mysql-percona-8.0.13-4-pcjczagj logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.8.0 mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.8.0]" profile=metricbeat
[2020-09-04T08:05:07.443Z] time="2020-09-04T08:05:07Z" level=debug msg="Updating state" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:05:07.443Z] time="2020-09-04T08:05:07Z" level=debug msg="State updated" dir=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:05:07.443Z] time="2020-09-04T08:05:07Z" level=debug msg="Docker compose down." composeFilePath="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/compose/profiles/metricbeat/docker-compose.yml]" profile=metricbeat
[2020-09-04T08:05:07.443Z] time="2020-09-04T08:05:07Z" level=debug msg="State destroyed" stateFile=/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258/.op/metricbeat-profile.run
[2020-09-04T08:05:07.443Z] <?xml version="1.0" encoding="UTF-8"?>
[2020-09-04T08:05:07.443Z] <testsuites name="main" tests="7" skipped="0" failures="0" errors="0" time="644.933281084">
[2020-09-04T08:05:07.443Z]   <testsuite name="Apache" tests="0" skipped="0" failures="0" errors="0" time="0"></testsuite>
[2020-09-04T08:05:07.443Z]   <testsuite name="Metricbeat" tests="0" skipped="0" failures="0" errors="0" time="0"></testsuite>
[2020-09-04T08:05:07.443Z]   <testsuite name="Mysql" tests="7" skipped="0" failures="0" errors="0" time="596.110303653">
[2020-09-04T08:05:07.443Z]     <testcase name="Check MariaDB-10.2.23 is sending metrics to Elasticsearch without errors" status="passed" time="89.94948577"></testcase>
[2020-09-04T08:05:07.443Z]     <testcase name="Check MariaDB-10.3.14 is sending metrics to Elasticsearch without errors" status="passed" time="75.813046427"></testcase>
[2020-09-04T08:05:07.443Z]     <testcase name="Check MariaDB-10.4.4 is sending metrics to Elasticsearch without errors" status="passed" time="74.3493368"></testcase>
[2020-09-04T08:05:07.443Z]     <testcase name="Check MySQL-5.7.12 is sending metrics to Elasticsearch without errors" status="passed" time="81.992117362"></testcase>
[2020-09-04T08:05:07.443Z]     <testcase name="Check MySQL-8.0.13 is sending metrics to Elasticsearch without errors" status="passed" time="79.549875374"></testcase>
[2020-09-04T08:05:07.443Z]     <testcase name="Check Percona-5.7.24 is sending metrics to Elasticsearch without errors" status="passed" time="82.528187107"></testcase>
[2020-09-04T08:05:07.443Z]     <testcase name="Check Percona-8.0.13-4 is sending metrics to Elasticsearch without errors" status="passed" time="88.818655048"></testcase>
[2020-09-04T08:05:07.443Z]   </testsuite>
[2020-09-04T08:05:07.443Z]   <testsuite name="Redis" tests="0" skipped="0" failures="0" errors="0" time="0"></testsuite>
[2020-09-04T08:05:07.443Z]   <testsuite name="vSphere" tests="0" skipped="0" failures="0" errors="0" time="0"></testsuite>
[2020-09-04T08:05:07.443Z] </testsuites>+ sed -e 's/^[ \t]*//; s#>.*failed$#>#g' outputs/TEST-metricbeat-mysql
[2020-09-04T08:05:07.443Z] + grep -E '^<.*>$'
[2020-09-04T08:05:07.443Z] + exit 0
[2020-09-04T08:05:07.498Z] Recording test results
[2020-09-04T08:05:07.986Z] Archiving artifacts
[2020-09-04T08:05:09.361Z] Stage "Release" skipped due to earlier failure(s)
[2020-09-04T08:05:09.638Z] Running on Jenkins in /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-258@2
[2020-09-04T08:05:09.690Z] [INFO] getVaultSecret: Getting secrets
[2020-09-04T08:05:09.756Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-04T08:05:10.438Z] + chmod 755 generate-build-data.sh
[2020-09-04T08:05:10.438Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-258/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-258/runs/5 FAILURE 1085115
[2020-09-04T08:05:10.438Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-258/runs/5/steps/?limit=10000 -o steps-info.json

@mdelapenya
Copy link
Contributor Author

mdelapenya commented Sep 3, 2020

Mmm, JJBB pre-commit step failing: https://beats-ci.elastic.co/blue/organizations/jenkins/e2e-tests%2Fe2e-testing-mbp%2FPR-258/detail/PR-258/1/tests

Traceback (most recent call last):
  File "/usr/local/bin/jenkins-jobs", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/jenkins_jobs/cli/entry.py", line 165, in main
    jjb.execute()
  File "/usr/local/lib/python3.6/dist-packages/jenkins_jobs/cli/entry.py", line 146, in execute
    ext.obj.execute(self.options, self.jjb_config)
  File "/usr/local/lib/python3.6/dist-packages/jenkins_jobs/cli/subcommand/test.py", line 63, in execute
    options, jjb_config)
  File "/usr/local/lib/python3.6/dist-packages/jenkins_jobs/cli/subcommand/update.py", line 115, in _generate_xmljobs
    xml_jobs = xml_job_generator.generateXML(job_data_list)
  File "/usr/local/lib/python3.6/dist-packages/jenkins_jobs/xml_config.py", line 82, in generateXML
    xml_objs.append(self._getXMLForData(data))
  File "/usr/local/lib/python3.6/dist-packages/jenkins_jobs/xml_config.py", line 92, in _getXMLForData
    xml = mod.root_xml(data)
  File "/usr/local/lib/python3.6/dist-packages/jenkins_jobs/modules/project_multibranch.py", line 246, in root_xml
    for scm_data in data.get('scm', None):
TypeError: 'NoneType' object is not iterable

Running precommit locally, with same result, even though the main MBP is already in the repo:

pre-commit run --files .ci/jobs/e2e-testing-mbp.yml --show-diff-on-failure --verbose
pre-commit run --files .ci/jobs/e2e-testing-helm-daily-mbp.yml --show-diff-on-failure --verbose

.ci/e2eTestingHelmDaily.groovy Outdated Show resolved Hide resolved
.ci/jobs/e2e-testing-helm-daily-mbp.yml Outdated Show resolved Hide resolved
.ci/jobs/e2e-testing-helm-daily-mbp.yml Outdated Show resolved Hide resolved
We won't deal with PRs, enforced by JJBB definition
@mdelapenya
Copy link
Contributor Author

Tests are not related. See #261 (comment)

@mdelapenya mdelapenya marked this pull request as ready for review September 4, 2020 14:36
@mdelapenya mdelapenya merged commit 8d4eca5 into elastic:master Sep 4, 2020
@mdelapenya mdelapenya deleted the maintenance-branches-mbp branch September 4, 2020 15:28
mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request Sep 4, 2020
* chore(ci): convert regular pipeline into MBP

* fix: proper YAML indentation

* chore: disable PR notifications

* chore: simplify triggered job name

We won't deal with PRs, enforced by JJBB definition
mdelapenya added a commit that referenced this pull request Sep 4, 2020
… definition of the script path (#263) backport for 7.9.x (#264)

* chore(ci): convert regular pipeline into MBP (#258)

* chore(ci): convert regular pipeline into MBP

* fix: proper YAML indentation

* chore: disable PR notifications

* chore: simplify triggered job name

We won't deal with PRs, enforced by JJBB definition

* fix: use proper definition of the script path (#263)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run 7.9.x tests for ingest-manager daily
4 participants