Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce statestore internal testing support #19177

Merged
merged 3 commits into from
Jun 16, 2020

Conversation

urso
Copy link

@urso urso commented Jun 12, 2020

  • Enhancement

What does this PR do?

This change introduces an internal package to the statestore that
provides testing helpers.
The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
here.

Once finalized, the libbeat/statestore package contains:

  • The statestore frontend and interface for use within Beats
  • Interfaces for the store backend
  • A common set of tests store backends need to support
  • a storetest package for testing new features that require a store. The
    testing helpers use map[string]interface{} that can be initialized or
    queried after the test run for validation purposes.
  • The default memlog backend + tests

This change include common testing support, that will be used by the
memlog backend, and to verify the storetest package.

Why is it important?

The statestore introduces a simple key-value store to Beats. The statestore will be used to replace the registry in filebeat in the future.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
    - [ ] I have added tests that prove my fix is effective or that my feature works
    - [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@urso urso added review Filebeat Filebeat libbeat needs_backport PR is waiting to be backported to other branches. Project:Filebeat-Input-v2 Team:Services (Deprecated) Label for the former Integrations-Services team v7.9.0 labels Jun 12, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 12, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 12, 2020

💔 Build Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #19177 updated]

  • Start Time: 2020-06-16T14:00:07.687+0000

  • Duration: 84 min 19 sec

Test stats 🧪

Test Results
Failed 0
Passed 9131
Skipped 1570
Total 10701

Steps errors

Expand to view the steps failures

  • Name: Make -C filebeat testsuite

    • Description: make -C filebeat testsuite

    • Duration: 7 min 57 sec

    • Start Time: 2020-06-16T14:22:43.470+0000

    • log

  • Name: Make -C generator/_templates/metricbeat test

    • Description: make -C generator/_templates/metricbeat test

    • Duration: 7 min 17 sec

    • Start Time: 2020-06-16T15:01:55.088+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-06-16T15:23:36.204Z]  OS/Arch:           darwin/amd64
[2020-06-16T15:23:36.204Z]  Experimental:      false
[2020-06-16T15:23:36.204Z] Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[2020-06-16T15:23:36.204Z] It requires Docker daemon to be installed and running
[2020-06-16T15:23:58.890Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats
[2020-06-16T15:23:59.207Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-06-16T15:23:59.220Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Lint
[2020-06-16T15:23:59.307Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Winlogbeat-oss
[2020-06-16T15:23:59.387Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-06-16T15:23:59.472Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-06-16T15:23:59.552Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Dockerlogbeat
[2020-06-16T15:23:59.630Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Journalbeat-oss
[2020-06-16T15:23:59.709Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-06-16T15:23:59.797Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-06-16T15:23:59.879Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-06-16T15:23:59.960Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-06-16T15:24:00.039Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-06-16T15:24:00.127Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-06-16T15:24:00.205Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-06-16T15:24:00.288Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Filebeat-oss
[2020-06-16T15:24:00.363Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-06-16T15:24:00.442Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-06-16T15:24:00.517Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Heartbeat-oss
[2020-06-16T15:24:00.595Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-06-16T15:24:00.671Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-06-16T15:24:00.752Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-06-16T15:24:00.827Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-06-16T15:24:00.905Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-06-16T15:24:00.980Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Libbeat-x-pack
[2020-06-16T15:24:01.056Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-06-16T15:24:01.132Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-06-16T15:24:01.206Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-06-16T15:24:01.284Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-06-16T15:24:01.360Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Filebeat-Windows
[2020-06-16T15:24:01.437Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Packetbeat-oss
[2020-06-16T15:24:01.513Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-06-16T15:24:01.590Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-06-16T15:24:01.665Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-06-16T15:24:01.746Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-06-16T15:24:01.821Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Filebeat-x-pack
[2020-06-16T15:24:01.897Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-Windows
[2020-06-16T15:24:01.971Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-06-16T15:24:02.052Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-06-16T15:24:02.133Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Heartbeat-Windows
[2020-06-16T15:24:02.209Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-06-16T15:24:02.294Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Libbeat-oss
[2020-06-16T15:24:02.369Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Functionbeat-Windows
[2020-06-16T15:24:02.446Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-06-16T15:24:02.523Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-06-16T15:24:02.600Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-06-16T15:24:02.675Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-06-16T15:24:02.749Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-06-16T15:24:03.114Z] + cat
[2020-06-16T15:24:03.115Z] + /usr/local/bin/runbld ./runbld-script
[2020-06-16T15:24:03.115Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-06-16T15:24:09.714Z] runbld>>> runbld started
[2020-06-16T15:24:09.714Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-06-16T15:24:11.632Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-19177' in order of occurrence in the config (last value wins).
[2020-06-16T15:24:13.019Z] runbld>>> Debug logging enabled.
[2020-06-16T15:24:13.019Z] runbld>>> Storing result
[2020-06-16T15:24:13.019Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-06-16T15:24:13.019Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200616152412-EEF0711B
[2020-06-16T15:24:13.019Z] runbld>>> Adding system facts.
[2020-06-16T15:24:13.962Z] runbld>>> Adding vcs info for the latest commit:  53f2d1e32facad36d9d6e39647de6a85c211cfb7
[2020-06-16T15:24:13.962Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-06-16T15:24:14.223Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-06-16T15:24:14.223Z] + echo 'Processing JUnit reports with runbld...'
[2020-06-16T15:24:14.223Z] Processing JUnit reports with runbld...
[2020-06-16T15:24:14.485Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-06-16T15:24:14.485Z] runbld>>> DURATION: 16ms
[2020-06-16T15:24:14.485Z] runbld>>> STDOUT: 40 bytes
[2020-06-16T15:24:14.485Z] runbld>>> STDERR: 49 bytes
[2020-06-16T15:24:14.485Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-06-16T15:24:14.485Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats
[2020-06-16T15:24:15.871Z] runbld>>> Storing build metadata: 
[2020-06-16T15:24:15.871Z] runbld>>> Adding test report.
[2020-06-16T15:24:15.871Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats
[2020-06-16T15:24:17.259Z] runbld>>> Found 114 test output files
[2020-06-16T15:24:17.842Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-06-16T15:24:17.842Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-06-16T15:24:17.842Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-06-16T15:24:17.842Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-cloudfoundry.xml
[2020-06-16T15:24:17.842Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-06-16T15:24:17.842Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-06-16T15:24:18.787Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-06-16T15:24:18.787Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-06-16T15:24:19.049Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 10551 Skipped: 1331
[2020-06-16T15:24:19.310Z] runbld>>> Storing result
[2020-06-16T15:24:19.310Z] runbld>>> FAILURES: 0
[2020-06-16T15:24:19.570Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-06-16T15:24:19.570Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200616152412-EEF0711B
[2020-06-16T15:24:19.570Z] runbld>>> Email notification disabled by environment variable.
[2020-06-16T15:24:19.570Z] runbld>>> Slack notification disabled by environment variable.
[2020-06-16T15:24:25.547Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19177
[2020-06-16T15:24:25.766Z] [INFO] getVaultSecret: Getting secrets
[2020-06-16T15:24:25.814Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-06-16T15:24:26.624Z] + chmod 755 generate-build-data.sh
[2020-06-16T15:24:26.624Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19177/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19177/runs/6 FAILURE 5058674
[2020-06-16T15:24:26.624Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19177/runs/6/steps/?limit=10000 -o steps-info.json
[2020-06-16T15:24:27.967Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19177/runs/6/tests/?status=FAILED -o tests-errors.json

urso added 2 commits June 15, 2020 15:47
This change introduces an internal package to the statestore that
provides testing helpers.
The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
[here](https://github.com/urso/beats/tree/fb-input-v2-combined/libbeat/statestore).

Once finalized, the libbeat/statestore package contains:
- The statestore frontend and interface for use within Beats
- Interfaces for the store backend
- A common set of tests store backends need to support
- a storetest package for testing new features that require a store. The
  testing helpers use map[string]interface{} that can be initialized or
  queried after the test run for validation purposes.
- The default memlog backend + tests

This change include common testing support, that will be used by the
memlog backend, and to verify the storetest package.
@urso urso force-pushed the statestore-testing branch from bb56448 to 8c567c1 Compare June 15, 2020 13:58
Copy link
Contributor

@sayden sayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and I just left a couple of comments. If you change them, feel free to merge afterwards


// Store helper for writing tests.
// The store needs a reference to the Registry with the current test context.
// The Store provides additional helpers for reopning the store, MustX methods
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Typo in reopning

}

// MustHas fails the test if an error occured in a call to Has.
func (s *Store) MustHas(key string) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I needed to do a search 😅 but it seems that the correct form is MustHave https://www.quora.com/Which-is-grammatically-correct-it-must-has-or-it-must-have

@urso
Copy link
Author

urso commented Jun 16, 2020

CI errors occured when building kafka containers. Relevant tests have passed.

@urso urso merged commit 8c23383 into elastic:master Jun 16, 2020
@urso urso deleted the statestore-testing branch June 16, 2020 20:54
urso pushed a commit to urso/beats that referenced this pull request Jul 6, 2020
This change introduces an internal package to the statestore that
provides testing helpers.
The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
[here](https://github.com/urso/beats/tree/fb-input-v2-combined/libbeat/statestore).

Once finalized, the libbeat/statestore package contains:
- The statestore frontend and interface for use within Beats
- Interfaces for the store backend
- A common set of tests store backends need to support
- a storetest package for testing new features that require a store. The
  testing helpers use map[string]interface{} that can be initialized or
  queried after the test run for validation purposes.
- The default memlog backend + tests

This change include common testing support, that will be used by the
memlog backend, and to verify the storetest package.

(cherry picked from commit 8c23383)
@urso urso removed the needs_backport PR is waiting to be backported to other branches. label Jul 6, 2020
urso pushed a commit that referenced this pull request Jul 7, 2020
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
This change introduces an internal package to the statestore that
provides testing helpers.
The addition of the statestore package is split up into multiple
changeset to ease review. The final version of the package can be found
[here](https://github.com/urso/beats/tree/fb-input-v2-combined/libbeat/statestore).

Once finalized, the libbeat/statestore package contains:
- The statestore frontend and interface for use within Beats
- Interfaces for the store backend
- A common set of tests store backends need to support
- a storetest package for testing new features that require a store. The
  testing helpers use map[string]interface{} that can be initialized or
  queried after the test run for validation purposes.
- The default memlog backend + tests

This change include common testing support, that will be used by the
memlog backend, and to verify the storetest package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Filebeat Filebeat libbeat Project:Filebeat-Input-v2 review skip-test-plan Team:Integrations Label for the Integrations team Team:Services (Deprecated) Label for the former Integrations-Services team v7.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants