Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Heartbeat] Add integration tests for browser monitors #31999

Closed
andrewvc opened this issue Jun 21, 2022 · 1 comment · Fixed by #32542
Closed

[Heartbeat] Add integration tests for browser monitors #31999

andrewvc opened this issue Jun 21, 2022 · 1 comment · Fixed by #32542
Assignees
@andrewvc
Labels
Heartbeat Team:obs-ds-hosted-services Label for the Observability Hosted Services team technical debt

Comments

@andrewvc
Copy link
Contributor

andrewvc commented Jun 21, 2022 •
edited
Loading

To prevent more bugs like #31997 we need better integration tests for browser and lightweight monitors. The python ones are hard to maintain.,

  • We can possible take inspiration from Port Python tests of log input to Golang with filestream input #24246
  • Should run synthetics agent, both latest and stable NPM labels
  • Tests should be written in go
  • We should investigate whether to index docs to ES as part of the process, if too cumbersome, we can use the current python approach of writing to files on disk
  • If we can re-use any go integ test tools written by the libbeat folks that would be worthwhile
  • Tests only run on linux / mac, no need for CI targets on windows etc.
@andrewvc andrewvc added Heartbeat Team:obs-ds-hosted-services Label for the Observability Hosted Services team technical debt labels Jun 21, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/uptime (Team:Uptime)

@andrewvc andrewvc self-assigned this Jul 25, 2022
@andrewvc andrewvc mentioned this issue Jul 29, 2022
Merged
6 tasks
mergify bot pushed a commit that referenced this issue Aug 8, 2022
@andrewvc @mergify
[Heartbeat] Add functional-ish tests in golang (#32542)
50eb2cd 
Fixes #31999

This PR turned out to be a bit of a monster. The main thing it does is add testing for functional 'scenarios', that is testing full heartbeat configs and the events they output going through all heartbeat specific code using the libbeat factory interface. This lets us do something sort of close to functional testing, where instead of using the heartbeat binary we actually use this factory interface. This gets us pretty close with a much simpler testing experience that's easier to debug and faster to run. It probably won't miss any error conditions from generating the full binary.

This also cleans up a bunch of linter issues in modified files, which is a bit distracting, things like missing error return values, shadowed variable names etc.

Finally, the new scenario test this adds, which checks that all events always contain a check group, revealed two bugs:

Synthetics test runs create an empty event at the start of each run (with no check group)
Any synth events prior to a journey start event would be missing a check group
This PR fixes this by:

Moving the check group generation code for synthexec up to the stream enricher, rather than the journey enricher.
Creating the check group UUIDs after journeys end, rather than waiting for them to start. The synthetics node agent can produce output before any journey starts

(cherry picked from commit 3fe6871)
andrewvc added a commit to andrewvc/beats that referenced this issue Aug 8, 2022
@andrewvc
[Heartbeat] Add functional-ish tests in golang (elastic#32542)
3fe6871 
Fixes elastic#31999

This PR turned out to be a bit of a monster. The main thing it does is add testing for functional 'scenarios', that is testing full heartbeat configs and the events they output going through all heartbeat specific code using the libbeat factory interface. This lets us do something sort of close to functional testing, where instead of using the heartbeat binary we actually use this factory interface. This gets us pretty close with a much simpler testing experience that's easier to debug and faster to run. It probably won't miss any error conditions from generating the full binary.

This also cleans up a bunch of linter issues in modified files, which is a bit distracting, things like missing error return values, shadowed variable names etc.

Finally, the new scenario test this adds, which checks that all events always contain a check group, revealed two bugs:

Synthetics test runs create an empty event at the start of each run (with no check group)
Any synth events prior to a journey start event would be missing a check group
This PR fixes this by:

Moving the check group generation code for synthexec up to the stream enricher, rather than the journey enricher.
Creating the check group UUIDs after journeys end, rather than waiting for them to start. The synthetics node agent can produce output before any journey starts
cmacknz pushed a commit that referenced this issue Aug 24, 2022
@mergify @andrewvc
[Heartbeat] Add functional-ish tests in golang (#32542) (#32630)
7f2073f 
Fixes #31999

This PR turned out to be a bit of a monster. The main thing it does is add testing for functional 'scenarios', that is testing full heartbeat configs and the events they output going through all heartbeat specific code using the libbeat factory interface. This lets us do something sort of close to functional testing, where instead of using the heartbeat binary we actually use this factory interface. This gets us pretty close with a much simpler testing experience that's easier to debug and faster to run. It probably won't miss any error conditions from generating the full binary.

This also cleans up a bunch of linter issues in modified files, which is a bit distracting, things like missing error return values, shadowed variable names etc.

Finally, the new scenario test this adds, which checks that all events always contain a check group, revealed two bugs:

Synthetics test runs create an empty event at the start of each run (with no check group)
Any synth events prior to a journey start event would be missing a check group
This PR fixes this by:

Moving the check group generation code for synthexec up to the stream enricher, rather than the journey enricher.
Creating the check group UUIDs after journeys end, rather than waiting for them to start. The synthetics node agent can produce output before any journey starts

(cherry picked from commit 3fe6871)

Co-authored-by: Andrew Cholakian <[email protected]>
chrisberkhout pushed a commit that referenced this issue Jun 1, 2023
@andrewvc @chrisberkhout
[Heartbeat] Add functional-ish tests in golang (#32542)
2445911 
Fixes #31999

This PR turned out to be a bit of a monster. The main thing it does is add testing for functional 'scenarios', that is testing full heartbeat configs and the events they output going through all heartbeat specific code using the libbeat factory interface. This lets us do something sort of close to functional testing, where instead of using the heartbeat binary we actually use this factory interface. This gets us pretty close with a much simpler testing experience that's easier to debug and faster to run. It probably won't miss any error conditions from generating the full binary.

This also cleans up a bunch of linter issues in modified files, which is a bit distracting, things like missing error return values, shadowed variable names etc.

Finally, the new scenario test this adds, which checks that all events always contain a check group, revealed two bugs:

Synthetics test runs create an empty event at the start of each run (with no check group)
Any synth events prior to a journey start event would be missing a check group
This PR fixes this by:

Moving the check group generation code for synthexec up to the stream enricher, rather than the journey enricher.
Creating the check group UUIDs after journeys end, rather than waiting for them to start. The synthetics node agent can produce output before any journey starts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andrewvc andrewvc

Labels
Heartbeat Team:obs-ds-hosted-services Label for the Observability Hosted Services team technical debt
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Heartbeat] Add functional-ish tests in golang / fix bugs andrewvc/beats
2 participants
@andrewvc @elasticmachine