Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tests][filebeat] fix flaky unit test cases #40389

Merged
merged 3 commits into from
Jul 31, 2024
Merged

[tests][filebeat] fix flaky unit test cases #40389

merged 3 commits into from
Jul 31, 2024

Conversation

VihasMakwana
Copy link
Contributor

@VihasMakwana VihasMakwana commented Jul 30, 2024 •
edited
Loading

  • Bug fix

Proposed commit message

  • Current behaviour:
    • In unit test case, we use /logs/* pattern.
    • This is problematic, because we can also read the ndjson file (which we shouldn't, as it's the output file) and this would cause test case failure.
  • This PR fixes the behaviour.
    • Use logs/*.log as the pattern
    • Use Runner: 'xyz' has stopped to determine the stoppage instead of Stopping runner. The later would fail if there are transient errors.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@VihasMakwana VihasMakwana added flaky-test Unstable or unreliable test cases. Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team bugfix Team:Ingest-EngProd labels Jul 30, 2024
@VihasMakwana VihasMakwana self-assigned this Jul 30, 2024
@VihasMakwana VihasMakwana requested a review from a team as a code owner July 30, 2024 12:21
@VihasMakwana VihasMakwana requested a review from AndersonQ July 30, 2024 12:21
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@VihasMakwana VihasMakwana requested a review from rdner July 30, 2024 12:21
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 30, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 30, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @VihasMakwana? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@VihasMakwana VihasMakwana requested a review from a team July 30, 2024 12:23
@pierrehilbert pierrehilbert added the backport-8.15 Automated backport to the 8.15 branch with mergify label Jul 30, 2024
@dliappis dliappis added macOS Enable builds in the CI for darwin testing :Windows labels Jul 30, 2024
@dliappis
Copy link
Contributor

/test

@dliappis
Copy link
Contributor

@VihasMakwana I added the macOS (and :Windows) labels to trigger the extended tests (see

beats/.buildkite/filebeat/filebeat-pipeline.yml

Line 161 in 49f3efe

if: build.env("BUILDKITE_PULL_REQUEST") == "false" || build.env("GITHUB_PR_LABELS") =~ /.*(macOS|arm).*/
) via #40389 (comment).
If CI is still green, suggest we run the tests using the trigger phrase a couple of more times -- since it's a flaky test -- to ensure we are good.

@VihasMakwana
Copy link
Contributor Author

/test

@VihasMakwana
Copy link
Contributor Author

/test

@pierrehilbert pierrehilbert linked an issue Jul 31, 2024 that may be closed by this pull request
filebeat/tests/system: Test.test_start_stop is failing on Windows #39756
Closed
@VihasMakwana
Copy link
Contributor Author

/test

@VihasMakwana
Copy link
Contributor Author

@dliappis looks like it's working.

I've ran CI for the fourth time. Let's see how it goes.

@dliappis
Copy link
Contributor

@dliappis looks like it's working.

I've ran CI for the fourth time. Let's see how it goes.

Thank you so much @VihasMakwana for testing this thoroughly, much appreciated! FYI @oakrizan

dliappis
dliappis approved these changes Jul 31, 2024
View reviewed changes
Copy link
Contributor

@dliappis dliappis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on #40389 (comment)
LGTM from an ingest-eng-prod PoV. Thank you!

rdner
rdner approved these changes Jul 31, 2024
View reviewed changes
Copy link
Member

@rdner rdner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix makes sense to me, I see it was well-tested too.

@VihasMakwana VihasMakwana mentioned this pull request Jul 31, 2024
Closed
6 tasks
@VihasMakwana VihasMakwana merged commit 4fbf1b1 into elastic:main Jul 31, 2024
36 checks passed
mergify bot pushed a commit that referenced this pull request Jul 31, 2024
@VihasMakwana @mergify
[tests][filebeat] fix flaky unit test cases (#40389)
c538c48 
* fix: fix flaky unit test cases

* fix: fix test_reload_same_config

(cherry picked from commit 4fbf1b1)
@mergify mergify bot mentioned this pull request Jul 31, 2024
Merged
6 tasks
@VihasMakwana VihasMakwana mentioned this pull request Jul 31, 2024
Closed
@VihasMakwana VihasMakwana added the backport-v8.14.0 Automated backport with mergify label Jul 31, 2024
mergify bot pushed a commit that referenced this pull request Jul 31, 2024
@VihasMakwana @mergify
[tests][filebeat] fix flaky unit test cases (#40389)
f0bf0a8 
* fix: fix flaky unit test cases

* fix: fix test_reload_same_config

(cherry picked from commit 4fbf1b1)
@mergify mergify bot mentioned this pull request Jul 31, 2024
Merged
6 tasks
pierrehilbert pushed a commit that referenced this pull request Aug 1, 2024
@mergify @VihasMakwana
[tests][filebeat] fix flaky unit test cases (#40389) (#40401)
5bdb7eb 
* fix: fix flaky unit test cases

* fix: fix test_reload_same_config

(cherry picked from commit 4fbf1b1)

Co-authored-by: VihasMakwana <[email protected]>
pierrehilbert pushed a commit that referenced this pull request Aug 1, 2024
@mergify @VihasMakwana
[tests][filebeat] fix flaky unit test cases (#40389) (#40406)
c181443 
* fix: fix flaky unit test cases

* fix: fix test_reload_same_config

(cherry picked from commit 4fbf1b1)

Co-authored-by: VihasMakwana <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dliappis dliappis dliappis approved these changes

@rdner rdner rdner approved these changes

@AndersonQ AndersonQ Awaiting requested review from AndersonQ AndersonQ is a code owner automatically assigned from elastic/elastic-agent-data-plane

Assignees

@VihasMakwana VihasMakwana

Labels
backport-8.15 Automated backport to the 8.15 branch with mergify backport-v8.14.0 Automated backport with mergify bugfix flaky-test Unstable or unreliable test cases. macOS Enable builds in the CI for darwin testing Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team Team:Ingest-EngProd :Windows
Projects
None yet
Milestone
No milestone
5 participants
@VihasMakwana @elasticmachine @dliappis @rdner @pierrehilbert