Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JMX Scraper: Refactoring integration tests assertion framework #1573

Open
robsunday opened this issue Nov 26, 2024 · 2 comments
Open

JMX Scraper: Refactoring integration tests assertion framework #1573

robsunday opened this issue Nov 26, 2024 · 2 comments

Comments

@robsunday
Copy link
Contributor

Component(s)

jmx-scraper

Is your feature request related to a problem? Please describe.

Current implementation of integration tests assertion framework (MetricAssertions class) reports assertion failures in a very verbose and non-optimal way.
In case of assertion failure it prints out all received metrics and assertions multiple times. It makes finding the issue really hard. The more metrics is checked the harder it is and often the only way to find out what went wrong is selectively commenting out the code.
Also some assertions are missing details forcing developer to guess what went wrong.

Describe the solution you'd like

New assertion framework should be created fulfilling the following requirements:

  1. Print only the metric that failed the assertion
  2. Provide meaningful description and the metric property details needed to identify root cause of assertion failure
  3. Fail when any of expected metrics was not received
  4. Fail when unexpected metric was received (there is no assertion defined for it). There should be mechanism to opt out form this default behavior. It may be useful when it is very hard/impossible to force tested target to report some metric.
  5. API must be easy to use, concise and reliable (it should be hard to miss some metric checks)

New framework will coexist for some time with the current one, so currently implemented integration tests may be ported incrementally in separate PRs.
Once all tests are ported the old framework will be removed from the code

Describe alternatives you've considered

No response

Additional context

No response

@robsunday robsunday self-assigned this Nov 26, 2024
@robsunday robsunday changed the title Refactoring integration tests assertion framework JMX Scraper: Refactoring integration tests assertion framework Nov 26, 2024
@SylvainJuge
Copy link
Contributor

#1566 relates to this.

The migration work will likely be spread over many PRs.

@SylvainJuge
Copy link
Contributor

SylvainJuge commented Dec 9, 2024

Remaining steps to close this:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants