Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce filterRegex parameter for performance report publisher #4555

Merged
merged 12 commits into from
Oct 11, 2023

Conversation

Thilaknath
Copy link
Contributor

@Thilaknath Thilaknath commented Sep 7, 2023

Changes

Step testsPublishResults use the performance plugin from jenkins to publish jmeter reports. It is possible to set thresholds per test and fail pipelines if they exceed. This feature was broken as the parameter filterRegex was not defined in piper-defaults and was always evaluating to null.

Error observed in pipelines before the change

Creating parser with percentiles:'0,50,90,100,' filterRegex:null
[2023-09-06T19:32:19.582Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-GetOrder-customFields.jtl' with filterRegex 'null'.
[2023-09-06T19:32:19.584Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-GetOrder-expanded.jtl' with filterRegex 'null'.
[2023-09-06T19:32:19.585Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-GetOrderItems-customFields.jtl' with filterRegex 'null'.
[2023-09-06T19:32:19.586Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-GetOrderItems-expanded.jtl' with filterRegex 'null'.
[2023-09-06T19:32:19.587Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-GetOrders-pgSize40-expanded.jtl' with filterRegex 'null'.
[2023-09-06T19:32:19.589Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-PostOrder-5Physical.jtl' with filterRegex 'null'.
[2023-09-06T19:32:19.590Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-PostOrder-5Subscription.jtl' with filterRegex 'null'.
[2023-09-06T19:32:19.591Z] Performance: Parsing report file '/var/jenkins_home/jobs/thilak-test-pipeline/builds/91/performance-reports/JMeter/v2-PostOrder-customFields.jtl' with filterRegex 'null'.

With these changes, the plugin is able to set the threshold and parse the file. Working output below

Creating parser with percentiles:'0,50,90,100,' filterRegex: 
[2023-09-06T21:16:30.330Z] Performance: Parsing report file '/var/jenkins_home/jobs/dom-ordercapture/branches/testJmeterResults/builds/11/performance-reports/JMeter/v2-PostOrder-5Physical.jtl' with filterRegex ''.
[2023-09-06T21:16:30.331Z] Performance: Parsing report file '/var/jenkins_home/jobs/dom-ordercapture/branches/testJmeterResults/builds/11/performance-reports/JMeter/v2-PostOrder-5Subscription.jtl' with filterRegex ''.
[2023-09-06T21:16:30.332Z] Performance: Parsing report file '/var/jenkins_home/jobs/dom-ordercapture/branches/testJmeterResults/builds/11/performance-reports/JMeter/v2-PostOrder-customFields.jtl' with filterRegex ''.
[2023-09-06T21:16:30.332Z] Performance: Percentage of errors greater or equal than 10% sets the build as unstable
[2023-09-06T21:16:30.332Z] Performance: Percentage of errors greater or equal than 20% sets the build as failure
[2023-09-06T21:16:30.332Z] Setting threshold: v2-PostOrder-5Physical.jtl:200
[2023-09-06T21:16:30.339Z] UNSTABLE: v2-PostOrder-5Physical.jtl has exceeded the threshold of [200] with the time of [319]
[2023-09-06T21:16:30.339Z] Performance: File v2-PostOrder-5Physical.jtl reported 0.0% of errors [UNSTABLE]. Build status is: UNSTABLE

This step and the feature from the performance plugin is actively used by our teams to notify on performance degradation for our services.

  • Tests
  • Documentation

Add filterRegex parameter to jmeter configuration. Set it to an empty string as generated from pipelinesyntax reference in jenkins
Add the filterRegex param to the publishJmeterReport plugin in testPublishResults
@Thilaknath Thilaknath requested a review from a team as a code owner September 7, 2023 12:57
@Thilaknath
Copy link
Contributor Author

/it-go

@Thilaknath
Copy link
Contributor Author

@CCFenner @anilkeshav27 Can someone take a look at this ?

@Thilaknath
Copy link
Contributor Author

@ashlymat Can someone from the piper community please help merge this change?

@Thilaknath
Copy link
Contributor Author

@sumeetpatil @OliverNocon Can someone from the code owners please look at this.

Copy link
Member

@anilkeshav27 anilkeshav27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a quick question would we need to add these

@@ -481,6 +481,7 @@ steps:
active: false
jmeter:
pattern: '**/*.jtl'
filterRegex: ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a quick question this filterRegex would also needed to be added to the piper defaults when using the sap specefic defaults ?. that comes from a different repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anilkeshav27 It probably should be. But why is a step loading values from two different configs ? Aren't certain steps meant to be coming from the piper-lib-os and not the internal one ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@anilkeshav27 anilkeshav27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@anilkeshav27
Copy link
Member

/it-go

@anilkeshav27 anilkeshav27 merged commit e22671a into SAP:master Oct 11, 2023
daskuznetsova pushed a commit to daskuznetsova/jenkins-library that referenced this pull request Oct 13, 2023
…#4555)

* Update default_pipeline_environment.yml

Add filterRegex parameter to jmeter configuration. Set it to an empty string as generated from pipelinesyntax reference in jenkins

* Update testsPublishResults.groovy

Add the filterRegex param to the publishJmeterReport plugin in testPublishResults

* Update documentation to include filterRegex

---------

Co-authored-by: Anil Keshav <[email protected]>
andrew-kireev pushed a commit that referenced this pull request Oct 17, 2023
* Update default_pipeline_environment.yml

Add filterRegex parameter to jmeter configuration. Set it to an empty string as generated from pipelinesyntax reference in jenkins

* Update testsPublishResults.groovy

Add the filterRegex param to the publishJmeterReport plugin in testPublishResults

* Update documentation to include filterRegex

---------

Co-authored-by: Anil Keshav <[email protected]>
maxatsap pushed a commit to maxatsap/jenkins-library that referenced this pull request Jul 23, 2024
…#4555)

* Update default_pipeline_environment.yml

Add filterRegex parameter to jmeter configuration. Set it to an empty string as generated from pipelinesyntax reference in jenkins

* Update testsPublishResults.groovy

Add the filterRegex param to the publishJmeterReport plugin in testPublishResults

* Update documentation to include filterRegex

---------

Co-authored-by: Anil Keshav <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants