Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve milestone automation #7553

Merged
merged 1 commit into from
Sep 11, 2024
Merged

Improve milestone automation #7553

merged 1 commit into from
Sep 11, 2024

Conversation

PerfectSlayer
Copy link
Contributor

@PerfectSlayer PerfectSlayer commented Sep 3, 2024 •
edited
Loading

What Does This Do

This PR improves milestone automation by:

  • Add permission description
  • Rewrite milestone closing / creation on tag creation using GitHub actions script and removing deprecated third party actions
  • Add workflow testing to locally work some jobs
  • Update README

Motivation

Remove deprecated third party actions
Add support for patch versions
Add missing permissions

Additional Notes

Related to #7531

Contributor Checklist

Jira ticket: VULN-8166

@PerfectSlayer PerfectSlayer added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Sep 3, 2024
@PerfectSlayer PerfectSlayer requested a review from a team as a code owner September 3, 2024 08:17
@pr-commenter
Copy link

pr-commenter bot commented Sep 3, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/gh-workflow
git_commit_date 1725979768 1725984516
git_commit_sha 04db2f0 724ce01
release_version 1.40.0-SNAPSHOT~04db2f0bfb 1.40.0-SNAPSHOT~724ce01c8c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1725987026 1725987026
ci_job_id 634142100 634142100
ci_pipeline_id 43998899 43998899
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064080
Total [baseline] (10.385 s) : 0, 10384989
Agent [candidate] (1.063 s) : 0, 1062601
Total [candidate] (10.347 s) : 0, 10346776
section appsec
Agent [baseline] (1.199 s) : 0, 1198994
Total [baseline] (10.614 s) : 0, 10613793
Agent [candidate] (1.198 s) : 0, 1198053
Total [candidate] (10.631 s) : 0, 10630767
section iast
Agent [baseline] (1.19 s) : 0, 1190261
Total [baseline] (10.828 s) : 0, 10827658
Agent [candidate] (1.189 s) : 0, 1188690
Total [candidate] (10.888 s) : 0, 10887784
section profiling
Agent [baseline] (1.268 s) : 0, 1268255
Total [baseline] (10.712 s) : 0, 10712410
Agent [candidate] (1.258 s) : 0, 1258489
Total [candidate] (10.563 s) : 0, 10563425
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.199 s 134.914 ms (12.7%)
Agent iast 1.19 s 126.181 ms (11.9%)
Agent profiling 1.268 s 204.175 ms (19.2%)
Total tracing 10.385 s -
Total appsec 10.614 s 228.805 ms (2.2%)
Total iast 10.828 s 442.669 ms (4.3%)
Total profiling 10.712 s 327.421 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.198 s 135.452 ms (12.7%)
Agent iast 1.189 s 126.089 ms (11.9%)
Agent profiling 1.258 s 195.889 ms (18.4%)
Total tracing 10.347 s -
Total appsec 10.631 s 283.991 ms (2.7%)
Total iast 10.888 s 541.008 ms (5.2%)
Total profiling 10.563 s 216.649 ms (2.1%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (679.042 ms) : 0, 679042
BytebuddyAgent [candidate] (678.137 ms) : 0, 678137
GlobalTracer [baseline] (311.61 ms) : 0, 311610
GlobalTracer [candidate] (310.525 ms) : 0, 310525
AppSec [baseline] (51.564 ms) : 0, 51564
AppSec [candidate] (51.966 ms) : 0, 51966
Remote Config [baseline] (694.164 µs) : 0, 694
Remote Config [candidate] (695.594 µs) : 0, 696
Telemetry [baseline] (7.547 ms) : 0, 7547
Telemetry [candidate] (7.578 ms) : 0, 7578
section appsec
BytebuddyAgent [baseline] (699.905 ms) : 0, 699905
BytebuddyAgent [candidate] (697.598 ms) : 0, 697598
GlobalTracer [baseline] (304.544 ms) : 0, 304544
GlobalTracer [candidate] (304.592 ms) : 0, 304592
AppSec [baseline] (160.748 ms) : 0, 160748
AppSec [candidate] (161.521 ms) : 0, 161521
Remote Config [baseline] (646.913 µs) : 0, 647
Remote Config [candidate] (633.739 µs) : 0, 634
Telemetry [baseline] (8.311 ms) : 0, 8311
Telemetry [candidate] (8.98 ms) : 0, 8980
IAST [baseline] (22.377 ms) : 0, 22377
IAST [candidate] (22.182 ms) : 0, 22182
section iast
BytebuddyAgent [baseline] (791.572 ms) : 0, 791572
BytebuddyAgent [candidate] (790.335 ms) : 0, 790335
GlobalTracer [baseline] (299.94 ms) : 0, 299940
GlobalTracer [candidate] (299.541 ms) : 0, 299541
AppSec [baseline] (55.009 ms) : 0, 55009
AppSec [candidate] (52.573 ms) : 0, 52573
Remote Config [baseline] (600.536 µs) : 0, 601
Remote Config [candidate] (607.991 µs) : 0, 608
Telemetry [baseline] (7.433 ms) : 0, 7433
Telemetry [candidate] (9.762 ms) : 0, 9762
IAST [baseline] (22.035 ms) : 0, 22035
IAST [candidate] (22.205 ms) : 0, 22205
section profiling
BytebuddyAgent [baseline] (676.943 ms) : 0, 676943
BytebuddyAgent [candidate] (671.115 ms) : 0, 671115
GlobalTracer [baseline] (395.46 ms) : 0, 395460
GlobalTracer [candidate] (393.02 ms) : 0, 393020
AppSec [baseline] (52.605 ms) : 0, 52605
AppSec [candidate] (52.196 ms) : 0, 52196
Remote Config [baseline] (712.621 µs) : 0, 713
Remote Config [candidate] (703.625 µs) : 0, 704
Telemetry [baseline] (7.43 ms) : 0, 7430
Telemetry [candidate] (7.395 ms) : 0, 7395
ProfilingAgent [baseline] (96.919 ms) : 0, 96919
ProfilingAgent [candidate] (96.076 ms) : 0, 96076
Profiling [baseline] (96.943 ms) : 0, 96943
Profiling [candidate] (96.1 ms) : 0, 96100
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064220
Total [baseline] (8.532 s) : 0, 8532344
Agent [candidate] (1.069 s) : 0, 1068799
Total [candidate] (8.538 s) : 0, 8537542
section iast
Agent [baseline] (1.188 s) : 0, 1188191
Total [baseline] (8.976 s) : 0, 8975813
Agent [candidate] (1.195 s) : 0, 1194699
Total [candidate] (9.042 s) : 0, 9042473
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.188 s) : 0, 1187641
Total [baseline] (8.967 s) : 0, 8967156
Agent [candidate] (1.189 s) : 0, 1188851
Total [candidate] (8.962 s) : 0, 8962274
section iast_TELEMETRY_OFF
Agent [baseline] (1.193 s) : 0, 1193448
Total [baseline] (8.984 s) : 0, 8983736
Agent [candidate] (1.183 s) : 0, 1183366
Total [candidate] (9.024 s) : 0, 9024140
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.188 s 123.971 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.188 s 123.421 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.193 s 129.228 ms (12.1%)
Total tracing 8.532 s -
Total iast 8.976 s 443.469 ms (5.2%)
Total iast_HARDCODED_SECRET_DISABLED 8.967 s 434.813 ms (5.1%)
Total iast_TELEMETRY_OFF 8.984 s 451.393 ms (5.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.069 s -
Agent iast 1.195 s 125.9 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.189 s 120.052 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.183 s 114.567 ms (10.7%)
Total tracing 8.538 s -
Total iast 9.042 s 504.932 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 8.962 s 424.733 ms (5.0%)
Total iast_TELEMETRY_OFF 9.024 s 486.598 ms (5.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (679.067 ms) : 0, 679067
BytebuddyAgent [candidate] (682.703 ms) : 0, 682703
GlobalTracer [baseline] (311.419 ms) : 0, 311419
GlobalTracer [candidate] (312.087 ms) : 0, 312087
AppSec [baseline] (51.857 ms) : 0, 51857
AppSec [candidate] (52.006 ms) : 0, 52006
Remote Config [baseline] (714.874 µs) : 0, 715
Remote Config [candidate] (692.386 µs) : 0, 692
Telemetry [baseline] (7.554 ms) : 0, 7554
Telemetry [candidate] (7.586 ms) : 0, 7586
section iast
BytebuddyAgent [baseline] (790.873 ms) : 0, 790873
BytebuddyAgent [candidate] (794.772 ms) : 0, 794772
GlobalTracer [baseline] (299.241 ms) : 0, 299241
GlobalTracer [candidate] (301.169 ms) : 0, 301169
AppSec [baseline] (52.092 ms) : 0, 52092
AppSec [candidate] (53.271 ms) : 0, 53271
IAST [baseline] (23.655 ms) : 0, 23655
IAST [candidate] (22.906 ms) : 0, 22906
Remote Config [baseline] (1.39 ms) : 0, 1390
Remote Config [candidate] (597.984 µs) : 0, 598
Telemetry [baseline] (7.312 ms) : 0, 7312
Telemetry [candidate] (8.206 ms) : 0, 8206
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (790.948 ms) : 0, 790948
BytebuddyAgent [candidate] (790.283 ms) : 0, 790283
GlobalTracer [baseline] (300.03 ms) : 0, 300030
GlobalTracer [candidate] (300.158 ms) : 0, 300158
AppSec [baseline] (53.136 ms) : 0, 53136
AppSec [candidate] (53.894 ms) : 0, 53894
IAST [baseline] (21.134 ms) : 0, 21134
IAST [candidate] (22.059 ms) : 0, 22059
Remote Config [baseline] (609.239 µs) : 0, 609
Remote Config [candidate] (610.437 µs) : 0, 610
Telemetry [baseline] (8.153 ms) : 0, 8153
Telemetry [candidate] (8.165 ms) : 0, 8165
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (793.296 ms) : 0, 793296
BytebuddyAgent [candidate] (786.387 ms) : 0, 786387
GlobalTracer [baseline] (301.751 ms) : 0, 301751
GlobalTracer [candidate] (299.511 ms) : 0, 299511
AppSec [baseline] (55.99 ms) : 0, 55990
AppSec [candidate] (53.732 ms) : 0, 53732
IAST [baseline] (20.736 ms) : 0, 20736
IAST [candidate] (21.54 ms) : 0, 21540
Remote Config [baseline] (592.558 µs) : 0, 593
Remote Config [candidate] (591.787 µs) : 0, 592
Telemetry [baseline] (7.346 ms) : 0, 7346
Telemetry [candidate] (7.918 ms) : 0, 7918
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-10T16:21:02 2024-09-10T16:27:51
git_branch master bbujon/gh-workflow
git_commit_date 1725979768 1725984516
git_commit_sha 04db2f0 724ce01
release_version 1.40.0-SNAPSHOT~04db2f0bfb 1.40.0-SNAPSHOT~724ce01c8c
start_time 2024-09-10T16:20:48 2024-09-10T16:27:38
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1725986017 1725986017
ci_job_id 634142103 634142103
ci_pipeline_id 43998899 43998899
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 19 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.346 ms) : 1327, 1366
.   : milestone, 1346,
appsec (1.707 ms) : 1683, 1730
.   : milestone, 1707,
appsec_no_iast (1.726 ms) : 1702, 1750
.   : milestone, 1726,
iast (1.477 ms) : 1454, 1499
.   : milestone, 1477,
profiling (1.476 ms) : 1452, 1499
.   : milestone, 1476,
tracing (1.471 ms) : 1447, 1495
.   : milestone, 1471,
section candidate
no_agent (1.33 ms) : 1311, 1350
.   : milestone, 1330,
appsec (1.729 ms) : 1705, 1753
.   : milestone, 1729,
appsec_no_iast (1.713 ms) : 1689, 1738
.   : milestone, 1713,
iast (1.462 ms) : 1439, 1485
.   : milestone, 1462,
profiling (1.52 ms) : 1495, 1546
.   : milestone, 1520,
tracing (1.459 ms) : 1434, 1484
.   : milestone, 1459,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.327 ms, 1.366 ms] -
appsec 1.707 ms [1.683 ms, 1.73 ms] 360.224 µs (26.8%)
appsec_no_iast 1.726 ms [1.702 ms, 1.75 ms] 379.675 µs (28.2%)
iast 1.477 ms [1.454 ms, 1.499 ms] 130.339 µs (9.7%)
profiling 1.476 ms [1.452 ms, 1.499 ms] 129.339 µs (9.6%)
tracing 1.471 ms [1.447 ms, 1.495 ms] 124.623 µs (9.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.33 ms [1.311 ms, 1.35 ms] -
appsec 1.729 ms [1.705 ms, 1.753 ms] 398.541 µs (30.0%)
appsec_no_iast 1.713 ms [1.689 ms, 1.738 ms] 382.907 µs (28.8%)
iast 1.462 ms [1.439 ms, 1.485 ms] 131.744 µs (9.9%)
profiling 1.52 ms [1.495 ms, 1.546 ms] 190.015 µs (14.3%)
tracing 1.459 ms [1.434 ms, 1.484 ms] 128.423 µs (9.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.801 µs) : 341, 380
.   : milestone, 361,
iast (477.787 µs) : 457, 499
.   : milestone, 478,
iast_FULL (555.376 µs) : 534, 576
.   : milestone, 555,
iast_GLOBAL (512.497 µs) : 491, 534
.   : milestone, 512,
iast_HARDCODED_SECRET_DISABLED (480.115 µs) : 459, 502
.   : milestone, 480,
iast_INACTIVE (445.988 µs) : 425, 467
.   : milestone, 446,
iast_TELEMETRY_OFF (472.062 µs) : 450, 495
.   : milestone, 472,
tracing (444.89 µs) : 424, 466
.   : milestone, 445,
section candidate
no_agent (363.672 µs) : 344, 383
.   : milestone, 364,
iast (478.806 µs) : 457, 501
.   : milestone, 479,
iast_FULL (553.956 µs) : 533, 575
.   : milestone, 554,
iast_GLOBAL (515.731 µs) : 493, 538
.   : milestone, 516,
iast_HARDCODED_SECRET_DISABLED (491.19 µs) : 468, 514
.   : milestone, 491,
iast_INACTIVE (440.292 µs) : 420, 461
.   : milestone, 440,
iast_TELEMETRY_OFF (472.177 µs) : 449, 495
.   : milestone, 472,
tracing (441.321 µs) : 421, 462
.   : milestone, 441,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.801 µs [341.319 µs, 380.282 µs] -
iast 477.787 µs [456.767 µs, 498.806 µs] 116.986 µs (32.4%)
iast_FULL 555.376 µs [534.312 µs, 576.44 µs] 194.576 µs (53.9%)
iast_GLOBAL 512.497 µs [490.553 µs, 534.441 µs] 151.696 µs (42.0%)
iast_HARDCODED_SECRET_DISABLED 480.115 µs [458.709 µs, 501.521 µs] 119.314 µs (33.1%)
iast_INACTIVE 445.988 µs [425.054 µs, 466.922 µs] 85.187 µs (23.6%)
iast_TELEMETRY_OFF 472.062 µs [449.622 µs, 494.502 µs] 111.261 µs (30.8%)
tracing 444.89 µs [423.535 µs, 466.245 µs] 84.089 µs (23.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.672 µs [343.956 µs, 383.388 µs] -
iast 478.806 µs [457.064 µs, 500.549 µs] 115.135 µs (31.7%)
iast_FULL 553.956 µs [532.64 µs, 575.273 µs] 190.285 µs (52.3%)
iast_GLOBAL 515.731 µs [493.361 µs, 538.101 µs] 152.06 µs (41.8%)
iast_HARDCODED_SECRET_DISABLED 491.19 µs [467.959 µs, 514.421 µs] 127.519 µs (35.1%)
iast_INACTIVE 440.292 µs [419.608 µs, 460.976 µs] 76.621 µs (21.1%)
iast_TELEMETRY_OFF 472.177 µs [449.49 µs, 494.864 µs] 108.505 µs (29.8%)
tracing 441.321 µs [420.939 µs, 461.703 µs] 77.649 µs (21.4%)

Dacapo

@PerfectSlayer PerfectSlayer enabled auto-merge (squash) September 4, 2024 12:45
bm1549
bm1549 reviewed Sep 9, 2024
View reviewed changes
Copy link
Contributor

@bm1549 bm1549 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly feedback on the docs and structure. The scripts and test files looks great!

.github/workflows/README.md Outdated
@@ -26,7 +26,7 @@ _Trigger:_ When closing a milestone.

_Action:_ Create a new milestone by incrementing minor version.

_Comment:_ Already done when closing a tag. To delete?
_Comment:_ Disaled as it is already done when closing a tag. It will be removed after some testing.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_Comment:_ Disaled as it is already done when closing a tag. It will be removed after some testing.
_Comment:_ Disabled as a comment is already added when closing a tag. This will be removed after some testing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t get the "comment" part? What about:

Disabled as also covered by increment-milestone-on-tag. This will be removed after some testing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We add a comment when closing the tag if iiuc

Your suggestion is much better 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s another automation. This one is supposed to create a new "milestone" when we close one. But we already create such milestone when creating the release tag 🤔

The comment thing is another automation that posts a comment on milestone issues to let reporters know the fix is released 😉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhhh, my bad. Thanks for improving the docs here!

.github/workflows/README.md Outdated Show resolved Hide resolved
.github/workflows/README.md Outdated Show resolved Hide resolved
.github/workflows/README.md Outdated Show resolved Hide resolved
.github/workflows/README.md Outdated Show resolved Hide resolved
.github/workflows/increment-milestone-on-tag.yaml Show resolved Hide resolved
@PerfectSlayer
feat: Improve milestone automation
724ce01 
Rewrite milestone update on tag
Add workflow testing
Update README
@PerfectSlayer PerfectSlayer merged commit efc6f52 into master Sep 11, 2024
98 checks passed
@PerfectSlayer PerfectSlayer deleted the bbujon/gh-workflow branch September 11, 2024 11:44
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bm1549 bm1549 bm1549 approved these changes

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
2 participants
@PerfectSlayer @bm1549