ci: Run physmon jobs in parallel

[benjaminhuth]

Some of the physmon jobs had numThreads=1, I think it would speed up the physmon to run all things in parallel (numThreads=-1).

[andiwand]

might be that I accidentally changed that. thanks for improving this!

[github-actions]

📊 Physics performance monitoring for 3cc8163

🟥 ERROR The result has missing elements!
This is likely a physmon job failure

Full report
CKF: ❌ seeded, ❌ truth smeared, ❌ truth estimated
IVF: ❌ seeded, ❌ truth smeared, ❌ truth estimated
❌ Ambiguity resolution
❌ Truth tracking

Vertexing ❌

❌ Vertexing vs. mu

❌ IVF seeded

❌ IVF truth_smeared

❌ IVF truth_estimated

CKF ❌

❌ CKF seeded

❌ CKF truth_smeared

❌ CKF truth_estimated

Ambiguity resolution ❌

❌ seeded

Truth tracking ❌

❌ Truth tracking

[codecov]

Codecov Report

Merging #1654 (831e407) into main (f3814fa) will increase coverage by 0.04%.
The diff coverage is n/a.

❗ Current head 831e407 differs from pull request most recent head 3cc8163. Consider uploading reports for the commit 3cc8163 to get more accurate results

@@            Coverage Diff             @@
##             main    #1654      +/-   ##
==========================================
+ Coverage   48.49%   48.54%   +0.04%     
==========================================
  Files         384      384              
  Lines       21060    21042      -18     
  Branches     9694     9693       -1     
==========================================
+ Hits        10213    10214       +1     
+ Misses       4123     4106      -17     
+ Partials     6724     6722       -2     

Impacted Files	Coverage Δ
...re/include/Acts/TrackFitting/GaussianSumFitter.hpp	64.63% <0.00%> (-0.22%)	⬇️
Core/include/Acts/Seeding/SeedFilter.ipp	0.00% <0.00%> (ø)
Core/include/Acts/Seeding/SeedFilterConfig.hpp	0.00% <0.00%> (ø)
...e/include/Acts/TrackFitting/BetheHeitlerApprox.hpp
...de/Acts/TrackFitting/detail/BetheHeitlerApprox.hpp	32.55% <0.00%> (ø)
Core/include/Acts/TrackFitting/detail/GsfActor.hpp	43.30% <0.00%> (+1.11%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[benjaminhuth]

Okay, seems like there was a reason for this maybe? Since now the physmon is failing? Any idea @andiwand @paulgessinger ?

[paulgessinger]

This should work though...
The failures seem to be from the pull and residual plots. @andiwand could you check those maybe, they're not in the automatic plotting yet?

[andiwand]

looking into the output https://github.com/acts-project/acts/suites/9129855748/artifacts/424126850 I cannot see any failed checks 🤔

[andiwand]

okay looks like there is a diff in the ambi root which is not shown in the html

2022-11-04T13:29:46.0187231Z ──────────────────────── all_range_res_eLOC0_prt (TH1F) ────────────────────────
2022-11-04T13:29:46.0192367Z Error in <TH1F::KolmogorovTest>: Histograms are not consistent: they have different bin edges
2022-11-04T13:29:46.0192950Z 🟡 Chi2Test
2022-11-04T13:29:46.0195993Z 🟡 KolmogorovTest
2022-11-04T13:29:46.0281432Z ##[error]all_range_res_eLOC0_prt: RatioCheck
(a/b - 1) / sigma(a/b) > 3 for 24/43 bins, cf. 6.557438524302
2022-11-04T13:29:46.0297304Z 🔴 RatioCheck (a/b - 1) / sigma(a/b) > 3 for 24/43 bins, cf. 6.557438524302
2022-11-04T13:29:46.0297772Z Error in <TH1F::KolmogorovTest>: Histograms are not consistent: they have different bin edges
2022-11-04T13:29:46.0298169Z Error in <TH1F::KolmogorovTest>: Histograms are not consistent: they have different bin edges
2022-11-04T13:29:46.0299221Z ##[error]all_range_res_eLOC0_prt: ResidualCheck
pull > 1 in 92/100 bins, cf. 10.0
2022-11-04T13:29:46.0299932Z 🔴 ResidualCheck pull > 1 in 92/100 bins, cf. 10.0
2022-11-04T13:29:46.0300292Z ✅ IntegralTest Intregal: 4773192.0+-2184.7636027726203 vs. 
2022-11-04T13:29:46.0300650Z 4773192.0+-2184.7636027726203: (int_a - int_b) / sqrt(sigma(int_a)^2 + 
2022-11-04T13:29:46.0300904Z sigma(int_b)^2) = 0.00 < 3
2022-11-04T13:29:46.0305801Z ──────────────────────── all_range_res_eLOC1_prt (TH1F) ────────────────────────
2022-11-04T13:29:46.0306088Z 🟡 Chi2Test
2022-11-04T13:29:46.0306305Z 🟡 KolmogorovTest
2022-11-04T13:29:46.0310170Z ##[error]all_range_res_eLOC1_prt: RatioCheck
(a/b - 1) / sigma(a/b) > 3 for 26/44 bins, cf. 6.6332495807108
2022-11-04T13:29:46.0310913Z 🔴 RatioCheck (a/b - 1) / sigma(a/b) > 3 for 26/44 bins, cf. 6.6332495807108
2022-11-04T13:29:46.0316153Z ##[error]all_range_res_eLOC1_prt: ResidualCheck
pull > 1 in 94/100 bins, cf. 10.0
2022-11-04T13:29:46.0319857Z 🔴 ResidualCheck pull > 1 in 94/100 bins, cf. 10.0
2022-11-04T13:29:46.0320227Z ✅ IntegralTest Intregal: 4773192.0+-2184.7636027726203 vs. 
2022-11-04T13:29:46.0320596Z 4773192.0+-2184.7636027726203: (int_a - int_b) / sqrt(sigma(int_a)^2 + 
2022-11-04T13:29:46.0320832Z sigma(int_b)^2) = 0.00 < 3
2022-11-04T13:29:46.0321201Z ──────────────────────── all_range_res_ePHI_prt (TH1F) ─────────────────────────
2022-11-04T13:29:46.0321503Z 🟡 Chi2Test
2022-11-04T13:29:46.0321733Z 🟡 KolmogorovTest
2022-11-04T13:29:46.0322063Z ✅ RatioCheck (a/b - 1) / sigma(a/b) > 3 for 5/72 bins, cf. 8.48528137423857
2022-11-04T13:29:46.0322833Z ##[error]all_range_res_ePHI_prt: ResidualCheck
pull > 1 in 77/100 bins, cf. 10.0
2022-11-04T13:29:46.0323444Z 🔴 ResidualCheck pull > 1 in 77/100 bins, cf. 10.0
2022-11-04T13:29:46.0323791Z Error in <TH1F::KolmogorovTest>: Histograms are not consistent: they have different bin edges
2022-11-04T13:29:46.0324203Z ✅ IntegralTest Intregal: 4773192.0+-2184.7636027726203 vs. 
2022-11-04T13:29:46.0328260Z 4773192.0+-2184.7636027726203: (int_a - int_b) / sqrt(sigma(int_a)^2 + 
2022-11-04T13:29:46.0328498Z sigma(int_b)^2) = 0.00 < 3
2022-11-04T13:29:46.0328876Z ──────────────────────── all_range_res_eQOP_prt (TH1F) ─────────────────────────
2022-11-04T13:29:46.0329157Z 🟡 Chi2Test
2022-11-04T13:29:46.0329373Z 🟡 KolmogorovTest
2022-11-04T13:29:46.0329726Z ✅ RatioCheck (a/b - 1) / sigma(a/b) > 3 for 2/87 bins, cf. 9.327379053088816
2022-11-04T13:29:46.0339444Z ##[error]all_range_res_eQOP_prt: ResidualCheck
pull > 1 in 76/100 bins, cf. 10.0
2022-11-04T13:29:46.0340242Z 🔴 ResidualCheck pull > 1 in 76/100 bins, cf. 10.0
2022-11-04T13:29:46.0345622Z ✅ IntegralTest Intregal: 4773192.0+-2184.7636027726203 vs. 
2022-11-04T13:29:46.0346009Z 4773192.0+-2184.7636027726203: (int_a - int_b) / sqrt(sigma(int_a)^2 + 
2022-11-04T13:29:46.0346244Z sigma(int_b)^2) = 0.00 < 3

[andiwand]

looks like the analysis outputs don't match up

2022-11-04T13:29:45.0173033Z ╭────────────────────────────── Comparing files: ──────────────────────────────╮
2022-11-04T13:29:45.0173855Z │ Monitored: physmon/acts_analysis_residuals_and_pulls.root                    │
2022-11-04T13:29:45.0220656Z │ Reference: CI/physmon/reference/acts_analysis_residuals_and_pulls.root       │
2022-11-04T13:29:45.0221399Z ╰──────────────────────────────────────────────────────────────────────────────╯

any ideas @paulgessinger @benjaminhuth ? maybe the analysis evaluation is sensitive to the order of events in the root files? if so an update of the reference data will not be sufficient I guess

[benjaminhuth]

Do all plots differ or just some of them?