Fix for the cosmetic issue for the GEM DQM summary plot

[quark2]

PR description:

In this PR, a cosmetic bug in the GEM DQM summary plots has been fixed. The bug occurs because we did not take into account the effect of the multi-core environment on the storage of the geometry information in histograms. The geometry information will be retrieved correctly by this fix.

PR validation:

Tests are done with runTheMatrix.py -l 141.107 with an additional run (367142) and runTheMatrix.py -l limited -i all --ibeos.

@jshlee @watson-ij @Dongwoon77

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45531/41002

[cmsbuild]

A new Pull Request was created by @quark2 for master.

It involves the following packages:

• DQM/GEM (dqm)

@antoniovagnerini, @cmsbuild, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please review it and eventually sign? Thanks.
@watson-ij this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[watson-ij]

please test

[cmsbuild]

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-193f21/40537/summary.html
COMMIT: c9b766a
CMSSW: CMSSW_14_1_X_2024-07-23-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45531/40537/install.sh to create a dev area with all the needed externals and cmssw changes.

• DAS Queries: The DAS query tests failed, see the summary page for details.

Comparison Summary

Summary:

• You potentially removed 2 lines from the logs
• Reco comparison results: 89 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3345208
• DQMHistoTests: Total failures: 2521
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3342667
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 202 log files, 165 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[quark2]

By the way, the cosmetic issue occurs on the run 382204, for example. If a replay is performed, is it possible to replay this run?

[tjavaid]

please test

• just to fetch bin-by-bin comparison plots (missing in previous iteration).

[cmsbuild]

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-193f21/40672/summary.html
COMMIT: c9b766a
CMSSW: CMSSW_14_1_X_2024-07-28-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45531/40672/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 1 lines to the logs
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 45
• DQMHistoTests: Total histograms compared: 3423961
• DQMHistoTests: Total failures: 63
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3423878
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
• Checked 196 log files, 165 edm output root files, 45 DQM output files
• TriggerResults: no differences found

[quark2]

I've checked that we have the desired differences in GEM DQM histograms. As you can see from the code, the differences are not displayed in the test GUI since they are on some underflow bins. I downloaded the histograms and checked them by myself.

[tjavaid]

I've checked that we have the desired differences in GEM DQM histograms. As you can see from the code, the differences are not displayed in the test GUI since they are on some underflow bins. I downloaded the histograms and checked them by myself.

Hi @quark2 , thanks for checking. We are also going to test this PR online at our playback machines. Get back to you soon.

[tjavaid]

I've checked that we have the desired differences in GEM DQM histograms. As you can see from the code, the differences are not displayed in the test GUI since they are on some underflow bins. I downloaded the histograms and checked them by myself.

Hi @quark2 , thanks for checking. We are also going to test this PR online at our playback machines. Get back to you soon.

hi @quark2 , all the clients are running without any crash while we test it on the top of CMSSW_14_0_12. Could you please have a look at GUI plots and see whether those look as expected ?

[quark2]

Hi @tjavaid,

Thanks for the test! The plots look good. However, the issue occurs only on the offlineDQM with a multicore environment. Although I have already tested this PR locally before, is it possible to test it in this environment?

Best regards,
Byeonghak Ko

[cmsbuild]

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-193f21/41024/summary.html
COMMIT: c9b766a
CMSSW: CMSSW_14_1_X_2024-08-19-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45531/41024/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 157 differences found in the comparisons
• DQMHistoTests: Total files compared: 45
• DQMHistoTests: Total histograms compared: 3422510
• DQMHistoTests: Total failures: 362
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3422128
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
• Checked 196 log files, 165 edm output root files, 45 DQM output files
• TriggerResults: no differences found

[tjavaid]

Hi @quark2 , We have tested this PR again on our playback machines. Looking at GUI plots, the summary plot looks okay. You can have a look and let us know.

In addition, with bin-by-bin plots from jenkins tests, you can have a look at (e.g. for W.F. 140.063 ) GEM plots, and let us know about your assessment.

[quark2]

Hi @tjavaid,

Sorry for my late...
Yep, it seems fine for me, as before. Thanks for your help!

[tjavaid]

+1

• P5 tests OK.
• spurious difference observed from WF 29634.911 see issue (Non-reproducibility in 29634.911 #45505)
• @quark2 , would like to make a backport PR for 14_0_X ?

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[quark2]

Yep, sure. I'll prepare the backport.

[cmsbuild]

Milestone for this pull request has been moved to CMSSW_14_2_X. Please open a backport if it should also go in to CMSSW_14_1_X.

[tjavaid]

Milestone for this pull request has been moved to CMSSW_14_2_X. Please open a backport if it should also go in to CMSSW_14_1_X.

@quark2 , could you please remove [14_1_0] from the title of this PR ? Thanks!

[mandrenguyen]

+1

[mandrenguyen]

type gem