new Prompt Lepton ID MVA (and renaming) #44503

vlimant · 2024-03-21T16:05:22Z

rework of #44322 due to #44392

This PR is to introduce the newest training of the Muon mvaTTH in CMSSW. The training has been performed following what was done already done for Run 2, but using Run 3 MC samples for training and validation.

Context of the PR

The goal for this new training is to be used as a substitute of the current mvaTTH only in Run 3 MC production. That is: for Run 2 we still need to keep the current mvaTTH implementation (can be found in these lines), and for Run 3 we want to make use of an era modifier to fetch the proper --new -- weights that have been computed.

Linked PRs

This is the link for the PR that includes the weights in CMS-data: cms-data/PhysicsTools-NanoAOD#16.

vlimant · 2024-03-21T16:05:34Z

enable nano

cmsbuild · 2024-03-21T16:05:45Z

cms-bot internal usage

vlimant · 2024-03-21T16:06:20Z

type egamma

vlimant · 2024-03-21T16:10:13Z

type muon

cmsbuild · 2024-03-21T16:12:25Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44503/39619

This PR adds an extra 40KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Eras/python/Era_Run3_cff.py modified in PR(s): Implemented new modifiers for mvaTTH Run 3 #44322, [Dummy] Try creating big-enough cache to cause splitting #44368
- File Configuration/Eras/python/Modifier_run3_muon_cff.py modified in PR(s): Implemented new modifiers for mvaTTH Run 3 #44322
- File PhysicsTools/NanoAOD/python/electrons_cff.py modified in PR(s): Implemented new modifiers for mvaTTH Run 3 #44322
- File PhysicsTools/NanoAOD/python/muons_cff.py modified in PR(s): Implemented new modifiers for mvaTTH Run 3 #44322
- File PhysicsTools/NanoAOD/python/nano_eras_cff.py modified in PR(s): Implemented new modifiers for mvaTTH Run 3 #44322
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Run3 era for 2024 pp reference and UPC reco #44316, Implemented new modifiers for mvaTTH Run 3 #44322, slimmedMETsNoHF and possible cleanup #44335

vlimant · 2024-03-21T16:12:41Z

test parameters:

pull_requests = Weights for ttH mva Run 3 cms-data/PhysicsTools-NanoAOD#16

cmsbuild · 2024-03-21T16:12:48Z

A new Pull Request was created by @vlimant for master.

It involves the following packages:

Configuration/Eras (operations)
PhysicsTools/NanoAOD (xpog)
PhysicsTools/PatAlgos (xpog, reconstruction)

@davidlange6, @vlimant, @hqucms, @antoniovilela, @cmsbuild, @mandrenguyen, @rappoccio, @fabiocos, @jfernan2 can you please review it and eventually sign? Thanks.
@ahinzmann, @fabiocos, @gpetruc, @Martin-Grunewald, @gouskos, @seemasharmafnal, @hatakeyamak, @azotz, @JyothsnaKomaragiri, @andrzejnovak, @mbluj, @AlexDeMoor, @AnnikaStein, @Ming-Yan, @mmarionncern, @demuller, @emilbols, @mariadalfonso, @gkasieczka, @jdolen, @jdamgov, @makortel, @missirol, @Senphy, @nhanvtran, @schoef, @rappoccio this is something you requested to watch as well.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

vlimant · 2024-03-21T16:12:52Z

please test

cmsbuild · 2024-03-21T19:44:06Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d05221/38332/summary.html
COMMIT: 110adb7
CMSSW: CMSSW_14_1_X_2024-03-21-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44503/38332/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 106 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 39 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3297369
DQMHistoTests: Total failures: 98
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3297251
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 202 log files, 165 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially removed 4 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 28 differences found in the comparisons
DQMHistoTests: Total files compared: 15
DQMHistoTests: Total histograms compared: 16402
DQMHistoTests: Total failures: 70
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 16332
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
Checked 47 log files, 27 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.0	2.548	2.548	0.000 ( +0.0% )	5.19	5.16	+0.5%	2.229	2.176
2500.001	2.698	2.698	0.000 ( +0.0% )	4.62	4.58	+0.8%	2.629	2.606
2500.002	2.638	2.638	0.000 ( +0.0% )	4.83	4.84	-0.2%	2.630	2.609
2500.01	1.318	1.318	0.000 ( +0.0% )	9.31	9.39	-0.9%	2.333	2.256
2500.011	1.740	1.740	0.000 ( +0.0% )	5.17	5.13	+0.8%	2.432	2.416
2500.012	1.581	1.581	0.000 ( +0.0% )	7.17	7.36	-2.5%	2.416	2.300
2500.1	2.192	2.192	0.000 ( +0.0% )	5.22	5.22	+0.0%	2.059	1.998
2500.2	2.308	2.308	0.000 ( +0.0% )	5.98	6.01	-0.5%	1.974	1.892
2500.21	1.181	1.181	0.000 ( +0.0% )	4.26	4.29	-0.6%	2.272	2.209
2500.211	1.545	1.545	0.000 ( +0.0% )	3.70	3.74	-1.1%	2.337	2.230
2500.3	2.058	2.058	0.001 ( +0.0% )	12.13	12.26	-1.1%	1.964	1.891
2500.301	2.631	2.631	0.000 ( +0.0% )	9.65	9.98	-3.3%	1.948	1.827
2500.31	1.254	1.254	0.000 ( +0.0% )	18.99	19.11	-0.6%	2.340	2.246
2500.311	1.640	1.640	0.000 ( +0.0% )	13.38	13.39	-0.1%	2.437	2.323
2500.312	7.159	7.159	0.000 ( +0.0% )	1.37	1.45	-5.9%	1.707	1.703
2500.313	1.564	1.564	0.000 ( +0.0% )	6.41	6.74	-5.0%	1.061	1.049
2500.314	1.163	1.163	0.000 ( +0.0% )	13.29	13.56	-2.0%	2.061	2.022
2500.315	1.758	1.758	0.000 ( +0.0% )	13.07	13.54	-3.5%	2.325	2.317
2500.316	3.091	3.091	0.000 ( +0.0% )	2.20	2.23	-1.7%	2.266	2.116
2500.317	1.787	1.787	0.000 ( +0.0% )	12.51	13.43	-6.9%	2.167	2.153
2500.4	2.058	2.058	0.001 ( +0.0% )	12.03	12.00	+0.2%	1.854	1.826
2500.401	1.824	1.824	0.000 ( +0.0% )	9.82	10.21	-3.8%	1.768	1.787
2500.402	2.631	2.631	0.000 ( +0.0% )	9.45	9.85	-4.1%	1.804	1.826
2500.403	5.188	5.188	0.000 ( +0.0% )	1.55	1.57	-0.9%	1.957	1.947
2500.404	2.620	2.620	0.000 ( +0.0% )	9.37	9.74	-3.8%	1.751	1.872
2500.5	19.575	19.575	0.000 ( +0.0% )	1.20	0.93	+29.6%	1.372	1.333

vlimant · 2024-03-28T08:07:02Z

+1

vlimant · 2024-03-28T08:10:00Z

please test

cmsbuild · 2024-03-28T08:12:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44503/39718

This PR adds an extra 40KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Eras/python/Era_Run3_cff.py modified in PR(s): [Dummy] Try creating big-enough cache to cause splitting #44368
- File PhysicsTools/NanoAOD/python/nanoDQM_cfi.py modified in PR(s): Computation of Isolation for GenParticles collection #44461, [NanoAOD] Add number of B & C hadrons for GenJetAK8 and GenJet #44479

cmsbuild · 2024-03-28T08:12:45Z

Pull request #44503 was updated. @antoniovilela, @rappoccio, @davidlange6, @fabiocos, @mandrenguyen, @jfernan2 can you please check and sign again.

cmsbuild · 2024-03-28T11:48:33Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d05221/38484/summary.html
COMMIT: b95f405
CMSSW: CMSSW_14_1_X_2024-03-27-2300/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44503/38484/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 6 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 51 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3297359
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3297330
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.23400000000000007 KiB( 47 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 0.018 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): 0.012 KiB Physics/NanoAODDQM
Checked 202 log files, 165 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

The workflows 2500.2 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 28 differences found in the comparisons
DQMHistoTests: Total files compared: 15
DQMHistoTests: Total histograms compared: 16332
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 16332
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.21 KiB( 14 files compared)
DQMHistoSizes: changed ( 2500.001,... ): 0.018 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.011,... ): 0.012 KiB Physics/NanoAODDQM
Checked 47 log files, 27 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.0	2.548	2.548	0.000 ( +0.0% )	4.33	4.53	-4.6%	2.093	2.173
2500.001	2.697	2.697	0.000 ( +0.0% )	3.89	4.06	-4.1%	2.489	2.598
2500.002	2.637	2.637	0.000 ( +0.0% )	4.02	4.22	-4.6%	2.493	2.591
2500.01	1.318	1.318	0.000 ( +0.0% )	7.50	8.07	-7.1%	2.206	2.291
2500.011	1.739	1.739	0.000 ( +0.0% )	4.31	4.53	-5.0%	2.341	2.430
2500.012	1.580	1.580	0.000 ( +0.0% )	6.06	6.35	-4.5%	2.338	2.377
2500.1	2.192	2.192	0.000 ( +0.0% )	4.32	4.57	-5.3%	1.946	2.065
2500.2	2.308	2.308	0.000 ( +0.0% )	4.82	5.17	-6.9%	1.847	1.974
2500.21	1.181	1.181	0.000 ( +0.0% )	3.35	3.49	-4.1%	2.133	2.256
2500.211	1.545	1.545	0.000 ( +0.0% )	2.82	3.03	-7.0%	2.159	2.337
2500.3	2.064	2.064	0.000 ( +0.0% )	8.96	9.63	-7.0%	1.847	1.959
2500.301	2.637	2.637	0.000 ( +0.0% )	7.60	8.02	-5.3%	1.787	1.947
2500.31	1.255	1.255	0.000 ( +0.0% )	11.56	12.25	-5.7%	2.188	2.344
2500.311	1.642	1.642	0.000 ( +0.0% )	8.71	9.57	-9.0%	2.246	2.412
2500.312	7.159	7.159	0.000 ( +0.0% )	1.29	1.34	-4.1%	1.655	1.705
2500.313	1.564	1.564	0.000 ( +0.0% )	6.09	6.65	-8.5%	1.059	1.059
2500.314	1.163	1.163	0.000 ( +0.0% )	9.17	9.29	-1.3%	2.056	2.236
2500.315	1.758	1.758	0.000 ( +0.0% )	9.03	9.68	-6.7%	2.131	2.294
2500.316	3.093	3.093	0.000 ( +0.0% )	1.59	1.73	-7.7%	2.048	2.248
2500.317	1.789	1.789	0.000 ( +0.0% )	8.95	9.36	-4.4%	2.155	2.159
2500.4	2.221	2.221	0.000 ( +0.0% )	8.77	9.34	-6.1%	1.795	1.878
2500.401	1.755	1.755	0.000 ( +0.0% )	7.68	7.85	-2.2%	1.694	1.690
2500.402	2.766	2.766	0.000 ( +0.0% )	7.21	7.91	-8.9%	1.729	1.843
2500.403	5.068	5.068	0.000 ( +0.0% )	1.21	1.28	-5.3%	1.797	1.829
2500.404	2.774	2.774	0.000 ( +0.0% )	7.08	7.28	-2.7%	1.701	1.700
2500.5	4.936	4.936	0.000 ( +0.0% )	14.38	15.19	-5.4%	1.479	1.599
2500.51	8.960	8.960	0.000 ( +0.0% )	8.90	9.33	-4.5%	1.409	1.441

vlimant · 2024-03-30T09:25:19Z

@cms-sw/orp-l2 : I think we’re good to go with this one. What is blocking @cms-sw/reconstruction-l2 and @cms-sw/operations-l2 signatures?

mandrenguyen · 2024-03-30T17:14:06Z

+reconstruction
@vlimant please consider a slightly more detailed PR title, e.g., new Prompt lepton ID MVA or something.

antoniovilela · 2024-03-31T23:04:40Z

+1

cmsbuild · 2024-03-31T23:05:04Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

vlimant added 2 commits March 21, 2024 17:01

new muon eras

6451805

new prompt mva, and renaming TTHMVA to PromptMVA

110adb7

cmsbuild added this to the CMSSW_14_1_X milestone Mar 21, 2024

cmsbuild added reconstruction-pending operations-pending pending-signatures tests-pending orp-pending code-checks-pending xpog-pending labels Mar 21, 2024

cmsbuild added the egamma label Mar 21, 2024

vlimant mentioned this pull request Mar 21, 2024

Implemented new modifiers for mvaTTH Run 3 #44322

Closed

cmsbuild added the muon label Mar 21, 2024

cmsbuild added requires-external code-checks-approved and removed code-checks-pending labels Mar 21, 2024

cmsbuild added tests-started and removed tests-pending labels Mar 21, 2024

vlimant mentioned this pull request Mar 21, 2024

BaseMVAValueMapProducer and new Prompt lepton MVA [Backport] #44504

Merged

cmsbuild added tests-approved and removed tests-started labels Mar 21, 2024

cmsbuild added tests-started xpog-approved and removed tests-pending xpog-pending labels Mar 28, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 28, 2024

cmsbuild added tests-approved and removed tests-started labels Mar 28, 2024

cmsbuild mentioned this pull request Mar 29, 2024

Explicit lazy parsing in SimpleFlatTableProducer #44575

Closed

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 30, 2024

vlimant changed the title ~~new Prompt MVA (and renaming)~~ new Prompt Lepton ID MVA (and renaming) Mar 31, 2024

cmsbuild added operations-approved fully-signed orp-approved and removed operations-pending pending-signatures orp-pending labels Mar 31, 2024

cmsbuild merged commit f8ce14a into cms-sw:master Mar 31, 2024
14 checks passed

cmsbuild mentioned this pull request Apr 1, 2024

[GCC13] Fix Woverloaded-virtual warning #44581

Merged

vlimant deleted the rebase_of_44322 branch April 2, 2024 12:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new Prompt Lepton ID MVA (and renaming) #44503

new Prompt Lepton ID MVA (and renaming) #44503

vlimant commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024 •

edited

Loading

vlimant commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024

vlimant commented Mar 28, 2024

vlimant commented Mar 28, 2024

cmsbuild commented Mar 28, 2024

cmsbuild commented Mar 28, 2024

cmsbuild commented Mar 28, 2024

vlimant commented Mar 30, 2024

mandrenguyen commented Mar 30, 2024

antoniovilela commented Mar 31, 2024

cmsbuild commented Mar 31, 2024

new Prompt Lepton ID MVA (and renaming) #44503

new Prompt Lepton ID MVA (and renaming) #44503

Conversation

vlimant commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024 • edited Loading

vlimant commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024

vlimant commented Mar 21, 2024

cmsbuild commented Mar 21, 2024

Comparison Summary

NANO Comparison Summary

vlimant commented Mar 28, 2024

vlimant commented Mar 28, 2024

cmsbuild commented Mar 28, 2024

cmsbuild commented Mar 28, 2024

cmsbuild commented Mar 28, 2024

Comparison Summary

NANO Comparison Summary

vlimant commented Mar 30, 2024

mandrenguyen commented Mar 30, 2024

antoniovilela commented Mar 31, 2024

cmsbuild commented Mar 31, 2024

cmsbuild commented Mar 21, 2024 •

edited

Loading