[JMENanoAOD] Updates for Run-3 #39000

nurfikri89 · 2022-08-08T19:02:38Z

PR description:

This PR updates JMENano to be compatible with Run-3 data and MC, starting with the upcoming NanoAODv10 production. The following major changes are made:

Sequences have been changed to Tasks and JMENano can now run in Prompt.
Updated names of NanoAOD-related config files and modules.
Add a dedicated function for reclustering and content customisation of AK4 Puppi jets.
Recluster both AK4 CHS and AK4 Puppi jet collections but modify their table names and content depending on era (i.e Run-2 or Run-3).
Increase pt threshold for stored reclustered AK8 reco jets to 15 GeV.
The JMENano workflows are enabled back again, which was disabled by PR Disable JME Nano workflows due to Nano-prompt dev #35140. Enable Run-3 workflows and added one wf: 11634.15.

PR validation:

passes the (re-enabled) JMENano workflows: runTheMatrix.py -i all --ibeos -l 10224.15,11024.15,25202.15,11634.15

With this PR, the event size is 6.24 kb/event. Test was done using 10K events from a TTTo2J1L1Nu Run3Winter22MiniAOD file. Below is the CPU time:

The full content and size details can be found in the JMENano twiki: content / size

if this PR is a backport please specify the original PR and why you need to backport that PR:

Once merged, this PR needs to be backported to 12_4_X for the JMENanoAODv10 production.

- Update name of NanoAOD-related config files and modules - Add adedicated function for reclustering and content customisation of AK4 Puppi jets. - Recluster both AK4 CHS and AK4 Puppi jet collections but modify their table names and content depending on eras (i.e Run-2 or Run-3).

cmsbuild · 2022-08-08T19:09:22Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39000/31469

This PR adds an extra 32KB to repository

cmsbuild · 2022-08-08T19:09:47Z

A new Pull Request was created by @nurfikri89 (Nurfikri Norjoharuddeen) for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)
PhysicsTools/NanoAOD (xpog)
PhysicsTools/PatAlgos (reconstruction)

@gouskos, @swertz, @jordan-martins, @vlimant, @bbilin, @clacaputo, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @jpata, @mariadalfonso can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @schoef, @emilbols, @kpedro88, @Martin-Grunewald, @trtomei, @mbluj, @ahinzmann, @demuller, @seemasharmafnal, @mmarionncern, @jdolen, @makortel, @fabiocos, @missirol, @azotz, @slomeo, @beaucero, @jdamgov, @nhanvtran, @gkasieczka, @hatakeyamak, @mariadalfonso, @AlexDeMoor, @swertz, @JyothsnaKomaragiri, @gpetruc, @andrzejnovak this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mariadalfonso · 2022-08-08T19:20:09Z

please test

mariadalfonso · 2022-08-08T19:32:30Z

size 6.33 kb/event is within the allocated budget
RecoBTag takes a lot of time, in the future should be optimized @hqucms maybe can have a look later
10224.15,11024.15,25202.15 which one is Run3 ?
What is the difference between the
FatJet and FatJetForJEC
https://twiki.cern.ch/twiki/pub/CMS/JMECustomNanoAOD/JMENano_version4_forPR_39000_content.html#FatJetForJEC)
https://twiki.cern.ch/twiki/pub/CMS/JMECustomNanoAOD/JMENano_version4_forPR_39000_content.html#FatJet
a note as discussed by email, we need a DQM to
--> compare JMEnano vs central nano so that we are sure the main collection of jet is ok despite the reclustering
(for this the current nanoDQM_cfi.py should be enough)
--> compare JMEnano different versions for the extra things added on top.
(for this we likely need a different nanoDQM_jme_cfi.py)

2 and 5 can be done in a future PRs

kpedro88 · 2022-08-08T19:37:36Z

PhysicsTools/NanoAOD/python/custom_jme_cff.py

+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leadTrackPt = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonPtRel = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonPtRatio = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonPtRelInv = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonPtRelv0 = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonPtRatiov0 = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonPtRelInvv0 = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonDeltaR = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leptonPt = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, vtxPt = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, vtxMass = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, vtx3dL = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, vtx3deL = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, ptD = None)
+  (~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, genPtwNu = None)


you can make multiple modifications in a single toModify() statement (and therefore you should, in order to reduce duplication). example:

(~run2_nanoAOD_ANY).toModify(proc.updatedJetsWithUserData.userFloats, leadTrackPt = None, leptonPtRel = None, )

Thanks for the suggestion. Will clean up.

kpedro88 · 2022-08-08T19:37:47Z

PhysicsTools/NanoAOD/python/custom_jme_cff.py

+      proc.nanoTableTaskCommon.add(proc.jetPuppiForMETTask)
+    return proc
+
+  jmeNano_addAK4JetTaks_switch = cms.PSet(


fix typo Taks -> Task

To be corrected.

nurfikri89 · 2022-08-08T19:54:27Z

10224.15,11024.15,25202.15 which one is Run3 ?

I don't think any of them is Run-3.

What is the difference between the
FatJet and FatJetForJEC
https://twiki.cern.ch/twiki/pub/CMS/JMECustomNanoAOD/JMENano_version4_forPR_39000_content.html#FatJetForJEC)
https://twiki.cern.ch/twiki/pub/CMS/JMECustomNanoAOD/JMENano_version4_forPR_39000_content.html#FatJet

FatJet is taken directly from slimmedJetsAK8 collection in MiniAOD with a minimum pt cut of 170 GeV while FatJetForJEC is a reclustered AK8 Puppi jet collection which can go as low as jet pt of 8 GeV with a basic set of jet variables saved for JEC studies.

cmsbuild · 2022-08-08T20:07:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39000/31470

This PR adds an extra 32KB to repository

mariadalfonso · 2022-08-15T07:30:30Z

@nurfikri89

seems we have twice this branch
https://twiki.cern.ch/twiki/pub/CMS/JMECustomNanoAOD/JMENano_version4_forPR_39000_content.html#HLTriggerFinalPath

nurfikri89 · 2022-08-15T08:35:06Z

seems we have twice this branch https://twiki.cern.ch/twiki/pub/CMS/JMECustomNanoAOD/JMENano_version4_forPR_39000_content.html#HLTriggerFinalPath

@mariadalfonso I could only find one branch with the name HLTriggerFinalPath. The other one is HLTriggerFirstPath.

mariadalfonso · 2022-08-15T12:16:02Z

+xpog

seems fine from technical point of view.
content wise not possible to verify the content w./o the DQM file
no confirmation that the 12B from Run2 UL were useful (AFIAK have never used to derive JEC)

to be done ASAP: 2 and 5 from here
#39000 (comment)
until then DO NOT set to central production

jpata · 2022-08-15T12:38:54Z

+reconstruction

resign since [JMENanoAOD] Updates for Run-3 #39000 (comment)

srimanob · 2022-08-15T12:44:49Z

+Upgrade

From the upgrade side, this PR update the workflows for JMENano. Workflow runs fine. For technical topics, i.e. contents or speed up, please follow up with @cms-sw/xpog-l2.

mariadalfonso · 2022-08-15T20:40:24Z

@nurfikri89 please backport to 12_4

kskovpen · 2022-08-18T07:46:29Z

+pdmv

cmsbuild · 2022-08-18T07:46:57Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-08-18T08:21:24Z

+1

As far as I understand from [JMENanoAOD] Updates for Run-3 #39000 (comment), this has to be further verified before backporting it in the nanoAOD production release 12_4_X: next week 12_5_0_pre5 relvals could be a good opportunity to do so

srimanob · 2022-08-18T08:30:08Z

@mariadalfonso @cms-sw/pdmv-l2 @nurfikri89
Just to confirm on the content validation, should the workflows with .15 be submitted with the next pre-release to get DQM for validation? Or this will be done from x-pog or JME side privately. Thanks.

mariadalfonso · 2022-08-18T10:32:35Z

@mariadalfonso @cms-sw/pdmv-l2 @nurfikri89
Just to confirm on the content validation, should the workflows with .15 be submitted with the next pre-release to get DQM for validation? Or this will be done from x-pog or JME side privately. Thanks.

The DQM was just never in the release.
We need something like this
https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/nanoDQM_cfi.py

Once this is added we can aim to test it:

JMEnano vs JMEnano
JMEnano vs central nano (w.o this pretty much I'm not sure what we use for calibration is what is going to be used for analysis given the large amount of cooking in the JMEnano).

The 12B of UL dataset were submitted by PPD for FWK validation and BLINDLY. Also all the presentation shown at PPD never included a single plot and nd PRs desciption never contain distributions.
Still waiting to close the loop on this unfortunately.

…un3_forPR [JMENanoAOD] Backport of #39000 (Updates for Run-3) to 12_4_X

nurfikri89 added 4 commits August 2, 2022 20:34

import pfCHS

c2db494

Switch from Sequence to Task for prompt production

70877c5

Re-enable JMENano workflows

1202498

cmsbuild added this to the CMSSW_12_5_X milestone Aug 8, 2022

cmsbuild added code-checks-pending orp-pending pdmv-pending pending-signatures reconstruction-pending tests-pending upgrade-pending xpog-pending labels Aug 8, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 8, 2022

cmsbuild added tests-started and removed tests-pending labels Aug 8, 2022

kpedro88 reviewed Aug 8, 2022

View reviewed changes

Clean up toModify and fix typo

c538786

cmsbuild added code-checks-pending tests-pending and removed tests-started code-checks-approved labels Aug 8, 2022

cmsbuild removed the code-checks-pending label Aug 8, 2022

cmsbuild added xpog-approved and removed xpog-pending labels Aug 15, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Aug 15, 2022

cmsbuild added upgrade-approved and removed upgrade-pending labels Aug 15, 2022

nurfikri89 mentioned this pull request Aug 16, 2022

[JMENanoAOD] Backport of #39000 (Updates for Run-3) to 12_4_X #39078

Merged

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Aug 18, 2022

cmsbuild added orp-approved and removed orp-pending labels Aug 18, 2022

cmsbuild merged commit e2e7961 into cms-sw:master Aug 18, 2022

This was referenced Aug 18, 2022

[GCC flags] enable -Wconversion flag cms-sw/cmsdist#8027

Closed

[llvm] Update to version 14.0.6 cms-sw/cmsdist#7729

Closed

cmsbuild added a commit that referenced this pull request Aug 22, 2022

Merge pull request #39078 from nurfikri89/portFrom125XTo124X_JMENanoR…

f3a9a2f

…un3_forPR [JMENanoAOD] Backport of #39000 (Updates for Run-3) to 12_4_X

alintulu mentioned this pull request Jan 17, 2023

Custom Run 3 PFScouting NanoAOD #40438

Merged

nurfikri89 deleted the from1250pre4_JMENanoRun3 branch October 29, 2023 06:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JMENanoAOD] Updates for Run-3 #39000

[JMENanoAOD] Updates for Run-3 #39000

nurfikri89 commented Aug 8, 2022 •

edited

Loading

cmsbuild commented Aug 8, 2022

cmsbuild commented Aug 8, 2022

mariadalfonso commented Aug 8, 2022

mariadalfonso commented Aug 8, 2022 •

edited

Loading

kpedro88 Aug 8, 2022

nurfikri89 Aug 8, 2022

kpedro88 Aug 8, 2022

nurfikri89 Aug 8, 2022

nurfikri89 commented Aug 8, 2022

cmsbuild commented Aug 8, 2022

mariadalfonso commented Aug 15, 2022

nurfikri89 commented Aug 15, 2022

mariadalfonso commented Aug 15, 2022

jpata commented Aug 15, 2022

srimanob commented Aug 15, 2022

mariadalfonso commented Aug 15, 2022

kskovpen commented Aug 18, 2022

cmsbuild commented Aug 18, 2022

perrotta commented Aug 18, 2022

srimanob commented Aug 18, 2022

mariadalfonso commented Aug 18, 2022

[JMENanoAOD] Updates for Run-3 #39000

[JMENanoAOD] Updates for Run-3 #39000

Conversation

nurfikri89 commented Aug 8, 2022 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Aug 8, 2022

cmsbuild commented Aug 8, 2022

mariadalfonso commented Aug 8, 2022

mariadalfonso commented Aug 8, 2022 • edited Loading

kpedro88 Aug 8, 2022

Choose a reason for hiding this comment

nurfikri89 Aug 8, 2022

Choose a reason for hiding this comment

kpedro88 Aug 8, 2022

Choose a reason for hiding this comment

nurfikri89 Aug 8, 2022

Choose a reason for hiding this comment

nurfikri89 commented Aug 8, 2022

cmsbuild commented Aug 8, 2022

mariadalfonso commented Aug 15, 2022

nurfikri89 commented Aug 15, 2022

mariadalfonso commented Aug 15, 2022

jpata commented Aug 15, 2022

srimanob commented Aug 15, 2022

mariadalfonso commented Aug 15, 2022

kskovpen commented Aug 18, 2022

cmsbuild commented Aug 18, 2022

perrotta commented Aug 18, 2022

srimanob commented Aug 18, 2022

mariadalfonso commented Aug 18, 2022

nurfikri89 commented Aug 8, 2022 •

edited

Loading

mariadalfonso commented Aug 8, 2022 •

edited

Loading