Implementation of beam halo tagger algorithm for the endcap photons #36901

jainshilpi · 2022-02-07T00:38:09Z

This PR concerns the newly developed BDT based algorithm to deal with a huge beam halo background in the ECAL endcap which is identified as good photons. Already presented in ECAL DPG, JetMET POG and E/gamma groups:

ECAL DPG: https://indico.cern.ch/event/991261/contributions/4283096/attachments/2219229/3757719/beamHalo_31march_v1.pdf
JetMET POG: https://indico.cern.ch/event/1027614/contributions/4314949/attachments/2224472/3767396/beamHalo_12April.pdf
E/gamma: https://indico.cern.ch/event/1095380/#13-beam-halo-tagging-in-ee
The last presentation has the checks done as well comparing the implementation using private framework
and the CMSSW implementation.
This implementation touches the DataFromats/EgammaCandidates/Photon.h
The PR for training file will be done as well in RecoEgamma/PhotonIdentification/data

This training is relevant for photons with PT > 100 GeV - the regime important for single photon (+MET) analyses.
The runMatrix tests have been done and they all ran successfully.

Currently the changes are done only at the miniAOD level. After the discussion with the xpog, the relevant variable will be put in the nanoAOD as well.

Thanks to @thomreis for the help.

…ith cms-merge-topic

cmsbuild · 2022-02-07T00:47:06Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36901/28185

This PR adds an extra 60KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): Fix calculation of smeared MET on MiniAOD #36165, Adding muon MVA ID #36179
- File RecoEgamma/EgammaPhotonProducers/src/GEDPhotonProducer.cc modified in PR(s): ele/ph mass fix #36889

cmsbuild · 2022-02-07T00:47:28Z

A new Pull Request was created by @jainshilpi for master.

It involves the following packages:

DataFormats/EgammaCandidates (reconstruction)
DataFormats/PatCandidates (reconstruction)
RecoEgamma/EgammaPhotonProducers (reconstruction)
RecoEgamma/PhotonIdentification (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@Sam-Harper, @sobhatta, @afiqaize, @valsdav, @rovere, @lgray, @gouskos, @lecriste, @hatakeyamak, @gpetruc, @wrtabb, @varuns23, @ram1123 this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

jpata · 2022-02-07T09:20:04Z

test parameters:

pull_requests = First version of BDT based BH tagger training files cms-data/RecoEgamma-PhotonIdentification#11

jpata · 2022-02-07T09:20:42Z

@cmsbuild please test

cmsbuild · 2022-02-07T10:51:42Z

-1

Failed Tests: Build ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b5b203/22257/summary.html
COMMIT: fdcfde2
CMSSW: CMSSW_12_3_X_2022-02-06-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36901/22257/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b5b203/22257/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b5b203/22257/git-merge-result

Build

I found compilation warning when building: See details on the summary page.

Clang Build

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 64 COMPILER='llvm compile'

See details on the summary page.

jpata · 2022-02-07T15:41:57Z

The following clang errors seem relevant:

RecoEgamma/PhotonIdentification/src/PhotonMVABasedHaloTagger.cc
/pool/condor/dir_122481/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2022-02-06-2300/src/RecoEgamma/PhotonIdentification/src/PhotonMVABasedHaloTagger.cc:157:21: warning: equality comparison with extraneous parentheses [-Wparentheses-equality]
/pool/condor/dir_122481/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2022-02-06-2300/src/RecoEgamma/PhotonIdentification/src/PhotonMVABasedHaloTagger.cc:382:12: warning: unused variable 'rhPhi' [-Wunused-variable]

cmsbuild · 2022-02-07T16:57:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36901/28199

This PR adds an extra 16KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): Fix calculation of smeared MET on MiniAOD #36165, Adding muon MVA ID #36179
- File RecoEgamma/EgammaPhotonProducers/src/GEDPhotonProducer.cc modified in PR(s): ele/ph mass fix #36889

cmsbuild · 2022-03-02T19:53:42Z

Pull request #36901 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

perrotta · 2022-03-02T20:39:29Z

@cmsbuild please test

cmsbuild · 2022-03-03T06:27:04Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b5b203/22784/summary.html
COMMIT: 73220c5
CMSSW: CMSSW_12_3_X_2022-03-02-1100/slc7_amd64_gcc10
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36901/22784/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b5b203/22784/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b5b203/22784/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 144 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3987333
DQMHistoTests: Total failures: 152
DQMHistoTests: Total nulls: 6
DQMHistoTests: Total successes: 3987153
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -6538.53 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -1091.170 KiB HLT/BPH
DQMHistoSizes: changed ( 11634.0,... ): 1.415 KiB HLT/Filters
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: found differences in 7 / 48 workflows

jpata · 2022-03-03T12:36:17Z

+reconstruction

on top of Implementation of beam halo tagger algorithm for the endcap photons #36901 (comment), some code optimization was done

cmsbuild · 2022-03-03T12:36:36Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-03-03T14:44:04Z

+1

francescobrivio · 2022-03-03T23:18:03Z

RecoEgamma/PhotonIdentification/python/mvaHaloVariable_cfi.py

+import FWCore.ParameterSet.Config as cms
+from RecoEgamma.EgammaIsolationAlgos.egammaHBHERecHitThreshold_cff import egammaHBHERecHit
+
+pathToHaloMVATrainingFile = "RecoEgamma/PhotonIdentification/data/beamHaloTaggerID/xgboostToTMVA_BHtagger.root"


Seems like the beamHaloTaggerID/xgboostToTMVA_BHtagger.root file wasn't included, so this PR broke the IB judging from #37130 (comment).

Thanks! Should be fixed now by cms-sw/cmsdist#7665

perrotta · 2022-03-06T18:34:30Z

@jainshilpi as suggested by @mmusich in #37143 (comment) there is quite likely some uninitialized variable in the PhotonMVABasedHaloTagger that causes some unphysical (and non reproducible) haloTaggerMVA value, see the results of the comparisons and the plots atteched to the #37143 github thread.
Could you please have a look and provide a fix, if needed?

jainshilpi · 2022-03-07T01:06:40Z

@perrotta - thanks for pointing to this problem and sorry that I missed to have the appropriate treatment to Run 1+2 files. It seems that the WF 136.7611 essentially picks the 2016 AOD files and tries to find the value of haloTaggerMVAVal_. Since this variable does not exist in those AOD files (which is added by this PR in the releases beyond 12_3_X), this just takes some junk value and puts in haloTaggerMVAVal(). I have fixed this issue in this update:
https://github.com/cms-sw/cmssw/compare/master...jainshilpi:BHtaggerImplementation_12_3_0pre4_codechecks?expand=1

Re-running over the same WF seems now to give -99 value for this variable. I also checked by running over the AOD files generated in 12_3_X and correct values can be seen.

Can you please confirm that the treatment to old files seem fine to you? If this is fine, I suppose I should do a new PR?

perrotta · 2022-03-07T07:51:33Z

@perrotta - thanks for pointing to this problem and sorry that I missed to have the appropriate treatment to Run 1+2 files. It seems that the WF 136.7611 essentially picks the 2016 AOD files and tries to find the value of haloTaggerMVAVal_. Since this variable does not exist in those AOD files (which is added by this PR in the releases beyond 12_3_X), this just takes some junk value and puts in haloTaggerMVAVal(). I have fixed this issue in this update: https://github.com/cms-sw/cmssw/compare/master...jainshilpi:BHtaggerImplementation_12_3_0pre4_codechecks?expand=1

Re-running over the same WF seems now to give -99 value for this variable. I also checked by running over the AOD files generated in 12_3_X and correct values can be seen.

Can you please confirm that the treatment to old files seem fine to you? If this is fine, I suppose I should do a new PR?

Thank you @jainshilpi for the prompt feedback!
Could you please make a PR out of it?

jainshilpi · 2022-03-07T08:15:38Z

@perrotta - thanks for pointing to this problem and sorry that I missed to have the appropriate treatment to Run 1+2 files. It seems that the WF 136.7611 essentially picks the 2016 AOD files and tries to find the value of haloTaggerMVAVal_. Since this variable does not exist in those AOD files (which is added by this PR in the releases beyond 12_3_X), this just takes some junk value and puts in haloTaggerMVAVal(). I have fixed this issue in this update: https://github.com/cms-sw/cmssw/compare/master...jainshilpi:BHtaggerImplementation_12_3_0pre4_codechecks?expand=1
Re-running over the same WF seems now to give -99 value for this variable. I also checked by running over the AOD files generated in 12_3_X and correct values can be seen.
Can you please confirm that the treatment to old files seem fine to you? If this is fine, I suppose I should do a new PR?

Thank you @jainshilpi for the prompt feedback! Could you please make a PR out of it?

Hi @perrotta I just created PR #37156

jainshilpi added 4 commits February 2, 2022 11:01

MVA based EE photon tagger implementation

b143f4b

added missing parameter for PhotonProducer

ee342ca

Merged BHtaggerImplementation_12_3_0pre4 from repository jainshilpi w…

9317de2

…ith cms-merge-topic

Code format checks

fdcfde2

cmsbuild added this to the CMSSW_12_3_X milestone Feb 7, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Feb 7, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 7, 2022

cmsbuild added the requires-external label Feb 7, 2022

cmsbuild added tests-started tests-rejected and removed tests-pending tests-started labels Feb 7, 2022

removed unwanted lines

f7d5e90

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Feb 7, 2022

cmsbuild removed the code-checks-pending label Feb 7, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 2, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 2, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 3, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Mar 3, 2022

cmsbuild added orp-approved and removed orp-pending labels Mar 3, 2022

cmsbuild merged commit 24fa880 into cms-sw:master Mar 3, 2022

cmsbuild mentioned this pull request Mar 3, 2022

Add AlcaPCCIntegrator modules to the AlCa no ConcurrentLumis list #37130

Merged

francescobrivio reviewed Mar 3, 2022

View reviewed changes

qliphy mentioned this pull request Mar 4, 2022

Update tag for RecoEgamma-PhotonIdentification to V01-04-00 cms-sw/cmsdist#7665

Merged

mmusich mentioned this pull request Mar 6, 2022

BeamSpot Payload Inspector: introduce payload parameters comparator #37143

Merged

jainshilpi mentioned this pull request Mar 7, 2022

Fix for the access of haloMVATaggerVal in the Run 1+2 files #37156

Merged

This was referenced Jul 25, 2022

Adding the relevant variable for BDT based beam halo tagger for photons in nanoAOD #38850

Merged

Backport of #38850: Adding the relevant variable for BDT based beam halo tagger for photons in nanoAOD #38851

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of beam halo tagger algorithm for the endcap photons #36901

Implementation of beam halo tagger algorithm for the endcap photons #36901

jainshilpi commented Feb 7, 2022 •

edited

Loading

cmsbuild commented Feb 7, 2022

cmsbuild commented Feb 7, 2022

jpata commented Feb 7, 2022

jpata commented Feb 7, 2022

cmsbuild commented Feb 7, 2022

jpata commented Feb 7, 2022

cmsbuild commented Feb 7, 2022

cmsbuild commented Mar 2, 2022

perrotta commented Mar 2, 2022

cmsbuild commented Mar 3, 2022

jpata commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

perrotta commented Mar 3, 2022

francescobrivio Mar 3, 2022

qliphy Mar 4, 2022

perrotta commented Mar 6, 2022

jainshilpi commented Mar 7, 2022

perrotta commented Mar 7, 2022

jainshilpi commented Mar 7, 2022

Implementation of beam halo tagger algorithm for the endcap photons #36901

Implementation of beam halo tagger algorithm for the endcap photons #36901

Conversation

jainshilpi commented Feb 7, 2022 • edited Loading

cmsbuild commented Feb 7, 2022

cmsbuild commented Feb 7, 2022

jpata commented Feb 7, 2022

jpata commented Feb 7, 2022

cmsbuild commented Feb 7, 2022

Build

Clang Build

jpata commented Feb 7, 2022

cmsbuild commented Feb 7, 2022

cmsbuild commented Mar 2, 2022

perrotta commented Mar 2, 2022

cmsbuild commented Mar 3, 2022

Comparison Summary

jpata commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

perrotta commented Mar 3, 2022

francescobrivio Mar 3, 2022

Choose a reason for hiding this comment

qliphy Mar 4, 2022

Choose a reason for hiding this comment

perrotta commented Mar 6, 2022

jainshilpi commented Mar 7, 2022

perrotta commented Mar 7, 2022

jainshilpi commented Mar 7, 2022

jainshilpi commented Feb 7, 2022 •

edited

Loading