TDC simulation for HFQIE10 (12_1)

[mseidel42]

PR description:

• This PR adds TDC timing simulation for the HF --> enables the use of data noise filters in MC
  • Replaces 11_3 PR TDC simulation for QIE10 (HF) #33829
  • Now precise samples are stored in PreMixStage1 and converted back to usual DIGIs in PreMixStage2
• Original presentation in HCAL DPG: https://indico.cern.ch/event/1026044/contributions/4324030/attachments/2227968/3774461/qie10tdc_mseidel.pdf
  • Pulse shape deemed sufficient (no update)
  • Eta-dependence fixed by new HF shower library

PR validation:

Tested with workflows 11634.0 (Run3 ttbar), 11834.0 (Run3 ttbar+PU), 250202.181 (Run2 ttbar+premix PU), no "Mismatched calo signals" messages.

MC Run3 ttbar+PU:

MC Run2 ttbar+premix PU (time alignment for Run2 MC is a little off, keep frozen):

Run2 data:

@abdoulline @kpedro88 @tvami

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35308/25324

• This PR adds an extra 36KB to repository

[cmsbuild]

A new Pull Request was created by @mseidel42 (Markus Seidel) for master.

It involves the following packages:

• CalibFormats/CaloObjects (alca)
• SimCalorimetry/CaloSimAlgos (simulation)
• SimCalorimetry/HcalSimAlgos (simulation)
• SimCalorimetry/HcalSimProducers (simulation)

@malbouis, @civanch, @yuanchao, @mdhildreth, @cmsbuild, @francescobrivio, @tvami can you please review it and eventually sign? Thanks.
@rovere, @bsunanda, @tocheng, @thomreis, @simonepigazzini, @mmusich, @abdoulline, @mariadalfonso this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[kpedro88]

please test

[kpedro88]

Are the MC hits with TDC=31 expected? These don't seem to occur in data.

[mseidel42]

Are the MC hits with TDC=31 expected? These don't seem to occur in data.

Those are the special codes, just divided by 2 to compare with the data plot (ns instead of TDC).

cmssw/SimCalorimetry/HcalSimAlgos/interface/HcalTDCParameters.h

Lines 12 to 15 in a87b522

	int alreadyTransitionCode() const { return 62; }
	int noTransitionCode() const { return 63; }
	int unlockedCode() const { return 61; }
	int invalidCode() const { return 60; }

[tvami]

@cmsbuild , please abort

[tvami]

test parameters:

• workflows = 11634.0,11834.0,250202.181

[tvami]

@cmsbuild , please test

[kpedro88]

@tvami for future reference, workflows in the short matrix are always tested, so it's not necessary to specify them explicitly.

[tvami]

@tvami for future reference, workflows in the short matrix are always tested, so it's not necessary to specify them explicitly.

@kpedro88 I had a look at another PR to see what's included
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2d40d9/18638/runTheMatrix-results/
11634.0 and 250202.181 is indeed there, but it seems 11834.0 is not part of the short matrix

[kpedro88]

Easier to check

cmssw/Configuration/PyReleaseValidation/scripts/runTheMatrix.py

Lines 61 to 100 in 04e25fa

	'limited' : [5.1, #FastSim ttbar
	7.3, #CosmicsSPLoose_UP17
	8, #BH/Cosmic MC
	25, #MC ttbar
	4.22, #cosmic data
	4.53, #run1 data + miniAOD
	9.0, #Higgs200 charged taus
	1000, #data+prompt
	1001, #data+express
	101.0, #SingleElectron120E120EHCAL
	136.731, #2016B Photon data
	136.7611, #2016E JetHT reMINIAOD from 80X legacy
	136.8311, #2017F JetHT reMINIAOD from 94X reprocessing
	136.88811,#2018D JetHT reMINIAOD from UL processing
	136.793, #2017C DoubleEG
	136.874, #2018C EGamma
	140.53, #2011 HI data
	140.56, #2018 HI data
	158.01, #reMiniAOD of 2018 HI MC with pp-like reco
	312.0, #2021/Run3 HI MC Pyquen_ZeemumuJets_pt10 with pp-like reco
	1306.0, #SingleMu Pt1 UP15
	1325.81, #test NanoAOD from existing MINI UL 106Xv1
	136.8523, #test NanoAOD from existing reMINI UL 106Xv2
	1330, #Run2 MC Zmm
	135.4, #Run 2 Zee ttbar
	10042.0, #2017 ZMM
	10024.0, #2017 ttbar
	10224.0, #2017 ttbar PU
	10824.0, #2018 ttbar
	11634.911, #2021 DD4hep ttbar reading geometry from XML
	11634.912, #2021 DD4hep ttbar reading geometry from the DB
	11634.0, #2021 ttbar
	12434.0, #2023 ttbar
	23234.0, #2026D49 ttbar (HLT TDR baseline w/ HGCal v11)
	28234.0, #2026D60 (exercise HF nose)
	34634.0, #2026D76 ttbar (2021 new baseline)
	34834.999, #2026D76 ttbar premixing stage1+stage2, PU50
	25202.0, #2016 ttbar UP15 PU
	250202.181, #2018 ttbar stage1 + stage2 premix
	],

Indeed, maybe a Run3 PU workflow should be considered for regular testing.

[abdoulline]

@toropin @igv4321
Please, remind us - what is there for MC as HFPhase1PMTParams content?
https://twiki.cern.ch/twiki/bin/view/CMS/HFPhase1PMTParams

There is HFPhase1PMTParams_2017_v1.0_mc tag since 2016 in all the MC GTs
Should be something like just "pass through"?

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-29891e/18672/summary.html
COMMIT: a87b522
CMSSW: CMSSW_12_1_X_2021-09-16-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35308/18672/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 2572 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 3000833
• DQMHistoTests: Total failures: 4214
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2996597
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: found differences in 1 / 38 workflows

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35308/25448

• This PR adds an extra 16KB to repository

[cmsbuild]

Pull request #35308 was updated. @malbouis, @civanch, @yuanchao, @mdhildreth, @cmsbuild, @francescobrivio, @tvami can you please check and sign again.

[tvami]

@cmsbuild , please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-29891e/18839/summary.html
COMMIT: ccaab31
CMSSW: CMSSW_12_1_X_2021-09-21-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35308/18839/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 2780 differences found in the comparisons
• DQMHistoTests: Total files compared: 40
• DQMHistoTests: Total histograms compared: 3211080
• DQMHistoTests: Total failures: 4893
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3206165
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 39 files compared)
• Checked 169 log files, 37 edm output root files, 40 DQM output files
• TriggerResults: no differences found

[tvami]

+alca

• was signed earlier: TDC simulation for HFQIE10 (12_1) #35308 (comment)

[civanch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1