Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[12.6.X] Adjust collections of Tracker clusters to be saved for Phase-2 ALCARECO #40394

Merged

Conversation

mmusich
Copy link
Contributor

@mmusich mmusich commented Dec 22, 2022

PR description:

This is a follow-up of PR #39858.
After the backport of that PR (#39898) was integrated, it was possible to test the production of RelVal samples (see e.g. on DAS: /RelValMinBias_14TeV/CMSSW_12_5_2-TkAlMinBias-125X_mcRun4_realistic_v3_2026D88PU-v1/ALCARECO).
When trying to run over it, it became clear that is not possible to refit tracks, due to the missing event products (the Phase-2 Outer Tracker clusters) that are necessary for CPE re-computation, leading to this error:

Begin processing the 1st record. Run 1, Event 301, LumiSection 4 on stream 6 at 13-Dec-2022 12:50:01.219 CET
----- Begin Fatal Exception 13-Dec-2022 12:50:57 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 4 event: 306 stream: 2
   [1] Running path 'p2'
   [2] Calling method for module TrackRefitter/'FinalTrackRefitter'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'edmNew::DetSetVector<Phase2TrackerCluster1D>' with ProductID '4:449'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

This PR adds the minimal amount of changes in the CommonTools/RecoAlgos package in order to be able to retain the Phase-2 OT clusters after running the AlignmentTrackSelector module that relies on it:

// the following include is necessary to clone all track branches
// including recoTrackExtras and TrackingRecHitsOwned (in future also "owned clusters"?).
// if you remove it the code will compile, but the cloned
// tracks have only the recoTracks branch!
#include "CommonTools/RecoAlgos/interface/TrackSelector.h"

This is done in commit 35ef2e2.
Secondly the outputCommands of all the Tracker Alignment ALCARECO producers is modified in order to retain Phase2TrackerCluster1DedmNewDetSetVector in the output - in case the production is run for Phase-2 - while removing SiStripClusteredmNewDetSetVector (since there is no SiStrip detector in phase-2). I profit of that to re-adjust the commands for event contents to cope with no SCAL in Run3 that were leftover from #38415 and #38590.
This is done in commit 08f72fa.
Additionally this contains commit 474efd4 which fixes a problem with the vector hits workflow spotted in master and commit e78b930 which makes the ClusterStorer::rekey methods non-const as per recommendations in #40363.

PR validation:

Run two typical workflows for Run-3 and Phase-2 with

runTheMatrix.py -l 11634.0,20834.0 -t 4 -j 8 --ibeos

And tested the event content of TkAlMinBias indeed changes as expected.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is a combined backport of #40323 and #40363 (partially backported). It is backported in this cycle in order to use it for the production of ALCARECOs within the context of the L1T production happening in that cycle, see https://its.cern.ch/jira/browse/PDMVRELVALS-178 and McM.

@cmsbuild
Copy link
Contributor

cmsbuild commented Dec 22, 2022

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_12_6_X.

It involves the following packages:

  • Alignment/CommonAlignmentProducer (alca)
  • CommonTools/RecoAlgos (reconstruction)
  • DataFormats/TrackerRecHit2D (reconstruction)

@malbouis, @yuanchao, @ChrisMisan, @clacaputo, @cmsbuild, @saumyaphor4252, @tvami, @mandrenguyen, @francescobrivio can you please review it and eventually sign? Thanks.
@pakhotin, @rappoccio, @jdolen, @tlampen, @VourMa, @adewit, @ahinzmann, @abbiendi, @JanFSchulte, @rovere, @VinInn, @missirol, @gkasieczka, @tocheng, @hatakeyamak, @gpetruc, @mmusich, @mtosi, @jhgoh this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9cc942/29752/summary.html
COMMIT: e78b930
CMSSW: CMSSW_12_6_X_2022-12-21-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40394/29752/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3458651
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3458623
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 206 log files, 158 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@mandrenguyen
Copy link
Contributor

Just checking, you don't also want this commit?
fe0f886

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

Just checking, you don't also want this commit?

I thought to leave it out, as it's only needed for future development (which will happen in master)

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

backport of #40323

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

backport of #40363 (partial)

@mandrenguyen
Copy link
Contributor

+reconstruction

@tvami
Copy link
Contributor

tvami commented Dec 22, 2022

+alca

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

test parameters:

  • workflow = 20834.9, 20818.9
  • relvals_opt= -w upgrade

@perrotta
Copy link
Contributor

please test

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

@cmsbuild please abort

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

@perrotta

wf 20818.9 was not backported, see #40394 (comment)
please change the test parameters

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

test parameters:

  • workflow = 20834.9
  • relvals_opt= -w upgrade

@mmusich
Copy link
Contributor Author

mmusich commented Dec 22, 2022

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9cc942/29760/summary.html
COMMIT: e78b930
CMSSW: CMSSW_12_6_X_2022-12-22-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40394/29760/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3554525
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3554503
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 211 log files, 162 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@perrotta
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 0e938af into cms-sw:CMSSW_12_6_X Dec 23, 2022
@mmusich mmusich deleted the AlignmentTrackSelectorPhase2_12_6_X branch December 23, 2022 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants