[MkFit] Tracker geometry extraction from Reco geom modules

[osschar]

This PR addresses the first long-term item in #36966 - removal of hardcoded phasI geometry from mkFit and implementation of proper geometry extraction from RECO geometry loaded in current cmsRun process.

• Extract mkFit layer r-z bounding regions and TEC r-coverage gaps from individual modules.
• Introduce ModuleInfo with module position, normal and principal direction. Each layer holds a std::vector and a mapping from detId to short in-layer module index that was constructed on the fly beforehand.
• LayerInfo: remove/cleanup is_pixb/pixe/tib/tob/tid/tec data members and functions. They were not really used. Put in 'int subdet' that can be used to the same effect in scoring / cleaning functions.
• Remove geometry creation files with hardcoded geometry information: MkFit/plugins/createPhase1TrackerGeometry.h/cc/acc

Corresponding changes for standalone extras are on https://github.com/trackreco/mkFit-external/tree/geom-extract

PR validation:

• TTbar + < PU >=50: http://uaf-10.t2.ucsd.edu/~mmasciov/MIC/rescaleError100/MTV_TTbarPU50_extractedGeometry_mkFitOnly/
• 10μ, pT=0.6 GeV, 1.5<|η|<2.5: http://uaf-10.t2.ucsd.edu/~mmasciov/MIC/rescaleError100/MTV_10muPt0p6Eta1p5to2p5_extractedGeometry_mkFitOnly/

There are minimal changes in efficiency and fake rates, below 1% level. These are expected as the hardcoded mkFit geometry was not identical to one obtained by proper extraction. There are differences in extent of endcap r-gaps (O(1cm)) and z-extent of endcap mkFit layers (O(5cm)).

Timing is not expected to change significantly (if at all) as this PR only affects extents of mkFit layers.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37418/29094

• This PR adds an extra 168KB to repository

• Found files with invalid states:

  • RecoTracker/MkFit/plugins/DumpMkFitGeometry.cc:
    • Added: 50fe873
    • Modified: 3608fb7, 95af9ef, c72da21, 28d3348
    • Deleted: 8ddfaa7
  • RecoTracker/MkFit/python/dumpMkFitGeometry.py:
    • Added: 50fe873
    • Modified: 95af9ef, 28d3348
    • Deleted: 8ddfaa7

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37418/29094/code-checks.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37418/29094/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37418/29094/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37418/29094/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37418/29095

• This PR adds an extra 180KB to repository

• Found files with invalid states:

  • RecoTracker/MkFit/plugins/DumpMkFitGeometry.cc:
    • Added: 50fe873
    • Modified: 3608fb7, 95af9ef, c72da21, 28d3348
    • Deleted: 8ddfaa7
  • RecoTracker/MkFit/python/dumpMkFitGeometry.py:
    • Added: 50fe873
    • Modified: 95af9ef, 28d3348
    • Deleted: 8ddfaa7

[cmsbuild]

A new Pull Request was created by @osschar (Matevž Tadel) for master.

It involves the following packages:

• RecoTracker/MkFit (reconstruction)
• RecoTracker/MkFitCMS (reconstruction)
• RecoTracker/MkFitCore (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @ebrondol, @gpetruc, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[slava77]

@cmsbuild please test

[cmsbuild]

Pull request #37418 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

[slava77]

@cmsbuild please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-520b22/23613/summary.html
COMMIT: b5343ce
CMSSW: CMSSW_12_4_X_2022-04-01-2300/slc7_amd64_gcc10
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37418/23613/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 13988 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3593039
• DQMHistoTests: Total failures: 26541
• DQMHistoTests: Total nulls: 4
• DQMHistoTests: Total successes: 3566472
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.012 KiB( 47 files compared)
• DQMHistoSizes: changed ( 136.874 ): -0.004 KiB MessageLogger/Warnings
• DQMHistoSizes: changed ( 138.4 ): -0.004 KiB JetMET/SUSYDQM
• DQMHistoSizes: changed ( 138.5 ): -0.008 KiB JetMET/SUSYDQM
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 200 log files, 45 edm output root files, 48 DQM output files
• TriggerResults: found differences in 5 / 47 workflows

[slava77]

Additional Tests: PROFILING

CPU

• MkFitProducer on 20 events 7.02%/24.79 -> 6.94%/24.77 seems like a no-change
• ES producer calls are registered, but are near 0 both before and after

Memory

• mkFit geometry 575,420 -> 1,389,137 should correspond to more per-module data; although I didn't try to do the bean counting.

So, this looks somewhat as expected.

[osschar]

Yes, we have O(22k) modules with 40 bytes per module -> 22000*40 = 880000

[jpata]

+reconstruction

• improves geometry handling in mkFit
• many small changes to tracking and everything downstream, but according to the tracking validation in the PR description, no physics changes are expected
• CPU and memory do not look to be significantly affected in the igprof test

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[osschar]

Rats, I forgot include MkFit/test/BuildFile.xml for building of the standalone geometry dumper, MkFit/test/DumpMkFitGeometry.cc.

Shall I put it in here or I wait until the next PR so we don't need to go through test etc. hooplas again? The file is rather trivial:

matevz@phi3 RecoTracker> cat MkFit/test/BuildFile.xml
<library file="DumpMkFitGeometry.cc" name="RecoTrackerMkFitTest">
  <flags EDM_PLUGIN="1"/>
  <use name="FWCore/Framework"/>
  <use name="RecoTracker/MkFit"/>
  <use name="RecoTracker/MkFitCore"/>
  <use name="RecoTracker/Record"/>
</library>

[qliphy]

@osschar You may add it within this PR, and can also take the chance to implement
https://github.com/cms-sw/cmssw/pull/37418/files#r839654493

[osschar]

Naaah, just go ahead and merge, I'll fix that print in another PR as well, it's almost ready -- the second long term item from our todo list, removal of Ice/ and consolidation of radix sort usage.

[qliphy]

+1
to follow up on #37418 (comment)

[osschar]

Thank you! :)

[mmusich]

For the record #37418 (comment) has been fixed at #37586