[MkFit] round up size arguments to aligned_alloc upwards to align value. #37111

osschar · 2022-03-01T19:34:32Z

std::aligned_alloc calls were passing the desired allocated size, not rounded up to a multiple of align argument which seems to result in UB according to the standard.

All allocations in mkFit are aligned to cache-line (64) required for vector intrinsics.

This PR addresses #37096.

cmsbuild · 2022-03-01T19:42:37Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37111/28609

This PR adds an extra 20KB to repository

cmsbuild · 2022-03-01T19:43:01Z

A new Pull Request was created by @osschar (Matevž Tadel) for master.

It involves the following packages:

RecoTracker/MkFitCore (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @ebrondol, @gpetruc, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

RecoTracker/MkFitCore/src/HitStructures.cc

Co-authored-by: Slava Krutelyov <[email protected]>

cmsbuild · 2022-03-01T20:12:53Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37111/28611

This PR adds an extra 20KB to repository

cmsbuild · 2022-03-01T20:13:17Z

Pull request #37111 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

cmsbuild · 2022-03-01T20:52:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37111/28613

This PR adds an extra 24KB to repository

cmsbuild · 2022-03-01T20:53:15Z

Pull request #37111 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

makortel · 2022-03-01T21:11:33Z

@cmsbuild, please test for CMSSW_12_3_ASAN_X

smuzaffar · 2022-03-01T21:22:37Z

please test for slc7_aarch64_gcc11

lets tests the change RecoTracker/MkFitCore/src/Matriplex/MatriplexCommon.h for non-x86_64 archs too

cmsbuild · 2022-03-01T22:46:07Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3d658/22753/summary.html
COMMIT: 4c2f7f8
CMSSW: CMSSW_12_3_X_2022-02-28-2300/slc7_aarch64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37111/22753/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

4.53

----- Begin Fatal Exception 01-Mar-2022 22:47:41 CET-----------------------
An exception of category 'Vertex' occurred while
   [0] Processing  Event run: 194533 lumi: 329 event: 462355458 stream: 0
   [1] Running path 'dqmofflineOnPAT_1_step'
   [2] Prefetching for module SingleTopTChannelLeptonDQM_miniAOD/'singleTopElectronMediumDQM_miniAOD'
   [3] Prefetching for module PATMuonSlimmer/'slimmedMuons'
   [4] Prefetching for module PATMuonSelector/'selectedPatMuons'
   [5] Prefetching for module PATMuonProducer/'patMuons'
   [6] Prefetching for module MuonProducer/'muons'
   [7] Prefetching for module PFProducer/'particleFlowTmp'
   [8] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [9] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [10] Prefetching for module PFConversionProducer/'pfConversions'
   [11] Calling method for module ConversionProducer/'allConversions'
Exception Message:
Refitted track not found in list
----- End Fatal Exception -------------------------------------------------

140.56

----- Begin Fatal Exception 01-Mar-2022 22:56:39 CET-----------------------
An exception of category 'Vertex' occurred while
   [0] Processing  Event run: 326479 lumi: 7 event: 1579493 stream: 0
   [1] Running path 'dqmoffline_8_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module PFProducer/'particleFlowTmp'
   [5] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [6] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [7] Prefetching for module PFConversionProducer/'pfConversions'
   [8] Calling method for module ConversionProducer/'allConversions'
Exception Message:
Refitted track not found in list
----- End Fatal Exception -------------------------------------------------

136.874

----- Begin Fatal Exception 01-Mar-2022 23:10:54 CET-----------------------
An exception of category 'Vertex' occurred while
   [0] Processing  Event run: 319450 lumi: 76 event: 106007323 stream: 0
   [1] Running path 'dqmoffline_10_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module PFProducer/'particleFlowTmp'
   [5] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [6] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [7] Prefetching for module PFConversionProducer/'pfConversions'
   [8] Calling method for module ConversionProducer/'allConversions'
Exception Message:
Refitted track not found in list
----- End Fatal Exception -------------------------------------------------

slava77 · 2022-03-01T23:20:39Z

Failed Tests: RelVals
CMSSW: CMSSW_12_3_X_2022-02-28-2300/slc7_aarch64_gcc11

I suppose, as in the previous update in #37014 these workflows are failing also in the aarch IB.
The fact that the code compiled was the main relevant part of the test.

cmsbuild · 2022-03-01T23:48:10Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3d658/22752/summary.html
COMMIT: 4c2f7f8
CMSSW: CMSSW_12_3_ASAN_X_2022-02-28-1100/slc7_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37111/22752/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

39496.039496.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal.log
39500.039500.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal.log
23234.023234.0_TTbar_14TeV+2026D49+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_TTbar_14TeV+2026D49+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal.log

Expand to see more relval errors ...

slava77 · 2022-03-02T00:00:18Z

Failed Tests: RelVals
CMSSW: CMSSW_12_3_ASAN_X_2022-02-28-1100/slc7_amd64_gcc11

it seems like only phase-2 (non-mkFit) workflows failed. Apparently the ones with mkFit completed OK.

makortel · 2022-03-02T02:09:00Z

Failed Tests: RelVals
CMSSW: CMSSW_12_3_ASAN_X_2022-02-28-1100/slc7_amd64_gcc11

it seems like only phase-2 (non-mkFit) workflows failed. Apparently the ones with mkFit completed OK.

I agree, and the phase2 workflow failures in ASAN occur also in IBs

makortel · 2022-03-02T02:09:15Z

@cmsbuild, please test

(regular tests too)

cmsbuild · 2022-03-02T06:20:40Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3d658/22754/summary.html
COMMIT: 4c2f7f8
CMSSW: CMSSW_12_3_X_2022-03-01-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37111/22754/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 4000857
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 4000821
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 48 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

clacaputo · 2022-03-02T10:30:44Z

+reconstruction

technical PR that addresses ASAN failure in mkFit [ASAN] Failure in mkfit #37096
wf with mkFit run properly in ASAN build (see [MkFit] round up size arguments to aligned_alloc upwards to align value. #37111 (comment) and [MkFit] round up size arguments to aligned_alloc upwards to align value. #37111 (comment))

cmsbuild · 2022-03-02T10:31:06Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-03-02T12:23:06Z

+1

[MkFit] round up size arguments to aligned_alloc upwards to align value.

699c85f

cmsbuild added this to the CMSSW_12_3_X milestone Mar 1, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 1, 2022

code-format

abecaed

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 1, 2022

slava77 reviewed Mar 1, 2022

View reviewed changes

RecoTracker/MkFitCore/src/HitStructures.cc Outdated Show resolved Hide resolved

fix include to localpath RecoTracker/MkFitCore/src/HitStructures.cc

4b36615

Co-authored-by: Slava Krutelyov <[email protected]>

cmsbuild added code-checks-pending and removed code-checks-approved labels Mar 1, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 1, 2022

fix include to localpath RecoTracker/MkFitCore/src/Pool.h

4c2f7f8

cmsbuild added code-checks-pending and removed code-checks-approved labels Mar 1, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 1, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 1, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 1, 2022

cmsbuild added tests-started and removed tests-rejected labels Mar 2, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 2, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Mar 2, 2022

cmsbuild added orp-approved and removed orp-pending labels Mar 2, 2022

cmsbuild merged commit 0ee444a into cms-sw:master Mar 2, 2022

slava77 mentioned this pull request Mar 3, 2022

review the patch for arm/power in MatriplexCommon.h trackreco/cmssw#81

Closed

perrotta mentioned this pull request Mar 8, 2022

[ASAN] Failure in mkfit #37096

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MkFit] round up size arguments to aligned_alloc upwards to align value. #37111

[MkFit] round up size arguments to aligned_alloc upwards to align value. #37111

osschar commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

makortel commented Mar 1, 2022

smuzaffar commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

slava77 commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

slava77 commented Mar 2, 2022

makortel commented Mar 2, 2022

makortel commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

clacaputo commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

perrotta commented Mar 2, 2022

[MkFit] round up size arguments to aligned_alloc upwards to align value. #37111

[MkFit] round up size arguments to aligned_alloc upwards to align value. #37111

Conversation

osschar commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

makortel commented Mar 1, 2022

smuzaffar commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

RelVals

slava77 commented Mar 1, 2022

cmsbuild commented Mar 1, 2022

RelVals

slava77 commented Mar 2, 2022

makortel commented Mar 2, 2022

makortel commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

Comparison Summary

clacaputo commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

perrotta commented Mar 2, 2022