New Heterogeneous Memory Pool #37952

VinInn · 2022-05-15T13:57:34Z

This PR replaces the old "notcub" cache allocator with a memory pool featuring

lockfree operations
backend agnostic implementation
The data interface is based on a simple Buffer that is completely backend agnostic
The allocation interface (makeBuffer) currently depends on cudaStream_t that can be easily hidden behind void * or a light opaque struct
A new feature is a "Bundle deleter": buffers can be bundle together and then freed in just one operation: this reduces the number of cuda calls.
All previous users of the cache allocator (at least for Pixel wf) have been migrated.

Tests passes: it is not slower than previous implementation. Need a free machine to make definitive tests.

Some cleanup is still required to remove debug statements.

Purely technical no regression expected.

Draft Slides for a possible presentation available @ https://cernbox.cern.ch/index.php/s/Ax4NHYGLHbG8N1C

cmsbuild · 2022-05-15T14:03:58Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37952/30020

This PR adds an extra 232KB to repository
Found files with invalid states:
- HeterogeneousCore/CUDAUtilities/src/cudaMemoryPool.cu:
  - Added: f37385a
  - Modified: 75ca0db, c429d13, c5e35f0
  - Deleted: 21a646e
- CUDADataFormats/TrackingRecHit/interface/TrackingRecHit2DHeterogeneousImpl.h:
  - Added: 5291489
  - Modified: 1a43ba7, e7d8632, c8d553a
  - Deleted: 521d4c0
- HeterogeneousCore/CUDAUtilities/interface/cudaMemoryPoolImpl.h:
  - Added: 5291489
  - Modified: 1a43ba7, 59bcb2b, 29df6e2, 849da8c, e7d8632, b4f4d46, 8b149ed
  - Deleted: 1487b88
- CUDADataFormats/SiPixelDigi/interface/SiPixelDigisCUDAImpl.h:
  - Added: 3ae45f7
  - Modified: b4f4d46, c8d553a
  - Deleted: 9402cb7
- CUDADataFormats/TrackingRecHit/src/TrackingRecHit2DHeterogeneous.cc:
  - Modified: 6b050bd, 0e49a36, e8e9c0f, 88da3bc, b4f4d46, 521d4c0
  - Deleted: 21a646e
  - Added: e7d8632
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/CUDAServices/src/CUDAService.cc modified in PR(s): Implement ResourceInformationService #37831
- File HeterogeneousCore/CUDAUtilities/test/BuildFile.xml modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoLocalTracker/SiPixelRecHits/plugins/PixelRecHitGPUKernel.cu modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoLocalTracker/SiPixelRecHits/plugins/SiPixelRecHitSoAFromLegacy.cc modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cc modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cu modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.h modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernelsAlloc.cc modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713

cmsbuild · 2022-05-15T14:04:21Z

A new Pull Request was created by @VinInn (Vincenzo Innocente) for master.

It involves the following packages:

CUDADataFormats/BeamSpot (heterogeneous, reconstruction)
CUDADataFormats/Common (heterogeneous)
CUDADataFormats/SiPixelDigi (heterogeneous, reconstruction)
CUDADataFormats/Track (heterogeneous, reconstruction)
CUDADataFormats/TrackingRecHit (heterogeneous, reconstruction)
CUDADataFormats/Vertex (heterogeneous, reconstruction)
EventFilter/SiPixelRawToDigi (reconstruction)
HeterogeneousCore/CUDACore (heterogeneous)
HeterogeneousCore/CUDAServices (heterogeneous)
HeterogeneousCore/CUDAUtilities (heterogeneous)
RecoLocalTracker/SiPixelRecHits (reconstruction)
RecoPixelVertexing/PixelTrackFitting (reconstruction)
RecoPixelVertexing/PixelTriplets (reconstruction)
RecoPixelVertexing/PixelVertexFinding (reconstruction)
RecoVertex/BeamSpotProducer (reconstruction, alca)

@malbouis, @yuanchao, @makortel, @slava77, @clacaputo, @cmsbuild, @fwyzard, @jpata, @tvami, @francescobrivio can you please review it and eventually sign? Thanks.
@tvami, @makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @Martin-Grunewald, @missirol, @OzAmram, @tocheng, @ferencek, @mtosi, @gpetruc, @mmusich, @dkotlins, @threus, @dgulhan, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

VinInn · 2022-05-15T15:47:35Z

@cmsbuild , please test

VinInn · 2022-05-15T15:47:40Z

enable gpu

cmsbuild · 2022-05-15T20:02:21Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-651b42/24728/summary.html
COMMIT: b8d0837
CMSSW: CMSSW_12_4_X_2022-05-15-0000/slc7_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37952/24728/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test cpuVertexFinderByDensity_t had ERRORS
---> test cpuVertexFinderIterative_t had ERRORS

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19874
DQMHistoTests: Total failures: 1171
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 18702
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: found differences in 3 / 3 workflows

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-651b42/11634.301_TTbar_14TeV+2021_Run3FS+TTbar_14TeV_TuneCP5_GenSim+HARVESTNano

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3741432
DQMHistoTests: Total failures: 92
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3741318
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

VinInn · 2022-05-16T06:55:30Z

@cmsbuild , please test

cmsbuild · 2022-05-16T07:02:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37952/30028

This PR adds an extra 236KB to repository
Found files with invalid states:
- HeterogeneousCore/CUDAUtilities/src/cudaMemoryPool.cu:
  - Added: f37385a
  - Modified: 75ca0db, c429d13, c5e35f0
  - Deleted: 21a646e
- CUDADataFormats/TrackingRecHit/interface/TrackingRecHit2DHeterogeneousImpl.h:
  - Added: 5291489
  - Modified: 1a43ba7, e7d8632, c8d553a
  - Deleted: 521d4c0
- HeterogeneousCore/CUDAUtilities/interface/cudaMemoryPoolImpl.h:
  - Added: 5291489
  - Modified: 1a43ba7, 59bcb2b, 29df6e2, 849da8c, e7d8632, b4f4d46, 8b149ed
  - Deleted: 1487b88
- CUDADataFormats/SiPixelDigi/interface/SiPixelDigisCUDAImpl.h:
  - Added: 3ae45f7
  - Modified: b4f4d46, c8d553a
  - Deleted: 9402cb7
- CUDADataFormats/TrackingRecHit/src/TrackingRecHit2DHeterogeneous.cc:
  - Modified: 6b050bd, 0e49a36, e8e9c0f, 88da3bc, b4f4d46, 521d4c0
  - Deleted: 21a646e
  - Added: e7d8632
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/CUDAServices/src/CUDAService.cc modified in PR(s): Implement ResourceInformationService #37831
- File HeterogeneousCore/CUDAUtilities/test/BuildFile.xml modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoLocalTracker/SiPixelRecHits/plugins/PixelRecHitGPUKernel.cu modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoLocalTracker/SiPixelRecHits/plugins/SiPixelRecHitSoAFromLegacy.cc modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cc modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cu modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.h modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernelsAlloc.cc modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713

mandrenguyen · 2023-12-06T06:49:41Z

-reconstruction
Cleaning reco queue, feel free to keep it open of course.

cmsbuild · 2024-02-06T10:09:44Z

Milestone for this pull request has been moved to CMSSW_14_1_X. Please open a backport if it should also go in to CMSSW_14_0_X.

smuzaffar · 2024-02-12T20:08:20Z

ping

cmsbuild · 2024-08-27T08:08:44Z

Milestone for this pull request has been moved to CMSSW_14_2_X. Please open a backport if it should also go in to CMSSW_14_1_X.

antoniovilela · 2024-09-03T09:44:05Z

ping (to make bot change milestone)

cmsbuild added this to the CMSSW_12_4_X milestone May 15, 2022

cmsbuild added alca-pending code-checks-pending heterogeneous-pending orp-pending pending-signatures reconstruction-pending tests-pending labels May 15, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 15, 2022

cmsbuild added tests-started and removed tests-pending labels May 15, 2022

cmsbuild mentioned this pull request May 15, 2022

BTV DQM Updates #37832

Merged

cmsbuild added tests-rejected code-checks-pending tests-pending and removed tests-started tests-rejected code-checks-approved labels May 15, 2022

cmsbuild added tests-started and removed tests-pending labels May 16, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 16, 2022

cmsbuild mentioned this pull request Nov 15, 2023

Pixel Alpaka Migration: Configs and Fixes [VII] #43294

Merged

cmsbuild modified the milestones: CMSSW_14_0_X, CMSSW_14_1_X Feb 6, 2024

cmsbuild modified the milestones: CMSSW_14_0_X, CMSSW_14_1_X Feb 12, 2024

cmsbuild mentioned this pull request Feb 14, 2024

Alpaka vs CUDA DQM compare modules for pixel tracks objects #43964

Closed

cmsbuild mentioned this pull request May 3, 2024

Introduce edm::Async service, and use it in CUDA and Alpaka modules #44901

Merged

cmsbuild mentioned this pull request Jun 7, 2024

Include HIon type traits for Alpaka pixel tracking #45151

Merged

cmsbuild mentioned this pull request Jul 25, 2024

Mark ScopedContextAcquire destructor as noexcept(false) #45560

Merged

cmsbuild modified the milestones: CMSSW_14_1_X, CMSSW_14_2_X Aug 27, 2024

cmsbuild added the changes-dataformats label Aug 27, 2024

cmsbuild mentioned this pull request Sep 1, 2024

Remove legacy CUDA modules for pixel track and vertex reconstruction #45853

Draft

cmsbuild modified the milestones: CMSSW_14_1_X, CMSSW_14_2_X Sep 3, 2024

cmsbuild mentioned this pull request Sep 20, 2024

Remove the configuration of the legacy CUDA workflows #46076

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Heterogeneous Memory Pool #37952

New Heterogeneous Memory Pool #37952

VinInn commented May 15, 2022 •

edited

Loading

cmsbuild commented May 15, 2022

cmsbuild commented May 15, 2022 •

edited

Loading

VinInn commented May 15, 2022

VinInn commented May 15, 2022

cmsbuild commented May 15, 2022

VinInn commented May 16, 2022

cmsbuild commented May 16, 2022

mandrenguyen commented Dec 6, 2023

cmsbuild commented Feb 6, 2024

smuzaffar commented Feb 12, 2024

cmsbuild commented Aug 27, 2024

antoniovilela commented Sep 3, 2024

New Heterogeneous Memory Pool #37952

Are you sure you want to change the base?

New Heterogeneous Memory Pool #37952

Conversation

VinInn commented May 15, 2022 • edited Loading

cmsbuild commented May 15, 2022

cmsbuild commented May 15, 2022 • edited Loading

VinInn commented May 15, 2022

VinInn commented May 15, 2022

cmsbuild commented May 15, 2022

Unit Tests

GPU Comparison Summary

Comparison Summary

VinInn commented May 16, 2022

cmsbuild commented May 16, 2022

mandrenguyen commented Dec 6, 2023

cmsbuild commented Feb 6, 2024

smuzaffar commented Feb 12, 2024

cmsbuild commented Aug 27, 2024

antoniovilela commented Sep 3, 2024

VinInn commented May 15, 2022 •

edited

Loading

cmsbuild commented May 15, 2022 •

edited

Loading