Add an option to cmsDriver for selecting the accelerators to use

[fwyzard]

PR description:

Add an option to cmsDriver for passing a comma-separated list of accelerators to enable. If none of those accelerators is available, and 'cpu' is not part of the list, the job will fail.
The default value is not set, which enables all available accelerators, including the cpu fallback.

PR validation:

Tested that cmsDriver.py ... --accelerator gpu-nvidia behaves as expected.

[fwyzard]

assign heterogeneous

[fwyzard]

@makortel does this seem the right approach to you ?

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37128/28657

• This PR adds an extra 40KB to repository

[cmsbuild]

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

• Configuration/Applications (operations)

@perrotta, @makortel, @cmsbuild, @fwyzard, @qliphy, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @missirol, @fabiocos, @Martin-Grunewald this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[fwyzard]

please test

[makortel]

does this seem the right approach to you ?

Looks reasonable, thanks.

Another thing I have been thinking about (but didn't have the moment to get to) was to make the runTheMatrix.py --gpu/--requires-gpu to set the process.options.accelerators (especially required leading to disabling the CPU fallback). But that's probably better to leave for subsequent development.

[fwyzard]

Another thing I have been thinking about (but didn't have the moment to get to) was to make the runTheMatrix.py --gpu/--requires-gpu to set the process.options.accelerators (especially required leading to disabling the CPU fallback). But that's probably better to leave for subsequent development.

Eh... I admit that I am a bit confused by the various "options", their intended behaviour, and their interplay.

[fwyzard]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37128/28663

• This PR adds an extra 36KB to repository

[cmsbuild]

Pull request #37128 was updated. @perrotta, @makortel, @fwyzard, @qliphy, @fabiocos, @davidlange6 can you please check and sign again.

[fwyzard]

+heterogeneous

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-986aef/22808/summary.html
COMMIT: 2030726
CMSSW: CMSSW_12_3_X_2022-03-03-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37128/22808/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3987741
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3987717
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[perrotta]

+operations

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1