failure of HLT-Validation tests in reading edm::HepMCProduct
#37392
Comments
|
A new Issue was created by @missirol Marino Missiroli. @Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks. cms-bot commands are listed here |
|
assign hlt, generators |
|
FYI @mkirsano |
|
New categories assigned: hlt,generators @mkirsano,@missirol,@alberto-sanchez,@SiewYan,@GurpreetSinghChahal,@Saptaparna,@Martin-Grunewald you have been requested to review this Pull request/Issue and eventually sign? Thanks |
cmsbuild
added
generators-pending
hlt-pending
pending-signatures
and removed
pending-assignment
labels
|
@missirol Could you give a recipe to reproduce? (this test is not part of addOnTests, right?) |
Here it goes: cmsrel CMSSW_12_4_X_2022-03-28-1100
cd CMSSW_12_4_X_2022-03-28-1100/src
cmsenv
cmsDriver.py RelValNEW --step=DIGI:pdigi,L1,DIGI2RAW --conditions=auto:run3_mc_GRun \
--custom_conditions= --fileout=RelVal_DigiL1Raw_GRun_MC.root --number=100 --mc --datatier GEN-SIM-DIGI-RAW --eventcontent=RAWSIM \
--customise=HLTrigger/Configuration/CustomConfigs.L1T --era=Run3 --customise= --scenario=pp --python_filename=RelValNEW_DigiL1Raw_GRun_MC.py \
--filein=root://eoscms.cern.ch//eos/cms/store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STORM/GEN-SIM/CMSSW_12/29297788-21c3-4bf2-afe0-5382f08fbcd5.root \
&> RelValNEW.log
cmsDriver.py RelValOLD --step=DIGI:pdigi,L1,DIGI2RAW --conditions=auto:run3_mc_GRun \
--custom_conditions= --fileout=RelVal_DigiL1Raw_GRun_MC.root --number=100 --mc --datatier GEN-SIM-DIGI-RAW --eventcontent=RAWSIM \
--customise=HLTrigger/Configuration/CustomConfigs.L1T --era=Run3 --customise= --scenario=pp --python_filename=RelValOLD_DigiL1Raw_GRun_MC.py \
--filein=root://eoscms.cern.ch//eos/cms/store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STORM/GEN-SIM/CMSSW_11/3ee9ba1e-0ef8-4242-8343-cff886c9f7b3.root \
&> RelValOLD.log
Indeed, it is not. It is part of additional tests designed by HLT which run as "HLT-Validation" tests in IB [*]. Afaiu, the HLT-Validation tests run in "DEFAULT" IBs, but not in other IB flavours (e.g. DEVEL IBs). cms-sw/cmsdist#7724 looks like a suspect because the issue showed up as soon as that cmsdist update made it into default IBs; it was already in DEVEL IBs, but it seems we don't run HLT-Validation tests for those (I'm not saying we should). [*] For reference, those tests can be run locally with cd HLTrigger/Configuration/test
./runAll.csh IB |
|
correct, the HLT tests are only run for default IBs and are not part of PR tests. @mkirsano any idea? |
|
assign core |
|
New categories assigned: core @Dr15Jones,@smuzaffar,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks |
|
Thanks @missirol, I was able to reproduce. Here is a stack trace of the exception |
|
@pcanal it looks like we have hit a problem reading back a file we could read before. |
|
@makortel did we change the compiler recently? |
|
No, compiler was not changed. |
|
Adding some debug symbols to the first stack frames |
|
I have opened a revert of pythia8 change cms-sw/cmsdist#7731 and will try to run the test mentioned here |
|
It may be related to the improvements/fixes I am currently making to the support for |
|
So |
|
It looks like it is not related to the We also had updated root cms-sw/cmsdist#7722 for this IB where we saw this issue. We updated root to the tip of v6.24.patches branch to fix the issue reported #37227. @pcanal root-project/root@292e441...4b08829 are the root changes which were included as compared to the previous version. |
|
unassign hlt,generators |
|
@smuzaffar root-project/root#10230 might/should help. |
|
thanks @pcanal , is there a v6-24-patches version of it? |
|
Not yet. I am still in the process of validating but let me see if it can applied cleanly as is. |
|
You can try: root-project/root#10280 |
|
many thanks @pcanal , I have started the tests cms-sw/root#167 . Once the cmssw dev area is available then I will run the HLT too |
|
@pcanal , root-project/root#10280 looks good, PR tests passed cms-sw/root#167 (comment) and also the additonal HLT tests mentioned #37392 (comment) worked too |
|
The patches have been merged in v6.24, v6.26 and master branches. |
|
thanks @pcanal , I have open a PR cms-sw/cmsdist#7762 to include root v6-24-00-patches changes in to cmssw |
|
@missirol The HLT validation test is working fine now, right? |
|
Yes, ever since we by-passed the issue by updating the input files in #37393. I didn't try to test with the old input file and the patch discussed lately here, but I understood Shahzad did so in #37392 (comment) . Does this answer the question, Matti? (I was waiting for signatures before closing the issue.) |
|
+1 Thanks @missirol |
cmsbuild
added
core-approved
fully-signed
and removed
core-pending
pending-signatures
labels
|
This issue is fully signed and ready to be closed. |
|
Closing. Thanks to experts for the follow-ups. |
The HLT-Validation tests failed in
CMSSW_12_4_X_2022-03-28-1100, with the error in [1], e.g.https://cmssdt.cern.ch/SDT/jenkins-artifacts/HLT-Validation/CMSSW_12_4_X_2022-03-28-1100/slc7_amd64_gcc10/RelVal_DigiL1Raw_GRun_MC.log
CMSSW_12_4_X_2022-03-28-1100, and the same job completes correctly forCMSSW_12_4_X_2022-03-27-2300(as per IB tests).cmsdisthaving to do withpythia8andHepMC(but I don't really know that this is related to this issue): configure pythia8 to use hepmc3 and hepmc2 cmsdist#7724 .It's worth noting that the input EDM file used in the failing HLT test is relatively old (GEN-SIM from 11_2_0_pre8).
A quick test suggests that using a more recent GEN-SIM file (from 12_3_0_pre6) fixes the problem.
I'm opening this issue for documentation, and to get insight from others on what the issue actually is here.
[1]
The text was updated successfully, but these errors were encountered: