Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failure of HLT-Validation tests in reading edm::HepMCProduct #37392

Closed
missirol opened this issue Mar 28, 2022 · 32 comments
Closed

failure of HLT-Validation tests in reading edm::HepMCProduct #37392

missirol opened this issue Mar 28, 2022 · 32 comments

Comments

@missirol
Copy link
Contributor

missirol commented Mar 28, 2022

The HLT-Validation tests failed in CMSSW_12_4_X_2022-03-28-1100, with the error in [1], e.g.
https://cmssdt.cern.ch/SDT/jenkins-artifacts/HLT-Validation/CMSSW_12_4_X_2022-03-28-1100/slc7_amd64_gcc10/RelVal_DigiL1Raw_GRun_MC.log

  • The same tests passed in the previous 12_4_X IB, and they are passing in earlier releases (e.g. 12_3_X).
  • Locally, the error can be reproduced in CMSSW_12_4_X_2022-03-28-1100, and the same job completes correctly for CMSSW_12_4_X_2022-03-27-2300 (as per IB tests).
  • I don't see any change in CMSSW itself that can explain it.
  • The only update that looks related to the error message is one in cmsdist having to do with pythia8 and HepMC (but I don't really know that this is related to this issue): configure pythia8 to use hepmc3 and hepmc2 cmsdist#7724 .

It's worth noting that the input EDM file used in the failing HLT test is relatively old (GEN-SIM from 11_2_0_pre8).

A quick test suggests that using a more recent GEN-SIM file (from 12_3_0_pre6) fixes the problem.

I'm opening this issue for documentation, and to get insight from others on what the issue actually is here.

[1]

Begin processing the 1st record. Run 1, Event 55, LumiSection 2 on stream 0 at 28-Mar-2022 23:32:50.956 CEST                                                                                               
----- Begin Fatal Exception 28-Mar-2022 23:32:57 CEST-----------------------                                                                                                                               
An exception of category 'FileReadError' occurred while                                                                                                                                                    
   [0] Processing  Event run: 1 lumi: 2 event: 55 stream: 0                                                                                                                                                
   [1] Running path 'RAWSIMoutput_step'                                                                                                                                                                    
   [2] Prefetching for module PoolOutputModule/'RAWSIMoutput'                                                                                                                                              
   [3] While reading from source edm::HepMCProduct generatorSmeared '' SIM                                                                                                                                 
   [4] Reading branch edmHepMCProduct_generatorSmeared__SIM.                                                                                                                                               
Exception Message:                                                                                                                                                                                         
basic_string::_M_construct null not valid                                                                                                                                                                  
----- End Fatal Exception -------------------------------------------------                                                                                                                                
@cmsbuild
Copy link
Contributor

A new Issue was created by @missirol Marino Missiroli.

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@makortel
Copy link
Contributor

assign hlt, generators

@makortel
Copy link
Contributor

FYI @mkirsano

@cmsbuild
Copy link
Contributor

New categories assigned: hlt,generators

@mkirsano,@missirol,@alberto-sanchez,@SiewYan,@GurpreetSinghChahal,@Saptaparna,@Martin-Grunewald you have been requested to review this Pull request/Issue and eventually sign? Thanks

@makortel
Copy link
Contributor

@missirol Could you give a recipe to reproduce? (this test is not part of addOnTests, right?)

@missirol
Copy link
Contributor Author

Could you give a recipe to reproduce?

Here it goes:

cmsrel CMSSW_12_4_X_2022-03-28-1100
cd CMSSW_12_4_X_2022-03-28-1100/src
cmsenv

cmsDriver.py RelValNEW --step=DIGI:pdigi,L1,DIGI2RAW --conditions=auto:run3_mc_GRun \
--custom_conditions= --fileout=RelVal_DigiL1Raw_GRun_MC.root --number=100 --mc --datatier GEN-SIM-DIGI-RAW --eventcontent=RAWSIM \
--customise=HLTrigger/Configuration/CustomConfigs.L1T --era=Run3 --customise= --scenario=pp --python_filename=RelValNEW_DigiL1Raw_GRun_MC.py \
--filein=root://eoscms.cern.ch//eos/cms/store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STORM/GEN-SIM/CMSSW_12/29297788-21c3-4bf2-afe0-5382f08fbcd5.root \
&> RelValNEW.log

cmsDriver.py RelValOLD --step=DIGI:pdigi,L1,DIGI2RAW --conditions=auto:run3_mc_GRun \
--custom_conditions= --fileout=RelVal_DigiL1Raw_GRun_MC.root --number=100 --mc --datatier GEN-SIM-DIGI-RAW --eventcontent=RAWSIM \
--customise=HLTrigger/Configuration/CustomConfigs.L1T --era=Run3 --customise= --scenario=pp --python_filename=RelValOLD_DigiL1Raw_GRun_MC.py \
--filein=root://eoscms.cern.ch//eos/cms/store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STORM/GEN-SIM/CMSSW_11/3ee9ba1e-0ef8-4242-8343-cff886c9f7b3.root \
&> RelValOLD.log

RelValOLD will fail, while RelValNEW will not. The only difference between the two is the input file (and HLT does not actually run here, see the steps).


(this test is not part of addOnTests, right?)

Indeed, it is not. It is part of additional tests designed by HLT which run as "HLT-Validation" tests in IB [*]. Afaiu, the HLT-Validation tests run in "DEFAULT" IBs, but not in other IB flavours (e.g. DEVEL IBs). cms-sw/cmsdist#7724 looks like a suspect because the issue showed up as soon as that cmsdist update made it into default IBs; it was already in DEVEL IBs, but it seems we don't run HLT-Validation tests for those (I'm not saying we should).

[*] For reference, those tests can be run locally with

cd HLTrigger/Configuration/test
./runAll.csh IB

@smuzaffar
Copy link
Contributor

correct, the HLT tests are only run for default IBs and are not part of PR tests. @mkirsano any idea?

@Dr15Jones
Copy link
Contributor

assign core

@cmsbuild
Copy link
Contributor

New categories assigned: core

@Dr15Jones,@smuzaffar,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

@makortel
Copy link
Contributor

Thanks @missirol, I was able to reproduce. Here is a stack trace of the exception

#0  0x00007ffff5b6dc9e in __cxxabiv1::__cxa_throw (obj=obj@entry=0x7fff803309e0, tinfo=0x7ffff5c881c8 <typeinfo for std::84898dea653199466402e67d73657f10/gcc-10.3.0/obj/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:212
#3  0x00007fffd0c435ea in std::pair<std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> >, bool> std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> > >::_M_emplace_unique<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long>&>(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long>&) () from /cvmfs/cms-ib.cern.ch/nweek-02726/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libDataFormatsStdDictionaries.so
#4  0x00007fffd0c436fc in ROOT::Detail::TCollectionProxyInfo::MapInsert<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned long long, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> > > >::feed(void*, void*, unsigned long) () from /cvmfs/cms-ib.cern.ch/nweek-02726/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libDataFormatsStdDictionaries.so
#5  0x00007ffff70d75e1 in TGenCollectionProxy::Commit(void*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#6  0x00007ffff7189894 in int TStreamerInfoActions::ReadSTL<&TStreamerInfoActions::ReadSTLMemberWiseSameClass, &TStreamerInfoActions::ReadSTLObjectWiseFastArray>(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#7  0x00007ffff703ebd5 in TBufferFile::ReadClassBuffer(TClass const*, void*, TClass const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#8  0x00007ffff728948b in int TStreamerInfo::ReadBuffer<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#9  0x00007ffff710604d in TStreamerInfoActions::GenericReadAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#10 0x00007ffff703ebd5 in TBufferFile::ReadClassBuffer(TClass const*, void*, TClass const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#11 0x00007ffff703f740 in TBufferFile::ReadObjectAny(TClass const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#12 0x00007ffff7037cf4 in TBufferFile::ReadFastArray(void**, TClass const*, int, bool, TMemberStreamer*, TClass const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#13 0x00007ffff7289344 in int TStreamerInfo::ReadBuffer<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#14 0x00007ffff710604d in TStreamerInfoActions::GenericReadAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#15 0x00007ffff703ebd5 in TBufferFile::ReadClassBuffer(TClass const*, void*, TClass const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#16 0x00007ffff728948b in int TStreamerInfo::ReadBuffer<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#17 0x00007ffff710604d in TStreamerInfoActions::GenericReadAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#18 0x00007ffff7036b15 in TBufferFile::ApplySequence(TStreamerInfoActions::TActionSequence const&, void*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libRIO.so
#19 0x00007ffff7767b17 in TBranchElement::ReadLeavesMember(TBuffer&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libTree.so
#20 0x00007ffff7760149 in TBranch::GetEntry(long long, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libTree.so
#21 0x00007ffff777ab98 in TBranchElement::GetEntry(long long, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libTree.so
#22 0x00007ffff777ab50 in TBranchElement::GetEntry(long long, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/external/slc7_amd64_gcc10/lib/libTree.so
#23 0x00007fffb4422fa1 in edm::RootTree::getEntry(TBranch*, long long) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/pluginIOPoolInput.so
#24 0x00007fffb43f3d3d in edm::RootDelayedReader::getProduct_(edm::BranchID const&, edm::EDProductGetter const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/pluginIOPoolInput.so
#25 0x00007ffff7b9bca7 in edm::DelayedReader::getProduct(edm::BranchID const&, edm::EDProductGetter const*, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#26 0x00007ffff7c72c1b in edm::DelayedReaderInputProductResolver::prefetchAsync_(edm::WaitingTaskHolder, edm::Principal const&, bool, edm::ServiceToken const&, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#27 0x00007ffff7c72e40 in edm::DelayedReaderInputProductResolver::prefetchAsync_(edm::WaitingTaskHolder, edm::Principal const&, bool, edm::ServiceToken const&, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const::{lambda()#1}::operator()() const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#28 0x00007ffff7c72f4f in void edm::SerialTaskQueueChain::actionToRun<edm::DelayedReaderInputProductResolver::prefetchAsync_(edm::WaitingTaskHolder, edm::Principal const&, bool, edm::ServiceToken const&, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const::{lambda()#1}&>(edm::DelayedReaderInputProductResolver::prefetchAsync_(edm::WaitingTaskHolder, edm::Principal const&, bool, edm::ServiceToken const&, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const::{lambda()#1}&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#29 0x00007ffff7c72fa1 in edm::SerialTaskQueue::QueuedTask<edm::SerialTaskQueueChain::push<edm::DelayedReaderInputProductResolver::prefetchAsync_(edm::WaitingTaskHolder, edm::Principal const&, bool, edm::ServiceToken const&, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const::{lambda()#1}&>(tbb::detail::d1::task_group&, edm::DelayedReaderInputProductResolver::prefetchAsync_(edm::WaitingTaskHolder, edm::Principal const&, bool, edm::ServiceToken const&, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const::{lambda()#1}&)::{lambda()#1}>::execute() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#30 0x00007ffff7e4a075 in tbb::detail::d1::function_task<edm::SerialTaskQueue::spawn(edm::SerialTaskQueue::TaskBase&)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreConcurrency.so
#31 0x00007ffff639bbec in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=0x7ffff1d52600, this=0x7ffff1d47e00) at /pool/condor/dir_35707/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-83b841c9062176110df527b0a6f75107/tbb-v2021.5.0/src/tbb/task_dispatcher.h:322
#32 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x7ffff1d47e00) at /pool/condor/dir_35707/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-83b841c9062176110df527b0a6f75107/tbb-v2021.5.0/src/tbb/task_dispatcher.h:463
#33 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=..., w_ctx=...) at /pool/condor/dir_35707/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-83b841c9062176110df527b0a6f75107/tbb-v2021.5.0/src/tbb/task_dispatcher.cpp:168
#34 0x00007ffff7bc19d8 in edm::EventProcessor::processLumis(std::shared_ptr<void> const&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#35 0x00007ffff7bcc82b in edm::EventProcessor::runToCompletion() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libFWCoreFramework.sologic_error>, dest=0x7ffff5b82080 <std::logic_error::~logic_error()>) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:78

@Dr15Jones
Copy link
Contributor

@pcanal it looks like we have hit a problem reading back a file we could read before.

@Dr15Jones
Copy link
Contributor

@makortel did we change the compiler recently?

@smuzaffar
Copy link
Contributor

No, compiler was not changed. pythia8 was updated to build with hepmc2 and hepmc3 support ( https://github.com/cms-sw/cmsdist/pull/7724/files ). May be pythia8 is loading both hepmc libs and that might be the reason behind this failure

@makortel
Copy link
Contributor

Adding some debug symbols to the first stack frames

#0  0x00007ffff5b6dc9e in __cxxabiv1::__cxa_throw (obj=obj@entry=0x7fff805903a0, tinfo=0x7ffff5c881c8 <typeinfo for std::logic_error>, dest=0x7ffff5b82080 <std::logic_error::~logic_error()>) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:78
#1  0x00007ffff5b64f92 in std::__throw_logic_error (__s=__s@entry=0x7ffff5c3d758 "basic_string::_M_construct null not valid") at ../../../../../libstdc++-v3/src/c++11/functexcept.cc:66
#2  0x00007ffff5bfe55f in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*> (this=0x7fff80421b90, __beg=0x0, __end=0x6 <error: Cannot access memory at address 0x6>)
    at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_3_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_3_0_pre2-build/BUILD/slc7_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/gcc-10.3.0/obj/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:212
#3  0x00007fffd0cb95ea in std::pair<std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> >, bool> std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> > >::_M_emplace_unique<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long>&>(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long>&) ()
   from /build/mkortela/debug/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libDataFormatsStdDictionaries.so
#4  0x00007fffd0cb96fc in ROOT::Detail::TCollectionProxyInfo::MapInsert<std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned long long, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long long> > > >::feed(void*, void*, unsigned long) ()
   from /build/mkortela/debug/CMSSW_12_4_X_2022-03-28-1100/lib/slc7_amd64_gcc10/libDataFormatsStdDictionaries.so

@smuzaffar
Copy link
Contributor

I have opened a revert of pythia8 change cms-sw/cmsdist#7731 and will try to run the test mentioned here

@pcanal
Copy link
Contributor

pcanal commented Mar 29, 2022

It may be related to the improvements/fixes I am currently making to the support for std::pair.

@makortel
Copy link
Contributor

So DataFormats/HepMCCandidate does not depend on hepmc (it's basically just a copy of the information). With strace I didn't see any other libraries whose name include hepmc or pythia except those related to PhysicsTools/HepMCCandAlgos package.

@smuzaffar
Copy link
Contributor

smuzaffar commented Mar 30, 2022

It looks like it is not related to the pythia8/hepmc update. cms-sw/cmsdist#7731 reverts the pythia8 change and we built a full release using it but test mentioned here still failed.

We also had updated root cms-sw/cmsdist#7722 for this IB where we saw this issue. We updated root to the tip of v6.24.patches branch to fix the issue reported #37227. @pcanal root-project/root@292e441...4b08829 are the root changes which were included as compared to the previous version.

@makortel
Copy link
Contributor

unassign hlt,generators

@pcanal
Copy link
Contributor

pcanal commented Mar 30, 2022

@smuzaffar root-project/root#10230 might/should help.

@smuzaffar
Copy link
Contributor

thanks @pcanal , is there a v6-24-patches version of it?

@pcanal
Copy link
Contributor

pcanal commented Mar 30, 2022

Not yet. I am still in the process of validating but let me see if it can applied cleanly as is.

@pcanal
Copy link
Contributor

pcanal commented Mar 30, 2022

You can try: root-project/root#10280

@smuzaffar
Copy link
Contributor

many thanks @pcanal , I have started the tests cms-sw/root#167 . Once the cmssw dev area is available then I will run the HLT too

@smuzaffar
Copy link
Contributor

@pcanal , root-project/root#10280 looks good, PR tests passed cms-sw/root#167 (comment) and also the additonal HLT tests mentioned #37392 (comment) worked too

@pcanal
Copy link
Contributor

pcanal commented Apr 6, 2022

The patches have been merged in v6.24, v6.26 and master branches.

@smuzaffar
Copy link
Contributor

thanks @pcanal , I have open a PR cms-sw/cmsdist#7762 to include root v6-24-00-patches changes in to cmssw

@makortel
Copy link
Contributor

@missirol The HLT validation test is working fine now, right?

@missirol
Copy link
Contributor Author

Yes, ever since we by-passed the issue by updating the input files in #37393. I didn't try to test with the old input file and the patch discussed lately here, but I understood Shahzad did so in #37392 (comment) .

Does this answer the question, Matti? (I was waiting for signatures before closing the issue.)

@makortel
Copy link
Contributor

+1

Thanks @missirol

@cmsbuild
Copy link
Contributor

This issue is fully signed and ready to be closed.

@missirol
Copy link
Contributor Author

Closing. Thanks to experts for the follow-ups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants