LeptonEfficiencyCorrector not found when running with crab

[amagnan]

Dear experts,
I have been running the leptonSFproducer - which calls the c++ code in LeptonEfficiencyCorrector - properly interactively, but when I submit to crab I get this error.
Could you please indicate me what is needed such that this file would be found by crab ?
Many thanks.

== CMSSW: /srv/CMSSW_9_4_10
== CMSSW:
== CMSSW: /srv
== CMSSW: mv: cannot stat `module': No such file or directory
== CMSSW: Found Proxy in: /srv/aebf959e130b2b40234b483b07243972d2e6d8eb
== CMSSW: Error in PyROOT::TPyROOTApplication::ProcessLine: macro /srv/CMSSW_9_4_10/src/PhysicsTools/NanoAODTools/python/postprocessing/helpers/LeptonEfficiencyCorrector.cc not found in path .:/cvmfs/cms.cern.ch/slc6_amd64_gcc630/lcg/root/6.10.08-elfike2/macros
== CMSSW: Error in PyROOT::TPyROOTApplication::ProcessLine: macro /srv/CMSSW_9_4_10/src/PhysicsTools/NanoAODTools/python/postprocessing/helpers/LeptonEfficiencyCorrector.cc not found in path .:/cvmfs/cms.cern.ch/slc6_amd64_gcc630/lcg/root/6.10.08-elfike2/macros
== CMSSW: ARGV: ['crab_script_vbf_mc.py', '2']
== CMSSW: cms.untracked.vstring('/store/mc/RunIIFall17NanoAOD/DYJetsToLL_1J_TuneCP5_13TeV-amcatnloFXFX-pythia8/NANOAODSIM/PU2017_12Apr2018_94X_mc2017_realistic_v14-v1/00000/0AC4DA82-D591-E811-BE97-00266CFFCD50.root')
== CMSSW: --------------- using edmFileUtil to convert PFN to LFN -------------------------
== CMSSW: Data is local
== CMSSW: /store/mc/RunIIFall17NanoAOD/DYJetsToLL_1J_TuneCP5_13TeV-amcatnloFXFX-pythia8/NANOAODSIM/PU2017_12Apr2018_94X_mc2017_realistic_v14-v1/00000/0AC4DA82-D591-E811-BE97-00266CFFCD50.root -> dcap://dcap01.jinr-t1.ru:22125/pnfs/jinr-t1.ru/data/cms/store/mc/RunIIFall17NanoAOD/DYJetsToLL_1J_TuneCP5_13TeV-amcatnloFXFX-pythia8/NANOAODSIM/PU2017_12Apr2018_94X_mc2017_realistic_v14-v1/00000/0AC4DA82-D591-E811-BE97-00266CFFCD50.root
== CMSSW: Testing file open
== CMSSW: Test OK
== CMSSW: Loading btagSF weights for CSV (v2) algorithm from file '/srv/CMSSW_9_4_10/src/PhysicsTools/NanoAODTools/data/btagSF/CSVv2_94XSF_V2_B_F.csv'
== CMSSW: Load Library 'CondFormatsBTauObjects'
== CMSSW: Load Library 'CondToolsBTau'
== CMSSW: Load C++ Worker
== CMSSW: Load C++ Worker
== CMSSW: Because you requested a FJR we assume you want the final hadd. No name specified for the output file, will use tree.root
== CMSSW: Will write selected trees to .
== CMSSW: Traceback (most recent call last):
== CMSSW: File "crab_script_vbf_mc.py", line 10, in
== CMSSW: p.run()
== CMSSW: File "/srv/CMSSW_9_4_10/python/PhysicsTools/NanoAODTools/postprocessing/framework/postprocessor.py", line 75, in run
== CMSSW: m.beginJob()
== CMSSW: File "/srv/CMSSW_9_4_10/python/VBFHToInv/NanoAODTools/postprocessing/modules/lepSFProducer.py", line 62, in beginJob
== CMSSW: self._worker_mu = ROOT.LeptonEfficiencyCorrector(self.mu_f,self.mu_h)
== CMSSW: File "/cvmfs/cms.cern.ch/slc6_amd64_gcc630/lcg/root/6.10.08-elfike2/lib/ROOT.py", line 468, in __getattr2
== CMSSW: raise AttributeError( name )
== CMSSW: AttributeError: LeptonEfficiencyCorrector

[arizzi]

it seems like it is not finding the library containing the C++ wrapper...
can you put somehwere the rest of the crab script output? does it work locally?
Are you sure the right release was used? Was the whole are compiled with scram?

[amagnan]

The full job output is here:
https://cmsweb.cern.ch/scheddmon/0194/cms1170/181025_202727:amagnan_crab_NanoPost6/job_out.2.0.txt

The code is here:
https://github.com/vukasinmilosevic/VBFHToInv-nanoAOD-tools/tree/master/crab

Locally I do:
python crab_script_vbf_mc.py 1
and it works fine, with only "cmsenv" as setup....

Note that I copied the leptonSFProducer module in our repository above - to change it for my needs. This is the code calling the LeptonEfficiencyCorrector. Could it be the problem somehow ?

[amagnan]

One more thing: to make the code compile, I needed to change this in the central file:
PhysicsTools/NanoAODTools/python/postprocessing/helpers/LeptonEfficiencyCorrector.cc

-#include "WeightCalculatorFromHistogram.cc"
+#include "PhysicsTools/NanoAODTools/src/WeightCalculatorFromHistogram.cc"

[arizzi]

mhh I'm not sure who is supposed to compile the /helpers folder... I would naively have put that under src/interface, if it works locally can you check which library contains the compiled version of that code?

[amagnan]

Indeed it is not in
nm -D --defined-only libPhysicsToolsNanoAODTools.so

OK, so you mean move the .cc inside src/ , and make a proper header file, instead of being in the helpers/ directory ?

[arizzi]

I'm not sure how you manage to use it locally then... I mean in order to use it as
ROOT.NameOfTheClass
you need to have it in classes.h and classes_def.xml

so yes you need to move it, make .h and include it in reflex dicts via classes*

[amagnan]

Yes I am not sure either, I am not the one who wrote that code, I was just trying to use it ;) But yes, I see and I'll test what you suggest. I think it works locally because there is this in the python code:
if "/LeptonEfficiencyCorrector_cc.so" not in ROOT.gSystem.GetLibraries():
print "Load C++ Worker"
ROOT.gROOT.ProcessLine(".L %s/src/PhysicsTools/NanoAODTools/python/postprocessing/helpers/LeptonEfficiencyCorrector.cc+" % os.environ['CMSSW_BASE'])

[arizzi]

ah ok, then the problem is that it should do it from the proper path that is something like
/python/PhysicsTools/NanoAODTools/postprocessing/helpers
instead of
src/....
because src/ is not shipped to crab but python/ is.

it should in fact work also locally with python/ given the scram symlink

[amagnan]

Hello,
coming back to this - I tested just changing the path of the library loaded to "python", it works fine interactively but still crashes with crab: this times it finds the module, but it doesn't load the library.... I added a "+" in the ".L" command -> ".L blah.cc++" and then it does make - and find - the library - but it doesn't find an include file, which is given in the standard way relative to the $CMSSW_BASE/src/ directory....
Bottom line, I abandon and will move the code properly into interface and src and make it part of the library....

[arizzi]

I guess that's the best option anyhow. Can you submit a PR?

[amagnan]

#110