Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Commented XML strings in the StringVec parameter are wrongly parsed? #43

Closed
dudarboh opened this issue Sep 5, 2021 · 4 comments · Fixed by #45
Closed

Commented XML strings in the StringVec parameter are wrongly parsed? #43

dudarboh opened this issue Sep 5, 2021 · 4 comments · Fixed by #45

Comments

@dudarboh
Copy link
Member

dudarboh commented Sep 5, 2021

Issue

Consider the following steering file:

<marlin>

<constants>
</constants>

<global>
    <parameter name="LCIOInputFiles">
    <!-- my_data1.slcio -->
         my_data2.slcio
    </parameter>
    <parameter name="MaxRecordNumber" value="0" />
    <parameter name="SupressCheck" value="false" />
    <parameter name="AllowToModifyEvent" value="false" />
</global>

<execute>
        <processor name="MyUsefulProcessor" />
</execute>


<processor name="MyUsefulProcessor" type="UsefulProcessor">
</processor>

</marlin>

I would expect the program to be processing only my_data2.slcio as the my_data1.slcio is commented.
However running Marlin on such an steering file will read my_data1.slcio instead of my_data2.slcio!

E.g. Marlin -c ./steer.xml will output:

. . .
Loading LCIO file [my_data1.slcio]
. . .

Tested with:

iLCSoft

source /cvmfs/ilc.desy.de/sw/x86_64_gcc82_centos7/v02-02-02/init_ilcsoft.sh

@dudarboh
Copy link
Member Author

@tmadlener @gaede

@tmadlener
Copy link
Contributor

If this is really an XML parsing issue then it might be hard to fix inside Marlin. I will have to check if that is truly the case.

In any case if you really only want one file, there is always the possibility to pass it in via the command line, like:

Marlin --global.LCIOInputFiles=my_data2.slcio <rest of the arguments>

@dudarboh
Copy link
Member Author

dudarboh commented Jun 13, 2022

Yes, I agree that this bug is very easy to avoid, when you already know it exists :)

In my case I have a test steering file which looks something like this:

    <global>
		<parameter name="LCIOInputFiles">
            /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/dst-merged/250-SetA/4f_ZZnunu_semileptonic/ILD_l5_o1_v02/v02-02/00015147/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500078.P4f_zznu_sl.eL.pR.n008.d_dstm_15147_38.slcio
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/dst-merged/250-SetA/4f_ZZ_semileptonic/ILD_l5_o1_v02/v02-02/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500074.P4f_zz_sl.eL.pR.n000.d_dstm_15116_1.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/dst-merged/250-SetA/4f_WW_semileptonic/ILD_l5_o1_v02/v02-02/00015173/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500082.P4f_ww_sl.eL.pR.n001.d_dstm_15173_7.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/dst-merged/250-SetA/4f_singleZee_semileptonic/ILD_l5_o1_v02/v02-02/00015179/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500102.P4f_sze_sl.eL.pR.n000.d_dstm_15179_2.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/rec/250-SetA/4f_WW_hadronic/ILD_l5_o1_v02/v02-02/00015155/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500066.P4f_ww_h.eL.pR.n000_001.d_rec_00015155_213.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/dst-merged/250-SetA/2f_hadronic_eL_pR/ILD_l5_o1_v02/v02-02/00015161/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500010.P2f_z_h.eL.pR.n000.d_dstm_15161_0.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/dst-merged/250-SetA/higgs_excl/ILD_l5_o1_v02/v02-02/00015201/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I402199.Pn23n23h_e2e2.eL.pR.n000.d_dstm_15201_1.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/rec/250-SetA/2f_hadronic_eL_pR/ILD_l5_o1_v02/v02-02/00015161/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500010.P2f_z_h.eL.pR.n000_052.d_rec_00015161_219.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/rec/250-SetA/2f_hadronic_eL_pR/ILD_l5_o1_v02/v02-02/00015161/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500010.P2f_z_h.eL.pR.n001_209.d_rec_00015161_259.slcio -->
            <!-- /pnfs/desy.de/ilc/prod/ilc/mc-2020/ild/rec/250-SetA/4f_WW_hadronic/ILD_l5_o1_v02/v02-02/00015128/000/rv02-02.sv02-02.mILD_l5_o1_v02.E250-SetA.I500068.P4f_ww_h.eR.pL.n000_350.d_rec_00015128_106.slcio -->
        </parameter>

And I always have to remember to move the un-commented file on top, because if the first line is commented, it will read the wrong file! And one, who is unaware of this issue, could spend hours (real-life story) to understand why the event doesn't look as you expect, when you test it locally and run as 1000 batch jobs which don't have any commented lines.

Also, always copy-pasting filename into a command line seems more time consuming than just one-click uncommenting the line.

If commenting in this field is e.g. unsupported, a warning/error would be nice.

@dudarboh
Copy link
Member Author

dudarboh commented Sep 9, 2022

image

[ MESSAGE "DecayChainDrawer"] -------------------------------------------------
 ***********************************************
 A runtime error occured - (uncaught exception):
      lcio::IOException: [LCReader::open()] File(s) not found:  54  event  seg  fault, need to check  
 Marlin will have to be terminated, sorry.
 ***********************************************

*** Error in `Marlin': malloc(): smallbin double linked list corrupted: 0x00000000014ec7d0 ***
======= Backtrace: =========
 . . .

Just another way I accidentally stumbled on this problem, forgetting about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants