TOF: Adding workflow for integrated TOF currents #10608
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The 1D integrated TOF currents (~1ms) are similar as the TPC 1D-IDCs and can be used for QA and calibration. The clusters are integrated in the `o2-tof-integrate-cluster-workflow` device and aggregated + merged to longer contiguous time intervalsi (1000TFs default ~ 11s) in the `o2-tof-merge-integrate-cluster-workflow` device using the `TimeSlotCalibration`. The workflow can be used using the CTFs as input: ``` o2-ctf-reader-workflow --copy-cmd no-copy --onlyDet TOF --ctf-input "ctf.list" --severity warning \ | o2-tof-reco-workflow --input-type compressed-clusters-ctf --output-type "clusters,disable-writer" --disable-mc --disable-root-output \ | o2-tof-integrate-cluster-workflow | o2-tof-merge-integrate-cluster-workflow --debug true \ | o2-calibration-ccdb-populator-workflow --ccdb-path "ccdb-test.cern.ch:8080" -b ``` The output will be stored in the CCDB or in local files using the `o2-tof-merge-integrate-cluster-workflow --debug true` option.
|
@chiarazampolli |
|
Example plot for around 3 minutes for Run 523141 |
|
Hello @matthias-kleiner , Thanks! We currently don’t do calibration from grid jobs since we have no aggregator, so we need to use the option to store to local files, right? Cheers, Chiara |
|
Dear @matthias-kleiner, In your tof-workflow config you have The input-type you are asking is not defined. It works because ctf already provides digits but it is misleading. Also disable-writer in the output-type doesn't have any meaning. You have --disable-root-output and it is enough (in any case for data tof-workflow is already defined in O2DPG scripts so you will inherit it from the global process) I also did't understand if and how this is supposed to run in async reco. |
noferini
requested changes
Jan 21, 2023
|
Error while checking build/O2/fullCI for 3d89a4d at 2023-01-22 12:15: Full log here. |
Hello @noferini
I am not sure how exactly the async reco is running, but I suppose if an input list of ctfs are used which are ordered in time then it should be fine? We want to run it for the apass 3 to have the currents which we can use for our space-charge distortions studies, but maybe we can also just run it on the GSI and produce the files before apass3. |
Dear @matthias-kleiner, |
- renaming variable - changing std::pow(10, -9) -> 1.E-9
|
Error while checking build/O2/fullCI for ab773d0 at 2023-01-22 18:06: Full log here. |
|
@chiarazampolli @noferini |
Hi @matthias-kleiner , Just in case it can help for TOF we do the following:
This allows us to merge all calib info as any root file and then just running on it with the reader + the workflow as in sync |
|
Error while checking build/O2/fullCI for 858a822 at 2023-01-23 20:14: Full log here. |
@noferini Many thanks for your help. I am currently trying to implement the reader, but I am struggling to inject the TFIDInfo data. I am doing it like here , but theo2-tof-merge-integrate-cluster-workflow which receives the data gets for each run call only the same TFInfos i.e. only the first value I injected in the loop. Is there something I am missing, like some update method or something?
I was also looking how it is done for the I was testing and implemented some dummy |
|
I think the problem is, that I was sending several messages per |
- writing TFIDInfo to TTree - moving writer to workflowIO
|
Hello @noferini I added the reader and tested it. and for merging the output files I think this would be fine? Do you have any more suggestions? |
noferini
approved these changes
Jan 24, 2023
There was a problem hiding this comment.
Hi @matthias-kleiner ,
many thanks!
You were very fast.
Cheers,
Francesco
|
Error while checking build/O2/fullCI for 773b62b at 2023-01-24 22:44: Full log here. |
Hi @noferini okay thanks. Should I squash the commits or is it fine like it is? |
|
Hi @matthias-kleiner , it is fine with me. FullCI fails with the error I think this is not related to this PR... pinging @shahor02 and @ktf . Can you merge it? |
matthias-kleiner
added a commit
to matthias-kleiner/AliceO2
that referenced
this pull request
Feb 1, 2023
FT0,FV0,TPC: The workflows for the integrated currents are analog to the worfklows for the TOF (integration of the cluster, writing of the clusters to file, reading in the clusters and merging the clusters to contigous intervals) - integrated currents can be stored in the CCDB or in local files - see for the TOF PR for further details AliceO2Group#10608 - examples of the integrated currents and use case for FT0,FV0,TOF,TPC https://indico.cern.ch/event/1237926/contributions/5251727/attachments/2585970/4461305/ATO-615O2%20Multiplicity%20counters%20-%20normalization%20and%20interactive%20visualization-3.pdf TOF - templating TOFIntegratedClusterCalibrator to be able to use the same calibratior for different detectors - Adding filtering of noise during integration of currents - Add option to store calibration objects to local files TPC: - IDCFactorization: parallelising writing of files for speed up - removing unneded code for integrated clusters which was in the IDC chain - Integrated cluster processing can be done in 1D (faster, less memory consumption, outliers e.g. out-of-sync FECs are not taken into account) or in 3D (outliers taken into account) Running the workflows for all the currents from list of input files: ARGS_ALL="--shm-segment-size 50000000000" o2-ctf-reader-workflow --copy-cmd no-copy --onlyDet FT0,FV0,TOF,TPC --ctf-input "list.txt" --severity warning $ARGS_ALL \ | o2-ft0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-ft0-integrate-cluster-workflow --disable-root-output \ | o2-fv0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-fv0-integrate-cluster-workflow --disable-root-output \ | o2-ft0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-fv0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tof-reco-workflow --input-type none --output-type "clusters" --disable-mc --disable-root-output $ARGS_ALL \ | o2-tof-integrate-cluster-workflow --disable-root-output \ | o2-tof-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tpc-reco-workflow --input-type compressed-clusters-ctf --output-type "clusters,disable-writer" --disable-mc \ | o2-tpc-integrate-cluster-workflow $ARGS_ALL --disable-root-output \ | o2-tpc-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none --nthreads 32 --enableWritingPadStatusMap -b
matthias-kleiner
added a commit
to matthias-kleiner/AliceO2
that referenced
this pull request
Feb 3, 2023
FT0,FV0,TPC: The workflows for the integrated currents are analog to the worfklows for the TOF (integration of the cluster, writing of the clusters to file, reading in the clusters and merging the clusters to contigous intervals) - integrated currents can be stored in the CCDB or in local files - see for the TOF PR for further details AliceO2Group#10608 - examples of the integrated currents and use case for FT0,FV0,TOF,TPC https://indico.cern.ch/event/1237926/contributions/5251727/attachments/2585970/4461305/ATO-615O2%20Multiplicity%20counters%20-%20normalization%20and%20interactive%20visualization-3.pdf TOF - templating TOFIntegratedClusterCalibrator to be able to use the same calibratior for different detectors - Adding filtering of noise during integration of currents - Add option to store calibration objects to local files TPC: - IDCFactorization: parallelising writing of files for speed up - removing unneded code for integrated clusters which was in the IDC chain - Integrated cluster processing can be done in 1D (faster, less memory consumption, outliers e.g. out-of-sync FECs are not taken into account) or in 3D (outliers taken into account) Running the workflows for all the currents from list of input files: ARGS_ALL="--shm-segment-size 50000000000" o2-ctf-reader-workflow --copy-cmd no-copy --onlyDet FT0,FV0,TOF,TPC --ctf-input "list.txt" --severity warning $ARGS_ALL \ | o2-ft0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-ft0-integrate-cluster-workflow --disable-root-output \ | o2-fv0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-fv0-integrate-cluster-workflow --disable-root-output \ | o2-ft0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-fv0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tof-reco-workflow --input-type none --output-type "clusters" --disable-mc --disable-root-output $ARGS_ALL \ | o2-tof-integrate-cluster-workflow --disable-root-output \ | o2-tof-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tpc-reco-workflow --input-type compressed-clusters-ctf --output-type "clusters,disable-writer" --disable-mc \ | o2-tpc-integrate-cluster-workflow $ARGS_ALL --disable-root-output \ | o2-tpc-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none --nthreads 32 --enableWritingPadStatusMap -b
davidrohr
pushed a commit
that referenced
this pull request
Feb 4, 2023
* FT0/FV0/TPC: adding workflows for integrated currents FT0,FV0,TPC: The workflows for the integrated currents are analog to the worfklows for the TOF (integration of the cluster, writing of the clusters to file, reading in the clusters and merging the clusters to contigous intervals) - integrated currents can be stored in the CCDB or in local files - see for the TOF PR for further details #10608 - examples of the integrated currents and use case for FT0,FV0,TOF,TPC https://indico.cern.ch/event/1237926/contributions/5251727/attachments/2585970/4461305/ATO-615O2%20Multiplicity%20counters%20-%20normalization%20and%20interactive%20visualization-3.pdf TOF - templating TOFIntegratedClusterCalibrator to be able to use the same calibratior for different detectors - Adding filtering of noise during integration of currents - Add option to store calibration objects to local files TPC: - IDCFactorization: parallelising writing of files for speed up - removing unneded code for integrated clusters which was in the IDC chain - Integrated cluster processing can be done in 1D (faster, less memory consumption, outliers e.g. out-of-sync FECs are not taken into account) or in 3D (outliers taken into account) Running the workflows for all the currents from list of input files: ARGS_ALL="--shm-segment-size 50000000000" o2-ctf-reader-workflow --copy-cmd no-copy --onlyDet FT0,FV0,TOF,TPC --ctf-input "list.txt" --severity warning $ARGS_ALL \ | o2-ft0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-ft0-integrate-cluster-workflow --disable-root-output \ | o2-fv0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-fv0-integrate-cluster-workflow --disable-root-output \ | o2-ft0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-fv0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tof-reco-workflow --input-type none --output-type "clusters" --disable-mc --disable-root-output $ARGS_ALL \ | o2-tof-integrate-cluster-workflow --disable-root-output \ | o2-tof-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tpc-reco-workflow --input-type compressed-clusters-ctf --output-type "clusters,disable-writer" --disable-mc \ | o2-tpc-integrate-cluster-workflow $ARGS_ALL --disable-root-output \ | o2-tpc-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none --nthreads 32 --enableWritingPadStatusMap -b * Fixing clang-format, wrong copyright notice * Moving TOFIntegratedClusterCalibrator to DetectorsCalibration/IntegratedClusterCalibrator
aphecetche
pushed a commit
to aphecetche/AliceO2
that referenced
this pull request
Feb 4, 2023
…0663) * FT0/FV0/TPC: adding workflows for integrated currents FT0,FV0,TPC: The workflows for the integrated currents are analog to the worfklows for the TOF (integration of the cluster, writing of the clusters to file, reading in the clusters and merging the clusters to contigous intervals) - integrated currents can be stored in the CCDB or in local files - see for the TOF PR for further details AliceO2Group#10608 - examples of the integrated currents and use case for FT0,FV0,TOF,TPC https://indico.cern.ch/event/1237926/contributions/5251727/attachments/2585970/4461305/ATO-615O2%20Multiplicity%20counters%20-%20normalization%20and%20interactive%20visualization-3.pdf TOF - templating TOFIntegratedClusterCalibrator to be able to use the same calibratior for different detectors - Adding filtering of noise during integration of currents - Add option to store calibration objects to local files TPC: - IDCFactorization: parallelising writing of files for speed up - removing unneded code for integrated clusters which was in the IDC chain - Integrated cluster processing can be done in 1D (faster, less memory consumption, outliers e.g. out-of-sync FECs are not taken into account) or in 3D (outliers taken into account) Running the workflows for all the currents from list of input files: ARGS_ALL="--shm-segment-size 50000000000" o2-ctf-reader-workflow --copy-cmd no-copy --onlyDet FT0,FV0,TOF,TPC --ctf-input "list.txt" --severity warning $ARGS_ALL \ | o2-ft0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-ft0-integrate-cluster-workflow --disable-root-output \ | o2-fv0-reco-workflow --disable-mc --disable-root-input --disable-root-output \ | o2-fv0-integrate-cluster-workflow --disable-root-output \ | o2-ft0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-fv0-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tof-reco-workflow --input-type none --output-type "clusters" --disable-mc --disable-root-output $ARGS_ALL \ | o2-tof-integrate-cluster-workflow --disable-root-output \ | o2-tof-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none \ | o2-tpc-reco-workflow --input-type compressed-clusters-ctf --output-type "clusters,disable-writer" --disable-mc \ | o2-tpc-integrate-cluster-workflow $ARGS_ALL --disable-root-output \ | o2-tpc-merge-integrate-cluster-workflow --dump-calib-data --meta-output-dir none --nthreads 32 --enableWritingPadStatusMap -b * Fixing clang-format, wrong copyright notice * Moving TOFIntegratedClusterCalibrator to DetectorsCalibration/IntegratedClusterCalibrator
This was referenced Feb 16, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The 1D integrated TOF currents (~1ms) are similar as the TPC 1D-IDCs and can be used for QA and calibration.
The clusters are integrated in the
o2-tof-integrate-cluster-workflowdevice and aggregated + merged to longer contiguous time intervals (1000TFs default ~ 11s) in theo2-tof-merge-integrate-cluster-workflowdevice using theTimeSlotCalibration. The workflow can be used using the CTFs as input:The output will be stored in the CCDB or in local files using the
o2-tof-merge-integrate-cluster-workflow --debug trueoption.