Missing output file(s) *igv.txt expected by process MCAS2 (...)

[ftucos]

I keep having this error and I don't understand why.
I'm running the pipeline with docker.
The .command.sh seems ok and if i try to run manually the last line of the script:
find * -type f -name "*.narrowPeak" -exec echo -e "bwa/mergedLibrary/macs/narrowPeak/"{}"\t0,0,178" \; > SCC9_CDK2AP1_OE_CHD4_R2_peaks.igv.txt in the work folder I finally see the igv.txt file appearing.
I don't understand why the command is not executed as expected.

[JoseEspinosa]

For what you report it should work, could you please paste what you get inside the work directory in the .command.err file to see if there is something there.

[ftucos]

This is what I get in the .command.err. It's not throwing any error actually. The error is thrown by the nf-core pipeline because it's missing the file but the step is marked as complete. Indeed, I managed to run through the whole pipeline resuming it every time.

# Command line: callpeak -t SCC9_CDK2AP1_OE_CHD4_R2.mLb.clN.sorted.bam -c SCC9_CDK2AP1_OE_IgG_R1.mLb.clN.sorted.bam -f BAM -g 2700000000 -n SCC9_CDK2AP1_OE_CHD4_R2 -B --SPMR --keep-dup all
# ARGUMENTS LIST:
# name = SCC9_CDK2AP1_OE_CHD4_R2
# format = BAM
# ChIP-seq file = ['SCC9_CDK2AP1_OE_CHD4_R2.mLb.clN.sorted.bam']
# control file = ['SCC9_CDK2AP1_OE_IgG_R1.mLb.clN.sorted.bam']
# effective genome size = 2.70e+09
# band width = 300
# model fold = [5, 50]
# qvalue cutoff = 5.00e-02
# The maximum gap between significant sites is assigned as the read length/tag size.
# The minimum length of peaks is assigned as the predicted fragment length "d".
# Larger dataset will be scaled towards smaller dataset.
# Range for calculating regional lambda is: 1000 bps and 10000 bps
# Broad region calling is off
# Paired-End mode is off
# MACS will save fragment pileup signal per million reads
 
INFO  @ Thu, 20 Jan 2022 20:22:02: #1 read tag files... 
INFO  @ Thu, 20 Jan 2022 20:22:02: #1 read treatment tags... 
INFO  @ Thu, 20 Jan 2022 20:22:04:  1000000 
INFO  @ Thu, 20 Jan 2022 20:22:07:  2000000 
INFO  @ Thu, 20 Jan 2022 20:22:09:  3000000 
INFO  @ Thu, 20 Jan 2022 20:22:12:  4000000 
INFO  @ Thu, 20 Jan 2022 20:22:14:  5000000 
INFO  @ Thu, 20 Jan 2022 20:22:16:  6000000 
INFO  @ Thu, 20 Jan 2022 20:22:19:  7000000 
INFO  @ Thu, 20 Jan 2022 20:22:21:  8000000 
INFO  @ Thu, 20 Jan 2022 20:22:23:  9000000 
INFO  @ Thu, 20 Jan 2022 20:22:25:  10000000 
INFO  @ Thu, 20 Jan 2022 20:22:28:  11000000 
INFO  @ Thu, 20 Jan 2022 20:22:30:  12000000 
INFO  @ Thu, 20 Jan 2022 20:22:32:  13000000 
INFO  @ Thu, 20 Jan 2022 20:22:35:  14000000 
INFO  @ Thu, 20 Jan 2022 20:22:36: 14698974 reads have been read. 
INFO  @ Thu, 20 Jan 2022 20:22:36: #1.2 read input tags... 
INFO  @ Thu, 20 Jan 2022 20:22:39:  1000000 
INFO  @ Thu, 20 Jan 2022 20:22:41:  2000000 
INFO  @ Thu, 20 Jan 2022 20:22:44:  3000000 
INFO  @ Thu, 20 Jan 2022 20:22:46:  4000000 
INFO  @ Thu, 20 Jan 2022 20:22:48:  5000000 
INFO  @ Thu, 20 Jan 2022 20:22:51:  6000000 
INFO  @ Thu, 20 Jan 2022 20:22:53:  7000000 
INFO  @ Thu, 20 Jan 2022 20:22:55:  8000000 
INFO  @ Thu, 20 Jan 2022 20:22:57:  9000000 
INFO  @ Thu, 20 Jan 2022 20:23:00:  10000000 
INFO  @ Thu, 20 Jan 2022 20:23:02:  11000000 
INFO  @ Thu, 20 Jan 2022 20:23:04:  12000000 
INFO  @ Thu, 20 Jan 2022 20:23:07:  13000000 
INFO  @ Thu, 20 Jan 2022 20:23:09:  14000000 
INFO  @ Thu, 20 Jan 2022 20:23:11:  15000000 
INFO  @ Thu, 20 Jan 2022 20:23:13: 15518469 reads have been read. 
INFO  @ Thu, 20 Jan 2022 20:23:13: #1 tag size is determined as 50 bps 
INFO  @ Thu, 20 Jan 2022 20:23:13: #1 tag size = 50.0 
INFO  @ Thu, 20 Jan 2022 20:23:13: #1  total tags in treatment: 14698974 
INFO  @ Thu, 20 Jan 2022 20:23:13: #1  total tags in control: 15518469 
INFO  @ Thu, 20 Jan 2022 20:23:13: #1 finished! 
INFO  @ Thu, 20 Jan 2022 20:23:13: #2 Build Peak Model... 
INFO  @ Thu, 20 Jan 2022 20:23:13: #2 looking for paired plus/minus strand peaks... 
INFO  @ Thu, 20 Jan 2022 20:23:16: #2 number of paired peaks: 28504 
INFO  @ Thu, 20 Jan 2022 20:23:16: start model_add_line... 
INFO  @ Thu, 20 Jan 2022 20:23:16: start X-correlation... 
INFO  @ Thu, 20 Jan 2022 20:23:16: end of X-cor 
INFO  @ Thu, 20 Jan 2022 20:23:16: #2 finished! 
INFO  @ Thu, 20 Jan 2022 20:23:16: #2 predicted fragment length is 250 bps 
INFO  @ Thu, 20 Jan 2022 20:23:16: #2 alternative fragment length(s) may be 250 bps 
INFO  @ Thu, 20 Jan 2022 20:23:16: #2.2 Generate R script for model : SCC9_CDK2AP1_OE_CHD4_R2_model.r 
INFO  @ Thu, 20 Jan 2022 20:23:16: #3 Call peaks... 
INFO  @ Thu, 20 Jan 2022 20:23:16: #3 Pre-compute pvalue-qvalue table... 
INFO  @ Thu, 20 Jan 2022 20:24:22: #3 In the peak calling step, the following will be performed simultaneously: 
INFO  @ Thu, 20 Jan 2022 20:24:22: #3   Write bedGraph files for treatment pileup (after scaling if necessary)... SCC9_CDK2AP1_OE_CHD4_R2_treat_pileup.bdg 
INFO  @ Thu, 20 Jan 2022 20:24:22: #3   Write bedGraph files for control lambda (after scaling if necessary)... SCC9_CDK2AP1_OE_CHD4_R2_control_lambda.bdg 
INFO  @ Thu, 20 Jan 2022 20:24:22: #3   --SPMR is requested, so pileup will be normalized by sequencing depth in million reads. 
INFO  @ Thu, 20 Jan 2022 20:24:22: #3 Call peaks for each chromosome... 
INFO  @ Thu, 20 Jan 2022 20:25:28: #4 Write output xls file... SCC9_CDK2AP1_OE_CHD4_R2_peaks.xls 
INFO  @ Thu, 20 Jan 2022 20:25:28: #4 Write peak in narrowPeak format file... SCC9_CDK2AP1_OE_CHD4_R2_peaks.narrowPeak 
INFO  @ Thu, 20 Jan 2022 20:25:28: #4 Write summits bed file... SCC9_CDK2AP1_OE_CHD4_R2_summits.bed 
INFO  @ Thu, 20 Jan 2022 20:25:28: Done! 
***** WARNING: File SCC9_CDK2AP1_OE_CHD4_R2_peaks.narrowPeak has inconsistent naming convention for record:
chr1	1058071	1058321	SCC9_CDK2AP1_OE_CHD4_R2_peak_11	25	.	4.23546	5.75554	2.54636	100

***** WARNING: File SCC9_CDK2AP1_OE_CHD4_R2_peaks.narrowPeak has inconsistent naming convention for record:
chr1	1058071	1058321	SCC9_CDK2AP1_OE_CHD4_R2_peak_11	25	.	4.23546	5.75554	2.54636	100

[JoseEspinosa]

Not sure what's going on... do you have any hints @drpatelh of what could be the reason behind this issue?

[JoseEspinosa]

Someone in my group came across a similar issue yesterday and I just realized that the problem here could be similar. See this. Although in your case, you are not using the storeDir it might be that there is some latency until the file gets available in your system, if it is too big. However, I can not fully understand why the process is set to be finished before the file is created. Maybe @drpatelh has a better idea

[drpatelh]

No idea. But yes, I suspect it could be due to latency when writing the file. Might need some more info including the .nextflow.log file, commands and any configs. Maybe we close this and re-open if the problem persists?