Error when no peaks are found

[ktrns]

Hi there!

This is an exception that you might want to catch: the experiment of the client failed and we don't see much of an enrichment, and hence no peaks are found at all. Your pipeline crashes with "Error executing process > 'PeakQC'" (plot_macs_qc.r, plot_homer_annotatepeaks.r, cat peak_annotation_header.txt macs_annotatePeaks.summary.txt > macs_annotatePeaks.summary_mqc.tsv)

  Error in read.table(PeakFiles[idx], sep = "\t", header = FALSE) :
    no lines available in input
  Execution halted

Would you be able to fix this so that the pipeline can finish off even if no peaks are found?

Best wishes
Katrin

[drpatelh]

Hi @ktrns ! Yes, this is likely related to #35 😓 It will take quite a bit of refactoring to ignore this error and resume for the rest of the samples mainly because of the way the experimental design is incorporated into the pipeline for differential binding analysis etc.

For now, the simplest fix is to just remove that sample from your design file and re-run. Sorry.

Would you mind sending me the output for the number of lines in each of the peak files. So if you called broad peaks then it would be something like:
wc <OUTPUT_DIR>/results/bwa/mergedLibrary/macs/broadPeak/*.broadPeak

or for narrow peaks:
wc <OUTPUT_DIR>/results/bwa/mergedLibrary/macs/narrowPeak/*.narrowPeak

[ktrns]

Hi @drpatelh,

Sure: 0 for both non-control samples.
bwa/mergedLibrary/Macs/narrowPeak

So if I remove my zero-peak samples, there is nothing left. But never mind, for now I can provide the atacseq pipeline results! I was just curious to also run the chipseq pipeline.

Best wishes and many thanks
Katrin

[drpatelh]

Ah, so what did you provide as a control to the chipseq pipeline then? I'm assuming this is Cut N Run data?

[ivokwee]

I have the same problem. Is it maybe possible to skip this 'peakQC' step using some run options variable?

[drpatelh]

Quite busy at the moment with the viralrecon pipeline but I can come back to this hopefully next week to see if I can do something in the next release which isnt far off now 👍

However, it would be great if you can share a minimal dataset with me to be able to reproduce this error @ivokwee @ktrns? Just an IP and control will be enough 🙂

[ivokwee]

Can you at least add an option to skip this step? I think it should be in the list of skip QA/QC steps.

…

On Sat, May 30, 2020, 14:56 Harshil Patel ***@***.***> wrote: Quite busy at the moment with the viralrecon <https://github.com/nf-core/viralrecon> pipeline but I can come back to this hopefully next week to see if I can do something in the next release which isnt far off now 👍 However, it would be great if you can share a minimal dataset with me to be able to reproduce this error @ivokwee <https://github.com/ivokwee> @ktrns <https://github.com/ktrns>? Just an IP and control will be enough 🙂 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#128 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVULWTWJK64LALPHQAPFRDRUD66TANCNFSM4JUYX6MQ> .

[drpatelh]

Yes, I was planning to do that already but that isn't going to solve this problem because the pipeline will just fail at a later step as there are no peaks called. This is why I would need an appropriate dataset to test this on.

In the meantime, you can just remove the sample from your design file and re-run 👍

[ivokwee]

Well, I cloned the git code, disabled the peakQC module, and re-ran.

Here are some example files to test:
https://www.dropbox.com/s/8z0q68yajhxoqa8/nftest.tgz?dl=0ll

Also, the error is in read.table (it doesn't like an empty file). I guess we need to make dummy file with zero length read segments or something. or wrap each read with try().

Ivo

[drpatelh]

Great! Thanks for the really nice dataset to reproduce the error @ivokwee 😎 Saved me a bit of work. I have added the ability to --skip_peak_qc and --skip_peak_annotation.

But as I mentioned this is a more deep-rooted issue in the way that the pipeline is currently written. Basically, variables like replicatesExist and multipleGroups may be affected by the fact that some samples drop-out due to having 0 peaks. This would need to be taking into account immediately after the MACS2 processes for all samples have completed. Will need to revisit this at some point.

[j-andrews7]

Another bump for this thread to say this has been a frequent issue for our group, as we frequently FLAG-tag proteins and do FLAG ChIPs but include samples with no FLAG proteins as controls for the antibody.

[JoseEspinosa]

@drpatelh proposed on slack the following issue that if the files have 0 bytes we may be able to do something like this

[JoseEspinosa]

Should have been fixed in #268 I tested and works in my hands but will be nice if someone else could give it a try. I will close the issue by now.