Travis CI Test Implementation

[apeltzer]

The EAGER 2.0 pipeline will have fully automatic consistency tests

• Human Genome, autosomal (hg19)
• Mitochondrial dataset
• An exemplary bacterial genome

I'll create test reference genomes with a couple of KB in size to test things quickly and then

[jfy133]

I'll look for:
non-human shotgun/mitochondria (assuming you still want human specific stuff)
low-complexity bacterial sample (probably pestis)
high-complexity bacteria sample (some calculus thing)

[apeltzer]

Yeah, need to update this ticket still, but this seems reasonable! We'll have some specific tweaks for human vs bacterial vs other stuff, but I'd say for most things it should be fairly identical.

[jfy133]

Actually, speaking of which. How do you envision the tweaks between human/bacterial/other?

Is it worth having default modifications that way, or is it better to let the user decide?

Just curious, as IIRC the only difference between human and bacterial in EAGER 1.9 was inclusion of QF values in report table.

[apeltzer]

Honestly, I don't see that much differences as you said already - the basic processing is identical, with some specific tools for human (e.g. schmutzi for example) and some specific tools for bacteria later (e.g. genConS or similar). One could in theory provide certain profiles that won't run these tools e.g. in the wrong cases but instead keep everything else identical.

Yes, thats mostly it. There were ideas to make things even smarter, but that resulted in an entire bunch of things that we'd need to implement for minimal user experience effects... so we instead focused on other things...

[jfy133]

Exactly. Then I suggest we remove organism-specific profiles. Maybe in the documentation we can provide 'suggested' or 'example' settings which the user can then input accordingly?

That'll save you some coding work too ;)

If you agree - do we still need the different organisms as test data?

[apeltzer]

I'm all in on that one

[jfy133]

eager2_mammoth_testing_package.tar.gz

Contains:

## Reference Mammoth MT genome
mammoth_testing_package/  
mammoth_testing_package/reference_genome/  
mammoth_testing_package/reference_genome/Mammoth_MT_Krause.fasta  

## EAGER 1.92.55 config file and subsequent (expected) results
mammoth_testing_package/eager_v1_9_55/  
mammoth_testing_package/eager_v1_9_55/2018-06-23-12-51-EAGER.xml  
mammoth_testing_package/eager_v1_9_55/Report_output_EAGER_v1_9_55.csv  

## Mammoth JK2782 HiSeq MT Capture library (~10,000 reads after merging) from Fellows Yates (2017) Sci. Rep
mammoth_testing_package/fastq_data/  
mammoth_testing_package/fastq_data/JK2782_TGGCCGATCAACGA_L008_R2_001.fastq.gz.tengrand.fq  
mammoth_testing_package/fastq_data/JK2782_TGGCCGATCAACGA_L008_R1_001.fastq.gz.tengrand.fq 

Let me know if that works/is useful.

Edit:

File sizes:

-rw-r--r-- 1 fellows domain users 5.4K Jun 23 13:02 eager_v1_9_55/2018-06-23-12-51-EAGER.xml
-rw-r--r-- 1 fellows domain users  690 Jun 23 12:55 eager_v1_9_55/Report_output_EAGER_v1_9_55.csv
-rw-r--r-- 1 fellows domain users 656K Jun 23 12:54 fastq_data/JK2782_TGGCCGATCAACGA_L008_R1_001.fastq.gz.tengrand.fq.gz
-rw-r--r-- 1 fellows domain users 704K Jun 23 12:54 fastq_data/JK2782_TGGCCGATCAACGA_L008_R2_001.fastq.gz.tengrand.fq.gz
-rw-r--r-- 1 fellows domain users  17K Jun 23 12:55 reference_genome/Mammoth_MT_Krause.fasta

[apeltzer]

Thats perfect! Thank you James! I will check this in under https://github.com/nf-core/test-datasets and use it for setting up testcases - probably even today as you took your time for this too :-)

[apeltzer]

Tests are up and running 🎉