Multithread / multicore

[AxVE]

Is there a way to specify the number of threads or cores to use ? Specifically for HaplotypeCaller ?
In GATK3, we had -nt and -nct (more of less reliable for what I read) but it doesn't seems to be here anymore.

[magicDGS]

For the new GATK framework the multi-thread support is through Spark (see #2345 for more details).

[AxVE]

Ok , thanks for the info ^^

[AxVE]

So it seems we can't set multithreading for non-spark tools ?

[lbergelson]

Yes, that's true in general. The -nt / -ntc flags options were never very good in gatk3. They usually scaled very poorly with number of cores, and were the cause of a lot of complexity and bugs. We decided not to try to roll our own map reduce framework for gatk4, but use an existing much better one, ie spark.

We recommend multiprocess parallelism with an external job runner like cromwell if you want parallelism in tools that aren't ready in spark yet. This is more complicated to setup and run, but it results in much more efficient use of compute resources.

There are few limited multithreaded options remaining in GATK4 outside of spark. One specific one is the option to use multiple threads with HaplotypeCaller's pairHmm. This is only available on linux systems and defaults to using 4 threads.

[AxVE]

Thanks for the information.
For info, I asked because I'm currently benchmarking SGE cluster vs Spark usage.