GATK Docker sets $HOME to /root which can result in failures on write #4782

Tintest · 2018-05-18T11:07:19Z

Hello,

I'm trying to use the CNVGermline Pipeline with Nextflow and a GATK Singularity image pulled from your Docker image.

As you can see, I'm trying to run the DetermineGermlineContigPloidy with 57 samples, and I got a permission error within my Singularity container.

This error is directly related to Singularity permissions to create a directory ('/root/.theano/compiledir_Linux-4.10--generic-x86_64-with-debian-stretch-sid-x86_64-3.6.2-64'), because it's running fine with Singularity when I'm root and Docker (without root). So it's not really a GATK4 problem but more a singularity-GATK4 related problem.

Maybe your GATK4 Germline CNV calling pipeline is not designed to work as a Singularity container (which is quite understandable), because for tools like HaplotypeCaller, MarkDuplicates ... It's working fine.

I didn't know where to post this error, singularity or GATK github, so why not both !

Do you have any idea make it run properly ? Change a bit the design of your CNV pipeline to make it compatible with Singularity ?

Version of softwares:

Singularity : 2.5.1, GATK : 4.0.4.0

Command

Singularity :
singularity build gatk-4.0.4.0.img docker://broadinstitute/gatk:4.0.4.0

GATK4 :
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /gatk/build/libs/gatk-package-4.0.4.0-local.jar DetermineGermlineContigPloidy --input 2044098202-8046_S5_sample.counts.hdf5 --input 2045946179-9076_S2_sample.counts.hdf5 --input 2045946166-9075_S1_sample.counts.hdf5 --input 2048220927-11022_S4_sample.counts.hdf5 --input 2045599261-9046ci_S1_sample.counts.hdf5 --input 2046745668-1007_S5_sample.counts.hdf5 --input 2044098101-8043_S2_sample.counts.hdf5 --input 2044098168-8044_S3_sample.counts.hdf5 --input 2046746598-1012_S4_sample.counts.hdf5 --input 2044395763-8064ci_S4_sample.counts.hdf5 --input 2044395647-8061ci_S1_sample.counts.hdf5 --input 70-20-CI_S3_sample.counts.hdf5 --input 70-18-CI_S1_sample.counts.hdf5 --input 2044395782-8065ci_S5_sample.counts.hdf5 --input 2048222393-11027_S3_sample.counts.hdf5 --input 2048220452-11020_S2_sample.counts.hdf5 --input 2048222667-11031_S7_sample.counts.hdf5 --input 2044395678-8062ci_S2_sample.counts.hdf5 --input 2044395611-8060_S6_sample.counts.hdf5 --input 2048221956-11025_S1_sample.counts.hdf5 --input 2045946131-9071ci_S6_sample.counts.hdf5 --input 2045599349-9047ci_S2_sample.counts.hdf5 --input 2045599158-9045ci_S6_sample.counts.hdf5 --input 2045946149-9073ci_S5_sample.counts.hdf5 --input 2048220431-11019_S1_sample.counts.hdf5 --input 2045946141-9072ci_S4_sample.counts.hdf5 --input 2046746758-1013_S5_sample.counts.hdf5 --input 2045598958-9043ci_S4_sample.counts.hdf5 --input 2046745451-1006_S4_sample.counts.hdf5 --input 2048220997-11023_S5_sample.counts.hdf5 --input 2044097928-8042_S1_sample.counts.hdf5 --input 2045598560-9041ci_S2_sample.counts.hdf5 --input 70-21-CI_S4_sample.counts.hdf5 --input 70-19-CI_S2_sample.counts.hdf5 --input 2046745923-1008_S6_sample.counts.hdf5 --input 2045598771-9042ci_S3_sample.counts.hdf5 --input 2048220861-11021_S3_sample.counts.hdf5 --input 204409765380-40Ci_S6_sample.counts.hdf5 --input 2048221977-11026_S2_sample.counts.hdf5 --input 2048222584-11029_S5_sample.counts.hdf5 --input 2046746353-1010_S2_sample.counts.hdf5 --input 2045946158-9074ci_S6_sample.counts.hdf5 --input 2045946119-9070ci_S5_sample.counts.hdf5 --input 2045599063-9044ci_S5_sample.counts.hdf5 --input 2046746482-1011_S3_sample.counts.hdf5 --input 204409784980-41ci_S7_sample.counts.hdf5 --input 2048222477-11028_S4_sample.counts.hdf5 --input 2045598355-9040ci_S1_sample.counts.hdf5 --input 2048221935-11024_S6_sample.counts.hdf5 --input 2044098183-8045_S4_sample.counts.hdf5 --input 2045946104-9069ci_S4_sample.counts.hdf5 --input 2048222618-11030_S6_sample.counts.hdf5 --input 2046747002-1014_S6_sample.counts.hdf5 --input 2044395737-8063ci_S3_sample.counts.hdf5 --input 2045946189-9077_S3_sample.counts.hdf5 --input 2045599415-9048ci_S3_sample.counts.hdf5 --input 2046746162-1009_S1_sample.counts.hdf5 --contig-ploidy-priors /home/qtestard/PROJECTS/aurexome/REF/common/prior_contig_diploidy.tsv --output /home/qtestard/PROJECTS/aurexome/SAMPLE/CNV_calling/CNV/CNV_calling_NaiveDetermineGermlineContigPloidy --output-prefix CNV_calling_NaiveDetermineGermlineContigPloidy

Actual behavior

PermissionError: [Errno 13] Permission denied: '/root/.theano'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/gcnvkernel/__init__.py", line 1, in <module>
      from pymc3 import __version__ as pymc3_version
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/pymc3/__init__.py", line 5, in <module>
      from .distributions import *
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/pymc3/distributions/__init__.py", line 1, in <module>
      from . import timeseries
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/pymc3/distributions/timeseries.py", line 1, in <module>
      import theano.tensor as tt
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/__init__.py", line 66, in <module>
      from theano.compile import (
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/compile/__init__.py", line 10, in <module>
      from theano.compile.function_module import *
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/compile/function_module.py", line 21, in <module>
      import theano.compile.mode
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/compile/mode.py", line 10, in <module>
      import theano.gof.vm
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/gof/vm.py", line 662, in <module>
      from . import lazylinker_c
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/gof/lazylinker_c.py", line 42, in <module>
      location = os.path.join(config.compiledir, 'lazylinker_ext')
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/confith.join(config.compiledir, 'lazylinker_ext')
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configparser.py", line 333, in __get__
      self.__set__(cls, val_str)
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configparser.py", line 344, in __set__
      self.val = self.filter(val)
    File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configdefaults.py", line 1745, in filter_compiledir
      " '%s'. Check the permissions." % path)
  ValueError: Unable to create the compiledir directory '/root/.theano/compiledir_Linux-4.10--generic-x86_64-with-debian-stretch-sid-x86_64-3.6.2-64'. Check the permissions.

        at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
        at org.broadinstitute.hellbender.utils.runtime.ScriptExecutor.executeCuratedArgs(ScriptExecutor.java:126)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeArgs(PythonScriptExecutor.java:170)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeCommand(PythonScriptExecutor.java:79)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.checkPythonEnvironmentForPackage(PythonScriptExecutor.java:192)
        at org.broadinstitute.hellbender.tools.copynumber.DetermineGermlineContigPloidy.onStartup(DetermineGermlineContigPloidy.java:240)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:132)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
        at org.broadinstitute.hellbender.Main.main(Main.java:289)

Steps to reproduce behavior

Run the DetermineGermlineContigPloidy tool within a Singularity container without root permission.

Thank you.

My thread on Singularity Github #1557

The text was updated successfully, but these errors were encountered:

Tintest · 2018-05-18T18:34:01Z

@jmstover smartly outlined in his post on the Singularity thread that having a tool asking for the /root directory should not be a normal behavior and might be hardcoded. That's why the command is successful when i'm using the container as root.

If you could change the /root directory with the /home/user one for the theano software, it would be great.

Thank you

samuelklee · 2018-05-18T19:05:51Z

Thanks for bringing this to our attention, @Tintest. I think that we may be able to address this by setting base_compiledir via os.environ["THEANO_FLAGS"] appropriately (see http://deeplearning.net/software/theano/library/config.html). @mbabadi @cmnbroad any thoughts?

In any case, thanks for trying out the GermlineCNVCaller pipeline. You may have to tune some parameters, depending on your data type. You may find the following discussions helpful:

https://gatkforums.broadinstitute.org/gatk/discussion/11711/germlinecnvcaller-interval-merging-rule-error

#4719

Note that we're still in beta, but our preliminary evaluations have demonstrated improved performance over other callers in both WES and WGS.

Tintest · 2018-05-18T19:43:12Z

Thank you for your links :)

I hoped you will be able to add the change in the next GATK version :)

If I can run this tool with singularity, I will be able to test a ton of different parameter !

cmnbroad · 2018-05-21T13:08:45Z

@samuelklee os.environ would probably work, though it might be easier to set theano.config.base_compiledir directly so you don't have to worry about clobbering any other flags.

shengqh · 2018-10-26T15:09:32Z

I downloaded the most recently gatk docker file yesterday and I met this problem today. I am wondering if you have solved this request? Thank you so much.

/gatk/gatk-package-4.0.11.0-local.jar
singularity 2.5.0-dist

sooheelee · 2018-10-26T16:25:19Z

Hi @shengqh, did you activate the gatk python environment within your Singularity container with source activate gatk? GATK CNV tools require this environment and it appears that launching the Singularity container does not automatically activate this. Docker containers automatically activate this environment.

shengqh · 2018-10-26T17:07:27Z

@sooheelee

Here is my command:

singularity exec gatk.simg test.pbs

Here is test.pbs:

gatk DetermineGermlineContigPloidy \
            -L filtered.interval_list \
             --input A1.count.hdf5 --input A2.count.hdf5 \
            --contig-ploidy-priors contig_ploidy_priors_homo_sapiens.tsv \
            --interval-merging-rule OVERLAPPING_ONLY \
            --output out \
            --output-prefix exomeseq \
            --verbosity DEBUG \
            --mean-bias-standard-deviation 0.01 \
            --mapping-error-rate 0.01 \
            --global-psi-scale 0.001 \
            --sample-psi-scale 0.0001

Here is the error message and I think the python environment has been activated by Singularity. The task failed when it tried to create directory of '/root/.theano'.

17:03:28.891 INFO  DetermineGermlineContigPloidy - Initializing engine
17:03:28.896 DEBUG ScriptExecutor - Executing:
17:03:28.896 DEBUG ScriptExecutor -   python
17:03:28.896 DEBUG ScriptExecutor -   -c
17:03:28.896 DEBUG ScriptExecutor -   import gcnvkernel

Traceback (most recent call last):
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configdefaults.py", line 1738, in filter_compiledir
    os.makedirs(path, 0o770)  # read-write-execute for user and group
  File "/opt/miniconda/envs/gatk/lib/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/opt/miniconda/envs/gatk/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/root/.theano'

sooheelee · 2018-10-26T17:53:27Z

You can tell if the python environment is active because your bash prompt will be prefaced with (gatk) as you see here from my testing:

shengqh · 2018-10-26T21:27:52Z

@sooheelee You are right, the python environment is not activated automatically. However, I manually activated the environment and then run my test.pbs, I still got the error.

[shengq2@cqs1 singularity]$ singularity shell gatk.simg
Singularity: Invoking an interactive shell within container...
Singularity gatk.simg:/scratch/cqs/softwares/singularity> source activate gatk
(gatk) Singularity gatk.simg:/scratch/cqs/softwares/singularity> sh test.pbs

...
21:27:03.205 INFO  DetermineGermlineContigPloidy - Initializing engine
21:27:03.210 DEBUG ScriptExecutor - Executing:
21:27:03.210 DEBUG ScriptExecutor -   python
21:27:03.210 DEBUG ScriptExecutor -   -c
21:27:03.210 DEBUG ScriptExecutor -   import gcnvkernel

Traceback (most recent call last):
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configdefaults.py", line 1738, in filter_compiledir
    os.makedirs(path, 0o770)  # read-write-execute for user and group
  File "/opt/miniconda/envs/gatk/lib/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/opt/miniconda/envs/gatk/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/root/.theano'

jmstover · 2018-10-26T21:31:03Z

@shengqh
Looking at it, it seems that test.pbs is basically a shell script. So at the top of that add:

source activate gatk

Or... whatever it is you need to do to activate the environment. What the exec command is going to do is launch whatever is passed in. So... something like:

singularity exec gatk.simg cat /etc/os-release

Would show you the /etc/os-release file from within the container. exec isn't going to load a shell environment, but launch the application directly. In a shell look at: /.singularity.d/actions/exec

shengqh · 2018-10-26T21:39:11Z

@jmstover Thank you for your recommendation. I added the following line in the beginning of test.pbs

source activate gatk

However, running the following command still threw the same error.

singularity exec gatk.simg ./test.pbs

jmstover · 2018-10-26T22:12:03Z

Are you building this from the docker broadinstitute/gatk image?

If so, I've found an error in it (@sooheelee, @samuelklee )... the Docker Manifest has set:

export HOME="/root"

... That's going to make some things not work if you're using it as a non-root user. If you are using that, try building from a Singularity definition file that looks like:

Bootstrap: docker
From: broadinstitute/gatk

%environment
    export HOME=/path/to/home

%runscript
    bash -c "source activate gatk; /gatk/gatk \"$@\""

I'm still testing this since I don't know the gatk software at all ... but with setting HOME in %environment you'll be overriding what the Docker manifest is setting. Setting %runscript you can then execute gatk by running:

singularity run gatk.simg DetermineGermlineContigPloidy \
            -L filtered.interval_list \
             --input A1.count.hdf5 --input A2.count.hdf5 \
            --contig-ploidy-priors contig_ploidy_priors_homo_sapiens.tsv \
            --interval-merging-rule OVERLAPPING_ONLY \
            --output out \
            --output-prefix exomeseq \
            --verbosity DEBUG \
            --mean-bias-standard-deviation 0.01 \
            --mapping-error-rate 0.01 \
            --global-psi-scale 0.001 \
            --sample-psi-scale 0.0001

Or if gatk.simg is executable, by just calling that image file if run-singularity is in your path. You just need to pass the options to gatk, not call gatk. So that would be:

./gatk.simg DetermineGermlineContigPloidy [...]

shengqh · 2018-10-26T22:36:56Z

@jmstover Yes, I pulled from broadinstiture/gatk.

singularity build gatk.simg docker://broadinstitute/gatk

I don't have root permission to build Singularity image from definition file now. I may need to install a virtual machine first to test it. However, just like the activation of environment, I added the export command in my test.pbs and it worked. :-)

export HOME=/scratch/cqs/shengq2
source activate gatk

jmstover · 2018-10-26T22:43:08Z

Good to hear. 👍

sooheelee · 2018-10-29T17:18:29Z

Yes, glad you found a workaround. 👍🏽

cmnbroad · 2018-10-29T19:35:55Z

Although there is a workaround, ideally we'd remove the assumption from our docker that it can access the root user's home dir.

samuelklee · 2019-02-25T18:01:06Z

@cmnbroad @droazen will there be any action on this? I'm not sure if #5714 might be related (although I'm not sure why we get a ulimit message there instead of a permission denied)?

droazen · 2019-02-27T19:57:34Z

@samuelklee I can have someone take another look at this, though it seems unlikely to me that it's the same problem reported in #5714

samuelklee · 2019-02-27T20:23:17Z

Thanks for looking into it! I guess Cromwell also essentially sets HOME, which is probably why we don't typically run into this. Do we have any non-Cromwell Docker tests that could catch a regression?

drifty914 · 2019-02-27T20:53:10Z

@samuelklee I forgot to note in #5714 that I had to manually set the path for HOME (to a mounted folder) when using the Docker image as it defaulted to ROOT and would complain otherwise.

samuelklee · 2019-02-27T20:55:47Z

OK, thanks @drifty914, that's very useful to know. Was that when using the Docker image with or without Cromwell---hopefully just the latter?

drifty914 · 2019-02-27T21:12:55Z

@samuelklee that was when using the Docker image with Cromwell (v34).

Tintest · 2019-05-26T14:59:15Z

Hello,

I did encounter the same behaviour that I reported one year ago with the official Docker GATK 4.1.0.0 container converted into a singularity image.

Version of softwares:

Singularity : 2.5.1, GATK : 4.1.0.0

Command

Singularity :
singularity build gatk-4.0.4.0.img docker://broadinstitute/gatk:4.0.4.0

Actual behavior

14:39:21.762 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.1.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
14:39:24.079 INFO  DetermineGermlineContigPloidy - ------------------------------------------------------------
14:39:24.079 INFO  DetermineGermlineContigPloidy - The Genome Analysis Toolkit (GATK) v4.1.0.0
14:39:24.080 INFO  DetermineGermlineContigPloidy - For support and documentation go to https://software.broadinstitute.org/gatk/
14:39:24.081 INFO  DetermineGermlineContigPloidy - Executing as tintest@dahu38 on Linux v4.9.0-8-amd64 amd64
14:39:24.081 INFO  DetermineGermlineContigPloidy - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_191-8u191-b12-0ubuntu0.16.04.1-b12
14:39:24.081 INFO  DetermineGermlineContigPloidy - Start Date/Time: May 26, 2019 2:39:21 PM UTC
14:39:24.081 INFO  DetermineGermlineContigPloidy - ------------------------------------------------------------
14:39:24.081 INFO  DetermineGermlineContigPloidy - ------------------------------------------------------------
14:39:24.082 INFO  DetermineGermlineContigPloidy - HTSJDK Version: 2.18.2
14:39:24.082 INFO  DetermineGermlineContigPloidy - Picard Version: 2.18.25
14:39:24.083 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.COMPRESSION_LEVEL : 2
14:39:24.083 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
14:39:24.083 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
14:39:24.083 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
14:39:24.083 INFO  DetermineGermlineContigPloidy - Deflater: IntelDeflater
14:39:24.083 INFO  DetermineGermlineContigPloidy - Inflater: IntelInflater
14:39:24.083 INFO  DetermineGermlineContigPloidy - GCS max retries/reopens: 20
14:39:24.083 INFO  DetermineGermlineContigPloidy - Requester pays: disabled
14:39:24.083 INFO  DetermineGermlineContigPloidy - Initializing engine
14:39:26.111 INFO  DetermineGermlineContigPloidy - Shutting down engine
[May 26, 2019 2:39:26 PM UTC] org.broadinstitute.hellbender.tools.copynumber.DetermineGermlineContigPloidy done. Elapsed time: 0.07 minutes.
Runtime.totalMemory()=1511522304
org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException:
python exited with 1
Command Line: python -c import gcnvkernel

Stdout:
Stderr: Traceback (most recent call last):
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configdefaults.py", line 1738, in filter_compiledir
    os.makedirs(path, 0o770)  # read-write-execute for user and group
  File "/opt/miniconda/envs/gatk/lib/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/opt/miniconda/envs/gatk/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/root/.theano'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/gcnvkernel/__init__.py", line 1, in <module>
    from pymc3 import __version__ as pymc3_version
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/pymc3/__init__.py", line 5, in <module>
    from .distributions import *
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/pymc3/distributions/__init__.py", line 1, in <module>
    from . import timeseries
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/pymc3/distributions/timeseries.py", line 1, in <module>
    import theano.tensor as tt
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/__init__.py", line 66, in <module>
    from theano.compile import (
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/compile/__init__.py", line 10, in <module>
    from theano.compile.function_module import *
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/compile/function_module.py", line 21, in <module>
    import theano.compile.mode
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/compile/mode.py", line 10, in <module>
    import theano.gof.vm
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/gof/vm.py", line 662, in <module>
    from . import lazylinker_c
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/gof/lazylinker_c.py", line 42, in <module>
    location = os.path.join(config.compiledir, 'lazylinker_ext')
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configparser.py", line 333, in __get__
    self.__set__(cls, val_str)
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configparser.py", line 344, in __set__
    self.val = self.filter(val)
  File "/opt/miniconda/envs/gatk/lib/python3.6/site-packages/theano/configdefaults.py", line 1745, in filter_compiledir
    " '%s'. Check the permissions." % path)
ValueError: Unable to create the compiledir directory '/root/.theano/compiledir_Linux-4.9--amd64-x86_64-with-debian-stretch-sid-x86_64-3.6.2-64'. Check the permissions.

        at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
        at org.broadinstitute.hellbender.utils.runtime.ScriptExecutor.executeCuratedArgs(ScriptExecutor.java:126)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeArgs(PythonScriptExecutor.java:170)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeCommand(PythonScriptExecutor.java:79)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.checkPythonEnvironmentForPackage(PythonScriptExecutor.java:192)
        at org.broadinstitute.hellbender.tools.copynumber.DetermineGermlineContigPloidy.onStartup(DetermineGermlineContigPloidy.java:269)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:136)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
        at org.broadinstitute.hellbender.Main.main(Main.java:291)
Using GATK jar /gatk/gatk-package-4.1.0.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /gatk/gatk-package-4.1.0.0-local.jar DetermineGermlineContigPloidy --input /bettik/tintest/PROJECTS/Test_nextflow_OAR/SAMPLE/test_grexome/cnv/tsv/0001/0001_grexome0505.tsv --input /bettik/tintest/PROJECTS/Test_nextflow_OAR/SAMPLE/test_grexome/cnv/tsv/0001/0001_grexome0507.tsv --input /bettik/tintest/PROJECTS/Test_nextflow_OAR/SAMPLE/test_grexome/cnv/tsv/0001/0001_grexome0498.tsv --model /bettik/tintest/PROJECTS/Test_nextflow_OAR/REF/hg38/CNV/DetermineGermlineContigPloidy/0001_DetermineGermlineContigPloidy --output /bettik/tintest/PROJECTS/Test_nextflow_OAR/SAMPLE/test_grexome/cnv/DetermineGermlineContigPloidy/0001_DetermineGermlineContigPloidy/ --output-prefix 0001_DetermineGermlineContigPloidy

Thank you.

shengqh · 2019-07-16T16:17:01Z

@Tintest I also met this problem. However, I solved it by "export HOME" before calling gatk command in my shell file call_gatk.sh

export HOME=/scratch/cqs/shengq2/temp/
source activate gatk
gatk --java-options "-Xmx40G" DetermineGermlineContigPloidy \
  ...

And, here is my shell file test.sh to call singularity :

export R_LIBS=
export PYTHONPATH=
export JAVA_HOME=

singularity exec gatk.4.1.0.0.simg  bash /scratch/cqs/shengq2/temp/call_gatk.sh

Putting the "export HOME" in test.sh with "export JAVA_HOME" before calling singularity didn't work. You have to put it in call_gatk.sh.

samuelklee · 2021-08-06T21:28:32Z

#6525 is still open and I don't know the plans for it.

Tintest mentioned this issue May 18, 2018

Singularity GATK4 permission problem error apptainer/singularity#1557

Closed

cmnbroad added the Python label May 21, 2018

samuelklee self-assigned this Jun 4, 2018

samuelklee added the Copy Number tools label Jun 4, 2018

cmnbroad assigned cmnbroad and unassigned samuelklee Oct 29, 2018

cmnbroad added the Docker label Oct 29, 2018

cmnbroad changed the title ~~Singularity Germline CNV caller pipeline permission problem error~~ GATK Docker sets $HOME to /root which can result in failures on write Oct 29, 2018

samuelklee mentioned this issue Feb 25, 2019

gCNV Open file limits exceeded at GermlineCNVCallerCohortMode step #5714

Closed

droazen added this to the Engine-Q12019 milestone Feb 27, 2019

This was referenced Oct 29, 2019

ramdom BUGS for germline copy number variants calling with GATK 4. PythonScriptExecutorException #6235

Closed

Added ability to override THEANO_FLAGS environment variable in gCNV tools. #6244

Merged

samuelklee linked a pull request Mar 25, 2020 that will close this issue

(DO NOT MERGE) Added non-root user to Dockerfile. #6525

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GATK Docker sets $HOME to /root which can result in failures on write #4782

GATK Docker sets $HOME to /root which can result in failures on write #4782

Tintest commented May 18, 2018 •

edited

Loading

Tintest commented May 18, 2018

samuelklee commented May 18, 2018 •

edited

Loading

Tintest commented May 18, 2018

cmnbroad commented May 21, 2018 •

edited

Loading

shengqh commented Oct 26, 2018

sooheelee commented Oct 26, 2018 •

edited

Loading

shengqh commented Oct 26, 2018

sooheelee commented Oct 26, 2018

shengqh commented Oct 26, 2018 •

edited

Loading

jmstover commented Oct 26, 2018 •

edited

Loading

shengqh commented Oct 26, 2018

jmstover commented Oct 26, 2018 •

edited

Loading

shengqh commented Oct 26, 2018

jmstover commented Oct 26, 2018

sooheelee commented Oct 29, 2018

cmnbroad commented Oct 29, 2018

samuelklee commented Feb 25, 2019

droazen commented Feb 27, 2019

samuelklee commented Feb 27, 2019

drifty914 commented Feb 27, 2019

samuelklee commented Feb 27, 2019

drifty914 commented Feb 27, 2019

Tintest commented May 26, 2019 •

edited

Loading

shengqh commented Jul 16, 2019

samuelklee commented Aug 6, 2021

GATK Docker sets $HOME to /root which can result in failures on write #4782

GATK Docker sets $HOME to /root which can result in failures on write #4782

Comments

Tintest commented May 18, 2018 • edited Loading

Version of softwares:

Command

Actual behavior

Steps to reproduce behavior

Tintest commented May 18, 2018

samuelklee commented May 18, 2018 • edited Loading

Tintest commented May 18, 2018

cmnbroad commented May 21, 2018 • edited Loading

shengqh commented Oct 26, 2018

sooheelee commented Oct 26, 2018 • edited Loading

shengqh commented Oct 26, 2018

sooheelee commented Oct 26, 2018

shengqh commented Oct 26, 2018 • edited Loading

jmstover commented Oct 26, 2018 • edited Loading

shengqh commented Oct 26, 2018

jmstover commented Oct 26, 2018 • edited Loading

shengqh commented Oct 26, 2018

jmstover commented Oct 26, 2018

sooheelee commented Oct 29, 2018

cmnbroad commented Oct 29, 2018

samuelklee commented Feb 25, 2019

droazen commented Feb 27, 2019

samuelklee commented Feb 27, 2019

drifty914 commented Feb 27, 2019

samuelklee commented Feb 27, 2019

drifty914 commented Feb 27, 2019

Tintest commented May 26, 2019 • edited Loading

Version of softwares:

Command

Actual behavior

shengqh commented Jul 16, 2019

samuelklee commented Aug 6, 2021

Tintest commented May 18, 2018 •

edited

Loading

samuelklee commented May 18, 2018 •

edited

Loading

cmnbroad commented May 21, 2018 •

edited

Loading

sooheelee commented Oct 26, 2018 •

edited

Loading

shengqh commented Oct 26, 2018 •

edited

Loading

jmstover commented Oct 26, 2018 •

edited

Loading

jmstover commented Oct 26, 2018 •

edited

Loading

Tintest commented May 26, 2019 •

edited

Loading