Presumable Installation Problem: Wall of Errors

[dpphill1]

Hi,
I have been trying for the last five hours to get this tool installed properly. After finally being able to run it without an immediate error, I thought the code was finally working. But I was wrong. It run for about half an hour before finishing but when I checked the output it was a very long list of errors (with lots of repetition). Here's the top:

[TRIO INFO] * Clir3-Trio pipeline start
[TRIO INFO] * 0 Check environment variables
[INFO] --include_all_ctgs not enabled, use chr{1..22,X,Y} and {1..22,X,Y} by default
[INFO] Call variant in contigs: chr1 chr2 chr3 chr4 chr5 chr6 chr7 chr8 chr9 chr10 chr11 chr12 chr13 chr14 chr15 chr16 chr17 chr18 chr19 chr20 chr21 chr22 chrX chrY
[INFO] Chunk number for each contig: 50 49 40 39 37 35 32 30 28 27 28 27 23 22 21 19 17 17 12 13 10 11 32 12
[TRIO INFO] * 1 call variants using pileup model
trio sample SAMPLE_C SAMPLE_P1 SAMPLE_P2
trio bam /rds/general/project/cebolalab_clinical_genomics/live/PRDM5_LSEC_ERG_KO/NanoPoreData/alignments/filtered_barcode01_mapped.sorted.bam /rds/general/project/cebolalab_clinical_genomics/live/PRDM5_LSEC_ERG_KO/NanoPoreData/alignments/filtered_barcode02_mapped.sorted.bam /rds/general/project/cebolalab_clinical_genomics/live/PRDM5_LSEC_ERG_KO/NanoPoreData/alignments/filtered_barcode03_mapped.sorted.bam
pileup threads 1
running pileup model
[INFO] Check environment variables
[INFO] --include_all_ctgs not enabled, use chr{1..22,X,Y} and {1..22,X,Y} by default
[INFO] Call variant in contigs: chr1 chr2 chr3 chr4 chr5 chr6 chr7 chr8 chr9 chr10 chr11 chr12 chr13 chr14 chr15 chr16 chr17 chr18 chr19 chr20 chr21 chr22 chrX chrY
[INFO] Chunk number for each contig: 50 49 40 39 37 35 32 30 28 27 28 27 23 22 21 19 17 17 12 13 10 11 32 12
[INFO] 1/7 Call variants using pileup model
[INFO] Delay 0 seconds before starting variant calling ...
[E::hts_parse_region] Coordinates must be > 0
[E::mpileup] fail to parse region 'chr1:-33-4979162' with /rds/general/project/cebolalab_clinical_genomics/live/PRDM5_LSEC_ERG_KO/NanoPoreData/alignments/filtered_barcode02_mapped.sorted.bam
Traceback (most recent call last):
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/bin/clair3/../clair3.py", line 112, in <module>
    main()
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/bin/clair3/../clair3.py", line 99, in main
    submodule = import_module("%s.%s" % (directory, submodule_name))
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/bin/clair3/CallVariants.py", line 5, in <module>
    import tensorflow as tf
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 64, in <module>
    from tensorflow.python.framework.framework_lib import *  # pylint: disable=redefined-builtin
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/tensorflow/python/framework/framework_lib.py", line 25, in <module>
    from tensorflow.python.framework.ops import Graph
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 47, in <module>
    from tensorflow.python.eager import context
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/tensorflow/python/eager/context.py", line 28, in <module>
    from absl import logging
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/absl/logging/__init__.py", line 97, in <module>
    from absl import flags
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/absl/flags/__init__.py", line 35, in <module>
    from absl.flags import _argument_parser
  File "/rds/general/user/dpphill1/home/anaconda3/envs/clair3-trio/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 82, in <module>
    class ArgumentParser(Generic[_T], metaclass=_ArgumentParserCache):
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases
[INFO] Delay 4 seconds before starting variant calling ...
Traceback (most recent call last):

Package (for this attempt) was installed using the bioconda method. Besides the models not being preinstalled in /bin/ and having to supply them myself, I thought it looked OK. Script looks like this:

run_clair3_trio.sh \
  --bam_fn_c=<longpath>/filtered_barcode01_mapped.sorted.bam \
  --bam_fn_p1=<longpath>/filtered_barcode02_mapped.sorted.bam \
  --bam_fn_p2=<longpath>/filtered_barcode03_mapped.sorted.bam \
  --output=<longpath>/variantCalls/clair3 \
  --ref_fn=<longpath>/GRCh38.p14.genome.fa \
  --threads=8 \
  --model_path_clair3=<longpath>/r1041_e82_400bps_sup_v400 \
  --model_path_clair3_trio=<longpath>/c3t_hg002_dna_r1041_e82_400bps_sup 

Anything obvious? I'd really like to get this to work rather than switching to a different tool.
Many thanks!
Dan

[sujunhao]

Hi Dan,

I'm sorry to know that the installation cost a lot of time. The log you provided indicates that the tool tries to call variants out of the defined region "[E::mpileup] fail to parse region 'chr1:-33-4979162' with". the "-33" position is illegal for Clair3-Trio and Samtools's mpileup. Clair3-Trio has a position boundary check, however, we might leave cases that cause bugs in our pipeline.

Could you please share your complete log and bam with us (to email address [email protected]) for further debugging on our side? Bam files for alignment in chr1 should be enough for debugging purposes.

On the other hand, one way to avoid the error is to provide bed files to tools like "--bed_fn=${_INPUT_DIR}/quick_demo.bed".
You can also try to use our tools in Singulary or Docker (https://github.com/HKU-BAL/Clair3-Trio?tab=readme-ov-file#option-2-singularity), which is more convenient for use compared to using the bioconda one.

Regards,
Junhao

[dpphill1]

Hi Junhao,
Thanks for the quick reply. I had already tried Singularity on our HPC and couldn't get it to work either, and so returned to trying an anaconda installation.

Unfortunately I don't think I can share any data as this is patient data. I did neglect to mention that the data comes from adaptive sampling (about 5% of the genome) and so was optimistic supplying a BED to bed_fun would solve the issue. Unfortunately, it did not resolve the issue. The error is the same:

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

I think I'll have to pivot to a different tool in the meantime but I'd like to ultimately get this one working as it is my preference. I'll retry the Singularity approach again today too.

Many thanks,
Daniel

[sujunhao]

Hi Daniel,

The error line "TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases" seems from the TensorFlow install problem.

I rechecked the bioconda installation script and found a version typo in the installation script. Sorry in case this is the reason for the inconvenience. I fixed it in README, you may try to reinstall the package via bioconda.

On the other hand, we recommend using our package via singularity, it should be the most convenient one. If you want to dig into the package, we recommend installing via Anaconda.

Regards,
Junhao