-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add COMEbin as another optional binner #234
Conversation
Weird test failure: STDERR was: b'11/28/2024 02:00:39 AM INFO: Time - 02:00:39 28-11-2024
11/28/2024 02:00:39 AM INFO: Command - /usr/share/miniconda/envs/test/bin/aviary assemble -1 /home/runner/work/aviary/aviary/test/data/wgsim.1.fq.gz -2 /home/runner/work/aviary/aviary/test/data/wgsim.2.fq.gz --output /tmp/tmpnd6jzx19/test --conda-prefix /home/runner/work/aviary/aviary/test/data/.conda --dryrun --tmpdir /tmp/tmpnd6jzx19 --snakemake-cmds --quiet
11/28/2024 02:00:39 AM INFO: Version - 0.10.0
11/28/2024 02:00:39 AM INFO: Configuration file written to /tmp/tmpnd6jzx19/test/config.yaml
11/28/2024 02:00:39 AM INFO: Executing: snakemake --snakefile /home/runner/work/aviary/aviary/aviary/modules/Snakefile --directory /tmp/tmpnd6jzx19/test --cores 16 --rerun-incomplete --keep-going --quiet --rerun-triggers mtime --configfile /tmp/tmpnd6jzx19/test/config.yaml --nolock --retries 0 --conda-frontend mamba --use-conda --conda-prefix /home/runner/work/aviary/aviary/test/data/.conda --dryrun complete_assembly_with_qc
Traceback (most recent call last):
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/__init__.py", line 671, in snakemake
workflow.include(
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/workflow.py", line 1362, in include
snakefile = infer_source_file(snakefile, basedir)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/sourcecache.py", line 323, in infer_source_file
if is_local_file(path_or_uri):
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/common/__init__.py", line 92, in is_local_file
return parse_uri(path_or_uri).scheme == "file"
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/common/__init__.py", line 96, in parse_uri
from smart_open import parse_uri
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/__init__.py", line 34, in <module>
from .smart_open_lib import open, parse_uri, smart_open, register_compressor # noqa: E402
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/smart_open_lib.py", line 35, in <module>
from smart_open import doctools
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/doctools.py", line 21, in <module>
from . import transport
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/transport.py", line 104, in <module>
register_transport("smart_open.s3")
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/transport.py", line 49, in register_transport
submodule = importlib.import_module(submodule)
File "/usr/share/miniconda/envs/test/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/s3.py", line 25, in <module>
import boto3
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/boto3/__init__.py", line 17, in <module>
from boto3.session import Session
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
import botocore.session
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/session.py", line 26, in <module>
import botocore.client
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/client.py", line 15, in <module>
from botocore import waiter, xform_name
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/waiter.py", line 18, in <module>
from botocore.docs.docstring import WaiterDocstring
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/__init__.py", line 15, in <module>
from botocore.docs.service import ServiceDocumenter
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in <module>
from botocore.docs.client import (
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/client.py", line 18, in <module>
from botocore.docs.example import ResponseExampleDocumenter
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in <module>
from botocore.docs.shape import ShapeDocumenter
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in <module>
from botocore.utils import is_json_value_header
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/utils.py", line 39, in <module>
import botocore.httpsession
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/httpsession.py", line 45, in <module>
from urllib3.contrib.pyopenssl import (
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/urllib3/contrib/pyopenssl.py", line 43, in <module>
import OpenSSL.SSL # type: ignore[import-untyped]
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/__init__.py", line 8, in <module>
from OpenSSL import crypto, SSL
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1579, in <module>
class X509StoreFlags(object):
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1598, in X509StoreFlags
NOTIFY_POLICY = _lib.X509_V_FLAG_NOTIFY_POLICY
AttributeError: module \'lib\' has no attribute \'X509_V_FLAG_NOTIFY_POLICY\'
11/28/2024 02:00:39 AM CRITICAL: Command \'[\'snakemake\', \'--snakefile\', \'/home/runner/work/aviary/aviary/aviary/modules/Snakefile\', \'--directory\', \'/tmp/tmpnd6jzx19/test\', \'--cores\', \'16\', \'--rerun-incomplete\', \'--keep-going\', \'--quiet\', \'--rerun-triggers\', \'mtime\', \'--configfile\', \'/tmp/tmpnd6jzx19/test/config.yaml\', \'--nolock\', \'--retries\', \'0\', \'--conda-frontend\', \'mamba\', \'--use-conda\', \'--conda-prefix\', \'/home/runner/work/aviary/aviary/test/data/.conda\', \'--dryrun\', \'complete_assembly_with_qc\']\' returned non-zero exit status 1.
'STDOUT was: b'' |
Fyi I tried this on the comebin branch I think. Gave up as it was funky in tests. Better to concentrate on vamb tax?
-\-------------
Ben Woodcroft
Group leader, Centre for Microbiome Research, QUT
…________________________________
From: Samuel Aroney ***@***.***>
Sent: Thursday, November 28, 2024 12:02:08 PM
To: rhysnewell/aviary ***@***.***>
Cc: Subscribed ***@***.***>
Subject: Re: [rhysnewell/aviary] Add COMEbin as another default binner (PR #234)
Weird test failure:
STDERR was: b'11/28/2024 02:00:39 AM INFO: Time - 02:00:39 28-11-2024
11/28/2024 02:00:39 AM INFO: Command - /usr/share/miniconda/envs/test/bin/aviary assemble -1 /home/runner/work/aviary/aviary/test/data/wgsim.1.fq.gz -2 /home/runner/work/aviary/aviary/test/data/wgsim.2.fq.gz --output /tmp/tmpnd6jzx19/test --conda-prefix /home/runner/work/aviary/aviary/test/data/.conda --dryrun --tmpdir /tmp/tmpnd6jzx19 --snakemake-cmds --quiet
11/28/2024 02:00:39 AM INFO: Version - 0.10.0
11/28/2024 02:00:39 AM INFO: Configuration file written to /tmp/tmpnd6jzx19/test/config.yaml
11/28/2024 02:00:39 AM INFO: Executing: snakemake --snakefile /home/runner/work/aviary/aviary/aviary/modules/Snakefile --directory /tmp/tmpnd6jzx19/test --cores 16 --rerun-incomplete --keep-going --quiet --rerun-triggers mtime --configfile /tmp/tmpnd6jzx19/test/config.yaml --nolock --retries 0 --conda-frontend mamba --use-conda --conda-prefix /home/runner/work/aviary/aviary/test/data/.conda --dryrun complete_assembly_with_qc
Traceback (most recent call last):
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/__init__.py", line 671, in snakemake
workflow.include(
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/workflow.py", line 1362, in include
snakefile = infer_source_file(snakefile, basedir)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/sourcecache.py", line 323, in infer_source_file
if is_local_file(path_or_uri):
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/common/__init__.py", line 92, in is_local_file
return parse_uri(path_or_uri).scheme == "file"
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/common/__init__.py", line 96, in parse_uri
from smart_open import parse_uri
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/__init__.py", line 34, in <module>
from .smart_open_lib import open, parse_uri, smart_open, register_compressor # noqa: E402
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/smart_open_lib.py", line 35, in <module>
from smart_open import doctools
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/doctools.py", line 21, in <module>
from . import transport
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/transport.py", line 104, in <module>
register_transport("smart_open.s3")
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/transport.py", line 49, in register_transport
submodule = importlib.import_module(submodule)
File "/usr/share/miniconda/envs/test/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/s3.py", line 25, in <module>
import boto3
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/boto3/__init__.py", line 17, in <module>
from boto3.session import Session
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
import botocore.session
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/session.py", line 26, in <module>
import botocore.client
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/client.py", line 15, in <module>
from botocore import waiter, xform_name
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/waiter.py", line 18, in <module>
from botocore.docs.docstring import WaiterDocstring
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/__init__.py", line 15, in <module>
from botocore.docs.service import ServiceDocumenter
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in <module>
from botocore.docs.client import (
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/client.py", line 18, in <module>
from botocore.docs.example import ResponseExampleDocumenter
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in <module>
from botocore.docs.shape import ShapeDocumenter
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in <module>
from botocore.utils import is_json_value_header
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/utils.py", line 39, in <module>
import botocore.httpsession
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/httpsession.py", line 45, in <module>
from urllib3.contrib.pyopenssl import (
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/urllib3/contrib/pyopenssl.py", line 43, in <module>
import OpenSSL.SSL # type: ignore[import-untyped]
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/__init__.py", line 8, in <module>
from OpenSSL import crypto, SSL
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1579, in <module>
class X509StoreFlags(object):
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1598, in X509StoreFlags
NOTIFY_POLICY = _lib.X509_V_FLAG_NOTIFY_POLICY
AttributeError: module \'lib\' has no attribute \'X509_V_FLAG_NOTIFY_POLICY\'
11/28/2024 02:00:39 AM CRITICAL: Command \'[\'snakemake\', \'--snakefile\', \'/home/runner/work/aviary/aviary/aviary/modules/Snakefile\', \'--directory\', \'/tmp/tmpnd6jzx19/test\', \'--cores\', \'16\', \'--rerun-incomplete\', \'--keep-going\', \'--quiet\', \'--rerun-triggers\', \'mtime\', \'--configfile\', \'/tmp/tmpnd6jzx19/test/config.yaml\', \'--nolock\', \'--retries\', \'0\', \'--conda-frontend\', \'mamba\', \'--use-conda\', \'--conda-prefix\', \'/home/runner/work/aviary/aviary/test/data/.conda\', \'--dryrun\', \'complete_assembly_with_qc\']\' returned non-zero exit status 1.
'STDOUT was: b''
―
Reply to this email directly, view it on GitHub<#234 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAADX5F55MHIYJAGQV77QNL2CZ2SBAVCNFSM6AAAAABSUCANVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBVGEYTSMJRHA>.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
Ok. Can't find that branch. It runs fine here in the integration tests. I think the failure here is unrelated. |
Happy for this to be added as an optional binner, but that weird test failure should probs get fixed up. I can't really make sense of what that error might be due to, perhaps a python versioning issue? |
@AroneyS let me know when you'd like me to review |
Ready. New integration test passes. |
- conda-forge | ||
- bioconda | ||
dependencies: | ||
- comebin=1.0.* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm imagining this will end up breaking at some point for someone without pinning an explicit version. But it's an optional binner so it's fine
Test passes, but it adds a bunch of tmp files at the position of the input assembly:
ls example/test_short_read_recovery_comebin/assembly.fasta* example/test_short_read_recovery_comebin/assembly.fasta example/test_short_read_recovery_comebin/assembly.fasta.frag.faa example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.2quarter_lencutoff_1001.seed example/test_short_read_recovery_comebin/assembly.fasta.frag.ffn example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.hmmout example/test_short_read_recovery_comebin/assembly.fasta.frag.gff example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.hmmout.err example/test_short_read_recovery_comebin/assembly.fasta.frag.out example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.hmmout.out example/test_short_read_recovery_comebin/assembly.fasta_lengths.txt example/test_short_read_recovery_comebin/assembly.fasta.frag.err
I guess we could symlink the assembly into the comebin output folder to contain the mess.