Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add COMEbin as another optional binner #234

Merged
merged 3 commits into from
Dec 3, 2024
Merged

Add COMEbin as another optional binner #234

merged 3 commits into from
Dec 3, 2024

Conversation

AroneyS
Copy link
Collaborator

@AroneyS AroneyS commented Nov 28, 2024

Test passes, but it adds a bunch of tmp files at the position of the input assembly:

ls example/test_short_read_recovery_comebin/assembly.fasta*
example/test_short_read_recovery_comebin/assembly.fasta                                            example/test_short_read_recovery_comebin/assembly.fasta.frag.faa
example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.2quarter_lencutoff_1001.seed  example/test_short_read_recovery_comebin/assembly.fasta.frag.ffn
example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.hmmout                        example/test_short_read_recovery_comebin/assembly.fasta.frag.gff
example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.hmmout.err                    example/test_short_read_recovery_comebin/assembly.fasta.frag.out
example/test_short_read_recovery_comebin/assembly.fasta.bacar_marker.hmmout.out                    example/test_short_read_recovery_comebin/assembly.fasta_lengths.txt
example/test_short_read_recovery_comebin/assembly.fasta.frag.err

I guess we could symlink the assembly into the comebin output folder to contain the mess.

@AroneyS
Copy link
Collaborator Author

AroneyS commented Nov 28, 2024

Weird test failure:

STDERR was: b'11/28/2024 02:00:39 AM INFO: Time - 02:00:39 28-11-2024
11/28/2024 02:00:39 AM INFO: Command - /usr/share/miniconda/envs/test/bin/aviary assemble -1 /home/runner/work/aviary/aviary/test/data/wgsim.1.fq.gz -2 /home/runner/work/aviary/aviary/test/data/wgsim.2.fq.gz --output /tmp/tmpnd6jzx19/test --conda-prefix /home/runner/work/aviary/aviary/test/data/.conda --dryrun --tmpdir /tmp/tmpnd6jzx19 --snakemake-cmds  --quiet
11/28/2024 02:00:39 AM INFO: Version - 0.10.0
11/28/2024 02:00:39 AM INFO: Configuration file written to /tmp/tmpnd6jzx19/test/config.yaml
11/28/2024 02:00:39 AM INFO: Executing: snakemake --snakefile /home/runner/work/aviary/aviary/aviary/modules/Snakefile --directory /tmp/tmpnd6jzx19/test --cores 16 --rerun-incomplete --keep-going  --quiet --rerun-triggers mtime --configfile /tmp/tmpnd6jzx19/test/config.yaml --nolock  --retries 0 --conda-frontend mamba  --use-conda --conda-prefix /home/runner/work/aviary/aviary/test/data/.conda --dryrun  complete_assembly_with_qc
Traceback (most recent call last):
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/__init__.py", line 671, in snakemake
    workflow.include(
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/workflow.py", line 1362, in include
    snakefile = infer_source_file(snakefile, basedir)
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/sourcecache.py", line 323, in infer_source_file
    if is_local_file(path_or_uri):
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/common/__init__.py", line 92, in is_local_file
    return parse_uri(path_or_uri).scheme == "file"
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/snakemake/common/__init__.py", line 96, in parse_uri
    from smart_open import parse_uri
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/__init__.py", line 34, in <module>
    from .smart_open_lib import open, parse_uri, smart_open, register_compressor  # noqa: E402
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/smart_open_lib.py", line 35, in <module>
    from smart_open import doctools
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/doctools.py", line 21, in <module>
    from . import transport
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/transport.py", line 104, in <module>
    register_transport("smart_open.s3")
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/transport.py", line 49, in register_transport
    submodule = importlib.import_module(submodule)
  File "/usr/share/miniconda/envs/test/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/smart_open/s3.py", line 25, in <module>
    import boto3
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/session.py", line 26, in <module>
    import botocore.client
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/client.py", line 15, in <module>
    from botocore import waiter, xform_name
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/waiter.py", line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import (
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/client.py", line 18, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/utils.py", line 39, in <module>
    import botocore.httpsession
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/botocore/httpsession.py", line 45, in <module>
    from urllib3.contrib.pyopenssl import (
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/urllib3/contrib/pyopenssl.py", line 43, in <module>
    import OpenSSL.SSL  # type: ignore[import-untyped]
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1579, in <module>
    class X509StoreFlags(object):
  File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1598, in X509StoreFlags
    NOTIFY_POLICY = _lib.X509_V_FLAG_NOTIFY_POLICY
AttributeError: module \'lib\' has no attribute \'X509_V_FLAG_NOTIFY_POLICY\'
11/28/2024 02:00:39 AM CRITICAL: Command \'[\'snakemake\', \'--snakefile\', \'/home/runner/work/aviary/aviary/aviary/modules/Snakefile\', \'--directory\', \'/tmp/tmpnd6jzx19/test\', \'--cores\', \'16\', \'--rerun-incomplete\', \'--keep-going\', \'--quiet\', \'--rerun-triggers\', \'mtime\', \'--configfile\', \'/tmp/tmpnd6jzx19/test/config.yaml\', \'--nolock\', \'--retries\', \'0\', \'--conda-frontend\', \'mamba\', \'--use-conda\', \'--conda-prefix\', \'/home/runner/work/aviary/aviary/test/data/.conda\', \'--dryrun\', \'complete_assembly_with_qc\']\' returned non-zero exit status 1.
'STDOUT was: b''

@AroneyS AroneyS requested a review from rhysnewell November 28, 2024 02:01
@wwood
Copy link
Collaborator

wwood commented Nov 28, 2024 via email

@AroneyS
Copy link
Collaborator Author

AroneyS commented Nov 28, 2024

Ok. Can't find that branch. It runs fine here in the integration tests. I think the failure here is unrelated.
I can just make it an optional binner. Seems slow without gpus.

@rhysnewell
Copy link
Owner

Happy for this to be added as an optional binner, but that weird test failure should probs get fixed up. I can't really make sense of what that error might be due to, perhaps a python versioning issue?

@AroneyS AroneyS changed the title Add COMEbin as another default binner Add COMEbin as another optional binner Dec 2, 2024
@rhysnewell
Copy link
Owner

@AroneyS let me know when you'd like me to review

@AroneyS
Copy link
Collaborator Author

AroneyS commented Dec 2, 2024

Ready. New integration test passes.

@AroneyS AroneyS changed the base branch from main to dev December 2, 2024 22:46
- conda-forge
- bioconda
dependencies:
- comebin=1.0.*
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm imagining this will end up breaking at some point for someone without pinning an explicit version. But it's an optional binner so it's fine

@AroneyS AroneyS merged commit de6a0b2 into dev Dec 3, 2024
2 checks passed
@AroneyS AroneyS deleted the add-comebin branch December 3, 2024 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants