Code

Python dev#8

Code

Merged

bkmarzouk merged 193 commits intopythoninstituteofcancerresearch/SOPRANO:pythonfrom python-devinstituteofcancerresearch/SOPRANO:python-devCopy head branch name to clipboardSep 20, 2023

+118,086-8,650

Commits

Commits on Aug 17, 2023

• argparse for required inputes

  bkmarzouk

  committed
  6eb0285

• missing required input added

  bkmarzouk

  committed
  890f3d9

• optional args parsing

  bkmarzouk

  committed
  35f1c5f

• promote str input to PosixPath where appropriate

  bkmarzouk

  committed
  91f5f5f

• proto validation check for input paths

  bkmarzouk

  committed
  47ccd75

• pass parsed args to main as kwargs

  bkmarzouk

  committed
  71580c1

• simple check to validate input path

  bkmarzouk

  committed
  7cb8d08

Commits on Aug 18, 2023

• rename ssb_selection script

  bkmarzouk

  committed
  c1322a0

• ruff cache

  bkmarzouk

  committed
  fbef219

• more type checks on _validate_input_path

  bkmarzouk

  committed
  43f5489

• reduce complexity of checks

  bkmarzouk

  committed
  695b68a

• test _validate_input_path

  bkmarzouk

  committed
  d85ee60

• rename of dest: bed_regions -> target

  bkmarzouk

  committed
  5a20464

• init prepare coordinates

  bkmarzouk

  committed
  e5bcca6

• filter_bed_file_transcript dev

  bkmarzouk

  committed
  96cca2c

Commits on Aug 21, 2023

• dummy functional methods for stage 2

  bkmarzouk

  committed
  82d5216

• skeleton functions for STEP 2

  bkmarzouk

  committed
  bcaeca1

Commits on Aug 22, 2023

• rename unit tests dir

  bkmarzouk

  committed
  4a0ae26

• init setup conda py script

  bkmarzouk

  committed
  aaa2037

• init setup conda py script

  bkmarzouk

  committed
  4d6738d

• test config dir

  bkmarzouk

  committed
  3f192bc

• local env yaml

  bkmarzouk

  committed
  9e306c2

• update tests for setup_conda.py

  bkmarzouk

  committed
  ae9f757

• update condarc methods

  bkmarzouk

  committed
  9563a71

• test _create_new_condarc

  bkmarzouk

  committed
  73754c6

• test _update_condarc

  bkmarzouk

  committed
  2d4aceb

• add pytest-dependency

  bkmarzouk

  committed
  2782cf5

• use subprocess to determine conda usage

  bkmarzouk

  committed
  5ea8f43

• init conda env build tests with dependencies

  bkmarzouk

  committed
  12bf2b9

• add dependencies for test_prepare_condarc

  bkmarzouk

  committed
  7fe1719

• ignore local env build dir

  bkmarzouk

  committed
  d7b4426

• (minor style fix) mem issue in formatting

  bkmarzouk

  committed
  7274fc3

• in test assert conda OR mamba

  bkmarzouk

  committed
  d660c69

Commits on Aug 23, 2023

• setup shell script to build conda env and install local editable project

  bkmarzouk

  committed
  7365781

• local yml file for installation updated. Strip python deps (other than core 3.11 + pip) and focus on control over perl

  bkmarzouk

  committed
  af72ae7

• flag removals

  bkmarzouk

  committed
  7b14ea2

• add bedtools & vep to conda env recipe

  bkmarzouk

  committed
  4904ec6

• bash installers

  bkmarzouk

  committed
  c03f2ca

• setup.sh update -> source environment constructor from src/

  bkmarzouk

  committed
  8cb930e

• test running soprano env

  bkmarzouk

  committed
  cb68dfc

• vep installation check

  bkmarzouk

  committed
  b2f20ea

• bedtools installation check

  bkmarzouk

  committed
  15279ec

• generalise method for detecting whether exec found & add secondary check for whether inside of current conda env

  bkmarzouk

  committed
  2b6274f

Commits on Aug 25, 2023

• rename sh_utils dir -> env_utils

  bkmarzouk

  committed
  6408b90

• src/SOPRANO/env_utils/setup_conda.py -> src/SOPRANO/env_utils/config_conda_env.py

  bkmarzouk

  committed
  001e91f

• remove redundent condarc efforts

  bkmarzouk

  committed
  4d05848

• mark dependencies for conda installations

  bkmarzouk

  committed
  b3422e1

• add cmd line option for installation (dev or ci). Add deps for ci in toml.

  bkmarzouk

  committed
  f3813c1

• update classifiers

  bkmarzouk

  committed
  999c5c5

• lint test workflow

  bkmarzouk

  committed
  05f1d5c

• update yaml: fix ruff syntax + add branch info

  bkmarzouk

  committed
  fe49119

• attempt conda env build and testing

  bkmarzouk

  committed
  c9d44f1

• missing checkout step added

  bkmarzouk

  committed
  03ea9e1

• update yaml: add x perms to shell scripts

  bkmarzouk

  committed
  477e777

• avoid shell script: build env using actions

  bkmarzouk

  committed
  d5a30a1

• try micromamba instead of miniconda & ensure env is activated in additional step

  bkmarzouk

  committed
  6170be9

• omit potentially redundent dependencies in local.yml

  bkmarzouk

  committed
  6c2c01e

• mamba -> micromamba cmd

  bkmarzouk

  committed
  7719a68

• add init-shell in context

  bkmarzouk

  committed
  24405bd

• ensure source ~/.bashrc & echo ~/.bashrc before (see if written as expected)

  bkmarzouk

  committed
  e664d55

• update shell for pytest steps. Remove activation step: Doesn't persist between steps (I think...)

  bkmarzouk

  committed
  bb7180a

• Merge branch 'python' into python-dev

  bkmarzouk

  authored
  0d8a85b

• Merge branch 'python' into python-dev

  bkmarzouk

  committed
  d5263ba

• minor: dbl quotes in eval

  bkmarzouk

  committed
  5703174

• add black style checks

  bkmarzouk

  committed
  b10c0ce

• add sh utils dir

  bkmarzouk

  committed
  604e3f6

• init testing for _filter_transcript_file

  bkmarzouk

  committed
  bb04446

• more details in docstring for _filter_transcript_file

  bkmarzouk

  committed
  7a22ad4

• subprocess pipe

  bkmarzouk

  committed
  66c4a6d

Commits on Aug 26, 2023

• rename sub_pipe -> subprocess_pipes

  bkmarzouk

  committed
  8a0411f

• dev and test output handlers for completed subprocesses

  bkmarzouk

  committed
  fe0b5e5

• dev and test and pipe wrapper for subprocesses

  bkmarzouk

  committed
  d66b54a

• dev and test _filter_transcript_file method

  bkmarzouk

  committed
  4b5d173

• dev test filter transcript files

  bkmarzouk

  committed
  1c0c5bb

• tmp ignore tmp dir

  bkmarzouk

  committed
  9651e8f

Commits on Aug 27, 2023

• _define_excluded_regions_for_randomization

  bkmarzouk

  committed
  6f021c7

• init sort excluded regs for rand

  bkmarzouk

  committed
  1cd27f3

Commits on Aug 29, 2023

• path objects implementation

  bkmarzouk

  committed
  d55a891

Commits on Aug 31, 2023

• dev and (mostly) test randomization prep

  bkmarzouk

  committed
  4cddd1f

• entry point

  bkmarzouk

  committed
  d42c259

• bug fix in filter processing

  bkmarzouk

  committed
  ccb18e1

• add pytest fixtures for content + helper methods for checking written content. Applied to filter_transcript_files methods

  bkmarzouk

  committed
  8c451e1

• Merge branch 'excl-pos' into python-dev

  Show description for 3bf0f53
  bkmarzouk

  committed
  3bf0f53

• apply fixtures to test__define_excluded_regions_for_randomization

  bkmarzouk

  committed
  3aa40fc

• apply fixtures to test__sort_excluded_regions_for_randomization

  bkmarzouk

  committed
  941bdfd

• fixtures applied to test__randomize_with_target_file

  bkmarzouk

  committed
  a47f55b

• apply fixtures to test__non_randomized

  bkmarzouk

  committed
  d401ab4

• Merge branch 'excl-pos' into python-dev

  Show description for 7bd7e1e
  bkmarzouk

  committed
  7bd7e1e

Commits on Sep 1, 2023

• exclusion of positively selected genes implemented and tested

  bkmarzouk

  committed
  2cc0b45

• develop and test get_protein complement

  bkmarzouk

  committed
  4ce4172

• prep ssb192

  bkmarzouk

  committed
  6cc1985

Commits on Sep 4, 2023

• transform coordinates implemented

  bkmarzouk

  committed
  5848d9c

• move examples into src dir. Prepare integration test config. Parsing of args into paths and transcript objects in main

  bkmarzouk

  committed
  5dce060

Commits on Sep 5, 2023

• introduce Parameters object for pipieline control

  bkmarzouk

  committed
  6e28a97

• introduce class methods for randomization protocol that depend on Parameters object: This helps keep consistent behaviour from single object

  bkmarzouk

  committed
  ac8ecd9

• update signature and init of transrcipts in Parameters object

  bkmarzouk

  committed
  d9e9283

• build Parameters object in main

  bkmarzouk

  committed
  3a487fc

• minor refactoring of existing methods. more complete treatment of input parameters. classmethod to intialize parameters from namespace

  bkmarzouk

  committed
  4b7cda6

• refactor slightly for improved readability. updated variable names in dest of argparse vars to be compatible with choices made in objects.py

  bkmarzouk

  committed
  10d9405

• update tests with refactored names, vars, funcs, etc

  bkmarzouk

  committed
  9cf834b

• include input annoted file path in AnalysisPaths object

  bkmarzouk

  committed
  792771a

• clean up cli methods: remove protected method indicators and unify notation for output commands

  bkmarzouk

  committed
  b7df74e

• integrate seed values for shuffling

  bkmarzouk

  committed
  d031e1b

• more explicit line length for isort in toml

  bkmarzouk

  committed
  1aff90d

• update defs in integration test following refactoring

  bkmarzouk

  committed
  e8acd62

• update pre-commit versioning (ruff)

  bkmarzouk

  committed
  d56b437

• add excluded genes objects

  bkmarzouk

  committed
  e8a3a8a

• integrate gene exclusions into pipeline and test

  bkmarzouk

  committed
  b50c5e1

• update workflow to include integration tests

  bkmarzouk

  committed
  ca6d95e

• implement class method for filtering

  bkmarzouk

  committed
  7e83257

• make application of design pattern more consistent. Apply to gene exclusions + protein complement

  bkmarzouk

  committed
  f9d4fd1

• minor name update

  bkmarzouk

  committed
  732bc6d

• apply pipeline pattern to complement, ssb and intra epitope methods and test integration

  bkmarzouk

  committed
  03fa944

• update docstrings and tidy comments in prep coords

  bkmarzouk

  committed
  781efd6

• update name of test file for detection at directory level

  bkmarzouk

  committed
  62f5c75

• fix indentation levels for easier readiblity

  bkmarzouk

  committed
  0e5a255

• methods for decompressing ensemble transcript ids

  bkmarzouk

  committed
  39eeb45

• rename integration test since caused conflict with units file name

  bkmarzouk

  committed
  4f08edb

• update TranscriptPaths object to include ensembl transcript ids

  bkmarzouk

  committed
  1a037c5

Commits on Sep 6, 2023

• add paths for *cds.fasta files. add paths for transcript:regions to estimate number of sites

  bkmarzouk

  committed
  838242f

• update defs for epitopes and intra epitope paths. Added missing def

  bkmarzouk

  committed
  5af9f05

• fix incorrectr path def for production of intra_epitopes_cds

  bkmarzouk

  committed
  ae46451

• more path checks on prep coords

  bkmarzouk

  committed
  e496e5d

• refactor test fixtures into seperate file and extract helper methods into _test_utils

  bkmarzouk

  committed
  59aa2ba

• extract pipeline component object and missing data error into pipeline_utils

  bkmarzouk

  committed
  741ce16

• integrate ObtainFastaRegions with fiducial test and fixtures

  bkmarzouk

  committed
  d4182c0

• rename integration test case to reference TCGA-05-4396

  bkmarzouk

  committed
  1d97cd3

• implement GetTranscriptRegionsForSites

  bkmarzouk

  committed
  1cee87c

Commits on Sep 11, 2023

• fix bug in writing of intra fasta file

  bkmarzouk

  committed
  b9a66d6

• estimate subs ssb192

  bkmarzouk

  committed
  8a91d5d

• raise runtime error for subprocess steps with non-zero exit

  bkmarzouk

  committed
  f1f4a33

• replace buggy method for obtaining transcript:regions

  bkmarzouk

  committed
  5390088

• compute sums over all possible target and non target region sites

  bkmarzouk

  committed
  1d8b0c1

• FixSimulated and ColumnCorrect methods

  bkmarzouk

  committed
  74cea6e

• entry point for downloading reference genome files

  bkmarzouk

  committed
  efff3d7

• update parameters to include attribute containing GenomePaths object

  bkmarzouk

  committed
  e147154

• path fix for ref genomes in frozen data instances

  bkmarzouk

  committed
  921cd3c

• Implement context corrections for ssb192

  bkmarzouk

  committed
  7b4503b

• add fasta index file for GRCh37

  bkmarzouk

  committed
  d3ad000

• Implement FlagComputations

  bkmarzouk

  committed
  f6ecc05

• Implement TripletCounts

  bkmarzouk

  committed
  bd58132

• note in master shell script: code block looks redundent

  bkmarzouk

  committed
  329e24d

Commits on Sep 12, 2023

• SiteCorrections

  bkmarzouk

  committed
  866b824

• typo fix

  bkmarzouk

  committed
  fe64f19

• omit integration tests from pipeline (tmp) - these have too many data dependencies that require download for remote runner

  bkmarzouk

  committed
  48f16f2

• Merge pull request #3 from instituteofcancerresearch/analysis-ssb192

  Show description for d76603c
  bkmarzouk

  authored
  d76603c

• IntersectByFrequency implemented

  bkmarzouk

  committed
  08a7480

• GetSilentCounts

  bkmarzouk

  committed
  0a87a9f

• GetNonSilentCounts

  bkmarzouk

  committed
  937d9db

• GetMissenseCounts

  bkmarzouk

  committed
  7ca5431

• GetIntronicCounts

  bkmarzouk

  committed
  aa1a963

• fix output of intro mutation count

  bkmarzouk

  committed
  ce2112a

• OnOffCounts

  bkmarzouk

  committed
  3a2e47b

• BuildEpitopesDataFile

  bkmarzouk

  committed
  5978835

• Merge pull request #4 from instituteofcancerresearch/step5

  Show description for bf05732
  bkmarzouk

  authored
  bf05732

Commits on Sep 13, 2023

• CheckTargetMutations - raise error if None found in target region

  bkmarzouk

  committed
  9451b31

• ComputeIntronRate

  bkmarzouk

  committed
  ab11c8e

• reimplementation of_build_flag_file due to bug in computing GOOD/BAD

  bkmarzouk

  committed
  602d740

• explanation of +1 in awk command

  bkmarzouk

  committed
  b79b848

• do not cleanup tmp files for debugging purposes. also added local paths for comparable execution to TCGA test case

  bkmarzouk

  committed
  623ed25

• minor notes in e2e test case for testing (tmp)

  bkmarzouk

  committed
  e7df483

• minor note on default behaviour

  bkmarzouk

  committed
  d829b1f

Commits on Sep 14, 2023

• Data prepocessing for intronic dnds calculation

  bkmarzouk

  committed
  4011ee9

• _compute_mutation_counts

  bkmarzouk

  committed
  bf21283

• ignore output file (used for local tests)

  bkmarzouk

  committed
  fb8db00

• _define_variables

  bkmarzouk

  committed
  dc972bf

• implment KaKs computations, define variables and combined quantities, etc

  bkmarzouk

  committed
  bd70b3b

• _compute_conf_interval

  bkmarzouk

  committed
  40e6d84

• implement confidence and pval computations

  bkmarzouk

  committed
  6874cdf

Commits on Sep 15, 2023

• dnds calculations in place

  bkmarzouk

  committed
  7bd81d7

Commits on Sep 18, 2023

• move methods from src/SOPRANO/calculate_KaKsEpiCorrected_Cl_intron.py into dnds.py

  bkmarzouk

  committed
  09c100c

• move path defs for tcga ssb192 test case into fixture

  bkmarzouk

  committed
  2ce8bae

• Merge pull request #5 from instituteofcancerresearch/step6

  Show description for 2e03fb2
  bkmarzouk

  authored
  2e03fb2

• add pandas to dependencies file

  bkmarzouk

  committed
  f925db6

Commits on Sep 19, 2023

• pre-computed fai and chrom files for GRCh37/38 for ensembl release 110

  bkmarzouk

  committed
  034f4cd

• delete old file versions (109)

  bkmarzouk

  committed
  d85a723

• update fixture for change in auxfile which now uses intron file

  bkmarzouk

  committed
  583d8a6

• provide values if assert fails in check_expected_content(...)

  bkmarzouk

  committed
  08624dc

• update directory config and download procedure / pre-processing of genome refs

  bkmarzouk

  committed
  a90bbec

• Update capitalization in ref genomes

  bkmarzouk

  committed
  3fabf85

• add coreutils (GNU) for osx as dependency

  bkmarzouk

  committed
  262c8c1

• generalise downloader & include option to download primary assembly

  bkmarzouk

  committed
  dcd1eda

• add 109 GRCh37 fai and chrom files

  bkmarzouk

  committed
  884c2d0

• add ensemble_transcriptID.translated.fasta index file

  bkmarzouk

  committed
  2ed8cb7

• add fai files for GRCh37/38 rel 110

  bkmarzouk

  committed
  d2a7998

Commits on Sep 20, 2023

• minor note

  bkmarzouk

  committed
  387e2f2

• homo sapiens vcf inputs (for testing)

  bkmarzouk

  committed
  e882256

• init vep annotation script

  bkmarzouk

  committed
  f375c35

• Merge pull request #6 from instituteofcancerresearch/parse-vcf

  Show description for cebad7c
  bkmarzouk

  authored
  cebad7c

• test workflow tests on macos

  bkmarzouk

  committed
  ac073ac

• update tests to run on osx-actions branch (tmp)

  bkmarzouk

  committed
  1c6f66e

• update workflow files

  bkmarzouk

  committed
  81bdbbf

• Merge pull request #7 from instituteofcancerresearch/osx-actions

  Show description for c52cbaa
  bkmarzouk

  authored
  c52cbaa