Guardrails clean history (#34)

* Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]>
edilytics · Apr 4, 2024 · 53b10aa · 53b10aa
1 parent e714626
commit 53b10aa
Show file tree

Hide file tree

Showing 8 changed files with 29,962 additions and 21,456 deletions.
diff --git a/CRISPResso2/CRISPResso2Align.c b/CRISPResso2/CRISPResso2Align.c
diff --git a/CRISPResso2/CRISPRessoCORE.py b/CRISPResso2/CRISPRessoCORE.py
diff --git a/CRISPResso2/CRISPRessoCOREResources.c b/CRISPResso2/CRISPRessoCOREResources.c
diff --git a/CRISPResso2/CRISPRessoCOREResources.pyx b/CRISPResso2/CRISPRessoCOREResources.pyx
@@ -7,7 +7,6 @@ cdef extern from "stdlib.h":
     ctypedef unsigned int size_t
     size_t strlen(char* s)
 
-
 cdef extern from "Python.h":
     ctypedef void PyObject
     int _PyBytes_Resize(PyObject **, size_t)
@@ -37,6 +36,7 @@ def find_indels_substitutions(read_seq_al, ref_seq_al, _include_indx):
     substitution_values=[]
 
     all_deletion_positions = []
+    all_deletion_coordinates = []
     deletion_positions = []
     deletion_coordinates = []
     deletion_sizes = []
@@ -93,6 +93,7 @@ def find_indels_substitutions(read_seq_al, ref_seq_al, _include_indx):
         elif read_seq_al[idx_c] != '-' and start_deletion != -1:  # this is the end of a deletion
             end_deletion = ref_positions[idx_c]
             all_deletion_positions.extend(range(start_deletion, end_deletion))
+            all_deletion_coordinates.append((start_deletion, end_deletion))
             if include_indx_set.intersection(range(start_deletion, end_deletion)):
                 deletion_positions.extend(range(start_deletion, end_deletion))
                 deletion_coordinates.append((start_deletion, end_deletion))
@@ -102,6 +103,7 @@ def find_indels_substitutions(read_seq_al, ref_seq_al, _include_indx):
     if start_deletion != -1:
         end_deletion = ref_positions[seq_len - 1]
         all_deletion_positions.extend(range(start_deletion, end_deletion))
+        all_deletion_coordinates.append((start_deletion, end_deletion))
         if include_indx_set.intersection(range(start_deletion, end_deletion)):
             deletion_positions.extend(range(start_deletion, end_deletion))
             deletion_coordinates.append((start_deletion, end_deletion))
@@ -120,6 +122,7 @@ def find_indels_substitutions(read_seq_al, ref_seq_al, _include_indx):
         'insertion_n': insertion_n,
 
         'all_deletion_positions': all_deletion_positions,
+        'all_deletion_coordinates': all_deletion_coordinates,
         'deletion_positions': deletion_positions,
         'deletion_coordinates': deletion_coordinates,
         'deletion_sizes': deletion_sizes,
@@ -130,14 +133,13 @@ def find_indels_substitutions(read_seq_al, ref_seq_al, _include_indx):
         'all_substitution_values': np.array(all_substitution_values),
         'substitution_values': np.array(substitution_values),
         'substitution_n': substitution_n,
-
         'ref_positions': ref_positions,
     }
-
-
 @cython.boundscheck(False)
 @cython.nonecheck(False)
 @cython.wraparound(False)
+
+
 def find_indels_substitutions_legacy(read_seq_al, ref_seq_al, _include_indx):
 
     cdef char* sub_seq=''
@@ -162,7 +164,6 @@ def find_indels_substitutions_legacy(read_seq_al, ref_seq_al, _include_indx):
     substitution_positions=[]
     all_substitution_values=[]
     substitution_values=[]
-
     nucSet = set(['A', 'T', 'C', 'G', 'N'])
     idx=0
     for idx_c, c in enumerate(ref_seq_al):
@@ -236,28 +237,27 @@ def find_indels_substitutions_legacy(read_seq_al, ref_seq_al, _include_indx):
 
     insertion_n = np.sum(insertion_sizes)
 
-
     retDict = {
-	    'all_insertion_positions':all_insertion_positions,
-	    'all_insertion_left_positions':all_insertion_left_positions,
-	    'insertion_positions':insertion_positions,
-	    'insertion_coordinates':insertion_coordinates,
-	    'insertion_sizes':insertion_sizes,
-	    'insertion_n':insertion_n,
-
-	    'all_deletion_positions':all_deletion_positions,
-	    'deletion_positions':deletion_positions,
-	    'deletion_coordinates':deletion_coordinates,
-	    'deletion_sizes':deletion_sizes,
-	    'deletion_n':deletion_n,
-
-	    'all_substitution_positions':all_substitution_positions,
-	    'substitution_positions':substitution_positions,
-	    'all_substitution_values':np.array(all_substitution_values),
-	    'substitution_values':np.array(substitution_values),
-	    'substitution_n':substitution_n,
-
-	    'ref_positions':ref_positions,
+        'all_insertion_positions':all_insertion_positions,
+        'all_insertion_left_positions':all_insertion_left_positions,
+        'insertion_positions':insertion_positions,
+        'insertion_coordinates':insertion_coordinates,
+        'insertion_sizes':insertion_sizes,
+        'insertion_n':insertion_n,
+        'all_deletion_positions':all_deletion_positions,
+
+        'deletion_positions':deletion_positions,
+        'deletion_coordinates':deletion_coordinates,
+        'deletion_sizes':deletion_sizes,
+        'deletion_n':deletion_n,
+
+        'all_substitution_positions':all_substitution_positions,
+        'substitution_positions':substitution_positions,
+        'all_substitution_values':np.array(all_substitution_values),
+        'substitution_values':np.array(substitution_values),
+        'substitution_n':substitution_n,
+
+        'ref_positions':ref_positions,
     }
     return retDict
 

diff --git a/CRISPResso2/CRISPRessoReports/templates/layout.html b/CRISPResso2/CRISPRessoReports/templates/layout.html
@@ -62,9 +62,6 @@
     <!-- Replace favicon.ico & apple-touch-icon.png in the root of your domain and delete these references -->
     {% if is_web %}
     <link rel="shortcut icon" href="/static/favicon.ico">
-    <link rel="stylesheet" href="/static/css/main.css">
-    <script src="/static/js/htmx-1.9.1.min.js"></script>
-    <link rel="apple-touch-icon" href="/apple-touch-icon.png">
     {% else %}
     <link rel="shortcut icon" href="http://crispresso.pinellolab.org/static/favicon.ico" />
     {% endif %}

diff --git a/CRISPResso2/CRISPRessoReports/templates/report.html b/CRISPResso2/CRISPRessoReports/templates/report.html
@@ -86,6 +86,7 @@
 					{% if report_data['report_display_name'] != '' %}
 					<h5>{{report_data['report_display_name']}}</h5>
 					{% endif %}
+					{{ render_partial('shared/partials/guardrail_warnings.html', report_data=report_data) }}
 					<h5>CRISPResso2 run information</h5>
 					<ul class="nav nav-tabs justify-content-center card-header-tabs" id="log-tab" role="tablist">
 						<li class="nav-item">

diff --git a/CRISPResso2/CRISPRessoReports/templates/shared/partials/guardrail_warnings.html b/CRISPResso2/CRISPRessoReports/templates/shared/partials/guardrail_warnings.html
@@ -0,0 +1,5 @@
+{% if 'guardrails_htmls' in report_data['run_data']['results'] and report_data['run_data']['results']['guardrails_htmls']|length > 0 %}
+	{% for message in report_data['run_data']['results']['guardrails_htmls'] %}
+		{{message}}
+	{% endfor %}
+{% endif %}