From 1f60486c42d2c44110e99bce6eabf050c662bd9a Mon Sep 17 00:00:00 2001 From: McKay Date: Mon, 15 Apr 2024 15:44:10 -0600 Subject: [PATCH] Squashed commit of the following: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit commit a98d35a5d38298fa68a83aecaf80b1cb9023b5a5 Author: McKay Date: Mon Apr 15 14:30:55 2024 -0600 Squashed commit of the following: commit 06f48ed73228580cfedaab389a6db55a62456b97 Author: McKay Date: Mon Apr 15 14:02:58 2024 -0600 c2pro installation in dockerfile commit 38dd5150a785e65edf7308bed35001b49bbbf017 Author: McKay Date: Mon Apr 15 13:20:46 2024 -0600 removed && commit ad38cee6ede27affaae6862173e912c999f32ce1 Author: McKay Date: Mon Apr 15 12:12:12 2024 -0600 moved d3 import to end commit c2ba22f35c3e0a7775b3b89405c3c5ca38b3c4ee Author: McKay Date: Mon Apr 15 12:11:03 2024 -0600 removed duplicate imports leave d3 in bottom plotly at top commit 44cbe598affd5176d3209fd1901db0d0c438f625 Author: McKay Date: Fri Apr 12 16:07:18 2024 -0600 fixed batch rendering for d3 commit 418a6205ce52d8eb45799b34bcb47e3e5372d726 Author: McKay Date: Fri Apr 12 15:47:45 2024 -0600 fixed d3 plot 2b rendering commit d4e00703ff6f023606505b787e5ddc31772d9e8c Author: McKay Date: Wed Apr 10 15:52:05 2024 -0600 move C2Pro install before app run commit e8f1947a3e5e5106d74f9200a6dac93616b171bd Author: McKay Date: Tue Apr 9 17:21:40 2024 -0600 fixed guardrails, paritals path commit 0a99c237b66970053a82221dba4b5aa6c2d2b568 Author: McKay Date: Tue Apr 9 13:33:40 2024 -0600 fastp, htmx, jquery dependencies commit 62e8e66f62ddc226a470e82ee761279fb957c71d Author: McKay Date: Tue Apr 9 12:15:49 2024 -0600 removed commented code commit 11fb95e02f6133df827798098765b07043723a4f Author: McKay Date: Tue Apr 9 11:19:58 2024 -0600 reports changes commit 3220cd8c6c031c1710c72c9cf0eb7d6070757bd3 Author: McKay Date: Mon Apr 8 15:50:54 2024 -0600 Squashed commit of the following: commit 53fecf71977f10c0d643887bf110cf7cbf044b8e Author: Sam Date: Thu Apr 4 15:35:39 2024 -0600 Squashed commit of the following: commit 6f4b0ad885e1d72413a034bf7abaaa0360a3b0c4 Author: Samuel Nichols Date: Thu Apr 4 15:18:09 2024 -0600 Batch d3 clean (#55) * imports C2Pro plots if available * added --use_matplotlib flag * added C2Pro matched api funciton signatures * added api args for plotly * added **kwargs * renamed config to custom_config, more specificity * added backend flag for plotly kaleido * added pro_installed boolean for templates, added plotly dependency to report templates * Squashed commit of the following: commit c909ea3b34e87ce637e00dac075d2bb2f8bfb954 Author: McKay Date: Thu Feb 15 15:55:23 2024 -0700 added plotly dependency for pro commit 76b3601f6a0144f100266153f1c999e0c5de65de Author: Samuel Nichols Date: Fri Jan 12 09:56:19 2024 -0700 Squashed commit of the following: commit 603f2eff9d1aa21ae95f3e134da303b8018d3a33 Author: Samuel Nichols Date: Fri Jan 12 09:48:20 2024 -0700 fix guardrials partial commit 22fc03183a8070c30dfb74d5c23575ac19019855 Author: Samuel Nichols Date: Fri Jan 12 08:54:01 2024 -0700 Add guardrail partial commit e55f6b21972b578261bc5a864ce1d653d98f9e34 Author: Samuel Nichols Date: Mon Jan 8 07:50:59 2024 -0700 Functional guardrails, needs reports update commit 6e968e9699ed59a47d88191d03768e042d8b60a4 Merge: 32b49685 e948ce10 Author: Samuel Nichols Date: Mon Dec 18 13:34:36 2023 -0700 Merge branch 'guardrails-clean-history' of https://github.com/edilytics/CRISPResso2 into guardrails-clean-history commit 32b49685da320501dad2b0ebbb57887b66220ba8 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit 4e309cf6f732565d635de3d4c5d074ada3027e2d Author: Cole Lyman Date: Mon Dec 18 10:51:55 2023 -0700 Refactor to use CRISPRessoReports module commit e648dc087c0055bc5d2fca13c64071a371dea941 Author: Cole Lyman Date: Mon Dec 18 10:51:11 2023 -0700 Add CRISPRessoReports subtree commit e948ce107ebb0d1d99010ed12e937f34b5e607d4 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05b bb13e007 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e8 d0b41483 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f909713 b913fcb4 Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f6788 5f909713 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2d e5afa478 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f0 51a943c3 Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f0 5a10d638 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd468 442a48c7 Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels… * refactored pro check for consistency * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * removed commented line * debugging lines * removed breakpoints * flattened allele data for plot * fix #377 to write info files in CRISPRessoPooled * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Introduce pandas sorting in CRISPRessoCompare (#47) * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * try, except on pio chromium flag * D3 version of batch summary * D3 batch summary and htmls * Fix pooled and WGS (WGS uses multiReport.html) * Prepare for merge with c2pro * Merge with c2pro * Print d3 json on crispresso base * CRISPResso pro changes * Reports changes * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Remove nested f strings * Update if statement to match pro * fixed handling of use_matplotlib flag * Point tests at repo branch * added flag to parsers, flag check to import * remove default False, fix help * - refactored import check to shared function - global in CORE formatting change * changed text_kwargs to kwargs * removed newline * Custom_colors change * Fix compare test * Remove d3 from core * remove plotly plots * remove guardrails and make reports changes * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: trevormartinj7 * change config to custom_config * Update integration_tests.yml point to master tests branch * Include guardrails and pro in README --------- Co-authored-by: McKay Co-authored-by: Kendell Clement Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: trevormartinj7 commit 123ffe093181e514894d74c4e9f0a5f72125157e Author: Cole Lyman Date: Thu Apr 4 13:15:41 2024 -0600 Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: trevormartinj7 commit 46f85de056ca541ae0852121f2e2897f3ab33539 Author: Samuel Nichols Date: Thu Apr 4 13:09:38 2024 -0600 Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit d75a9ba74fa1e67d2e53725cfee5203d754c09dc Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Thu Apr 4 12:25:43 2024 -0600 Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols commit ff07f62a0328ac9f551bf3e09bfc1546e5827f90 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Apr 4 11:22:48 2024 -0600 Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit fa03d16ddd10e3163c035bd1a8d18416f7da35a8 Author: Cole Lyman Date: Thu Mar 28 16:28:36 2024 -0600 Decrease Docker image size and fix PE naming and parameter behavior (#404) * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit b2cfb911171dd8550f057e67239f51a8dfff91aa Author: Cole Lyman Date: Thu Mar 28 14:41:31 2024 -0600 Fix the assignment of multiple quantification window coordinates (#38) (#403) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- * Extract out quantification window coordinate function * Refactor get_quant_window_coordinates function into two The rationale behind this is that the behavior around the cloned amplicon is quite different than if the qwc are specified directly for the amplicon. * Handling qwc: add unit tests, refactor some more and add documentation * Extract out get_relative_coordinates function This function just computes the relative indexes without doing an alignment. * Add clarifying unit tests for `get_relative_coordinates` * Refactor cloned indexes to use ref_positions instead of s1inds * fixed function for getting cloned qwc idxs * added tests for cloned qwc function * Introduce pandas sorting in CRISPRessoCompare (#47) * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- --------- * removed if check * implemented last test * changed NT to BadParameterException * changed tests, NT to BadParameter exceptions * Uncomment and correct tests for `get_relative_coordinates` * finished qwc tests * 0 is an acceptable qwc * new get_relative_coords function * added relative coordinate tests * removed unused functions * formatting * check for 0 qwc * remove test code * remove comment * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: McKay Co-authored-by: Samuel Nichols commit d171561b4a98f1f5b323b9f1b19c027ef40b059a Author: Kendell Clement Date: Thu Mar 28 14:33:57 2024 -0600 Update integration_tests.yml commit e87d92ebaf95a1147b657a4c356df137d9aae04e Author: Cole Lyman Date: Mon Mar 25 09:50:30 2024 -0600 Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master commit 42e59597039eb4580abdfb0a9beb636f404b0f7a Author: Samuel Nichols Date: Fri Mar 22 17:38:37 2024 -0600 Run tests on PR only when opening PR (#53) commit e30cec33d9f0ce867251e170ffa820a51d31724e Author: Samuel Nichols Date: Fri Mar 22 14:31:38 2024 -0600 GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request commit 9191e687b246f6758afd8349c871207130b7e1ee Author: Cole Lyman Date: Fri Mar 22 14:05:11 2024 -0600 Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators commit cee64891a6d0a803906c686d27b1c597db91345a Author: Samuel Nichols Date: Fri Mar 15 10:42:53 2024 -0600 GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman commit ad3bd4ef552cc1b2f7628617c200174ab56cf255 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Mar 14 15:11:00 2024 -0600 Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman commit 0c834b36e72dbc6674a5cfa608c398b5f411f52f Author: Cole Lyman Date: Thu Mar 14 13:57:07 2024 -0600 Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly commit 0af8fe1377ea7a548250115d3a3adc9af7881395 Author: Samuel Nichols Date: Thu Mar 14 12:28:59 2024 -0600 Introduce pandas sorting in CRISPRessoCompare (#47) commit ebd14ba9fb51cb6ff5ac02b03737c3f4ba536793 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Mon Mar 11 16:15:56 2024 -0600 Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman commit de182d268518202a07c59780c3de0da760ef7409 Author: Kendell Clement Date: Thu Mar 7 15:04:12 2024 -0700 fix #377 to write info files in CRISPRessoPooled commit 448d244c1f990046a8a09662d220387a47a87edb Author: McKay Date: Fri Feb 23 17:18:22 2024 -0700 flattened allele data for plot commit 24310ff6353bd701b8d8a7b57d49fe10f0089792 Author: McKay Date: Fri Feb 23 13:23:09 2024 -0700 removed breakpoints commit 1f165794516df82b5dd68a93e7f8e7b7ca031e98 Author: McKay Date: Thu Feb 22 14:36:31 2024 -0700 debugging lines commit a1f275f603c6dfdb923b7a4e2536291e4a649e65 Author: Samuel Nichols Date: Thu Feb 22 12:35:21 2024 -0700 GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e commit 2b163927faa4ca37a1dc8294ffd1c18dd057b62e Author: Kendell Clement Date: Tue Feb 13 10:14:56 2024 -0700 Fix #376 - CRISPRessoPooled chunking When running CRISPressoPooled with > 10000000 reads aligned, if there are reads overlapping the chunk boundary, instead of increasing the end with 500 bases, it takes the last position of chromosome as end position. This fixes the bug. commit 31c65dfbe9d9e146eb1d12a91e0bfaad76b263dd Author: Samuel Nichols Date: Mon Feb 5 13:03:29 2024 -0700 Cole/fix assigning multiple qwc (#37) * Remove extraneous whitespace * Implement the ability to assign multiple quantification window coordinates * Updating documentation for quantification window coordinates * Update to allow 0 to prevent qwc being set. Update description. --------- Co-authored-by: Cole Lyman commit 751d9ea215707e145eda1446b20328215f90d6b7 Author: Kendell Clement Date: Mon Feb 5 12:31:56 2024 -0700 Fix #374 naming of CRISPRessoPooled file name commit 3f84c7dccf62885583eb6b74825c2d63557a6aa2 Author: Kendell Clement Date: Thu Jan 25 00:13:37 2024 -0700 Revert "denoise multiprocessing" This reverts commit 3bc04213fcc730d6a7dcd51067998f7c220df135. commit 3bc04213fcc730d6a7dcd51067998f7c220df135 Author: Kendell Clement Date: Thu Jan 25 00:11:52 2024 -0700 denoise multiprocessing commit 4723e4ac81c38d0fb2da7682f4213a0586bfc881 Author: Kendell Clement Date: Wed Jan 24 23:45:59 2024 -0700 Improve multiprocessing in Batch to utilize more processors Previously, sub-crispresso runs are run on 1 process (with the rationale that parallelization of the alignment bit is the most important for batches). Now, with parallelized plotting, if more processes are provided than batches, each sub-crispresso run will be run with int(#processes/#batches) processes, thus maximizing the processor utilization for small batches. commit ba8882dff10602a55cb2d1e4317863d55a9b50b7 Author: Kendell Clement Date: Mon Jan 22 23:12:07 2024 -0700 Include all jinja templates recursively commit 08a25495ee53e293ec5d85b843f381d561a428e8 Author: Kendell Clement Date: Mon Jan 22 22:54:49 2024 -0700 Include shared partials in manifest commit 6d4f6ceb8188e77b94f5666b4106b2ffee494c54 Author: Kendell Clement Date: Mon Jan 22 15:19:27 2024 -0700 Include locations for report templates commit 0dee4aba063fde2e25f160bb9e30d84dd5a274cc Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Jan 22 12:26:46 2024 -0700 Failed batch runs (#33) * Reports, add reports to packages, colors, ordered pandas sort (#28) * Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 7d9b4e5..e18807d * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from e18807d..e9da7bf git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: e9da7bff794058e1fcdb3dc9ced79871c6a30e18 * Add CRISPRessoReports to packages * Colors only with pro * changed tuple to list for matplotlib change (#31) * wgs and batch failed runs implementation * Added failed run functionality including shared function, edits to Report, and displaying with HTML and Javascript * Merge CRISPRessoReports master into failed-batch-runs * Cole's failed-batch-runs review and changes (#36) * Fix showing link to report in CLI (only show in web) * Remove styling of jumbotron The p-5 added some weird space at the top of the container, the rounded-3 did not make a difference (because there is no background), and the h-100 also did not make a difference. * Remove extra spaces at end of the line * Remove color legend from figure caption in plot 4f * Refactor fig_reports.html partial to reduce duplication * Add opacity to custom colors on allele quilt plot * Remove extra spaces * Change default color of deletion It looked too similar to `N` and was difficult to tell apart. * Refactor plot 10c, refactor displaying of figures This commit adds flexbox to the plots, this was mainly for plots 10b and 10c because their alignment was off. * Add more plots to get the correct percentages for width * Remove setting the height of the plots * Check for failed batch info before retrieving it in `make_multi_report_from_folder` * Fix extraneous whitespace in `fig_reports` partial * Only load certain resources when on web mode * Move jQuery import to bottom of the page to improve performance * Extract out report footer buttons to partial * Fix too many closing divs in batchReport.html * Refactor failed runs to be a partial * Move the failed run JS to the partial This has the benefit of keeping the relevant code close, and also prevents the error that we were running into before where `chevronIcon` wasn't found when there were no failed runs (because the element wasn't there). * Remove `report_name` id because it probably has spaces * Move existing Plotly plots to batchReport from multiReport * Fix typo in fig 11c and resize it to 40% --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05 bb13e00 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e d0b4148 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f90971 b913fcb Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f678 5f90971 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2 e5afa47 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f 51a943c Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f 5a10d63 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd46 442a48c Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. Co-authored-by: Kendell Clement commit a7ffd46822ce195d51ff4d3dba0f02fe9bc73c1e Author: Kendell Clement Date: Tue Jan 11 16:29:37 2022 -0500 Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9990790a0081b765c1f54f4a9b18db522ab4693 Author: Kendell Clement Date: Wed Jan 5 16:48:17 2022 -0500 Allow for mixed-case prime editing input, fix #187 commit 4acdcddbef3e812655b7af9def772f0bb0dc30b5 Author: Kendell Clement Date: Wed Jan 5 16:37:22 2022 -0500 Update insertion annotation for fastq_output and bam_output Fix index issue for fastq_output, add reporting of inserted bases to bam_output. Index for fastq_output is increased because the insertion happens immediately after the insertion coordinate. commit 2f84dd02787abffa6d39efbc50c82c92d1c87528 Author: kclem Date: Fri Dec 10 16:39:41 2021 -0500 Fastq_output report inserted bases when the `--fastq_output` parameter is provided, the inserted bases will be written to the output fastq file. Previously, a string like "DEL= INS=78(1) SUB= " would indicate a 1bp insertion at site 78. This update outputs strings like "DEL= INS=78(1+G) SUB= " with a plus character followed by the inserted bases. commit ba742bbfa533a672321b27b5122d7ea6658c9014 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Implement algorithmic improvements for find_indels_substitutions" This reverts commit 13414833ef6cd5b232097f6ff0325a2b8e28b214. commit 5c8e1c5de6e800c820d9ec5ffe4809737f1211de Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Add test cases for find_indels_substitutions" This reverts commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808. commit d9a072368ca991ffde52aa1ee29e17f2758ed279 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Try some additional cython optimizations" This reverts commit 38b101d57384b9f7d664ac8719c1585f5f7142a5. commit 38b101d57384b9f7d664ac8719c1585f5f7142a5 Author: Cole Lyman Date: Fri Oct 8 14:42:49 2021 -0600 Try some additional cython optimizations commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808 Author: Cole Lyman Date: Fri Oct 8 14:15:11 2021 -0600 Add test cases for find_indels_substitutions commit 13414833ef6cd5b232097f6ff0325a2b8e28b214 Author: Cole Lyman Date: Fri Oct 8 11:50:25 2021 -0600 Implement algorithmic improvements for find_indels_substitutions These changes remove the need for iterating over the alignment multiple times. commit f4b6cfc03951215c3b9019dc47beb2913a5448ab Author: Kendell Clement Date: Fri Nov 19 00:10:05 2021 -0500 Fix CRISPRessoPooled calls to deprecated pands ix commit 751fbdb4ce432f11f3fb66c5a34f7db3d1d75dc1 Author: swrosati <40470095+swrosati@users.noreply.github.com> Date: Fri Nov 12 12:33:04 2021 -0600 Update ylabel_values -> y_label_values Typo causing error in certain modes. commit 76c80713b7b8209c9399d67f718d7ade20f20d91 Author: kclem Date: Wed Nov 10 11:37:16 2021 -0500 Update dockerfile for pyparsing commit ef15caee29380f58aaae392c897fabe47587486e Author: Kendell Clement Date: Wed Nov 10 00:45:51 2021 -0500 Fix int bug for n_reads in CRISPRessoPooled commit fc1ae890712ab2dc36d5ff36c81f64a10a2c2337 Author: Kendell Clement Date: Wed Nov 10 00:34:34 2021 -0500 Spelling fix and version bump to 2.2.7 commit 08369e294d6756f727167af50f14ff75cb4864b5 Author: Kendell Clement Date: Wed Nov 10 00:30:03 2021 -0500 Add bam input and selected demuxing for CRISPRessoPooled Adds features for providing aligned bams as input to CRISPRessoPooled and for a faster demultiplexing when amplicons and genome are provided. The parameters are: --aligned_pooled_bam: Path to aligned input for CRISPRessoPooled processing. If this parameter is specified, the alignments in the given bam will be used to demultiplex reads. If this parameter is not set (default), input reads provided by --fastq_r1 (and optionally --fastq_r2) will be aligned to the reference genome using bowtie2. If the input bam is given, the corresponding reference fasta must also be given to extract reference genomic sequences via the parameter --bowtie2_index. Note that the aligned reads are paired-end seqenced, they should already be merged into 1 read (e.g. via Flash) before alignment. --demultiplex_only_at_amplicons: If set, and an amplicon file (--amplicons_file) and reference sequence (--bowtie2_index) are provided, reads overlapping alignment positions of amplicons will be demultiplexed and assigned to that amplicon. If this flag is not set, the entire genome will be demultiplexed and reads with the same start and stop coordinates as an amplicon will be assigned to that amplicon. commit 0cdcfd7c4af834f3270e61b4db4b5f6d3da10d7c Author: Kendell Clement Date: Wed Nov 10 00:24:15 2021 -0500 Remove unused var for bam_index commit 3647ace73c95a2f44e45b6b42b4f1748d3a47f4c Author: kclem Date: Wed Nov 3 09:43:28 2021 -0400 Add --min-overlap param for flash in CRISPRessoPooled commit eea442a763e0c6c41da16a15d6e11ddf6d222dc8 Author: kclem Date: Mon Nov 1 11:25:55 2021 -0400 Remove deprecated pandas .ix from WGS commit 3e6c281c931756acd26acca96249b2fa1ad1db31 Author: Cole Lyman Date: Thu Oct 21 11:10:19 2021 -0600 Add unit tests These unit tests were in the other repo, but weren't moved over for some reason. commit 50e7cb21570ddb757904728800e31c2bcbd0d060 Author: Cole Lyman Date: Thu Oct 21 11:09:38 2021 -0600 Add Makefile to run tests This makes it easy to test the integrations cases because it will automatically install the package when needed. commit de91d51bcd9b725533a45c86ae72bc27dd5d3150 Author: Cole Lyman Date: Thu Oct 21 11:28:02 2021 -0600 Replace `np.int` with `int` Apparently `np.int` has been deprecated, so in places where precision isn't important `np.int` has been replaced with `int` according to the instructions from the warning. commit cabebbefb2646967dbeee80af08ac14156b1b53c Author: kclem Date: Wed Oct 20 12:33:49 2021 -0400 Convert cols to numeric for PE commit c2bdd96651eef5af38fb7bbc11d257a827ac080d Author: kclem Date: Wed Oct 20 12:00:43 2021 -0400 Make loggers module-specific Matplolib sometimes logs verbosely to info, so this stops using the root logger commit 8196b6a81f477ddcb0e34d61dfb54085de20c1a0 Author: Kendell Clement Date: Tue Oct 19 22:00:23 2021 -0400 Fix unicode error for bam read/write commit a923a7c2ef182238bd6b8aa77289bac487b7679b Author: Kendell Clement Date: Tue Oct 19 21:59:47 2021 -0400 All sub-crispresso processes are run with 1 process commit 75205b0d41423e4f62796e9674b0edbee68a11b9 Author: Kendell Clement Date: Mon Oct 18 23:03:59 2021 -0400 Fix #161 add params for prime editing guide analysis commit ecf23ef23e5701b232bba547a6d7d4b96f085f26 Author: kclem Date: Mon Oct 18 15:02:29 2021 -0400 Add param for plotting custom plots centered at point Add parameter to plotCustomAllelePlot script: --plot_center which if set, will produce plots centered at this point instead of sgRNAs. commit 71bf32ca789dde1d44cf41b9d3b702f12336e010 Author: Kendell Clement Date: Wed Oct 13 00:48:14 2021 -0400 Update README.md commit 53197e62706e37db54f7ed50c94f38a938955e59 Author: Kendell Clement Date: Tue Oct 12 15:43:08 2021 -0400 Fix #160 plotting for cut sites in plot 5 commit 90b43eaa03c0ea0fdee62a7b244204cad50056cc Author: Kendell Clement Date: Mon Oct 4 15:45:20 2021 -0400 Remove version checks for old seaborn and numpy, fix #155 commit 5e9dedf6bd7c7b3ef998bed811760a41b5313c6e Author: Kendell Clement Date: Tue Sep 28 21:16:54 2021 -0400 Optimize get_command_output, fix gzip binary bug commit 46953c39b769a4fa43b2ef0822dd92f07c4f1b9b Author: Kendell Clement Date: Wed Sep 22 01:58:36 2021 -0400 Update expected tests for batch commit 856354aadebc8410956e724b555224a55941a618 Author: Kendell Clement Date: Wed Sep 22 01:57:59 2021 -0400 Update CRISPRessoBatchCORE.py Move parsing of batch params to after assignment of n_processes so this value is applied to sub-processes commit f7f81747207cb279fd2c0d91986c7d4e7137fde4 Author: Kendell Clement Date: Wed Sep 22 00:23:04 2021 -0400 Fix #149 bug when read is much longer than the given reference commit 798eb4322899f70e2e2a3df0fdfad9b5598d3a41 Author: Kendell Clement Date: Tue Sep 21 17:43:38 2021 -0400 Fix #150 Open fastq.gz in 'rt' mode commit fa168843e6036c6d4ec4a5d7acb097c6a1532a98 Author: Kendell Clement Date: Fri Sep 17 12:33:12 2021 -0400 Remove requirement for python 2.7 commit 80fe12efbb0ee5c321262822b237fc8220a3f2c6 Author: Kendell Clement Date: Thu Sep 9 17:41:27 2021 -0400 Print batch summaries for amplicons with only one sample Previously, batch summaries were only printed for amplicons with more than one sample to save bits. This change produces batch summaries for amplicons with only one sample, and we shift responsibility to the user for purchasing carbon offsets to compensate for the generation and storage of these redundant bits. commit 43065310bf28e6a08780222250abd0d39ff6d0c5 Author: Kendell Clement Date: Thu Sep 9 17:38:58 2021 -0400 Add parameter 'assign_ambiguous_alignements_to_first_allele' For ambiguous alignments, setting this flag will force them to be assigned to the first (as provided by the references -a first and then -e second) amplicon. Thus, no reads will be discarded as 'ambiguous' and all reads will be counted once in the analysis. Version bump to 2.2.4 commit 11eaacd3bac9ebeabad69c1d5d92f1fd1a783a17 Author: Kendell Clement Date: Fri Sep 3 22:34:59 2021 -0400 push down crispresso2_info items commit 20272e1e95305888ca744ac56af2c08554eb9f1b Author: Kendell Clement Date: Fri Sep 3 22:32:24 2021 -0400 remove mention of pickle commit 3517285473d423dc32c6e3fdbac69eecf0caa7fc Author: Kendell Clement Date: Fri Sep 3 22:30:36 2021 -0400 minor pushdown of report name in crispresso2_info commit 8129494607488bf270567d40d43e01e043699f06 Author: Cole Lyman Date: Fri Sep 3 17:31:54 2021 -0400 fixup! Move region related objects to results in crispresso2_info commit 88a82d27e840c29b95b1602ae1594bf161108979 Author: Cole Lyman Date: Fri Sep 3 16:40:40 2021 -0400 Move some objects in CRISPRessoPooled crispresso2_info objects Move the objects: - 'final_data' -> 'results'/'final_data' - 'running_mode' -> 'running_info'/'running_mode' commit b702ab3e53e88170c7517591ce7a7050ccf1c2ba Author: Cole Lyman Date: Fri Sep 3 16:36:20 2021 -0400 Move some objects in CRISPRessoBatch crispresso2_info Move the following objects: - 'completed_batch_arr' -> 'results'/'completed_batch_arr' - 'batch_names_arr' -> 'results'/'batch_names_arr' - 'batch_input_names' -> 'results'/'batch_input_names' - 'nucleotide_frequency_summary_filename' -> 'results'/'nucleotide_frequency_filename' - 'nucleotide_percentage_summary_filename' -> 'results'/'nucleotide_percentage_filename' - 'modification_frequency_summary_filename' -> 'results'/'modification_frequency_summary_filename' - 'modification_percentage_summary_filename' -> 'results'/'modification_percentage_summary_filename' commit 2416c80e952cb58fa082ead5ef719ed7eab3eeb5 Author: Cole Lyman Date: Fri Sep 3 15:39:18 2021 -0400 Move nuc_quilt related objects to 'results'/'general_plots' to crispresso2_info The following objects have been moved: - 'window_nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'window_nuc_pct_quilt_plot_names' - 'nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'nuc_pct_quilt_plot_names' - 'window_nuc_conv_plot_names' -> 'results'/'general_plots'/'window_nuc_conv_plot_names' - 'nuc_conv_plot_names' -> 'results'/'general_plots'/'nuc_conv_plot_names' commit 37e354f94980d304e1cecf11fee95e61988dd3e3 Author: Cole Lyman Date: Fri Sep 3 14:58:36 2021 -0400 Move summary objects to 'results'/'general_plots' in crispresso2_info The following objects have been moved: - 'summary_plot_names' -> 'results'/'general_plots'/'summary_plot_names' - 'summary_plot_titles' -> 'results'/'general_plots'/'summary_plot_titles' - 'summary_plot_labels' -> 'results'/'general_plots'/'summary_plot_labels' - 'summary_plot_datas' -> 'results'/'general_plots'/'summary_plot_datas' - 'summary_plot_root' -> 'results'/'general_plots'/'summary_plot_root' - 'reads_summary_plot' -> 'results'/'general_plots'/'reads_summary_plot' - 'modification_summary_plot' -> 'results'/'general_plots'/'modification_summary_plot' commit 6df988151c602602470c944ccf561cd28f2dc30c Author: Cole Lyman Date: Fri Sep 3 14:04:12 2021 -0400 Move region related objects to results in crispresso2_info This includes: - 'regions' -> 'results'/'regions' - 'all_region_names' -> 'results'/'all_region_names' - 'good_region_names' -> 'results'/'good_region_names' - 'good_region_folders' -> 'results'/'good_region_folders' commit 053691311b158a2d3620670d20983d546a9c2c7b Author: Cole Lyman Date: Fri Sep 3 13:44:46 2021 -0400 Move 'samples_quantification_summary_filename' to 'results'/'alignment_stats'/'sample_quantification_summary_filename' in crispresso2_info commit eda3d50b6fafde4ea3145980cb2010417b799d88 Author: Cole Lyman Date: Fri Sep 3 13:27:42 2021 -0400 Move 'finished_steps' to 'running_info'/'finished_steps' in crispresso2_info commit 1da829c1c3cfd2bda9aaccca0aaa28c78a8f7271 Author: blasvicco@gmail.com Date: Mon Aug 30 17:48:02 2021 +0200 BugFix: database_id referenced before declared. commit c9321cc2cfcac34e4b6d8e1aa9fde9f25382e2de Author: Kendell Clement Date: Sat Aug 21 00:15:52 2021 -0400 Version bump to 2.2.2 for some reason the last release didn't pick up the last commits. commit 3aadab69ef1e3a44f12ee277a7767648e943a787 Author: Cole Lyman Date: Fri Aug 20 12:09:49 2021 -0400 Update filterFastqs so that the numpy arrays are writable commit 7ca129d2471aeebef2ba6d2dadf36265810f1a5c Author: Kendell Clement Date: Fri Aug 20 02:00:42 2021 -0400 Update CRISPRessoPlot.py Undo attempted matplotlib deprecation warning fix commit 8e8a65c0f29b6f99662ac1d272aa7199871f5cf5 Author: Kendell Clement Date: Fri Aug 20 01:26:50 2021 -0400 Plotting bug fixes Display and plotting fixes for batch report labels, and general plots, as well as a matplotlib deprecation fix commit 3f114752c5eca1554fcebf044b604b60a3eebeae Author: Kendell Clement Date: Fri Aug 20 00:06:38 2021 -0400 add batch summary of frameshift/splicing Closes #116 by adding frameshift/splice summary to batch Version bump to 2.2.1 Fix unicode error in slugify commit 5ad9fbc6645475885fc0f35bc26afd353a2b3d5f Author: Cole Lyman Date: Mon Aug 16 14:16:13 2021 -0400 Read plain text fastq as bytes in filterFastqs commit 6c20f28678469875392e3fc8946018b53a00256b Author: Kendell Clement Date: Mon Aug 16 13:35:12 2021 -0400 Remove test for seaborn version commit cec965feff65b4228d42784e498398b73b8caa49 Author: Cole Lyman Date: Mon Aug 16 12:16:04 2021 -0400 Fix filterFastqs This commit fixes the filterFastqs script by properly casting to strings (from bytes) in the appropriate places. It also replaces the deprecated `np.fromstring` function with `np.frombuffer`. commit 3c30e56a15fa15a3537c3d51e429c1ff8738d6fe Author: Cole Lyman Date: Thu Aug 12 09:57:50 2021 -0400 Add .gitignore file commit 2461e30770d5ae8a6686530981a4bf5c913e2f68 Author: Cole Lyman Date: Thu Aug 12 09:50:01 2021 -0400 Decode result from Popen from bytes to string This is done only where a string is necessary, the instances where this is not done are where the result is cast to a float or int (because they can accept bytes as input directly). commit 64ec8bacf9ae8cb0d3a9093ad12a916983f32207 Author: Cole Lyman Date: Sat Aug 7 13:49:05 2021 -0400 Refactor `results/refs` and `results/ref_names` crispresso2_info fields commit ebb33a7d691b524ed7ab92b5a870da09df045e00 Author: Cole Lyman Date: Fri Aug 6 20:32:03 2021 -0400 Refactor `results/general_plots` crispresso2_info fields commit 94768e209e2424394efb4d902aac4f0e4268aad3 Author: Cole Lyman Date: Fri Aug 6 14:58:25 2021 -0400 Refactor `results/alignment_stats` crispresso2_info fields commit a58b7a2b40bc99346a9ea8f6240034963b3d6b31 Author: Cole Lyman Date: Fri Aug 6 14:10:33 2021 -0400 Refactor `running_info` crispresso2_info fields commit a92ea4687e0cc69844c5d4a6250757540076ed59 Author: Kendell Clement Date: Thu Aug 5 14:28:29 2021 -0400 Version bump to 2.2.0 commit 20e9b6f228949886ebe592d4542aaddbb9fb123a Author: Cole Lyman Date: Thu Aug 5 11:52:22 2021 -0400 Remove argparse dependency Remove the argparse dependency from setup.py because argparse is a standard module in Python 3. commit ecf70ea4379e079e7669fc5ed8baabdcd11a8c61 Author: Kendell Clement Date: Fri Jul 30 01:23:38 2021 -0400 Update Dockerfile bowtie throws an error 'Can't locate Sys/Hostname.pm in @INC' This fixes it. commit 30fb34e17787858d4218eb0b0fcc1baf6259ee23 Author: Kendell Clement Date: Fri Jul 30 00:39:48 2021 -0400 Ignore python egg for copying, pin tbb for bowtie error https://github.com/BenLangmead/bowtie2/issues/336 commit c850abb672019a3684a544fccde9550f7eeb86f5 Author: Kendell Clement Date: Thu Jul 29 20:13:32 2021 -0400 Update config.yml commit eb70cb18d6910f418bb8052f45b58c05c397af43 Author: Kendell Clement Date: Thu Jul 29 17:47:06 2021 -0400 Update config.yml commit 01f079fd663027574115a0af974564d5b5efbe97 Author: Kendell Clement Date: Thu Jul 29 17:22:48 2021 -0400 Update CRISPResso2_router.py commit 2e3b5d42b8ea6705dbd2c2f59312b93390083da3 Author: Kendell Clement Date: Thu Jul 29 17:20:16 2021 -0400 Update Dockerfile commit 46d8c44f2dbe7a67531d3ccae6b0a3c51b12b9ca Author: Kendell Clement Date: Thu Jul 29 17:07:02 2021 -0400 Update Dockerfile commit 0999e82dd8e184a6b0aefa7a35fc9decaa1fc20d Author: Kendell Clement Date: Thu Jul 29 16:59:47 2021 -0400 Update Dockerfile commit 34b93cfeeb95d0a1375378996e3ca29e79fe5311 Author: Kendell Clement Date: Thu Jul 29 16:50:20 2021 -0400 Python 3 commit e040ac488b050d6f316d21196de002ef09383915 Author: Kendell Clement Date: Tue Jul 27 10:04:43 2021 -0400 Add testing file for batch, remove debug print lines commit aa7b9998c4660438f4303a57386f01617ddec1bb Author: Kendell Clement Date: Thu Jul 22 10:43:52 2021 -0400 Update Batch to allow for max processes usage commit 1566a1110215e32f338497040f1279c3581b6dcd Author: Kendell Clement Date: Wed Jul 21 01:02:24 2021 -0400 Fix reference to BadParameterException commit fea5017109ab56c955b8053eebad5f6f4218bc65 Author: Kendell Clement Date: Mon Jul 12 16:11:05 2021 -0400 Allow spaces in run names, print run name to report commit b8594354c96131ba33c9277fb719c95b1cf72f3d Author: Kendell Clement Date: Tue Jun 29 14:12:03 2021 -0400 Update CRISPRessoPooledCORE.py Fix debug statement commit 9bc9b9959cbc8faf9dc462669e333298a80a71d1 Author: Kendell Clement Date: Tue Jun 29 14:09:13 2021 -0400 Update CRISPRessoPooled for samtools sort status updates Redirect samtools sort status updates to log file. Version bump. Previously this would cause an error: `ERROR: list index out of range samtools view: writing to standard output failed: Broken pipe samtools view: error closing standard output: -1`. commit ad5b9d853ac3559e58a5ad569e7d3ac9c3d8a719 Author: Kendell Clement Date: Thu Jun 24 12:10:31 2021 -0400 Allow max processes for CRISPRessoPooledWGSCompare Users can provide -p max to use all processes available for comparisons commit 48d6c8724f541b285e5d6889723cc37fc99e5cfa Author: Kendell Clement Date: Wed Jun 23 20:53:51 2021 -0400 Create html report for CRISPRessoComparePooledWGS commit abe3054dd9b95f51cbf34e3c37e088f6beb8287c Author: Kendell Clement Date: Wed Jun 23 20:53:22 2021 -0400 Fix allele plot bug #103 If no regions are returned, max of a pandas dataframe returns an error because the df is empty commit 1ccb507858d92516e28286358d3aae9cce2cf7ea Author: Kendell Clement Date: Wed Jun 23 20:51:16 2021 -0400 Fix command prompt logo lines commit 293c249c0f380576121a123dec982e40409c977e Author: Kendell Clement Date: Sun Jun 20 00:26:34 2021 -0400 Update plotCustomAllelePlot.py Get rid of debug statements commit 947fbab70e0f4fa78cc43273b4fe2be5225043cc Author: Kendell Clement Date: Sun Jun 20 00:22:56 2021 -0400 Add plotCustomAllelePlot script for replotting allele plots commit 167a48659684ce1669da915ddb6995935c6a1aa6 Author: Kendell Clement Date: Wed May 26 11:16:51 2021 -0400 Update README with explicit instructions to activate conda environment commit af0b7e441d2a4f1e035fabfd353c58784f688371 Author: Kendell Clement Date: Sun May 23 00:22:00 2021 -0400 Update test results for more flexible pooled alignment The relaxing of bowtie parameters allowed more reads to align, which changed the expected test results for CRISPRessoPooled commit 0e08cd05c2f3279ac95a068be76f1d36a4b0224d Author: Kendell Clement Date: Sun May 23 00:06:21 2021 -0400 Fix double rows in Alleles_frequency_table due to read directionality Previous to this fix, forward and reverse reads having the same sequence would appear in different rows in the Alleles_frequency_table. This fix doesn't change counts or results, just combines the two rows in the Alleles_frequency_table. commit 7d2d761a3d4c25915be0d27b36f3b4a87068587d Author: Kendell Clement Date: Thu May 20 16:05:27 2021 -0400 CRISPRessoPooled - get rid of -k bowtie2 parameter The -k 1 parameter may not yield the best alignment. This commit gets rid of that parameter. commit 44dc9e75c660f4b3b683f1c80f5a964aa55e75bd Author: Kendell Clement Date: Wed May 19 22:52:46 2021 -0400 CRISPRessoPooled alignment more tolerant When given a genome file, CRISPRessoPooled aligns reads to the genome using the Bowtie2 aligner. The legacy parameters were somewhat strict. The new parameters reflect the 'default_min_aln_score' parameter in allowing for substantially more indels and mismatches than previous. The parameter `--use_legacy_bowtie2_options_string` has been added to use the legacy settings. Otherwise, the bowtie2 alignment settings will be calculated as follows: commit 84b870ce0d489295501aa57711edd6b18c011b92 Author: Kendell Clement Date: Tue May 4 10:22:22 2021 -0400 Raise exception if quantification_window_coordinates looks like an int Previously, if quantification_window_coordinates looks like an int when pandas parsed a batch file, it would throw an error trying to split the int. Now an exception will be raised. commit e25a6dbb13fcd0565f4c81e3ea42cfd115aa1bc0 Author: Kendell Clement Date: Mon Apr 26 16:19:00 2021 -0400 Fix bug if all reads are discarded If all reads are discarded, CRISPResso would fail because the df_alleles is empty. This adds discarded alleles to the df_alleles. commit 156d7d640355ad4755c6ce4cbab1b75bf31677c9 Author: Kendell Clement Date: Fri Apr 9 13:24:51 2021 -0400 CRISPRessoPooled - close active file in demux commit c5d9fcbbf5bd9b307c9050498d08e72dd98d42aa Author: Kendell Clement Date: Fri Apr 9 12:29:02 2021 -0400 Keep old awk command for speed for samples with <50 amplicons commit c7539661fc5bd29f4f6ee1e9c7795be932b53a8e Author: Cole Lyman Date: Fri Apr 9 12:18:45 2021 -0400 Alt pooled processing implementation (#87) These changes implement separating reads to their corresponding amplicons via Python instead of through awk. This is to get around the maximum number of open files that is limited on many operating systems. Co-authored-by: Kendell Clement commit e57d98738fa832766c78dc7eecfdb0588d92634d Author: Kendell Clement Date: Thu Apr 8 12:36:43 2021 -0400 Alt pooled processing implementation commit 4598226837cd4b726ae38f50958f977bde4794ca Author: Kendell Clement Date: Sun Mar 21 00:16:19 2021 -0400 HDR Updates - yw #82 Multiple amplicon names are resolved before adding the HDR amplicon -- unnamed amplicons are named Amplicon{i} for each amplicon. Plot 4g data (nuc pct table, mod pct table for all reads aligned to the first reference) is output and linked to from the plot display Ambiguous reads don't contribute to plot 4g data (which would otherwise lead to double counting and pct values > 1) commit d7915b2c561173238c6b0e82863f76a783db7c4d Author: Kendell Clement Date: Sat Mar 20 01:12:55 2021 -0400 Fix #72 bam_input error commit 32a9e98ffc6ceb1dd56e5e29c369176ed788c985 Author: Kendell Clement Date: Sat Mar 20 01:01:17 2021 -0400 Prime editing, fastq_out updates Prime editing input parameters are forced to be in the RNA 3'->5' direction. This makes sure that the scaffold incorporation happens on the correct side of the extension sequence. Errors are thrown if improper directionality is detected. Fastq_out now includes alignment scores and details for every run (it may be time to upgrade that SSD to hold these new fastq output files, but it makes debugging particular reads a lot easier!) Update to linked data for plot 3b in report commit c1b6ea2ecebd920ea12ab91f2d50e35c274f94fe Author: Kyle McChesney Date: Wed Mar 10 13:40:44 2021 -0700 fix missing import path on NaN commit 1b24ca351f4337e0a7bece7ef7addb4558f950d8 Author: Kendell Clement Date: Sat Feb 20 22:57:07 2021 -0500 Update links in readme to https - fix #79 commit d550fd1db8ce0e1d11ed77fd082c53a3d887bbc2 Author: Kendell Clement Date: Fri Feb 12 16:57:37 2021 -0500 Insertion quantification change + fix #76 Starting in version 2.1.0, insertion quantification has been changed to only include insertions completely contained by the quantification window. To use the legacy quantification method (i.e. include insertions directly adjacent to the quantification window) please use the parameter --use_legacy_insertion_quantification commit 798f661031236ee1aa5611f491ca135ec0432dc9 Author: Kendell Clement Date: Thu Jan 14 23:51:29 2021 -0500 Allow for int-looking chromosome names in WGS input file In CRISPRessoWGS, the region file contains a 'chr_id' column which is sometimes mis-recognized as ints when read by pandas if using the chromosome notation without 'chr' (e.g. 1,2,3 in stead of chr1,chr2,chr3). This bug fix forces chr_ids to be read as strs. commit 92c008673690a3cd32e33fe8cfcf07060cf6cb0f Author: Kendell Clement Date: Wed Dec 30 23:48:32 2020 -0500 Update README.md commit 8d590c5588d93ef0129512a5156fe3b45e2d3c4b Author: Kendell Clement Date: Wed Dec 30 23:46:36 2020 -0500 Introduction of CRISPRessoAggregate to aggregate stats across runs Adds CRISPRessoAggregate Adds start/end time to CRISPRessoBatch info Started removing pickle dependencies from Pooled and Report commit c8447246ada2758831a870f30ed99c496c18feb1 Author: Kendell Clement Date: Sat Dec 26 10:52:12 2020 -0500 Output alignment details for unaligned reads in fastq_out or bam_out commit dedc36cadc64e9302d88c3472652d2a4a2385d25 Author: Kendell Clement Date: Tue Nov 24 13:45:34 2020 -0500 Add scripts folder for one-off analyses commit 88b6e9c50c6d97da357534c67aaeba64c00ddc23 Author: Kendell Clement Date: Sat Nov 14 02:46:36 2020 -0500 Fix plot window cloning from Ref1 to HDR Plot window for sgRNA will be the same length after cloning even if the window is shorter or longer after comparing between ref1 and HDR. commit 7403a46e186c4b70ad606dbb0eebbbde684fac61 Author: Kendell Clement Date: Sat Nov 14 01:52:55 2020 -0500 Frameshift plot and HDR quant window updates Frameshift plots don't show 0-bp changes (these dwarf all other changes). The number of reads not shown are added to the legend. Addressed cloning quantification windows when bases are inserted in the clone-ee (previously these cloned bases would be ignored. Force HDR to clone all quantification windows from Ref 1 Fix #60 and #59 commit f3f9c122cc25ab62bdd7f30fb9d6ee4c9ab820a8 Author: Kendell Clement Date: Sat Nov 7 23:55:55 2020 -0500 Update histogram x-limits, caption, and data commit e9332f7eabed32e65430b44677c841dd0150ac5a Author: Kendell Clement Date: Sat Nov 7 02:26:59 2020 -0500 Fixes for when no reads align #63 commit 9fae60a57d99238e4f7a9ad2e992ae71cca49c60 Author: Kendell Clement Date: Sat Nov 7 02:08:59 2020 -0500 Standardize pie plot appearances commit f1738bd17b496a31d6f68a91ed7ae67a36f8f178 Author: Kendell Clement Date: Sat Nov 7 00:36:40 2020 -0500 plotting and pe fixes - Special bonus for y'all to keep you company during covid - axis ticks on most plots! - added parameter --plot_histogram_outliers to plot all insertion sizes in histogram - all insertion sizes are reported in .hist output files #64 - add HDR reference plot (may change this later to set ref1 to the longer reference of WT/HDR but for now it is always WT..) Allow reverse complement of extension seq if PE sequence is specified. commit 5a2d9271b3ea99342f22e59259149d30dc193e47 Merge: bd03668 9812651 Author: Kendell Clement Date: Wed Oct 21 16:52:12 2020 -0400 Merge pull request #62 from matandro/patch-1 Fix a bug when generating compare plot commit 9812651c2aae1218393e8ce0d734812247cd59ab Author: Matan Drory Retwitzer Date: Wed Oct 21 10:45:01 2020 -0700 Fix a bug when generating compare plot Related to issue #61 This happens when N_ROWS < 1 which I assume has something to do with no results -> negative control commit bd03668ef1626a54e0b5bfbc010afacee60f45e3 Author: kclem Date: Fri Oct 2 17:39:52 2020 -0400 delete merging intermediate files commit 3c9b1344e19dee04b169c0860bc11304d6a594b5 Author: Kendell Clement Date: Wed Sep 30 23:51:08 2020 -0400 Version bump to v2.0.42 commit 2f8c6f9c7d31b276ff18000d579ef722f693c433 Author: Kendell Clement Date: Wed Sep 30 23:44:02 2020 -0400 Update new parameters, fix docker biuld problem commit f130270135b84e0904e0003858c263285834b6dd Author: Kendell Clement Date: Wed Sep 30 14:18:58 2020 -0400 Fix bug in read counting for interleaved fastqs commit faac4e1045e5b617b3743e328c0203ced27e3f31 Author: Kendell Clement Date: Thu Sep 24 22:37:50 2020 -0400 Fix bug in mode to write fastq out commit 388e8a97fc18648dd28e35063cb12680887395e2 Author: Kendell Clement Date: Wed Sep 23 23:49:44 2020 -0400 Update postrun reference output file Rename postrun references file to be more standardized with other output files. Output is now "CRISPResso2Pooled_postrun_references.txt" commit ee5be76e5e24e494abd93940622d51faa83a2371 Author: Kendell Clement Date: Wed Sep 23 23:32:34 2020 -0400 Multiple allele support for pooled mode Most common alleles for each pooled target are output if the flag '--compile_postrun_references' is provided. This writes alleles with frequncy defined by the parameter to --compile_postrun_reference_allele_cutoff This file can be manually edited to remove noisy alleles, and then used to run CRISPRessoPooled again but to provide alternate alleles to each CRISPResso run by using the parameter '--alternate_alleles'. This is particularly useful in cases where control experiments are available. The running pattern would be: 1) CRISPRessoPooled --compile_postrun_references {control} 2) CRISPRessoPooled --alternate_alleles {produced in step 1} {control} CRISPRessoPooled --alternate_alleles {produced in step 1} {experiment} commit fe5bee2ac7ca4a8dd165e2c7305a1c41a79b8d9d Author: Kendell Clement Date: Wed Sep 23 23:27:37 2020 -0400 Output + PE updates Add parameter '--fastq_output' to output a fastq file with annotations for each read. Also, the substitution frequency table is only produced in base editor mode -- in an effort to slim down CRISPResso2 output files. Also, especially for long Prime Editing insertions or deletions, the inference algorithm may incorrectly infer the prime-edited allele. I added the parameter '--prime_editing_override_prime_edited_ref_seq' which can be used to specify the prime-edited sequence manually. commit ca61c483a8a63fec2e85889f554648ea6f3a903c Author: Kendell Clement Date: Sat Sep 5 16:15:16 2020 -0400 update report caption for fig 10 commit 346c6f6e62097ea7690204cfe879ebb918fb244d Merge: e52cb73 44ccc5e Author: kclem Date: Fri Sep 4 15:05:02 2020 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit e52cb73471f4538ecd98f943a38771f0d07a4476 Author: kclem Date: Fri Sep 4 15:04:48 2020 -0400 Update on help message for mark wildtype allele commit 44ccc5ec3ff6a57d265807b559010512f053d82a Author: Kendell Clement Date: Fri Sep 4 14:59:05 2020 -0400 Update to plotting quantification window plot vertically centered, pooled/wgs plots are not limited in height to allow analysis of large pools. commit ff675b12196593ceb8e8d8b58dfd6a8ca8865501 Author: Kendell Clement Date: Mon Jul 27 09:55:30 2020 -0400 Update parameters and description in README commit 25c6a1b45ef1e1c1243057b9b3ee06f638d55d26 Author: Kendell Clement Date: Sat Jul 25 00:29:14 2020 -0400 Update CRISPRessoBatchCORE.py If amplicon sequence is empty, auto mode is run for batches commit 2978a6ebc0cd977b36b4ea7e1965f5c43b46d325 Author: Kendell Clement Date: Thu Jul 23 08:46:01 2020 -0400 Removed WGS logging For some reason, piping output breaks multiprocessing blocking commit 8bc9214ba0a1b628b69a49a2cf306bf559e008b3 Author: Kendell Clement Date: Wed Jul 22 22:58:51 2020 -0400 Update CRISPRessoWGSCORE.py commit a9484fd481bfa8ad716c6326aba9c57f5271f885 Author: Kendell Clement Date: Wed Jul 22 14:24:38 2020 -0400 Add parameter discard_guide_positions_overhanging_amplicon_edge If run with param -- discard_guide_positions_overhanging_amplicon_edge, for guides that align to multiple positions, guide positions will be discarded if plotting around those regions would included bp that extend beyond the end of the amplicon. Normally this would cause CRISPResso to fail if plotting were requested beyond the end of an amplicon. commit 8d26edeed89e33451bd539c242f7ffaa560db3e6 Author: kclem Date: Wed Jul 22 13:58:10 2020 -0400 WGS doesn't print crispresso output to screen WGS printing error fixed commit 5ed03059d09aaf8a416c5fec553801eeca73355f Author: kclem Date: Tue Jul 21 00:00:40 2020 -0400 bam processing update of cached not-alignments commit cf593183d7b3d8200ec295ebcc653e518775046a Author: Kendell Clement Date: Thu Jul 16 21:49:21 2020 -0400 Fix naming of scaffold-incorporated reference links on plots commit 676ff623373b0cabf992017bd5eca255c4073d2a Author: Kendell Clement Date: Fri Jul 10 00:02:59 2020 -0400 Prime editing updates prime editing guides are shown as input on report output file names are produced without spaces commit 9de370148b2ada7210c10532247b3fa4b18a76de Author: Kendell Clement Date: Tue Jul 7 20:46:30 2020 -0400 Update batch for multiple quant window input commit 6ad1822f478d7f108b92f25378cc3ddf8dc3a2d3 Author: Kendell Clement Date: Wed Jul 1 16:26:38 2020 -0400 Bam processing + Prime Editing updates -Input can now be read from bam using the parameter `--bam_input` and (optionally) `--bam_chr_loc` to use the reads in the bam at this location as input. An output bam is produced with an additional soace-separated field prefixed by c2 (e.g. c2:Z:ALN=Inferred CLASS=Inferred_MODIFIED MODS=D47;I0;S0 DEL=56(47) INS= SUB= ALN_REF=TTGGCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGAAGTAGGGCCTTCGCGCACCTCATGGAATCCCTTCTGCAGCACCTGGATCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCT----------------------------------------------- ALN_SEQ=ACACCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGA-----------------------------------------------TCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCTGGAGCGGCGGCTGCACAACCAGTGGAGGCAAGAGGGCGGCTTTGGGC). Note that the alignment details (location, cigar string, etc) are not modified.. this may be done in the future). Bam file input cannot be trimmed or pre-processed with quality filtering. -Prime editing scaffold incorporation is now more accurate (looks for the scaffold sequence at the expected position directly after the extension sequence). A plot showing the number of bases matching the scaffold, as well as insertions after the extension sequence, and a data file with these numbers is produced. Added parameter `--prime_editing_pegRNA_scaffold_min_match_length` to define the minimum length required to classify a read as 'Scaffold-incorporated' -Renamed split_paired_end parameter to `--split_interleaved_input` for interleaved input -Auto mode now considers 5000 reads to detect amplicon sequences -Add new paramter `--annotate_wildtype_allele` to annotate wildtype alleles on the allele plots -Update output when reporting missing files -- only lists first 15 files in the current directory and directory of input parameter --reference https instead of http commit c0f2871befed86c4b314100584bf844f13d71d0e Author: Kendell Clement Date: Tue Jun 2 18:54:52 2020 -0400 Update CRISPRessoWGSCORE.py Remove debug print commit e5450b1cc6be518706969d27bda843cb7c16b082 Author: Kendell Clement Date: Tue Jun 2 18:53:20 2020 -0400 Updates for Pooled and WGS WGS gene annotations compatability fix and pooled gzip fix commit c2b286dbda3807752f34cdf6274e41d5640b408f Author: Kendell Clement Date: Tue May 12 00:33:36 2020 -0400 Fix docker bug, print version to Pooled + WGS commit 2a06cc18c3157e6c135dae7ce53adec94fe8a83f Author: kclem Date: Sun May 10 00:09:55 2020 -0400 Fix plotting bug if no sgRNAs given commit 1e3ca605e0c5ae65e8acc727667f952bc4c0d3ee Author: kclem Date: Sun May 10 00:00:32 2020 -0400 flexiguide fix commit 4e1d6b2b3424e725010e3e1a13522a7386228853 Author: Kendell Clement Date: Sat May 9 23:30:39 2020 -0400 Prime editing refinement and Pooled filesystem demand reduction - Prime editing parameters renamed, nicking guides match with flexibility - Prime editing extension seq is shown as a guide (with no cut site) - Prime editing summary plot included in report - Nucleotide plots are shaded when no changes from the reference sequence - sgRNA annotations are plotted on multiple lines if they overlap - N's don't count as substitutions - extended read analysis data available with --write_detailed_allele_table flag commit 8c584719b9771c01e53cfe409789b29a29fad665 Merge: f506936 7624815 Author: Kendell Clement Date: Sat May 9 23:14:30 2020 -0400 Merge pull request #42 from ronaldhause/patch-1 ZipFile: set allowZip64=True to write larger allele frequency tables commit f5069365d37bd698ff3bf35e5c39a1b75e10dc1d Author: kclem Date: Sat May 9 12:04:10 2020 -0400 CRISPRessoPooled updates, fix #37 for too many files Demultiplexing in the case of amplicons + genome is parallelized to reduce sorting Only files with sufficient reads are demultiplexed and written Additional output file REPORT_READS_ALIGNED_TO_GENOME_ALL_DEPTHS.txt shows all alignment locations commit 7624815e3159d926d8a0710f674f44c616b68bd5 Author: Ron Hause Date: Sun May 3 22:50:07 2020 -0700 ZipFile: set allowZip64=True to write larger allele frequency tables Addresses terminating ERROR: Filesize would require ZIP64 extensions when trying to write compressed allele frequency tables > 2 GB commit 6af15a09033166b713df159a6ef850dde8867253 Author: kclem Date: Tue Apr 28 03:28:41 2020 -0400 int bug fixes commit 531753c0f5be89c255f65d876cba5e9bf00dd4a2 Author: Kendell Clement Date: Tue Apr 28 02:00:37 2020 -0400 Introduces support for prime editing, multiple window sizes and offsets, max processors commit 8e29c1e0966ebb2073d52a221ae77e56bb146431 Merge: adb0d8b 039013e Author: Kendell Clement Date: Fri Apr 24 13:06:46 2020 -0400 Merge pull request #41 from natecarlson/fix-name-error If the name column is called 'name', refer to it as 'name', not '#name'. commit 039013efe363603dfe89f10b857d58bb1ef8e8d9 Author: Nate Carlson Date: Fri Apr 24 09:46:21 2020 -0500 If the name column is called 'name', refer to it as 'name', not '#name'. commit adb0d8b791686846fa8522f46e064418cbfbdc1c Author: Kendell Clement Date: Mon Apr 20 16:26:32 2020 -0400 Update LICENSE.txt commit b514a68eea135e805333c67bf33bf0f1ea41a034 Author: Kendell Clement Date: Sun Apr 19 00:36:26 2020 -0400 Print CRISPResso command on multiprocessing fail commit 4bfadd08d30e1a8b926757c1af4a9c0c0dc0b484 Author: Kendell Clement Date: Sun Apr 19 00:03:58 2020 -0400 Index fix for crispresso multiprocessing Indexes were incremented for user enjoyment (1-based) but the more accurate approach is 0-based Also, no_rerun ignores changes to the flags 'debug' or 'n_processes' which shouldn't affect the outcome commit 867e692b326acd19ed5f291bd5699f5885b1d569 Author: kclem Date: Thu Apr 16 00:25:47 2020 -0400 Allele plot sgRNA labels stay on plot commit b0d89b4effc4242ec55ed4a3e20e8835a90e3588 Author: kclem Date: Wed Apr 15 23:58:46 2020 -0400 WGS fastq seqs are now uppercase, so guides match even in lowercase-masked genomes commit 8098f1f1dda6efdaf773b9f85622d79f32ac49c9 Author: kclem Date: Thu Apr 9 01:11:26 2020 -0400 Pooled multiprocessing updates commit 034ff2b5858dd29ab60017835695fad527a5213e Author: Kendell Clement Date: Tue Apr 7 02:16:39 2020 -0400 add n_processes param for pooled analysis commit 7fb477ff15c7f8d62d3acad4d14434942cd25bff Author: Kendell Clement Date: Tue Apr 7 00:36:47 2020 -0400 Pooled Set flag to skip reporting problematic regions commit 6c3aeff8b96d918ef2b3c518b73860d3f74480b8 Author: Kendell Clement Date: Mon Apr 6 19:56:56 2020 -0400 Pooled parallelization by chr Parallelized CRISPRessoPooled extraction to operate by chr Attempt to appease the dockerhub requrements -- require cython for compilation commit a66c4020f473d2dee80fa1257c04049b2ba6dbd3 Author: kclem Date: Fri Apr 3 13:41:38 2020 -0400 v2.0.33 plot updates allele plot sgRNAs that extend beyond plot are marked quantification window shading and right-side correction version commit 90e677f453ef971b478e4582120b17cbb572212c Author: Kendell Clement Date: Fri Apr 3 01:30:57 2020 -0400 Pooled bug fixes for regions with the same location and different names commit d795479d117e689a6679ffe463df413bff2f6a5a Author: Kendell Clement Date: Fri Apr 3 01:23:30 2020 -0400 Fix open error for docker commit 9aee866e5f65bf8df6495e81684651436a0b0a30 Author: Kendell Clement Date: Fri Apr 3 01:17:09 2020 -0400 Parallelization of Pooled and introduction of checkpoints for WGS and Pooled Alignment of amplicons is done in one bowtie2 call instead of one bowtiecall per amplicon Parallelize several time-intensive steps in Pooled (splitting by region, etc) --no_rerun flag will skip already-processed steps in Pooled and WGS commit fb1e7e25404bd80722824755c1b4ff4478449c48 Author: Kendell Clement Date: Thu Apr 2 01:34:52 2020 -0400 WGS updates - multiprocessing and no rerun WGS multiprocesses extraction of reads across multiple cores. WGS extraction doesn't occur if the --no_rerun flag is set and the files are all present. commit 45d4377f678fceeff83d60efa22dbd1078e6840e Author: Kendell Clement Date: Tue Mar 31 10:35:43 2020 -0400 Remove biopython for fastq parser Living life on the edge -- dropping the biopython fastq parser to remove biopython package dependency. This will discard the minimal error-checking provided by the biopython package. commit d52f69c9fa16ea38e919f9e3dc70fb6d53246610 Author: kclem Date: Tue Feb 25 17:05:08 2020 -0500 Pin dependency versions commit 86ff4bebcfcaa5c618ff124822ecb45de2b7c9ca Author: kclem Date: Tue Jan 28 16:17:57 2020 -0500 get rid of test for CRISPRessoCompare commit b7e96d6f4d1e0966cc0847b620349ee7fe21f43f Author: kclem Date: Tue Jan 28 15:26:20 2020 -0500 Fix problem with circleci testing Switch columns for output quantification files so that total reads is before the number of aligned reads commit ac976074c03967a386ed386d9ce3436c5fa1f2d5 Author: kclem Date: Fri Jan 24 14:02:14 2020 -0500 Version 2.0.32 - guide updates and other updates Introduced flexiguides - can match with flexibility to the reference sequence - useful for pooled screens of offtargets (parameter --fg or --flexiguide, with --fg or --flexiguide_homology to control how many mismatches) Flexiguide mismatches and other mismatches are shown on plots sgRNAs can be labeled (parameter -gn or --guide_name) sgRNA position shown on allele frequency plots detection of dsODN -- shown in plot 1d (parameter --dsODN) CRISPRessoPooled gene set input flexibility - more formats accepted plot 8 shown on html report commit ca9273377acbabf01f97f04a028d4fd87a09d6b5 Author: Kendell Clement Date: Fri Dec 6 15:14:38 2019 -0500 Fix docker bug #30 commit a3ae575f870ace49a459447e13288cfd50487e2f Author: kclem Date: Wed Oct 2 20:57:03 2019 -0400 remove debug statements commit 53d70eb83bad2aa8456b744dd29611a788f3dbbd Author: kclem Date: Tue Oct 1 15:41:39 2019 -0400 Fix #25 to accept bt2l bowtie2 index extension commit 039cc8e1b4a145e53cf06c1d52f102497928f3ac Author: kclem Date: Wed Sep 25 17:53:49 2019 -0400 v2.0.31 CRISPRessoPooled chr names fix, allele plot colors CRISPRessoPooled compatability with chr names with underscores (alternate scaffolds) Additional function to plot allele table with custom set of colors for a completed run commit c35d0151efcd70a6044399e16c841ee9d0ad0535 Merge: 491247e b1d1518 Author: kclem Date: Thu Sep 19 16:23:13 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 491247e1a07e2991a74341b4424c7c722a3db6a9 Author: kclem Date: Thu Sep 19 16:22:56 2019 -0400 updates to command line output, batch bug commit b1d1518a7ed4b72e92fd9d10586ccce653585630 Author: Kendell Clement Date: Fri Aug 23 11:26:53 2019 -0400 Update conda installation path commit cdfd78772de27bdb78a380e591d8857acd143562 Author: kclem Date: Tue Aug 13 11:24:58 2019 -0400 Use python 2.7 pandas commit 1417d1aa387d45f37953b93304a545e08943cc45 Author: kclem Date: Tue Jul 16 17:18:28 2019 -0400 force merge reads and fix #19 add optional param for force merging R1 and R2 in case they don't cover the entire amplicon fix labels for expected amplicon plots commit 60f90053ab65958871402b609d0b72b31021bc6b Author: kclem Date: Tue Jul 2 17:28:27 2019 -0400 v2.0.30 case-insensitive guides, fix #17 commit 702b9dce89e070d96064db314ac1c5155fedd42e Author: Kendell Clement Date: Tue Jul 2 16:18:17 2019 -0400 Update README.md commit 5a10eb9a9d24371d2bedf8b173f9c7401d7cbd53 Merge: bc7b332 bc006c8 Author: kclem Date: Tue Jun 25 15:32:51 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit bc7b3321e1bb6b7ed0c8af48d609e86e55081f6b Author: kclem Date: Tue Jun 25 15:32:45 2019 -0400 plot window bug update commit bc006c826f338b9a1fcaec2286b506bdd2c548db Merge: 1f7171b feee2c2 Author: Kendell Clement Date: Thu Jun 20 00:56:45 2019 -0400 Merge pull request #16 from PEHGP/patch-1 args.trimmomatic_command for CRISPRessoPooled commit feee2c2eb20807933376f01a88102767ce5e42e4 Author: kuan <396777306@qq.com> Date: Thu Jun 20 09:44:32 2019 +0800 args.trimmomatic_command args.trimmomatic_command commit 1f7171b8eb2dcd63fada80fa6cac561e576f727c Merge: f6c9eed 3d4a37a Author: kclem Date: Thu Jun 13 13:13:33 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit f6c9eed206b16fede7c88724c94d8d091f8657f3 Author: kclem Date: Thu Jun 13 13:13:20 2019 -0400 update pooled names commit 3d4a37a626f366f8f024ee7f62e017e8d68092b3 Author: Kendell Clement Date: Tue Jun 11 12:33:40 2019 -0400 Add web link to readme commit 89348066ede5c447db9f04b2337118a0624b8f63 Author: kclem Date: Tue Jun 11 11:14:55 2019 -0400 add batch percentage report commit 3bcd50d67c9b320c9f908eb2e3469143461e7df6 Author: kclem Date: Thu May 30 17:25:05 2019 -0400 document report param commit 79303435747a70b288b88bb076dda90b8d379f91 Author: kclem Date: Thu May 30 17:16:11 2019 -0400 output updates commit 9f14424f275f132a148308042db48c77cbda9b1d Author: kclem Date: Fri May 24 17:57:57 2019 -0400 plot updates, compare bug #12 commit 7f5482c411a3d56a73d7f0c66f0159b623653c2a Author: kclem Date: Tue May 21 17:47:08 2019 -0400 remove crispressocompare test condition (cuz has floats) commit 5e2f4094ec607f63981cd5aa2ee7f99ce1a20d12 Merge: aac9dfe 5933d93 Author: kclem Date: Tue May 21 17:40:44 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit aac9dfe70292a90d6981b0859ff9ede8da7094a4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test changed test to something that won't be affected by float formatting commit 5933d93c5f1408f754895254306701b5b0eaadd4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test commit 7d15cdfaa4a323f4baad96bf36c97fd455dd6684 Author: kclem Date: Tue May 21 17:11:09 2019 -0400 Add compiled c file commit 50134d64d33dc984ac81a066e7c370b160b861b3 Author: kclem Date: Tue May 21 16:58:57 2019 -0400 v2.0.28 Add report for CRISPRessoCompare Standardize naming conventions for files and plots from CRISPResso Add data links to CRISPRessoBatch report CRISPRessoBatch plots using the plot window around the cut site instead of only the quantification window If only one reference, 'Reference' is not shown in data plots or as a file prefix Set plotting indexes once for each guide (previously, they were specific to the amplicon) Base editing plots now plot for each guide (previously, they were one for each amplicon) commit 9e86bef89884a0e5980c7781cfcd56243fdd42f0 Author: kclem Date: Wed May 15 14:28:14 2019 -0400 Standardize concept of windows for quantification and plotting #11 commit dd63974334c94816efe5878e4aab1ec3c3e1a6c5 Author: kclem Date: Wed May 1 14:49:24 2019 -0400 python division bug.. <3<3 commit 4a4b88885ab2e034bdcbca9566aebe911c58f427 Author: kclem Date: Wed May 1 14:35:33 2019 -0400 fix bug for spaces in filepaths commit f7e0aee5d948abd876b5048c87cae59e265d0c6f Author: kclem Date: Fri Apr 26 11:56:51 2019 -0400 min merge size commit 12cc6a93c0b02be86575c6c7fe8b7569a96e0edd Author: kclem Date: Fri Apr 26 11:41:09 2019 -0400 Relax flash merging, add parameter for stringent flash merging (#7), remove debug statements commit 38bfb3174d8d37297587243cb9e04469e7e54a20 Author: Kendell Clement Date: Thu Apr 25 22:50:08 2019 -0400 Update CRISPRessoPooledCORE.py fix numpy -> string error commit 273fe3a1ab731a32c26efdcab58a502cd2162104 Author: kclem Date: Mon Apr 15 13:38:17 2019 -0400 Fix CRISPRessoWGS tests commit 2c1ec9f5eadaf62ac7df35535aa26965d993e96a Author: kclem Date: Mon Apr 15 13:15:03 2019 -0400 add tests for CRISPRessoWGS commit 6632700fbde6b7f5e49e402471fea88a0966d2c0 Author: kclem Date: Fri Apr 12 10:29:43 2019 -0400 update docker run message, enable local testing, remove networkx commit c31355e15afc49f6aa069968c94408f5f7b484c0 Author: kclem Date: Thu Apr 11 16:00:44 2019 -0400 ignore test directory for docker commit aad55c922fa9b3948e5b194b26da3a8a3ccc97d2 Author: kclem Date: Thu Apr 11 15:52:45 2019 -0400 fix plot label bug for 10a, update fig 2a data commit c40b2de4f21e69404de153b9e12dee6654568b67 Merge: 560b65e 88990db Author: kclem Date: Wed Apr 10 17:30:41 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 560b65e720a6dc5329203ecbc8c1b38b47aee219 Author: kclem Date: Wed Apr 10 17:30:34 2019 -0400 update tests for decimal shift for percentage in summary report commit 88990dbb29135d160879ed02dcac6874b0fed9f2 Author: Kendell Clement Date: Wed Apr 10 17:26:20 2019 -0400 Update README.md commit e4deb9211dadb3e9622f2eee38f0cc4930a0cf17 Author: kclem Date: Wed Apr 10 17:19:07 2019 -0400 v2.0.28 commit 098f5a68ba632987930fd70038af667e228b7a1f Author: kclem Date: Tue Apr 9 22:13:13 2019 -0400 update circleci path commit bb78ade66d20a826bbd8beee53711620869b86aa Author: kclem Date: Tue Apr 9 17:40:43 2019 -0400 circleci test path update2 commit e760c77bdd23fd087733c26eda86f15de6877bbf Author: kclem Date: Tue Apr 9 17:36:23 2019 -0400 circleci test path update commit 0e1b576959b09da72b101983d312842e06ea4c56 Author: kclem Date: Tue Apr 9 17:33:22 2019 -0400 circleci artifact storage commit 28c23d2172cf5c39faffd1ea498f6d771237567d Merge: c7da846 e42b3a6 Author: kclem Date: Tue Apr 9 14:17:34 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit c7da84668556b897b43dd53eb2370c3404ab3fa2 Author: kclem Date: Tue Apr 9 14:17:23 2019 -0400 circleci testing for batch and pooled commit e42b3a6b4c11cab0ac9c29c378858706b7fd328e Author: Kendell Clement Date: Tue Apr 9 11:55:04 2019 -0400 Update badge links commit 6a435376fe43e6408507f1078518515f1f522ba8 Author: Kendell Clement Date: Tue Apr 9 11:42:21 2019 -0400 Got me some badges! commit f8c9713444472f5e3cef4fd7f7bce0704dd6a266 Author: kclem Date: Tue Apr 9 11:07:18 2019 -0400 circleci artifact update commit 8d4b825dcf01887db96b90640aafea564031a7e9 Author: kclem Date: Tue Apr 9 11:03:59 2019 -0400 circleci updates commit bfb3bc82abd22647554b80517554ef58e81d2090 Author: kclem Date: Tue Apr 9 10:51:53 2019 -0400 add expected results for circleci commit 8466e146d73a2c2149fdbcd67d4db656fe8c13e0 Author: kclem Date: Tue Apr 9 10:29:57 2019 -0400 circleci update commit 19c437975e57119531bcb0b9b2a6587a7d6b3ea7 Author: kclem Date: Tue Apr 9 10:14:42 2019 -0400 circleci update commit c665c19c97f4c6d4b45f40b3ac9522bf53ce05ba Author: kclem Date: Mon Apr 8 16:27:38 2019 -0400 circleci - use custom docker commit caa46ce2bc1de5e100bfd5db25179fb6b54f4d95 Author: kclem Date: Mon Apr 8 14:45:16 2019 -0400 python2 virtualenv commit a13e2fd2c9eea59652ec1ad7c6a9862a708bc33e Author: kclem Date: Mon Apr 8 13:54:32 2019 -0400 CircleCI testing commit 7257b54f77391da21532bf06ed84b1ce03d460e0 Author: Kendell Clement Date: Fri Apr 5 11:15:22 2019 -0400 Remove dependency on zip commit 7b694f507a61e424e994384cd765855d7f69dfb4 Author: kclem Date: Thu Apr 4 12:12:37 2019 -0400 add networkx requirement for py27 commit 099acbc8149dbbd5c99729ddc8e0928e3f890023 Author: kclem Date: Thu Apr 4 10:16:10 2019 -0400 Fixes for dockerhub commit 3a3bfbdd2373348901faac6fda468b70cb5ce725 Author: kclem Date: Thu Apr 4 10:09:06 2019 -0400 Bioconda submission fixes commit dff86e16812f2b9345ef86bd3185786f4f82d25a Author: kclem Date: Wed Apr 3 18:49:13 2019 -0400 More precise cleavage window and quant window plotting commit c8caf7fbdad415d4d3fb47cc5b9b0b76dd65deab Author: kclem Date: Wed Apr 3 17:49:54 2019 -0400 v2.0.27 add reports for pooled and WGS commit c68e3cb922d037c12a62c695c0ae1f6c148ace82 Author: kclem Date: Mon Mar 18 11:15:40 2019 -0400 batch info pickle, WGS bai location, meta mode commit 768c75c3a7864c365cf13fd65573859b9aa86ebe Author: kclem Date: Wed Mar 6 17:13:56 2019 -0500 v2.0.26 add report display name, remove paths from stored files, fix sgRNA plot, CRISPRessoPooled report HTML, add citation to report commit 58257b54fc440427e5437f8e7458fd5824020b6e Author: Kendell Clement Date: Fri Mar 1 16:55:27 2019 -0500 Update issue templates commit 50fb2d58f0d3777ba51d0f5a37e82cfc1a47ebff Author: kclem Date: Fri Mar 1 16:38:13 2019 -0500 Fix file naming and flash incompatibility commit eca34aebf86dc1b03c6107ee405cdf16898c8d51 Author: kclem Date: Wed Feb 20 17:26:42 2019 -0500 v2.0.25 Add inferring of guides commit 2ccec08691db17e6a2cfab8e310f5f29621319ca Author: kclem Date: Wed Feb 20 13:42:57 2019 -0500 quant window updates commit 21d558da1f8f5fed8f59de0eb154e5a8a505ff7a Author: kclem Date: Tue Feb 19 17:21:12 2019 -0500 add flash outies, fix quant window coords bug commit 22954e6d40ad2d237cb75c521a09f30c2b294066 Author: kclem Date: Tue Feb 19 11:17:30 2019 -0500 Update entrypoint for docker commit 0216329d8a8d1c072a269d14786820f943443153 Author: kclem Date: Wed Feb 13 16:40:39 2019 -0500 v2.0.24 update docker, setup.py commit e11b60fe1cf3c3e67e41964d45e843f28c2975a5 Merge: fb1687b d6a7b97 Author: kclem Date: Mon Feb 11 16:22:09 2019 -0500 v2.0.23 suppress plots, custom flash version commit fb1687b87de0cb7e5d1ab0acc2a2651d1be1fcec Author: kclem Date: Mon Feb 11 16:12:26 2019 -0500 v2.0.23 suppress plots, custom flash version commit d6a7b979e1ecd49b26c648f2007e1ecfa905ec14 Author: Kendell Clement Date: Fri Feb 1 12:20:50 2019 -0500 Update readme formatting commit 9e4c87c4f60edd82539b01b24f646962c0f06f4c Author: Kendell Clement Date: Thu Jan 24 17:13:28 2019 -0500 Add conda install instructions commit 4b2b06e52ee1aba4f3bdd0458c13813f2417fbf7 Author: kclem Date: Thu Jan 24 14:02:05 2019 -0500 add manifest.in commit 495e1f9829d6e72048b87a6972472c4242411286 Author: kclem Date: Wed Jan 23 11:05:44 2019 -0500 Change license location, license update commit 24c3b1b6ba66557b99469c0e12291fae2fcc800d Author: kclem Date: Wed Jan 23 11:01:44 2019 -0500 v2.0.22 commit 4a426bf715e37cbb8d619d54fc3a92385e9dcf1b Author: kclem Date: Tue Jan 22 15:25:13 2019 -0500 update batch amplicon naming commit f4fbe96b335c6e1a5b3dc252bf090e3d36ffb8cd Author: kclem Date: Tue Jan 22 12:44:00 2019 -0500 v2.0.21 detangled root location dependency from params commit 9d29737de257a2999f0763afd5f97ddafd6d2fe7 Author: kclem Date: Tue Jan 22 12:36:03 2019 -0500 Update CRISPRessoShared.py commit 573aa90cf70cd941c3e26361b2e656fbf34d8e5e Author: kclem Date: Fri Jan 18 18:10:47 2019 -0500 allow no cython commit 69811cf1eb58ac014a1ac34c56748aaffcead187 Author: kclem Date: Fri Jan 18 17:55:23 2019 -0500 require cython for installation commit 37ac0e6278c4825bb816c7cfe0a1fc3819abd516 Author: kclem Date: Fri Jan 18 17:44:19 2019 -0500 v2.0.20b prepare for bioconda integration commit 31c5ad02127366d0ace2849a6e996a86d690f4d1 Author: Kendell Clement Date: Tue Jan 15 17:22:53 2019 -0500 Update README.md commit 5b8cf82bfee3eeefcf2ba5bb31b4a60b25960df0 Author: Kendell Clement Date: Tue Jan 15 16:46:46 2019 -0500 Update README.md commit c932b8b4ad7c08d3fc94d5c57d0017001a78a42f Author: Kendell Clement Date: Tue Jan 15 16:40:59 2019 -0500 Update README.md commit 5cf5aa34b2ef97e38e45ee3394cd8b4aade50c6d Author: Kendell Clement Date: Fri Dec 21 15:03:50 2018 -0500 Add trimmomatic_command parameter commit 2ec374962c169c1a56cd48ff9080f7941433d016 Author: kclem Date: Thu Dec 20 18:30:01 2018 -0500 v2.0.19b HDR and WGS changes commit 78483624993c6fdb5ccc889a2a6f37036fb0f2c8 Author: kclem Date: Thu Dec 6 14:55:18 2018 -0500 add filtering for fastqs commit 17940c70b36d74d8b9134664b515f3b164c678b0 Author: kclem Date: Wed Nov 28 15:00:35 2018 -0500 v2.0.18b - fix bug with batch names, add param to suppress plots commit 038042e3cea983fc264460402d88e15766cc50b0 Author: kclem Date: Wed Nov 14 15:00:47 2018 -0500 Add router for docker commit 3ef96cc08a18f6eff9bb6115366e27304986ed27 Author: kclem Date: Wed Nov 14 14:52:41 2018 -0500 add Docker file commit 3e1cbf1dffc4dbc555942d45f91ad942237b81c3 Author: kclem Date: Wed Nov 14 14:29:44 2018 -0500 set white background for plots commit dd2cb254170b8363a7feb0427ed587d2093ebd3e Author: kclem Date: Wed Nov 14 11:13:02 2018 -0500 Set seaborn style commit dde6a675a5ba4e9ec100c29c2d59534aa39ef4fc Author: kclem Date: Tue Nov 13 16:13:55 2018 -0500 Fix line endings commit db0a67017f57c5a77bed9eb442cb7efa9df4b97a Author: kclem Date: Tue Nov 13 10:31:45 2018 -0500 v2.0.17b - bug with multiple references of different lengths commit 7f4afffa1485094aee4c0399a319a30c12ec473e Author: Kendell Clement Date: Tue Oct 23 17:22:07 2018 -0400 Update README.md commit 0843923b50d2db953600230ac23614f52591c4b4 Author: Kendell Clement Date: Tue Oct 23 16:59:48 2018 -0400 Update README.md commit 6f79084ce88502a40fe8b9b1839733ca224d14dd Author: Kendell Clement Date: Tue Oct 23 16:56:38 2018 -0400 Add files via upload commit 72d0b355b5d39164405b6311bda6231dbbdef371 Author: Kendell Clement Date: Tue Oct 23 15:44:26 2018 -0400 Update README.md commit 30fdc7fd5d73594b332efd546a1f4d85004a80d6 Author: Kendell Clement Date: Tue Oct 23 13:48:25 2018 -0400 Update README.md commit 5b11e51083ca87cbc1a8dff02b4634fe12176d29 Author: Kendell Clement Date: Tue Oct 23 13:42:11 2018 -0400 Update README.md commit 33d367310d702667d86202aba389a0fee4eba691 Author: Kendell Clement Date: Tue Oct 23 13:24:50 2018 -0400 Update README.md commit d6c647d32cdded7803f6b023949202c9486f5caa Author: Kendell Clement Date: Tue Oct 23 12:01:41 2018 -0400 Update README.md commit 5cb8fccf048a49b3798f6932c0b0db90569697fa Author: Kendell Clement Date: Mon Oct 22 17:42:41 2018 -0400 Update README.md commit 7b60f691450c932b5d32ff48218f187328d0726f Author: kclem Date: Tue Oct 16 15:27:34 2018 -0400 2.0.16b - batch mode report commit 6037c2945efdea6fecf67b037a70c30d4bc6696b Author: kclem Date: Fri Oct 12 18:14:27 2018 -0400 2.0.15 updates to pooled, adjust merging commit 326e1c9f370e1d6956ad565605309f37e214e927 Author: kclem Date: Thu Oct 4 10:28:07 2018 -0400 2.0.14b - adjusted flash overlap params, cannot take mult aln gap penlty commit 26ec80198dc06ca09eb525deaf387c330f701cac Author: kclem Date: Fri Sep 28 15:13:00 2018 -0400 default val for n_processes commit bec0d312629d006169495b241fb9ea0018380e62 Author: kclem Date: Tue Sep 25 10:55:21 2018 -0400 2.0.13b commit 51d1386856849af7f04be0ce587e97349c7149b8 Author: kclem Date: Wed Aug 22 18:18:23 2018 -0400 Produce report commit 017c409c19a6af99c09c97faff67c26ac902e157 Author: kclem Date: Mon May 14 16:44:32 2018 -0400 Initial Commit of files commit 4324c954cc4efa10fc01fc6d69f88253ef5a7483 Author: kclem Date: Mon May 14 16:31:29 2018 -0400 first commit commit 2eff6a160159a5e64deb12f33110ee62b245f59e Author: Cole Lyman Date: Mon Mar 25 11:02:27 2024 -0600 Update reports to align with master (#20) * Update from recent changes with the CLI Most have to do with failed runs reporting * Update README to not automatically track master on new Reports branch * Remove extraneous `crispresso_data_path` commit b8dae4c8ed854f70d3dde7fdd0de396e842f8789 Author: Kendell Clement Date: Fri Mar 22 14:16:12 2024 -0600 Clean reports (#19) * clean reports * ignore README line endings * fix line endings * Remove commented out code for resizing Plotly plots when going fullscreen This would make Uncle Bob happy... * Update cup URL --------- Co-authored-by: kclem Co-authored-by: Cole Lyman commit 76b3601f6a0144f100266153f1c999e0c5de65de Author: Samuel Nichols Date: Fri Jan 12 09:56:19 2024 -0700 Squashed commit of the following: commit 603f2eff9d1aa21ae95f3e134da303b8018d3a33 Author: Samuel Nichols Date: Fri Jan 12 09:48:20 2024 -0700 fix guardrials partial commit 22fc03183a8070c30dfb74d5c23575ac19019855 Author: Samuel Nichols Date: Fri Jan 12 08:54:01 2024 -0700 Add guardrail partial commit e55f6b21972b578261bc5a864ce1d653d98f9e34 Author: Samuel Nichols Date: Mon Jan 8 07:50:59 2024 -0700 Functional guardrails, needs reports update commit 6e968e9699ed59a47d88191d03768e042d8b60a4 Merge: 32b49685 e948ce10 Author: Samuel Nichols Date: Mon Dec 18 13:34:36 2023 -0700 Merge branch 'guardrails-clean-history' of https://github.com/edilytics/CRISPResso2 into guardrails-clean-history commit 32b49685da320501dad2b0ebbb57887b66220ba8 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit 4e309cf6f732565d635de3d4c5d074ada3027e2d Author: Cole Lyman Date: Mon Dec 18 10:51:55 2023 -0700 Refactor to use CRISPRessoReports module commit e648dc087c0055bc5d2fca13c64071a371dea941 Author: Cole Lyman Date: Mon Dec 18 10:51:11 2023 -0700 Add CRISPRessoReports subtree commit e948ce107ebb0d1d99010ed12e937f34b5e607d4 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05b bb13e007 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e8 d0b41483 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f909713 b913fcb4 Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f6788 5f909713 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2d e5afa478 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f0 51a943c3 Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f0 5a10d638 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd468 442a48c7 Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. Co-authored-by: Kendell Clement commit a7ffd46822ce195d51ff4d3dba0f02fe9bc73c1e Author: Kendell Clement Date: Tue Jan 11 16:29:37 2022 -0500 Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9990790a0081b765c1f54f4a9b18db522ab4693 Author: Kendell Clement Date: Wed Jan 5 16:48:17 2022 -0500 Allow for mixed-case prime editing input, fix #187 commit 4acdcddbef3e812655b7af9def772f0bb0dc30b5 Author: Kendell Clement Date: Wed Jan 5 16:37:22 2022 -0500 Update insertion annotation for fastq_output and bam_output Fix index issue for fastq_output, add reporting of inserted bases to bam_output. Index for fastq_output is increased because the insertion happens immediately after the insertion coordinate. commit 2f84dd02787abffa6d39efbc50c82c92d1c87528 Author: kclem Date: Fri Dec 10 16:39:41 2021 -0500 Fastq_output report inserted bases when the `--fastq_output` parameter is provided, the inserted bases will be written to the output fastq file. Previously, a string like "DEL= INS=78(1) SUB= " would indicate a 1bp insertion at site 78. This update outputs strings like "DEL= INS=78(1+G) SUB= " with a plus character followed by the inserted bases. commit ba742bbfa533a672321b27b5122d7ea6658c9014 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Implement algorithmic improvements for find_indels_substitutions" This reverts commit 13414833ef6cd5b232097f6ff0325a2b8e28b214. commit 5c8e1c5de6e800c820d9ec5ffe4809737f1211de Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Add test cases for find_indels_substitutions" This reverts commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808. commit d9a072368ca991ffde52aa1ee29e17f2758ed279 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Try some additional cython optimizations" This reverts commit 38b101d57384b9f7d664ac8719c1585f5f7142a5. commit 38b101d57384b9f7d664ac8719c1585f5f7142a5 Author: Cole Lyman Date: Fri Oct 8 14:42:49 2021 -0600 Try some additional cython optimizations commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808 Author: Cole Lyman Date: Fri Oct 8 14:15:11 2021 -0600 Add test cases for find_indels_substitutions commit 13414833ef6cd5b232097f6ff0325a2b8e28b214 Author: Cole Lyman Date: Fri Oct 8 11:50:25 2021 -0600 Implement algorithmic improvements for find_indels_substitutions These changes remove the need for iterating over the alignment multiple times. commit f4b6cfc03951215c3b9019dc47beb2913a5448ab Author: Kendell Clement Date: Fri Nov 19 00:10:05 2021 -0500 Fix CRISPRessoPooled calls to deprecated pands ix commit 751fbdb4ce432f11f3fb66c5a34f7db3d1d75dc1 Author: swrosati <40470095+swrosati@users.noreply.github.com> Date: Fri Nov 12 12:33:04 2021 -0600 Update ylabel_values -> y_label_values Typo causing error in certain modes. commit 76c80713b7b8209c9399d67f718d7ade20f20d91 Author: kclem Date: Wed Nov 10 11:37:16 2021 -0500 Update dockerfile for pyparsing commit ef15caee29380f58aaae392c897fabe47587486e Author: Kendell Clement Date: Wed Nov 10 00:45:51 2021 -0500 Fix int bug for n_reads in CRISPRessoPooled commit fc1ae890712ab2dc36d5ff36c81f64a10a2c2337 Author: Kendell Clement Date: Wed Nov 10 00:34:34 2021 -0500 Spelling fix and version bump to 2.2.7 commit 08369e294d6756f727167af50f14ff75cb4864b5 Author: Kendell Clement Date: Wed Nov 10 00:30:03 2021 -0500 Add bam input and selected demuxing for CRISPRessoPooled Adds features for providing aligned bams as input to CRISPRessoPooled and for a faster demultiplexing when amplicons and genome are provided. The parameters are: --aligned_pooled_bam: Path to aligned input for CRISPRessoPooled processing. If this parameter is specified, the alignments in the given bam will be used to demultiplex reads. If this parameter is not set (default), input reads provided by --fastq_r1 (and optionally --fastq_r2) will be aligned to the reference genome using bowtie2. If the input bam is given, the corresponding reference fasta must also be given to extract reference genomic sequences via the parameter --bowtie2_index. Note that the aligned reads are paired-end seqenced, they should already be merged into 1 read (e.g. via Flash) before alignment. --demultiplex_only_at_amplicons: If set, and an amplicon file (--amplicons_file) and reference sequence (--bowtie2_index) are provided, reads overlapping alignment positions of amplicons will be demultiplexed and assigned to that amplicon. If this flag is not set, the entire genome will be demultiplexed and reads with the same start and stop coordinates as an amplicon will be assigned to that amplicon. commit 0cdcfd7c4af834f3270e61b4db4b5f6d3da10d7c Author: Kendell Clement Date: Wed Nov 10 00:24:15 2021 -0500 Remove unused var for bam_index commit 3647ace73c95a2f44e45b6b42b4f1748d3a47f4c Author: kclem Date: Wed Nov 3 09:43:28 2021 -0400 Add --min-overlap param for flash in CRISPRessoPooled commit eea442a763e0c6c41da16a15d6e11ddf6d222dc8 Author: kclem Date: Mon Nov 1 11:25:55 2021 -0400 Remove deprecated pandas .ix from WGS commit 3e6c281c931756acd26acca96249b2fa1ad1db31 Author: Cole Lyman Date: Thu Oct 21 11:10:19 2021 -0600 Add unit tests These unit tests were in the other repo, but weren't moved over for some reason. commit 50e7cb21570ddb757904728800e31c2bcbd0d060 Author: Cole Lyman Date: Thu Oct 21 11:09:38 2021 -0600 Add Makefile to run tests This makes it easy to test the integrations cases because it will automatically install the package when needed. commit de91d51bcd9b725533a45c86ae72bc27dd5d3150 Author: Cole Lyman Date: Thu Oct 21 11:28:02 2021 -0600 Replace `np.int` with `int` Apparently `np.int` has been deprecated, so in places where precision isn't important `np.int` has been replaced with `int` according to the instructions from the warning. commit cabebbefb2646967dbeee80af08ac14156b1b53c Author: kclem Date: Wed Oct 20 12:33:49 2021 -0400 Convert cols to numeric for PE commit c2bdd96651eef5af38fb7bbc11d257a827ac080d Author: kclem Date: Wed Oct 20 12:00:43 2021 -0400 Make loggers module-specific Matplolib sometimes logs verbosely to info, so this stops using the root logger commit 8196b6a81f477ddcb0e34d61dfb54085de20c1a0 Author: Kendell Clement Date: Tue Oct 19 22:00:23 2021 -0400 Fix unicode error for bam read/write commit a923a7c2ef182238bd6b8aa77289bac487b7679b Author: Kendell Clement Date: Tue Oct 19 21:59:47 2021 -0400 All sub-crispresso processes are run with 1 process commit 75205b0d41423e4f62796e9674b0edbee68a11b9 Author: Kendell Clement Date: Mon Oct 18 23:03:59 2021 -0400 Fix #161 add params for prime editing guide analysis commit ecf23ef23e5701b232bba547a6d7d4b96f085f26 Author: kclem Date: Mon Oct 18 15:02:29 2021 -0400 Add param for plotting custom plots centered at point Add parameter to plotCustomAllelePlot script: --plot_center which if set, will produce plots centered at this point instead of sgRNAs. commit 71bf32ca789dde1d44cf41b9d3b702f12336e010 Author: Kendell Clement Date: Wed Oct 13 00:48:14 2021 -0400 Update README.md commit 53197e62706e37db54f7ed50c94f38a938955e59 Author: Kendell Clement Date: Tue Oct 12 15:43:08 2021 -0400 Fix #160 plotting for cut sites in plot 5 commit 90b43eaa03c0ea0fdee62a7b244204cad50056cc Author: Kendell Clement Date: Mon Oct 4 15:45:20 2021 -0400 Remove version checks for old seaborn and numpy, fix #155 commit 5e9dedf6bd7c7b3ef998bed811760a41b5313c6e Author: Kendell Clement Date: Tue Sep 28 21:16:54 2021 -0400 Optimize get_command_output, fix gzip binary bug commit 46953c39b769a4fa43b2ef0822dd92f07c4f1b9b Author: Kendell Clement Date: Wed Sep 22 01:58:36 2021 -0400 Update expected tests for batch commit 856354aadebc8410956e724b555224a55941a618 Author: Kendell Clement Date: Wed Sep 22 01:57:59 2021 -0400 Update CRISPRessoBatchCORE.py Move parsing of batch params to after assignment of n_processes so this value is applied to sub-processes commit f7f81747207cb279fd2c0d91986c7d4e7137fde4 Author: Kendell Clement Date: Wed Sep 22 00:23:04 2021 -0400 Fix #149 bug when read is much longer than the given reference commit 798eb4322899f70e2e2a3df0fdfad9b5598d3a41 Author: Kendell Clement Date: Tue Sep 21 17:43:38 2021 -0400 Fix #150 Open fastq.gz in 'rt' mode commit fa168843e6036c6d4ec4a5d7acb097c6a1532a98 Author: Kendell Clement Date: Fri Sep 17 12:33:12 2021 -0400 Remove requirement for python 2.7 commit 80fe12efbb0ee5c321262822b237fc8220a3f2c6 Author: Kendell Clement Date: Thu Sep 9 17:41:27 2021 -0400 Print batch summaries for amplicons with only one sample Previously, batch summaries were only printed for amplicons with more than one sample to save bits. This change produces batch summaries for amplicons with only one sample, and we shift responsibility to the user for purchasing carbon offsets to compensate for the generation and storage of these redundant bits. commit 43065310bf28e6a08780222250abd0d39ff6d0c5 Author: Kendell Clement Date: Thu Sep 9 17:38:58 2021 -0400 Add parameter 'assign_ambiguous_alignements_to_first_allele' For ambiguous alignments, setting this flag will force them to be assigned to the first (as provided by the references -a first and then -e second) amplicon. Thus, no reads will be discarded as 'ambiguous' and all reads will be counted once in the analysis. Version bump to 2.2.4 commit 11eaacd3bac9ebeabad69c1d5d92f1fd1a783a17 Author: Kendell Clement Date: Fri Sep 3 22:34:59 2021 -0400 push down crispresso2_info items commit 20272e1e95305888ca744ac56af2c08554eb9f1b Author: Kendell Clement Date: Fri Sep 3 22:32:24 2021 -0400 remove mention of pickle commit 3517285473d423dc32c6e3fdbac69eecf0caa7fc Author: Kendell Clement Date: Fri Sep 3 22:30:36 2021 -0400 minor pushdown of report name in crispresso2_info commit 8129494607488bf270567d40d43e01e043699f06 Author: Cole Lyman Date: Fri Sep 3 17:31:54 2021 -0400 fixup! Move region related objects to results in crispresso2_info commit 88a82d27e840c29b95b1602ae1594bf161108979 Author: Cole Lyman Date: Fri Sep 3 16:40:40 2021 -0400 Move some objects in CRISPRessoPooled crispresso2_info objects Move the objects: - 'final_data' -> 'results'/'final_data' - 'running_mode' -> 'running_info'/'running_mode' commit b702ab3e53e88170c7517591ce7a7050ccf1c2ba Author: Cole Lyman Date: Fri Sep 3 16:36:20 2021 -0400 Move some objects in CRISPRessoBatch crispresso2_info Move the following objects: - 'completed_batch_arr' -> 'results'/'completed_batch_arr' - 'batch_names_arr' -> 'results'/'batch_names_arr' - 'batch_input_names' -> 'results'/'batch_input_names' - 'nucleotide_frequency_summary_filename' -> 'results'/'nucleotide_frequency_filename' - 'nucleotide_percentage_summary_filename' -> 'results'/'nucleotide_percentage_filename' - 'modification_frequency_summary_filename' -> 'results'/'modification_frequency_summary_filename' - 'modification_percentage_summary_filename' -> 'results'/'modification_percentage_summary_filename' commit 2416c80e952cb58fa082ead5ef719ed7eab3eeb5 Author: Cole Lyman Date: Fri Sep 3 15:39:18 2021 -0400 Move nuc_quilt related objects to 'results'/'general_plots' to crispresso2_info The following objects have been moved: - 'window_nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'window_nuc_pct_quilt_plot_names' - 'nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'nuc_pct_quilt_plot_names' - 'window_nuc_conv_plot_names' -> 'results'/'general_plots'/'window_nuc_conv_plot_names' - 'nuc_conv_plot_names' -> 'results'/'general_plots'/'nuc_conv_plot_names' commit 37e354f94980d304e1cecf11fee95e61988dd3e3 Author: Cole Lyman Date: Fri Sep 3 14:58:36 2021 -0400 Move summary objects to 'results'/'general_plots' in crispresso2_info The following objects have been moved: - 'summary_plot_names' -> 'results'/'general_plots'/'summary_plot_names' - 'summary_plot_titles' -> 'results'/'general_plots'/'summary_plot_titles' - 'summary_plot_labels' -> 'results'/'general_plots'/'summary_plot_labels' - 'summary_plot_datas' -> 'results'/'general_plots'/'summary_plot_datas' - 'summary_plot_root' -> 'results'/'general_plots'/'summary_plot_root' - 'reads_summary_plot' -> 'results'/'general_plots'/'reads_summary_plot' - 'modification_summary_plot' -> 'results'/'general_plots'/'modification_summary_plot' commit 6df988151c602602470c944ccf561cd28f2dc30c Author: Cole Lyman Date: Fri Sep 3 14:04:12 2021 -0400 Move region related objects to results in crispresso2_info This includes: - 'regions' -> 'results'/'regions' - 'all_region_names' -> 'results'/'all_region_names' - 'good_region_names' -> 'results'/'good_region_names' - 'good_region_folders' -> 'results'/'good_region_folders' commit 053691311b158a2d3620670d20983d546a9c2c7b Author: Cole Lyman Date: Fri Sep 3 13:44:46 2021 -0400 Move 'samples_quantification_summary_filename' to 'results'/'alignment_stats'/'sample_quantification_summary_filename' in crispresso2_info commit eda3d50b6fafde4ea3145980cb2010417b799d88 Author: Cole Lyman Date: Fri Sep 3 13:27:42 2021 -0400 Move 'finished_steps' to 'running_info'/'finished_steps' in crispresso2_info commit 1da829c1c3cfd2bda9aaccca0aaa28c78a8f7271 Author: blasvicco@gmail.com Date: Mon Aug 30 17:48:02 2021 +0200 BugFix: database_id referenced before declared. commit c9321cc2cfcac34e4b6d8e1aa9fde9f25382e2de Author: Kendell Clement Date: Sat Aug 21 00:15:52 2021 -0400 Version bump to 2.2.2 for some reason the last release didn't pick up the last commits. commit 3aadab69ef1e3a44f12ee277a7767648e943a787 Author: Cole Lyman Date: Fri Aug 20 12:09:49 2021 -0400 Update filterFastqs so that the numpy arrays are writable commit 7ca129d2471aeebef2ba6d2dadf36265810f1a5c Author: Kendell Clement Date: Fri Aug 20 02:00:42 2021 -0400 Update CRISPRessoPlot.py Undo attempted matplotlib deprecation warning fix commit 8e8a65c0f29b6f99662ac1d272aa7199871f5cf5 Author: Kendell Clement Date: Fri Aug 20 01:26:50 2021 -0400 Plotting bug fixes Display and plotting fixes for batch report labels, and general plots, as well as a matplotlib deprecation fix commit 3f114752c5eca1554fcebf044b604b60a3eebeae Author: Kendell Clement Date: Fri Aug 20 00:06:38 2021 -0400 add batch summary of frameshift/splicing Closes #116 by adding frameshift/splice summary to batch Version bump to 2.2.1 Fix unicode error in slugify commit 5ad9fbc6645475885fc0f35bc26afd353a2b3d5f Author: Cole Lyman Date: Mon Aug 16 14:16:13 2021 -0400 Read plain text fastq as bytes in filterFastqs commit 6c20f28678469875392e3fc8946018b53a00256b Author: Kendell Clement Date: Mon Aug 16 13:35:12 2021 -0400 Remove test for seaborn version commit cec965feff65b4228d42784e498398b73b8caa49 Author: Cole Lyman Date: Mon Aug 16 12:16:04 2021 -0400 Fix filterFastqs This commit fixes the filterFastqs script by properly casting to strings (from bytes) in the appropriate places. It also replaces the deprecated `np.fromstring` function with `np.frombuffer`. commit 3c30e56a15fa15a3537c3d51e429c1ff8738d6fe Author: Cole Lyman Date: Thu Aug 12 09:57:50 2021 -0400 Add .gitignore file commit 2461e30770d5ae8a6686530981a4bf5c913e2f68 Author: Cole Lyman Date: Thu Aug 12 09:50:01 2021 -0400 Decode result from Popen from bytes to string This is done only where a string is necessary, the instances where this is not done are where the result is cast to a float or int (because they can accept bytes as input directly). commit 64ec8bacf9ae8cb0d3a9093ad12a916983f32207 Author: Cole Lyman Date: Sat Aug 7 13:49:05 2021 -0400 Refactor `results/refs` and `results/ref_names` crispresso2_info fields commit ebb33a7d691b524ed7ab92b5a870da09df045e00 Author: Cole Lyman Date: Fri Aug 6 20:32:03 2021 -0400 Refactor `results/general_plots` crispresso2_info fields commit 94768e209e2424394efb4d902aac4f0e4268aad3 Author: Cole Lyman Date: Fri Aug 6 14:58:25 2021 -0400 Refactor `results/alignment_stats` crispresso2_info fields commit a58b7a2b40bc99346a9ea8f6240034963b3d6b31 Author: Cole Lyman Date: Fri Aug 6 14:10:33 2021 -0400 Refactor `running_info` crispresso2_info fields commit a92ea4687e0cc69844c5d4a6250757540076ed59 Author: Kendell Clement Date: Thu Aug 5 14:28:29 2021 -0400 Version bump to 2.2.0 commit 20e9b6f228949886ebe592d4542aaddbb9fb123a Author: Cole Lyman Date: Thu Aug 5 11:52:22 2021 -0400 Remove argparse dependency Remove the argparse dependency from setup.py because argparse is a standard module in Python 3. commit ecf70ea4379e079e7669fc5ed8baabdcd11a8c61 Author: Kendell Clement Date: Fri Jul 30 01:23:38 2021 -0400 Update Dockerfile bowtie throws an error 'Can't locate Sys/Hostname.pm in @INC' This fixes it. commit 30fb34e17787858d4218eb0b0fcc1baf6259ee23 Author: Kendell Clement Date: Fri Jul 30 00:39:48 2021 -0400 Ignore python egg for copying, pin tbb for bowtie error https://github.com/BenLangmead/bowtie2/issues/336 commit c850abb672019a3684a544fccde9550f7eeb86f5 Author: Kendell Clement Date: Thu Jul 29 20:13:32 2021 -0400 Update config.yml commit eb70cb18d6910f418bb8052f45b58c05c397af43 Author: Kendell Clement Date: Thu Jul 29 17:47:06 2021 -0400 Update config.yml commit 01f079fd663027574115a0af974564d5b5efbe97 Author: Kendell Clement Date: Thu Jul 29 17:22:48 2021 -0400 Update CRISPResso2_router.py commit 2e3b5d42b8ea6705dbd2c2f59312b93390083da3 Author: Kendell Clement Date: Thu Jul 29 17:20:16 2021 -0400 Update Dockerfile commit 46d8c44f2dbe7a67531d3ccae6b0a3c51b12b9ca Author: Kendell Clement Date: Thu Jul 29 17:07:02 2021 -0400 Update Dockerfile commit 0999e82dd8e184a6b0aefa7a35fc9decaa1fc20d Author: Kendell Clement Date: Thu Jul 29 16:59:47 2021 -0400 Update Dockerfile commit 34b93cfeeb95d0a1375378996e3ca29e79fe5311 Author: Kendell Clement Date: Thu Jul 29 16:50:20 2021 -0400 Python 3 commit e040ac488b050d6f316d21196de002ef09383915 Author: Kendell Clement Date: Tue Jul 27 10:04:43 2021 -0400 Add testing file for batch, remove debug print lines commit aa7b9998c4660438f4303a57386f01617ddec1bb Author: Kendell Clement Date: Thu Jul 22 10:43:52 2021 -0400 Update Batch to allow for max processes usage commit 1566a1110215e32f338497040f1279c3581b6dcd Author: Kendell Clement Date: Wed Jul 21 01:02:24 2021 -0400 Fix reference to BadParameterException commit fea5017109ab56c955b8053eebad5f6f4218bc65 Author: Kendell Clement Date: Mon Jul 12 16:11:05 2021 -0400 Allow spaces in run names, print run name to report commit b8594354c96131ba33c9277fb719c95b1cf72f3d Author: Kendell Clement Date: Tue Jun 29 14:12:03 2021 -0400 Update CRISPRessoPooledCORE.py Fix debug statement commit 9bc9b9959cbc8faf9dc462669e333298a80a71d1 Author: Kendell Clement Date: Tue Jun 29 14:09:13 2021 -0400 Update CRISPRessoPooled for samtools sort status updates Redirect samtools sort status updates to log file. Version bump. Previously this would cause an error: `ERROR: list index out of range samtools view: writing to standard output failed: Broken pipe samtools view: error closing standard output: -1`. commit ad5b9d853ac3559e58a5ad569e7d3ac9c3d8a719 Author: Kendell Clement Date: Thu Jun 24 12:10:31 2021 -0400 Allow max processes for CRISPRessoPooledWGSCompare Users can provide -p max to use all processes available for comparisons commit 48d6c8724f541b285e5d6889723cc37fc99e5cfa Author: Kendell Clement Date: Wed Jun 23 20:53:51 2021 -0400 Create html report for CRISPRessoComparePooledWGS commit abe3054dd9b95f51cbf34e3c37e088f6beb8287c Author: Kendell Clement Date: Wed Jun 23 20:53:22 2021 -0400 Fix allele plot bug #103 If no regions are returned, max of a pandas dataframe returns an error because the df is empty commit 1ccb507858d92516e28286358d3aae9cce2cf7ea Author: Kendell Clement Date: Wed Jun 23 20:51:16 2021 -0400 Fix command prompt logo lines commit 293c249c0f380576121a123dec982e40409c977e Author: Kendell Clement Date: Sun Jun 20 00:26:34 2021 -0400 Update plotCustomAllelePlot.py Get rid of debug statements commit 947fbab70e0f4fa78cc43273b4fe2be5225043cc Author: Kendell Clement Date: Sun Jun 20 00:22:56 2021 -0400 Add plotCustomAllelePlot script for replotting allele plots commit 167a48659684ce1669da915ddb6995935c6a1aa6 Author: Kendell Clement Date: Wed May 26 11:16:51 2021 -0400 Update README with explicit instructions to activate conda environment commit af0b7e441d2a4f1e035fabfd353c58784f688371 Author: Kendell Clement Date: Sun May 23 00:22:00 2021 -0400 Update test results for more flexible pooled alignment The relaxing of bowtie parameters allowed more reads to align, which changed the expected test results for CRISPRessoPooled commit 0e08cd05c2f3279ac95a068be76f1d36a4b0224d Author: Kendell Clement Date: Sun May 23 00:06:21 2021 -0400 Fix double rows in Alleles_frequency_table due to read directionality Previous to this fix, forward and reverse reads having the same sequence would appear in different rows in the Alleles_frequency_table. This fix doesn't change counts or results, just combines the two rows in the Alleles_frequency_table. commit 7d2d761a3d4c25915be0d27b36f3b4a87068587d Author: Kendell Clement Date: Thu May 20 16:05:27 2021 -0400 CRISPRessoPooled - get rid of -k bowtie2 parameter The -k 1 parameter may not yield the best alignment. This commit gets rid of that parameter. commit 44dc9e75c660f4b3b683f1c80f5a964aa55e75bd Author: Kendell Clement Date: Wed May 19 22:52:46 2021 -0400 CRISPRessoPooled alignment more tolerant When given a genome file, CRISPRessoPooled aligns reads to the genome using the Bowtie2 aligner. The legacy parameters were somewhat strict. The new parameters reflect the 'default_min_aln_score' parameter in allowing for substantially more indels and mismatches than previous. The parameter `--use_legacy_bowtie2_options_string` has been added to use the legacy settings. Otherwise, the bowtie2 alignment settings will be calculated as follows: commit 84b870ce0d489295501aa57711edd6b18c011b92 Author: Kendell Clement Date: Tue May 4 10:22:22 2021 -0400 Raise exception if quantification_window_coordinates looks like an int Previously, if quantification_window_coordinates looks like an int when pandas parsed a batch file, it would throw an error trying to split the int. Now an exception will be raised. commit e25a6dbb13fcd0565f4c81e3ea42cfd115aa1bc0 Author: Kendell Clement Date: Mon Apr 26 16:19:00 2021 -0400 Fix bug if all reads are discarded If all reads are discarded, CRISPResso would fail because the df_alleles is empty. This adds discarded alleles to the df_alleles. commit 156d7d640355ad4755c6ce4cbab1b75bf31677c9 Author: Kendell Clement Date: Fri Apr 9 13:24:51 2021 -0400 CRISPRessoPooled - close active file in demux commit c5d9fcbbf5bd9b307c9050498d08e72dd98d42aa Author: Kendell Clement Date: Fri Apr 9 12:29:02 2021 -0400 Keep old awk command for speed for samples with <50 amplicons commit c7539661fc5bd29f4f6ee1e9c7795be932b53a8e Author: Cole Lyman Date: Fri Apr 9 12:18:45 2021 -0400 Alt pooled processing implementation (#87) These changes implement separating reads to their corresponding amplicons via Python instead of through awk. This is to get around the maximum number of open files that is limited on many operating systems. Co-authored-by: Kendell Clement commit e57d98738fa832766c78dc7eecfdb0588d92634d Author: Kendell Clement Date: Thu Apr 8 12:36:43 2021 -0400 Alt pooled processing implementation commit 4598226837cd4b726ae38f50958f977bde4794ca Author: Kendell Clement Date: Sun Mar 21 00:16:19 2021 -0400 HDR Updates - yw #82 Multiple amplicon names are resolved before adding the HDR amplicon -- unnamed amplicons are named Amplicon{i} for each amplicon. Plot 4g data (nuc pct table, mod pct table for all reads aligned to the first reference) is output and linked to from the plot display Ambiguous reads don't contribute to plot 4g data (which would otherwise lead to double counting and pct values > 1) commit d7915b2c561173238c6b0e82863f76a783db7c4d Author: Kendell Clement Date: Sat Mar 20 01:12:55 2021 -0400 Fix #72 bam_input error commit 32a9e98ffc6ceb1dd56e5e29c369176ed788c985 Author: Kendell Clement Date: Sat Mar 20 01:01:17 2021 -0400 Prime editing, fastq_out updates Prime editing input parameters are forced to be in the RNA 3'->5' direction. This makes sure that the scaffold incorporation happens on the correct side of the extension sequence. Errors are thrown if improper directionality is detected. Fastq_out now includes alignment scores and details for every run (it may be time to upgrade that SSD to hold these new fastq output files, but it makes debugging particular reads a lot easier!) Update to linked data for plot 3b in report commit c1b6ea2ecebd920ea12ab91f2d50e35c274f94fe Author: Kyle McChesney Date: Wed Mar 10 13:40:44 2021 -0700 fix missing import path on NaN commit 1b24ca351f4337e0a7bece7ef7addb4558f950d8 Author: Kendell Clement Date: Sat Feb 20 22:57:07 2021 -0500 Update links in readme to https - fix #79 commit d550fd1db8ce0e1d11ed77fd082c53a3d887bbc2 Author: Kendell Clement Date: Fri Feb 12 16:57:37 2021 -0500 Insertion quantification change + fix #76 Starting in version 2.1.0, insertion quantification has been changed to only include insertions completely contained by the quantification window. To use the legacy quantification method (i.e. include insertions directly adjacent to the quantification window) please use the parameter --use_legacy_insertion_quantification commit 798f661031236ee1aa5611f491ca135ec0432dc9 Author: Kendell Clement Date: Thu Jan 14 23:51:29 2021 -0500 Allow for int-looking chromosome names in WGS input file In CRISPRessoWGS, the region file contains a 'chr_id' column which is sometimes mis-recognized as ints when read by pandas if using the chromosome notation without 'chr' (e.g. 1,2,3 in stead of chr1,chr2,chr3). This bug fix forces chr_ids to be read as strs. commit 92c008673690a3cd32e33fe8cfcf07060cf6cb0f Author: Kendell Clement Date: Wed Dec 30 23:48:32 2020 -0500 Update README.md commit 8d590c5588d93ef0129512a5156fe3b45e2d3c4b Author: Kendell Clement Date: Wed Dec 30 23:46:36 2020 -0500 Introduction of CRISPRessoAggregate to aggregate stats across runs Adds CRISPRessoAggregate Adds start/end time to CRISPRessoBatch info Started removing pickle dependencies from Pooled and Report commit c8447246ada2758831a870f30ed99c496c18feb1 Author: Kendell Clement Date: Sat Dec 26 10:52:12 2020 -0500 Output alignment details for unaligned reads in fastq_out or bam_out commit dedc36cadc64e9302d88c3472652d2a4a2385d25 Author: Kendell Clement Date: Tue Nov 24 13:45:34 2020 -0500 Add scripts folder for one-off analyses commit 88b6e9c50c6d97da357534c67aaeba64c00ddc23 Author: Kendell Clement Date: Sat Nov 14 02:46:36 2020 -0500 Fix plot window cloning from Ref1 to HDR Plot window for sgRNA will be the same length after cloning even if the window is shorter or longer after comparing between ref1 and HDR. commit 7403a46e186c4b70ad606dbb0eebbbde684fac61 Author: Kendell Clement Date: Sat Nov 14 01:52:55 2020 -0500 Frameshift plot and HDR quant window updates Frameshift plots don't show 0-bp changes (these dwarf all other changes). The number of reads not shown are added to the legend. Addressed cloning quantification windows when bases are inserted in the clone-ee (previously these cloned bases would be ignored. Force HDR to clone all quantification windows from Ref 1 Fix #60 and #59 commit f3f9c122cc25ab62bdd7f30fb9d6ee4c9ab820a8 Author: Kendell Clement Date: Sat Nov 7 23:55:55 2020 -0500 Update histogram x-limits, caption, and data commit e9332f7eabed32e65430b44677c841dd0150ac5a Author: Kendell Clement Date: Sat Nov 7 02:26:59 2020 -0500 Fixes for when no reads align #63 commit 9fae60a57d99238e4f7a9ad2e992ae71cca49c60 Author: Kendell Clement Date: Sat Nov 7 02:08:59 2020 -0500 Standardize pie plot appearances commit f1738bd17b496a31d6f68a91ed7ae67a36f8f178 Author: Kendell Clement Date: Sat Nov 7 00:36:40 2020 -0500 plotting and pe fixes - Special bonus for y'all to keep you company during covid - axis ticks on most plots! - added parameter --plot_histogram_outliers to plot all insertion sizes in histogram - all insertion sizes are reported in .hist output files #64 - add HDR reference plot (may change this later to set ref1 to the longer reference of WT/HDR but for now it is always WT..) Allow reverse complement of extension seq if PE sequence is specified. commit 5a2d9271b3ea99342f22e59259149d30dc193e47 Merge: bd03668e 9812651c Author: Kendell Clement Date: Wed Oct 21 16:52:12 2020 -0400 Merge pull request #62 from matandro/patch-1 Fix a bug when generating compare plot commit 9812651c2aae1218393e8ce0d734812247cd59ab Author: Matan Drory Retwitzer Date: Wed Oct 21 10:45:01 2020 -0700 Fix a bug when generating compare plot Related to issue #61 This happens when N_ROWS < 1 which I assume has something to do with no results -> negative control commit bd03668ef1626a54e0b5bfbc010afacee60f45e3 Author: kclem Date: Fri Oct 2 17:39:52 2020 -0400 delete merging intermediate files commit 3c9b1344e19dee04b169c0860bc11304d6a594b5 Author: Kendell Clement Date: Wed Sep 30 23:51:08 2020 -0400 Version bump to v2.0.42 commit 2f8c6f9c7d31b276ff18000d579ef722f693c433 Author: Kendell Clement Date: Wed Sep 30 23:44:02 2020 -0400 Update new parameters, fix docker biuld problem commit f130270135b84e0904e0003858c263285834b6dd Author: Kendell Clement Date: Wed Sep 30 14:18:58 2020 -0400 Fix bug in read counting for interleaved fastqs commit faac4e1045e5b617b3743e328c0203ced27e3f31 Author: Kendell Clement Date: Thu Sep 24 22:37:50 2020 -0400 Fix bug in mode to write fastq out commit 388e8a97fc18648dd28e35063cb12680887395e2 Author: Kendell Clement Date: Wed Sep 23 23:49:44 2020 -0400 Update postrun reference output file Rename postrun references file to be more standardized with other output files. Output is now "CRISPResso2Pooled_postrun_references.txt" commit ee5be76e5e24e494abd93940622d51faa83a2371 Author: Kendell Clement Date: Wed Sep 23 23:32:34 2020 -0400 Multiple allele support for pooled mode Most common alleles for each pooled target are output if the flag '--compile_postrun_references' is provided. This writes alleles with frequncy defined by the parameter to --compile_postrun_reference_allele_cutoff This file can be manually edited to remove noisy alleles, and then used to run CRISPRessoPooled again but to provide alternate alleles to each CRISPResso run by using the parameter '--alternate_alleles'. This is particularly useful in cases where control experiments are available. The running pattern would be: 1) CRISPRessoPooled --compile_postrun_references {control} 2) CRISPRessoPooled --alternate_alleles {produced in step 1} {control} CRISPRessoPooled --alternate_alleles {produced in step 1} {experiment} commit fe5bee2ac7ca4a8dd165e2c7305a1c41a79b8d9d Author: Kendell Clement Date: Wed Sep 23 23:27:37 2020 -0400 Output + PE updates Add parameter '--fastq_output' to output a fastq file with annotations for each read. Also, the substitution frequency table is only produced in base editor mode -- in an effort to slim down CRISPResso2 output files. Also, especially for long Prime Editing insertions or deletions, the inference algorithm may incorrectly infer the prime-edited allele. I added the parameter '--prime_editing_override_prime_edited_ref_seq' which can be used to specify the prime-edited sequence manually. commit ca61c483a8a63fec2e85889f554648ea6f3a903c Author: Kendell Clement Date: Sat Sep 5 16:15:16 2020 -0400 update report caption for fig 10 commit 346c6f6e62097ea7690204cfe879ebb918fb244d Merge: e52cb734 44ccc5ec Author: kclem Date: Fri Sep 4 15:05:02 2020 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit e52cb73471f4538ecd98f943a38771f0d07a4476 Author: kclem Date: Fri Sep 4 15:04:48 2020 -0400 Update on help message for mark wildtype allele commit 44ccc5ec3ff6a57d265807b559010512f053d82a Author: Kendell Clement Date: Fri Sep 4 14:59:05 2020 -0400 Update to plotting quantification window plot vertically centered, pooled/wgs plots are not limited in height to allow analysis of large pools. commit ff675b12196593ceb8e8d8b58dfd6a8ca8865501 Author: Kendell Clement Date: Mon Jul 27 09:55:30 2020 -0400 Update parameters and description in README commit 25c6a1b45ef1e1c1243057b9b3ee06f638d55d26 Author: Kendell Clement Date: Sat Jul 25 00:29:14 2020 -0400 Update CRISPRessoBatchCORE.py If amplicon sequence is empty, auto mode is run for batches commit 2978a6ebc0cd977b36b4ea7e1965f5c43b46d325 Author: Kendell Clement Date: Thu Jul 23 08:46:01 2020 -0400 Removed WGS logging For some reason, piping output breaks multiprocessing blocking commit 8bc9214ba0a1b628b69a49a2cf306bf559e008b3 Author: Kendell Clement Date: Wed Jul 22 22:58:51 2020 -0400 Update CRISPRessoWGSCORE.py commit a9484fd481bfa8ad716c6326aba9c57f5271f885 Author: Kendell Clement Date: Wed Jul 22 14:24:38 2020 -0400 Add parameter discard_guide_positions_overhanging_amplicon_edge If run with param -- discard_guide_positions_overhanging_amplicon_edge, for guides that align to multiple positions, guide positions will be discarded if plotting around those regions would included bp that extend beyond the end of the amplicon. Normally this would cause CRISPResso to fail if plotting were requested beyond the end of an amplicon. commit 8d26edeed89e33451bd539c242f7ffaa560db3e6 Author: kclem Date: Wed Jul 22 13:58:10 2020 -0400 WGS doesn't print crispresso output to screen WGS printing error fixed commit 5ed03059d09aaf8a416c5fec553801eeca73355f Author: kclem Date: Tue Jul 21 00:00:40 2020 -0400 bam processing update of cached not-alignments commit cf593183d7b3d8200ec295ebcc653e518775046a Author: Kendell Clement Date: Thu Jul 16 21:49:21 2020 -0400 Fix naming of scaffold-incorporated reference links on plots commit 676ff623373b0cabf992017bd5eca255c4073d2a Author: Kendell Clement Date: Fri Jul 10 00:02:59 2020 -0400 Prime editing updates prime editing guides are shown as input on report output file names are produced without spaces commit 9de370148b2ada7210c10532247b3fa4b18a76de Author: Kendell Clement Date: Tue Jul 7 20:46:30 2020 -0400 Update batch for multiple quant window input commit 6ad1822f478d7f108b92f25378cc3ddf8dc3a2d3 Author: Kendell Clement Date: Wed Jul 1 16:26:38 2020 -0400 Bam processing + Prime Editing updates -Input can now be read from bam using the parameter `--bam_input` and (optionally) `--bam_chr_loc` to use the reads in the bam at this location as input. An output bam is produced with an additional soace-separated field prefixed by c2 (e.g. c2:Z:ALN=Inferred CLASS=Inferred_MODIFIED MODS=D47;I0;S0 DEL=56(47) INS= SUB= ALN_REF=TTGGCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGAAGTAGGGCCTTCGCGCACCTCATGGAATCCCTTCTGCAGCACCTGGATCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCT----------------------------------------------- ALN_SEQ=ACACCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGA-----------------------------------------------TCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCTGGAGCGGCGGCTGCACAACCAGTGGAGGCAAGAGGGCGGCTTTGGGC). Note that the alignment details (location, cigar string, etc) are not modified.. this may be done in the future). Bam file input cannot be trimmed or pre-processed with quality filtering. -Prime editing scaffold incorporation is now more accurate (looks for the scaffold sequence at the expected position directly after the extension sequence). A plot showing the number of bases matching the scaffold, as well as insertions after the extension sequence, and a data file with these numbers is produced. Added parameter `--prime_editing_pegRNA_scaffold_min_match_length` to define the minimum length required to classify a read as 'Scaffold-incorporated' -Renamed split_paired_end parameter to `--split_interleaved_input` for interleaved input -Auto mode now considers 5000 reads to detect amplicon sequences -Add new paramter `--annotate_wildtype_allele` to annotate wildtype alleles on the allele plots -Update output when reporting missing files -- only lists first 15 files in the current directory and directory of input parameter --reference https instead of http commit c0f2871befed86c4b314100584bf844f13d71d0e Author: Kendell Clement Date: Tue Jun 2 18:54:52 2020 -0400 Update CRISPRessoWGSCORE.py Remove debug print commit e5450b1cc6be518706969d27bda843cb7c16b082 Author: Kendell Clement Date: Tue Jun 2 18:53:20 2020 -0400 Updates for Pooled and WGS WGS gene annotations compatability fix and pooled gzip fix commit c2b286dbda3807752f34cdf6274e41d5640b408f Author: Kendell Clement Date: Tue May 12 00:33:36 2020 -0400 Fix docker bug, print version to Pooled + WGS commit 2a06cc18c3157e6c135dae7ce53adec94fe8a83f Author: kclem Date: Sun May 10 00:09:55 2020 -0400 Fix plotting bug if no sgRNAs given commit 1e3ca605e0c5ae65e8acc727667f952bc4c0d3ee Author: kclem Date: Sun May 10 00:00:32 2020 -0400 flexiguide fix commit 4e1d6b2b3424e725010e3e1a13522a7386228853 Author: Kendell Clement Date: Sat May 9 23:30:39 2020 -0400 Prime editing refinement and Pooled filesystem demand reduction - Prime editing parameters renamed, nicking guides match with flexibility - Prime editing extension seq is shown as a guide (with no cut site) - Prime editing summary plot included in report - Nucleotide plots are shaded when no changes from the reference sequence - sgRNA annotations are plotted on multiple lines if they overlap - N's don't count as substitutions - extended read analysis data available with --write_detailed_allele_table flag commit 8c584719b9771c01e53cfe409789b29a29fad665 Merge: f5069365 7624815e Author: Kendell Clement Date: Sat May 9 23:14:30 2020 -0400 Merge pull request #42 from ronaldhause/patch-1 ZipFile: set allowZip64=True to write larger allele frequency tables commit f5069365d37bd698ff3bf35e5c39a1b75e10dc1d Author: kclem Date: Sat May 9 12:04:10 2020 -0400 CRISPRessoPooled updates, fix #37 for too many files Demultiplexing in the case of amplicons + genome is parallelized to reduce sorting Only files with sufficient reads are demultiplexed and written Additional output file REPORT_READS_ALIGNED_TO_GENOME_ALL_DEPTHS.txt shows all alignment locations commit 7624815e3159d926d8a0710f674f44c616b68bd5 Author: Ron Hause Date: Sun May 3 22:50:07 2020 -0700 ZipFile: set allowZip64=True to write larger allele frequency tables Addresses terminating ERROR: Filesize would require ZIP64 extensions when trying to write compressed allele frequency tables > 2 GB commit 6af15a09033166b713df159a6ef850dde8867253 Author: kclem Date: Tue Apr 28 03:28:41 2020 -0400 int bug fixes commit 531753c0f5be89c255f65d876cba5e9bf00dd4a2 Author: Kendell Clement Date: Tue Apr 28 02:00:37 2020 -0400 Introduces support for prime editing, multiple window sizes and offsets, max processors commit 8e29c1e0966ebb2073d52a221ae77e56bb146431 Merge: adb0d8b7 039013ef Author: Kendell Clement Date: Fri Apr 24 13:06:46 2020 -0400 Merge pull request #41 from natecarlson/fix-name-error If the name column is called 'name', refer to it as 'name', not '#name'. commit 039013efe363603dfe89f10b857d58bb1ef8e8d9 Author: Nate Carlson Date: Fri Apr 24 09:46:21 2020 -0500 If the name column is called 'name', refer to it as 'name', not '#name'. commit adb0d8b791686846fa8522f46e064418cbfbdc1c Author: Kendell Clement Date: Mon Apr 20 16:26:32 2020 -0400 Update LICENSE.txt commit b514a68eea135e805333c67bf33bf0f1ea41a034 Author: Kendell Clement Date: Sun Apr 19 00:36:26 2020 -0400 Print CRISPResso command on multiprocessing fail commit 4bfadd08d30e1a8b926757c1af4a9c0c0dc0b484 Author: Kendell Clement Date: Sun Apr 19 00:03:58 2020 -0400 Index fix for crispresso multiprocessing Indexes were incremented for user enjoyment (1-based) but the more accurate approach is 0-based Also, no_rerun ignores changes to the flags 'debug' or 'n_processes' which shouldn't affect the outcome commit 867e692b326acd19ed5f291bd5699f5885b1d569 Author: kclem Date: Thu Apr 16 00:25:47 2020 -0400 Allele plot sgRNA labels stay on plot commit b0d89b4effc4242ec55ed4a3e20e8835a90e3588 Author: kclem Date: Wed Apr 15 23:58:46 2020 -0400 WGS fastq seqs are now uppercase, so guides match even in lowercase-masked genomes commit 8098f1f1dda6efdaf773b9f85622d79f32ac49c9 Author: kclem Date: Thu Apr 9 01:11:26 2020 -0400 Pooled multiprocessing updates commit 034ff2b5858dd29ab60017835695fad527a5213e Author: Kendell Clement Date: Tue Apr 7 02:16:39 2020 -0400 add n_processes param for pooled analysis commit 7fb477ff15c7f8d62d3acad4d14434942cd25bff Author: Kendell Clement Date: Tue Apr 7 00:36:47 2020 -0400 Pooled Set flag to skip reporting problematic regions commit 6c3aeff8b96d918ef2b3c518b73860d3f74480b8 Author: Kendell Clement Date: Mon Apr 6 19:56:56 2020 -0400 Pooled parallelization by chr Parallelized CRISPRessoPooled extraction to operate by chr Attempt to appease the dockerhub requrements -- require cython for compilation commit a66c4020f473d2dee80fa1257c04049b2ba6dbd3 Author: kclem Date: Fri Apr 3 13:41:38 2020 -0400 v2.0.33 plot updates allele plot sgRNAs that extend beyond plot are marked quantification window shading and right-side correction version commit 90e677f453ef971b478e4582120b17cbb572212c Author: Kendell Clement Date: Fri Apr 3 01:30:57 2020 -0400 Pooled bug fixes for regions with the same location and different names commit d795479d117e689a6679ffe463df413bff2f6a5a Author: Kendell Clement Date: Fri Apr 3 01:23:30 2020 -0400 Fix open error for docker commit 9aee866e5f65bf8df6495e81684651436a0b0a30 Author: Kendell Clement Date: Fri Apr 3 01:17:09 2020 -0400 Parallelization of Pooled and introduction of checkpoints for WGS and Pooled Alignment of amplicons is done in one bowtie2 call instead of one bowtiecall per amplicon Parallelize several time-intensive steps in Pooled (splitting by region, etc) --no_rerun flag will skip already-processed steps in Pooled and WGS commit fb1e7e25404bd80722824755c1b4ff4478449c48 Author: Kendell Clement Date: Thu Apr 2 01:34:52 2020 -0400 WGS updates - multiprocessing and no rerun WGS multiprocesses extraction of reads across multiple cores. WGS extraction doesn't occur if the --no_rerun flag is set and the files are all present. commit 45d4377f678fceeff83d60efa22dbd1078e6840e Author: Kendell Clement Date: Tue Mar 31 10:35:43 2020 -0400 Remove biopython for fastq parser Living life on the edge -- dropping the biopython fastq parser to remove biopython package dependency. This will discard the minimal error-checking provided by the biopython package. commit d52f69c9fa16ea38e919f9e3dc70fb6d53246610 Author: kclem Date: Tue Feb 25 17:05:08 2020 -0500 Pin dependency versions commit 86ff4bebcfcaa5c618ff124822ecb45de2b7c9ca Author: kclem Date: Tue Jan 28 16:17:57 2020 -0500 get rid of test for CRISPRessoCompare commit b7e96d6f4d1e0966cc0847b620349ee7fe21f43f Author: kclem Date: Tue Jan 28 15:26:20 2020 -0500 Fix problem with circleci testing Switch columns for output quantification files so that total reads is before the number of aligned reads commit ac976074c03967a386ed386d9ce3436c5fa1f2d5 Author: kclem Date: Fri Jan 24 14:02:14 2020 -0500 Version 2.0.32 - guide updates and other updates Introduced flexiguides - can match with flexibility to the reference sequence - useful for pooled screens of offtargets (parameter --fg or --flexiguide, with --fg or --flexiguide_homology to control how many mismatches) Flexiguide mismatches and other mismatches are shown on plots sgRNAs can be labeled (parameter -gn or --guide_name) sgRNA position shown on allele frequency plots detection of dsODN -- shown in plot 1d (parameter --dsODN) CRISPRessoPooled gene set input flexibility - more formats accepted plot 8 shown on html report commit ca9273377acbabf01f97f04a028d4fd87a09d6b5 Author: Kendell Clement Date: Fri Dec 6 15:14:38 2019 -0500 Fix docker bug #30 commit a3ae575f870ace49a459447e13288cfd50487e2f Author: kclem Date: Wed Oct 2 20:57:03 2019 -0400 remove debug statements commit 53d70eb83bad2aa8456b744dd29611a788f3dbbd Author: kclem Date: Tue Oct 1 15:41:39 2019 -0400 Fix #25 to accept bt2l bowtie2 index extension commit 039cc8e1b4a145e53cf06c1d52f102497928f3ac Author: kclem Date: Wed Sep 25 17:53:49 2019 -0400 v2.0.31 CRISPRessoPooled chr names fix, allele plot colors CRISPRessoPooled compatability with chr names with underscores (alternate scaffolds) Additional function to plot allele table with custom set of colors for a completed run commit c35d0151efcd70a6044399e16c841ee9d0ad0535 Merge: 491247e1 b1d1518a Author: kclem Date: Thu Sep 19 16:23:13 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 491247e1a07e2991a74341b4424c7c722a3db6a9 Author: kclem Date: Thu Sep 19 16:22:56 2019 -0400 updates to command line output, batch bug commit b1d1518a7ed4b72e92fd9d10586ccce653585630 Author: Kendell Clement Date: Fri Aug 23 11:26:53 2019 -0400 Update conda installation path commit cdfd78772de27bdb78a380e591d8857acd143562 Author: kclem Date: Tue Aug 13 11:24:58 2019 -0400 Use python 2.7 pandas commit 1417d1aa387d45f37953b93304a545e08943cc45 Author: kclem Date: Tue Jul 16 17:18:28 2019 -0400 force merge reads and fix #19 add optional param for force merging R1 and R2 in case they don't cover the entire amplicon fix labels for expected amplicon plots commit 60f90053ab65958871402b609d0b72b31021bc6b Author: kclem Date: Tue Jul 2 17:28:27 2019 -0400 v2.0.30 case-insensitive guides, fix #17 commit 702b9dce89e070d96064db314ac1c5155fedd42e Author: Kendell Clement Date: Tue Jul 2 16:18:17 2019 -0400 Update README.md commit 5a10eb9a9d24371d2bedf8b173f9c7401d7cbd53 Merge: bc7b3321 bc006c82 Author: kclem Date: Tue Jun 25 15:32:51 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit bc7b3321e1bb6b7ed0c8af48d609e86e55081f6b Author: kclem Date: Tue Jun 25 15:32:45 2019 -0400 plot window bug update commit bc006c826f338b9a1fcaec2286b506bdd2c548db Merge: 1f7171b8 feee2c2e Author: Kendell Clement Date: Thu Jun 20 00:56:45 2019 -0400 Merge pull request #16 from PEHGP/patch-1 args.trimmomatic_command for CRISPRessoPooled commit feee2c2eb20807933376f01a88102767ce5e42e4 Author: kuan <396777306@qq.com> Date: Thu Jun 20 09:44:32 2019 +0800 args.trimmomatic_command args.trimmomatic_command commit 1f7171b8eb2dcd63fada80fa6cac561e576f727c Merge: f6c9eed2 3d4a37a6 Author: kclem Date: Thu Jun 13 13:13:33 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit f6c9eed206b16fede7c88724c94d8d091f8657f3 Author: kclem Date: Thu Jun 13 13:13:20 2019 -0400 update pooled names commit 3d4a37a626f366f8f024ee7f62e017e8d68092b3 Author: Kendell Clement Date: Tue Jun 11 12:33:40 2019 -0400 Add web link to readme commit 89348066ede5c447db9f04b2337118a0624b8f63 Author: kclem Date: Tue Jun 11 11:14:55 2019 -0400 add batch percentage report commit 3bcd50d67c9b320c9f908eb2e3469143461e7df6 Author: kclem Date: Thu May 30 17:25:05 2019 -0400 document report param commit 79303435747a70b288b88bb076dda90b8d379f91 Author: kclem Date: Thu May 30 17:16:11 2019 -0400 output updates commit 9f14424f275f132a148308042db48c77cbda9b1d Author: kclem Date: Fri May 24 17:57:57 2019 -0400 plot updates, compare bug #12 commit 7f5482c411a3d56a73d7f0c66f0159b623653c2a Author: kclem Date: Tue May 21 17:47:08 2019 -0400 remove crispressocompare test condition (cuz has floats) commit 5e2f4094ec607f63981cd5aa2ee7f99ce1a20d12 Merge: aac9dfe7 5933d93c Author: kclem Date: Tue May 21 17:40:44 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit aac9dfe70292a90d6981b0859ff9ede8da7094a4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test changed test to something that won't be affected by float formatting commit 5933d93c5f1408f754895254306701b5b0eaadd4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test commit 7d15cdfaa4a323f4baad96bf36c97fd455dd6684 Author: kclem Date: Tue May 21 17:11:09 2019 -0400 Add compiled c file commit 50134d64d33dc984ac81a066e7c370b160b861b3 Author: kclem Date: Tue May 21 16:58:57 2019 -0400 v2.0.28 Add report for CRISPRessoCompare Standardize naming conventions for files and plots from CRISPResso Add data links to CRISPRessoBatch report CRISPRessoBatch plots using the plot window around the cut site instead of only the quantification window If only one reference, 'Reference' is not shown in data plots or as a file prefix Set plotting indexes once for each guide (previously, they were specific to the amplicon) Base editing plots now plot for each guide (previously, they were one for each amplicon) commit 9e86bef89884a0e5980c7781cfcd56243fdd42f0 Author: kclem Date: Wed May 15 14:28:14 2019 -0400 Standardize concept of windows for quantification and plotting #11 commit dd63974334c94816efe5878e4aab1ec3c3e1a6c5 Author: kclem Date: Wed May 1 14:49:24 2019 -0400 python division bug.. <3<3 commit 4a4b88885ab2e034bdcbca9566aebe911c58f427 Author: kclem Date: Wed May 1 14:35:33 2019 -0400 fix bug for spaces in filepaths commit f7e0aee5d948abd876b5048c87cae59e265d0c6f Author: kclem Date: Fri Apr 26 11:56:51 2019 -0400 min merge size commit 12cc6a93c0b02be86575c6c7fe8b7569a96e0edd Author: kclem Date: Fri Apr 26 11:41:09 2019 -0400 Relax flash merging, add parameter for stringent flash merging (#7), remove debug statements commit 38bfb3174d8d37297587243cb9e04469e7e54a20 Author: Kendell Clement Date: Thu Apr 25 22:50:08 2019 -0400 Update CRISPRessoPooledCORE.py fix numpy -> string error commit 273fe3a1ab731a32c26efdcab58a502cd2162104 Author: kclem Date: Mon Apr 15 13:38:17 2019 -0400 Fix CRISPRessoWGS tests commit 2c1ec9f5eadaf62ac7df35535aa26965d993e96a Author: kclem Date: Mon Apr 15 13:15:03 2019 -0400 add tests for CRISPRessoWGS commit 6632700fbde6b7f5e49e402471fea88a0966d2c0 Author: kclem Date: Fri Apr 12 10:29:43 2019 -0400 update docker run message, enable local testing, remove networkx commit c31355e15afc49f6aa069968c94408f5f7b484c0 Author: kclem Date: Thu Apr 11 16:00:44 2019 -0400 ignore test directory for docker commit aad55c922fa9b3948e5b194b26da3a8a3ccc97d2 Author: kclem Date: Thu Apr 11 15:52:45 2019 -0400 fix plot label bug for 10a, update fig 2a data commit c40b2de4f21e69404de153b9e12dee6654568b67 Merge: 560b65e7 88990dbb Author: kclem Date: Wed Apr 10 17:30:41 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 560b65e720a6dc5329203ecbc8c1b38b47aee219 Author: kclem Date: Wed Apr 10 17:30:34 2019 -0400 update tests for decimal shift for percentage in summary report commit 88990dbb29135d160879ed02dcac6874b0fed9f2 Author: Kendell Clement Date: Wed Apr 10 17:26:20 2019 -0400 Update README.md commit e4deb9211dadb3e9622f2eee38f0cc4930a0cf17 Author: kclem Date: Wed Apr 10 17:19:07 2019 -0400 v2.0.28 commit 098f5a68ba632987930fd70038af667e228b7a1f Author: kclem Date: Tue Apr 9 22:13:13 2019 -0400 update circleci path commit bb78ade66d20a826bbd8beee53711620869b86aa Author: kclem Date: Tue Apr 9 17:40:43 2019 -0400 circleci test path update2 commit e760c77bdd23fd087733c26eda86f15de6877bbf Author: kclem Date: Tue Apr 9 17:36:23 2019 -0400 circleci test path update commit 0e1b576959b09da72b101983d312842e06ea4c56 Author: kclem Date: Tue Apr 9 17:33:22 2019 -0400 circleci artifact storage commit 28c23d2172cf5c39faffd1ea498f6d771237567d Merge: c7da8466 e42b3a6b Author: kclem Date: Tue Apr 9 14:17:34 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit c7da84668556b897b43dd53eb2370c3404ab3fa2 Author: kclem Date: Tue Apr 9 14:17:23 2019 -0400 circleci testing for batch and pooled commit e42b3a6b4c11cab0ac9c29c378858706b7fd328e Author: Kendell Clement Date: Tue Apr 9 11:55:04 2019 -0400 Update badge links commit 6a435376fe43e6408507f1078518515f1f522ba8 Author: Kendell Clement Date: Tue Apr 9 11:42:21 2019 -0400 Got me some badges! commit f8c9713444472f5e3cef4fd7f7bce0704dd6a266 Author: kclem Date: Tue Apr 9 11:07:18 2019 -0400 circleci artifact update commit 8d4b825dcf01887db96b90640aafea564031a7e9 Author: kclem Date: Tue Apr 9 11:03:59 2019 -0400 circleci updates commit bfb3bc82abd22647554b80517554ef58e81d2090 Author: kclem Date: Tue Apr 9 10:51:53 2019 -0400 add expected results for circleci commit 8466e146d73a2c2149fdbcd67d4db656fe8c13e0 Author: kclem Date: Tue Apr 9 10:29:57 2019 -0400 circleci update commit 19c437975e57119531bcb0b9b2a6587a7d6b3ea7 Author: kclem Date: Tue Apr 9 10:14:42 2019 -0400 circleci update commit c665c19c97f4c6d4b45f40b3ac9522bf53ce05ba Author: kclem Date: Mon Apr 8 16:27:38 2019 -0400 circleci - use custom docker commit caa46ce2bc1de5e100bfd5db25179fb6b54f4d95 Author: kclem Date: Mon Apr 8 14:45:16 2019 -0400 python2 virtualenv commit a13e2fd2c9eea59652ec1ad7c6a9862a708bc33e Author: kclem Date: Mon Apr 8 13:54:32 2019 -0400 CircleCI testing commit 7257b54f77391da21532bf06ed84b1ce03d460e0 Author: Kendell Clement Date: Fri Apr 5 11:15:22 2019 -0400 Remove dependency on zip commit 7b694f507a61e424e994384cd765855d7f69dfb4 Author: kclem Date: Thu Apr 4 12:12:37 2019 -0400 add networkx requirement for py27 commit 099acbc8149dbbd5c99729ddc8e0928e3f890023 Author: kclem Date: Thu Apr 4 10:16:10 2019 -0400 Fixes for dockerhub commit 3a3bfbdd2373348901faac6fda468b70cb5ce725 Author: kclem Date: Thu Apr 4 10:09:06 2019 -0400 Bioconda submission fixes commit dff86e16812f2b9345ef86bd3185786f4f82d25a Author: kclem Date: Wed Apr 3 18:49:13 2019 -0400 More precise cleavage window and quant window plotting commit c8caf7fbdad415d4d3fb47cc5b9b0b76dd65deab Author: kclem Date: Wed Apr 3 17:49:54 2019 -0400 v2.0.27 add reports for pooled and WGS commit c68e3cb922d037c12a62c695c0ae1f6c148ace82 Author: kclem Date: Mon Mar 18 11:15:40 2019 -0400 batch info pickle, WGS bai location, meta mode commit 768c75c3a7864c365cf13fd65573859b9aa86ebe Author: kclem Date: Wed Mar 6 17:13:56 2019 -0500 v2.0.26 add report display name, remove paths from stored files, fix sgRNA plot, CRISPRessoPooled report HTML, add citation to report commit 58257b54fc440427e5437f8e7458fd5824020b6e Author: Kendell Clement Date: Fri Mar 1 16:55:27 2019 -0500 Update issue templates commit 50fb2d58f0d3777ba51d0f5a37e82cfc1a47ebff Author: kclem Date: Fri Mar 1 16:38:13 2019 -0500 Fix file naming and flash incompatibility commit eca34aebf86dc1b03c6107ee405cdf16898c8d51 Author: kclem Date: Wed Feb 20 17:26:42 2019 -0500 v2.0.25 Add inferring of guides commit 2ccec08691db17e6a2cfab8e310f5f29621319ca Author: kclem Date: Wed Feb 20 13:42:57 2019 -0500 quant window updates commit 21d558da1f8f5fed8f59de0eb154e5a8a505ff7a Author: kclem Date: Tue Feb 19 17:21:12 2019 -0500 add flash outies, fix quant window coords bug commit 22954e6d40ad2d237cb75c521a09f30c2b294066 Author: kclem Date: Tue Feb 19 11:17:30 2019 -0500 Update entrypoint for docker commit 0216329d8a8d1c072a269d14786820f943443153 Author: kclem Date: Wed Feb 13 16:40:39 2019 -0500 v2.0.24 update docker, setup.py commit e11b60fe1cf3c3e67e41964d45e843f28c2975a5 Merge: fb1687b8 d6a7b979 Author: kclem Date: Mon Feb 11 16:22:09 2019 -0500 v2.0.23 suppress plots, custom flash version commit fb1687b87de0cb7e5d1ab0acc2a2651d1be1fcec Author: kclem Date: Mon Feb 11 16:12:26 2019 -0500 v2.0.23 suppress plots, custom flash version commit d6a7b979e1ecd49b26c648f2007e1ecfa905ec14 Author: Kendell Clement Date: Fri Feb 1 12:20:50 2019 -0500 Update readme formatting commit 9e4c87c4f60edd82539b01b24f646962c0f06f4c Author: Kendell Clement Date: Thu Jan 24 17:13:28 2019 -0500 Add conda install instructions commit 4b2b06e52ee1aba4f3bdd0458c13813f2417fbf7 Author: kclem Date: Thu Jan 24 14:02:05 2019 -0500 add manifest.in commit 495e1f9829d6e72048b87a6972472c4242411286 Author: kclem Date: Wed Jan 23 11:05:44 2019 -0500 Change license location, license update commit 24c3b1b6ba66557b99469c0e12291fae2fcc800d Author: kclem Date: Wed Jan 23 11:01:44 2019 -0500 v2.0.22 commit 4a426bf715e37cbb8d619d54fc3a92385e9dcf1b Author: kclem Date: Tue Jan 22 15:25:13 2019 -0500 update batch amplicon naming commit f4fbe96b335c6e1a5b3dc252bf090e3d36ffb8cd Author: kclem Date: Tue Jan 22 12:44:00 2019 -0500 v2.0.21 detangled root location dependency from params commit 9d29737de257a2999f0763afd5f97ddafd6d2fe7 Author: kclem Date: Tue Jan 22 12:36:03 2019 -0500 Update CRISPRessoShared.py commit 573aa90cf70cd941c3e26361b2e656fbf34d8e5e Author: kclem Date: Fri Jan 18 18:10:47 2019 -0500 allow no cython commit 69811cf1eb58ac014a1ac34c56748aaffcead187 Author: kclem Date: Fri Jan 18 17:55:23 2019 -0500 require cython for installation commit 37ac0e6278c4825bb816c7cfe0a1fc3819abd516 Author: kclem Date: Fri Jan 18 17:44:19 2019 -0500 v2.0.20b prepare for bioconda integration commit 31c5ad02127366d0ace2849a6e996a86d690f4d1 Author: Kendell Clement Date: Tue Jan 15 17:22:53 2019 -0500 Update README.md commit 5b8cf82bfee3eeefcf2ba5bb31b4a60b25960df0 Author: Kendell Clement Date: Tue Jan 15 16:46:46 2019 -0500 Update README.md commit c932b8b4ad7c08d3fc94d5c57d0017001a78a42f Author: Kendell Clement Date: Tue Jan 15 16:40:59 2019 -0500 Update README.md commit 5cf5aa34b2ef97e38e45ee3394cd8b4aade50c6d Author: Kendell Clement Date: Fri Dec 21 15:03:50 2018 -0500 Add trimmomatic_command parameter commit 2ec374962c169c1a56cd48ff9080f7941433d016 Author: kclem Date: Thu Dec 20 18:30:01 2018 -0500 v2.0.19b HDR and WGS changes commit 78483624993c6fdb5ccc889a2a6f37036fb0f2c8 Author: kclem Date: Thu Dec 6 14:55:18 2018 -0500 add filtering for fastqs commit 17940c70b36d74d8b9134664b515f3b164c678b0 Author: kclem Date: Wed Nov 28 15:00:35 2018 -0500 v2.0.18b - fix bug with batch names, add param to suppress plots commit 038042e3cea983fc264460402d88e15766cc50b0 Author: kclem Date: Wed Nov 14 15:00:47 2018 -0500 Add router for docker commit 3ef96cc08a18f6eff9bb6115366e27304986ed27 Author: kclem Date: Wed Nov 14 14:52:41 2018 -0500 add Docker file commit 3e1cbf1dffc4dbc555942d45f91ad942237b81c3 Author: kclem Date: Wed Nov 14 14:29:44 2018 -0500 set white background for plots commit dd2cb254170b8363a7feb0427ed587d2093ebd3e Author: kclem Date: Wed Nov 14 11:13:02 2018 -0500 Set seaborn style commit dde6a675a5ba4e9ec100c29c2d59534aa39ef4fc Author: kclem Date: Tue Nov 13 16:13:55 2018 -0500 Fix line endings commit db0a67017f57c5a77bed9eb442cb7efa9df4b97a Author: kclem Date: Tue Nov 13 10:31:45 2018 -0500 v2.0.17b - bug with multiple references of different lengths commit 7f4afffa1485094aee4c0399a319a30c12ec473e Author: Kendell Clement Date: Tue Oct 23 17:22:07 2018 -0400 Update README.md commit 0843923b50d2db953600230ac23614f52591c4b4 Author: Kendell Clement Date: Tue Oct 23 16:59:48 2018 -0400 Update README.md commit 6f79084ce88502a40fe8b9b1839733ca224d14dd Author: Kendell Clement Date: Tue Oct 23 16:56:38 2018 -0400 Add files via upload commit 72d0b355b5d39164405b6311bda6231dbbdef371 Author: Kendell Clement Date: Tue Oct 23 15:44:26 2018 -0400 Update README.md commit 30fdc7fd5d73594b332efd546a1f4d85004a80d6 Author: Kendell Clement Date: Tue Oct 23 13:48:25 2018 -0400 Update README.md commit 5b11e51083ca87cbc1a8dff02b4634fe12176d29 Author: Kendell Clement Date: Tue Oct 23 13:42:11 2018 -0400 Update README.md commit 33d367310d702667d86202aba389a0fee4eba691 Author: Kendell Clement Date: Tue Oct 23 13:24:50 2018 -0400 Update README.md commit d6c647d32cdded7803f6b023949202c9486f5caa Author: Kendell Clement Date: Tue Oct 23 12:01:41 2018 -0400 Update README.md commit 5cb8fccf048a49b3798f6932c0b0db90569697fa Author: Kendell Clement Date: Mon Oct 22 17:42:41 2018 -0400 Update README.md commit 7b60f691450c932b5d32ff48218f187328d0726f Author: kclem Date: Tue Oct 16 15:27:34 2018 -0400 2.0.16b - batch mode report commit 6037c2945efdea6fecf67b037a70c30d4bc6696b Author: kclem Date: Fri Oct 12 18:14:27 2018 -0400 2.0.15 updates to pooled, adjust merging commit 326e1c9f370e1d6956ad565605309f37e214e927 Author: kclem Date: Thu Oct 4 10:28:07 2018 -0400 2.0.14b - adjusted flash overlap params, cannot take mult aln gap penlty commit 26ec80198dc06ca09eb525deaf387c330f701cac Author: kclem Date: Fri Sep 28 15:13:00 2018 -0400 default val for n_processes commit bec0d312629d006169495b241fb9ea0018380e62 Author: kclem Date: Tue Sep 25 10:55:21 2018 -0400 2.0.13b commit 51d1386856849af7f04be0ce587e97349c7149b8 Author: kclem Date: Wed Aug 22 18:18:23 2018 -0400 Produce report commit 017c409c19a6af99c09c97faff67c26ac902e157 Author: kclem Date: Mon May 14 16:44:32 2018 -0400 Initial Commit of files commit 4324c954cc4efa10fc01fc6d69f88253ef5a7483 Author: kclem Date: Mon May 14 16:31:29 2018 -0400 first commit commit 0c37209616db2848a623bb3fe84f17bf8429d402 Author: Samuel Nichols Date: Fri Jan 12 08:55:40 2024 -0700 Squashed commit of the following: commit 22fc03183a8070c30dfb74d5c23575ac19019855 Author: Samuel Nichols Date: Fri Jan 12 08:54:01 2024 -0700 Add guardrail partial commit e55f6b21972b578261bc5a864ce1d653d98f9e34 Author: Samuel Nichols Date: Mon Jan 8 07:50:59 2024 -0700 Functional guardrails, needs reports update commit 6e968e9699ed59a47d88191d03768e042d8b60a4 Merge: 32b49685 e948ce10 Author: Samuel Nichols Date: Mon Dec 18 13:34:36 2023 -0700 Merge branch 'guardrails-clean-history' of https://github.com/edilytics/CRISPResso2 into guardrails-clean-history commit 32b49685da320501dad2b0ebbb57887b66220ba8 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit 4e309cf6f732565d635de3d4c5d074ada3027e2d Author: Cole Lyman Date: Mon Dec 18 10:51:55 2023 -0700 Refactor to use CRISPRessoReports module commit e648dc087c0055bc5d2fca13c64071a371dea941 Author: Cole Lyman Date: Mon Dec 18 10:51:11 2023 -0700 Add CRISPRessoReports subtree commit e948ce107ebb0d1d99010ed12e937f34b5e607d4 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05b bb13e007 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e8 d0b41483 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f909713 b913fcb4 Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f6788 5f909713 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2d e5afa478 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f0 51a943c3 Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f0 5a10d638 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd468 442a48c7 Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. Co-authored-by: Kendell Clement commit a7ffd46822ce195d51ff4d3dba0f02fe9bc73c1e Author: Kendell Clement Date: Tue Jan 11 16:29:37 2022 -0500 Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9990790a0081b765c1f54f4a9b18db522ab4693 Author: Kendell Clement Date: Wed Jan 5 16:48:17 2022 -0500 Allow for mixed-case prime editing input, fix #187 commit 4acdcddbef3e812655b7af9def772f0bb0dc30b5 Author: Kendell Clement Date: Wed Jan 5 16:37:22 2022 -0500 Update insertion annotation for fastq_output and bam_output Fix index issue for fastq_output, add reporting of inserted bases to bam_output. Index for fastq_output is increased because the insertion happens immediately after the insertion coordinate. commit 2f84dd02787abffa6d39efbc50c82c92d1c87528 Author: kclem Date: Fri Dec 10 16:39:41 2021 -0500 Fastq_output report inserted bases when the `--fastq_output` parameter is provided, the inserted bases will be written to the output fastq file. Previously, a string like "DEL= INS=78(1) SUB= " would indicate a 1bp insertion at site 78. This update outputs strings like "DEL= INS=78(1+G) SUB= " with a plus character followed by the inserted bases. commit ba742bbfa533a672321b27b5122d7ea6658c9014 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Implement algorithmic improvements for find_indels_substitutions" This reverts commit 13414833ef6cd5b232097f6ff0325a2b8e28b214. commit 5c8e1c5de6e800c820d9ec5ffe4809737f1211de Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Add test cases for find_indels_substitutions" This reverts commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808. commit d9a072368ca991ffde52aa1ee29e17f2758ed279 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Try some additional cython optimizations" This reverts commit 38b101d57384b9f7d664ac8719c1585f5f7142a5. commit 38b101d57384b9f7d664ac8719c1585f5f7142a5 Author: Cole Lyman Date: Fri Oct 8 14:42:49 2021 -0600 Try some additional cython optimizations commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808 Author: Cole Lyman Date: Fri Oct 8 14:15:11 2021 -0600 Add test cases for find_indels_substitutions commit 13414833ef6cd5b232097f6ff0325a2b8e28b214 Author: Cole Lyman Date: Fri Oct 8 11:50:25 2021 -0600 Implement algorithmic improvements for find_indels_substitutions These changes remove the need for iterating over the alignment multiple times. commit f4b6cfc03951215c3b9019dc47beb2913a5448ab Author: Kendell Clement Date: Fri Nov 19 00:10:05 2021 -0500 Fix CRISPRessoPooled calls to deprecated pands ix commit 751fbdb4ce432f11f3fb66c5a34f7db3d1d75dc1 Author: swrosati <40470095+swrosati@users.noreply.github.com> Date: Fri Nov 12 12:33:04 2021 -0600 Update ylabel_values -> y_label_values Typo causing error in certain modes. commit 76c80713b7b8209c9399d67f718d7ade20f20d91 Author: kclem Date: Wed Nov 10 11:37:16 2021 -0500 Update dockerfile for pyparsing commit ef15caee29380f58aaae392c897fabe47587486e Author: Kendell Clement Date: Wed Nov 10 00:45:51 2021 -0500 Fix int bug for n_reads in CRISPRessoPooled commit fc1ae890712ab2dc36d5ff36c81f64a10a2c2337 Author: Kendell Clement Date: Wed Nov 10 00:34:34 2021 -0500 Spelling fix and version bump to 2.2.7 commit 08369e294d6756f727167af50f14ff75cb4864b5 Author: Kendell Clement Date: Wed Nov 10 00:30:03 2021 -0500 Add bam input and selected demuxing for CRISPRessoPooled Adds features for providing aligned bams as input to CRISPRessoPooled and for a faster demultiplexing when amplicons and genome are provided. The parameters are: --aligned_pooled_bam: Path to aligned input for CRISPRessoPooled processing. If this parameter is specified, the alignments in the given bam will be used to demultiplex reads. If this parameter is not set (default), input reads provided by --fastq_r1 (and optionally --fastq_r2) will be aligned to the reference genome using bowtie2. If the input bam is given, the corresponding reference fasta must also be given to extract reference genomic sequences via the parameter --bowtie2_index. Note that the aligned reads are paired-end seqenced, they should already be merged into 1 read (e.g. via Flash) before alignment. --demultiplex_only_at_amplicons: If set, and an amplicon file (--amplicons_file) and reference sequence (--bowtie2_index) are provided, reads overlapping alignment positions of amplicons will be demultiplexed and assigned to that amplicon. If this flag is not set, the entire genome will be demultiplexed and reads with the same start and stop coordinates as an amplicon will be assigned to that amplicon. commit 0cdcfd7c4af834f3270e61b4db4b5f6d3da10d7c Author: Kendell Clement Date: Wed Nov 10 00:24:15 2021 -0500 Remove unused var for bam_index commit 3647ace73c95a2f44e45b6b42b4f1748d3a47f4c Author: kclem Date: Wed Nov 3 09:43:28 2021 -0400 Add --min-overlap param for flash in CRISPRessoPooled commit eea442a763e0c6c41da16a15d6e11ddf6d222dc8 Author: kclem Date: Mon Nov 1 11:25:55 2021 -0400 Remove deprecated pandas .ix from WGS commit 3e6c281c931756acd26acca96249b2fa1ad1db31 Author: Cole Lyman Date: Thu Oct 21 11:10:19 2021 -0600 Add unit tests These unit tests were in the other repo, but weren't moved over for some reason. commit 50e7cb21570ddb757904728800e31c2bcbd0d060 Author: Cole Lyman Date: Thu Oct 21 11:09:38 2021 -0600 Add Makefile to run tests This makes it easy to test the integrations cases because it will automatically install the package when needed. commit de91d51bcd9b725533a45c86ae72bc27dd5d3150 Author: Cole Lyman Date: Thu Oct 21 11:28:02 2021 -0600 Replace `np.int` with `int` Apparently `np.int` has been deprecated, so in places where precision isn't important `np.int` has been replaced with `int` according to the instructions from the warning. commit cabebbefb2646967dbeee80af08ac14156b1b53c Author: kclem Date: Wed Oct 20 12:33:49 2021 -0400 Convert cols to numeric for PE commit c2bdd96651eef5af38fb7bbc11d257a827ac080d Author: kclem Date: Wed Oct 20 12:00:43 2021 -0400 Make loggers module-specific Matplolib sometimes logs verbosely to info, so this stops using the root logger commit 8196b6a81f477ddcb0e34d61dfb54085de20c1a0 Author: Kendell Clement Date: Tue Oct 19 22:00:23 2021 -0400 Fix unicode error for bam read/write commit a923a7c2ef182238bd6b8aa77289bac487b7679b Author: Kendell Clement Date: Tue Oct 19 21:59:47 2021 -0400 All sub-crispresso processes are run with 1 process commit 75205b0d41423e4f62796e9674b0edbee68a11b9 Author: Kendell Clement Date: Mon Oct 18 23:03:59 2021 -0400 Fix #161 add params for prime editing guide analysis commit ecf23ef23e5701b232bba547a6d7d4b96f085f26 Author: kclem Date: Mon Oct 18 15:02:29 2021 -0400 Add param for plotting custom plots centered at point Add parameter to plotCustomAllelePlot script: --plot_center which if set, will produce plots centered at this point instead of sgRNAs. commit 71bf32ca789dde1d44cf41b9d3b702f12336e010 Author: Kendell Clement Date: Wed Oct 13 00:48:14 2021 -0400 Update README.md commit 53197e62706e37db54f7ed50c94f38a938955e59 Author: Kendell Clement Date: Tue Oct 12 15:43:08 2021 -0400 Fix #160 plotting for cut sites in plot 5 commit 90b43eaa03c0ea0fdee62a7b244204cad50056cc Author: Kendell Clement Date: Mon Oct 4 15:45:20 2021 -0400 Remove version checks for old seaborn and numpy, fix #155 commit 5e9dedf6bd7c7b3ef998bed811760a41b5313c6e Author: Kendell Clement Date: Tue Sep 28 21:16:54 2021 -0400 Optimize get_command_output, fix gzip binary bug commit 46953c39b769a4fa43b2ef0822dd92f07c4f1b9b Author: Kendell Clement Date: Wed Sep 22 01:58:36 2021 -0400 Update expected tests for batch commit 856354aadebc8410956e724b555224a55941a618 Author: Kendell Clement Date: Wed Sep 22 01:57:59 2021 -0400 Update CRISPRessoBatchCORE.py Move parsing of batch params to after assignment of n_processes so this value is applied to sub-processes commit f7f81747207cb279fd2c0d91986c7d4e7137fde4 Author: Kendell Clement Date: Wed Sep 22 00:23:04 2021 -0400 Fix #149 bug when read is much longer than the given reference commit 798eb4322899f70e2e2a3df0fdfad9b5598d3a41 Author: Kendell Clement Date: Tue Sep 21 17:43:38 2021 -0400 Fix #150 Open fastq.gz in 'rt' mode commit fa168843e6036c6d4ec4a5d7acb097c6a1532a98 Author: Kendell Clement Date: Fri Sep 17 12:33:12 2021 -0400 Remove requirement for python 2.7 commit 80fe12efbb0ee5c321262822b237fc8220a3f2c6 Author: Kendell Clement Date: Thu Sep 9 17:41:27 2021 -0400 Print batch summaries for amplicons with only one sample Previously, batch summaries were only printed for amplicons with more than one sample to save bits. This change produces batch summaries for amplicons with only one sample, and we shift responsibility to the user for purchasing carbon offsets to compensate for the generation and storage of these redundant bits. commit 43065310bf28e6a08780222250abd0d39ff6d0c5 Author: Kendell Clement Date: Thu Sep 9 17:38:58 2021 -0400 Add parameter 'assign_ambiguous_alignements_to_first_allele' For ambiguous alignments, setting this flag will force them to be assigned to the first (as provided by the references -a first and then -e second) amplicon. Thus, no reads will be discarded as 'ambiguous' and all reads will be counted once in the analysis. Version bump to 2.2.4 commit 11eaacd3bac9ebeabad69c1d5d92f1fd1a783a17 Author: Kendell Clement Date: Fri Sep 3 22:34:59 2021 -0400 push down crispresso2_info items commit 20272e1e95305888ca744ac56af2c08554eb9f1b Author: Kendell Clement Date: Fri Sep 3 22:32:24 2021 -0400 remove mention of pickle commit 3517285473d423dc32c6e3fdbac69eecf0caa7fc Author: Kendell Clement Date: Fri Sep 3 22:30:36 2021 -0400 minor pushdown of report name in crispresso2_info commit 8129494607488bf270567d40d43e01e043699f06 Author: Cole Lyman Date: Fri Sep 3 17:31:54 2021 -0400 fixup! Move region related objects to results in crispresso2_info commit 88a82d27e840c29b95b1602ae1594bf161108979 Author: Cole Lyman Date: Fri Sep 3 16:40:40 2021 -0400 Move some objects in CRISPRessoPooled crispresso2_info objects Move the objects: - 'final_data' -> 'results'/'final_data' - 'running_mode' -> 'running_info'/'running_mode' commit b702ab3e53e88170c7517591ce7a7050ccf1c2ba Author: Cole Lyman Date: Fri Sep 3 16:36:20 2021 -0400 Move some objects in CRISPRessoBatch crispresso2_info Move the following objects: - 'completed_batch_arr' -> 'results'/'completed_batch_arr' - 'batch_names_arr' -> 'results'/'batch_names_arr' - 'batch_input_names' -> 'results'/'batch_input_names' - 'nucleotide_frequency_summary_filename' -> 'results'/'nucleotide_frequency_filename' - 'nucleotide_percentage_summary_filename' -> 'results'/'nucleotide_percentage_filename' - 'modification_frequency_summary_filename' -> 'results'/'modification_frequency_summary_filename' - 'modification_percentage_summary_filename' -> 'results'/'modification_percentage_summary_filename' commit 2416c80e952cb58fa082ead5ef719ed7eab3eeb5 Author: Cole Lyman Date: Fri Sep 3 15:39:18 2021 -0400 Move nuc_quilt related objects to 'results'/'general_plots' to crispresso2_info The following objects have been moved: - 'window_nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'window_nuc_pct_quilt_plot_names' - 'nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'nuc_pct_quilt_plot_names' - 'window_nuc_conv_plot_names' -> 'results'/'general_plots'/'window_nuc_conv_plot_names' - 'nuc_conv_plot_names' -> 'results'/'general_plots'/'nuc_conv_plot_names' commit 37e354f94980d304e1cecf11fee95e61988dd3e3 Author: Cole Lyman Date: Fri Sep 3 14:58:36 2021 -0400 Move summary objects to 'results'/'general_plots' in crispresso2_info The following objects have been moved: - 'summary_plot_names' -> 'results'/'general_plots'/'summary_plot_names' - 'summary_plot_titles' -> 'results'/'general_plots'/'summary_plot_titles' - 'summary_plot_labels' -> 'results'/'general_plots'/'summary_plot_labels' - 'summary_plot_datas' -> 'results'/'general_plots'/'summary_plot_datas' - 'summary_plot_root' -> 'results'/'general_plots'/'summary_plot_root' - 'reads_summary_plot' -> 'results'/'general_plots'/'reads_summary_plot' - 'modification_summary_plot' -> 'results'/'general_plots'/'modification_summary_plot' commit 6df988151c602602470c944ccf561cd28f2dc30c Author: Cole Lyman Date: Fri Sep 3 14:04:12 2021 -0400 Move region related objects to results in crispresso2_info This includes: - 'regions' -> 'results'/'regions' - 'all_region_names' -> 'results'/'all_region_names' - 'good_region_names' -> 'results'/'good_region_names' - 'good_region_folders' -> 'results'/'good_region_folders' commit 053691311b158a2d3620670d20983d546a9c2c7b Author: Cole Lyman Date: Fri Sep 3 13:44:46 2021 -0400 Move 'samples_quantification_summary_filename' to 'results'/'alignment_stats'/'sample_quantification_summary_filename' in crispresso2_info commit eda3d50b6fafde4ea3145980cb2010417b799d88 Author: Cole Lyman Date: Fri Sep 3 13:27:42 2021 -0400 Move 'finished_steps' to 'running_info'/'finished_steps' in crispresso2_info commit 1da829c1c3cfd2bda9aaccca0aaa28c78a8f7271 Author: blasvicco@gmail.com Date: Mon Aug 30 17:48:02 2021 +0200 BugFix: database_id referenced before declared. commit c9321cc2cfcac34e4b6d8e1aa9fde9f25382e2de Author: Kendell Clement Date: Sat Aug 21 00:15:52 2021 -0400 Version bump to 2.2.2 for some reason the last release didn't pick up the last commits. commit 3aadab69ef1e3a44f12ee277a7767648e943a787 Author: Cole Lyman Date: Fri Aug 20 12:09:49 2021 -0400 Update filterFastqs so that the numpy arrays are writable commit 7ca129d2471aeebef2ba6d2dadf36265810f1a5c Author: Kendell Clement Date: Fri Aug 20 02:00:42 2021 -0400 Update CRISPRessoPlot.py Undo attempted matplotlib deprecation warning fix commit 8e8a65c0f29b6f99662ac1d272aa7199871f5cf5 Author: Kendell Clement Date: Fri Aug 20 01:26:50 2021 -0400 Plotting bug fixes Display and plotting fixes for batch report labels, and general plots, as well as a matplotlib deprecation fix commit 3f114752c5eca1554fcebf044b604b60a3eebeae Author: Kendell Clement Date: Fri Aug 20 00:06:38 2021 -0400 add batch summary of frameshift/splicing Closes #116 by adding frameshift/splice summary to batch Version bump to 2.2.1 Fix unicode error in slugify commit 5ad9fbc6645475885fc0f35bc26afd353a2b3d5f Author: Cole Lyman Date: Mon Aug 16 14:16:13 2021 -0400 Read plain text fastq as bytes in filterFastqs commit 6c20f28678469875392e3fc8946018b53a00256b Author: Kendell Clement Date: Mon Aug 16 13:35:12 2021 -0400 Remove test for seaborn version commit cec965feff65b4228d42784e498398b73b8caa49 Author: Cole Lyman Date: Mon Aug 16 12:16:04 2021 -0400 Fix filterFastqs This commit fixes the filterFastqs script by properly casting to strings (from bytes) in the appropriate places. It also replaces the deprecated `np.fromstring` function with `np.frombuffer`. commit 3c30e56a15fa15a3537c3d51e429c1ff8738d6fe Author: Cole Lyman Date: Thu Aug 12 09:57:50 2021 -0400 Add .gitignore file commit 2461e30770d5ae8a6686530981a4bf5c913e2f68 Author: Cole Lyman Date: Thu Aug 12 09:50:01 2021 -0400 Decode result from Popen from bytes to string This is done only where a string is necessary, the instances where this is not done are where the result is cast to a float or int (because they can accept bytes as input directly). commit 64ec8bacf9ae8cb0d3a9093ad12a916983f32207 Author: Cole Lyman Date: Sat Aug 7 13:49:05 2021 -0400 Refactor `results/refs` and `results/ref_names` crispresso2_info fields commit ebb33a7d691b524ed7ab92b5a870da09df045e00 Author: Cole Lyman Date: Fri Aug 6 20:32:03 2021 -0400 Refactor `results/general_plots` crispresso2_info fields commit 94768e209e2424394efb4d902aac4f0e4268aad3 Author: Cole Lyman Date: Fri Aug 6 14:58:25 2021 -0400 Refactor `results/alignment_stats` crispresso2_info fields commit a58b7a2b40bc99346a9ea8f6240034963b3d6b31 Author: Cole Lyman Date: Fri Aug 6 14:10:33 2021 -0400 Refactor `running_info` crispresso2_info fields commit a92ea4687e0cc69844c5d4a6250757540076ed59 Author: Kendell Clement Date: Thu Aug 5 14:28:29 2021 -0400 Version bump to 2.2.0 commit 20e9b6f228949886ebe592d4542aaddbb9fb123a Author: Cole Lyman Date: Thu Aug 5 11:52:22 2021 -0400 Remove argparse dependency Remove the argparse dependency from setup.py because argparse is a standard module in Python 3. commit ecf70ea4379e079e7669fc5ed8baabdcd11a8c61 Author: Kendell Clement Date: Fri Jul 30 01:23:38 2021 -0400 Update Dockerfile bowtie throws an error 'Can't locate Sys/Hostname.pm in @INC' This fixes it. commit 30fb34e17787858d4218eb0b0fcc1baf6259ee23 Author: Kendell Clement Date: Fri Jul 30 00:39:48 2021 -0400 Ignore python egg for copying, pin tbb for bowtie error https://github.com/BenLangmead/bowtie2/issues/336 commit c850abb672019a3684a544fccde9550f7eeb86f5 Author: Kendell Clement Date: Thu Jul 29 20:13:32 2021 -0400 Update config.yml commit eb70cb18d6910f418bb8052f45b58c05c397af43 Author: Kendell Clement Date: Thu Jul 29 17:47:06 2021 -0400 Update config.yml commit 01f079fd663027574115a0af974564d5b5efbe97 Author: Kendell Clement Date: Thu Jul 29 17:22:48 2021 -0400 Update CRISPResso2_router.py commit 2e3b5d42b8ea6705dbd2c2f59312b93390083da3 Author: Kendell Clement Date: Thu Jul 29 17:20:16 2021 -0400 Update Dockerfile commit 46d8c44f2dbe7a67531d3ccae6b0a3c51b12b9ca Author: Kendell Clement Date: Thu Jul 29 17:07:02 2021 -0400 Update Dockerfile commit 0999e82dd8e184a6b0aefa7a35fc9decaa1fc20d Author: Kendell Clement Date: Thu Jul 29 16:59:47 2021 -0400 Update Dockerfile commit 34b93cfeeb95d0a1375378996e3ca29e79fe5311 Author: Kendell Clement Date: Thu Jul 29 16:50:20 2021 -0400 Python 3 commit e040ac488b050d6f316d21196de002ef09383915 Author: Kendell Clement Date: Tue Jul 27 10:04:43 2021 -0400 Add testing file for batch, remove debug print lines commit aa7b9998c4660438f4303a57386f01617ddec1bb Author: Kendell Clement Date: Thu Jul 22 10:43:52 2021 -0400 Update Batch to allow for max processes usage commit 1566a1110215e32f338497040f1279c3581b6dcd Author: Kendell Clement Date: Wed Jul 21 01:02:24 2021 -0400 Fix reference to BadParameterException commit fea5017109ab56c955b8053eebad5f6f4218bc65 Author: Kendell Clement Date: Mon Jul 12 16:11:05 2021 -0400 Allow spaces in run names, print run name to report commit b8594354c96131ba33c9277fb719c95b1cf72f3d Author: Kendell Clement Date: Tue Jun 29 14:12:03 2021 -0400 Update CRISPRessoPooledCORE.py Fix debug statement commit 9bc9b9959cbc8faf9dc462669e333298a80a71d1 Author: Kendell Clement Date: Tue Jun 29 14:09:13 2021 -0400 Update CRISPRessoPooled for samtools sort status updates Redirect samtools sort status updates to log file. Version bump. Previously this would cause an error: `ERROR: list index out of range samtools view: writing to standard output failed: Broken pipe samtools view: error closing standard output: -1`. commit ad5b9d853ac3559e58a5ad569e7d3ac9c3d8a719 Author: Kendell Clement Date: Thu Jun 24 12:10:31 2021 -0400 Allow max processes for CRISPRessoPooledWGSCompare Users can provide -p max to use all processes available for comparisons commit 48d6c8724f541b285e5d6889723cc37fc99e5cfa Author: Kendell Clement Date: Wed Jun 23 20:53:51 2021 -0400 Create html report for CRISPRessoComparePooledWGS commit abe3054dd9b95f51cbf34e3c37e088f6beb8287c Author: Kendell Clement Date: Wed Jun 23 20:53:22 2021 -0400 Fix allele plot bug #103 If no regions are returned, max of a pandas dataframe returns an error because the df is empty commit 1ccb507858d92516e28286358d3aae9cce2cf7ea Author: Kendell Clement Date: Wed Jun 23 20:51:16 2021 -0400 Fix command prompt logo lines commit 293c249c0f380576121a123dec982e40409c977e Author: Kendell Clement Date: Sun Jun 20 00:26:34 2021 -0400 Update plotCustomAllelePlot.py Get rid of debug statements commit 947fbab70e0f4fa78cc43273b4fe2be5225043cc Author: Kendell Clement Date: Sun Jun 20 00:22:56 2021 -0400 Add plotCustomAllelePlot script for replotting allele plots commit 167a48659684ce1669da915ddb6995935c6a1aa6 Author: Kendell Clement Date: Wed May 26 11:16:51 2021 -0400 Update README with explicit instructions to activate conda environment commit af0b7e441d2a4f1e035fabfd353c58784f688371 Author: Kendell Clement Date: Sun May 23 00:22:00 2021 -0400 Update test results for more flexible pooled alignment The relaxing of bowtie parameters allowed more reads to align, which changed the expected test results for CRISPRessoPooled commit 0e08cd05c2f3279ac95a068be76f1d36a4b0224d Author: Kendell Clement Date: Sun May 23 00:06:21 2021 -0400 Fix double rows in Alleles_frequency_table due to read directionality Previous to this fix, forward and reverse reads having the same sequence would appear in different rows in the Alleles_frequency_table. This fix doesn't change counts or results, just combines the two rows in the Alleles_frequency_table. commit 7d2d761a3d4c25915be0d27b36f3b4a87068587d Author: Kendell Clement Date: Thu May 20 16:05:27 2021 -0400 CRISPRessoPooled - get rid of -k bowtie2 parameter The -k 1 parameter may not yield the best alignment. This commit gets rid of that parameter. commit 44dc9e75c660f4b3b683f1c80f5a964aa55e75bd Author: Kendell Clement Date: Wed May 19 22:52:46 2021 -0400 CRISPRessoPooled alignment more tolerant When given a genome file, CRISPRessoPooled aligns reads to the genome using the Bowtie2 aligner. The legacy parameters were somewhat strict. The new parameters reflect the 'default_min_aln_score' parameter in allowing for substantially more indels and mismatches than previous. The parameter `--use_legacy_bowtie2_options_string` has been added to use the legacy settings. Otherwise, the bowtie2 alignment settings will be calculated as follows: commit 84b870ce0d489295501aa57711edd6b18c011b92 Author: Kendell Clement Date: Tue May 4 10:22:22 2021 -0400 Raise exception if quantification_window_coordinates looks like an int Previously, if quantification_window_coordinates looks like an int when pandas parsed a batch file, it would throw an error trying to split the int. Now an exception will be raised. commit e25a6dbb13fcd0565f4c81e3ea42cfd115aa1bc0 Author: Kendell Clement Date: Mon Apr 26 16:19:00 2021 -0400 Fix bug if all reads are discarded If all reads are discarded, CRISPResso would fail because the df_alleles is empty. This adds discarded alleles to the df_alleles. commit 156d7d640355ad4755c6ce4cbab1b75bf31677c9 Author: Kendell Clement Date: Fri Apr 9 13:24:51 2021 -0400 CRISPRessoPooled - close active file in demux commit c5d9fcbbf5bd9b307c9050498d08e72dd98d42aa Author: Kendell Clement Date: Fri Apr 9 12:29:02 2021 -0400 Keep old awk command for speed for samples with <50 amplicons commit c7539661fc5bd29f4f6ee1e9c7795be932b53a8e Author: Cole Lyman Date: Fri Apr 9 12:18:45 2021 -0400 Alt pooled processing implementation (#87) These changes implement separating reads to their corresponding amplicons via Python instead of through awk. This is to get around the maximum number of open files that is limited on many operating systems. Co-authored-by: Kendell Clement commit e57d98738fa832766c78dc7eecfdb0588d92634d Author: Kendell Clement Date: Thu Apr 8 12:36:43 2021 -0400 Alt pooled processing implementation commit 4598226837cd4b726ae38f50958f977bde4794ca Author: Kendell Clement Date: Sun Mar 21 00:16:19 2021 -0400 HDR Updates - yw #82 Multiple amplicon names are resolved before adding the HDR amplicon -- unnamed amplicons are named Amplicon{i} for each amplicon. Plot 4g data (nuc pct table, mod pct table for all reads aligned to the first reference) is output and linked to from the plot display Ambiguous reads don't contribute to plot 4g data (which would otherwise lead to double counting and pct values > 1) commit d7915b2c561173238c6b0e82863f76a783db7c4d Author: Kendell Clement Date: Sat Mar 20 01:12:55 2021 -0400 Fix #72 bam_input error commit 32a9e98ffc6ceb1dd56e5e29c369176ed788c985 Author: Kendell Clement Date: Sat Mar 20 01:01:17 2021 -0400 Prime editing, fastq_out updates Prime editing input parameters are forced to be in the RNA 3'->5' direction. This makes sure that the scaffold incorporation happens on the correct side of the extension sequence. Errors are thrown if improper directionality is detected. Fastq_out now includes alignment scores and details for every run (it may be time to upgrade that SSD to hold these new fastq output files, but it makes debugging particular reads a lot easier!) Update to linked data for plot 3b in report commit c1b6ea2ecebd920ea12ab91f2d50e35c274f94fe Author: Kyle McChesney Date: Wed Mar 10 13:40:44 2021 -0700 fix missing import path on NaN commit 1b24ca351f4337e0a7bece7ef7addb4558f950d8 Author: Kendell Clement Date: Sat Feb 20 22:57:07 2021 -0500 Update links in readme to https - fix #79 commit d550fd1db8ce0e1d11ed77fd082c53a3d887bbc2 Author: Kendell Clement Date: Fri Feb 12 16:57:37 2021 -0500 Insertion quantification change + fix #76 Starting in version 2.1.0, insertion quantification has been changed to only include insertions completely contained by the quantification window. To use the legacy quantification method (i.e. include insertions directly adjacent to the quantification window) please use the parameter --use_legacy_insertion_quantification commit 798f661031236ee1aa5611f491ca135ec0432dc9 Author: Kendell Clement Date: Thu Jan 14 23:51:29 2021 -0500 Allow for int-looking chromosome names in WGS input file In CRISPRessoWGS, the region file contains a 'chr_id' column which is sometimes mis-recognized as ints when read by pandas if using the chromosome notation without 'chr' (e.g. 1,2,3 in stead of chr1,chr2,chr3). This bug fix forces chr_ids to be read as strs. commit 92c008673690a3cd32e33fe8cfcf07060cf6cb0f Author: Kendell Clement Date: Wed Dec 30 23:48:32 2020 -0500 Update README.md commit 8d590c5588d93ef0129512a5156fe3b45e2d3c4b Author: Kendell Clement Date: Wed Dec 30 23:46:36 2020 -0500 Introduction of CRISPRessoAggregate to aggregate stats across runs Adds CRISPRessoAggregate Adds start/end time to CRISPRessoBatch info Started removing pickle dependencies from Pooled and Report commit c8447246ada2758831a870f30ed99c496c18feb1 Author: Kendell Clement Date: Sat Dec 26 10:52:12 2020 -0500 Output alignment details for unaligned reads in fastq_out or bam_out commit dedc36cadc64e9302d88c3472652d2a4a2385d25 Author: Kendell Clement Date: Tue Nov 24 13:45:34 2020 -0500 Add scripts folder for one-off analyses commit 88b6e9c50c6d97da357534c67aaeba64c00ddc23 Author: Kendell Clement Date: Sat Nov 14 02:46:36 2020 -0500 Fix plot window cloning from Ref1 to HDR Plot window for sgRNA will be the same length after cloning even if the window is shorter or longer after comparing between ref1 and HDR. commit 7403a46e186c4b70ad606dbb0eebbbde684fac61 Author: Kendell Clement Date: Sat Nov 14 01:52:55 2020 -0500 Frameshift plot and HDR quant window updates Frameshift plots don't show 0-bp changes (these dwarf all other changes). The number of reads not shown are added to the legend. Addressed cloning quantification windows when bases are inserted in the clone-ee (previously these cloned bases would be ignored. Force HDR to clone all quantification windows from Ref 1 Fix #60 and #59 commit f3f9c122cc25ab62bdd7f30fb9d6ee4c9ab820a8 Author: Kendell Clement Date: Sat Nov 7 23:55:55 2020 -0500 Update histogram x-limits, caption, and data commit e9332f7eabed32e65430b44677c841dd0150ac5a Author: Kendell Clement Date: Sat Nov 7 02:26:59 2020 -0500 Fixes for when no reads align #63 commit 9fae60a57d99238e4f7a9ad2e992ae71cca49c60 Author: Kendell Clement Date: Sat Nov 7 02:08:59 2020 -0500 Standardize pie plot appearances commit f1738bd17b496a31d6f68a91ed7ae67a36f8f178 Author: Kendell Clement Date: Sat Nov 7 00:36:40 2020 -0500 plotting and pe fixes - Special bonus for y'all to keep you company during covid - axis ticks on most plots! - added parameter --plot_histogram_outliers to plot all insertion sizes in histogram - all insertion sizes are reported in .hist output files #64 - add HDR reference plot (may change this later to set ref1 to the longer reference of WT/HDR but for now it is always WT..) Allow reverse complement of extension seq if PE sequence is specified. commit 5a2d9271b3ea99342f22e59259149d30dc193e47 Merge: bd03668e 9812651c Author: Kendell Clement Date: Wed Oct 21 16:52:12 2020 -0400 Merge pull request #62 from matandro/patch-1 Fix a bug when generating compare plot commit 9812651c2aae1218393e8ce0d734812247cd59ab Author: Matan Drory Retwitzer Date: Wed Oct 21 10:45:01 2020 -0700 Fix a bug when generating compare plot Related to issue #61 This happens when N_ROWS < 1 which I assume has something to do with no results -> negative control commit bd03668ef1626a54e0b5bfbc010afacee60f45e3 Author: kclem Date: Fri Oct 2 17:39:52 2020 -0400 delete merging intermediate files commit 3c9b1344e19dee04b169c0860bc11304d6a594b5 Author: Kendell Clement Date: Wed Sep 30 23:51:08 2020 -0400 Version bump to v2.0.42 commit 2f8c6f9c7d31b276ff18000d579ef722f693c433 Author: Kendell Clement Date: Wed Sep 30 23:44:02 2020 -0400 Update new parameters, fix docker biuld problem commit f130270135b84e0904e0003858c263285834b6dd Author: Kendell Clement Date: Wed Sep 30 14:18:58 2020 -0400 Fix bug in read counting for interleaved fastqs commit faac4e1045e5b617b3743e328c0203ced27e3f31 Author: Kendell Clement Date: Thu Sep 24 22:37:50 2020 -0400 Fix bug in mode to write fastq out commit 388e8a97fc18648dd28e35063cb12680887395e2 Author: Kendell Clement Date: Wed Sep 23 23:49:44 2020 -0400 Update postrun reference output file Rename postrun references file to be more standardized with other output files. Output is now "CRISPResso2Pooled_postrun_references.txt" commit ee5be76e5e24e494abd93940622d51faa83a2371 Author: Kendell Clement Date: Wed Sep 23 23:32:34 2020 -0400 Multiple allele support for pooled mode Most common alleles for each pooled target are output if the flag '--compile_postrun_references' is provided. This writes alleles with frequncy defined by the parameter to --compile_postrun_reference_allele_cutoff This file can be manually edited to remove noisy alleles, and then used to run CRISPRessoPooled again but to provide alternate alleles to each CRISPResso run by using the parameter '--alternate_alleles'. This is particularly useful in cases where control experiments are available. The running pattern would be: 1) CRISPRessoPooled --compile_postrun_references {control} 2) CRISPRessoPooled --alternate_alleles {produced in step 1} {control} CRISPRessoPooled --alternate_alleles {produced in step 1} {experiment} commit fe5bee2ac7ca4a8dd165e2c7305a1c41a79b8d9d Author: Kendell Clement Date: Wed Sep 23 23:27:37 2020 -0400 Output + PE updates Add parameter '--fastq_output' to output a fastq file with annotations for each read. Also, the substitution frequency table is only produced in base editor mode -- in an effort to slim down CRISPResso2 output files. Also, especially for long Prime Editing insertions or deletions, the inference algorithm may incorrectly infer the prime-edited allele. I added the parameter '--prime_editing_override_prime_edited_ref_seq' which can be used to specify the prime-edited sequence manually. commit ca61c483a8a63fec2e85889f554648ea6f3a903c Author: Kendell Clement Date: Sat Sep 5 16:15:16 2020 -0400 update report caption for fig 10 commit 346c6f6e62097ea7690204cfe879ebb918fb244d Merge: e52cb734 44ccc5ec Author: kclem Date: Fri Sep 4 15:05:02 2020 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit e52cb73471f4538ecd98f943a38771f0d07a4476 Author: kclem Date: Fri Sep 4 15:04:48 2020 -0400 Update on help message for mark wildtype allele commit 44ccc5ec3ff6a57d265807b559010512f053d82a Author: Kendell Clement Date: Fri Sep 4 14:59:05 2020 -0400 Update to plotting quantification window plot vertically centered, pooled/wgs plots are not limited in height to allow analysis of large pools. commit ff675b12196593ceb8e8d8b58dfd6a8ca8865501 Author: Kendell Clement Date: Mon Jul 27 09:55:30 2020 -0400 Update parameters and description in README commit 25c6a1b45ef1e1c1243057b9b3ee06f638d55d26 Author: Kendell Clement Date: Sat Jul 25 00:29:14 2020 -0400 Update CRISPRessoBatchCORE.py If amplicon sequence is empty, auto mode is run for batches commit 2978a6ebc0cd977b36b4ea7e1965f5c43b46d325 Author: Kendell Clement Date: Thu Jul 23 08:46:01 2020 -0400 Removed WGS logging For some reason, piping output breaks multiprocessing blocking commit 8bc9214ba0a1b628b69a49a2cf306bf559e008b3 Author: Kendell Clement Date: Wed Jul 22 22:58:51 2020 -0400 Update CRISPRessoWGSCORE.py commit a9484fd481bfa8ad716c6326aba9c57f5271f885 Author: Kendell Clement Date: Wed Jul 22 14:24:38 2020 -0400 Add parameter discard_guide_positions_overhanging_amplicon_edge If run with param -- discard_guide_positions_overhanging_amplicon_edge, for guides that align to multiple positions, guide positions will be discarded if plotting around those regions would included bp that extend beyond the end of the amplicon. Normally this would cause CRISPResso to fail if plotting were requested beyond the end of an amplicon. commit 8d26edeed89e33451bd539c242f7ffaa560db3e6 Author: kclem Date: Wed Jul 22 13:58:10 2020 -0400 WGS doesn't print crispresso output to screen WGS printing error fixed commit 5ed03059d09aaf8a416c5fec553801eeca73355f Author: kclem Date: Tue Jul 21 00:00:40 2020 -0400 bam processing update of cached not-alignments commit cf593183d7b3d8200ec295ebcc653e518775046a Author: Kendell Clement Date: Thu Jul 16 21:49:21 2020 -0400 Fix naming of scaffold-incorporated reference links on plots commit 676ff623373b0cabf992017bd5eca255c4073d2a Author: Kendell Clement Date: Fri Jul 10 00:02:59 2020 -0400 Prime editing updates prime editing guides are shown as input on report output file names are produced without spaces commit 9de370148b2ada7210c10532247b3fa4b18a76de Author: Kendell Clement Date: Tue Jul 7 20:46:30 2020 -0400 Update batch for multiple quant window input commit 6ad1822f478d7f108b92f25378cc3ddf8dc3a2d3 Author: Kendell Clement Date: Wed Jul 1 16:26:38 2020 -0400 Bam processing + Prime Editing updates -Input can now be read from bam using the parameter `--bam_input` and (optionally) `--bam_chr_loc` to use the reads in the bam at this location as input. An output bam is produced with an additional soace-separated field prefixed by c2 (e.g. c2:Z:ALN=Inferred CLASS=Inferred_MODIFIED MODS=D47;I0;S0 DEL=56(47) INS= SUB= ALN_REF=TTGGCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGAAGTAGGGCCTTCGCGCACCTCATGGAATCCCTTCTGCAGCACCTGGATCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCT----------------------------------------------- ALN_SEQ=ACACCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGA-----------------------------------------------TCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCTGGAGCGGCGGCTGCACAACCAGTGGAGGCAAGAGGGCGGCTTTGGGC). Note that the alignment details (location, cigar string, etc) are not modified.. this may be done in the future). Bam file input cannot be trimmed or pre-processed with quality filtering. -Prime editing scaffold incorporation is now more accurate (looks for the scaffold sequence at the expected position directly after the extension sequence). A plot showing the number of bases matching the scaffold, as well as insertions after the extension sequence, and a data file with these numbers is produced. Added parameter `--prime_editing_pegRNA_scaffold_min_match_length` to define the minimum length required to classify a read as 'Scaffold-incorporated' -Renamed split_paired_end parameter to `--split_interleaved_input` for interleaved input -Auto mode now considers 5000 reads to detect amplicon sequences -Add new paramter `--annotate_wildtype_allele` to annotate wildtype alleles on the allele plots -Update output when reporting missing files -- only lists first 15 files in the current directory and directory of input parameter --reference https instead of http commit c0f2871befed86c4b314100584bf844f13d71d0e Author: Kendell Clement Date: Tue Jun 2 18:54:52 2020 -0400 Update CRISPRessoWGSCORE.py Remove debug print commit e5450b1cc6be518706969d27bda843cb7c16b082 Author: Kendell Clement Date: Tue Jun 2 18:53:20 2020 -0400 Updates for Pooled and WGS WGS gene annotations compatability fix and pooled gzip fix commit c2b286dbda3807752f34cdf6274e41d5640b408f Author: Kendell Clement Date: Tue May 12 00:33:36 2020 -0400 Fix docker bug, print version to Pooled + WGS commit 2a06cc18c3157e6c135dae7ce53adec94fe8a83f Author: kclem Date: Sun May 10 00:09:55 2020 -0400 Fix plotting bug if no sgRNAs given commit 1e3ca605e0c5ae65e8acc727667f952bc4c0d3ee Author: kclem Date: Sun May 10 00:00:32 2020 -0400 flexiguide fix commit 4e1d6b2b3424e725010e3e1a13522a7386228853 Author: Kendell Clement Date: Sat May 9 23:30:39 2020 -0400 Prime editing refinement and Pooled filesystem demand reduction - Prime editing parameters renamed, nicking guides match with flexibility - Prime editing extension seq is shown as a guide (with no cut site) - Prime editing summary plot included in report - Nucleotide plots are shaded when no changes from the reference sequence - sgRNA annotations are plotted on multiple lines if they overlap - N's don't count as substitutions - extended read analysis data available with --write_detailed_allele_table flag commit 8c584719b9771c01e53cfe409789b29a29fad665 Merge: f5069365 7624815e Author: Kendell Clement Date: Sat May 9 23:14:30 2020 -0400 Merge pull request #42 from ronaldhause/patch-1 ZipFile: set allowZip64=True to write larger allele frequency tables commit f5069365d37bd698ff3bf35e5c39a1b75e10dc1d Author: kclem Date: Sat May 9 12:04:10 2020 -0400 CRISPRessoPooled updates, fix #37 for too many files Demultiplexing in the case of amplicons + genome is parallelized to reduce sorting Only files with sufficient reads are demultiplexed and written Additional output file REPORT_READS_ALIGNED_TO_GENOME_ALL_DEPTHS.txt shows all alignment locations commit 7624815e3159d926d8a0710f674f44c616b68bd5 Author: Ron Hause Date: Sun May 3 22:50:07 2020 -0700 ZipFile: set allowZip64=True to write larger allele frequency tables Addresses terminating ERROR: Filesize would require ZIP64 extensions when trying to write compressed allele frequency tables > 2 GB commit 6af15a09033166b713df159a6ef850dde8867253 Author: kclem Date: Tue Apr 28 03:28:41 2020 -0400 int bug fixes commit 531753c0f5be89c255f65d876cba5e9bf00dd4a2 Author: Kendell Clement Date: Tue Apr 28 02:00:37 2020 -0400 Introduces support for prime editing, multiple window sizes and offsets, max processors commit 8e29c1e0966ebb2073d52a221ae77e56bb146431 Merge: adb0d8b7 039013ef Author: Kendell Clement Date: Fri Apr 24 13:06:46 2020 -0400 Merge pull request #41 from natecarlson/fix-name-error If the name column is called 'name', refer to it as 'name', not '#name'. commit 039013efe363603dfe89f10b857d58bb1ef8e8d9 Author: Nate Carlson Date: Fri Apr 24 09:46:21 2020 -0500 If the name column is called 'name', refer to it as 'name', not '#name'. commit adb0d8b791686846fa8522f46e064418cbfbdc1c Author: Kendell Clement Date: Mon Apr 20 16:26:32 2020 -0400 Update LICENSE.txt commit b514a68eea135e805333c67bf33bf0f1ea41a034 Author: Kendell Clement Date: Sun Apr 19 00:36:26 2020 -0400 Print CRISPResso command on multiprocessing fail commit 4bfadd08d30e1a8b926757c1af4a9c0c0dc0b484 Author: Kendell Clement Date: Sun Apr 19 00:03:58 2020 -0400 Index fix for crispresso multiprocessing Indexes were incremented for user enjoyment (1-based) but the more accurate approach is 0-based Also, no_rerun ignores changes to the flags 'debug' or 'n_processes' which shouldn't affect the outcome commit 867e692b326acd19ed5f291bd5699f5885b1d569 Author: kclem Date: Thu Apr 16 00:25:47 2020 -0400 Allele plot sgRNA labels stay on plot commit b0d89b4effc4242ec55ed4a3e20e8835a90e3588 Author: kclem Date: Wed Apr 15 23:58:46 2020 -0400 WGS fastq seqs are now uppercase, so guides match even in lowercase-masked genomes commit 8098f1f1dda6efdaf773b9f85622d79f32ac49c9 Author: kclem Date: Thu Apr 9 01:11:26 2020 -0400 Pooled multiprocessing updates commit 034ff2b5858dd29ab60017835695fad527a5213e Author: Kendell Clement Date: Tue Apr 7 02:16:39 2020 -0400 add n_processes param for pooled analysis commit 7fb477ff15c7f8d62d3acad4d14434942cd25bff Author: Kendell Clement Date: Tue Apr 7 00:36:47 2020 -0400 Pooled Set flag to skip reporting problematic regions commit 6c3aeff8b96d918ef2b3c518b73860d3f74480b8 Author: Kendell Clement Date: Mon Apr 6 19:56:56 2020 -0400 Pooled parallelization by chr Parallelized CRISPRessoPooled extraction to operate by chr Attempt to appease the dockerhub requrements -- require cython for compilation commit a66c4020f473d2dee80fa1257c04049b2ba6dbd3 Author: kclem Date: Fri Apr 3 13:41:38 2020 -0400 v2.0.33 plot updates allele plot sgRNAs that extend beyond plot are marked quantification window shading and right-side correction version commit 90e677f453ef971b478e4582120b17cbb572212c Author: Kendell Clement Date: Fri Apr 3 01:30:57 2020 -0400 Pooled bug fixes for regions with the same location and different names commit d795479d117e689a6679ffe463df413bff2f6a5a Author: Kendell Clement Date: Fri Apr 3 01:23:30 2020 -0400 Fix open error for docker commit 9aee866e5f65bf8df6495e81684651436a0b0a30 Author: Kendell Clement Date: Fri Apr 3 01:17:09 2020 -0400 Parallelization of Pooled and introduction of checkpoints for WGS and Pooled Alignment of amplicons is done in one bowtie2 call instead of one bowtiecall per amplicon Parallelize several time-intensive steps in Pooled (splitting by region, etc) --no_rerun flag will skip already-processed steps in Pooled and WGS commit fb1e7e25404bd80722824755c1b4ff4478449c48 Author: Kendell Clement Date: Thu Apr 2 01:34:52 2020 -0400 WGS updates - multiprocessing and no rerun WGS multiprocesses extraction of reads across multiple cores. WGS extraction doesn't occur if the --no_rerun flag is set and the files are all present. commit 45d4377f678fceeff83d60efa22dbd1078e6840e Author: Kendell Clement Date: Tue Mar 31 10:35:43 2020 -0400 Remove biopython for fastq parser Living life on the edge -- dropping the biopython fastq parser to remove biopython package dependency. This will discard the minimal error-checking provided by the biopython package. commit d52f69c9fa16ea38e919f9e3dc70fb6d53246610 Author: kclem Date: Tue Feb 25 17:05:08 2020 -0500 Pin dependency versions commit 86ff4bebcfcaa5c618ff124822ecb45de2b7c9ca Author: kclem Date: Tue Jan 28 16:17:57 2020 -0500 get rid of test for CRISPRessoCompare commit b7e96d6f4d1e0966cc0847b620349ee7fe21f43f Author: kclem Date: Tue Jan 28 15:26:20 2020 -0500 Fix problem with circleci testing Switch columns for output quantification files so that total reads is before the number of aligned reads commit ac976074c03967a386ed386d9ce3436c5fa1f2d5 Author: kclem Date: Fri Jan 24 14:02:14 2020 -0500 Version 2.0.32 - guide updates and other updates Introduced flexiguides - can match with flexibility to the reference sequence - useful for pooled screens of offtargets (parameter --fg or --flexiguide, with --fg or --flexiguide_homology to control how many mismatches) Flexiguide mismatches and other mismatches are shown on plots sgRNAs can be labeled (parameter -gn or --guide_name) sgRNA position shown on allele frequency plots detection of dsODN -- shown in plot 1d (parameter --dsODN) CRISPRessoPooled gene set input flexibility - more formats accepted plot 8 shown on html report commit ca9273377acbabf01f97f04a028d4fd87a09d6b5 Author: Kendell Clement Date: Fri Dec 6 15:14:38 2019 -0500 Fix docker bug #30 commit a3ae575f870ace49a459447e13288cfd50487e2f Author: kclem Date: Wed Oct 2 20:57:03 2019 -0400 remove debug statements commit 53d70eb83bad2aa8456b744dd29611a788f3dbbd Author: kclem Date: Tue Oct 1 15:41:39 2019 -0400 Fix #25 to accept bt2l bowtie2 index extension commit 039cc8e1b4a145e53cf06c1d52f102497928f3ac Author: kclem Date: Wed Sep 25 17:53:49 2019 -0400 v2.0.31 CRISPRessoPooled chr names fix, allele plot colors CRISPRessoPooled compatability with chr names with underscores (alternate scaffolds) Additional function to plot allele table with custom set of colors for a completed run commit c35d0151efcd70a6044399e16c841ee9d0ad0535 Merge: 491247e1 b1d1518a Author: kclem Date: Thu Sep 19 16:23:13 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 491247e1a07e2991a74341b4424c7c722a3db6a9 Author: kclem Date: Thu Sep 19 16:22:56 2019 -0400 updates to command line output, batch bug commit b1d1518a7ed4b72e92fd9d10586ccce653585630 Author: Kendell Clement Date: Fri Aug 23 11:26:53 2019 -0400 Update conda installation path commit cdfd78772de27bdb78a380e591d8857acd143562 Author: kclem Date: Tue Aug 13 11:24:58 2019 -0400 Use python 2.7 pandas commit 1417d1aa387d45f37953b93304a545e08943cc45 Author: kclem Date: Tue Jul 16 17:18:28 2019 -0400 force merge reads and fix #19 add optional param for force merging R1 and R2 in case they don't cover the entire amplicon fix labels for expected amplicon plots commit 60f90053ab65958871402b609d0b72b31021bc6b Author: kclem Date: Tue Jul 2 17:28:27 2019 -0400 v2.0.30 case-insensitive guides, fix #17 commit 702b9dce89e070d96064db314ac1c5155fedd42e Author: Kendell Clement Date: Tue Jul 2 16:18:17 2019 -0400 Update README.md commit 5a10eb9a9d24371d2bedf8b173f9c7401d7cbd53 Merge: bc7b3321 bc006c82 Author: kclem Date: Tue Jun 25 15:32:51 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit bc7b3321e1bb6b7ed0c8af48d609e86e55081f6b Author: kclem Date: Tue Jun 25 15:32:45 2019 -0400 plot window bug update commit bc006c826f338b9a1fcaec2286b506bdd2c548db Merge: 1f7171b8 feee2c2e Author: Kendell Clement Date: Thu Jun 20 00:56:45 2019 -0400 Merge pull request #16 from PEHGP/patch-1 args.trimmomatic_command for CRISPRessoPooled commit feee2c2eb20807933376f01a88102767ce5e42e4 Author: kuan <396777306@qq.com> Date: Thu Jun 20 09:44:32 2019 +0800 args.trimmomatic_command args.trimmomatic_command commit 1f7171b8eb2dcd63fada80fa6cac561e576f727c Merge: f6c9eed2 3d4a37a6 Author: kclem Date: Thu Jun 13 13:13:33 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit f6c9eed206b16fede7c88724c94d8d091f8657f3 Author: kclem Date: Thu Jun 13 13:13:20 2019 -0400 update pooled names commit 3d4a37a626f366f8f024ee7f62e017e8d68092b3 Author: Kendell Clement Date: Tue Jun 11 12:33:40 2019 -0400 Add web link to readme commit 89348066ede5c447db9f04b2337118a0624b8f63 Author: kclem Date: Tue Jun 11 11:14:55 2019 -0400 add batch percentage report commit 3bcd50d67c9b320c9f908eb2e3469143461e7df6 Author: kclem Date: Thu May 30 17:25:05 2019 -0400 document report param commit 79303435747a70b288b88bb076dda90b8d379f91 Author: kclem Date: Thu May 30 17:16:11 2019 -0400 output updates commit 9f14424f275f132a148308042db48c77cbda9b1d Author: kclem Date: Fri May 24 17:57:57 2019 -0400 plot updates, compare bug #12 commit 7f5482c411a3d56a73d7f0c66f0159b623653c2a Author: kclem Date: Tue May 21 17:47:08 2019 -0400 remove crispressocompare test condition (cuz has floats) commit 5e2f4094ec607f63981cd5aa2ee7f99ce1a20d12 Merge: aac9dfe7 5933d93c Author: kclem Date: Tue May 21 17:40:44 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit aac9dfe70292a90d6981b0859ff9ede8da7094a4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test changed test to something that won't be affected by float formatting commit 5933d93c5f1408f754895254306701b5b0eaadd4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test commit 7d15cdfaa4a323f4baad96bf36c97fd455dd6684 Author: kclem Date: Tue May 21 17:11:09 2019 -0400 Add compiled c file commit 50134d64d33dc984ac81a066e7c370b160b861b3 Author: kclem Date: Tue May 21 16:58:57 2019 -0400 v2.0.28 Add report for CRISPRessoCompare Standardize naming conventions for files and plots from CRISPResso Add data links to CRISPRessoBatch report CRISPRessoBatch plots using the plot window around the cut site instead of only the quantification window If only one reference, 'Reference' is not shown in data plots or as a file prefix Set plotting indexes once for each guide (previously, they were specific to the amplicon) Base editing plots now plot for each guide (previously, they were one for each amplicon) commit 9e86bef89884a0e5980c7781cfcd56243fdd42f0 Author: kclem Date: Wed May 15 14:28:14 2019 -0400 Standardize concept of windows for quantification and plotting #11 commit dd63974334c94816efe5878e4aab1ec3c3e1a6c5 Author: kclem Date: Wed May 1 14:49:24 2019 -0400 python division bug.. <3<3 commit 4a4b88885ab2e034bdcbca9566aebe911c58f427 Author: kclem Date: Wed May 1 14:35:33 2019 -0400 fix bug for spaces in filepaths commit f7e0aee5d948abd876b5048c87cae59e265d0c6f Author: kclem Date: Fri Apr 26 11:56:51 2019 -0400 min merge size commit 12cc6a93c0b02be86575c6c7fe8b7569a96e0edd Author: kclem Date: Fri Apr 26 11:41:09 2019 -0400 Relax flash merging, add parameter for stringent flash merging (#7), remove debug statements commit 38bfb3174d8d37297587243cb9e04469e7e54a20 Author: Kendell Clement Date: Thu Apr 25 22:50:08 2019 -0400 Update CRISPRessoPooledCORE.py fix numpy -> string error commit 273fe3a1ab731a32c26efdcab58a502cd2162104 Author: kclem Date: Mon Apr 15 13:38:17 2019 -0400 Fix CRISPRessoWGS tests commit 2c1ec9f5eadaf62ac7df35535aa26965d993e96a Author: kclem Date: Mon Apr 15 13:15:03 2019 -0400 add tests for CRISPRessoWGS commit 6632700fbde6b7f5e49e402471fea88a0966d2c0 Author: kclem Date: Fri Apr 12 10:29:43 2019 -0400 update docker run message, enable local testing, remove networkx commit c31355e15afc49f6aa069968c94408f5f7b484c0 Author: kclem Date: Thu Apr 11 16:00:44 2019 -0400 ignore test directory for docker commit aad55c922fa9b3948e5b194b26da3a8a3ccc97d2 Author: kclem Date: Thu Apr 11 15:52:45 2019 -0400 fix plot label bug for 10a, update fig 2a data commit c40b2de4f21e69404de153b9e12dee6654568b67 Merge: 560b65e7 88990dbb Author: kclem Date: Wed Apr 10 17:30:41 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 560b65e720a6dc5329203ecbc8c1b38b47aee219 Author: kclem Date: Wed Apr 10 17:30:34 2019 -0400 update tests for decimal shift for percentage in summary report commit 88990dbb29135d160879ed02dcac6874b0fed9f2 Author: Kendell Clement Date: Wed Apr 10 17:26:20 2019 -0400 Update README.md commit e4deb9211dadb3e9622f2eee38f0cc4930a0cf17 Author: kclem Date: Wed Apr 10 17:19:07 2019 -0400 v2.0.28 commit 098f5a68ba632987930fd70038af667e228b7a1f Author: kclem Date: Tue Apr 9 22:13:13 2019 -0400 update circleci path commit bb78ade66d20a826bbd8beee53711620869b86aa Author: kclem Date: Tue Apr 9 17:40:43 2019 -0400 circleci test path update2 commit e760c77bdd23fd087733c26eda86f15de6877bbf Author: kclem Date: Tue Apr 9 17:36:23 2019 -0400 circleci test path update commit 0e1b576959b09da72b101983d312842e06ea4c56 Author: kclem Date: Tue Apr 9 17:33:22 2019 -0400 circleci artifact storage commit 28c23d2172cf5c39faffd1ea498f6d771237567d Merge: c7da8466 e42b3a6b Author: kclem Date: Tue Apr 9 14:17:34 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit c7da84668556b897b43dd53eb2370c3404ab3fa2 Author: kclem Date: Tue Apr 9 14:17:23 2019 -0400 circleci testing for batch and pooled commit e42b3a6b4c11cab0ac9c29c378858706b7fd328e Author: Kendell Clement Date: Tue Apr 9 11:55:04 2019 -0400 Update badge links commit 6a435376fe43e6408507f1078518515f1f522ba8 Author: Kendell Clement Date: Tue Apr 9 11:42:21 2019 -0400 Got me some badges! commit f8c9713444472f5e3cef4fd7f7bce0704dd6a266 Author: kclem Date: Tue Apr 9 11:07:18 2019 -0400 circleci artifact update commit 8d4b825dcf01887db96b90640aafea564031a7e9 Author: kclem Date: Tue Apr 9 11:03:59 2019 -0400 circleci updates commit bfb3bc82abd22647554b80517554ef58e81d2090 Author: kclem Date: Tue Apr 9 10:51:53 2019 -0400 add expected results for circleci commit 8466e146d73a2c2149fdbcd67d4db656fe8c13e0 Author: kclem Date: Tue Apr 9 10:29:57 2019 -0400 circleci update commit 19c437975e57119531bcb0b9b2a6587a7d6b3ea7 Author: kclem Date: Tue Apr 9 10:14:42 2019 -0400 circleci update commit c665c19c97f4c6d4b45f40b3ac9522bf53ce05ba Author: kclem Date: Mon Apr 8 16:27:38 2019 -0400 circleci - use custom docker commit caa46ce2bc1de5e100bfd5db25179fb6b54f4d95 Author: kclem Date: Mon Apr 8 14:45:16 2019 -0400 python2 virtualenv commit a13e2fd2c9eea59652ec1ad7c6a9862a708bc33e Author: kclem Date: Mon Apr 8 13:54:32 2019 -0400 CircleCI testing commit 7257b54f77391da21532bf06ed84b1ce03d460e0 Author: Kendell Clement Date: Fri Apr 5 11:15:22 2019 -0400 Remove dependency on zip commit 7b694f507a61e424e994384cd765855d7f69dfb4 Author: kclem Date: Thu Apr 4 12:12:37 2019 -0400 add networkx requirement for py27 commit 099acbc8149dbbd5c99729ddc8e0928e3f890023 Author: kclem Date: Thu Apr 4 10:16:10 2019 -0400 Fixes for dockerhub commit 3a3bfbdd2373348901faac6fda468b70cb5ce725 Author: kclem Date: Thu Apr 4 10:09:06 2019 -0400 Bioconda submission fixes commit dff86e16812f2b9345ef86bd3185786f4f82d25a Author: kclem Date: Wed Apr 3 18:49:13 2019 -0400 More precise cleavage window and quant window plotting commit c8caf7fbdad415d4d3fb47cc5b9b0b76dd65deab Author: kclem Date: Wed Apr 3 17:49:54 2019 -0400 v2.0.27 add reports for pooled and WGS commit c68e3cb922d037c12a62c695c0ae1f6c148ace82 Author: kclem Date: Mon Mar 18 11:15:40 2019 -0400 batch info pickle, WGS bai location, meta mode commit 768c75c3a7864c365cf13fd65573859b9aa86ebe Author: kclem Date: Wed Mar 6 17:13:56 2019 -0500 v2.0.26 add report display name, remove paths from stored files, fix sgRNA plot, CRISPRessoPooled report HTML, add citation to report commit 58257b54fc440427e5437f8e7458fd5824020b6e Author: Kendell Clement Date: Fri Mar 1 16:55:27 2019 -0500 Update issue templates commit 50fb2d58f0d3777ba51d0f5a37e82cfc1a47ebff Author: kclem Date: Fri Mar 1 16:38:13 2019 -0500 Fix file naming and flash incompatibility commit eca34aebf86dc1b03c6107ee405cdf16898c8d51 Author: kclem Date: Wed Feb 20 17:26:42 2019 -0500 v2.0.25 Add inferring of guides commit 2ccec08691db17e6a2cfab8e310f5f29621319ca Author: kclem Date: Wed Feb 20 13:42:57 2019 -0500 quant window updates commit 21d558da1f8f5fed8f59de0eb154e5a8a505ff7a Author: kclem Date: Tue Feb 19 17:21:12 2019 -0500 add flash outies, fix quant window coords bug commit 22954e6d40ad2d237cb75c521a09f30c2b294066 Author: kclem Date: Tue Feb 19 11:17:30 2019 -0500 Update entrypoint for docker commit 0216329d8a8d1c072a269d14786820f943443153 Author: kclem Date: Wed Feb 13 16:40:39 2019 -0500 v2.0.24 update docker, setup.py commit e11b60fe1cf3c3e67e41964d45e843f28c2975a5 Merge: fb1687b8 d6a7b979 Author: kclem Date: Mon Feb 11 16:22:09 2019 -0500 v2.0.23 suppress plots, custom flash version commit fb1687b87de0cb7e5d1ab0acc2a2651d1be1fcec Author: kclem Date: Mon Feb 11 16:12:26 2019 -0500 v2.0.23 suppress plots, custom flash version commit d6a7b979e1ecd49b26c648f2007e1ecfa905ec14 Author: Kendell Clement Date: Fri Feb 1 12:20:50 2019 -0500 Update readme formatting commit 9e4c87c4f60edd82539b01b24f646962c0f06f4c Author: Kendell Clement Date: Thu Jan 24 17:13:28 2019 -0500 Add conda install instructions commit 4b2b06e52ee1aba4f3bdd0458c13813f2417fbf7 Author: kclem Date: Thu Jan 24 14:02:05 2019 -0500 add manifest.in commit 495e1f9829d6e72048b87a6972472c4242411286 Author: kclem Date: Wed Jan 23 11:05:44 2019 -0500 Change license location, license update commit 24c3b1b6ba66557b99469c0e12291fae2fcc800d Author: kclem Date: Wed Jan 23 11:01:44 2019 -0500 v2.0.22 commit 4a426bf715e37cbb8d619d54fc3a92385e9dcf1b Author: kclem Date: Tue Jan 22 15:25:13 2019 -0500 update batch amplicon naming commit f4fbe96b335c6e1a5b3dc252bf090e3d36ffb8cd Author: kclem Date: Tue Jan 22 12:44:00 2019 -0500 v2.0.21 detangled root location dependency from params commit 9d29737de257a2999f0763afd5f97ddafd6d2fe7 Author: kclem Date: Tue Jan 22 12:36:03 2019 -0500 Update CRISPRessoShared.py commit 573aa90cf70cd941c3e26361b2e656fbf34d8e5e Author: kclem Date: Fri Jan 18 18:10:47 2019 -0500 allow no cython commit 69811cf1eb58ac014a1ac34c56748aaffcead187 Author: kclem Date: Fri Jan 18 17:55:23 2019 -0500 require cython for installation commit 37ac0e6278c4825bb816c7cfe0a1fc3819abd516 Author: kclem Date: Fri Jan 18 17:44:19 2019 -0500 v2.0.20b prepare for bioconda integration commit 31c5ad02127366d0ace2849a6e996a86d690f4d1 Author: Kendell Clement Date: Tue Jan 15 17:22:53 2019 -0500 Update README.md commit 5b8cf82bfee3eeefcf2ba5bb31b4a60b25960df0 Author: Kendell Clement Date: Tue Jan 15 16:46:46 2019 -0500 Update README.md commit c932b8b4ad7c08d3fc94d5c57d0017001a78a42f Author: Kendell Clement Date: Tue Jan 15 16:40:59 2019 -0500 Update README.md commit 5cf5aa34b2ef97e38e45ee3394cd8b4aade50c6d Author: Kendell Clement Date: Fri Dec 21 15:03:50 2018 -0500 Add trimmomatic_command parameter commit 2ec374962c169c1a56cd48ff9080f7941433d016 Author: kclem Date: Thu Dec 20 18:30:01 2018 -0500 v2.0.19b HDR and WGS changes commit 78483624993c6fdb5ccc889a2a6f37036fb0f2c8 Author: kclem Date: Thu Dec 6 14:55:18 2018 -0500 add filtering for fastqs commit 17940c70b36d74d8b9134664b515f3b164c678b0 Author: kclem Date: Wed Nov 28 15:00:35 2018 -0500 v2.0.18b - fix bug with batch names, add param to suppress plots commit 038042e3cea983fc264460402d88e15766cc50b0 Author: kclem Date: Wed Nov 14 15:00:47 2018 -0500 Add router for docker commit 3ef96cc08a18f6eff9bb6115366e27304986ed27 Author: kclem Date: Wed Nov 14 14:52:41 2018 -0500 add Docker file commit 3e1cbf1dffc4dbc555942d45f91ad942237b81c3 Author: kclem Date: Wed Nov 14 14:29:44 2018 -0500 set white background for plots commit dd2cb254170b8363a7feb0427ed587d2093ebd3e Author: kclem Date: Wed Nov 14 11:13:02 2018 -0500 Set seaborn style commit dde6a675a5ba4e9ec100c29c2d59534aa39ef4fc Author: kclem Date: Tue Nov 13 16:13:55 2018 -0500 Fix line endings commit db0a67017f57c5a77bed9eb442cb7efa9df4b97a Author: kclem Date: Tue Nov 13 10:31:45 2018 -0500 v2.0.17b - bug with multiple references of different lengths commit 7f4afffa1485094aee4c0399a319a30c12ec473e Author: Kendell Clement Date: Tue Oct 23 17:22:07 2018 -0400 Update README.md commit 0843923b50d2db953600230ac23614f52591c4b4 Author: Kendell Clement Date: Tue Oct 23 16:59:48 2018 -0400 Update README.md commit 6f79084ce88502a40fe8b9b1839733ca224d14dd Author: Kendell Clement Date: Tue Oct 23 16:56:38 2018 -0400 Add files via upload commit 72d0b355b5d39164405b6311bda6231dbbdef371 Author: Kendell Clement Date: Tue Oct 23 15:44:26 2018 -0400 Update README.md commit 30fdc7fd5d73594b332efd546a1f4d85004a80d6 Author: Kendell Clement Date: Tue Oct 23 13:48:25 2018 -0400 Update README.md commit 5b11e51083ca87cbc1a8dff02b4634fe12176d29 Author: Kendell Clement Date: Tue Oct 23 13:42:11 2018 -0400 Update README.md commit 33d367310d702667d86202aba389a0fee4eba691 Author: Kendell Clement Date: Tue Oct 23 13:24:50 2018 -0400 Update README.md commit d6c647d32cdded7803f6b023949202c9486f5caa Author: Kendell Clement Date: Tue Oct 23 12:01:41 2018 -0400 Update README.md commit 5cb8fccf048a49b3798f6932c0b0db90569697fa Author: Kendell Clement Date: Mon Oct 22 17:42:41 2018 -0400 Update README.md commit 7b60f691450c932b5d32ff48218f187328d0726f Author: kclem Date: Tue Oct 16 15:27:34 2018 -0400 2.0.16b - batch mode report commit 6037c2945efdea6fecf67b037a70c30d4bc6696b Author: kclem Date: Fri Oct 12 18:14:27 2018 -0400 2.0.15 updates to pooled, adjust merging commit 326e1c9f370e1d6956ad565605309f37e214e927 Author: kclem Date: Thu Oct 4 10:28:07 2018 -0400 2.0.14b - adjusted flash overlap params, cannot take mult aln gap penlty commit 26ec80198dc06ca09eb525deaf387c330f701cac Author: kclem Date: Fri Sep 28 15:13:00 2018 -0400 default val for n_processes commit bec0d312629d006169495b241fb9ea0018380e62 Author: kclem Date: Tue Sep 25 10:55:21 2018 -0400 2.0.13b commit 51d1386856849af7f04be0ce587e97349c7149b8 Author: kclem Date: Wed Aug 22 18:18:23 2018 -0400 Produce report commit 017c409c19a6af99c09c97faff67c26ac902e157 Author: kclem Date: Mon May 14 16:44:32 2018 -0400 Initial Commit of files commit 4324c954cc4efa10fc01fc6d69f88253ef5a7483 Author: kclem Date: Mon May 14 16:31:29 2018 -0400 first commit commit a433b57c059e5c3fbba6dc4dbd6c66a4843741ad Author: Cole Lyman Date: Thu Dec 14 16:05:24 2023 -0700 Add in buttons to reports and row for multiReport styling (#17) commit e1a3d81115255a3db4107150af9162d43d4a2d29 Author: Cole Lyman Date: Tue Dec 12 14:50:45 2023 -0700 Run metadata (#16) * Reports refactor (#37) * Changes necessary for selenium tests * Changes to make interleaved and pooled tests possible * PopulatePooled error * Changes for pooled testing * Changes for WGS selenium test file loading * Changes for WGS selenium tests. All tests functional. * Files for testing * Writing text for pooled * Add smallGenome.fa * Jinja partial for color picker and pip install in dockerfile * Names for color fields * Color picker input added to cmd_to_run * Adding color routes to other versions * Fix for responsiveness on cup and title * Adding color-picker partial to wgs and pooled * Tabs for different style options * New style menu with tabs * Style menu completed * Rough framework for style admin page * Working style FileAdmin, access button, and further partial refactoring * Checkbox for custom colors that shows and hides color selectors, box on home page for style folder * Optional save file * Updated Docker file and style_files.html * Changes to pooled and wgs, reset Dockerfile * Add style files to pooled and wgs * Adding style_files to partial * Debuging * Adding styling * Colors function refactored and working for all types * Remove style from Compare * Style file check * Style dropdown - allow save json only for admin * DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files * Restyle the color pickers These changes make it so that the colors are in a row instead of a column when the screen is wide enough. They are also responsive, so when the screen is small the color pickers will return to a column. * Add margins around style file elements * Move style folder inside of server folder * Refactor styles to be part of the database instead of files This allows for greater flexibility in displaying and updating them through the web interface. * Refactor `style_handler` to read the style from the database This completes the refactor to store style parameters using the database instead of storing the styles as files. * Fix error when the default style can't be read from the database The intended and proper behavior is that no style parameter is added to the command, and that is what happens now. * Restyle the colors in the admin view Add border around the colors in the admin view as well as moving them to be more vertically centered with the name. * Succesfully implemented selecting default style * Implement color pickers in style admin view * Refactor saving style files when there is no name specified * Remove style file card from admin index page * Add some default styles and rename the default to "Original" * Rename style_file references to style This is a final clean up of refactoring how the different style properties are stored. They are now stored in the database, so it doesn't make much sense to call them style files. * Implement creating styles from the admin panel * Move where the style files are stored in Docker * Implement Docker Compose for both production and development * Replace sub, ins, del with Substitution, Insertion, Deletion * Replace WSGI with Gunicorn for both development and production This allows us to hotreload the code when running the development version and also adds the Flask Debug Toolbar Extension which will be helpful in debugging. * Add a Makefile for commonly used Docker compose commands * Add reverse proxy to make Apache redirects work properly * Layout and report update * Bootstrap 5 changes * Working, missing custom label * Working file upload in partial. * Changes to submission.js for bootstrap 5 and load file upload partial * New submission.js template file * Jinja partials for all submissions * Move styling to main.css file * Add server file to render js * Commit before adding subtree * Removing reports found in subtree * Adding files * Web updates refactoring done * Add function to render template partials without using Flask * Use the `render_template` function for each report * Update path to template directory to include `CRISPRessoReports` * Update indentation in report.html and extract log params into partial * Added a few changes from the selenium-tests branch on C2Web * fig_reports and replacement * summaries partials and html updates * Fix error when rendering multi reports * Layout.html for C2WEB and CLI * Bootstrap 5 and partials changes * Path correction * Jinja choice loader * Subtree working * Centering issue and submit button fix * Styling and bootstrap changes * Spacing changes, submission_compared fix, and submission_wgs file upload fix * Remove extra files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca933c065a0f0765ce899eb537ab1ace41d43 * Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes * Removing CRISPRessoReport files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca933c065a0f0765ce899eb537ab1ace41d43 * Removing unecessary logic from submission_compare * Set up standalone Apache server container in Docker Compose * Add C2Web conda environment file * Make C2Web Docker image smaller (now 2.25 GB uncompressed) * Change style to styles * Fix for updating labels * Make C2Web Dockerfile multi-stage and make CRISPResso2 hot-reloadable These changes modify the C2Web Dockerfile to be multi-stage, so that there is a base stage (shared between dev and prod), a dev stage, and a prod stage. The dev stage doesn't install CRISPResso2, but binds a local copy of CRISPResso2 so that it can be hot reloaded. In the prod stage, this installs CRISPResso2 via conda. * Clean up Dockerfile and add CRISPResso2 dependencies to C2Web Docker These dependencies (plotly, seaborn-base, and matplotlib-base) are added so that they don't need to be added when CRISPResso2 is installed. * Final style fixes, color circles for style files * Update README.md with Docker compose details and update ignore files * Install CRISPResso2 in the build stage of Dockerfile * Removed docker-compose.prod.yml and created docker-compose.public.yml Also, updated the Makefile. Now, the default docker-compose.yml should be a suitable configuration for client facing production. The docker-compose.public.yml is a good configuration for the public facing site and the docker-compose.override.yml is a good configuration for development. * Share environment variables between web and celery and update README * Add spacing around body and footer tags * Replace spacing utilities classes with Bootstrap 5 versions * Resize images and fix filepath * Hide base editing if checkbox unchecked * Base editing partial * Add padding around pegRNA radio buttons and plot window size Also, add a margin around the JSON file upload box. * Increase size of Submit buttons * Fix hamburger menu and add -bs- to data-target and data-toggle * Fix plot window size spacing in Pooled * Fix the vertical span of the input labels in WGS, Pooled and Batch * Fix Pooled layout * Make input labels in the forms the same width * Remove ALLOW_USER_STYLE_UPLOAD parameter * Remove jinja loader from report_routes * Reformat style in submit_routes.py and update docs * Convert tabs to spaces in style_selection.html * Use string interpolation instead of concatenation in submission.js * Add an authentication check before exposing server_files in submission.js * Fix indentation and convert tabs to spaces in many templates * Clean up old files and comments * Replace tabs with spaces and reindent template files * Update README.md with git alias and subtree information * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 461ca93..1efad70 1efad70 Replace tabs with spaces and reindent template files e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle df896b0 Resize images and fix filepath 2f70855 Add spacing around body and footer tags 5cd6d27 Final style fixes, color circles for style files f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' 321815d Removing CRISPRessoReport files 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 04558fb Remove extra files 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix 980fdc4 Styling and bootstrap changes 9d40474 Centering issue and submit button fix aa5071c Subtree working db30843 Jinja choice loader 3b67ac0 Path correction 3aaca48 Bootstrap 5 and partials changes 8f5d8a1 Layout.html for C2WEB and CLI 290d829 Fix error when rendering multi reports 546397a summaries partials and html updates 858a751 fig_reports and replacement 073f1fe Added a few changes from the selenium-tests branch on C2Web 1061ebb Update indentation in report.html and extract log params into partial c3781e9 Update path to template directory to include `CRISPRessoReports` 84e0969 Use the `render_template` function for each report ee721b3 Add function to render template partials without using Flask 08fcd4e Web updates refactoring done 99c8e22 Adding files ef333f0 Removing reports found in subtree 1bae0df Commit before adding subtree 1fbb427 Add server file to render js d1d6fdf Move styling to main.css file 1241569 Jinja partials for all submissions 0534637 New submission.js template file c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial ecd03f6 Working file upload in partial. ce5d20f Working, missing custom label 6ba73e7 Bootstrap 5 changes e05d146 Layout and report update 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion ea44128 Move where the style files are stored in Docker 7f03e98 Implement creating styles from the admin panel 9b27a2e Rename style_file references to style a233d10 Add some default styles and rename the default to "Original" 43a8d29 Remove style file card from admin index page 1a8f332 Refactor saving style files when there is no name specified 64a7b1c Implement color pickers in style admin view 17c93c1 Succesfully implemented selecting default style fd79cdd Restyle the colors in the admin view 3cd94b8 Fix error when the default style can't be read from the database 5e626bd Refactor `style_handler` to read the style from the database 0f66d4a Refactor styles to be part of the database instead of files 6c7d3c8 Move style folder inside of server folder 9f71f21 Add margins around style file elements 2a28549 Restyle the color pickers 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files dc4f2c7 Style dropdown - allow save json only for admin 15e7483 Style file check 7bd0e91 Remove style from Compare 0ab45f5 Colors function refactored and working for all types 2e24f8b Adding styling d6621f1 Debuging ed00c82 Merge with master 5150f9b Adding style_files to partial 957a9ca Add style files to pooled and wgs 66dc2d3 Changes to pooled and wgs, reset Dockerfile fa6b1cf Updated Docker file and style_files.html ee0fcfc Optional save file 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder 0f26e2c Working style FileAdmin, access button, and further partial refactoring b3b70bd Rough framework for style admin page e4731d7 Style menu completed 1bb37bc New style menu with tabs 58f7e56 Tabs for different style options 3de893d Compare (#34) e66bef1 Update AWS EB instructions.docx 658a218 Fix bug when trying to send recovery password with bad email creds ee32e36 Adding color-picker partial to wgs and pooled 34ea688 Fix for responsiveness on cup and title f0c4d07 Adding color routes to other versions 110fe14 Color picker input added to cmd_to_run e732478 Names for color fields 2934631 Jinja partial for color picker and pip install in dockerfile 48bbf9c Cup animation (#33) 2905248 Selenium tests (#31) 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides 570e42a Don't remove commas from amplicons or guides 0d70425 Add smallGenome.fa fc33197 Writing text for pooled dccfcb3 Files for testing 4cea67c Changes for WGS selenium tests. All tests functional. ff05713 Changes for WGS selenium test file loading 495a98d Changes for pooled testing 0ad86a5 Merge pull request #30 from edilytics/pooled-upload-fix 127eb8f PopulatePooled error 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests 7847687 Add link to CRISPRessoWGS from profile page and change header 666f73b Remove example block from CRISPRessoWGS submission page 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled 8d979a4 Fix bug where files_to_delete was being replaced and standardize append 09e55fc Changes to make interleaved and pooled tests possible f89eca8 Changes necessary for selenium tests 3efe4f9 Clean up test files a696363 Merge pull request #28 from edilytics/s3 dcef708 Remove changes for CRISPRessoCompare e0c79cf Add demo config file for eb 03aba8e Update AWS EB instructions.docx a671c4e Set version to 2.6.3 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled da5b15b Timezone for history is displayed in user local timezone e11691f Update history to show time of previous run be675fb Update pooled with s3 4c7d429 Add data links to pooled report 353e88f Update admin portal landing page 712e828 Show run type in history 2802252 s3 and user updates efc3ed8 S3 error catching af68341 New S3 Validation f7d64e0 AWS validation before submission 8446093 Update s3 for batch and paired modes 0e7d327 S3_Upload function imrpvoed -JF b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 c991d52 added s3 user database model ab4aa54 add model for s3 bucket 853cda9 S3_Functionality improved -JF 2f060a6 Implemented front-end s3 browsing e082a5f stub out viewing method c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length c85a93f Merge pull request #15 from edilytics/wgs-interface 712270a Add support for CRISPRessoWGS deaacee Extract out function to get server files in submit_routes 151eb15 Update crispresso2_info object fields b2a974d Bump CRISPResso verion to 2.2.4 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 7f2dc1c Stop trimming json error messages, fix #11 d28c03b Update reporting logic to use the new CRISPResso2_info schema 03ee46f Bump CRISPResso version in Dockerfile and download release from Github 9151c5d Add CRISPRessoPooled report template 25a6e37 Merge pull request #6 from edilytics/pooled-interface b47d288 Check length of amplicons for hosted version, closes #4 54c28b6 Update submission file extension check 8fcadee Add a link to CRISPRessoPooled interface in user dashboard 7fd0283 Implement CRISPRessoPooled backend and report functionality 4063eb3 Modify submission.js to accept .txt and .tsv files b770323 Create template file for CRISPRessoPooled submission interface d4f2ed0 Merge pull request #5 from edilytics/flask-modularization 8527384 Convert some celery configurations settings to new format 962a209 Install less and vim in Dockerfile c693668 Read CRISPResso2_info from json files instead of pickle files a469e08 Move LoginManager to user_routes.py f62e67a Create db tables in init_db.py 0d85c90 Move login_required to user_routes 6f5e33e Reformatting of remaining __init__.py e615c0b Extract report routes out of __init__.py 20f2601 Extract user routes out from __init__.py 5582612 Extract status routes out from __init__.py 2406a10 Extract submit routes out from __init__.py b562fcd Extract celery tasks from __init__.py faa785d Extract views out from __init__.py ff44576 Extract model classes out from __init__.py 914498f Merge pull request #3 from edilytics/2to3 86ea7da Replace RabbitMQ with Redis adca9fb Upgrade celery to version 5.0.5 244ec33 Convert from Python 2 to Python 3 28b4f37 Refactor Docker image to use Python 3 via micromamba 2359800 Allow interleaved batches 428720b Add features: Allow admin init, server discovery depth 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon 5062365 Update README.md 51e02f4 Update README.md ac4a6d5 delete other images 4f3ad88 Update README.md fc0de1d Update README.md 08defa1 Update README.md 9604983 Trycatch pickle loads c1facd7 get rid of debug print of email d699d4d crispresso2.0.45 e7ff079 Update param descriptions 1f12d59 2.0.44 b81febe crispresso to 2.0.42 1a967a8 update report 178c56d 2.4 e41076d Job expiration 41d1a4c check progress on setinterval 756e488 server-side files ad19c3c Update to crispresso 2.0.40 prime editing e3a194a update errors and ignore email config 2efb0bb Update README.md 58844a6 initial commit 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 1efad706b7c147dd45601111d697534fa993c007 * Indentation and parenthesis * Semi-colon to README * Add targets for .env and clean in Makefile * Add flower support to the development version * Add support to map individual directories in Docker Compose * Fix typo in README * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 1efad70..13f7ae2 13f7ae2 Fix command used and parameters elements. Increase print width and height to 100% dcf7391 Adding styling for print-only and screen only git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 13f7ae215990b155ecf9b7659800f3790407ebc2 * Working in docker * Increase the size of the center column when printing * Remove some page breaks * Restore block statement * Spacing fix for empty page problem * Change gunicorn reload engine to poll This is because when running through x86 emulation (on M1 Mac), the inotify file system events are blocked. But reloading works with polling! * Pin SQLAlchemy version to 2.5.1 because newer versions don't work Also, this fixes the entrypoint for the dev build stage. * Make clarifications in README and fix more spelling * Add back in deleted report.html * Don't add styles to the database if they already exist * Upgrade to python 3.9 * Add report_data object to render templates * Reset subtree * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 69cb5e2 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 69cb5e235bd62abd34c779a9701ae77b77b319a7 * Fix region_name to be run_names * Update sizing on graphs * Create environmental variable for crispresso_version and load colors if version is above 2.2.12 * Make env variable use ARG * Add version check * Changes to work with Docker compose * Fix style selection database path * Add pe_ref_seq to WGS input and update to data-bs-toggle * Remove extra div that affected the height of prime edited ref seq * Fix gene annotation file label cut off on WGS * Move style function to after folder_id is declared * Docker compose pin versions and search C2 args for --config_files * Remove gtag code * Update CRISPResso to 2.2.14 * Specify platform in docker compose file * Move jinja_partials install from dev to build stage * Fix running when no default style is selected (or when selected style isn't available) * Add zip and unzip to prod step * Clean execution logs after run finishes This will clean the full paths from the execution logs so that those aren't exposed to the user. This also ensures that this only happens in one place in the code (instead of multiple). * Update path to favicon * Reorder conda channels, remove flask-sqlalchemy pin, and fix wtforms The latest version of wtforms has moved ColorInput from `wtforms.widgets.html5` to `wtforms.widgets`, which is reflected in this commit. * Fix S3 file upload by adding form field * Remove pyopenssl pin * Add create_styles and createUsers to app context in init_db.py * Fixed padding in S3 file upload * Removed commented out S3 upload code for CRISPRessoPooled * Add volume mount to Apache in dev version This allows any files that are edited in the static folder to hot reload. * Don't show root folder of S3 buckets because it leads to weird behavior * Fix old Bootstrap margin and padding utilities The new version of Bootstrap has replaced `ml-1` with `ms-1` and `mr-1` with `me-1`, etc. Instead of being "left" and "right", it is now "start" and "end" to account for right to left languages. * Fix the close button on the S3 modal * Fix positive quantification window in radio button label * Add another app context to init_db.py * Update IDs for jQuery examples and how radio buttons are selected Because of upgrading to Bootstrap 5, the way that labels and inputs needed to be formatted, the previous way of selecting a radio button input no longer worked. Now to select a radio button element programmatically, you can issue a `.click()` and it will be selected. * Remove extra report file * Replace deprecated padding utility classes in report * Remove duplicate id's and add a few aria labels * Add correct MIME type to submission.js file * Disable caching on submission pages to improve back button behavior * Add + to quantification window for pooled and WGS * Add S3 buckets to WGS and Compare * Remove S3 buckets from WGS Not going to implement this now, because it would be a significant effort to do it correctly. * Add note to S3 modal about large files being expensive * Don't show style selector when `--config_file` parameter isn't available * Install CRISPRessoPro in the dev environment * Fix error with app contexts and databases in unit tests * Implement handling duplicate style names when saving to db * Move Plotly JS import to reports templates and out of layout.html * Remove font installer, less, and vim dependencies --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman * added metadata to report partial * fixed logout button condition in banner * Fix loading favicon.ico, remove duplicate log_params, and fix README typos * Fix bug where `metadata` key is not found in `report_data` --------- Co-authored-by: Samuel Nichols Co-authored-by: Cole Lyman Co-authored-by: McKay commit 5083058ccaaf41426c389bd8e20f7ca6164429ec Author: Cole Lyman Date: Fri Sep 29 14:55:41 2023 -0600 Finishing touches on README commit 7da597a224a355d6cb4e16b2162f2dd7f503030a Author: Cole Lyman Date: Fri Sep 29 14:45:30 2023 -0600 Update README with tip about merge conflicts and remove `-s subtree` commit b4db4966247910f978c959015967cbb708fa81d8 Merge: 31c3bbe 8fafd56 Author: Cole Lyman Date: Fri Sep 29 14:36:36 2023 -0600 Merge pull request #15 from edilytics/cole/reports-readme-updates-reports Update README commit 8fafd5628fbab893388eec8f8476c61d793d5481 Merge: 4b6a5c3 31c3bbe Author: Cole Lyman Date: Fri Sep 29 14:35:58 2023 -0600 Merge branch 'master' into cole/reports-readme-updates-reports commit 4b6a5c32588686957babe0a555a90e99ac7b310f Author: Cole Lyman Date: Fri Sep 29 14:26:14 2023 -0600 Add sources and helpful links commit ff0a04f7c1a7cd9980a4b14d145499d13bfc4199 Author: Cole Lyman Date: Fri Sep 29 13:44:05 2023 -0600 Update README.md with instructions on how to develop on branches commit 31c3bbe4e804f8e4e2f9303de20290110bef221c Author: Cole Lyman Date: Fri Sep 29 13:31:38 2023 -0600 Add in the shortcut for `git merge` commit 3256a60b20b7206f84262373335f85a4c1e2e340 Author: Cole Lyman Date: Mon Sep 25 13:29:34 2023 -0600 Update the README.md with updates steps on pushing commit b1b2fd1eb9027b7d3852cd4c035dfb9fd55971b0 Author: Cole Lyman Date: Mon Sep 25 10:42:37 2023 -0600 Update README.md commit 5c714b96951b73ef0f881ddbce75e6e9d9d8a585 Author: Cole Lyman Date: Mon Sep 25 13:18:49 2023 -0600 Fix missing remote parameter in README.md commit 32325fd8b60aeae83dfa92a08a5365839879f6e5 Author: Cole Lyman Date: Fri Sep 22 16:00:30 2023 -0600 Further clarificatoin in the README.md commit 758a92c2ac3082279c98b3fc47b4ebfc06a96015 Author: Cole Lyman Date: Fri Sep 22 15:53:57 2023 -0600 Add instructins to README about how to bring in commits from subtree commit 0e54f0416923c3daf54b7331ec1abbcf0070ede1 Author: Cole Lyman Date: Fri Sep 22 15:16:03 2023 -0600 Add clarification to frequency of first few steps commit 9059230b9a6b0153c54cdc5516f9dfd33ee864a6 Author: Cole Lyman Date: Fri Sep 22 15:04:43 2023 -0600 Update README.md with new instructions on how to work with this repo commit 4f996f1523f52696bade121e96c2307d5ac64fab Author: Cole Lyman Date: Fri Sep 22 14:10:16 2023 -0600 Add README.md, __init__.py, and .github files/directory After the history rewrite these files were removed (because they were present in other repos) commit 3058e7d634ba559b80699bfdcb57c3158d5bd305 Author: Cole Lyman Date: Wed Sep 20 14:47:23 2023 -0600 Add proper link to CRISPResso website when on CLI commit 69c3e736c9dd34f1e330debe9bfe5cdbe0789a83 Merge: c5a678c 47a2625 Author: Samuel Nichols Date: Wed Sep 20 10:47:05 2023 -0600 Merge commit '47a2625280243f3da4a890b2c9f9d56ab404b0ca' into obscure_variables commit 47a2625280243f3da4a890b2c9f9d56ab404b0ca Author: Samuel Nichols Date: Wed Sep 20 09:31:17 2023 -0600 Guardrails fix (#13) commit 2cf6aedecf5d52b05f86a191577a07a5eae1af91 Author: Samuel Nichols Date: Mon Sep 18 15:09:11 2023 -0600 Origin/master (#12) * Remove gtag code * Update path to favicon * Replace deprecated padding utility classes in report * Move Plotly JS import to reports templates and out of layout.html * Update reports to use is_default_user to obscure variables --------- Co-authored-by: Cole Lyman commit 4d7ff013ab19913bf63e1da5d6ebe5ce4a998af2 Author: Samuel Nichols Date: Wed Aug 16 14:29:43 2023 -0600 Guardrails (#11) * Display guardrails * Check that guardrails dict exists * Add guardrails to report_data if it exists commit 6829bcb104bdf022ee5db273d8d094a4524a9d63 Author: Samuel Nichols Date: Tue Aug 15 14:56:50 2023 -0600 Guardrails (#10) * Display guardrails * Check that guardrails dict exists commit c5a678c5277448bff5dffb4b1cb4e17d601a7d51 Author: Samuel Nichols Date: Mon Aug 14 14:35:27 2023 -0600 Reports, add reports to packages, colors, ordered pandas sort (#28) * Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 461ca93..f41627e * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from f41627e..c9a09ec git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: c9a09ecd3daf015009a39cce80e5183cdc0eaf1c * Add CRISPRessoReports to packages * Colors only with pro commit c9a09ecd3daf015009a39cce80e5183cdc0eaf1c Merge: 69cb5e2 4adf34f Author: Samuel Nichols Date: Mon Aug 7 13:08:51 2023 -0600 Merge pull request #9 from edilytics/origin/master Origin/master commit 4adf34f59cd902ecf3ad5cbd1d7703c5401ca5b3 Author: Samuel Nichols Date: Mon Aug 7 13:05:35 2023 -0600 Update sizing on graphs commit 3f31714d83aba60f19d3d33c6add8fed174d8e2e Merge: 6d31590 69cb5e2 Author: Samuel Nichols Date: Thu Aug 3 13:14:43 2023 -0600 Merge commit 'b8c11d51d65ab8e0cbe0a37cfd00389aa84edbf8' as 'CRISPRessoWEB/CRISPRessoReports' commit 69cb5e235bd62abd34c779a9701ae77b77b319a7 Merge: ab74bc6 d66ed48 Author: Samuel Nichols Date: Thu Aug 3 13:09:21 2023 -0600 Merge pull request #8 from edilytics/web_report_refactor Cup script and if htmls statement commit d66ed48bfa140112abb111ad734145b3e96c6ec7 Author: Samuel Nichols Date: Thu Aug 3 13:01:28 2023 -0600 Cup script and if htmls statement commit ab74bc62ba6213b1537b540ef3311fba6e24980b Merge: 4c1c03b f41627e Author: Samuel Nichols Date: Thu Jul 27 13:07:46 2023 -0600 Merge pull request #7 from edilytics/fig_name_fix Fix 10f and 10g not showing up error, bootstrap 5 changes, and githubActions. commit f41627e83e2c162663c5b9c826ab1f706d5e837f Merge: 294c8ec 8dc60cd Author: Samuel Nichols Date: Thu Jul 27 12:25:33 2023 -0600 Merge remote-tracking branch 'origin/githubActions' into fig_name_fix commit 294c8ec82ee21c3be86b1c83bdce33c5bf798e5c Merge: bbb75d0 74f1450 Author: Samuel Nichols Date: Thu Jul 27 12:24:50 2023 -0600 Merge remote-tracking branch 'origin/Reports_refactor' into fig_name_fix commit bbb75d05f77015611aac9fe8e0501c2d4ebb280a Author: Samuel Nichols Date: Wed Jul 26 13:24:16 2023 -0600 Fix 10f and 10g not showing up error commit 8dc60cd2d96372ea278eaed6992fd63614356300 Author: Samuel Nichols Date: Tue Mar 21 14:14:40 2023 -0600 Pylint fixes: unused variables commit 4efa1d5243b1bc8f0165c4d8718ebd71466bd590 Author: Samuel Nichols Date: Tue Mar 21 13:07:35 2023 -0600 Dangerous defaults fix commit 749d244d3c3ffb5799fd175e12395e46eaada927 Author: Samuel Nichols Date: Tue Mar 21 12:31:48 2023 -0600 Pylint fixes commit 4c1c03b2688344f327cd3bd0df8a8c4a20b56317 Merge: 461ca93 a2a1b41 Author: Samuel Nichols Date: Mon Mar 6 10:00:39 2023 -0700 Merge pull request #6 from edilytics/print_styles Print styles commit a2a1b4149d93ec303beacb2dafd4a6782251d5f5 Author: Samuel Nichols Date: Mon Feb 27 14:24:00 2023 -0700 Remove borders when printing commit 78f3680bea3da1ee0cacd4e116530ebba76d63b1 Author: Samuel Nichols Date: Fri Feb 24 14:11:00 2023 -0700 Fix div issue, breakinpage at all points commit 7922efa4f2c5c46a1688d675aed420db4f8d9d85 Author: Samuel Nichols Date: Fri Feb 24 11:16:15 2023 -0700 Debugging for error commit 5d7576b8dca7ede0b6ad4469e3a3e57b985c62ce Author: Samuel Nichols Date: Fri Feb 17 13:15:44 2023 -0700 Spacing fix for empty page problem commit 785816bb266b6e6a13ac62a3d46ac442b7c032f4 Author: Samuel Nichols Date: Thu Feb 16 10:32:20 2023 -0700 Restore block statement commit fbdece1396f21843b07378c085232ae8b6d795fd Author: Samuel Nichols Date: Tue Feb 14 14:52:40 2023 -0700 Remove some page breaks commit 03151eaadd30f24fe921b573d550b02ba04597ed Author: Samuel Nichols Date: Tue Feb 14 13:31:24 2023 -0700 Increase the size of the center column when printing commit 4224800c16a9803de0a3bd93bb8bd776386c8f47 Author: Samuel Nichols Date: Tue Feb 14 12:58:05 2023 -0700 Working in docker commit 6d31590f66631f65658e72bf86e340b49c341f37 Merge: 85038b8 938f86d Author: Samuel Nichols Date: Tue Feb 14 10:52:52 2023 -0700 Switch reports branch commit 8d4ce1ffb445c1c116c6641f9a5aeb77e8ce2d50 Merge: e80a139 13f7ae2 Author: Samuel Nichols Date: Tue Feb 14 10:52:52 2023 -0700 Switch reports branch commit 938f86dfff7254fe53c9189c35460079931239e1 Author: Samuel Nichols Date: Tue Feb 14 10:51:45 2023 -0700 Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 1efad70..13f7ae2 13f7ae2 Fix command used and parameters elements. Increase print width and height to 100% dcf7391 Adding styling for print-only and screen only REVERT: 1efad70 Replace tabs with spaces and reindent template files REVERT: e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle REVERT: df896b0 Resize images and fix filepath REVERT: 2f70855 Add spacing around body and footer tags REVERT: 5cd6d27 Final style fixes, color circles for style files REVERT: f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' REVERT: 321815d Removing CRISPRessoReport files REVERT: 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes REVERT: 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 REVERT: 04558fb Remove extra files REVERT: 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix REVERT: 980fdc4 Styling and bootstrap changes REVERT: 9d40474 Centering issue and submit button fix REVERT: aa5071c Subtree working REVERT: db30843 Jinja choice loader REVERT: 3b67ac0 Path correction REVERT: 3aaca48 Bootstrap 5 and partials changes REVERT: 8f5d8a1 Layout.html for C2WEB and CLI REVERT: 290d829 Fix error when rendering multi reports REVERT: 546397a summaries partials and html updates REVERT: 858a751 fig_reports and replacement REVERT: 073f1fe Added a few changes from the selenium-tests branch on C2Web REVERT: 1061ebb Update indentation in report.html and extract log params into partial REVERT: c3781e9 Update path to template directory to include `CRISPRessoReports` REVERT: 84e0969 Use the `render_template` function for each report REVERT: ee721b3 Add function to render template partials without using Flask REVERT: 08fcd4e Web updates refactoring done REVERT: 99c8e22 Adding files REVERT: ef333f0 Removing reports found in subtree REVERT: 1bae0df Commit before adding subtree REVERT: 1fbb427 Add server file to render js REVERT: d1d6fdf Move styling to main.css file REVERT: 1241569 Jinja partials for all submissions REVERT: 0534637 New submission.js template file REVERT: c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial REVERT: ecd03f6 Working file upload in partial. REVERT: ce5d20f Working, missing custom label REVERT: 6ba73e7 Bootstrap 5 changes REVERT: e05d146 Layout and report update REVERT: 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion REVERT: ea44128 Move where the style files are stored in Docker REVERT: 7f03e98 Implement creating styles from the admin panel REVERT: 9b27a2e Rename style_file references to style REVERT: a233d10 Add some default styles and rename the default to "Original" REVERT: 43a8d29 Remove style file card from admin index page REVERT: 1a8f332 Refactor saving style files when there is no name specified REVERT: 64a7b1c Implement color pickers in style admin view REVERT: 17c93c1 Succesfully implemented selecting default style REVERT: fd79cdd Restyle the colors in the admin view REVERT: 3cd94b8 Fix error when the default style can't be read from the database REVERT: 5e626bd Refactor `style_handler` to read the style from the database REVERT: 0f66d4a Refactor styles to be part of the database instead of files REVERT: 6c7d3c8 Move style folder inside of server folder REVERT: 9f71f21 Add margins around style file elements REVERT: 2a28549 Restyle the color pickers REVERT: 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files REVERT: dc4f2c7 Style dropdown - allow save json only for admin REVERT: 15e7483 Style file check REVERT: 7bd0e91 Remove style from Compare REVERT: 0ab45f5 Colors function refactored and working for all types REVERT: 2e24f8b Adding styling REVERT: d6621f1 Debuging REVERT: ed00c82 Merge with master REVERT: 5150f9b Adding style_files to partial REVERT: 957a9ca Add style files to pooled and wgs REVERT: 66dc2d3 Changes to pooled and wgs, reset Dockerfile REVERT: fa6b1cf Updated Docker file and style_files.html REVERT: ee0fcfc Optional save file REVERT: 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder REVERT: 0f26e2c Working style FileAdmin, access button, and further partial refactoring REVERT: b3b70bd Rough framework for style admin page REVERT: e4731d7 Style menu completed REVERT: 1bb37bc New style menu with tabs REVERT: 58f7e56 Tabs for different style options REVERT: 3de893d Compare (#34) REVERT: e66bef1 Update AWS EB instructions.docx REVERT: 658a218 Fix bug when trying to send recovery password with bad email creds REVERT: ee32e36 Adding color-picker partial to wgs and pooled REVERT: 34ea688 Fix for responsiveness on cup and title REVERT: f0c4d07 Adding color routes to other versions REVERT: 110fe14 Color picker input added to cmd_to_run REVERT: e732478 Names for color fields REVERT: 2934631 Jinja partial for color picker and pip install in dockerfile REVERT: 48bbf9c Cup animation (#33) REVERT: 2905248 Selenium tests (#31) REVERT: 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides REVERT: 570e42a Don't remove commas from amplicons or guides REVERT: 0d70425 Add smallGenome.fa REVERT: fc33197 Writing text for pooled REVERT: dccfcb3 Files for testing REVERT: 4cea67c Changes for WGS selenium tests. All tests functional. REVERT: ff05713 Changes for WGS selenium test file loading REVERT: 495a98d Changes for pooled testing REVERT: 0ad86a5 Merge pull request #30 from edilytics/pooled-upload-fix REVERT: 127eb8f PopulatePooled error REVERT: 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests REVERT: 7847687 Add link to CRISPRessoWGS from profile page and change header REVERT: 666f73b Remove example block from CRISPRessoWGS submission page REVERT: 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled REVERT: 8d979a4 Fix bug where files_to_delete was being replaced and standardize append REVERT: 09e55fc Changes to make interleaved and pooled tests possible REVERT: f89eca8 Changes necessary for selenium tests REVERT: 3efe4f9 Clean up test files REVERT: a696363 Merge pull request #28 from edilytics/s3 REVERT: dcef708 Remove changes for CRISPRessoCompare REVERT: e0c79cf Add demo config file for eb REVERT: 03aba8e Update AWS EB instructions.docx REVERT: a671c4e Set version to 2.6.3 REVERT: 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled REVERT: da5b15b Timezone for history is displayed in user local timezone REVERT: e11691f Update history to show time of previous run REVERT: be675fb Update pooled with s3 REVERT: 4c7d429 Add data links to pooled report REVERT: 353e88f Update admin portal landing page REVERT: 712e828 Show run type in history REVERT: 2802252 s3 and user updates REVERT: efc3ed8 S3 error catching REVERT: af68341 New S3 Validation REVERT: f7d64e0 AWS validation before submission REVERT: 8446093 Update s3 for batch and paired modes REVERT: 0e7d327 S3_Upload function imrpvoed -JF REVERT: b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 REVERT: c991d52 added s3 user database model REVERT: ab4aa54 add model for s3 bucket REVERT: 853cda9 S3_Functionality improved -JF REVERT: 2f060a6 Implemented front-end s3 browsing REVERT: e082a5f stub out viewing method REVERT: c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length REVERT: c85a93f Merge pull request #15 from edilytics/wgs-interface REVERT: 712270a Add support for CRISPRessoWGS REVERT: deaacee Extract out function to get server files in submit_routes REVERT: 151eb15 Update crispresso2_info object fields REVERT: b2a974d Bump CRISPResso verion to 2.2.4 REVERT: 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 REVERT: 7f2dc1c Stop trimming json error messages, fix #11 REVERT: d28c03b Update reporting logic to use the new CRISPResso2_info schema REVERT: 03ee46f Bump CRISPResso version in Dockerfile and download release from Github REVERT: 9151c5d Add CRISPRessoPooled report template REVERT: 25a6e37 Merge pull request #6 from edilytics/pooled-interface REVERT: b47d288 Check length of amplicons for hosted version, closes #4 REVERT: 54c28b6 Update submission file extension check REVERT: 8fcadee Add a link to CRISPRessoPooled interface in user dashboard REVERT: 7fd0283 Implement CRISPRessoPooled backend and report functionality REVERT: 4063eb3 Modify submission.js to accept .txt and .tsv files REVERT: b770323 Create template file for CRISPRessoPooled submission interface REVERT: d4f2ed0 Merge pull request #5 from edilytics/flask-modularization REVERT: 8527384 Convert some celery configurations settings to new format REVERT: 962a209 Install less and vim in Dockerfile REVERT: c693668 Read CRISPResso2_info from json files instead of pickle files REVERT: a469e08 Move LoginManager to user_routes.py REVERT: f62e67a Create db tables in init_db.py REVERT: 0d85c90 Move login_required to user_routes REVERT: 6f5e33e Reformatting of remaining __init__.py REVERT: e615c0b Extract report routes out of __init__.py REVERT: 20f2601 Extract user routes out from __init__.py REVERT: 5582612 Extract status routes out from __init__.py REVERT: 2406a10 Extract submit routes out from __init__.py REVERT: b562fcd Extract celery tasks from __init__.py REVERT: faa785d Extract views out from __init__.py REVERT: ff44576 Extract model classes out from __init__.py REVERT: 914498f Merge pull request #3 from edilytics/2to3 REVERT: 86ea7da Replace RabbitMQ with Redis REVERT: adca9fb Upgrade celery to version 5.0.5 REVERT: 244ec33 Convert from Python 2 to Python 3 REVERT: 28b4f37 Refactor Docker image to use Python 3 via micromamba REVERT: 2359800 Allow interleaved batches REVERT: 428720b Add features: Allow admin init, server discovery depth REVERT: 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon REVERT: 5062365 Update README.md REVERT: 51e02f4 Update README.md REVERT: ac4a6d5 delete other images REVERT: 4f3ad88 Update README.md REVERT: fc0de1d Update README.md REVERT: 08defa1 Update README.md REVERT: 9604983 Trycatch pickle loads REVERT: c1facd7 get rid of debug print of email REVERT: d699d4d crispresso2.0.45 REVERT: e7ff079 Update param descriptions REVERT: 1f12d59 2.0.44 REVERT: b81febe crispresso to 2.0.42 REVERT: 1a967a8 update report REVERT: 178c56d 2.4 REVERT: e41076d Job expiration REVERT: 41d1a4c check progress on setinterval REVERT: 756e488 server-side files REVERT: ad19c3c Update to crispresso 2.0.40 prime editing REVERT: e3a194a update errors and ignore email config REVERT: 2efb0bb Update README.md REVERT: 58844a6 initial commit REVERT: 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 13f7ae215990b155ecf9b7659800f3790407ebc2 commit 13f7ae215990b155ecf9b7659800f3790407ebc2 Author: Samuel Nichols Date: Mon Feb 13 09:38:21 2023 -0700 Fix command used and parameters elements. Increase print width and height to 100% commit dcf739175ff8db098d63b9b000afd0bdb59e6dff Author: Samuel Nichols Date: Mon Feb 6 16:24:45 2023 -0700 Adding styling for print-only and screen only commit 74f1450207fcc45d074ad59683654fa100bde31a Author: Samuel Nichols Date: Tue Oct 4 10:48:36 2022 -0600 Load favicon from web server commit e80a13932ef1651db312ee4d0cf31290dbc8df6f Author: Samuel Nichols Date: Tue Oct 4 10:36:52 2022 -0600 Indentation and parenthesis commit 85038b83b2a156d9217fe71cc6d1d102bcb5e2bc Merge: cba4399 15c9fdf Author: Samuel Nichols Date: Tue Oct 4 10:32:20 2022 -0600 Merge commit '15c9fdf7d12d515e45d8bfaddfa3aebd0123c293' into Reports_refactor commit 15c9fdf7d12d515e45d8bfaddfa3aebd0123c293 Author: Samuel Nichols Date: Tue Oct 4 10:32:20 2022 -0600 Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 461ca93..1efad70 1efad70 Replace tabs with spaces and reindent template files e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle df896b0 Resize images and fix filepath 2f70855 Add spacing around body and footer tags 5cd6d27 Final style fixes, color circles for style files f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' 321815d Removing CRISPRessoReport files 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 04558fb Remove extra files 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix 980fdc4 Styling and bootstrap changes 9d40474 Centering issue and submit button fix aa5071c Subtree working db30843 Jinja choice loader 3b67ac0 Path correction 3aaca48 Bootstrap 5 and partials changes 8f5d8a1 Layout.html for C2WEB and CLI 290d829 Fix error when rendering multi reports 546397a summaries partials and html updates 858a751 fig_reports and replacement 073f1fe Added a few changes from the selenium-tests branch on C2Web 1061ebb Update indentation in report.html and extract log params into partial c3781e9 Update path to template directory to include `CRISPRessoReports` 84e0969 Use the `render_template` function for each report ee721b3 Add function to render template partials without using Flask 08fcd4e Web updates refactoring done 99c8e22 Adding files ef333f0 Removing reports found in subtree 1bae0df Commit before adding subtree 1fbb427 Add server file to render js d1d6fdf Move styling to main.css file 1241569 Jinja partials for all submissions 0534637 New submission.js template file c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial ecd03f6 Working file upload in partial. ce5d20f Working, missing custom label 6ba73e7 Bootstrap 5 changes e05d146 Layout and report update 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion ea44128 Move where the style files are stored in Docker 7f03e98 Implement creating styles from the admin panel 9b27a2e Rename style_file references to style a233d10 Add some default styles and rename the default to "Original" 43a8d29 Remove style file card from admin index page 1a8f332 Refactor saving style files when there is no name specified 64a7b1c Implement color pickers in style admin view 17c93c1 Succesfully implemented selecting default style fd79cdd Restyle the colors in the admin view 3cd94b8 Fix error when the default style can't be read from the database 5e626bd Refactor `style_handler` to read the style from the database 0f66d4a Refactor styles to be part of the database instead of files 6c7d3c8 Move style folder inside of server folder 9f71f21 Add margins around style file elements 2a28549 Restyle the color pickers 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files dc4f2c7 Style dropdown - allow save json only for admin 15e7483 Style file check 7bd0e91 Remove style from Compare 0ab45f5 Colors function refactored and working for all types 2e24f8b Adding styling d6621f1 Debuging ed00c82 Merge with master 5150f9b Adding style_files to partial 957a9ca Add style files to pooled and wgs 66dc2d3 Changes to pooled and wgs, reset Dockerfile fa6b1cf Updated Docker file and style_files.html ee0fcfc Optional save file 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder 0f26e2c Working style FileAdmin, access button, and further partial refactoring b3b70bd Rough framework for style admin page e4731d7 Style menu completed 1bb37bc New style menu with tabs 58f7e56 Tabs for different style options 3de893d Compare (#34) e66bef1 Update AWS EB instructions.docx 658a218 Fix bug when trying to send recovery password with bad email creds ee32e36 Adding color-picker partial to wgs and pooled 34ea688 Fix for responsiveness on cup and title f0c4d07 Adding color routes to other versions 110fe14 Color picker input added to cmd_to_run e732478 Names for color fields 2934631 Jinja partial for color picker and pip install in dockerfile 48bbf9c Cup animation (#33) 2905248 Selenium tests (#31) 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides 570e42a Don't remove commas from amplicons or guides 0d70425 Add smallGenome.fa fc33197 Writing text for pooled dccfcb3 Files for testing 4cea67c Changes for WGS selenium tests. All tests functional. ff05713 Changes for WGS selenium test file loading 495a98d Changes for pooled testing 0ad86a5 Merge pull request #30 from edilytics/pooled-upload-fix 127eb8f PopulatePooled error 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests 7847687 Add link to CRISPRessoWGS from profile page and change header 666f73b Remove example block from CRISPRessoWGS submission page 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled 8d979a4 Fix bug where files_to_delete was being replaced and standardize append 09e55fc Changes to make interleaved and pooled tests possible f89eca8 Changes necessary for selenium tests 3efe4f9 Clean up test files a696363 Merge pull request #28 from edilytics/s3 dcef708 Remove changes for CRISPRessoCompare e0c79cf Add demo config file for eb 03aba8e Update AWS EB instructions.docx a671c4e Set version to 2.6.3 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled da5b15b Timezone for history is displayed in user local timezone e11691f Update history to show time of previous run be675fb Update pooled with s3 4c7d429 Add data links to pooled report 353e88f Update admin portal landing page 712e828 Show run type in history 2802252 s3 and user updates efc3ed8 S3 error catching af68341 New S3 Validation f7d64e0 AWS validation before submission 8446093 Update s3 for batch and paired modes 0e7d327 S3_Upload function imrpvoed -JF b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 c991d52 added s3 user database model ab4aa54 add model for s3 bucket 853cda9 S3_Functionality improved -JF 2f060a6 Implemented front-end s3 browsing e082a5f stub out viewing method c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length c85a93f Merge pull request #15 from edilytics/wgs-interface 712270a Add support for CRISPRessoWGS deaacee Extract out function to get server files in submit_routes 151eb15 Update crispresso2_info object fields b2a974d Bump CRISPResso verion to 2.2.4 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 7f2dc1c Stop trimming json error messages, fix #11 d28c03b Update reporting logic to use the new CRISPResso2_info schema 03ee46f Bump CRISPResso version in Dockerfile and download release from Github 9151c5d Add CRISPRessoPooled report template 25a6e37 Merge pull request #6 from edilytics/pooled-interface b47d288 Check length of amplicons for hosted version, closes #4 54c28b6 Update submission file extension check 8fcadee Add a link to CRISPRessoPooled interface in user dashboard 7fd0283 Implement CRISPRessoPooled backend and report functionality 4063eb3 Modify submission.js to accept .txt and .tsv files b770323 Create template file for CRISPRessoPooled submission interface d4f2ed0 Merge pull request #5 from edilytics/flask-modularization 8527384 Convert some celery configurations settings to new format 962a209 Install less and vim in Dockerfile c693668 Read CRISPResso2_info from json files instead of pickle files a469e08 Move LoginManager to user_routes.py f62e67a Create db tables in init_db.py 0d85c90 Move login_required to user_routes 6f5e33e Reformatting of remaining __init__.py e615c0b Extract report routes out of __init__.py 20f2601 Extract user routes out from __init__.py 5582612 Extract status routes out from __init__.py 2406a10 Extract submit routes out from __init__.py b562fcd Extract celery tasks from __init__.py faa785d Extract views out from __init__.py ff44576 Extract model classes out from __init__.py 914498f Merge pull request #3 from edilytics/2to3 86ea7da Replace RabbitMQ with Redis adca9fb Upgrade celery to version 5.0.5 244ec33 Convert from Python 2 to Python 3 28b4f37 Refactor Docker image to use Python 3 via micromamba 2359800 Allow interleaved batches 428720b Add features: Allow admin init, server discovery depth 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon 5062365 Update README.md 51e02f4 Update README.md ac4a6d5 delete other images 4f3ad88 Update README.md fc0de1d Update README.md 08defa1 Update README.md 9604983 Trycatch pickle loads c1facd7 get rid of debug print of email d699d4d crispresso2.0.45 e7ff079 Update param descriptions 1f12d59 2.0.44 b81febe crispresso to 2.0.42 1a967a8 update report 178c56d 2.4 e41076d Job expiration 41d1a4c check progress on setinterval 756e488 server-side files ad19c3c Update to crispresso 2.0.40 prime editing e3a194a update errors and ignore email config 2efb0bb Update README.md 58844a6 initial commit 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 1efad706b7c147dd45601111d697534fa993c007 commit 1efad706b7c147dd45601111d697534fa993c007 Author: Cole Lyman Date: Mon Oct 3 15:21:07 2022 -0600 Replace tabs with spaces and reindent template files commit e7ef285dbc92b309b674c8c3f54b88cb8e07a2a0 Author: Samuel Nichols Date: Wed Sep 28 11:31:42 2022 -0600 Fix hamburger menu and add -bs- to data-target and data-toggle commit df896b0f4630e5fd282fb3ac414c47039ce0db34 Author: Samuel Nichols Date: Wed Sep 28 10:41:37 2022 -0600 Resize images and fix filepath commit 2f70855b57de16584ee0fe85f4f30c1a0d13cf97 Author: Cole Lyman Date: Wed Sep 28 10:21:16 2022 -0600 Add spacing around body and footer tags commit 5cd6d2707e6c95a20939531f1be770be4169625b Author: Samuel Nichols Date: Fri Sep 23 11:48:32 2022 -0600 Final style fixes, color circles for style files commit cba43990501386df65d38d88e34a8d7176a9c5e6 Merge: 321815d e7de9b7 Author: Samuel Nichols Date: Tue Sep 13 11:02:06 2022 -0600 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' commit e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5 Author: Samuel Nichols Date: Tue Sep 13 11:02:06 2022 -0600 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca933c065a0f0765ce899eb537ab1ace41d43 commit f8d7d92393f1e3293990ab841c350861fc6e49d3 Merge: 321815d 461ca93 Author: Samuel Nichols Date: Tue Sep 13 11:02:06 2022 -0600 Merge commit '90392b44c4bf86da0940887f85401072f4190428' as 'CRISPRessoWEB/CRISPRessoReports' commit 321815d0c3599408afb6b1506f793275ff673f42 Author: Samuel Nichols Date: Tue Sep 13 11:01:52 2022 -0600 Removing CRISPRessoReport files commit 84174e6393ac68874698e910e1ba3d5daf098e3e Author: Samuel Nichols Date: Sat Sep 10 13:02:11 2022 -0600 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 7d9b4e5 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 7d9b4e54795c7459c917da50797c0fc5e30f8de7 commit aa5071c0b112f2a30457587408cc8f688a9390ca Author: Samuel Nichols Date: Fri Sep 9 10:34:52 2022 -0600 Subtree working commit db30843e77f9f66f96ec91b3e02058e0ddbaa111 Author: Samuel Nichols Date: Thu Sep 8 10:27:28 2022 -0600 Jinja choice loader commit 3b67ac0944c3f9c264603d4b3958d06d98d80030 Author: Samuel Nichols Date: Thu Sep 8 10:24:58 2022 -0600 Path correction commit 3aaca4875ffe6eddbcdc6e81b37eb07522c7311d Author: Samuel Nichols Date: Wed Aug 24 18:13:20 2022 -0600 Bootstrap 5 and partials changes commit 8f5d8a1136cbe16c5daf78da24655374e01b4f5d Author: Samuel Nichols Date: Mon Aug 22 10:21:10 2022 -0600 Layout.html for C2WEB and CLI commit 290d829f3b20f4186694a3895bb31200ef4a6d8f Author: Cole Lyman Date: Thu Jul 7 09:11:02 2022 -0600 Fix error when rendering multi reports commit 546397a73d59288205b1a5772cf83a8f513b7e5b Author: Samuel Nichols Date: Thu Jul 7 10:11:15 2022 -0400 summaries partials and html updates commit 858a7517d24caee7e3333594477d9aa0dad56ac4 Author: Samuel Nichols Date: Wed Jul 6 11:27:46 2022 -0400 fig_reports and replacement commit 073f1fe5c55941cc5a759c63782783bbbfb2a5a8 Author: Cole Lyman Date: Wed Jul 6 15:02:30 2022 -0600 Added a few changes from the selenium-tests branch on C2Web commit 1061ebb3de0f0c0a4773213f5f0b5aa97f28fc79 Author: Cole Lyman Date: Thu Jun 30 00:25:14 2022 -0600 Update indentation in report.html and extract log params into partial commit c3781e9c71e20f8b11bfa51d709df1f50bb95e6b Author: Cole Lyman Date: Thu Jun 30 00:05:40 2022 -0600 Update path to template directory to include `CRISPRessoReports` commit 84e0969968218bf74478a62754370937f1bc7312 Author: Cole Lyman Date: Wed Jun 29 23:42:26 2022 -0600 Use the `render_template` function for each report commit ee721b37a97155da72d5cddba01bfded58461748 Author: Cole Lyman Date: Wed Jun 29 23:19:05 2022 -0600 Add function to render template partials without using Flask commit 08fcd4eda347886e66c4d2a170eb03b00a02781a Author: Samuel Nichols Date: Tue May 17 10:08:08 2022 -0600 Web updates refactoring done commit 99c8e2243b7c37f2284189984e37a26742a21b3b Author: Samuel Nichols Date: Fri May 6 10:59:05 2022 -0600 Adding files commit 461ca933c065a0f0765ce899eb537ab1ace41d43 Author: Cole Lyman Date: Fri Sep 2 14:53:30 2022 -0600 Fix tab layout in report by removing extra closing div tags commit 9f7380b0c997c4ac634a5bf1ed6903046013f77d Author: Cole Lyman Date: Fri Sep 2 14:51:00 2022 -0600 Fix footer on web pages commit 68936690a00725a6c700a291d8091f5429de5dd6 Author: Samuel Nichols Date: Fri Sep 2 10:30:42 2022 -0600 Testing commit 06f1ee74220f6b233462ea6c3e2bca374eb633b4 Merge: 04cc7d9 c956fbd Author: Cole Lyman Date: Wed Aug 17 15:06:24 2022 -0600 Merge commit 'a92f38b66455728992f58a506edbd14bd6ef0e8b' as 'CRISPResso2/CRISPRessoReports' commit c956fbdc7964706ac90a3c228619d2b39b3511da Merge: 2f6086e a952878 Author: Cole Lyman Date: Wed Aug 17 14:51:21 2022 -0600 Merge pull request #1 from edilytics/jinja-partials Jinja partials commit a952878c37ae1e27ac5b3890c5e60633c26c6a4b Merge: e7b1888 ca3b6a1 Author: Cole Lyman Date: Wed Aug 17 14:50:42 2022 -0600 Merge pull request #2 from edilytics/selenium-tests Added a few changes from the selenium-tests branch on C2Web commit ca3b6a1ed8233e3784db4e9a4f1cc1d7dfb2d8d8 Merge: 1fd7df3 e7b1888 Author: Cole Lyman Date: Wed Aug 17 14:50:31 2022 -0600 Merge branch 'jinja-partials' into selenium-tests commit e7b1888d639b7048bc62a6363f49e993e2a11bb0 Merge: 904b05a 3673981 Author: Samuel Nichols Date: Thu Jul 7 11:55:08 2022 -0400 Merge fix, working templates for all multi reports commit 04cc7d9781325a2b53b2c8e4721db71db905e297 Merge: 95bc1f6 a511ec6 Author: Cole Lyman Date: Thu Jul 7 09:15:47 2022 -0600 Merge commit 'a511ec62614b38b8783863a593611be23367c3d6' into reports commit 367398122714d854d3fa6dd81b6ecea08c5e77ca Merge: d279edd a511ec6 Author: Cole Lyman Date: Thu Jul 7 09:15:47 2022 -0600 Merge commit 'a511ec62614b38b8783863a593611be23367c3d6' into reports commit a511ec62614b38b8783863a593611be23367c3d6 Author: Cole Lyman Date: Thu Jul 7 09:11:02 2022 -0600 Fix error when rendering multi reports commit 904b05a6fd7752e751e5817e780f2b380e13e697 Author: Samuel Nichols Date: Thu Jul 7 10:11:15 2022 -0400 summaries partials and html updates commit 1fd7df3a020dbdf08fde0d7e72ed25bc8a7abd7c Author: Cole Lyman Date: Wed Jul 6 15:02:30 2022 -0600 Added a few changes from the selenium-tests branch on C2Web commit d279eddb2168ff9edf0269080948b92c7803af7f Author: Samuel Nichols Date: Wed Jul 6 11:27:46 2022 -0400 fig_reports and replacement commit 95bc1f6dac7393959c0359ce37b0e6cd49e55805 Merge: 652ccf1 d6ca6f9 Author: Cole Lyman Date: Thu Jun 30 00:27:13 2022 -0600 Merge commit 'd6ca6f9ff3b79b9519237da0a8dcc32dd8dead73' into reports commit d6ca6f9ff3b79b9519237da0a8dcc32dd8dead73 Author: Cole Lyman Date: Thu Jun 30 00:25:14 2022 -0600 Update indentation in report.html and extract log params into partial commit 652ccf1574a62d39664f0bd6b4171f24722611e7 Merge: d6d7a54 ef18bb9 Author: Cole Lyman Date: Thu Jun 30 00:10:48 2022 -0600 Merge commit 'ef18bb9779f446d8be53aabcd966063bc186b32a' into reports commit ef18bb9779f446d8be53aabcd966063bc186b32a Author: Cole Lyman Date: Thu Jun 30 00:05:40 2022 -0600 Update path to template directory to include `CRISPRessoReports` commit d6d7a5496fd9b015ae4ac6fd8d0659ae25a82a91 Author: Cole Lyman Date: Wed Jun 29 23:43:33 2022 -0600 Add 'CRISPResso2/CRISPRessoReports/' from commit '2c25436b458d830df70aaaf19da170c5815de93f' git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-mainline: e8a796f5f451409cbafed4404dfba4b6b8a124ca git-subtree-split: 2c25436b458d830df70aaaf19da170c5815de93f commit 2c25436b458d830df70aaaf19da170c5815de93f Author: Cole Lyman Date: Wed Jun 29 23:42:26 2022 -0600 Use the `render_template` function for each report commit 293c0c937f39a19363dc567737230580b7f68fa4 Author: Cole Lyman Date: Wed Jun 29 23:19:05 2022 -0600 Add function to render template partials without using Flask commit 2f6086ef1daa56d48fd23761ea08cf3c7fa6a059 Author: Samuel Nichols Date: Tue May 17 10:08:08 2022 -0600 Web updates refactoring done commit 79d25cac42da71bb77f2bea0b565f6319830e015 Author: Samuel Nichols Date: Fri May 6 10:59:05 2022 -0600 Adding files commit 7e9e54b513ef06428ff5f809969ebdc73f4304da Merge: 4aa0bcc aa37b3f Author: McKay Date: Mon Apr 8 10:39:12 2024 -0600 Merge remote-tracking branch 'origin/master' into C2Pro commit aa37b3f3342148bf1b73cd85925230b1b2988524 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Apr 4 13:18:14 2024 -0600 Read CRISPResso_status.txt as JSON (#81) * added json access for run status * changed file to .json commit a960b071a540e1dfde9141b9ef2b9ff5d1929c3d Author: Cole Lyman Date: Fri Mar 22 15:45:57 2024 -0600 Don't delete sqlite3 binary from base Docker layer (#79) * Don't delete sqlite3 binary from base Docker layer This allows us to keep the sqlite3 binary in the dev version, but remove it in prod. * Install sqlite3 binary in dev container commit 4aa0bccb0fcd2769bbc51ff7d4ef14dc679248a6 Author: McKay Date: Fri Feb 16 10:55:57 2024 -0700 added pro check for styles view commit 24d7d9c8d44bfd1245ed52aad7f36d72a7460eb5 Author: McKay Date: Thu Feb 15 16:43:03 2024 -0700 added is_pro context processor to app. commit 85ccdfa60c3f7961cdd623d09dfe23657b3ad418 Author: McKay Date: Thu Feb 15 16:21:43 2024 -0700 merge c2pro-reports into C2Pro commit 23f6ddd803251231c6798cb5d5ea9757c1a7d554 Author: McKay Date: Thu Feb 15 14:30:39 2024 -0700 added plotly htmls to wgs render template commit 8a3ffcdc03e74179b169f66a13e4cbd32a655e9e Author: McKay Date: Thu Feb 15 14:21:12 2024 -0700 added plotly htmls to pooled plot templates commit 8bd55fd912c8f25a62ab737f250e173db9eacdaf Author: McKay Date: Thu Feb 15 14:20:37 2024 -0700 added plotly dependency, python-kaleido commit 901afaf1ee53bc841432dda5b625a86bc21ef3d8 Author: McKay Date: Tue Feb 13 19:24:18 2024 -0700 added plotly htmls to report_routes commit 21ac49ec5a02e60718a2b0bc57af6e8b929a59c7 Author: Cole Lyman Date: Tue Jan 23 13:25:22 2024 -0700 Bump version number to 2.7.1 commit 82cf9f4fb9c274af2a1d524341686a2ee20ccbfd Author: Samuel Nichols Date: Fri Jan 19 15:51:43 2024 -0700 S3 report (#80) * Add default report save to db * Add model and function to send files to s3 * link save_report_to_s3 to celery task * Remove style from Compare * Prep for cron job * Cron task, file cleanup, and view reports refactor * Add file_cleanup.py * Pass run id instead of run object to be json serializable * Modifications to view_reports * Bug fixes for saving with a name * Add default report save to db * Add model and function to send files to s3 * link save_report_to_s3 to celery task * Remove style from Compare * Prep for cron job * Cron task, file cleanup, and view reports refactor * Add file_cleanup.py * Pass run id instead of run object to be json serializable * Modifications to view_reports * Bug fixes for saving with a name * Merge fixes * Add WGS demo input files back * adding flask migrate * imported migrate functions * implement flask-migrate * removed comment * Remove automigration from init_db.py * Don't delete sqlite3 binary from base Docker layer This allows us to keep the sqlite3 binary in the dev version, but remove it in prod. * flask_migrate initialization fixed. * removed comments, used_db * fix * added s3_report migration script * Add validate_cache and refine caching method --------- Co-authored-by: Cole Lyman Co-authored-by: McKay commit cc4ff9e98b5ff79e85891238dd2c9fad4a245eef Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Wed Dec 13 11:25:23 2023 -0700 Example runs (#57) * added example_block to pooled form, not functional * - added WGS and Pooled examples * - added examples to WGS and Pooled * - added new output data for compare - added example modal to compare form - added populateCompare function * added example run to compare route * added links for report and download buttons * added view, download buttons for pooled and wgs * removed print statements * - set min_reads_to_use_region to 100 for pooled * -updated demo file download names * added example_block to pooled form, not functional * - added WGS and Pooled examples * - added examples to WGS and Pooled * - added new output data for compare - added example modal to compare form - added populateCompare function * added example run to compare route * added links for report and download buttons * added view, download buttons for pooled and wgs * removed print statements * fixed label populate for pooled demo * - added pooled demo files - fixed wgs demo files * Fix bug in displaying metadata This change fixes a bug where metadata was trying to be retrieved from `report_data`, but it wasn't being passed to it for Pooled, Compare, or WGS runs. * Remove flask-debugtoolbar install and from __init__.py * Add categories to Admin page view * Fix Pooled and WGS example zip files This includes fixing the name of the CRISPRessoPooled example zip file. Also, removing the unnecessary nested directories of the WGS zip file (`var/www/...`). * Fix loading of Compare example parameters * Refactor log_params.html to only have one `if` instead of two * Add Compare example zip with report included * Add `row` around multiReport for proper styling * Add download, link, and print buttons to reports pages * Remove extra and incorrect WGS input files --------- Co-authored-by: Cole Lyman commit 691e50afa3876c90bf4d61cca3f8b0f03c947a70 Merge: a96568c 43b15f8 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Fri Dec 8 16:31:00 2023 -0700 Merge pull request #69 from edilytics/cole/run-metadata Metadata for runs commit 43b15f86245c27805bd26541a5b00da5397ea211 Author: McKay Date: Fri Dec 8 16:26:51 2023 -0700 fixed import from merge commit 518d74a0f096774933424dbfcf5a07954695b0c0 Merge: f682fdf a96568c Author: McKay Date: Fri Dec 8 16:25:36 2023 -0700 Merge branch 'master' into cole/run-metadata commit a96568c87939c7ba96c5804104407f26c33a43f9 Author: Cole Lyman Date: Fri Dec 8 15:28:54 2023 -0700 Batch upload automatic paired end read matching (#76) * Update run_unit_tests.sh to select proper image and use entrypoint * Implement matching paired end reads function * Allow for .fa or .fq.gz paired end files to be matched * Implement interface to upload multiple paired end files * Show multiple uploaded files in UI * Add error message if there was at least one paired end file matched And if there were unmatched files present. I think this is the least surprising behavior, and still allows for single end reads to be processed. * Move the name column to be on the left in batch single end read upload * Move MAX_BATCHES check to properly handle paired end reads The previous check was only checking the number of files uploaded, but now it will truly check the number of samples in the batch. * Implement better behavior for when there are unmatched paired end files Now, the user can confirm whether the files should be run single end instead of just displaying an error. * Acheive debugging Nirvana, with stacktraces (and a console!) in the browser * Properly check for MAX_BATCHES when uploading a zip file * Refactor batch upload paired end read matching into separate file * Properly check for MAX_BATCHES when unzipping batch files * Fix expansion of Jinja variable in htmx-trigger * Move `read_and_sanitize_batch_file_into_dataframe` and use in submit_routes.py and tasks.py * Fix serving static files using Flask in debug mode * Rename batch_status_warning_message to warning_message in batch_stats template * Update verbage of unmatched paired end read files * Fix bug when returning from match_paired_reads function Apparently `[] and [] == []`, where I would have assumed it equald False. Now explicitly casting it to a `bool` will fix this bug. * Move unit tests for batch upload into separate file * Make `prep_run` a module * Imported render_template in celery, added check_again to download.py * Added edge case if user uploads r1.fq and r2.fq without other names * Renaming and clarifying functions * Fix typo in `send_static_report_data` function name --------- Co-authored-by: trevormartinj7 commit f682fdf2443480635bc87c48d3a9474c31421c4d Author: Cole Lyman Date: Fri Dec 8 12:31:08 2023 -0700 Add dropdown arrow to metadata keys commit e1830971f9d1879d2b2023898eee852c2f9bcad8 Author: Cole Lyman Date: Fri Dec 8 12:14:44 2023 -0700 Fix loading favicon.ico commit 1a5975e8882f33c39cd6b752773c1b92c1a17e8b Author: Cole Lyman Date: Fri Dec 8 12:04:32 2023 -0700 Remove duplicated log_params section and refactor metadata display This now uses a `` to display the metadata information commit dbc25fb97b6c225ab2c18df64d5c36e2cf27e012 Merge: ce83a62 aac520f Author: Cole Lyman Date: Fri Dec 8 11:16:06 2023 -0700 Merge branch 'master' into cole/run-metadata commit aac520fe39d5f9e62487b7393b32b8edadf3027e Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Dec 4 15:00:32 2023 -0700 Renamed batch upload in partial to prevent same-named batch files (#75) commit 60520467176aeb888d2a0fd9a3466d5191735b2d Author: Cole Lyman Date: Wed Nov 29 16:10:07 2023 -0700 Fix file upload limit and add Docker user (#74) * Allow for large files to be uploaded in form requests * Update the file permissions for the new Docker user * Make CRISPResso_TMP directory before trying to change the owner * Add a target to the Makefile that will push up a version to ECR commit 6c1e2c54b4f418ccc402d2faf08f3e54fb264bcf Author: Cole Lyman Date: Wed Nov 29 16:09:48 2023 -0700 Update Apache version, remove vine version pin, add matplotlib pin (#73) * Update Apache version, remove vine version pin, add matplotlib pin By removing the vine version pin the version of celery is updated. We are keeping the werkzeug version because it keeps Flask at 2.3.3. Flask Debug Toolbar does not support Flask version 3.0.0 yet, so we still need this pin. The matplotlib version pin is because matplotlib 3.8.0 or greater results in allele plots with missing text elements. * Remove S3 buckets from Compare (until implemented later) commit ce83a62de45b3f89eda0fb2f7144c345a300c217 Author: McKay Date: Wed Nov 15 16:15:30 2023 -0700 fixed context for being logged out commit 4f8f235fb18ab98b86cd1ca2a8beeafeb736d77b Author: McKay Date: Wed Nov 15 15:25:13 2023 -0700 fixed favicon commit 106b69ee0f9aa351e37a3d519dc247fef4c6fc09 Author: McKay Date: Wed Nov 15 15:23:31 2023 -0700 fixed history table styling commit a3753cce9993cf106fad71c656bb863d02c3f63e Author: McKay Date: Wed Nov 15 13:17:48 2023 -0700 fixed typo in README commit f747dc446ef2d6aca1f6812de2cb554bb600001e Author: McKay Date: Tue Nov 14 16:31:00 2023 -0700 changed to row design commit 05e5d922629ad19f9c82e275e30cc3778f258e23 Author: McKay Date: Tue Nov 14 16:17:59 2023 -0700 added metadata to report commit bed87a997f747fdd1a7b97845ef49c375a5fdf69 Author: McKay Date: Mon Nov 13 15:53:33 2023 -0700 add key form requires name commit 97b937e1193d438f6a8b5dbe726dd1ee6ccddd1c Author: McKay Date: Mon Nov 13 15:53:21 2023 -0700 fixed select 1 option update commit 69dafee3c635921d898f8a52495f04fcc19080ab Author: McKay Date: Mon Nov 13 15:52:55 2023 -0700 removed comment commit 3edf5575d62c02733e3a1cb64ceaa95a78df101d Author: McKay Date: Mon Nov 13 14:41:51 2023 -0700 key is required for RunMetadata() commit 51af1f3c6dcf1c4ab0031a1b80522aa53ba0d3d3 Author: McKay Date: Mon Nov 13 13:45:20 2023 -0700 - run delete cascades to entries - RunMetadata only deletes with no Entries commit 6eb4d920b22b0642cf045d44a237eba8ad202d5b Author: McKay Date: Mon Nov 13 13:44:14 2023 -0700 only keys from form are submitted commit 7a33dece7939fbd035aba2ba2939f90548edb211 Author: McKay Date: Fri Nov 10 14:42:32 2023 -0700 empty values aren't added to table commit 5440a181d81baf2b4810fd5497621cf2412972fa Author: McKay Date: Fri Nov 10 12:48:29 2023 -0700 fixed input field margin commit 1450351076e3863e4063e1ba4ca3e9f2533e0088 Merge: 6c7e311 09fa341 Author: McKay Date: Fri Nov 10 12:26:12 2023 -0700 Merge branch 'cole/run-metadata' of https://github.com/edilytics/C2Web into cole/run-metadata commit 09fa34156b2000f971f3d28cb566aeccd69e9e9e Author: Cole Lyman Date: Wed Nov 8 16:02:54 2023 -0700 Remove extraneous layout.html commit 6c7e311483d872a79b897e75b4224a46e56d90b8 Author: McKay Date: Wed Nov 8 15:49:21 2023 -0700 alert flashes when key already exists commit a63401b63b067881dea288078bebd947a5ed9125 Author: Cole Lyman Date: Wed Nov 8 14:26:49 2023 -0700 Update CRISPRessoReports using `run_metadata` branch commit 9ea0a7d51291e4635831130a64a8c301880d9e29 Author: Cole Lyman Date: Wed Nov 8 13:57:11 2023 -0700 Remove extraneous newlines and add space around elements The elements where space was added was the metadata trash can and the modal for "Add key" in the metadata seciton. commit d5c1fdecb07581c929d8edeb8913eab7d3e8d569 Merge: c08bcdd 9b81f6e Author: Cole Lyman Date: Wed Nov 8 13:04:15 2023 -0700 Merge branch 'master' into cole/run-metadata commit c08bcddf1ade265fdafccec42e71a2d3a464b722 Author: McKay Date: Fri Nov 3 11:22:03 2023 -0600 fixed table formatting commit 9b81f6e4989d7fadd14a36c161c5ef8b827c53b7 Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Fri Oct 13 16:29:52 2023 -0600 Added code that swaps text and color for batch upload (#72) commit 14a0d295a4cef4d2495273c7ee61849f80d1074b Author: Cole Lyman Date: Fri Oct 13 15:49:21 2023 -0600 Revert submission check to use vanilla form instead of htmx trigger (#71) commit ade74ce6fb88cff55cac0808c87880e758b20ab7 Author: Cole Lyman Date: Fri Oct 13 15:32:57 2023 -0600 Add layout to batch_status.html (#70) commit 91d149c1683b1bed439bebdbcc71a0ef96f05ad0 Merge: ca66822 4403568 Author: Cole Lyman Date: Fri Oct 13 14:46:29 2023 -0600 Merge branch 'master' into cole/run-metadata commit 4403568405c24fa8f9916fb05b15df4ef0ee9517 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Fri Oct 13 14:23:12 2023 -0600 Named amplicons (#67) * added amplicon models * added amplicon api endpoints * added amplicons array to template * added view in admin for amplicon table * added __init__ file for api * added amplicon partials to submission form * increased sequence max length in db for amplicons * Add pin to werkzeug * changed route to api/amplicon/save * changed route to api/amplicon * removed print statement * removed print statement * added padding around inputs * removed get_amplicons() * amplicon names must be unique * amplicon name required * Add saved amplicons interface to single and interleaved inputs --------- Co-authored-by: Cole Lyman commit f7ee22c9412909a3b1f38f61340ee44c8c289c24 Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Wed Oct 11 15:56:56 2023 -0600 Batch Upload Project (#63) * Implement Docker Compose for both production and development * Replace WSGI with Gunicorn for both development and production This allows us to hotreload the code when running the development version and also adds the Flask Debug Toolbar Extension which will be helpful in debugging. * Add a Makefile for commonly used Docker compose commands * Add reverse proxy to make Apache redirects work properly * Set up standalone Apache server container in Docker Compose * Add C2Web conda environment file * Make C2Web Docker image smaller (now 2.25 GB uncompressed) * Make C2Web Dockerfile multi-stage and make CRISPResso2 hot-reloadable These changes modify the C2Web Dockerfile to be multi-stage, so that there is a base stage (shared between dev and prod), a dev stage, and a prod stage. The dev stage doesn't install CRISPResso2, but binds a local copy of CRISPResso2 so that it can be hot reloaded. In the prod stage, this installs CRISPResso2 via conda. * Clean up Dockerfile and add CRISPResso2 dependencies to C2Web Docker These dependencies (plotly, seaborn-base, and matplotlib-base) are added so that they don't need to be added when CRISPResso2 is installed. * Update README.md with Docker compose details and update ignore files * Install CRISPResso2 in the build stage of Dockerfile * Removed docker-compose.prod.yml and created docker-compose.public.yml Also, updated the Makefile. Now, the default docker-compose.yml should be a suitable configuration for client facing production. The docker-compose.public.yml is a good configuration for the public facing site and the docker-compose.override.yml is a good configuration for development. * Share environment variables between web and celery and update README * Add targets for .env and clean in Makefile * Add flower support to the development version * Add support to map individual directories in Docker Compose * Fix typo in README * initial uploading and parsing of tsv files * Added batch file button and javascript and python basic parsing functions * structured more of the parsing for batch tsv files * adding comments laying out work to be done * Added dynamic parsing for both tsv and csv files and downloading of connected S3 buckets * Creating dynamic batch file based on whether or not r2 files are present in tsv/csv upload, adding to command line arguments * Added function to get s3 file by url, added parsing with pandas for xls,csv,tsv files * completed preliminary batch file processesing * Fixed Radio Buttons * Added interleaved checkbox, fixed bugs * Added endpoint to retrieve status of S3 download This adds a table to the database that tracks the S3 files that are being downloaded and adds the endpoint to retrieve their status. * Configure WSGI to run in single interpreter mode This prevents errors when using the `requests` library and also removes the strange numpy warning that has been at the top of the error logs. * Install requests library and make a test request in `get_s3_file_by_url` * Change gunicorn reload engine to poll This is because when running through x86 emulation (on M1 Mac), the inotify file system events are blocked. But reloading works with polling! * Pin SQLAlchemy version to 2.5.1 because newer versions don't work Also, this fixes the entrypoint for the dev build stage. * Make clarifications in README and fix more spelling * Ignore all __pycache__ directories, not just the one in CRISPRessoWEB * Implement posting the job submission via htmx * Implement background S3 file download S3 files can now be downloaded via a celery task and the status will be written using Redis. * Implement htmx loading for S3 batch download status * initial batch-redux commit * Swapped batch file urls with local path for better batch parsing * Batch status initally done * Added initial zip file uploading and celery task unzipping * Added initial zip upload functionality * removed plot * zip htmx implementation * Added multiple file upload, fixed bugs * Update .gitignore to recursively include __pycache__ and .pyc files * Fixed non-zip plus batch path * Added error handling, batch tutorials, updated redis and makefile * Tracking Downloaded Files proof of concept with Button * Fixed data frame replacements, added download tracker * Commit prior to removing s3file redis * Fixed bugs, added multiple file support to celery tasks and beyond * Fixed html form errors, added xlsx reading, added js guardrails * Batch Upload: fixed front end formatting, removed print statements, removed uneccesary template * Removed .DS_Store and fixed .gitignore * Removed extraneous .pyc files * Remove merge conflicts from README and make additional improvements * Remove extra imports from sumbit_routes.py and extra whitespace * updating bootstrap classes, improving file upload partial, streamlining functions * Fixed amplicon and sgrna propogation, added and improved partials * Fix bug in `get_s3_files_background` where `s3_match` was renamed to `match` * Removed InputFile DB, minor tweaks for release * Fix the sgRNA label in Batch upload S3 upload * improved dataframe reading, sharpened phrasing on buttons * altering display text for input fields * Fix back button behavior when submitting a run * Fix uneven sgRNA labels and extraneous whitespace * Fixed undefined variable bug * Fixed vanilla form submission (without htmx) by identifying fancy quotes --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Trevor Martin commit 36bcf014d40ffe9cfd455afffc5c4b249d9f6598 Author: Cole Lyman Date: Wed Oct 11 11:45:41 2023 -0600 Remove style check (for now), update Docker (#68) * Temporarily disable the check_arg_exists_in_C2 function because it is slow * Add S3 category to admin view * Comment out style dropdowns until it is released on the CLI * Bump version number * Parameterize micromamba version * Install vim and pin werkzeug version (to be compatible with flask-login) * Add non-root user to Dockerfile * Update Docker save command in Makefile * Add proper Python versioning commit ca668222388607ceaf26bd681f816421ad72e493 Author: Cole Lyman Date: Tue Oct 10 15:23:08 2023 -0600 Merge run_amplicons into branch to delete erroneous git history commit 71dac3d9fab7b7fcab0cb79ea070daa287f34633 Merge: e49aa0d 0ae4e38 Author: Samuel Nichols Date: Tue Oct 3 14:18:15 2023 -0600 Merge pull request #65 from edilytics/compare_zip_upload If interior folder doesn't exist, look for non-nested files commit 0ae4e38530129649c758fdab95558a0979dcbe2c Author: Samuel Nichols Date: Mon Oct 2 15:17:01 2023 -0600 Remove extra import and print statements commit ddb495513825b4a2c880b92f93b4ac8d0a860832 Author: Samuel Nichols Date: Mon Oct 2 15:15:13 2023 -0600 If interior folder doesn't exist, look for none nested files commit e49aa0dd1ba65401a9171706e88d29c77af2e561 Author: Cole Lyman Date: Fri Sep 15 14:30:45 2023 -0600 Add app context block to create db tables (#64) commit 9ac610bee4f55efe9e358135032a0d5083c3bf67 Author: Samuel Nichols Date: Wed Sep 6 12:13:17 2023 -0600 Reports refactor (#37) * Changes necessary for selenium tests * Changes to make interleaved and pooled tests possible * PopulatePooled error * Changes for pooled testing * Changes for WGS selenium test file loading * Changes for WGS selenium tests. All tests functional. * Files for testing * Writing text for pooled * Add smallGenome.fa * Jinja partial for color picker and pip install in dockerfile * Names for color fields * Color picker input added to cmd_to_run * Adding color routes to other versions * Fix for responsiveness on cup and title * Adding color-picker partial to wgs and pooled * Tabs for different style options * New style menu with tabs * Style menu completed * Rough framework for style admin page * Working style FileAdmin, access button, and further partial refactoring * Checkbox for custom colors that shows and hides color selectors, box on home page for style folder * Optional save file * Updated Docker file and style_files.html * Changes to pooled and wgs, reset Dockerfile * Add style files to pooled and wgs * Adding style_files to partial * Debuging * Adding styling * Colors function refactored and working for all types * Remove style from Compare * Style file check * Style dropdown - allow save json only for admin * DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files * Restyle the color pickers These changes make it so that the colors are in a row instead of a column when the screen is wide enough. They are also responsive, so when the screen is small the color pickers will return to a column. * Add margins around style file elements * Move style folder inside of server folder * Refactor styles to be part of the database instead of files This allows for greater flexibility in displaying and updating them through the web interface. * Refactor `style_handler` to read the style from the database This completes the refactor to store style parameters using the database instead of storing the styles as files. * Fix error when the default style can't be read from the database The intended and proper behavior is that no style parameter is added to the command, and that is what happens now. * Restyle the colors in the admin view Add border around the colors in the admin view as well as moving them to be more vertically centered with the name. * Succesfully implemented selecting default style * Implement color pickers in style admin view * Refactor saving style files when there is no name specified * Remove style file card from admin index page * Add some default styles and rename the default to "Original" * Rename style_file references to style This is a final clean up of refactoring how the different style properties are stored. They are now stored in the database, so it doesn't make much sense to call them style files. * Implement creating styles from the admin panel * Move where the style files are stored in Docker * Implement Docker Compose for both production and development * Replace sub, ins, del with Substitution, Insertion, Deletion * Replace WSGI with Gunicorn for both development and production This allows us to hotreload the code when running the development version and also adds the Flask Debug Toolbar Extension which will be helpful in debugging. * Add a Makefile for commonly used Docker compose commands * Add reverse proxy to make Apache redirects work properly * Layout and report update * Bootstrap 5 changes * Working, missing custom label * Working file upload in partial. * Changes to submission.js for bootstrap 5 and load file upload partial * New submission.js template file * Jinja partials for all submissions * Move styling to main.css file * Add server file to render js * Commit before adding subtree * Removing reports found in subtree * Adding files * Web updates refactoring done * Add function to render template partials without using Flask * Use the `render_template` function for each report * Update path to template directory to include `CRISPRessoReports` * Update indentation in report.html and extract log params into partial * Added a few changes from the selenium-tests branch on C2Web * fig_reports and replacement * summaries partials and html updates * Fix error when rendering multi reports * Layout.html for C2WEB and CLI * Bootstrap 5 and partials changes * Path correction * Jinja choice loader * Subtree working * Centering issue and submit button fix * Styling and bootstrap changes * Spacing changes, submission_compared fix, and submission_wgs file upload fix * Remove extra files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 7d9b4e5 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 7d9b4e54795c7459c917da50797c0fc5e30f8de7 * Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes * Removing CRISPRessoReport files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 7d9b4e5 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 7d9b4e54795c7459c917da50797c0fc5e30f8de7 * Removing unecessary logic from submission_compare * Set up standalone Apache server container in Docker Compose * Add C2Web conda environment file * Make C2Web Docker image smaller (now 2.25 GB uncompressed) * Change style to styles * Fix for updating labels * Make C2Web Dockerfile multi-stage and make CRISPResso2 hot-reloadable These changes modify the C2Web Dockerfile to be multi-stage, so that there is a base stage (shared between dev and prod), a dev stage, and a prod stage. The dev stage doesn't install CRISPResso2, but binds a local copy of CRISPResso2 so that it can be hot reloaded. In the prod stage, this installs CRISPResso2 via conda. * Clean up Dockerfile and add CRISPResso2 dependencies to C2Web Docker These dependencies (plotly, seaborn-base, and matplotlib-base) are added so that they don't need to be added when CRISPResso2 is installed. * Final style fixes, color circles for style files * Update README.md with Docker compose details and update ignore files * Install CRISPResso2 in the build stage of Dockerfile * Removed docker-compose.prod.yml and created docker-compose.public.yml Also, updated the Makefile. Now, the default docker-compose.yml should be a suitable configuration for client facing production. The docker-compose.public.yml is a good configuration for the public facing site and the docker-compose.override.yml is a good configuration for development. * Share environment variables between web and celery and update README * Add spacing around body and footer tags * Replace spacing utilities classes with Bootstrap 5 versions * Resize images and fix filepath * Hide base editing if checkbox unchecked * Base editing partial * Add padding around pegRNA radio buttons and plot window size Also, add a margin around the JSON file upload box. * Increase size of Submit buttons * Fix hamburger menu and add -bs- to data-target and data-toggle * Fix plot window size spacing in Pooled * Fix the vertical span of the input labels in WGS, Pooled and Batch * Fix Pooled layout * Make input labels in the forms the same width * Remove ALLOW_USER_STYLE_UPLOAD parameter * Remove jinja loader from report_routes * Reformat style in submit_routes.py and update docs * Convert tabs to spaces in style_selection.html * Use string interpolation instead of concatenation in submission.js * Add an authentication check before exposing server_files in submission.js * Fix indentation and convert tabs to spaces in many templates * Clean up old files and comments * Replace tabs with spaces and reindent template files * Update README.md with git alias and subtree information * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 7d9b4e5..21f63d0 21f63d0 Replace tabs with spaces and reindent template files a3bcfeb Fix hamburger menu and add -bs- to data-target and data-toggle bd0c0f1 Resize images and fix filepath 02f94fd Add spacing around body and footer tags e514cdc Final style fixes, color circles for style files a6700c0 Merge commit '90392b44c4bf86da0940887f85401072f4190428' as 'CRISPRessoWEB/CRISPRessoReports' 1343942 Removing CRISPRessoReport files 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes 9ebd458 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 7d9b4e5 04558fb Remove extra files 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix 980fdc4 Styling and bootstrap changes 9d40474 Centering issue and submit button fix 4cbbda7 Subtree working 35741c3 Jinja choice loader e30fc40 Path correction 89864b1 Bootstrap 5 and partials changes 6740185 Layout.html for C2WEB and CLI 61f5287 Fix error when rendering multi reports 240e910 summaries partials and html updates bc7535f fig_reports and replacement dd02b44 Added a few changes from the selenium-tests branch on C2Web c1e572a Update indentation in report.html and extract log params into partial c7a6974 Update path to template directory to include `CRISPRessoReports` 90108fa Use the `render_template` function for each report 125e989 Add function to render template partials without using Flask 56b1d26 Web updates refactoring done 40ac3cb Adding files ef333f0 Removing reports found in subtree 1bae0df Commit before adding subtree 1fbb427 Add server file to render js d1d6fdf Move styling to main.css file 1241569 Jinja partials for all submissions 0534637 New submission.js template file c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial ecd03f6 Working file upload in partial. ce5d20f Working, missing custom label 6ba73e7 Bootstrap 5 changes e05d146 Layout and report update 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion ea44128 Move where the style files are stored in Docker 7f03e98 Implement creating styles from the admin panel 9b27a2e Rename style_file references to style a233d10 Add some default styles and rename the default to "Original" 43a8d29 Remove style file card from admin index page 1a8f332 Refactor saving style files when there is no name specified 64a7b1c Implement color pickers in style admin view 17c93c1 Succesfully implemented selecting default style fd79cdd Restyle the colors in the admin view 3cd94b8 Fix error when the default style can't be read from the database 5e626bd Refactor `style_handler` to read the style from the database 0f66d4a Refactor styles to be part of the database instead of files 6c7d3c8 Move style folder inside of server folder 9f71f21 Add margins around style file elements 2a28549 Restyle the color pickers 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files dc4f2c7 Style dropdown - allow save json only for admin 15e7483 Style file check 7bd0e91 Remove style from Compare 0ab45f5 Colors function refactored and working for all types 2e24f8b Adding styling d6621f1 Debuging ed00c82 Merge with master 5150f9b Adding style_files to partial 957a9ca Add style files to pooled and wgs 66dc2d3 Changes to pooled and wgs, reset Dockerfile fa6b1cf Updated Docker file and style_files.html ee0fcfc Optional save file 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder 0f26e2c Working style FileAdmin, access button, and further partial refactoring b3b70bd Rough framework for style admin page e4731d7 Style menu completed 1bb37bc New style menu with tabs 58f7e56 Tabs for different style options 3de893d Compare (#34) e66bef1 Update AWS EB instructions.docx 658a218 Fix bug when trying to send recovery password with bad email creds ee32e36 Adding color-picker partial to wgs and pooled 34ea688 Fix for responsiveness on cup and title f0c4d07 Adding color routes to other versions 110fe14 Color picker input added to cmd_to_run e732478 Names for color fields 2934631 Jinja partial for color picker and pip install in dockerfile 48bbf9c Cup animation (#33) 2905248 Selenium tests (#31) 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides 570e42a Don't remove commas from amplicons or guides 0d70425 Add smallGenome.fa fc33197 Writing text for pooled dccfcb3 Files for testing 4cea67c Changes for WGS selenium tests. All tests functional. ff05713 Changes for WGS selenium test file loading 495a98d Changes for pooled testing 0ad86a5 Merge pull request #30 from edilytics/pooled-upload-fix 127eb8f PopulatePooled error 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests 7847687 Add link to CRISPRessoWGS from profile page and change header 666f73b Remove example block from CRISPRessoWGS submission page 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled 8d979a4 Fix bug where files_to_delete was being replaced and standardize append 09e55fc Changes to make interleaved and pooled tests possible f89eca8 Changes necessary for selenium tests 3efe4f9 Clean up test files a696363 Merge pull request #28 from edilytics/s3 dcef708 Remove changes for CRISPRessoCompare e0c79cf Add demo config file for eb 03aba8e Update AWS EB instructions.docx a671c4e Set version to 2.6.3 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled da5b15b Timezone for history is displayed in user local timezone e11691f Update history to show time of previous run be675fb Update pooled with s3 4c7d429 Add data links to pooled report 353e88f Update admin portal landing page 712e828 Show run type in history 2802252 s3 and user updates efc3ed8 S3 error catching af68341 New S3 Validation f7d64e0 AWS validation before submission 8446093 Update s3 for batch and paired modes 0e7d327 S3_Upload function imrpvoed -JF b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 c991d52 added s3 user database model ab4aa54 add model for s3 bucket 853cda9 S3_Functionality improved -JF 2f060a6 Implemented front-end s3 browsing e082a5f stub out viewing method c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length c85a93f Merge pull request #15 from edilytics/wgs-interface 712270a Add support for CRISPRessoWGS deaacee Extract out function to get server files in submit_routes 151eb15 Update crispresso2_info object fields b2a974d Bump CRISPResso verion to 2.2.4 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 7f2dc1c Stop trimming json error messages, fix #11 d28c03b Update reporting logic to use the new CRISPResso2_info schema 03ee46f Bump CRISPResso version in Dockerfile and download release from Github 9151c5d Add CRISPRessoPooled report template 25a6e37 Merge pull request #6 from edilytics/pooled-interface b47d288 Check length of amplicons for hosted version, closes #4 54c28b6 Update submission file extension check 8fcadee Add a link to CRISPRessoPooled interface in user dashboard 7fd0283 Implement CRISPRessoPooled backend and report functionality 4063eb3 Modify submission.js to accept .txt and .tsv files b770323 Create template file for CRISPRessoPooled submission interface d4f2ed0 Merge pull request #5 from edilytics/flask-modularization 8527384 Convert some celery configurations settings to new format 962a209 Install less and vim in Dockerfile c693668 Read CRISPResso2_info from json files instead of pickle files a469e08 Move LoginManager to user_routes.py f62e67a Create db tables in init_db.py 0d85c90 Move login_required to user_routes 6f5e33e Reformatting of remaining __init__.py e615c0b Extract report routes out of __init__.py 20f2601 Extract user routes out from __init__.py 5582612 Extract status routes out from __init__.py 2406a10 Extract submit routes out from __init__.py b562fcd Extract celery tasks from __init__.py faa785d Extract views out from __init__.py ff44576 Extract model classes out from __init__.py 914498f Merge pull request #3 from edilytics/2to3 86ea7da Replace RabbitMQ with Redis adca9fb Upgrade celery to version 5.0.5 244ec33 Convert from Python 2 to Python 3 28b4f37 Refactor Docker image to use Python 3 via micromamba 2359800 Allow interleaved batches 428720b Add features: Allow admin init, server discovery depth 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon 5062365 Update README.md 51e02f4 Update README.md ac4a6d5 delete other images 4f3ad88 Update README.md fc0de1d Update README.md 08defa1 Update README.md 9604983 Trycatch pickle loads c1facd7 get rid of debug print of email d699d4d crispresso2.0.45 e7ff079 Update param descriptions 1f12d59 2.0.44 b81febe crispresso to 2.0.42 1a967a8 update report 178c56d 2.4 e41076d Job expiration 41d1a4c check progress on setinterval 756e488 server-side files ad19c3c Update to crispresso 2.0.40 prime editing e3a194a update errors and ignore email config 2efb0bb Update README.md 58844a6 initial commit 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 21f63d05a15e1f48ec14db46fa0e5bc5f0ea5344 * Indentation and parenthesis * Semi-colon to README * Add targets for .env and clean in Makefile * Add flower support to the development version * Add support to map individual directories in Docker Compose * Fix typo in README * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 21f63d0..418d811 418d811 Fix command used and parameters elements. Increase print width and height to 100% 40330d5 Adding styling for print-only and screen only git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 418d811a007d782bef7c319358bb13702e97b1bf * Working in docker * Increase the size of the center column when printing * Remove some page breaks * Restore block statement * Spacing fix for empty page problem * Change gunicorn reload engine to poll This is because when running through x86 emulation (on M1 Mac), the inotify file system events are blocked. But reloading works with polling! * Pin SQLAlchemy version to 2.5.1 because newer versions don't work Also, this fixes the entrypoint for the dev build stage. * Make clarifications in README and fix more spelling * Add back in deleted report.html * Don't add styles to the database if they already exist * Upgrade to python 3.9 * Add report_data object to render templates * Reset subtree * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit de11bc5 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: de11bc5fb1df31c1420451be4b32c39f366b0383 * Fix region_name to be run_names * Update sizing on graphs * Create environmental variable for crispresso_version and load colors if version is above 2.2.12 * Make env variable use ARG * Add version check * Changes to work with Docker compose * Fix style selection database path * Add pe_ref_seq to WGS input and update to data-bs-toggle * Remove extra div that affected the height of prime edited ref seq * Fix gene annotation file label cut off on WGS * Move style function to after folder_id is declared * Docker compose pin versions and search C2 args for --config_files * Remove gtag code * Update CRISPResso to 2.2.14 * Specify platform in docker compose file * Move jinja_partials install from dev to build stage * Fix running when no default style is selected (or when selected style isn't available) * Add zip and unzip to prod step * Clean execution logs after run finishes This will clean the full paths from the execution logs so that those aren't exposed to the user. This also ensures that this only happens in one place in the code (instead of multiple). * Update path to favicon * Reorder conda channels, remove flask-sqlalchemy pin, and fix wtforms The latest version of wtforms has moved ColorInput from `wtforms.widgets.html5` to `wtforms.widgets`, which is reflected in this commit. * Fix S3 file upload by adding form field * Remove pyopenssl pin * Add create_styles and createUsers to app context in init_db.py * Fixed padding in S3 file upload * Removed commented out S3 upload code for CRISPRessoPooled * Add volume mount to Apache in dev version This allows any files that are edited in the static folder to hot reload. * Don't show root folder of S3 buckets because it leads to weird behavior * Fix old Bootstrap margin and padding utilities The new version of Bootstrap has replaced `ml-1` with `ms-1` and `mr-1` with `me-1`, etc. Instead of being "left" and "right", it is now "start" and "end" to account for right to left languages. * Fix the close button on the S3 modal * Fix positive quantification window in radio button label * Add another app context to init_db.py * Update IDs for jQuery examples and how radio buttons are selected Because of upgrading to Bootstrap 5, the way that labels and inputs needed to be formatted, the previous way of selecting a radio button input no longer worked. Now to select a radio button element programmatically, you can issue a `.click()` and it will be selected. * Remove extra report file * Replace deprecated padding utility classes in report * Remove duplicate id's and add a few aria labels * Add correct MIME type to submission.js file * Disable caching on submission pages to improve back button behavior * Add + to quantification window for pooled and WGS * Add S3 buckets to WGS and Compare * Remove S3 buckets from WGS Not going to implement this now, because it would be a significant effort to do it correctly. * Add note to S3 modal about large files being expensive * Don't show style selector when `--config_file` parameter isn't available * Install CRISPRessoPro in the dev environment * Fix error with app contexts and databases in unit tests * Implement handling duplicate style names when saving to db * Move Plotly JS import to reports templates and out of layout.html * Remove font installer, less, and vim dependencies --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman commit a156590508905f840ebe4ac2287dec4ff3ac09c3 Author: Cole Lyman Date: Thu Aug 24 16:01:09 2023 -0600 Cole/fix status page (#62) * Fix message when execution log can't be found * Refactor the naming of CRISPRessoCompare to CRISPRessoCompareRun * Refactor getting the report and output paths in status_routes. With this refactor, now the EXECUTION.log file will be picked up and displayed if there is an error where CRISPResso doesn't even start. * Add test case to check report_path * Refactor test cases to have different IDs and add test case for get_output_path This will also actually delete the test directories. * Modify layout of status loading page This moves the progress bar to the top of the page and brings the status mesages down below the cup. commit e106a13b44ec791fd0c7d64ea8217013e23efd08 Author: Cole Lyman Date: Thu Aug 17 19:20:09 2023 -0600 Unit tests (#61) * Add pytest dependency to Docker container * Create initial unit tests with pytest fixtures * Refactor tests to use a separate test database and add context to app * Logged in user is working and able to make requests to Flask app * Remove setting user password (because it is very slow!) * Add a few unit tests on user routes * Add script to run unit tests * Update README with information about the unit tests * Update .gitignore for __pycache__ and *.pyc commit 53e27ebf0ef0ba1f29654d0e2c3f6bd95d7772ca Author: Cole Lyman Date: Mon Jul 24 15:30:13 2023 -0600 Improved status page (#60) * Display loading percent on checkprogress page * Use htmx to get the status of the running reports * Add working progress bar! * Don't show progress bar when percent_complete is 0 * Extract functions to get report path and status file. This change allows the status to be retreived from different types of runs (Batch, Pooled, WGS). Also, change the status to update every 5 seconds instead of every 1 second. * Add functionality to retrieve and fiew the execution log on error * Implement partial steaming cup and ASCII error cup * Refactor how the partial cup is displayed and add coffee filling * Implement go back button on error * Implement constantly steaming cup with htmx * Remove extra space from left side of cups * Set the cup_index and percent_complete to -1 to show error cup and remove progress bar * Add correct check for app.config['SEND_MAIL'] because it is a string * Refactor `app.config['SEND_MAIL']`to be a bool instead of string --------- Co-authored-by: Samuel Bowcut commit c4fe8cc2667bd60028e623bf9e2fa84207d15070 Author: Cole Lyman Date: Fri Apr 28 15:01:36 2023 -0600 Add two prime editing parameters to web interface (#59) commit 28839fdc60030528e74214b8ca249ce8cecd5860 Author: Cole Lyman Date: Wed Apr 26 13:01:08 2023 -0600 Cole/bug fixes (#58) * Remove animated cup from loading screen when errors are present * Implement custom quantification window and center options * Fix for label on submission_wgs * Account for if no custom value is added in quantification window parameters * Add custom field for minimum alignment score * Add custom input for pegRNA extension quantification window size * Add custom input fields for plot window size * Add custom input for average read quality filter * Add custom input for single bp quality filtering * Add custom input to min bp quality or N * Add custom input to exclude bp from left * Add custom input to exclude bp from right * Remove duplicate CRISPResso2 header * Redirect DEFAULTUSER to CRISPResso when accessing WGS and Pooled commit bed7230a32ce34ab5d956430156015da645060e2 Merge: fe1a352 16e4f6f Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Thu Feb 9 15:33:58 2023 -0700 Merge pull request #53 from edilytics/updating-crispresso2-version Updating Crispresso2 Version to avoid numpy float error commit 16e4f6f7291cc4a45cfec3129936654c544c456d Author: trevormartinj7 Date: Thu Feb 9 15:32:52 2023 -0700 Updating Crispresso2 Version to avoid numpy float error commit fe1a352e3b18e1409d39e59f675077fcea1a0a10 Merge: efea5d9 d64a52b Author: Kendell Clement Date: Thu Feb 2 10:58:15 2023 -0500 Merge pull request #43 from edilytics/fix-crispresso-cup Fix crispresso cup commit efea5d9b4ee0fb29259f12101941b829352a3cab Merge: 15a87dc 1754051 Author: Kendell Clement Date: Thu Feb 2 10:57:37 2023 -0500 Merge pull request #36 from edilytics/admin-user Make users edittable from admin view and add ability to reset password commit 15a87dc75f60f6dece751b1427c7cb8a52225161 Author: Cole Lyman Date: Thu Jan 26 13:17:55 2023 -0700 https redirect fix (#50) * Pin the version of pyopenssl When this version is not pinned, running the Docker container will fail when `boto3` is imported because of a breaking change introduced in OpenSSL. * Add fix for proper redirection when using https When using AWS Elastic Beanstalk and having https configured with a classic load balancer redirecting in Flask will go from https to http without this fix. This fix tells Flask to respect the `X-Forwarded-Proto` headers which come from NGINX. * Update instructions to properly configure https on AWS EB Co-authored-by: Cole Lyman commit 7640e7da8a03240e2922725f52640ff49cb1bab8 Author: Kendell Clement Date: Thu Jan 5 17:54:07 2023 -0500 Update README.md commit d16f63e0cdf7e59ad8c1871ab9cbb0b73581f825 Author: Cole Lyman Date: Thu Jan 5 09:50:04 2023 -0700 Fix README on running Docker using Apple Silicon (#48) commit 19786e0fd6551882bf6a42c56b21fb2f26fea19c Merge: d19bcd0 521e750 Author: Kendell Clement Date: Thu Jan 5 08:52:24 2023 -0500 Merge pull request #47 from edilytics/cole/readme-updates Cole/readme updates commit 521e7503842437591cec40017f6281bb52bde269 Author: Cole Lyman Date: Wed Jan 4 19:54:08 2023 -0700 Add details on how to build Docker image using Apple silicon commit 33befb7755b69777a467a52a2a147e5e94e1817d Author: Cole Lyman Date: Wed Jan 4 19:46:05 2023 -0700 Add more information about debugging and error locations commit d64a52bdafb84ce9508bfd07cbd1ed3c90045aee Author: Cole Lyman Date: Tue Nov 8 22:30:35 2022 -0700 Fix the CRISPResso cup animation to look more like an espresso commit 17540510872acb79d1131394600e7ea6d683cc00 Author: Kendell Clement Date: Thu Sep 29 16:49:59 2022 -0400 Format reset link display commit 5a781754769c9dfddc844deeda119b8b2457d323 Author: Kendell Clement Date: Tue Sep 27 16:09:22 2022 -0400 Logout on password reset if logged in commit 112f70e1aa7384f4167fe64874261c1e166ca790 Author: Samuel Nichols Date: Tue Sep 27 11:07:18 2022 -0600 Remove escape char commit 3595fa0819e281fd77e0ba53040d8813a179777b Author: Samuel Nichols Date: Mon Sep 26 11:50:15 2022 -0600 ResetPassword db table up and logic working commit e90a7d2f07dbbb918fa1fe4ccaac8213513c17e4 Author: Cole Lyman Date: Mon Sep 12 16:51:58 2022 -0600 Add clipboard copy button to registration link flash message commit 45221108ce85ab95961eba5178d6299437348824 Author: Cole Lyman Date: Mon Sep 12 16:43:04 2022 -0600 Add messages for when the reset password link will expire commit 206d362795ab1f05c18f2f096d707a15e754faa7 Author: Cole Lyman Date: Wed Sep 7 13:02:06 2022 -0600 Add logic for correct grammar (referring to plural words) on admin index commit 710d01a18548cbef1f066e9c517aaf91d4ecbcb2 Author: Cole Lyman Date: Wed Sep 7 13:00:30 2022 -0600 Implement extra row action to reset a password for users in admin commit 36162b8068b1a147c8bef894307fc76d93eb3e79 Author: Cole Lyman Date: Wed Sep 7 12:59:13 2022 -0600 Extract reset password logic into standalone functions commit d19bcd086de3a8f173e46278711f0a23332d9890 Author: Kendell Clement Date: Tue Sep 6 17:53:45 2022 -0400 Update AWS EB instructions.docx commit 3154a5c6d98ab024a9e55e740b82d9b1c27b92c4 Author: Cole Lyman Date: Tue Sep 6 14:36:51 2022 -0600 Make users edittable from admin view and add ability to reset password These changes implements the ability to reset passwords (either by email or by providing a link directly) for user accounts. commit 3de893d5451cdacf053c4fe9d8f325d54d78516a Author: Samuel Nichols Date: Mon Jul 25 12:48:55 2022 -0400 Compare (#34) * Admin Compare Report button added -JF * Load filepath for compare * Passing files to sumbit routes * CRISPResso Compare working with Server side files * File upload paths working * Clean up print statements * Removing pycache * Fix bug where files_to_delete was being replaced and standardize append * Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled * Remove example block from CRISPRessoWGS submission page * Add link to CRISPRessoWGS from profile page and change header * Don't remove commas from amplicons or guides When supplying multiple amplicons (or sgRNAs) the `clean_dna_letters` (`clean_rna_letters`) function will remove the commas and make your two or more comma separated sequences into one super-sequence that is incorrect. * Create .gitignore * Rebase * files_to_delete removes directories and unzip files include run names * Add a few files to .gitignore * Fix whitespace in report_routes.py * Minor formatting fixes mainly around string formatting * Add -f to rm to ensure that the files are deleted * Add missing `
` tag to CRISPRessoCompare submission form * Change textarea elements to input text * Update indentation for CRISPRessoCompare submission template * Update cup animation steam to look more natural * Added row div to multiReport so that the report is centered I also removed an unecessary closing div tag from the end of the template. * Fix compressing the CRISPRessoCompare report results * Make the "Compare" button in previous runs an outline * Add CRISPResso2 to base submission form if logged in Co-authored-by: Jeffrey Forte Co-authored-by: Cole Lyman commit e66bef17c3619aca95f7fcc23923816bb9c2dbf8 Author: Kendell Clement Date: Thu Jul 21 22:26:03 2022 -0400 Update AWS EB instructions.docx commit 658a2185f0e53ab9d1fec7dc3a2715ce5055dc94 Author: Kendell Clement Date: Thu Jul 21 22:25:55 2022 -0400 Fix bug when trying to send recovery password with bad email creds commit 48bbf9c46c35e92986bc8d58c6405850f9d03757 Author: Samuel Nichols Date: Fri Jul 8 13:21:39 2022 -0600 Cup animation (#33) * Adding js * Animation working, wrong background color * New color for background * Update background color of loading page to original blue Co-authored-by: Cole Lyman commit 29052484ec691d853a59952cfbbc047d612babc8 Author: Samuel Nichols Date: Wed Jul 6 15:01:08 2022 -0600 Selenium tests (#31) * Changes necessary for selenium tests * Changes to make interleaved and pooled tests possible * PopulatePooled error * Changes for pooled testing * Changes for WGS selenium test file loading * Changes for WGS selenium tests. All tests functional. * Files for testing * Writing text for pooled * Add smallGenome.fa Co-authored-by: Cole Lyman commit 5641fd34aa531bac07c5094474aad4b68527bf00 Merge: 0ad86a5 570e42a Author: Kendell Clement Date: Wed May 11 21:00:09 2022 -0400 Merge pull request #32 from edilytics/multi-amplicon-guides Don't remove commas from amplicons or guides commit 570e42a45168b52809dbbc47d48a6a292ac4729c Author: Cole Lyman Date: Wed May 11 09:18:42 2022 -0600 Don't remove commas from amplicons or guides When supplying multiple amplicons (or sgRNAs) the `clean_dna_letters` (`clean_rna_letters`) function will remove the commas and make your two or more comma separated sequences into one super-sequence that is incorrect. commit 0ad86a542588338a9474022ef9f8f1aee58135ec Merge: 3efe4f9 7847687 Author: Kendell Clement Date: Thu Apr 28 15:42:44 2022 -0400 Merge pull request #30 from edilytics/pooled-upload-fix Pooled upload fix commit 7847687657d2e7c2eea9205515c2384170a36c37 Author: Cole Lyman Date: Tue Apr 26 13:00:20 2022 -0600 Add link to CRISPRessoWGS from profile page and change header commit 666f73b637cbe2c1937452ef52069013e87c167f Author: Cole Lyman Date: Tue Apr 26 12:59:48 2022 -0600 Remove example block from CRISPRessoWGS submission page commit 27fcc13d23c8ed684bfac0c1f1373d87835ac636 Author: Cole Lyman Date: Tue Apr 26 10:46:59 2022 -0600 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled commit 8d979a42c9d1afe4425b9cd41240dc990292499f Author: Cole Lyman Date: Tue Apr 26 10:42:19 2022 -0600 Fix bug where files_to_delete was being replaced and standardize append commit 3efe4f9ce9f2712f9d2d014be3615ead61a0caac Author: Kendell Clement Date: Fri Apr 15 00:39:16 2022 -0400 Clean up test files commit a6963630957f63eefde088ba327b7bd3328f7c0f Merge: c5b6d13 dcef708 Author: Kendell Clement Date: Fri Apr 15 00:19:42 2022 -0400 Merge pull request #28 from edilytics/s3 S3 commit dcef708b1a77c87aa353d427efd7a6f5999f7f19 Author: Kendell Clement Date: Fri Apr 15 00:18:55 2022 -0400 Remove changes for CRISPRessoCompare commit e0c79cfe3d824c6ea1c960ad55a5e9b1cf8d4d84 Author: Kendell Clement Date: Fri Apr 8 03:52:53 2022 -0400 Add demo config file for eb commit 03aba8e8c56307fc0300078545fcfd44c7119ac9 Author: Kendell Clement Date: Fri Apr 8 03:52:04 2022 -0400 Update AWS EB instructions.docx commit a671c4edcbca6d9bd52e18c5ebc1145836b661c1 Author: Kendell Clement Date: Fri Apr 8 00:52:36 2022 -0400 Set version to 2.6.3 commit 3bb3a8d244267c66de4461fdea645488cb0fb002 Author: Kendell Clement Date: Sun Mar 27 01:44:14 2022 -0400 Pull out s3 javascript for use in crispresso and crispressopooled commit da5b15bd85f9addac7390a84b83c67caf999d714 Author: Kendell Clement Date: Sun Mar 27 01:21:23 2022 -0400 Timezone for history is displayed in user local timezone Ok. This turned out to be a little more work... but it works (hackety hack hack) commit e11691ffaff0759f254a2819d77db837fd1d20dd Author: Kendell Clement Date: Sun Mar 27 00:18:53 2022 -0400 Update history to show time of previous run commit be675fb33172fd30ae27bf682a256b9b739ae3f7 Author: Kendell Clement Date: Fri Mar 25 16:14:04 2022 -0400 Update pooled with s3 commit 4c7d42950dd1e8faf1ad62a5aa2ca9f45e7f4bf9 Author: Kendell Clement Date: Fri Mar 25 16:13:25 2022 -0400 Add data links to pooled report commit 353e88f7c90bd3a4507cc7f929bfd39e7071eef8 Author: Kendell Clement Date: Fri Mar 25 16:13:08 2022 -0400 Update admin portal landing page Update admin portal to bootstrap4 commit 712e82837d1770e80e2e0c8568636ba99f89937e Author: Kendell Clement Date: Fri Mar 25 16:12:42 2022 -0400 Show run type in history commit 2802252806a82859d5a1784bc7d960a0ded6e99c Author: Kendell Clement Date: Thu Mar 24 23:47:47 2022 -0400 s3 and user updates Add action to give all users access to s3 bucket Add ability to edit users commit efc3ed8493a3d31ccada73b7434963304063c5bb Author: Kendell Clement Date: Thu Mar 24 17:03:56 2022 -0400 S3 error catching - Catch and display errors for users who aren't logged in. - Download s3 file to temp file commit af683411e915044766f913a3cffc2a8733f94e08 Author: Kendell Clement Date: Thu Mar 24 17:02:54 2022 -0400 New S3 Validation - New S3 validation via wtforms - Get rid of 'write access' column for s3 users - take that suckas! commit f7d64e000ed65ec04404c6590fe015f8ee48655b Author: Kendell Clement Date: Wed Mar 23 01:22:34 2022 -0400 AWS validation before submission commit 84460936974cfbcdf63cbdc24c512b7556a04d7c Author: Kendell Clement Date: Wed Mar 23 00:25:09 2022 -0400 Update s3 for batch and paired modes Also added spinner for ajax query commit 0e7d32719704b7b8d4c744f74f02e2d86c185be4 Author: Jeffrey Forte Date: Fri Mar 18 23:20:41 2022 -0500 S3_Upload function imrpvoed -JF commit b48e0dc22512da300844a7f5edafcff2b47bf217 Merge: c991d52 ab4aa54 Author: Jeffrey Forte Date: Wed Mar 2 19:02:29 2022 -0600 Merge branch 's3' of https://github.com/edilytics/C2Web into s3 commit c991d526240f944450f112ce39e53e9a353214f9 Author: Jeffrey Forte Date: Wed Mar 2 18:57:13 2022 -0600 added s3 user database model commit ab4aa54ab870442c60ca1d7694b02206f0d6dad2 Author: Kendell Clement Date: Tue Mar 1 20:44:03 2022 -0500 add model for s3 bucket commit 853cda96099c21aa647e98a7d70f30f7413a89d1 Author: Jeffrey Forte Date: Tue Mar 1 19:13:00 2022 -0600 S3_Functionality improved -JF commit 2f060a6b9fed0088c9c5e9a0c96437ec58f35fa1 Author: Kendell Clement Date: Fri Feb 4 02:42:33 2022 -0500 Implemented front-end s3 browsing commit e082a5ff5270396a3c1910e608a17971003bbaed Author: Kendell Clement Date: Thu Feb 3 23:05:40 2022 -0500 stub out viewing method commit c5b6d13f09f68b5f9ee2fd17419a23f109aec171 Merge: c85a93f b47d288 Author: Kendell Clement Date: Fri Sep 10 22:25:41 2021 -0400 Merge pull request #7 from edilytics/check-amplicon-length Check length of amplicons for hosted version, closes #4 commit c85a93f9ca73b6efe76f851320ab2a44943ae870 Merge: 58ae313 712270a Author: Kendell Clement Date: Fri Sep 10 21:50:38 2021 -0400 Merge pull request #15 from edilytics/wgs-interface WGS interface commit 712270a3af8cae040637a2f2a6546e60412b4ec4 Author: Cole Lyman Date: Fri Sep 3 13:15:24 2021 -0400 Add support for CRISPRessoWGS commit deaaceedc25e92db78fbb5b9a7caa925c72b790b Author: Cole Lyman Date: Fri Aug 20 13:49:15 2021 -0400 Extract out function to get server files in submit_routes commit 151eb158530ae10e421aa42690a9edcd1b3fc363 Author: Cole Lyman Date: Fri Sep 10 14:16:47 2021 -0400 Update crispresso2_info object fields Most of these changes are for CRISPRessoBatch and CRISPRessoPooled commit b2a974d7d76c57d8016515efb0725fe666ad06e1 Author: Cole Lyman Date: Fri Sep 10 14:15:48 2021 -0400 Bump CRISPResso verion to 2.2.4 commit 58ae313d7b3ab73f74a7e57c6ae1502d87ba3e82 Merge: 7f2dc1c d28c03b Author: Kendell Clement Date: Thu Sep 2 12:02:12 2021 -0400 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 Update to crispresso 2.2.2 commit 7f2dc1caededaf2345ce8b6bcb1742f4a7b173a2 Author: Kendell Clement Date: Thu Sep 2 11:46:38 2021 -0400 Stop trimming json error messages, fix #11 commit d28c03b7f48d5fd50f81bc7856662b200544bfce Author: Cole Lyman Date: Thu Sep 2 11:21:08 2021 -0400 Update reporting logic to use the new CRISPResso2_info schema commit 03ee46f66f54fc9e1cd7e60de62c8f12f657a4d0 Author: Cole Lyman Date: Fri Aug 27 14:28:17 2021 -0400 Bump CRISPResso version in Dockerfile and download release from Github commit 9151c5d495c30e0736c1f377dbf5c6f8aa658353 Author: Cole Lyman Date: Fri Aug 27 14:10:25 2021 -0400 Add CRISPRessoPooled report template commit 25a6e37035886ffc672a1bb15faf5875905c3755 Merge: d4f2ed0 54c28b6 Author: Cole Lyman Date: Fri Aug 20 12:59:49 2021 -0400 Merge pull request #6 from edilytics/pooled-interface CRISPRessoPooled interface commit b47d28840b0f23955c50bd3c7f32515cca81d800 Author: Cole Lyman Date: Thu Jul 29 13:19:40 2021 -0400 Check length of amplicons for hosted version, closes #4 commit 54c28b6f5898236cdfab1ea80120a68cf93fdde8 Author: Cole Lyman Date: Thu Jul 29 11:47:39 2021 -0400 Update submission file extension check Now when a file is submitted that does not match the file extensions of the `accept` field, the alert will be shown with the proper file extensions for that field. commit 8fcadee364c92f0df0720dae49836c8005230269 Author: Cole Lyman Date: Fri Jul 16 21:18:07 2021 -0600 Add a link to CRISPRessoPooled interface in user dashboard commit 7fd0283a422965912d2d033ec2039dc8a16a3c0d Author: Cole Lyman Date: Fri Jul 16 21:17:36 2021 -0600 Implement CRISPRessoPooled backend and report functionality commit 4063eb3fd79e92dcda9b3474659684c885797599 Author: Cole Lyman Date: Fri Jul 16 21:16:31 2021 -0600 Modify submission.js to accept .txt and .tsv files commit b770323c55a6953e49fe1cefee332cf05e796383 Author: Cole Lyman Date: Thu Jul 8 16:13:38 2021 -0600 Create template file for CRISPRessoPooled submission interface commit d4f2ed09ecbd1ab86f2f57946b5e8fef5a509f6e Merge: 914498f 8527384 Author: Kendell Clement Date: Mon Jul 12 09:12:13 2021 -0400 Merge pull request #5 from edilytics/flask-modularization Flask modularization commit 85273847ad77e1663049e6786406e46ead834777 Author: Cole Lyman Date: Fri Jun 25 16:36:53 2021 -0400 Convert some celery configurations settings to new format When I try to convert the other two settings (CELERY_BROKER_URL and CELERY_RESULT_BACKEND) things broke, so I'm not sure how that will be handled when we move to Celery 6.0.0. commit 962a20912d2c63078316d3df89651406d0de3827 Author: Cole Lyman Date: Fri Jun 25 14:06:58 2021 -0400 Install less and vim in Dockerfile commit c6936684f902435aee443c3cad4d7131a3f05a36 Author: Cole Lyman Date: Fri Jun 25 14:06:35 2021 -0400 Read CRISPResso2_info from json files instead of pickle files commit a469e082ee87f6cb3ce0122eead6b06734254a63 Author: Cole Lyman Date: Thu Jun 24 14:15:25 2021 -0400 Move LoginManager to user_routes.py The location of this chunk of code is important because it needs to be run when the module is initialized. commit f62e67a6b2d936212eed608155f0bbf920e6804b Author: Cole Lyman Date: Thu Jun 24 14:14:48 2021 -0400 Create db tables in init_db.py commit 0d85c908b38c549f724b1dcdf3f0a970d5201abd Author: Cole Lyman Date: Mon Jun 21 11:26:26 2021 -0400 Move login_required to user_routes commit 6f5e33e54dcff22fa22a73df56d857af8cdac21a Author: Cole Lyman Date: Sat Jun 19 00:22:52 2021 -0400 Reformatting of remaining __init__.py commit e615c0b057cbdfd589453f063a13159b3f8c8374 Author: Cole Lyman Date: Sat Jun 19 00:19:33 2021 -0400 Extract report routes out of __init__.py commit 20f260131f1b6fd5079da84ac6c1f6a016c2778c Author: Cole Lyman Date: Sat Jun 19 00:15:42 2021 -0400 Extract user routes out from __init__.py commit 5582612c47fcbf263271ee2dccc0b0b8cb792e8a Author: Cole Lyman Date: Sat Jun 19 00:09:45 2021 -0400 Extract status routes out from __init__.py commit 2406a104472c491258a815f5554a986dec558465 Author: Cole Lyman Date: Sat Jun 19 00:04:08 2021 -0400 Extract submit routes out from __init__.py commit b562fcd93225097a23fd1bd9f14630b9fe0cf930 Author: Cole Lyman Date: Sat Jun 19 00:00:09 2021 -0400 Extract celery tasks from __init__.py commit faa785da37a385000ee5c67f5542b17254e6628e Author: Cole Lyman Date: Fri Jun 18 23:52:58 2021 -0400 Extract views out from __init__.py commit ff445768bd99489b95209f36094de93410d63cb2 Author: Cole Lyman Date: Fri Jun 18 15:30:12 2021 -0400 Extract model classes out from __init__.py commit 914498fac9d60fb9526d7be6ed9ee1e8e95864d1 Merge: 2359800 86ea7da Author: Kendell Clement Date: Fri Jun 11 11:25:41 2021 -0400 Merge pull request #3 from edilytics/2to3 2to3 commit 86ea7daa2ba6e838f1edc33df2d63fe82593b458 Author: Cole Lyman Date: Thu Jun 10 15:59:13 2021 -0400 Replace RabbitMQ with Redis The RPC backend (using RabbitMQ) seems to have a bug that hasn't been fixed in 4 years https://github.com/celery/celery/issues/4084 where the states of the tasks aren't properly updated. This makes redirecting the results page difficult if not impossible. Using Redis, we do not encounter these issues. commit adca9fb2bcc48e1e4c4c61f43f02e6787c0f72db Author: Cole Lyman Date: Fri Jun 4 10:34:01 2021 -0400 Upgrade celery to version 5.0.5 This upgrade includes moving away from the AMQP backend to the RPC backend (because AMQP is now deprecated). commit 244ec33ee77fcf02323c8ef7d503f5db4908ebf5 Author: Cole Lyman Date: Thu Mar 18 11:02:37 2021 -0400 Convert from Python 2 to Python 3 commit 28b4f3767c3170983d418e459a1576d29aa40696 Author: Cole Lyman Date: Fri Mar 12 16:04:47 2021 -0500 Refactor Docker image to use Python 3 via micromamba This commit also includes some cleanup and refactoring of the Dockerfiles. This also merges the previously separate Dockerfiles. Currently this assumes that the Python 3 version of the CRISPResso2 source code is found in the root directory of the project under the name `CRISPResso2`. commit 23598000c53d2c8a01c16674bfbbc53bfd47883a Author: Kendell Clement Date: Fri May 7 21:08:32 2021 -0400 Allow interleaved batches commit 428720b6da4c85395d322419cf8c481b42fad93a Author: Kendell Clement Date: Thu Mar 25 16:16:16 2021 -0400 Add features: Allow admin init, server discovery depth commit 11df5d88f803c4d732493394b78a066f671795c6 Author: Kendell Clement Date: Wed Mar 24 20:19:16 2021 -0400 Client and server-side checks for invalid characters on sgRNA and amplicon commit 506236500d2cd1c613d6439b4aadcfbe2619b888 Author: edilytics <57826186+edilytics@users.noreply.github.com> Date: Fri Mar 12 12:21:09 2021 -0500 Update README.md commit 51e02f4968b87ec2e3ff71aa4baf53825743f9c6 Author: edilytics <57826186+edilytics@users.noreply.github.com> Date: Fri Mar 12 12:19:21 2021 -0500 Update README.md commit ac4a6d5ad03600e2f4386cb8c49f6daf64339cf0 Author: Kendell Clement Date: Fri Mar 12 11:37:04 2021 -0500 delete other images commit 4f3ad8898bcfcb5c8ead25c5ca1d0fede13d0023 Author: edilytics <57826186+edilytics@users.noreply.github.com> Date: Fri Mar 12 11:20:15 2021 -0500 Update README.md commit fc0de1dec68f4ca3ee4c9be7e063e8c12ddb6400 Author: edilytics <57826186+edilytics@users.noreply.github.com> Date: Fri Mar 12 11:16:35 2021 -0500 Update README.md commit 08defa1dba7ab85bed3d1a111b505e1b6777bcad Author: edilytics <57826186+edilytics@users.noreply.github.com> Date: Fri Mar 12 11:14:51 2021 -0500 Update README.md commit 9604983239cbbbf72920481e335dff28d0e8406a Author: Kendell Clement Date: Mon Mar 8 23:27:52 2021 -0500 Trycatch pickle loads commit c1facd7c31b0e14ef185731ab497c8ddbb215e36 Author: Kendell Clement Date: Mon Mar 8 23:05:06 2021 -0500 get rid of debug print of email commit d699d4d98ef168063b0355b96597164421db5e0d Author: Kendell Clement Date: Mon Mar 8 22:43:05 2021 -0500 crispresso2.0.45 CRISPResso 2.0.45 log report viewers commit e7ff079b107925f1c9a571160ffd435ed7919598 Author: Kendell Clement Date: Sat Dec 26 10:38:01 2020 -0500 Update param descriptions Update param descriptions in readme and usage.docx for REQUIRE_SUBMITTER_EMAIL and REQUIRE_RUN_NAME commit 1f12d5951e9f73aef63f0d332b5c372fc3420a62 Author: Kendell Clement Date: Tue Dec 22 21:52:31 2020 -0500 2.0.44 Add options for REQUIRE_SUBMITTER EMAIL and REQUIRE_RUN_NAME Also change login decorator commit b81febed93178815be8caa671cbcc1e362fb8e24 Author: Kendell Clement Date: Sun Nov 8 00:13:51 2020 -0500 crispresso to 2.0.42 commit 1a967a8a28cf9d36c6ad6f0bf6b4f649965a42ee Author: Kendell Clement Date: Sat Nov 7 02:11:09 2020 -0500 update report commit 178c56db2070e02b36ace317a9c4764815595649 Author: Kendell Clement Date: Sat Nov 7 00:31:23 2020 -0500 2.4 admin logging commit e41076dffb5ad8cea976cc5f38c58cad2da70dfe Author: Kendell Clement Date: Tue Nov 3 15:47:32 2020 -0500 Job expiration commit 41d1a4c7262f639eac2f17a1ff5f39dbee32408a Author: Kendell Clement Date: Wed Aug 5 12:22:50 2020 -0400 check progress on setinterval commit 756e48801e9bd6dba83bb7e7ed16941d2882fe5d Author: Kendell Clement Date: Sun Jul 26 01:23:13 2020 -0400 server-side files commit ad19c3c4017e4428ac0a1d2556a616dfe1335fb4 Author: Kendell Clement Date: Fri Jul 10 00:55:30 2020 -0400 Update to crispresso 2.0.40 prime editing commit e3a194ace2dd615c2105bc01c0ab835764223e41 Author: Kendell Clement Date: Wed May 6 10:20:01 2020 -0400 update errors and ignore email config commit 2efb0bbb08a7e5b667749e1984f0b789409ede49 Author: Kendell Clement Date: Fri Apr 17 23:57:17 2020 -0400 Update README.md commit 58844a6b17b8857a0d2418647afd8483dfe6b3e5 Author: Kendell Clement Date: Fri Apr 17 23:55:19 2020 -0400 initial commit commit 8ff1878d5380863de2548e0cb1deb720841856ae Author: Kendell Clement Date: Fri Apr 17 23:37:51 2020 -0400 Initial commit commit 53fecf71977f10c0d643887bf110cf7cbf044b8e Author: Sam Date: Thu Apr 4 15:35:39 2024 -0600 Squashed commit of the following: commit 6f4b0ad885e1d72413a034bf7abaaa0360a3b0c4 Author: Samuel Nichols Date: Thu Apr 4 15:18:09 2024 -0600 Batch d3 clean (#55) * imports C2Pro plots if available * added --use_matplotlib flag * added C2Pro matched api funciton signatures * added api args for plotly * added **kwargs * renamed config to custom_config, more specificity * added backend flag for plotly kaleido * added pro_installed boolean for templates, added plotly dependency to report templates * Squashed commit of the following: commit c909ea3b34e87ce637e00dac075d2bb2f8bfb954 Author: McKay Date: Thu Feb 15 15:55:23 2024 -0700 added plotly dependency for pro commit 76b3601f6a0144f100266153f1c999e0c5de65de Author: Samuel Nichols Date: Fri Jan 12 09:56:19 2024 -0700 Squashed commit of the following: commit 603f2eff9d1aa21ae95f3e134da303b8018d3a33 Author: Samuel Nichols Date: Fri Jan 12 09:48:20 2024 -0700 fix guardrials partial commit 22fc03183a8070c30dfb74d5c23575ac19019855 Author: Samuel Nichols Date: Fri Jan 12 08:54:01 2024 -0700 Add guardrail partial commit e55f6b21972b578261bc5a864ce1d653d98f9e34 Author: Samuel Nichols Date: Mon Jan 8 07:50:59 2024 -0700 Functional guardrails, needs reports update commit 6e968e9699ed59a47d88191d03768e042d8b60a4 Merge: 32b49685 e948ce10 Author: Samuel Nichols Date: Mon Dec 18 13:34:36 2023 -0700 Merge branch 'guardrails-clean-history' of https://github.com/edilytics/CRISPResso2 into guardrails-clean-history commit 32b49685da320501dad2b0ebbb57887b66220ba8 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit 4e309cf6f732565d635de3d4c5d074ada3027e2d Author: Cole Lyman Date: Mon Dec 18 10:51:55 2023 -0700 Refactor to use CRISPRessoReports module commit e648dc087c0055bc5d2fca13c64071a371dea941 Author: Cole Lyman Date: Mon Dec 18 10:51:11 2023 -0700 Add CRISPRessoReports subtree commit e948ce107ebb0d1d99010ed12e937f34b5e607d4 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05b bb13e007 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e8 d0b41483 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f909713 b913fcb4 Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f6788 5f909713 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2d e5afa478 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f0 51a943c3 Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f0 5a10d638 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd468 442a48c7 Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels… * refactored pro check for consistency * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * removed commented line * debugging lines * removed breakpoints * flattened allele data for plot * fix #377 to write info files in CRISPRessoPooled * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Introduce pandas sorting in CRISPRessoCompare (#47) * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * try, except on pio chromium flag * D3 version of batch summary * D3 batch summary and htmls * Fix pooled and WGS (WGS uses multiReport.html) * Prepare for merge with c2pro * Merge with c2pro * Print d3 json on crispresso base * CRISPResso pro changes * Reports changes * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Remove nested f strings * Update if statement to match pro * fixed handling of use_matplotlib flag * Point tests at repo branch * added flag to parsers, flag check to import * remove default False, fix help * - refactored import check to shared function - global in CORE formatting change * changed text_kwargs to kwargs * removed newline * Custom_colors change * Fix compare test * Remove d3 from core * remove plotly plots * remove guardrails and make reports changes * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: trevormartinj7 * change config to custom_config * Update integration_tests.yml point to master tests branch * Include guardrails and pro in README --------- Co-authored-by: McKay Co-authored-by: Kendell Clement Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: trevormartinj7 commit 123ffe093181e514894d74c4e9f0a5f72125157e Author: Cole Lyman Date: Thu Apr 4 13:15:41 2024 -0600 Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: trevormartinj7 commit 46f85de056ca541ae0852121f2e2897f3ab33539 Author: Samuel Nichols Date: Thu Apr 4 13:09:38 2024 -0600 Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit d75a9ba74fa1e67d2e53725cfee5203d754c09dc Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Thu Apr 4 12:25:43 2024 -0600 Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols commit ff07f62a0328ac9f551bf3e09bfc1546e5827f90 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Apr 4 11:22:48 2024 -0600 Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit fa03d16ddd10e3163c035bd1a8d18416f7da35a8 Author: Cole Lyman Date: Thu Mar 28 16:28:36 2024 -0600 Decrease Docker image size and fix PE naming and parameter behavior (#404) * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit b2cfb911171dd8550f057e67239f51a8dfff91aa Author: Cole Lyman Date: Thu Mar 28 14:41:31 2024 -0600 Fix the assignment of multiple quantification window coordinates (#38) (#403) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- * Extract out quantification window coordinate function * Refactor get_quant_window_coordinates function into two The rationale behind this is that the behavior around the cloned amplicon is quite different than if the qwc are specified directly for the amplicon. * Handling qwc: add unit tests, refactor some more and add documentation * Extract out get_relative_coordinates function This function just computes the relative indexes without doing an alignment. * Add clarifying unit tests for `get_relative_coordinates` * Refactor cloned indexes to use ref_positions instead of s1inds * fixed function for getting cloned qwc idxs * added tests for cloned qwc function * Introduce pandas sorting in CRISPRessoCompare (#47) * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- --------- * removed if check * implemented last test * changed NT to BadParameterException * changed tests, NT to BadParameter exceptions * Uncomment and correct tests for `get_relative_coordinates` * finished qwc tests * 0 is an acceptable qwc * new get_relative_coords function * added relative coordinate tests * removed unused functions * formatting * check for 0 qwc * remove test code * remove comment * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: McKay Co-authored-by: Samuel Nichols commit d171561b4a98f1f5b323b9f1b19c027ef40b059a Author: Kendell Clement Date: Thu Mar 28 14:33:57 2024 -0600 Update integration_tests.yml commit e87d92ebaf95a1147b657a4c356df137d9aae04e Author: Cole Lyman Date: Mon Mar 25 09:50:30 2024 -0600 Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master commit 42e59597039eb4580abdfb0a9beb636f404b0f7a Author: Samuel Nichols Date: Fri Mar 22 17:38:37 2024 -0600 Run tests on PR only when opening PR (#53) commit e30cec33d9f0ce867251e170ffa820a51d31724e Author: Samuel Nichols Date: Fri Mar 22 14:31:38 2024 -0600 GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request commit 9191e687b246f6758afd8349c871207130b7e1ee Author: Cole Lyman Date: Fri Mar 22 14:05:11 2024 -0600 Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators commit cee64891a6d0a803906c686d27b1c597db91345a Author: Samuel Nichols Date: Fri Mar 15 10:42:53 2024 -0600 GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman commit ad3bd4ef552cc1b2f7628617c200174ab56cf255 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Mar 14 15:11:00 2024 -0600 Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman commit 0c834b36e72dbc6674a5cfa608c398b5f411f52f Author: Cole Lyman Date: Thu Mar 14 13:57:07 2024 -0600 Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly commit 0af8fe1377ea7a548250115d3a3adc9af7881395 Author: Samuel Nichols Date: Thu Mar 14 12:28:59 2024 -0600 Introduce pandas sorting in CRISPRessoCompare (#47) commit ebd14ba9fb51cb6ff5ac02b03737c3f4ba536793 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Mon Mar 11 16:15:56 2024 -0600 Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman commit de182d268518202a07c59780c3de0da760ef7409 Author: Kendell Clement Date: Thu Mar 7 15:04:12 2024 -0700 fix #377 to write info files in CRISPRessoPooled commit 448d244c1f990046a8a09662d220387a47a87edb Author: McKay Date: Fri Feb 23 17:18:22 2024 -0700 flattened allele data for plot commit 24310ff6353bd701b8d8a7b57d49fe10f0089792 Author: McKay Date: Fri Feb 23 13:23:09 2024 -0700 removed breakpoints commit 1f165794516df82b5dd68a93e7f8e7b7ca031e98 Author: McKay Date: Thu Feb 22 14:36:31 2024 -0700 debugging lines commit a1f275f603c6dfdb923b7a4e2536291e4a649e65 Author: Samuel Nichols Date: Thu Feb 22 12:35:21 2024 -0700 GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e commit 2b163927faa4ca37a1dc8294ffd1c18dd057b62e Author: Kendell Clement Date: Tue Feb 13 10:14:56 2024 -0700 Fix #376 - CRISPRessoPooled chunking When running CRISPressoPooled with > 10000000 reads aligned, if there are reads overlapping the chunk boundary, instead of increasing the end with 500 bases, it takes the last position of chromosome as end position. This fixes the bug. commit 31c65dfbe9d9e146eb1d12a91e0bfaad76b263dd Author: Samuel Nichols Date: Mon Feb 5 13:03:29 2024 -0700 Cole/fix assigning multiple qwc (#37) * Remove extraneous whitespace * Implement the ability to assign multiple quantification window coordinates * Updating documentation for quantification window coordinates * Update to allow 0 to prevent qwc being set. Update description. --------- Co-authored-by: Cole Lyman commit 751d9ea215707e145eda1446b20328215f90d6b7 Author: Kendell Clement Date: Mon Feb 5 12:31:56 2024 -0700 Fix #374 naming of CRISPRessoPooled file name commit 3f84c7dccf62885583eb6b74825c2d63557a6aa2 Author: Kendell Clement Date: Thu Jan 25 00:13:37 2024 -0700 Revert "denoise multiprocessing" This reverts commit 3bc04213fcc730d6a7dcd51067998f7c220df135. commit 3bc04213fcc730d6a7dcd51067998f7c220df135 Author: Kendell Clement Date: Thu Jan 25 00:11:52 2024 -0700 denoise multiprocessing commit 4723e4ac81c38d0fb2da7682f4213a0586bfc881 Author: Kendell Clement Date: Wed Jan 24 23:45:59 2024 -0700 Improve multiprocessing in Batch to utilize more processors Previously, sub-crispresso runs are run on 1 process (with the rationale that parallelization of the alignment bit is the most important for batches). Now, with parallelized plotting, if more processes are provided than batches, each sub-crispresso run will be run with int(#processes/#batches) processes, thus maximizing the processor utilization for small batches. commit ba8882dff10602a55cb2d1e4317863d55a9b50b7 Author: Kendell Clement Date: Mon Jan 22 23:12:07 2024 -0700 Include all jinja templates recursively commit 08a25495ee53e293ec5d85b843f381d561a428e8 Author: Kendell Clement Date: Mon Jan 22 22:54:49 2024 -0700 Include shared partials in manifest commit 6d4f6ceb8188e77b94f5666b4106b2ffee494c54 Author: Kendell Clement Date: Mon Jan 22 15:19:27 2024 -0700 Include locations for report templates commit 0dee4aba063fde2e25f160bb9e30d84dd5a274cc Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Jan 22 12:26:46 2024 -0700 Failed batch runs (#33) * Reports, add reports to packages, colors, ordered pandas sort (#28) * Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 7d9b4e5..e18807d * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from e18807d..e9da7bf git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: e9da7bff794058e1fcdb3dc9ced79871c6a30e18 * Add CRISPRessoReports to packages * Colors only with pro * changed tuple to list for matplotlib change (#31) * wgs and batch failed runs implementation * Added failed run functionality including shared function, edits to Report, and displaying with HTML and Javascript * Merge CRISPRessoReports master into failed-batch-runs * Cole's failed-batch-runs review and changes (#36) * Fix showing link to report in CLI (only show in web) * Remove styling of jumbotron The p-5 added some weird space at the top of the container, the rounded-3 did not make a difference (because there is no background), and the h-100 also did not make a difference. * Remove extra spaces at end of the line * Remove color legend from figure caption in plot 4f * Refactor fig_reports.html partial to reduce duplication * Add opacity to custom colors on allele quilt plot * Remove extra spaces * Change default color of deletion It looked too similar to `N` and was difficult to tell apart. * Refactor plot 10c, refactor displaying of figures This commit adds flexbox to the plots, this was mainly for plots 10b and 10c because their alignment was off. * Add more plots to get the correct percentages for width * Remove setting the height of the plots * Check for failed batch info before retrieving it in `make_multi_report_from_folder` * Fix extraneous whitespace in `fig_reports` partial * Only load certain resources when on web mode * Move jQuery import to bottom of the page to improve performance * Extract out report footer buttons to partial * Fix too many closing divs in batchReport.html * Refactor failed runs to be a partial * Move the failed run JS to the partial This has the benefit of keeping the relevant code close, and also prevents the error that we were running into before where `chevronIcon` wasn't found when there were no failed runs (because the element wasn't there). * Remove `report_name` id because it probably has spaces * Move existing Plotly plots to batchReport from multiReport * Fix typo in fig 11c and resize it to 40% --------- Co-authored-by: Samuel Nichols Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05 bb13e00 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e d0b4148 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f90971 b913fcb Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f678 5f90971 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2 e5afa47 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f 51a943c Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f 5a10d63 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd46 442a48c Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. Co-authored-by: Kendell Clement commit a7ffd46822ce195d51ff4d3dba0f02fe9bc73c1e Author: Kendell Clement Date: Tue Jan 11 16:29:37 2022 -0500 Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9990790a0081b765c1f54f4a9b18db522ab4693 Author: Kendell Clement Date: Wed Jan 5 16:48:17 2022 -0500 Allow for mixed-case prime editing input, fix #187 commit 4acdcddbef3e812655b7af9def772f0bb0dc30b5 Author: Kendell Clement Date: Wed Jan 5 16:37:22 2022 -0500 Update insertion annotation for fastq_output and bam_output Fix index issue for fastq_output, add reporting of inserted bases to bam_output. Index for fastq_output is increased because the insertion happens immediately after the insertion coordinate. commit 2f84dd02787abffa6d39efbc50c82c92d1c87528 Author: kclem Date: Fri Dec 10 16:39:41 2021 -0500 Fastq_output report inserted bases when the `--fastq_output` parameter is provided, the inserted bases will be written to the output fastq file. Previously, a string like "DEL= INS=78(1) SUB= " would indicate a 1bp insertion at site 78. This update outputs strings like "DEL= INS=78(1+G) SUB= " with a plus character followed by the inserted bases. commit ba742bbfa533a672321b27b5122d7ea6658c9014 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Implement algorithmic improvements for find_indels_substitutions" This reverts commit 13414833ef6cd5b232097f6ff0325a2b8e28b214. commit 5c8e1c5de6e800c820d9ec5ffe4809737f1211de Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Add test cases for find_indels_substitutions" This reverts commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808. commit d9a072368ca991ffde52aa1ee29e17f2758ed279 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Try some additional cython optimizations" This reverts commit 38b101d57384b9f7d664ac8719c1585f5f7142a5. commit 38b101d57384b9f7d664ac8719c1585f5f7142a5 Author: Cole Lyman Date: Fri Oct 8 14:42:49 2021 -0600 Try some additional cython optimizations commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808 Author: Cole Lyman Date: Fri Oct 8 14:15:11 2021 -0600 Add test cases for find_indels_substitutions commit 13414833ef6cd5b232097f6ff0325a2b8e28b214 Author: Cole Lyman Date: Fri Oct 8 11:50:25 2021 -0600 Implement algorithmic improvements for find_indels_substitutions These changes remove the need for iterating over the alignment multiple times. commit f4b6cfc03951215c3b9019dc47beb2913a5448ab Author: Kendell Clement Date: Fri Nov 19 00:10:05 2021 -0500 Fix CRISPRessoPooled calls to deprecated pands ix commit 751fbdb4ce432f11f3fb66c5a34f7db3d1d75dc1 Author: swrosati <40470095+swrosati@users.noreply.github.com> Date: Fri Nov 12 12:33:04 2021 -0600 Update ylabel_values -> y_label_values Typo causing error in certain modes. commit 76c80713b7b8209c9399d67f718d7ade20f20d91 Author: kclem Date: Wed Nov 10 11:37:16 2021 -0500 Update dockerfile for pyparsing commit ef15caee29380f58aaae392c897fabe47587486e Author: Kendell Clement Date: Wed Nov 10 00:45:51 2021 -0500 Fix int bug for n_reads in CRISPRessoPooled commit fc1ae890712ab2dc36d5ff36c81f64a10a2c2337 Author: Kendell Clement Date: Wed Nov 10 00:34:34 2021 -0500 Spelling fix and version bump to 2.2.7 commit 08369e294d6756f727167af50f14ff75cb4864b5 Author: Kendell Clement Date: Wed Nov 10 00:30:03 2021 -0500 Add bam input and selected demuxing for CRISPRessoPooled Adds features for providing aligned bams as input to CRISPRessoPooled and for a faster demultiplexing when amplicons and genome are provided. The parameters are: --aligned_pooled_bam: Path to aligned input for CRISPRessoPooled processing. If this parameter is specified, the alignments in the given bam will be used to demultiplex reads. If this parameter is not set (default), input reads provided by --fastq_r1 (and optionally --fastq_r2) will be aligned to the reference genome using bowtie2. If the input bam is given, the corresponding reference fasta must also be given to extract reference genomic sequences via the parameter --bowtie2_index. Note that the aligned reads are paired-end seqenced, they should already be merged into 1 read (e.g. via Flash) before alignment. --demultiplex_only_at_amplicons: If set, and an amplicon file (--amplicons_file) and reference sequence (--bowtie2_index) are provided, reads overlapping alignment positions of amplicons will be demultiplexed and assigned to that amplicon. If this flag is not set, the entire genome will be demultiplexed and reads with the same start and stop coordinates as an amplicon will be assigned to that amplicon. commit 0cdcfd7c4af834f3270e61b4db4b5f6d3da10d7c Author: Kendell Clement Date: Wed Nov 10 00:24:15 2021 -0500 Remove unused var for bam_index commit 3647ace73c95a2f44e45b6b42b4f1748d3a47f4c Author: kclem Date: Wed Nov 3 09:43:28 2021 -0400 Add --min-overlap param for flash in CRISPRessoPooled commit eea442a763e0c6c41da16a15d6e11ddf6d222dc8 Author: kclem Date: Mon Nov 1 11:25:55 2021 -0400 Remove deprecated pandas .ix from WGS commit 3e6c281c931756acd26acca96249b2fa1ad1db31 Author: Cole Lyman Date: Thu Oct 21 11:10:19 2021 -0600 Add unit tests These unit tests were in the other repo, but weren't moved over for some reason. commit 50e7cb21570ddb757904728800e31c2bcbd0d060 Author: Cole Lyman Date: Thu Oct 21 11:09:38 2021 -0600 Add Makefile to run tests This makes it easy to test the integrations cases because it will automatically install the package when needed. commit de91d51bcd9b725533a45c86ae72bc27dd5d3150 Author: Cole Lyman Date: Thu Oct 21 11:28:02 2021 -0600 Replace `np.int` with `int` Apparently `np.int` has been deprecated, so in places where precision isn't important `np.int` has been replaced with `int` according to the instructions from the warning. commit cabebbefb2646967dbeee80af08ac14156b1b53c Author: kclem Date: Wed Oct 20 12:33:49 2021 -0400 Convert cols to numeric for PE commit c2bdd96651eef5af38fb7bbc11d257a827ac080d Author: kclem Date: Wed Oct 20 12:00:43 2021 -0400 Make loggers module-specific Matplolib sometimes logs verbosely to info, so this stops using the root logger commit 8196b6a81f477ddcb0e34d61dfb54085de20c1a0 Author: Kendell Clement Date: Tue Oct 19 22:00:23 2021 -0400 Fix unicode error for bam read/write commit a923a7c2ef182238bd6b8aa77289bac487b7679b Author: Kendell Clement Date: Tue Oct 19 21:59:47 2021 -0400 All sub-crispresso processes are run with 1 process commit 75205b0d41423e4f62796e9674b0edbee68a11b9 Author: Kendell Clement Date: Mon Oct 18 23:03:59 2021 -0400 Fix #161 add params for prime editing guide analysis commit ecf23ef23e5701b232bba547a6d7d4b96f085f26 Author: kclem Date: Mon Oct 18 15:02:29 2021 -0400 Add param for plotting custom plots centered at point Add parameter to plotCustomAllelePlot script: --plot_center which if set, will produce plots centered at this point instead of sgRNAs. commit 71bf32ca789dde1d44cf41b9d3b702f12336e010 Author: Kendell Clement Date: Wed Oct 13 00:48:14 2021 -0400 Update README.md commit 53197e62706e37db54f7ed50c94f38a938955e59 Author: Kendell Clement Date: Tue Oct 12 15:43:08 2021 -0400 Fix #160 plotting for cut sites in plot 5 commit 90b43eaa03c0ea0fdee62a7b244204cad50056cc Author: Kendell Clement Date: Mon Oct 4 15:45:20 2021 -0400 Remove version checks for old seaborn and numpy, fix #155 commit 5e9dedf6bd7c7b3ef998bed811760a41b5313c6e Author: Kendell Clement Date: Tue Sep 28 21:16:54 2021 -0400 Optimize get_command_output, fix gzip binary bug commit 46953c39b769a4fa43b2ef0822dd92f07c4f1b9b Author: Kendell Clement Date: Wed Sep 22 01:58:36 2021 -0400 Update expected tests for batch commit 856354aadebc8410956e724b555224a55941a618 Author: Kendell Clement Date: Wed Sep 22 01:57:59 2021 -0400 Update CRISPRessoBatchCORE.py Move parsing of batch params to after assignment of n_processes so this value is applied to sub-processes commit f7f81747207cb279fd2c0d91986c7d4e7137fde4 Author: Kendell Clement Date: Wed Sep 22 00:23:04 2021 -0400 Fix #149 bug when read is much longer than the given reference commit 798eb4322899f70e2e2a3df0fdfad9b5598d3a41 Author: Kendell Clement Date: Tue Sep 21 17:43:38 2021 -0400 Fix #150 Open fastq.gz in 'rt' mode commit fa168843e6036c6d4ec4a5d7acb097c6a1532a98 Author: Kendell Clement Date: Fri Sep 17 12:33:12 2021 -0400 Remove requirement for python 2.7 commit 80fe12efbb0ee5c321262822b237fc8220a3f2c6 Author: Kendell Clement Date: Thu Sep 9 17:41:27 2021 -0400 Print batch summaries for amplicons with only one sample Previously, batch summaries were only printed for amplicons with more than one sample to save bits. This change produces batch summaries for amplicons with only one sample, and we shift responsibility to the user for purchasing carbon offsets to compensate for the generation and storage of these redundant bits. commit 43065310bf28e6a08780222250abd0d39ff6d0c5 Author: Kendell Clement Date: Thu Sep 9 17:38:58 2021 -0400 Add parameter 'assign_ambiguous_alignements_to_first_allele' For ambiguous alignments, setting this flag will force them to be assigned to the first (as provided by the references -a first and then -e second) amplicon. Thus, no reads will be discarded as 'ambiguous' and all reads will be counted once in the analysis. Version bump to 2.2.4 commit 11eaacd3bac9ebeabad69c1d5d92f1fd1a783a17 Author: Kendell Clement Date: Fri Sep 3 22:34:59 2021 -0400 push down crispresso2_info items commit 20272e1e95305888ca744ac56af2c08554eb9f1b Author: Kendell Clement Date: Fri Sep 3 22:32:24 2021 -0400 remove mention of pickle commit 3517285473d423dc32c6e3fdbac69eecf0caa7fc Author: Kendell Clement Date: Fri Sep 3 22:30:36 2021 -0400 minor pushdown of report name in crispresso2_info commit 8129494607488bf270567d40d43e01e043699f06 Author: Cole Lyman Date: Fri Sep 3 17:31:54 2021 -0400 fixup! Move region related objects to results in crispresso2_info commit 88a82d27e840c29b95b1602ae1594bf161108979 Author: Cole Lyman Date: Fri Sep 3 16:40:40 2021 -0400 Move some objects in CRISPRessoPooled crispresso2_info objects Move the objects: - 'final_data' -> 'results'/'final_data' - 'running_mode' -> 'running_info'/'running_mode' commit b702ab3e53e88170c7517591ce7a7050ccf1c2ba Author: Cole Lyman Date: Fri Sep 3 16:36:20 2021 -0400 Move some objects in CRISPRessoBatch crispresso2_info Move the following objects: - 'completed_batch_arr' -> 'results'/'completed_batch_arr' - 'batch_names_arr' -> 'results'/'batch_names_arr' - 'batch_input_names' -> 'results'/'batch_input_names' - 'nucleotide_frequency_summary_filename' -> 'results'/'nucleotide_frequency_filename' - 'nucleotide_percentage_summary_filename' -> 'results'/'nucleotide_percentage_filename' - 'modification_frequency_summary_filename' -> 'results'/'modification_frequency_summary_filename' - 'modification_percentage_summary_filename' -> 'results'/'modification_percentage_summary_filename' commit 2416c80e952cb58fa082ead5ef719ed7eab3eeb5 Author: Cole Lyman Date: Fri Sep 3 15:39:18 2021 -0400 Move nuc_quilt related objects to 'results'/'general_plots' to crispresso2_info The following objects have been moved: - 'window_nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'window_nuc_pct_quilt_plot_names' - 'nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'nuc_pct_quilt_plot_names' - 'window_nuc_conv_plot_names' -> 'results'/'general_plots'/'window_nuc_conv_plot_names' - 'nuc_conv_plot_names' -> 'results'/'general_plots'/'nuc_conv_plot_names' commit 37e354f94980d304e1cecf11fee95e61988dd3e3 Author: Cole Lyman Date: Fri Sep 3 14:58:36 2021 -0400 Move summary objects to 'results'/'general_plots' in crispresso2_info The following objects have been moved: - 'summary_plot_names' -> 'results'/'general_plots'/'summary_plot_names' - 'summary_plot_titles' -> 'results'/'general_plots'/'summary_plot_titles' - 'summary_plot_labels' -> 'results'/'general_plots'/'summary_plot_labels' - 'summary_plot_datas' -> 'results'/'general_plots'/'summary_plot_datas' - 'summary_plot_root' -> 'results'/'general_plots'/'summary_plot_root' - 'reads_summary_plot' -> 'results'/'general_plots'/'reads_summary_plot' - 'modification_summary_plot' -> 'results'/'general_plots'/'modification_summary_plot' commit 6df988151c602602470c944ccf561cd28f2dc30c Author: Cole Lyman Date: Fri Sep 3 14:04:12 2021 -0400 Move region related objects to results in crispresso2_info This includes: - 'regions' -> 'results'/'regions' - 'all_region_names' -> 'results'/'all_region_names' - 'good_region_names' -> 'results'/'good_region_names' - 'good_region_folders' -> 'results'/'good_region_folders' commit 053691311b158a2d3620670d20983d546a9c2c7b Author: Cole Lyman Date: Fri Sep 3 13:44:46 2021 -0400 Move 'samples_quantification_summary_filename' to 'results'/'alignment_stats'/'sample_quantification_summary_filename' in crispresso2_info commit eda3d50b6fafde4ea3145980cb2010417b799d88 Author: Cole Lyman Date: Fri Sep 3 13:27:42 2021 -0400 Move 'finished_steps' to 'running_info'/'finished_steps' in crispresso2_info commit 1da829c1c3cfd2bda9aaccca0aaa28c78a8f7271 Author: blasvicco@gmail.com Date: Mon Aug 30 17:48:02 2021 +0200 BugFix: database_id referenced before declared. commit c9321cc2cfcac34e4b6d8e1aa9fde9f25382e2de Author: Kendell Clement Date: Sat Aug 21 00:15:52 2021 -0400 Version bump to 2.2.2 for some reason the last release didn't pick up the last commits. commit 3aadab69ef1e3a44f12ee277a7767648e943a787 Author: Cole Lyman Date: Fri Aug 20 12:09:49 2021 -0400 Update filterFastqs so that the numpy arrays are writable commit 7ca129d2471aeebef2ba6d2dadf36265810f1a5c Author: Kendell Clement Date: Fri Aug 20 02:00:42 2021 -0400 Update CRISPRessoPlot.py Undo attempted matplotlib deprecation warning fix commit 8e8a65c0f29b6f99662ac1d272aa7199871f5cf5 Author: Kendell Clement Date: Fri Aug 20 01:26:50 2021 -0400 Plotting bug fixes Display and plotting fixes for batch report labels, and general plots, as well as a matplotlib deprecation fix commit 3f114752c5eca1554fcebf044b604b60a3eebeae Author: Kendell Clement Date: Fri Aug 20 00:06:38 2021 -0400 add batch summary of frameshift/splicing Closes #116 by adding frameshift/splice summary to batch Version bump to 2.2.1 Fix unicode error in slugify commit 5ad9fbc6645475885fc0f35bc26afd353a2b3d5f Author: Cole Lyman Date: Mon Aug 16 14:16:13 2021 -0400 Read plain text fastq as bytes in filterFastqs commit 6c20f28678469875392e3fc8946018b53a00256b Author: Kendell Clement Date: Mon Aug 16 13:35:12 2021 -0400 Remove test for seaborn version commit cec965feff65b4228d42784e498398b73b8caa49 Author: Cole Lyman Date: Mon Aug 16 12:16:04 2021 -0400 Fix filterFastqs This commit fixes the filterFastqs script by properly casting to strings (from bytes) in the appropriate places. It also replaces the deprecated `np.fromstring` function with `np.frombuffer`. commit 3c30e56a15fa15a3537c3d51e429c1ff8738d6fe Author: Cole Lyman Date: Thu Aug 12 09:57:50 2021 -0400 Add .gitignore file commit 2461e30770d5ae8a6686530981a4bf5c913e2f68 Author: Cole Lyman Date: Thu Aug 12 09:50:01 2021 -0400 Decode result from Popen from bytes to string This is done only where a string is necessary, the instances where this is not done are where the result is cast to a float or int (because they can accept bytes as input directly). commit 64ec8bacf9ae8cb0d3a9093ad12a916983f32207 Author: Cole Lyman Date: Sat Aug 7 13:49:05 2021 -0400 Refactor `results/refs` and `results/ref_names` crispresso2_info fields commit ebb33a7d691b524ed7ab92b5a870da09df045e00 Author: Cole Lyman Date: Fri Aug 6 20:32:03 2021 -0400 Refactor `results/general_plots` crispresso2_info fields commit 94768e209e2424394efb4d902aac4f0e4268aad3 Author: Cole Lyman Date: Fri Aug 6 14:58:25 2021 -0400 Refactor `results/alignment_stats` crispresso2_info fields commit a58b7a2b40bc99346a9ea8f6240034963b3d6b31 Author: Cole Lyman Date: Fri Aug 6 14:10:33 2021 -0400 Refactor `running_info` crispresso2_info fields commit a92ea4687e0cc69844c5d4a6250757540076ed59 Author: Kendell Clement Date: Thu Aug 5 14:28:29 2021 -0400 Version bump to 2.2.0 commit 20e9b6f228949886ebe592d4542aaddbb9fb123a Author: Cole Lyman Date: Thu Aug 5 11:52:22 2021 -0400 Remove argparse dependency Remove the argparse dependency from setup.py because argparse is a standard module in Python 3. commit ecf70ea4379e079e7669fc5ed8baabdcd11a8c61 Author: Kendell Clement Date: Fri Jul 30 01:23:38 2021 -0400 Update Dockerfile bowtie throws an error 'Can't locate Sys/Hostname.pm in @INC' This fixes it. commit 30fb34e17787858d4218eb0b0fcc1baf6259ee23 Author: Kendell Clement Date: Fri Jul 30 00:39:48 2021 -0400 Ignore python egg for copying, pin tbb for bowtie error https://github.com/BenLangmead/bowtie2/issues/336 commit c850abb672019a3684a544fccde9550f7eeb86f5 Author: Kendell Clement Date: Thu Jul 29 20:13:32 2021 -0400 Update config.yml commit eb70cb18d6910f418bb8052f45b58c05c397af43 Author: Kendell Clement Date: Thu Jul 29 17:47:06 2021 -0400 Update config.yml commit 01f079fd663027574115a0af974564d5b5efbe97 Author: Kendell Clement Date: Thu Jul 29 17:22:48 2021 -0400 Update CRISPResso2_router.py commit 2e3b5d42b8ea6705dbd2c2f59312b93390083da3 Author: Kendell Clement Date: Thu Jul 29 17:20:16 2021 -0400 Update Dockerfile commit 46d8c44f2dbe7a67531d3ccae6b0a3c51b12b9ca Author: Kendell Clement Date: Thu Jul 29 17:07:02 2021 -0400 Update Dockerfile commit 0999e82dd8e184a6b0aefa7a35fc9decaa1fc20d Author: Kendell Clement Date: Thu Jul 29 16:59:47 2021 -0400 Update Dockerfile commit 34b93cfeeb95d0a1375378996e3ca29e79fe5311 Author: Kendell Clement Date: Thu Jul 29 16:50:20 2021 -0400 Python 3 commit e040ac488b050d6f316d21196de002ef09383915 Author: Kendell Clement Date: Tue Jul 27 10:04:43 2021 -0400 Add testing file for batch, remove debug print lines commit aa7b9998c4660438f4303a57386f01617ddec1bb Author: Kendell Clement Date: Thu Jul 22 10:43:52 2021 -0400 Update Batch to allow for max processes usage commit 1566a1110215e32f338497040f1279c3581b6dcd Author: Kendell Clement Date: Wed Jul 21 01:02:24 2021 -0400 Fix reference to BadParameterException commit fea5017109ab56c955b8053eebad5f6f4218bc65 Author: Kendell Clement Date: Mon Jul 12 16:11:05 2021 -0400 Allow spaces in run names, print run name to report commit b8594354c96131ba33c9277fb719c95b1cf72f3d Author: Kendell Clement Date: Tue Jun 29 14:12:03 2021 -0400 Update CRISPRessoPooledCORE.py Fix debug statement commit 9bc9b9959cbc8faf9dc462669e333298a80a71d1 Author: Kendell Clement Date: Tue Jun 29 14:09:13 2021 -0400 Update CRISPRessoPooled for samtools sort status updates Redirect samtools sort status updates to log file. Version bump. Previously this would cause an error: `ERROR: list index out of range samtools view: writing to standard output failed: Broken pipe samtools view: error closing standard output: -1`. commit ad5b9d853ac3559e58a5ad569e7d3ac9c3d8a719 Author: Kendell Clement Date: Thu Jun 24 12:10:31 2021 -0400 Allow max processes for CRISPRessoPooledWGSCompare Users can provide -p max to use all processes available for comparisons commit 48d6c8724f541b285e5d6889723cc37fc99e5cfa Author: Kendell Clement Date: Wed Jun 23 20:53:51 2021 -0400 Create html report for CRISPRessoComparePooledWGS commit abe3054dd9b95f51cbf34e3c37e088f6beb8287c Author: Kendell Clement Date: Wed Jun 23 20:53:22 2021 -0400 Fix allele plot bug #103 If no regions are returned, max of a pandas dataframe returns an error because the df is empty commit 1ccb507858d92516e28286358d3aae9cce2cf7ea Author: Kendell Clement Date: Wed Jun 23 20:51:16 2021 -0400 Fix command prompt logo lines commit 293c249c0f380576121a123dec982e40409c977e Author: Kendell Clement Date: Sun Jun 20 00:26:34 2021 -0400 Update plotCustomAllelePlot.py Get rid of debug statements commit 947fbab70e0f4fa78cc43273b4fe2be5225043cc Author: Kendell Clement Date: Sun Jun 20 00:22:56 2021 -0400 Add plotCustomAllelePlot script for replotting allele plots commit 167a48659684ce1669da915ddb6995935c6a1aa6 Author: Kendell Clement Date: Wed May 26 11:16:51 2021 -0400 Update README with explicit instructions to activate conda environment commit af0b7e441d2a4f1e035fabfd353c58784f688371 Author: Kendell Clement Date: Sun May 23 00:22:00 2021 -0400 Update test results for more flexible pooled alignment The relaxing of bowtie parameters allowed more reads to align, which changed the expected test results for CRISPRessoPooled commit 0e08cd05c2f3279ac95a068be76f1d36a4b0224d Author: Kendell Clement Date: Sun May 23 00:06:21 2021 -0400 Fix double rows in Alleles_frequency_table due to read directionality Previous to this fix, forward and reverse reads having the same sequence would appear in different rows in the Alleles_frequency_table. This fix doesn't change counts or results, just combines the two rows in the Alleles_frequency_table. commit 7d2d761a3d4c25915be0d27b36f3b4a87068587d Author: Kendell Clement Date: Thu May 20 16:05:27 2021 -0400 CRISPRessoPooled - get rid of -k bowtie2 parameter The -k 1 parameter may not yield the best alignment. This commit gets rid of that parameter. commit 44dc9e75c660f4b3b683f1c80f5a964aa55e75bd Author: Kendell Clement Date: Wed May 19 22:52:46 2021 -0400 CRISPRessoPooled alignment more tolerant When given a genome file, CRISPRessoPooled aligns reads to the genome using the Bowtie2 aligner. The legacy parameters were somewhat strict. The new parameters reflect the 'default_min_aln_score' parameter in allowing for substantially more indels and mismatches than previous. The parameter `--use_legacy_bowtie2_options_string` has been added to use the legacy settings. Otherwise, the bowtie2 alignment settings will be calculated as follows: commit 84b870ce0d489295501aa57711edd6b18c011b92 Author: Kendell Clement Date: Tue May 4 10:22:22 2021 -0400 Raise exception if quantification_window_coordinates looks like an int Previously, if quantification_window_coordinates looks like an int when pandas parsed a batch file, it would throw an error trying to split the int. Now an exception will be raised. commit e25a6dbb13fcd0565f4c81e3ea42cfd115aa1bc0 Author: Kendell Clement Date: Mon Apr 26 16:19:00 2021 -0400 Fix bug if all reads are discarded If all reads are discarded, CRISPResso would fail because the df_alleles is empty. This adds discarded alleles to the df_alleles. commit 156d7d640355ad4755c6ce4cbab1b75bf31677c9 Author: Kendell Clement Date: Fri Apr 9 13:24:51 2021 -0400 CRISPRessoPooled - close active file in demux commit c5d9fcbbf5bd9b307c9050498d08e72dd98d42aa Author: Kendell Clement Date: Fri Apr 9 12:29:02 2021 -0400 Keep old awk command for speed for samples with <50 amplicons commit c7539661fc5bd29f4f6ee1e9c7795be932b53a8e Author: Cole Lyman Date: Fri Apr 9 12:18:45 2021 -0400 Alt pooled processing implementation (#87) These changes implement separating reads to their corresponding amplicons via Python instead of through awk. This is to get around the maximum number of open files that is limited on many operating systems. Co-authored-by: Kendell Clement commit e57d98738fa832766c78dc7eecfdb0588d92634d Author: Kendell Clement Date: Thu Apr 8 12:36:43 2021 -0400 Alt pooled processing implementation commit 4598226837cd4b726ae38f50958f977bde4794ca Author: Kendell Clement Date: Sun Mar 21 00:16:19 2021 -0400 HDR Updates - yw #82 Multiple amplicon names are resolved before adding the HDR amplicon -- unnamed amplicons are named Amplicon{i} for each amplicon. Plot 4g data (nuc pct table, mod pct table for all reads aligned to the first reference) is output and linked to from the plot display Ambiguous reads don't contribute to plot 4g data (which would otherwise lead to double counting and pct values > 1) commit d7915b2c561173238c6b0e82863f76a783db7c4d Author: Kendell Clement Date: Sat Mar 20 01:12:55 2021 -0400 Fix #72 bam_input error commit 32a9e98ffc6ceb1dd56e5e29c369176ed788c985 Author: Kendell Clement Date: Sat Mar 20 01:01:17 2021 -0400 Prime editing, fastq_out updates Prime editing input parameters are forced to be in the RNA 3'->5' direction. This makes sure that the scaffold incorporation happens on the correct side of the extension sequence. Errors are thrown if improper directionality is detected. Fastq_out now includes alignment scores and details for every run (it may be time to upgrade that SSD to hold these new fastq output files, but it makes debugging particular reads a lot easier!) Update to linked data for plot 3b in report commit c1b6ea2ecebd920ea12ab91f2d50e35c274f94fe Author: Kyle McChesney Date: Wed Mar 10 13:40:44 2021 -0700 fix missing import path on NaN commit 1b24ca351f4337e0a7bece7ef7addb4558f950d8 Author: Kendell Clement Date: Sat Feb 20 22:57:07 2021 -0500 Update links in readme to https - fix #79 commit d550fd1db8ce0e1d11ed77fd082c53a3d887bbc2 Author: Kendell Clement Date: Fri Feb 12 16:57:37 2021 -0500 Insertion quantification change + fix #76 Starting in version 2.1.0, insertion quantification has been changed to only include insertions completely contained by the quantification window. To use the legacy quantification method (i.e. include insertions directly adjacent to the quantification window) please use the parameter --use_legacy_insertion_quantification commit 798f661031236ee1aa5611f491ca135ec0432dc9 Author: Kendell Clement Date: Thu Jan 14 23:51:29 2021 -0500 Allow for int-looking chromosome names in WGS input file In CRISPRessoWGS, the region file contains a 'chr_id' column which is sometimes mis-recognized as ints when read by pandas if using the chromosome notation without 'chr' (e.g. 1,2,3 in stead of chr1,chr2,chr3). This bug fix forces chr_ids to be read as strs. commit 92c008673690a3cd32e33fe8cfcf07060cf6cb0f Author: Kendell Clement Date: Wed Dec 30 23:48:32 2020 -0500 Update README.md commit 8d590c5588d93ef0129512a5156fe3b45e2d3c4b Author: Kendell Clement Date: Wed Dec 30 23:46:36 2020 -0500 Introduction of CRISPRessoAggregate to aggregate stats across runs Adds CRISPRessoAggregate Adds start/end time to CRISPRessoBatch info Started removing pickle dependencies from Pooled and Report commit c8447246ada2758831a870f30ed99c496c18feb1 Author: Kendell Clement Date: Sat Dec 26 10:52:12 2020 -0500 Output alignment details for unaligned reads in fastq_out or bam_out commit dedc36cadc64e9302d88c3472652d2a4a2385d25 Author: Kendell Clement Date: Tue Nov 24 13:45:34 2020 -0500 Add scripts folder for one-off analyses commit 88b6e9c50c6d97da357534c67aaeba64c00ddc23 Author: Kendell Clement Date: Sat Nov 14 02:46:36 2020 -0500 Fix plot window cloning from Ref1 to HDR Plot window for sgRNA will be the same length after cloning even if the window is shorter or longer after comparing between ref1 and HDR. commit 7403a46e186c4b70ad606dbb0eebbbde684fac61 Author: Kendell Clement Date: Sat Nov 14 01:52:55 2020 -0500 Frameshift plot and HDR quant window updates Frameshift plots don't show 0-bp changes (these dwarf all other changes). The number of reads not shown are added to the legend. Addressed cloning quantification windows when bases are inserted in the clone-ee (previously these cloned bases would be ignored. Force HDR to clone all quantification windows from Ref 1 Fix #60 and #59 commit f3f9c122cc25ab62bdd7f30fb9d6ee4c9ab820a8 Author: Kendell Clement Date: Sat Nov 7 23:55:55 2020 -0500 Update histogram x-limits, caption, and data commit e9332f7eabed32e65430b44677c841dd0150ac5a Author: Kendell Clement Date: Sat Nov 7 02:26:59 2020 -0500 Fixes for when no reads align #63 commit 9fae60a57d99238e4f7a9ad2e992ae71cca49c60 Author: Kendell Clement Date: Sat Nov 7 02:08:59 2020 -0500 Standardize pie plot appearances commit f1738bd17b496a31d6f68a91ed7ae67a36f8f178 Author: Kendell Clement Date: Sat Nov 7 00:36:40 2020 -0500 plotting and pe fixes - Special bonus for y'all to keep you company during covid - axis ticks on most plots! - added parameter --plot_histogram_outliers to plot all insertion sizes in histogram - all insertion sizes are reported in .hist output files #64 - add HDR reference plot (may change this later to set ref1 to the longer reference of WT/HDR but for now it is always WT..) Allow reverse complement of extension seq if PE sequence is specified. commit 5a2d9271b3ea99342f22e59259149d30dc193e47 Merge: bd03668 9812651 Author: Kendell Clement Date: Wed Oct 21 16:52:12 2020 -0400 Merge pull request #62 from matandro/patch-1 Fix a bug when generating compare plot commit 9812651c2aae1218393e8ce0d734812247cd59ab Author: Matan Drory Retwitzer Date: Wed Oct 21 10:45:01 2020 -0700 Fix a bug when generating compare plot Related to issue #61 This happens when N_ROWS < 1 which I assume has something to do with no results -> negative control commit bd03668ef1626a54e0b5bfbc010afacee60f45e3 Author: kclem Date: Fri Oct 2 17:39:52 2020 -0400 delete merging intermediate files commit 3c9b1344e19dee04b169c0860bc11304d6a594b5 Author: Kendell Clement Date: Wed Sep 30 23:51:08 2020 -0400 Version bump to v2.0.42 commit 2f8c6f9c7d31b276ff18000d579ef722f693c433 Author: Kendell Clement Date: Wed Sep 30 23:44:02 2020 -0400 Update new parameters, fix docker biuld problem commit f130270135b84e0904e0003858c263285834b6dd Author: Kendell Clement Date: Wed Sep 30 14:18:58 2020 -0400 Fix bug in read counting for interleaved fastqs commit faac4e1045e5b617b3743e328c0203ced27e3f31 Author: Kendell Clement Date: Thu Sep 24 22:37:50 2020 -0400 Fix bug in mode to write fastq out commit 388e8a97fc18648dd28e35063cb12680887395e2 Author: Kendell Clement Date: Wed Sep 23 23:49:44 2020 -0400 Update postrun reference output file Rename postrun references file to be more standardized with other output files. Output is now "CRISPResso2Pooled_postrun_references.txt" commit ee5be76e5e24e494abd93940622d51faa83a2371 Author: Kendell Clement Date: Wed Sep 23 23:32:34 2020 -0400 Multiple allele support for pooled mode Most common alleles for each pooled target are output if the flag '--compile_postrun_references' is provided. This writes alleles with frequncy defined by the parameter to --compile_postrun_reference_allele_cutoff This file can be manually edited to remove noisy alleles, and then used to run CRISPRessoPooled again but to provide alternate alleles to each CRISPResso run by using the parameter '--alternate_alleles'. This is particularly useful in cases where control experiments are available. The running pattern would be: 1) CRISPRessoPooled --compile_postrun_references {control} 2) CRISPRessoPooled --alternate_alleles {produced in step 1} {control} CRISPRessoPooled --alternate_alleles {produced in step 1} {experiment} commit fe5bee2ac7ca4a8dd165e2c7305a1c41a79b8d9d Author: Kendell Clement Date: Wed Sep 23 23:27:37 2020 -0400 Output + PE updates Add parameter '--fastq_output' to output a fastq file with annotations for each read. Also, the substitution frequency table is only produced in base editor mode -- in an effort to slim down CRISPResso2 output files. Also, especially for long Prime Editing insertions or deletions, the inference algorithm may incorrectly infer the prime-edited allele. I added the parameter '--prime_editing_override_prime_edited_ref_seq' which can be used to specify the prime-edited sequence manually. commit ca61c483a8a63fec2e85889f554648ea6f3a903c Author: Kendell Clement Date: Sat Sep 5 16:15:16 2020 -0400 update report caption for fig 10 commit 346c6f6e62097ea7690204cfe879ebb918fb244d Merge: e52cb73 44ccc5e Author: kclem Date: Fri Sep 4 15:05:02 2020 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit e52cb73471f4538ecd98f943a38771f0d07a4476 Author: kclem Date: Fri Sep 4 15:04:48 2020 -0400 Update on help message for mark wildtype allele commit 44ccc5ec3ff6a57d265807b559010512f053d82a Author: Kendell Clement Date: Fri Sep 4 14:59:05 2020 -0400 Update to plotting quantification window plot vertically centered, pooled/wgs plots are not limited in height to allow analysis of large pools. commit ff675b12196593ceb8e8d8b58dfd6a8ca8865501 Author: Kendell Clement Date: Mon Jul 27 09:55:30 2020 -0400 Update parameters and description in README commit 25c6a1b45ef1e1c1243057b9b3ee06f638d55d26 Author: Kendell Clement Date: Sat Jul 25 00:29:14 2020 -0400 Update CRISPRessoBatchCORE.py If amplicon sequence is empty, auto mode is run for batches commit 2978a6ebc0cd977b36b4ea7e1965f5c43b46d325 Author: Kendell Clement Date: Thu Jul 23 08:46:01 2020 -0400 Removed WGS logging For some reason, piping output breaks multiprocessing blocking commit 8bc9214ba0a1b628b69a49a2cf306bf559e008b3 Author: Kendell Clement Date: Wed Jul 22 22:58:51 2020 -0400 Update CRISPRessoWGSCORE.py commit a9484fd481bfa8ad716c6326aba9c57f5271f885 Author: Kendell Clement Date: Wed Jul 22 14:24:38 2020 -0400 Add parameter discard_guide_positions_overhanging_amplicon_edge If run with param -- discard_guide_positions_overhanging_amplicon_edge, for guides that align to multiple positions, guide positions will be discarded if plotting around those regions would included bp that extend beyond the end of the amplicon. Normally this would cause CRISPResso to fail if plotting were requested beyond the end of an amplicon. commit 8d26edeed89e33451bd539c242f7ffaa560db3e6 Author: kclem Date: Wed Jul 22 13:58:10 2020 -0400 WGS doesn't print crispresso output to screen WGS printing error fixed commit 5ed03059d09aaf8a416c5fec553801eeca73355f Author: kclem Date: Tue Jul 21 00:00:40 2020 -0400 bam processing update of cached not-alignments commit cf593183d7b3d8200ec295ebcc653e518775046a Author: Kendell Clement Date: Thu Jul 16 21:49:21 2020 -0400 Fix naming of scaffold-incorporated reference links on plots commit 676ff623373b0cabf992017bd5eca255c4073d2a Author: Kendell Clement Date: Fri Jul 10 00:02:59 2020 -0400 Prime editing updates prime editing guides are shown as input on report output file names are produced without spaces commit 9de370148b2ada7210c10532247b3fa4b18a76de Author: Kendell Clement Date: Tue Jul 7 20:46:30 2020 -0400 Update batch for multiple quant window input commit 6ad1822f478d7f108b92f25378cc3ddf8dc3a2d3 Author: Kendell Clement Date: Wed Jul 1 16:26:38 2020 -0400 Bam processing + Prime Editing updates -Input can now be read from bam using the parameter `--bam_input` and (optionally) `--bam_chr_loc` to use the reads in the bam at this location as input. An output bam is produced with an additional soace-separated field prefixed by c2 (e.g. c2:Z:ALN=Inferred CLASS=Inferred_MODIFIED MODS=D47;I0;S0 DEL=56(47) INS= SUB= ALN_REF=TTGGCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGAAGTAGGGCCTTCGCGCACCTCATGGAATCCCTTCTGCAGCACCTGGATCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCT----------------------------------------------- ALN_SEQ=ACACCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGA-----------------------------------------------TCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCTGGAGCGGCGGCTGCACAACCAGTGGAGGCAAGAGGGCGGCTTTGGGC). Note that the alignment details (location, cigar string, etc) are not modified.. this may be done in the future). Bam file input cannot be trimmed or pre-processed with quality filtering. -Prime editing scaffold incorporation is now more accurate (looks for the scaffold sequence at the expected position directly after the extension sequence). A plot showing the number of bases matching the scaffold, as well as insertions after the extension sequence, and a data file with these numbers is produced. Added parameter `--prime_editing_pegRNA_scaffold_min_match_length` to define the minimum length required to classify a read as 'Scaffold-incorporated' -Renamed split_paired_end parameter to `--split_interleaved_input` for interleaved input -Auto mode now considers 5000 reads to detect amplicon sequences -Add new paramter `--annotate_wildtype_allele` to annotate wildtype alleles on the allele plots -Update output when reporting missing files -- only lists first 15 files in the current directory and directory of input parameter --reference https instead of http commit c0f2871befed86c4b314100584bf844f13d71d0e Author: Kendell Clement Date: Tue Jun 2 18:54:52 2020 -0400 Update CRISPRessoWGSCORE.py Remove debug print commit e5450b1cc6be518706969d27bda843cb7c16b082 Author: Kendell Clement Date: Tue Jun 2 18:53:20 2020 -0400 Updates for Pooled and WGS WGS gene annotations compatability fix and pooled gzip fix commit c2b286dbda3807752f34cdf6274e41d5640b408f Author: Kendell Clement Date: Tue May 12 00:33:36 2020 -0400 Fix docker bug, print version to Pooled + WGS commit 2a06cc18c3157e6c135dae7ce53adec94fe8a83f Author: kclem Date: Sun May 10 00:09:55 2020 -0400 Fix plotting bug if no sgRNAs given commit 1e3ca605e0c5ae65e8acc727667f952bc4c0d3ee Author: kclem Date: Sun May 10 00:00:32 2020 -0400 flexiguide fix commit 4e1d6b2b3424e725010e3e1a13522a7386228853 Author: Kendell Clement Date: Sat May 9 23:30:39 2020 -0400 Prime editing refinement and Pooled filesystem demand reduction - Prime editing parameters renamed, nicking guides match with flexibility - Prime editing extension seq is shown as a guide (with no cut site) - Prime editing summary plot included in report - Nucleotide plots are shaded when no changes from the reference sequence - sgRNA annotations are plotted on multiple lines if they overlap - N's don't count as substitutions - extended read analysis data available with --write_detailed_allele_table flag commit 8c584719b9771c01e53cfe409789b29a29fad665 Merge: f506936 7624815 Author: Kendell Clement Date: Sat May 9 23:14:30 2020 -0400 Merge pull request #42 from ronaldhause/patch-1 ZipFile: set allowZip64=True to write larger allele frequency tables commit f5069365d37bd698ff3bf35e5c39a1b75e10dc1d Author: kclem Date: Sat May 9 12:04:10 2020 -0400 CRISPRessoPooled updates, fix #37 for too many files Demultiplexing in the case of amplicons + genome is parallelized to reduce sorting Only files with sufficient reads are demultiplexed and written Additional output file REPORT_READS_ALIGNED_TO_GENOME_ALL_DEPTHS.txt shows all alignment locations commit 7624815e3159d926d8a0710f674f44c616b68bd5 Author: Ron Hause Date: Sun May 3 22:50:07 2020 -0700 ZipFile: set allowZip64=True to write larger allele frequency tables Addresses terminating ERROR: Filesize would require ZIP64 extensions when trying to write compressed allele frequency tables > 2 GB commit 6af15a09033166b713df159a6ef850dde8867253 Author: kclem Date: Tue Apr 28 03:28:41 2020 -0400 int bug fixes commit 531753c0f5be89c255f65d876cba5e9bf00dd4a2 Author: Kendell Clement Date: Tue Apr 28 02:00:37 2020 -0400 Introduces support for prime editing, multiple window sizes and offsets, max processors commit 8e29c1e0966ebb2073d52a221ae77e56bb146431 Merge: adb0d8b 039013e Author: Kendell Clement Date: Fri Apr 24 13:06:46 2020 -0400 Merge pull request #41 from natecarlson/fix-name-error If the name column is called 'name', refer to it as 'name', not '#name'. commit 039013efe363603dfe89f10b857d58bb1ef8e8d9 Author: Nate Carlson Date: Fri Apr 24 09:46:21 2020 -0500 If the name column is called 'name', refer to it as 'name', not '#name'. commit adb0d8b791686846fa8522f46e064418cbfbdc1c Author: Kendell Clement Date: Mon Apr 20 16:26:32 2020 -0400 Update LICENSE.txt commit b514a68eea135e805333c67bf33bf0f1ea41a034 Author: Kendell Clement Date: Sun Apr 19 00:36:26 2020 -0400 Print CRISPResso command on multiprocessing fail commit 4bfadd08d30e1a8b926757c1af4a9c0c0dc0b484 Author: Kendell Clement Date: Sun Apr 19 00:03:58 2020 -0400 Index fix for crispresso multiprocessing Indexes were incremented for user enjoyment (1-based) but the more accurate approach is 0-based Also, no_rerun ignores changes to the flags 'debug' or 'n_processes' which shouldn't affect the outcome commit 867e692b326acd19ed5f291bd5699f5885b1d569 Author: kclem Date: Thu Apr 16 00:25:47 2020 -0400 Allele plot sgRNA labels stay on plot commit b0d89b4effc4242ec55ed4a3e20e8835a90e3588 Author: kclem Date: Wed Apr 15 23:58:46 2020 -0400 WGS fastq seqs are now uppercase, so guides match even in lowercase-masked genomes commit 8098f1f1dda6efdaf773b9f85622d79f32ac49c9 Author: kclem Date: Thu Apr 9 01:11:26 2020 -0400 Pooled multiprocessing updates commit 034ff2b5858dd29ab60017835695fad527a5213e Author: Kendell Clement Date: Tue Apr 7 02:16:39 2020 -0400 add n_processes param for pooled analysis commit 7fb477ff15c7f8d62d3acad4d14434942cd25bff Author: Kendell Clement Date: Tue Apr 7 00:36:47 2020 -0400 Pooled Set flag to skip reporting problematic regions commit 6c3aeff8b96d918ef2b3c518b73860d3f74480b8 Author: Kendell Clement Date: Mon Apr 6 19:56:56 2020 -0400 Pooled parallelization by chr Parallelized CRISPRessoPooled extraction to operate by chr Attempt to appease the dockerhub requrements -- require cython for compilation commit a66c4020f473d2dee80fa1257c04049b2ba6dbd3 Author: kclem Date: Fri Apr 3 13:41:38 2020 -0400 v2.0.33 plot updates allele plot sgRNAs that extend beyond plot are marked quantification window shading and right-side correction version commit 90e677f453ef971b478e4582120b17cbb572212c Author: Kendell Clement Date: Fri Apr 3 01:30:57 2020 -0400 Pooled bug fixes for regions with the same location and different names commit d795479d117e689a6679ffe463df413bff2f6a5a Author: Kendell Clement Date: Fri Apr 3 01:23:30 2020 -0400 Fix open error for docker commit 9aee866e5f65bf8df6495e81684651436a0b0a30 Author: Kendell Clement Date: Fri Apr 3 01:17:09 2020 -0400 Parallelization of Pooled and introduction of checkpoints for WGS and Pooled Alignment of amplicons is done in one bowtie2 call instead of one bowtiecall per amplicon Parallelize several time-intensive steps in Pooled (splitting by region, etc) --no_rerun flag will skip already-processed steps in Pooled and WGS commit fb1e7e25404bd80722824755c1b4ff4478449c48 Author: Kendell Clement Date: Thu Apr 2 01:34:52 2020 -0400 WGS updates - multiprocessing and no rerun WGS multiprocesses extraction of reads across multiple cores. WGS extraction doesn't occur if the --no_rerun flag is set and the files are all present. commit 45d4377f678fceeff83d60efa22dbd1078e6840e Author: Kendell Clement Date: Tue Mar 31 10:35:43 2020 -0400 Remove biopython for fastq parser Living life on the edge -- dropping the biopython fastq parser to remove biopython package dependency. This will discard the minimal error-checking provided by the biopython package. commit d52f69c9fa16ea38e919f9e3dc70fb6d53246610 Author: kclem Date: Tue Feb 25 17:05:08 2020 -0500 Pin dependency versions commit 86ff4bebcfcaa5c618ff124822ecb45de2b7c9ca Author: kclem Date: Tue Jan 28 16:17:57 2020 -0500 get rid of test for CRISPRessoCompare commit b7e96d6f4d1e0966cc0847b620349ee7fe21f43f Author: kclem Date: Tue Jan 28 15:26:20 2020 -0500 Fix problem with circleci testing Switch columns for output quantification files so that total reads is before the number of aligned reads commit ac976074c03967a386ed386d9ce3436c5fa1f2d5 Author: kclem Date: Fri Jan 24 14:02:14 2020 -0500 Version 2.0.32 - guide updates and other updates Introduced flexiguides - can match with flexibility to the reference sequence - useful for pooled screens of offtargets (parameter --fg or --flexiguide, with --fg or --flexiguide_homology to control how many mismatches) Flexiguide mismatches and other mismatches are shown on plots sgRNAs can be labeled (parameter -gn or --guide_name) sgRNA position shown on allele frequency plots detection of dsODN -- shown in plot 1d (parameter --dsODN) CRISPRessoPooled gene set input flexibility - more formats accepted plot 8 shown on html report commit ca9273377acbabf01f97f04a028d4fd87a09d6b5 Author: Kendell Clement Date: Fri Dec 6 15:14:38 2019 -0500 Fix docker bug #30 commit a3ae575f870ace49a459447e13288cfd50487e2f Author: kclem Date: Wed Oct 2 20:57:03 2019 -0400 remove debug statements commit 53d70eb83bad2aa8456b744dd29611a788f3dbbd Author: kclem Date: Tue Oct 1 15:41:39 2019 -0400 Fix #25 to accept bt2l bowtie2 index extension commit 039cc8e1b4a145e53cf06c1d52f102497928f3ac Author: kclem Date: Wed Sep 25 17:53:49 2019 -0400 v2.0.31 CRISPRessoPooled chr names fix, allele plot colors CRISPRessoPooled compatability with chr names with underscores (alternate scaffolds) Additional function to plot allele table with custom set of colors for a completed run commit c35d0151efcd70a6044399e16c841ee9d0ad0535 Merge: 491247e b1d1518 Author: kclem Date: Thu Sep 19 16:23:13 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 491247e1a07e2991a74341b4424c7c722a3db6a9 Author: kclem Date: Thu Sep 19 16:22:56 2019 -0400 updates to command line output, batch bug commit b1d1518a7ed4b72e92fd9d10586ccce653585630 Author: Kendell Clement Date: Fri Aug 23 11:26:53 2019 -0400 Update conda installation path commit cdfd78772de27bdb78a380e591d8857acd143562 Author: kclem Date: Tue Aug 13 11:24:58 2019 -0400 Use python 2.7 pandas commit 1417d1aa387d45f37953b93304a545e08943cc45 Author: kclem Date: Tue Jul 16 17:18:28 2019 -0400 force merge reads and fix #19 add optional param for force merging R1 and R2 in case they don't cover the entire amplicon fix labels for expected amplicon plots commit 60f90053ab65958871402b609d0b72b31021bc6b Author: kclem Date: Tue Jul 2 17:28:27 2019 -0400 v2.0.30 case-insensitive guides, fix #17 commit 702b9dce89e070d96064db314ac1c5155fedd42e Author: Kendell Clement Date: Tue Jul 2 16:18:17 2019 -0400 Update README.md commit 5a10eb9a9d24371d2bedf8b173f9c7401d7cbd53 Merge: bc7b332 bc006c8 Author: kclem Date: Tue Jun 25 15:32:51 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit bc7b3321e1bb6b7ed0c8af48d609e86e55081f6b Author: kclem Date: Tue Jun 25 15:32:45 2019 -0400 plot window bug update commit bc006c826f338b9a1fcaec2286b506bdd2c548db Merge: 1f7171b feee2c2 Author: Kendell Clement Date: Thu Jun 20 00:56:45 2019 -0400 Merge pull request #16 from PEHGP/patch-1 args.trimmomatic_command for CRISPRessoPooled commit feee2c2eb20807933376f01a88102767ce5e42e4 Author: kuan <396777306@qq.com> Date: Thu Jun 20 09:44:32 2019 +0800 args.trimmomatic_command args.trimmomatic_command commit 1f7171b8eb2dcd63fada80fa6cac561e576f727c Merge: f6c9eed 3d4a37a Author: kclem Date: Thu Jun 13 13:13:33 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit f6c9eed206b16fede7c88724c94d8d091f8657f3 Author: kclem Date: Thu Jun 13 13:13:20 2019 -0400 update pooled names commit 3d4a37a626f366f8f024ee7f62e017e8d68092b3 Author: Kendell Clement Date: Tue Jun 11 12:33:40 2019 -0400 Add web link to readme commit 89348066ede5c447db9f04b2337118a0624b8f63 Author: kclem Date: Tue Jun 11 11:14:55 2019 -0400 add batch percentage report commit 3bcd50d67c9b320c9f908eb2e3469143461e7df6 Author: kclem Date: Thu May 30 17:25:05 2019 -0400 document report param commit 79303435747a70b288b88bb076dda90b8d379f91 Author: kclem Date: Thu May 30 17:16:11 2019 -0400 output updates commit 9f14424f275f132a148308042db48c77cbda9b1d Author: kclem Date: Fri May 24 17:57:57 2019 -0400 plot updates, compare bug #12 commit 7f5482c411a3d56a73d7f0c66f0159b623653c2a Author: kclem Date: Tue May 21 17:47:08 2019 -0400 remove crispressocompare test condition (cuz has floats) commit 5e2f4094ec607f63981cd5aa2ee7f99ce1a20d12 Merge: aac9dfe 5933d93 Author: kclem Date: Tue May 21 17:40:44 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit aac9dfe70292a90d6981b0859ff9ede8da7094a4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test changed test to something that won't be affected by float formatting commit 5933d93c5f1408f754895254306701b5b0eaadd4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test commit 7d15cdfaa4a323f4baad96bf36c97fd455dd6684 Author: kclem Date: Tue May 21 17:11:09 2019 -0400 Add compiled c file commit 50134d64d33dc984ac81a066e7c370b160b861b3 Author: kclem Date: Tue May 21 16:58:57 2019 -0400 v2.0.28 Add report for CRISPRessoCompare Standardize naming conventions for files and plots from CRISPResso Add data links to CRISPRessoBatch report CRISPRessoBatch plots using the plot window around the cut site instead of only the quantification window If only one reference, 'Reference' is not shown in data plots or as a file prefix Set plotting indexes once for each guide (previously, they were specific to the amplicon) Base editing plots now plot for each guide (previously, they were one for each amplicon) commit 9e86bef89884a0e5980c7781cfcd56243fdd42f0 Author: kclem Date: Wed May 15 14:28:14 2019 -0400 Standardize concept of windows for quantification and plotting #11 commit dd63974334c94816efe5878e4aab1ec3c3e1a6c5 Author: kclem Date: Wed May 1 14:49:24 2019 -0400 python division bug.. <3<3 commit 4a4b88885ab2e034bdcbca9566aebe911c58f427 Author: kclem Date: Wed May 1 14:35:33 2019 -0400 fix bug for spaces in filepaths commit f7e0aee5d948abd876b5048c87cae59e265d0c6f Author: kclem Date: Fri Apr 26 11:56:51 2019 -0400 min merge size commit 12cc6a93c0b02be86575c6c7fe8b7569a96e0edd Author: kclem Date: Fri Apr 26 11:41:09 2019 -0400 Relax flash merging, add parameter for stringent flash merging (#7), remove debug statements commit 38bfb3174d8d37297587243cb9e04469e7e54a20 Author: Kendell Clement Date: Thu Apr 25 22:50:08 2019 -0400 Update CRISPRessoPooledCORE.py fix numpy -> string error commit 273fe3a1ab731a32c26efdcab58a502cd2162104 Author: kclem Date: Mon Apr 15 13:38:17 2019 -0400 Fix CRISPRessoWGS tests commit 2c1ec9f5eadaf62ac7df35535aa26965d993e96a Author: kclem Date: Mon Apr 15 13:15:03 2019 -0400 add tests for CRISPRessoWGS commit 6632700fbde6b7f5e49e402471fea88a0966d2c0 Author: kclem Date: Fri Apr 12 10:29:43 2019 -0400 update docker run message, enable local testing, remove networkx commit c31355e15afc49f6aa069968c94408f5f7b484c0 Author: kclem Date: Thu Apr 11 16:00:44 2019 -0400 ignore test directory for docker commit aad55c922fa9b3948e5b194b26da3a8a3ccc97d2 Author: kclem Date: Thu Apr 11 15:52:45 2019 -0400 fix plot label bug for 10a, update fig 2a data commit c40b2de4f21e69404de153b9e12dee6654568b67 Merge: 560b65e 88990db Author: kclem Date: Wed Apr 10 17:30:41 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 560b65e720a6dc5329203ecbc8c1b38b47aee219 Author: kclem Date: Wed Apr 10 17:30:34 2019 -0400 update tests for decimal shift for percentage in summary report commit 88990dbb29135d160879ed02dcac6874b0fed9f2 Author: Kendell Clement Date: Wed Apr 10 17:26:20 2019 -0400 Update README.md commit e4deb9211dadb3e9622f2eee38f0cc4930a0cf17 Author: kclem Date: Wed Apr 10 17:19:07 2019 -0400 v2.0.28 commit 098f5a68ba632987930fd70038af667e228b7a1f Author: kclem Date: Tue Apr 9 22:13:13 2019 -0400 update circleci path commit bb78ade66d20a826bbd8beee53711620869b86aa Author: kclem Date: Tue Apr 9 17:40:43 2019 -0400 circleci test path update2 commit e760c77bdd23fd087733c26eda86f15de6877bbf Author: kclem Date: Tue Apr 9 17:36:23 2019 -0400 circleci test path update commit 0e1b576959b09da72b101983d312842e06ea4c56 Author: kclem Date: Tue Apr 9 17:33:22 2019 -0400 circleci artifact storage commit 28c23d2172cf5c39faffd1ea498f6d771237567d Merge: c7da846 e42b3a6 Author: kclem Date: Tue Apr 9 14:17:34 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit c7da84668556b897b43dd53eb2370c3404ab3fa2 Author: kclem Date: Tue Apr 9 14:17:23 2019 -0400 circleci testing for batch and pooled commit e42b3a6b4c11cab0ac9c29c378858706b7fd328e Author: Kendell Clement Date: Tue Apr 9 11:55:04 2019 -0400 Update badge links commit 6a435376fe43e6408507f1078518515f1f522ba8 Author: Kendell Clement Date: Tue Apr 9 11:42:21 2019 -0400 Got me some badges! commit f8c9713444472f5e3cef4fd7f7bce0704dd6a266 Author: kclem Date: Tue Apr 9 11:07:18 2019 -0400 circleci artifact update commit 8d4b825dcf01887db96b90640aafea564031a7e9 Author: kclem Date: Tue Apr 9 11:03:59 2019 -0400 circleci updates commit bfb3bc82abd22647554b80517554ef58e81d2090 Author: kclem Date: Tue Apr 9 10:51:53 2019 -0400 add expected results for circleci commit 8466e146d73a2c2149fdbcd67d4db656fe8c13e0 Author: kclem Date: Tue Apr 9 10:29:57 2019 -0400 circleci update commit 19c437975e57119531bcb0b9b2a6587a7d6b3ea7 Author: kclem Date: Tue Apr 9 10:14:42 2019 -0400 circleci update commit c665c19c97f4c6d4b45f40b3ac9522bf53ce05ba Author: kclem Date: Mon Apr 8 16:27:38 2019 -0400 circleci - use custom docker commit caa46ce2bc1de5e100bfd5db25179fb6b54f4d95 Author: kclem Date: Mon Apr 8 14:45:16 2019 -0400 python2 virtualenv commit a13e2fd2c9eea59652ec1ad7c6a9862a708bc33e Author: kclem Date: Mon Apr 8 13:54:32 2019 -0400 CircleCI testing commit 7257b54f77391da21532bf06ed84b1ce03d460e0 Author: Kendell Clement Date: Fri Apr 5 11:15:22 2019 -0400 Remove dependency on zip commit 7b694f507a61e424e994384cd765855d7f69dfb4 Author: kclem Date: Thu Apr 4 12:12:37 2019 -0400 add networkx requirement for py27 commit 099acbc8149dbbd5c99729ddc8e0928e3f890023 Author: kclem Date: Thu Apr 4 10:16:10 2019 -0400 Fixes for dockerhub commit 3a3bfbdd2373348901faac6fda468b70cb5ce725 Author: kclem Date: Thu Apr 4 10:09:06 2019 -0400 Bioconda submission fixes commit dff86e16812f2b9345ef86bd3185786f4f82d25a Author: kclem Date: Wed Apr 3 18:49:13 2019 -0400 More precise cleavage window and quant window plotting commit c8caf7fbdad415d4d3fb47cc5b9b0b76dd65deab Author: kclem Date: Wed Apr 3 17:49:54 2019 -0400 v2.0.27 add reports for pooled and WGS commit c68e3cb922d037c12a62c695c0ae1f6c148ace82 Author: kclem Date: Mon Mar 18 11:15:40 2019 -0400 batch info pickle, WGS bai location, meta mode commit 768c75c3a7864c365cf13fd65573859b9aa86ebe Author: kclem Date: Wed Mar 6 17:13:56 2019 -0500 v2.0.26 add report display name, remove paths from stored files, fix sgRNA plot, CRISPRessoPooled report HTML, add citation to report commit 58257b54fc440427e5437f8e7458fd5824020b6e Author: Kendell Clement Date: Fri Mar 1 16:55:27 2019 -0500 Update issue templates commit 50fb2d58f0d3777ba51d0f5a37e82cfc1a47ebff Author: kclem Date: Fri Mar 1 16:38:13 2019 -0500 Fix file naming and flash incompatibility commit eca34aebf86dc1b03c6107ee405cdf16898c8d51 Author: kclem Date: Wed Feb 20 17:26:42 2019 -0500 v2.0.25 Add inferring of guides commit 2ccec08691db17e6a2cfab8e310f5f29621319ca Author: kclem Date: Wed Feb 20 13:42:57 2019 -0500 quant window updates commit 21d558da1f8f5fed8f59de0eb154e5a8a505ff7a Author: kclem Date: Tue Feb 19 17:21:12 2019 -0500 add flash outies, fix quant window coords bug commit 22954e6d40ad2d237cb75c521a09f30c2b294066 Author: kclem Date: Tue Feb 19 11:17:30 2019 -0500 Update entrypoint for docker commit 0216329d8a8d1c072a269d14786820f943443153 Author: kclem Date: Wed Feb 13 16:40:39 2019 -0500 v2.0.24 update docker, setup.py commit e11b60fe1cf3c3e67e41964d45e843f28c2975a5 Merge: fb1687b d6a7b97 Author: kclem Date: Mon Feb 11 16:22:09 2019 -0500 v2.0.23 suppress plots, custom flash version commit fb1687b87de0cb7e5d1ab0acc2a2651d1be1fcec Author: kclem Date: Mon Feb 11 16:12:26 2019 -0500 v2.0.23 suppress plots, custom flash version commit d6a7b979e1ecd49b26c648f2007e1ecfa905ec14 Author: Kendell Clement Date: Fri Feb 1 12:20:50 2019 -0500 Update readme formatting commit 9e4c87c4f60edd82539b01b24f646962c0f06f4c Author: Kendell Clement Date: Thu Jan 24 17:13:28 2019 -0500 Add conda install instructions commit 4b2b06e52ee1aba4f3bdd0458c13813f2417fbf7 Author: kclem Date: Thu Jan 24 14:02:05 2019 -0500 add manifest.in commit 495e1f9829d6e72048b87a6972472c4242411286 Author: kclem Date: Wed Jan 23 11:05:44 2019 -0500 Change license location, license update commit 24c3b1b6ba66557b99469c0e12291fae2fcc800d Author: kclem Date: Wed Jan 23 11:01:44 2019 -0500 v2.0.22 commit 4a426bf715e37cbb8d619d54fc3a92385e9dcf1b Author: kclem Date: Tue Jan 22 15:25:13 2019 -0500 update batch amplicon naming commit f4fbe96b335c6e1a5b3dc252bf090e3d36ffb8cd Author: kclem Date: Tue Jan 22 12:44:00 2019 -0500 v2.0.21 detangled root location dependency from params commit 9d29737de257a2999f0763afd5f97ddafd6d2fe7 Author: kclem Date: Tue Jan 22 12:36:03 2019 -0500 Update CRISPRessoShared.py commit 573aa90cf70cd941c3e26361b2e656fbf34d8e5e Author: kclem Date: Fri Jan 18 18:10:47 2019 -0500 allow no cython commit 69811cf1eb58ac014a1ac34c56748aaffcead187 Author: kclem Date: Fri Jan 18 17:55:23 2019 -0500 require cython for installation commit 37ac0e6278c4825bb816c7cfe0a1fc3819abd516 Author: kclem Date: Fri Jan 18 17:44:19 2019 -0500 v2.0.20b prepare for bioconda integration commit 31c5ad02127366d0ace2849a6e996a86d690f4d1 Author: Kendell Clement Date: Tue Jan 15 17:22:53 2019 -0500 Update README.md commit 5b8cf82bfee3eeefcf2ba5bb31b4a60b25960df0 Author: Kendell Clement Date: Tue Jan 15 16:46:46 2019 -0500 Update README.md commit c932b8b4ad7c08d3fc94d5c57d0017001a78a42f Author: Kendell Clement Date: Tue Jan 15 16:40:59 2019 -0500 Update README.md commit 5cf5aa34b2ef97e38e45ee3394cd8b4aade50c6d Author: Kendell Clement Date: Fri Dec 21 15:03:50 2018 -0500 Add trimmomatic_command parameter commit 2ec374962c169c1a56cd48ff9080f7941433d016 Author: kclem Date: Thu Dec 20 18:30:01 2018 -0500 v2.0.19b HDR and WGS changes commit 78483624993c6fdb5ccc889a2a6f37036fb0f2c8 Author: kclem Date: Thu Dec 6 14:55:18 2018 -0500 add filtering for fastqs commit 17940c70b36d74d8b9134664b515f3b164c678b0 Author: kclem Date: Wed Nov 28 15:00:35 2018 -0500 v2.0.18b - fix bug with batch names, add param to suppress plots commit 038042e3cea983fc264460402d88e15766cc50b0 Author: kclem Date: Wed Nov 14 15:00:47 2018 -0500 Add router for docker commit 3ef96cc08a18f6eff9bb6115366e27304986ed27 Author: kclem Date: Wed Nov 14 14:52:41 2018 -0500 add Docker file commit 3e1cbf1dffc4dbc555942d45f91ad942237b81c3 Author: kclem Date: Wed Nov 14 14:29:44 2018 -0500 set white background for plots commit dd2cb254170b8363a7feb0427ed587d2093ebd3e Author: kclem Date: Wed Nov 14 11:13:02 2018 -0500 Set seaborn style commit dde6a675a5ba4e9ec100c29c2d59534aa39ef4fc Author: kclem Date: Tue Nov 13 16:13:55 2018 -0500 Fix line endings commit db0a67017f57c5a77bed9eb442cb7efa9df4b97a Author: kclem Date: Tue Nov 13 10:31:45 2018 -0500 v2.0.17b - bug with multiple references of different lengths commit 7f4afffa1485094aee4c0399a319a30c12ec473e Author: Kendell Clement Date: Tue Oct 23 17:22:07 2018 -0400 Update README.md commit 0843923b50d2db953600230ac23614f52591c4b4 Author: Kendell Clement Date: Tue Oct 23 16:59:48 2018 -0400 Update README.md commit 6f79084ce88502a40fe8b9b1839733ca224d14dd Author: Kendell Clement Date: Tue Oct 23 16:56:38 2018 -0400 Add files via upload commit 72d0b355b5d39164405b6311bda6231dbbdef371 Author: Kendell Clement Date: Tue Oct 23 15:44:26 2018 -0400 Update README.md commit 30fdc7fd5d73594b332efd546a1f4d85004a80d6 Author: Kendell Clement Date: Tue Oct 23 13:48:25 2018 -0400 Update README.md commit 5b11e51083ca87cbc1a8dff02b4634fe12176d29 Author: Kendell Clement Date: Tue Oct 23 13:42:11 2018 -0400 Update README.md commit 33d367310d702667d86202aba389a0fee4eba691 Author: Kendell Clement Date: Tue Oct 23 13:24:50 2018 -0400 Update README.md commit d6c647d32cdded7803f6b023949202c9486f5caa Author: Kendell Clement Date: Tue Oct 23 12:01:41 2018 -0400 Update README.md commit 5cb8fccf048a49b3798f6932c0b0db90569697fa Author: Kendell Clement Date: Mon Oct 22 17:42:41 2018 -0400 Update README.md commit 7b60f691450c932b5d32ff48218f187328d0726f Author: kclem Date: Tue Oct 16 15:27:34 2018 -0400 2.0.16b - batch mode report commit 6037c2945efdea6fecf67b037a70c30d4bc6696b Author: kclem Date: Fri Oct 12 18:14:27 2018 -0400 2.0.15 updates to pooled, adjust merging commit 326e1c9f370e1d6956ad565605309f37e214e927 Author: kclem Date: Thu Oct 4 10:28:07 2018 -0400 2.0.14b - adjusted flash overlap params, cannot take mult aln gap penlty commit 26ec80198dc06ca09eb525deaf387c330f701cac Author: kclem Date: Fri Sep 28 15:13:00 2018 -0400 default val for n_processes commit bec0d312629d006169495b241fb9ea0018380e62 Author: kclem Date: Tue Sep 25 10:55:21 2018 -0400 2.0.13b commit 51d1386856849af7f04be0ce587e97349c7149b8 Author: kclem Date: Wed Aug 22 18:18:23 2018 -0400 Produce report commit 017c409c19a6af99c09c97faff67c26ac902e157 Author: kclem Date: Mon May 14 16:44:32 2018 -0400 Initial Commit of files commit 4324c954cc4efa10fc01fc6d69f88253ef5a7483 Author: kclem Date: Mon May 14 16:31:29 2018 -0400 first commit commit 2eff6a160159a5e64deb12f33110ee62b245f59e Author: Cole Lyman Date: Mon Mar 25 11:02:27 2024 -0600 Update reports to align with master (#20) * Update from recent changes with the CLI Most have to do with failed runs reporting * Update README to not automatically track master on new Reports branch * Remove extraneous `crispresso_data_path` commit b8dae4c8ed854f70d3dde7fdd0de396e842f8789 Author: Kendell Clement Date: Fri Mar 22 14:16:12 2024 -0600 Clean reports (#19) * clean reports * ignore README line endings * fix line endings * Remove commented out code for resizing Plotly plots when going fullscreen This would make Uncle Bob happy... * Update cup URL --------- Co-authored-by: kclem Co-authored-by: Cole Lyman commit 76b3601f6a0144f100266153f1c999e0c5de65de Author: Samuel Nichols Date: Fri Jan 12 09:56:19 2024 -0700 Squashed commit of the following: commit 603f2eff9d1aa21ae95f3e134da303b8018d3a33 Author: Samuel Nichols Date: Fri Jan 12 09:48:20 2024 -0700 fix guardrials partial commit 22fc03183a8070c30dfb74d5c23575ac19019855 Author: Samuel Nichols Date: Fri Jan 12 08:54:01 2024 -0700 Add guardrail partial commit e55f6b21972b578261bc5a864ce1d653d98f9e34 Author: Samuel Nichols Date: Mon Jan 8 07:50:59 2024 -0700 Functional guardrails, needs reports update commit 6e968e9699ed59a47d88191d03768e042d8b60a4 Merge: 32b49685 e948ce10 Author: Samuel Nichols Date: Mon Dec 18 13:34:36 2023 -0700 Merge branch 'guardrails-clean-history' of https://github.com/edilytics/CRISPResso2 into guardrails-clean-history commit 32b49685da320501dad2b0ebbb57887b66220ba8 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit 4e309cf6f732565d635de3d4c5d074ada3027e2d Author: Cole Lyman Date: Mon Dec 18 10:51:55 2023 -0700 Refactor to use CRISPRessoReports module commit e648dc087c0055bc5d2fca13c64071a371dea941 Author: Cole Lyman Date: Mon Dec 18 10:51:11 2023 -0700 Add CRISPRessoReports subtree commit e948ce107ebb0d1d99010ed12e937f34b5e607d4 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05b bb13e007 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e8 d0b41483 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f909713 b913fcb4 Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f6788 5f909713 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2d e5afa478 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f0 51a943c3 Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f0 5a10d638 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd468 442a48c7 Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. Co-authored-by: Kendell Clement commit a7ffd46822ce195d51ff4d3dba0f02fe9bc73c1e Author: Kendell Clement Date: Tue Jan 11 16:29:37 2022 -0500 Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9990790a0081b765c1f54f4a9b18db522ab4693 Author: Kendell Clement Date: Wed Jan 5 16:48:17 2022 -0500 Allow for mixed-case prime editing input, fix #187 commit 4acdcddbef3e812655b7af9def772f0bb0dc30b5 Author: Kendell Clement Date: Wed Jan 5 16:37:22 2022 -0500 Update insertion annotation for fastq_output and bam_output Fix index issue for fastq_output, add reporting of inserted bases to bam_output. Index for fastq_output is increased because the insertion happens immediately after the insertion coordinate. commit 2f84dd02787abffa6d39efbc50c82c92d1c87528 Author: kclem Date: Fri Dec 10 16:39:41 2021 -0500 Fastq_output report inserted bases when the `--fastq_output` parameter is provided, the inserted bases will be written to the output fastq file. Previously, a string like "DEL= INS=78(1) SUB= " would indicate a 1bp insertion at site 78. This update outputs strings like "DEL= INS=78(1+G) SUB= " with a plus character followed by the inserted bases. commit ba742bbfa533a672321b27b5122d7ea6658c9014 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Implement algorithmic improvements for find_indels_substitutions" This reverts commit 13414833ef6cd5b232097f6ff0325a2b8e28b214. commit 5c8e1c5de6e800c820d9ec5ffe4809737f1211de Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Add test cases for find_indels_substitutions" This reverts commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808. commit d9a072368ca991ffde52aa1ee29e17f2758ed279 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Try some additional cython optimizations" This reverts commit 38b101d57384b9f7d664ac8719c1585f5f7142a5. commit 38b101d57384b9f7d664ac8719c1585f5f7142a5 Author: Cole Lyman Date: Fri Oct 8 14:42:49 2021 -0600 Try some additional cython optimizations commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808 Author: Cole Lyman Date: Fri Oct 8 14:15:11 2021 -0600 Add test cases for find_indels_substitutions commit 13414833ef6cd5b232097f6ff0325a2b8e28b214 Author: Cole Lyman Date: Fri Oct 8 11:50:25 2021 -0600 Implement algorithmic improvements for find_indels_substitutions These changes remove the need for iterating over the alignment multiple times. commit f4b6cfc03951215c3b9019dc47beb2913a5448ab Author: Kendell Clement Date: Fri Nov 19 00:10:05 2021 -0500 Fix CRISPRessoPooled calls to deprecated pands ix commit 751fbdb4ce432f11f3fb66c5a34f7db3d1d75dc1 Author: swrosati <40470095+swrosati@users.noreply.github.com> Date: Fri Nov 12 12:33:04 2021 -0600 Update ylabel_values -> y_label_values Typo causing error in certain modes. commit 76c80713b7b8209c9399d67f718d7ade20f20d91 Author: kclem Date: Wed Nov 10 11:37:16 2021 -0500 Update dockerfile for pyparsing commit ef15caee29380f58aaae392c897fabe47587486e Author: Kendell Clement Date: Wed Nov 10 00:45:51 2021 -0500 Fix int bug for n_reads in CRISPRessoPooled commit fc1ae890712ab2dc36d5ff36c81f64a10a2c2337 Author: Kendell Clement Date: Wed Nov 10 00:34:34 2021 -0500 Spelling fix and version bump to 2.2.7 commit 08369e294d6756f727167af50f14ff75cb4864b5 Author: Kendell Clement Date: Wed Nov 10 00:30:03 2021 -0500 Add bam input and selected demuxing for CRISPRessoPooled Adds features for providing aligned bams as input to CRISPRessoPooled and for a faster demultiplexing when amplicons and genome are provided. The parameters are: --aligned_pooled_bam: Path to aligned input for CRISPRessoPooled processing. If this parameter is specified, the alignments in the given bam will be used to demultiplex reads. If this parameter is not set (default), input reads provided by --fastq_r1 (and optionally --fastq_r2) will be aligned to the reference genome using bowtie2. If the input bam is given, the corresponding reference fasta must also be given to extract reference genomic sequences via the parameter --bowtie2_index. Note that the aligned reads are paired-end seqenced, they should already be merged into 1 read (e.g. via Flash) before alignment. --demultiplex_only_at_amplicons: If set, and an amplicon file (--amplicons_file) and reference sequence (--bowtie2_index) are provided, reads overlapping alignment positions of amplicons will be demultiplexed and assigned to that amplicon. If this flag is not set, the entire genome will be demultiplexed and reads with the same start and stop coordinates as an amplicon will be assigned to that amplicon. commit 0cdcfd7c4af834f3270e61b4db4b5f6d3da10d7c Author: Kendell Clement Date: Wed Nov 10 00:24:15 2021 -0500 Remove unused var for bam_index commit 3647ace73c95a2f44e45b6b42b4f1748d3a47f4c Author: kclem Date: Wed Nov 3 09:43:28 2021 -0400 Add --min-overlap param for flash in CRISPRessoPooled commit eea442a763e0c6c41da16a15d6e11ddf6d222dc8 Author: kclem Date: Mon Nov 1 11:25:55 2021 -0400 Remove deprecated pandas .ix from WGS commit 3e6c281c931756acd26acca96249b2fa1ad1db31 Author: Cole Lyman Date: Thu Oct 21 11:10:19 2021 -0600 Add unit tests These unit tests were in the other repo, but weren't moved over for some reason. commit 50e7cb21570ddb757904728800e31c2bcbd0d060 Author: Cole Lyman Date: Thu Oct 21 11:09:38 2021 -0600 Add Makefile to run tests This makes it easy to test the integrations cases because it will automatically install the package when needed. commit de91d51bcd9b725533a45c86ae72bc27dd5d3150 Author: Cole Lyman Date: Thu Oct 21 11:28:02 2021 -0600 Replace `np.int` with `int` Apparently `np.int` has been deprecated, so in places where precision isn't important `np.int` has been replaced with `int` according to the instructions from the warning. commit cabebbefb2646967dbeee80af08ac14156b1b53c Author: kclem Date: Wed Oct 20 12:33:49 2021 -0400 Convert cols to numeric for PE commit c2bdd96651eef5af38fb7bbc11d257a827ac080d Author: kclem Date: Wed Oct 20 12:00:43 2021 -0400 Make loggers module-specific Matplolib sometimes logs verbosely to info, so this stops using the root logger commit 8196b6a81f477ddcb0e34d61dfb54085de20c1a0 Author: Kendell Clement Date: Tue Oct 19 22:00:23 2021 -0400 Fix unicode error for bam read/write commit a923a7c2ef182238bd6b8aa77289bac487b7679b Author: Kendell Clement Date: Tue Oct 19 21:59:47 2021 -0400 All sub-crispresso processes are run with 1 process commit 75205b0d41423e4f62796e9674b0edbee68a11b9 Author: Kendell Clement Date: Mon Oct 18 23:03:59 2021 -0400 Fix #161 add params for prime editing guide analysis commit ecf23ef23e5701b232bba547a6d7d4b96f085f26 Author: kclem Date: Mon Oct 18 15:02:29 2021 -0400 Add param for plotting custom plots centered at point Add parameter to plotCustomAllelePlot script: --plot_center which if set, will produce plots centered at this point instead of sgRNAs. commit 71bf32ca789dde1d44cf41b9d3b702f12336e010 Author: Kendell Clement Date: Wed Oct 13 00:48:14 2021 -0400 Update README.md commit 53197e62706e37db54f7ed50c94f38a938955e59 Author: Kendell Clement Date: Tue Oct 12 15:43:08 2021 -0400 Fix #160 plotting for cut sites in plot 5 commit 90b43eaa03c0ea0fdee62a7b244204cad50056cc Author: Kendell Clement Date: Mon Oct 4 15:45:20 2021 -0400 Remove version checks for old seaborn and numpy, fix #155 commit 5e9dedf6bd7c7b3ef998bed811760a41b5313c6e Author: Kendell Clement Date: Tue Sep 28 21:16:54 2021 -0400 Optimize get_command_output, fix gzip binary bug commit 46953c39b769a4fa43b2ef0822dd92f07c4f1b9b Author: Kendell Clement Date: Wed Sep 22 01:58:36 2021 -0400 Update expected tests for batch commit 856354aadebc8410956e724b555224a55941a618 Author: Kendell Clement Date: Wed Sep 22 01:57:59 2021 -0400 Update CRISPRessoBatchCORE.py Move parsing of batch params to after assignment of n_processes so this value is applied to sub-processes commit f7f81747207cb279fd2c0d91986c7d4e7137fde4 Author: Kendell Clement Date: Wed Sep 22 00:23:04 2021 -0400 Fix #149 bug when read is much longer than the given reference commit 798eb4322899f70e2e2a3df0fdfad9b5598d3a41 Author: Kendell Clement Date: Tue Sep 21 17:43:38 2021 -0400 Fix #150 Open fastq.gz in 'rt' mode commit fa168843e6036c6d4ec4a5d7acb097c6a1532a98 Author: Kendell Clement Date: Fri Sep 17 12:33:12 2021 -0400 Remove requirement for python 2.7 commit 80fe12efbb0ee5c321262822b237fc8220a3f2c6 Author: Kendell Clement Date: Thu Sep 9 17:41:27 2021 -0400 Print batch summaries for amplicons with only one sample Previously, batch summaries were only printed for amplicons with more than one sample to save bits. This change produces batch summaries for amplicons with only one sample, and we shift responsibility to the user for purchasing carbon offsets to compensate for the generation and storage of these redundant bits. commit 43065310bf28e6a08780222250abd0d39ff6d0c5 Author: Kendell Clement Date: Thu Sep 9 17:38:58 2021 -0400 Add parameter 'assign_ambiguous_alignements_to_first_allele' For ambiguous alignments, setting this flag will force them to be assigned to the first (as provided by the references -a first and then -e second) amplicon. Thus, no reads will be discarded as 'ambiguous' and all reads will be counted once in the analysis. Version bump to 2.2.4 commit 11eaacd3bac9ebeabad69c1d5d92f1fd1a783a17 Author: Kendell Clement Date: Fri Sep 3 22:34:59 2021 -0400 push down crispresso2_info items commit 20272e1e95305888ca744ac56af2c08554eb9f1b Author: Kendell Clement Date: Fri Sep 3 22:32:24 2021 -0400 remove mention of pickle commit 3517285473d423dc32c6e3fdbac69eecf0caa7fc Author: Kendell Clement Date: Fri Sep 3 22:30:36 2021 -0400 minor pushdown of report name in crispresso2_info commit 8129494607488bf270567d40d43e01e043699f06 Author: Cole Lyman Date: Fri Sep 3 17:31:54 2021 -0400 fixup! Move region related objects to results in crispresso2_info commit 88a82d27e840c29b95b1602ae1594bf161108979 Author: Cole Lyman Date: Fri Sep 3 16:40:40 2021 -0400 Move some objects in CRISPRessoPooled crispresso2_info objects Move the objects: - 'final_data' -> 'results'/'final_data' - 'running_mode' -> 'running_info'/'running_mode' commit b702ab3e53e88170c7517591ce7a7050ccf1c2ba Author: Cole Lyman Date: Fri Sep 3 16:36:20 2021 -0400 Move some objects in CRISPRessoBatch crispresso2_info Move the following objects: - 'completed_batch_arr' -> 'results'/'completed_batch_arr' - 'batch_names_arr' -> 'results'/'batch_names_arr' - 'batch_input_names' -> 'results'/'batch_input_names' - 'nucleotide_frequency_summary_filename' -> 'results'/'nucleotide_frequency_filename' - 'nucleotide_percentage_summary_filename' -> 'results'/'nucleotide_percentage_filename' - 'modification_frequency_summary_filename' -> 'results'/'modification_frequency_summary_filename' - 'modification_percentage_summary_filename' -> 'results'/'modification_percentage_summary_filename' commit 2416c80e952cb58fa082ead5ef719ed7eab3eeb5 Author: Cole Lyman Date: Fri Sep 3 15:39:18 2021 -0400 Move nuc_quilt related objects to 'results'/'general_plots' to crispresso2_info The following objects have been moved: - 'window_nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'window_nuc_pct_quilt_plot_names' - 'nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'nuc_pct_quilt_plot_names' - 'window_nuc_conv_plot_names' -> 'results'/'general_plots'/'window_nuc_conv_plot_names' - 'nuc_conv_plot_names' -> 'results'/'general_plots'/'nuc_conv_plot_names' commit 37e354f94980d304e1cecf11fee95e61988dd3e3 Author: Cole Lyman Date: Fri Sep 3 14:58:36 2021 -0400 Move summary objects to 'results'/'general_plots' in crispresso2_info The following objects have been moved: - 'summary_plot_names' -> 'results'/'general_plots'/'summary_plot_names' - 'summary_plot_titles' -> 'results'/'general_plots'/'summary_plot_titles' - 'summary_plot_labels' -> 'results'/'general_plots'/'summary_plot_labels' - 'summary_plot_datas' -> 'results'/'general_plots'/'summary_plot_datas' - 'summary_plot_root' -> 'results'/'general_plots'/'summary_plot_root' - 'reads_summary_plot' -> 'results'/'general_plots'/'reads_summary_plot' - 'modification_summary_plot' -> 'results'/'general_plots'/'modification_summary_plot' commit 6df988151c602602470c944ccf561cd28f2dc30c Author: Cole Lyman Date: Fri Sep 3 14:04:12 2021 -0400 Move region related objects to results in crispresso2_info This includes: - 'regions' -> 'results'/'regions' - 'all_region_names' -> 'results'/'all_region_names' - 'good_region_names' -> 'results'/'good_region_names' - 'good_region_folders' -> 'results'/'good_region_folders' commit 053691311b158a2d3620670d20983d546a9c2c7b Author: Cole Lyman Date: Fri Sep 3 13:44:46 2021 -0400 Move 'samples_quantification_summary_filename' to 'results'/'alignment_stats'/'sample_quantification_summary_filename' in crispresso2_info commit eda3d50b6fafde4ea3145980cb2010417b799d88 Author: Cole Lyman Date: Fri Sep 3 13:27:42 2021 -0400 Move 'finished_steps' to 'running_info'/'finished_steps' in crispresso2_info commit 1da829c1c3cfd2bda9aaccca0aaa28c78a8f7271 Author: blasvicco@gmail.com Date: Mon Aug 30 17:48:02 2021 +0200 BugFix: database_id referenced before declared. commit c9321cc2cfcac34e4b6d8e1aa9fde9f25382e2de Author: Kendell Clement Date: Sat Aug 21 00:15:52 2021 -0400 Version bump to 2.2.2 for some reason the last release didn't pick up the last commits. commit 3aadab69ef1e3a44f12ee277a7767648e943a787 Author: Cole Lyman Date: Fri Aug 20 12:09:49 2021 -0400 Update filterFastqs so that the numpy arrays are writable commit 7ca129d2471aeebef2ba6d2dadf36265810f1a5c Author: Kendell Clement Date: Fri Aug 20 02:00:42 2021 -0400 Update CRISPRessoPlot.py Undo attempted matplotlib deprecation warning fix commit 8e8a65c0f29b6f99662ac1d272aa7199871f5cf5 Author: Kendell Clement Date: Fri Aug 20 01:26:50 2021 -0400 Plotting bug fixes Display and plotting fixes for batch report labels, and general plots, as well as a matplotlib deprecation fix commit 3f114752c5eca1554fcebf044b604b60a3eebeae Author: Kendell Clement Date: Fri Aug 20 00:06:38 2021 -0400 add batch summary of frameshift/splicing Closes #116 by adding frameshift/splice summary to batch Version bump to 2.2.1 Fix unicode error in slugify commit 5ad9fbc6645475885fc0f35bc26afd353a2b3d5f Author: Cole Lyman Date: Mon Aug 16 14:16:13 2021 -0400 Read plain text fastq as bytes in filterFastqs commit 6c20f28678469875392e3fc8946018b53a00256b Author: Kendell Clement Date: Mon Aug 16 13:35:12 2021 -0400 Remove test for seaborn version commit cec965feff65b4228d42784e498398b73b8caa49 Author: Cole Lyman Date: Mon Aug 16 12:16:04 2021 -0400 Fix filterFastqs This commit fixes the filterFastqs script by properly casting to strings (from bytes) in the appropriate places. It also replaces the deprecated `np.fromstring` function with `np.frombuffer`. commit 3c30e56a15fa15a3537c3d51e429c1ff8738d6fe Author: Cole Lyman Date: Thu Aug 12 09:57:50 2021 -0400 Add .gitignore file commit 2461e30770d5ae8a6686530981a4bf5c913e2f68 Author: Cole Lyman Date: Thu Aug 12 09:50:01 2021 -0400 Decode result from Popen from bytes to string This is done only where a string is necessary, the instances where this is not done are where the result is cast to a float or int (because they can accept bytes as input directly). commit 64ec8bacf9ae8cb0d3a9093ad12a916983f32207 Author: Cole Lyman Date: Sat Aug 7 13:49:05 2021 -0400 Refactor `results/refs` and `results/ref_names` crispresso2_info fields commit ebb33a7d691b524ed7ab92b5a870da09df045e00 Author: Cole Lyman Date: Fri Aug 6 20:32:03 2021 -0400 Refactor `results/general_plots` crispresso2_info fields commit 94768e209e2424394efb4d902aac4f0e4268aad3 Author: Cole Lyman Date: Fri Aug 6 14:58:25 2021 -0400 Refactor `results/alignment_stats` crispresso2_info fields commit a58b7a2b40bc99346a9ea8f6240034963b3d6b31 Author: Cole Lyman Date: Fri Aug 6 14:10:33 2021 -0400 Refactor `running_info` crispresso2_info fields commit a92ea4687e0cc69844c5d4a6250757540076ed59 Author: Kendell Clement Date: Thu Aug 5 14:28:29 2021 -0400 Version bump to 2.2.0 commit 20e9b6f228949886ebe592d4542aaddbb9fb123a Author: Cole Lyman Date: Thu Aug 5 11:52:22 2021 -0400 Remove argparse dependency Remove the argparse dependency from setup.py because argparse is a standard module in Python 3. commit ecf70ea4379e079e7669fc5ed8baabdcd11a8c61 Author: Kendell Clement Date: Fri Jul 30 01:23:38 2021 -0400 Update Dockerfile bowtie throws an error 'Can't locate Sys/Hostname.pm in @INC' This fixes it. commit 30fb34e17787858d4218eb0b0fcc1baf6259ee23 Author: Kendell Clement Date: Fri Jul 30 00:39:48 2021 -0400 Ignore python egg for copying, pin tbb for bowtie error https://github.com/BenLangmead/bowtie2/issues/336 commit c850abb672019a3684a544fccde9550f7eeb86f5 Author: Kendell Clement Date: Thu Jul 29 20:13:32 2021 -0400 Update config.yml commit eb70cb18d6910f418bb8052f45b58c05c397af43 Author: Kendell Clement Date: Thu Jul 29 17:47:06 2021 -0400 Update config.yml commit 01f079fd663027574115a0af974564d5b5efbe97 Author: Kendell Clement Date: Thu Jul 29 17:22:48 2021 -0400 Update CRISPResso2_router.py commit 2e3b5d42b8ea6705dbd2c2f59312b93390083da3 Author: Kendell Clement Date: Thu Jul 29 17:20:16 2021 -0400 Update Dockerfile commit 46d8c44f2dbe7a67531d3ccae6b0a3c51b12b9ca Author: Kendell Clement Date: Thu Jul 29 17:07:02 2021 -0400 Update Dockerfile commit 0999e82dd8e184a6b0aefa7a35fc9decaa1fc20d Author: Kendell Clement Date: Thu Jul 29 16:59:47 2021 -0400 Update Dockerfile commit 34b93cfeeb95d0a1375378996e3ca29e79fe5311 Author: Kendell Clement Date: Thu Jul 29 16:50:20 2021 -0400 Python 3 commit e040ac488b050d6f316d21196de002ef09383915 Author: Kendell Clement Date: Tue Jul 27 10:04:43 2021 -0400 Add testing file for batch, remove debug print lines commit aa7b9998c4660438f4303a57386f01617ddec1bb Author: Kendell Clement Date: Thu Jul 22 10:43:52 2021 -0400 Update Batch to allow for max processes usage commit 1566a1110215e32f338497040f1279c3581b6dcd Author: Kendell Clement Date: Wed Jul 21 01:02:24 2021 -0400 Fix reference to BadParameterException commit fea5017109ab56c955b8053eebad5f6f4218bc65 Author: Kendell Clement Date: Mon Jul 12 16:11:05 2021 -0400 Allow spaces in run names, print run name to report commit b8594354c96131ba33c9277fb719c95b1cf72f3d Author: Kendell Clement Date: Tue Jun 29 14:12:03 2021 -0400 Update CRISPRessoPooledCORE.py Fix debug statement commit 9bc9b9959cbc8faf9dc462669e333298a80a71d1 Author: Kendell Clement Date: Tue Jun 29 14:09:13 2021 -0400 Update CRISPRessoPooled for samtools sort status updates Redirect samtools sort status updates to log file. Version bump. Previously this would cause an error: `ERROR: list index out of range samtools view: writing to standard output failed: Broken pipe samtools view: error closing standard output: -1`. commit ad5b9d853ac3559e58a5ad569e7d3ac9c3d8a719 Author: Kendell Clement Date: Thu Jun 24 12:10:31 2021 -0400 Allow max processes for CRISPRessoPooledWGSCompare Users can provide -p max to use all processes available for comparisons commit 48d6c8724f541b285e5d6889723cc37fc99e5cfa Author: Kendell Clement Date: Wed Jun 23 20:53:51 2021 -0400 Create html report for CRISPRessoComparePooledWGS commit abe3054dd9b95f51cbf34e3c37e088f6beb8287c Author: Kendell Clement Date: Wed Jun 23 20:53:22 2021 -0400 Fix allele plot bug #103 If no regions are returned, max of a pandas dataframe returns an error because the df is empty commit 1ccb507858d92516e28286358d3aae9cce2cf7ea Author: Kendell Clement Date: Wed Jun 23 20:51:16 2021 -0400 Fix command prompt logo lines commit 293c249c0f380576121a123dec982e40409c977e Author: Kendell Clement Date: Sun Jun 20 00:26:34 2021 -0400 Update plotCustomAllelePlot.py Get rid of debug statements commit 947fbab70e0f4fa78cc43273b4fe2be5225043cc Author: Kendell Clement Date: Sun Jun 20 00:22:56 2021 -0400 Add plotCustomAllelePlot script for replotting allele plots commit 167a48659684ce1669da915ddb6995935c6a1aa6 Author: Kendell Clement Date: Wed May 26 11:16:51 2021 -0400 Update README with explicit instructions to activate conda environment commit af0b7e441d2a4f1e035fabfd353c58784f688371 Author: Kendell Clement Date: Sun May 23 00:22:00 2021 -0400 Update test results for more flexible pooled alignment The relaxing of bowtie parameters allowed more reads to align, which changed the expected test results for CRISPRessoPooled commit 0e08cd05c2f3279ac95a068be76f1d36a4b0224d Author: Kendell Clement Date: Sun May 23 00:06:21 2021 -0400 Fix double rows in Alleles_frequency_table due to read directionality Previous to this fix, forward and reverse reads having the same sequence would appear in different rows in the Alleles_frequency_table. This fix doesn't change counts or results, just combines the two rows in the Alleles_frequency_table. commit 7d2d761a3d4c25915be0d27b36f3b4a87068587d Author: Kendell Clement Date: Thu May 20 16:05:27 2021 -0400 CRISPRessoPooled - get rid of -k bowtie2 parameter The -k 1 parameter may not yield the best alignment. This commit gets rid of that parameter. commit 44dc9e75c660f4b3b683f1c80f5a964aa55e75bd Author: Kendell Clement Date: Wed May 19 22:52:46 2021 -0400 CRISPRessoPooled alignment more tolerant When given a genome file, CRISPRessoPooled aligns reads to the genome using the Bowtie2 aligner. The legacy parameters were somewhat strict. The new parameters reflect the 'default_min_aln_score' parameter in allowing for substantially more indels and mismatches than previous. The parameter `--use_legacy_bowtie2_options_string` has been added to use the legacy settings. Otherwise, the bowtie2 alignment settings will be calculated as follows: commit 84b870ce0d489295501aa57711edd6b18c011b92 Author: Kendell Clement Date: Tue May 4 10:22:22 2021 -0400 Raise exception if quantification_window_coordinates looks like an int Previously, if quantification_window_coordinates looks like an int when pandas parsed a batch file, it would throw an error trying to split the int. Now an exception will be raised. commit e25a6dbb13fcd0565f4c81e3ea42cfd115aa1bc0 Author: Kendell Clement Date: Mon Apr 26 16:19:00 2021 -0400 Fix bug if all reads are discarded If all reads are discarded, CRISPResso would fail because the df_alleles is empty. This adds discarded alleles to the df_alleles. commit 156d7d640355ad4755c6ce4cbab1b75bf31677c9 Author: Kendell Clement Date: Fri Apr 9 13:24:51 2021 -0400 CRISPRessoPooled - close active file in demux commit c5d9fcbbf5bd9b307c9050498d08e72dd98d42aa Author: Kendell Clement Date: Fri Apr 9 12:29:02 2021 -0400 Keep old awk command for speed for samples with <50 amplicons commit c7539661fc5bd29f4f6ee1e9c7795be932b53a8e Author: Cole Lyman Date: Fri Apr 9 12:18:45 2021 -0400 Alt pooled processing implementation (#87) These changes implement separating reads to their corresponding amplicons via Python instead of through awk. This is to get around the maximum number of open files that is limited on many operating systems. Co-authored-by: Kendell Clement commit e57d98738fa832766c78dc7eecfdb0588d92634d Author: Kendell Clement Date: Thu Apr 8 12:36:43 2021 -0400 Alt pooled processing implementation commit 4598226837cd4b726ae38f50958f977bde4794ca Author: Kendell Clement Date: Sun Mar 21 00:16:19 2021 -0400 HDR Updates - yw #82 Multiple amplicon names are resolved before adding the HDR amplicon -- unnamed amplicons are named Amplicon{i} for each amplicon. Plot 4g data (nuc pct table, mod pct table for all reads aligned to the first reference) is output and linked to from the plot display Ambiguous reads don't contribute to plot 4g data (which would otherwise lead to double counting and pct values > 1) commit d7915b2c561173238c6b0e82863f76a783db7c4d Author: Kendell Clement Date: Sat Mar 20 01:12:55 2021 -0400 Fix #72 bam_input error commit 32a9e98ffc6ceb1dd56e5e29c369176ed788c985 Author: Kendell Clement Date: Sat Mar 20 01:01:17 2021 -0400 Prime editing, fastq_out updates Prime editing input parameters are forced to be in the RNA 3'->5' direction. This makes sure that the scaffold incorporation happens on the correct side of the extension sequence. Errors are thrown if improper directionality is detected. Fastq_out now includes alignment scores and details for every run (it may be time to upgrade that SSD to hold these new fastq output files, but it makes debugging particular reads a lot easier!) Update to linked data for plot 3b in report commit c1b6ea2ecebd920ea12ab91f2d50e35c274f94fe Author: Kyle McChesney Date: Wed Mar 10 13:40:44 2021 -0700 fix missing import path on NaN commit 1b24ca351f4337e0a7bece7ef7addb4558f950d8 Author: Kendell Clement Date: Sat Feb 20 22:57:07 2021 -0500 Update links in readme to https - fix #79 commit d550fd1db8ce0e1d11ed77fd082c53a3d887bbc2 Author: Kendell Clement Date: Fri Feb 12 16:57:37 2021 -0500 Insertion quantification change + fix #76 Starting in version 2.1.0, insertion quantification has been changed to only include insertions completely contained by the quantification window. To use the legacy quantification method (i.e. include insertions directly adjacent to the quantification window) please use the parameter --use_legacy_insertion_quantification commit 798f661031236ee1aa5611f491ca135ec0432dc9 Author: Kendell Clement Date: Thu Jan 14 23:51:29 2021 -0500 Allow for int-looking chromosome names in WGS input file In CRISPRessoWGS, the region file contains a 'chr_id' column which is sometimes mis-recognized as ints when read by pandas if using the chromosome notation without 'chr' (e.g. 1,2,3 in stead of chr1,chr2,chr3). This bug fix forces chr_ids to be read as strs. commit 92c008673690a3cd32e33fe8cfcf07060cf6cb0f Author: Kendell Clement Date: Wed Dec 30 23:48:32 2020 -0500 Update README.md commit 8d590c5588d93ef0129512a5156fe3b45e2d3c4b Author: Kendell Clement Date: Wed Dec 30 23:46:36 2020 -0500 Introduction of CRISPRessoAggregate to aggregate stats across runs Adds CRISPRessoAggregate Adds start/end time to CRISPRessoBatch info Started removing pickle dependencies from Pooled and Report commit c8447246ada2758831a870f30ed99c496c18feb1 Author: Kendell Clement Date: Sat Dec 26 10:52:12 2020 -0500 Output alignment details for unaligned reads in fastq_out or bam_out commit dedc36cadc64e9302d88c3472652d2a4a2385d25 Author: Kendell Clement Date: Tue Nov 24 13:45:34 2020 -0500 Add scripts folder for one-off analyses commit 88b6e9c50c6d97da357534c67aaeba64c00ddc23 Author: Kendell Clement Date: Sat Nov 14 02:46:36 2020 -0500 Fix plot window cloning from Ref1 to HDR Plot window for sgRNA will be the same length after cloning even if the window is shorter or longer after comparing between ref1 and HDR. commit 7403a46e186c4b70ad606dbb0eebbbde684fac61 Author: Kendell Clement Date: Sat Nov 14 01:52:55 2020 -0500 Frameshift plot and HDR quant window updates Frameshift plots don't show 0-bp changes (these dwarf all other changes). The number of reads not shown are added to the legend. Addressed cloning quantification windows when bases are inserted in the clone-ee (previously these cloned bases would be ignored. Force HDR to clone all quantification windows from Ref 1 Fix #60 and #59 commit f3f9c122cc25ab62bdd7f30fb9d6ee4c9ab820a8 Author: Kendell Clement Date: Sat Nov 7 23:55:55 2020 -0500 Update histogram x-limits, caption, and data commit e9332f7eabed32e65430b44677c841dd0150ac5a Author: Kendell Clement Date: Sat Nov 7 02:26:59 2020 -0500 Fixes for when no reads align #63 commit 9fae60a57d99238e4f7a9ad2e992ae71cca49c60 Author: Kendell Clement Date: Sat Nov 7 02:08:59 2020 -0500 Standardize pie plot appearances commit f1738bd17b496a31d6f68a91ed7ae67a36f8f178 Author: Kendell Clement Date: Sat Nov 7 00:36:40 2020 -0500 plotting and pe fixes - Special bonus for y'all to keep you company during covid - axis ticks on most plots! - added parameter --plot_histogram_outliers to plot all insertion sizes in histogram - all insertion sizes are reported in .hist output files #64 - add HDR reference plot (may change this later to set ref1 to the longer reference of WT/HDR but for now it is always WT..) Allow reverse complement of extension seq if PE sequence is specified. commit 5a2d9271b3ea99342f22e59259149d30dc193e47 Merge: bd03668e 9812651c Author: Kendell Clement Date: Wed Oct 21 16:52:12 2020 -0400 Merge pull request #62 from matandro/patch-1 Fix a bug when generating compare plot commit 9812651c2aae1218393e8ce0d734812247cd59ab Author: Matan Drory Retwitzer Date: Wed Oct 21 10:45:01 2020 -0700 Fix a bug when generating compare plot Related to issue #61 This happens when N_ROWS < 1 which I assume has something to do with no results -> negative control commit bd03668ef1626a54e0b5bfbc010afacee60f45e3 Author: kclem Date: Fri Oct 2 17:39:52 2020 -0400 delete merging intermediate files commit 3c9b1344e19dee04b169c0860bc11304d6a594b5 Author: Kendell Clement Date: Wed Sep 30 23:51:08 2020 -0400 Version bump to v2.0.42 commit 2f8c6f9c7d31b276ff18000d579ef722f693c433 Author: Kendell Clement Date: Wed Sep 30 23:44:02 2020 -0400 Update new parameters, fix docker biuld problem commit f130270135b84e0904e0003858c263285834b6dd Author: Kendell Clement Date: Wed Sep 30 14:18:58 2020 -0400 Fix bug in read counting for interleaved fastqs commit faac4e1045e5b617b3743e328c0203ced27e3f31 Author: Kendell Clement Date: Thu Sep 24 22:37:50 2020 -0400 Fix bug in mode to write fastq out commit 388e8a97fc18648dd28e35063cb12680887395e2 Author: Kendell Clement Date: Wed Sep 23 23:49:44 2020 -0400 Update postrun reference output file Rename postrun references file to be more standardized with other output files. Output is now "CRISPResso2Pooled_postrun_references.txt" commit ee5be76e5e24e494abd93940622d51faa83a2371 Author: Kendell Clement Date: Wed Sep 23 23:32:34 2020 -0400 Multiple allele support for pooled mode Most common alleles for each pooled target are output if the flag '--compile_postrun_references' is provided. This writes alleles with frequncy defined by the parameter to --compile_postrun_reference_allele_cutoff This file can be manually edited to remove noisy alleles, and then used to run CRISPRessoPooled again but to provide alternate alleles to each CRISPResso run by using the parameter '--alternate_alleles'. This is particularly useful in cases where control experiments are available. The running pattern would be: 1) CRISPRessoPooled --compile_postrun_references {control} 2) CRISPRessoPooled --alternate_alleles {produced in step 1} {control} CRISPRessoPooled --alternate_alleles {produced in step 1} {experiment} commit fe5bee2ac7ca4a8dd165e2c7305a1c41a79b8d9d Author: Kendell Clement Date: Wed Sep 23 23:27:37 2020 -0400 Output + PE updates Add parameter '--fastq_output' to output a fastq file with annotations for each read. Also, the substitution frequency table is only produced in base editor mode -- in an effort to slim down CRISPResso2 output files. Also, especially for long Prime Editing insertions or deletions, the inference algorithm may incorrectly infer the prime-edited allele. I added the parameter '--prime_editing_override_prime_edited_ref_seq' which can be used to specify the prime-edited sequence manually. commit ca61c483a8a63fec2e85889f554648ea6f3a903c Author: Kendell Clement Date: Sat Sep 5 16:15:16 2020 -0400 update report caption for fig 10 commit 346c6f6e62097ea7690204cfe879ebb918fb244d Merge: e52cb734 44ccc5ec Author: kclem Date: Fri Sep 4 15:05:02 2020 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit e52cb73471f4538ecd98f943a38771f0d07a4476 Author: kclem Date: Fri Sep 4 15:04:48 2020 -0400 Update on help message for mark wildtype allele commit 44ccc5ec3ff6a57d265807b559010512f053d82a Author: Kendell Clement Date: Fri Sep 4 14:59:05 2020 -0400 Update to plotting quantification window plot vertically centered, pooled/wgs plots are not limited in height to allow analysis of large pools. commit ff675b12196593ceb8e8d8b58dfd6a8ca8865501 Author: Kendell Clement Date: Mon Jul 27 09:55:30 2020 -0400 Update parameters and description in README commit 25c6a1b45ef1e1c1243057b9b3ee06f638d55d26 Author: Kendell Clement Date: Sat Jul 25 00:29:14 2020 -0400 Update CRISPRessoBatchCORE.py If amplicon sequence is empty, auto mode is run for batches commit 2978a6ebc0cd977b36b4ea7e1965f5c43b46d325 Author: Kendell Clement Date: Thu Jul 23 08:46:01 2020 -0400 Removed WGS logging For some reason, piping output breaks multiprocessing blocking commit 8bc9214ba0a1b628b69a49a2cf306bf559e008b3 Author: Kendell Clement Date: Wed Jul 22 22:58:51 2020 -0400 Update CRISPRessoWGSCORE.py commit a9484fd481bfa8ad716c6326aba9c57f5271f885 Author: Kendell Clement Date: Wed Jul 22 14:24:38 2020 -0400 Add parameter discard_guide_positions_overhanging_amplicon_edge If run with param -- discard_guide_positions_overhanging_amplicon_edge, for guides that align to multiple positions, guide positions will be discarded if plotting around those regions would included bp that extend beyond the end of the amplicon. Normally this would cause CRISPResso to fail if plotting were requested beyond the end of an amplicon. commit 8d26edeed89e33451bd539c242f7ffaa560db3e6 Author: kclem Date: Wed Jul 22 13:58:10 2020 -0400 WGS doesn't print crispresso output to screen WGS printing error fixed commit 5ed03059d09aaf8a416c5fec553801eeca73355f Author: kclem Date: Tue Jul 21 00:00:40 2020 -0400 bam processing update of cached not-alignments commit cf593183d7b3d8200ec295ebcc653e518775046a Author: Kendell Clement Date: Thu Jul 16 21:49:21 2020 -0400 Fix naming of scaffold-incorporated reference links on plots commit 676ff623373b0cabf992017bd5eca255c4073d2a Author: Kendell Clement Date: Fri Jul 10 00:02:59 2020 -0400 Prime editing updates prime editing guides are shown as input on report output file names are produced without spaces commit 9de370148b2ada7210c10532247b3fa4b18a76de Author: Kendell Clement Date: Tue Jul 7 20:46:30 2020 -0400 Update batch for multiple quant window input commit 6ad1822f478d7f108b92f25378cc3ddf8dc3a2d3 Author: Kendell Clement Date: Wed Jul 1 16:26:38 2020 -0400 Bam processing + Prime Editing updates -Input can now be read from bam using the parameter `--bam_input` and (optionally) `--bam_chr_loc` to use the reads in the bam at this location as input. An output bam is produced with an additional soace-separated field prefixed by c2 (e.g. c2:Z:ALN=Inferred CLASS=Inferred_MODIFIED MODS=D47;I0;S0 DEL=56(47) INS= SUB= ALN_REF=TTGGCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGAAGTAGGGCCTTCGCGCACCTCATGGAATCCCTTCTGCAGCACCTGGATCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCT----------------------------------------------- ALN_SEQ=ACACCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGA-----------------------------------------------TCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCTGGAGCGGCGGCTGCACAACCAGTGGAGGCAAGAGGGCGGCTTTGGGC). Note that the alignment details (location, cigar string, etc) are not modified.. this may be done in the future). Bam file input cannot be trimmed or pre-processed with quality filtering. -Prime editing scaffold incorporation is now more accurate (looks for the scaffold sequence at the expected position directly after the extension sequence). A plot showing the number of bases matching the scaffold, as well as insertions after the extension sequence, and a data file with these numbers is produced. Added parameter `--prime_editing_pegRNA_scaffold_min_match_length` to define the minimum length required to classify a read as 'Scaffold-incorporated' -Renamed split_paired_end parameter to `--split_interleaved_input` for interleaved input -Auto mode now considers 5000 reads to detect amplicon sequences -Add new paramter `--annotate_wildtype_allele` to annotate wildtype alleles on the allele plots -Update output when reporting missing files -- only lists first 15 files in the current directory and directory of input parameter --reference https instead of http commit c0f2871befed86c4b314100584bf844f13d71d0e Author: Kendell Clement Date: Tue Jun 2 18:54:52 2020 -0400 Update CRISPRessoWGSCORE.py Remove debug print commit e5450b1cc6be518706969d27bda843cb7c16b082 Author: Kendell Clement Date: Tue Jun 2 18:53:20 2020 -0400 Updates for Pooled and WGS WGS gene annotations compatability fix and pooled gzip fix commit c2b286dbda3807752f34cdf6274e41d5640b408f Author: Kendell Clement Date: Tue May 12 00:33:36 2020 -0400 Fix docker bug, print version to Pooled + WGS commit 2a06cc18c3157e6c135dae7ce53adec94fe8a83f Author: kclem Date: Sun May 10 00:09:55 2020 -0400 Fix plotting bug if no sgRNAs given commit 1e3ca605e0c5ae65e8acc727667f952bc4c0d3ee Author: kclem Date: Sun May 10 00:00:32 2020 -0400 flexiguide fix commit 4e1d6b2b3424e725010e3e1a13522a7386228853 Author: Kendell Clement Date: Sat May 9 23:30:39 2020 -0400 Prime editing refinement and Pooled filesystem demand reduction - Prime editing parameters renamed, nicking guides match with flexibility - Prime editing extension seq is shown as a guide (with no cut site) - Prime editing summary plot included in report - Nucleotide plots are shaded when no changes from the reference sequence - sgRNA annotations are plotted on multiple lines if they overlap - N's don't count as substitutions - extended read analysis data available with --write_detailed_allele_table flag commit 8c584719b9771c01e53cfe409789b29a29fad665 Merge: f5069365 7624815e Author: Kendell Clement Date: Sat May 9 23:14:30 2020 -0400 Merge pull request #42 from ronaldhause/patch-1 ZipFile: set allowZip64=True to write larger allele frequency tables commit f5069365d37bd698ff3bf35e5c39a1b75e10dc1d Author: kclem Date: Sat May 9 12:04:10 2020 -0400 CRISPRessoPooled updates, fix #37 for too many files Demultiplexing in the case of amplicons + genome is parallelized to reduce sorting Only files with sufficient reads are demultiplexed and written Additional output file REPORT_READS_ALIGNED_TO_GENOME_ALL_DEPTHS.txt shows all alignment locations commit 7624815e3159d926d8a0710f674f44c616b68bd5 Author: Ron Hause Date: Sun May 3 22:50:07 2020 -0700 ZipFile: set allowZip64=True to write larger allele frequency tables Addresses terminating ERROR: Filesize would require ZIP64 extensions when trying to write compressed allele frequency tables > 2 GB commit 6af15a09033166b713df159a6ef850dde8867253 Author: kclem Date: Tue Apr 28 03:28:41 2020 -0400 int bug fixes commit 531753c0f5be89c255f65d876cba5e9bf00dd4a2 Author: Kendell Clement Date: Tue Apr 28 02:00:37 2020 -0400 Introduces support for prime editing, multiple window sizes and offsets, max processors commit 8e29c1e0966ebb2073d52a221ae77e56bb146431 Merge: adb0d8b7 039013ef Author: Kendell Clement Date: Fri Apr 24 13:06:46 2020 -0400 Merge pull request #41 from natecarlson/fix-name-error If the name column is called 'name', refer to it as 'name', not '#name'. commit 039013efe363603dfe89f10b857d58bb1ef8e8d9 Author: Nate Carlson Date: Fri Apr 24 09:46:21 2020 -0500 If the name column is called 'name', refer to it as 'name', not '#name'. commit adb0d8b791686846fa8522f46e064418cbfbdc1c Author: Kendell Clement Date: Mon Apr 20 16:26:32 2020 -0400 Update LICENSE.txt commit b514a68eea135e805333c67bf33bf0f1ea41a034 Author: Kendell Clement Date: Sun Apr 19 00:36:26 2020 -0400 Print CRISPResso command on multiprocessing fail commit 4bfadd08d30e1a8b926757c1af4a9c0c0dc0b484 Author: Kendell Clement Date: Sun Apr 19 00:03:58 2020 -0400 Index fix for crispresso multiprocessing Indexes were incremented for user enjoyment (1-based) but the more accurate approach is 0-based Also, no_rerun ignores changes to the flags 'debug' or 'n_processes' which shouldn't affect the outcome commit 867e692b326acd19ed5f291bd5699f5885b1d569 Author: kclem Date: Thu Apr 16 00:25:47 2020 -0400 Allele plot sgRNA labels stay on plot commit b0d89b4effc4242ec55ed4a3e20e8835a90e3588 Author: kclem Date: Wed Apr 15 23:58:46 2020 -0400 WGS fastq seqs are now uppercase, so guides match even in lowercase-masked genomes commit 8098f1f1dda6efdaf773b9f85622d79f32ac49c9 Author: kclem Date: Thu Apr 9 01:11:26 2020 -0400 Pooled multiprocessing updates commit 034ff2b5858dd29ab60017835695fad527a5213e Author: Kendell Clement Date: Tue Apr 7 02:16:39 2020 -0400 add n_processes param for pooled analysis commit 7fb477ff15c7f8d62d3acad4d14434942cd25bff Author: Kendell Clement Date: Tue Apr 7 00:36:47 2020 -0400 Pooled Set flag to skip reporting problematic regions commit 6c3aeff8b96d918ef2b3c518b73860d3f74480b8 Author: Kendell Clement Date: Mon Apr 6 19:56:56 2020 -0400 Pooled parallelization by chr Parallelized CRISPRessoPooled extraction to operate by chr Attempt to appease the dockerhub requrements -- require cython for compilation commit a66c4020f473d2dee80fa1257c04049b2ba6dbd3 Author: kclem Date: Fri Apr 3 13:41:38 2020 -0400 v2.0.33 plot updates allele plot sgRNAs that extend beyond plot are marked quantification window shading and right-side correction version commit 90e677f453ef971b478e4582120b17cbb572212c Author: Kendell Clement Date: Fri Apr 3 01:30:57 2020 -0400 Pooled bug fixes for regions with the same location and different names commit d795479d117e689a6679ffe463df413bff2f6a5a Author: Kendell Clement Date: Fri Apr 3 01:23:30 2020 -0400 Fix open error for docker commit 9aee866e5f65bf8df6495e81684651436a0b0a30 Author: Kendell Clement Date: Fri Apr 3 01:17:09 2020 -0400 Parallelization of Pooled and introduction of checkpoints for WGS and Pooled Alignment of amplicons is done in one bowtie2 call instead of one bowtiecall per amplicon Parallelize several time-intensive steps in Pooled (splitting by region, etc) --no_rerun flag will skip already-processed steps in Pooled and WGS commit fb1e7e25404bd80722824755c1b4ff4478449c48 Author: Kendell Clement Date: Thu Apr 2 01:34:52 2020 -0400 WGS updates - multiprocessing and no rerun WGS multiprocesses extraction of reads across multiple cores. WGS extraction doesn't occur if the --no_rerun flag is set and the files are all present. commit 45d4377f678fceeff83d60efa22dbd1078e6840e Author: Kendell Clement Date: Tue Mar 31 10:35:43 2020 -0400 Remove biopython for fastq parser Living life on the edge -- dropping the biopython fastq parser to remove biopython package dependency. This will discard the minimal error-checking provided by the biopython package. commit d52f69c9fa16ea38e919f9e3dc70fb6d53246610 Author: kclem Date: Tue Feb 25 17:05:08 2020 -0500 Pin dependency versions commit 86ff4bebcfcaa5c618ff124822ecb45de2b7c9ca Author: kclem Date: Tue Jan 28 16:17:57 2020 -0500 get rid of test for CRISPRessoCompare commit b7e96d6f4d1e0966cc0847b620349ee7fe21f43f Author: kclem Date: Tue Jan 28 15:26:20 2020 -0500 Fix problem with circleci testing Switch columns for output quantification files so that total reads is before the number of aligned reads commit ac976074c03967a386ed386d9ce3436c5fa1f2d5 Author: kclem Date: Fri Jan 24 14:02:14 2020 -0500 Version 2.0.32 - guide updates and other updates Introduced flexiguides - can match with flexibility to the reference sequence - useful for pooled screens of offtargets (parameter --fg or --flexiguide, with --fg or --flexiguide_homology to control how many mismatches) Flexiguide mismatches and other mismatches are shown on plots sgRNAs can be labeled (parameter -gn or --guide_name) sgRNA position shown on allele frequency plots detection of dsODN -- shown in plot 1d (parameter --dsODN) CRISPRessoPooled gene set input flexibility - more formats accepted plot 8 shown on html report commit ca9273377acbabf01f97f04a028d4fd87a09d6b5 Author: Kendell Clement Date: Fri Dec 6 15:14:38 2019 -0500 Fix docker bug #30 commit a3ae575f870ace49a459447e13288cfd50487e2f Author: kclem Date: Wed Oct 2 20:57:03 2019 -0400 remove debug statements commit 53d70eb83bad2aa8456b744dd29611a788f3dbbd Author: kclem Date: Tue Oct 1 15:41:39 2019 -0400 Fix #25 to accept bt2l bowtie2 index extension commit 039cc8e1b4a145e53cf06c1d52f102497928f3ac Author: kclem Date: Wed Sep 25 17:53:49 2019 -0400 v2.0.31 CRISPRessoPooled chr names fix, allele plot colors CRISPRessoPooled compatability with chr names with underscores (alternate scaffolds) Additional function to plot allele table with custom set of colors for a completed run commit c35d0151efcd70a6044399e16c841ee9d0ad0535 Merge: 491247e1 b1d1518a Author: kclem Date: Thu Sep 19 16:23:13 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 491247e1a07e2991a74341b4424c7c722a3db6a9 Author: kclem Date: Thu Sep 19 16:22:56 2019 -0400 updates to command line output, batch bug commit b1d1518a7ed4b72e92fd9d10586ccce653585630 Author: Kendell Clement Date: Fri Aug 23 11:26:53 2019 -0400 Update conda installation path commit cdfd78772de27bdb78a380e591d8857acd143562 Author: kclem Date: Tue Aug 13 11:24:58 2019 -0400 Use python 2.7 pandas commit 1417d1aa387d45f37953b93304a545e08943cc45 Author: kclem Date: Tue Jul 16 17:18:28 2019 -0400 force merge reads and fix #19 add optional param for force merging R1 and R2 in case they don't cover the entire amplicon fix labels for expected amplicon plots commit 60f90053ab65958871402b609d0b72b31021bc6b Author: kclem Date: Tue Jul 2 17:28:27 2019 -0400 v2.0.30 case-insensitive guides, fix #17 commit 702b9dce89e070d96064db314ac1c5155fedd42e Author: Kendell Clement Date: Tue Jul 2 16:18:17 2019 -0400 Update README.md commit 5a10eb9a9d24371d2bedf8b173f9c7401d7cbd53 Merge: bc7b3321 bc006c82 Author: kclem Date: Tue Jun 25 15:32:51 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit bc7b3321e1bb6b7ed0c8af48d609e86e55081f6b Author: kclem Date: Tue Jun 25 15:32:45 2019 -0400 plot window bug update commit bc006c826f338b9a1fcaec2286b506bdd2c548db Merge: 1f7171b8 feee2c2e Author: Kendell Clement Date: Thu Jun 20 00:56:45 2019 -0400 Merge pull request #16 from PEHGP/patch-1 args.trimmomatic_command for CRISPRessoPooled commit feee2c2eb20807933376f01a88102767ce5e42e4 Author: kuan <396777306@qq.com> Date: Thu Jun 20 09:44:32 2019 +0800 args.trimmomatic_command args.trimmomatic_command commit 1f7171b8eb2dcd63fada80fa6cac561e576f727c Merge: f6c9eed2 3d4a37a6 Author: kclem Date: Thu Jun 13 13:13:33 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit f6c9eed206b16fede7c88724c94d8d091f8657f3 Author: kclem Date: Thu Jun 13 13:13:20 2019 -0400 update pooled names commit 3d4a37a626f366f8f024ee7f62e017e8d68092b3 Author: Kendell Clement Date: Tue Jun 11 12:33:40 2019 -0400 Add web link to readme commit 89348066ede5c447db9f04b2337118a0624b8f63 Author: kclem Date: Tue Jun 11 11:14:55 2019 -0400 add batch percentage report commit 3bcd50d67c9b320c9f908eb2e3469143461e7df6 Author: kclem Date: Thu May 30 17:25:05 2019 -0400 document report param commit 79303435747a70b288b88bb076dda90b8d379f91 Author: kclem Date: Thu May 30 17:16:11 2019 -0400 output updates commit 9f14424f275f132a148308042db48c77cbda9b1d Author: kclem Date: Fri May 24 17:57:57 2019 -0400 plot updates, compare bug #12 commit 7f5482c411a3d56a73d7f0c66f0159b623653c2a Author: kclem Date: Tue May 21 17:47:08 2019 -0400 remove crispressocompare test condition (cuz has floats) commit 5e2f4094ec607f63981cd5aa2ee7f99ce1a20d12 Merge: aac9dfe7 5933d93c Author: kclem Date: Tue May 21 17:40:44 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit aac9dfe70292a90d6981b0859ff9ede8da7094a4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test changed test to something that won't be affected by float formatting commit 5933d93c5f1408f754895254306701b5b0eaadd4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test commit 7d15cdfaa4a323f4baad96bf36c97fd455dd6684 Author: kclem Date: Tue May 21 17:11:09 2019 -0400 Add compiled c file commit 50134d64d33dc984ac81a066e7c370b160b861b3 Author: kclem Date: Tue May 21 16:58:57 2019 -0400 v2.0.28 Add report for CRISPRessoCompare Standardize naming conventions for files and plots from CRISPResso Add data links to CRISPRessoBatch report CRISPRessoBatch plots using the plot window around the cut site instead of only the quantification window If only one reference, 'Reference' is not shown in data plots or as a file prefix Set plotting indexes once for each guide (previously, they were specific to the amplicon) Base editing plots now plot for each guide (previously, they were one for each amplicon) commit 9e86bef89884a0e5980c7781cfcd56243fdd42f0 Author: kclem Date: Wed May 15 14:28:14 2019 -0400 Standardize concept of windows for quantification and plotting #11 commit dd63974334c94816efe5878e4aab1ec3c3e1a6c5 Author: kclem Date: Wed May 1 14:49:24 2019 -0400 python division bug.. <3<3 commit 4a4b88885ab2e034bdcbca9566aebe911c58f427 Author: kclem Date: Wed May 1 14:35:33 2019 -0400 fix bug for spaces in filepaths commit f7e0aee5d948abd876b5048c87cae59e265d0c6f Author: kclem Date: Fri Apr 26 11:56:51 2019 -0400 min merge size commit 12cc6a93c0b02be86575c6c7fe8b7569a96e0edd Author: kclem Date: Fri Apr 26 11:41:09 2019 -0400 Relax flash merging, add parameter for stringent flash merging (#7), remove debug statements commit 38bfb3174d8d37297587243cb9e04469e7e54a20 Author: Kendell Clement Date: Thu Apr 25 22:50:08 2019 -0400 Update CRISPRessoPooledCORE.py fix numpy -> string error commit 273fe3a1ab731a32c26efdcab58a502cd2162104 Author: kclem Date: Mon Apr 15 13:38:17 2019 -0400 Fix CRISPRessoWGS tests commit 2c1ec9f5eadaf62ac7df35535aa26965d993e96a Author: kclem Date: Mon Apr 15 13:15:03 2019 -0400 add tests for CRISPRessoWGS commit 6632700fbde6b7f5e49e402471fea88a0966d2c0 Author: kclem Date: Fri Apr 12 10:29:43 2019 -0400 update docker run message, enable local testing, remove networkx commit c31355e15afc49f6aa069968c94408f5f7b484c0 Author: kclem Date: Thu Apr 11 16:00:44 2019 -0400 ignore test directory for docker commit aad55c922fa9b3948e5b194b26da3a8a3ccc97d2 Author: kclem Date: Thu Apr 11 15:52:45 2019 -0400 fix plot label bug for 10a, update fig 2a data commit c40b2de4f21e69404de153b9e12dee6654568b67 Merge: 560b65e7 88990dbb Author: kclem Date: Wed Apr 10 17:30:41 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 560b65e720a6dc5329203ecbc8c1b38b47aee219 Author: kclem Date: Wed Apr 10 17:30:34 2019 -0400 update tests for decimal shift for percentage in summary report commit 88990dbb29135d160879ed02dcac6874b0fed9f2 Author: Kendell Clement Date: Wed Apr 10 17:26:20 2019 -0400 Update README.md commit e4deb9211dadb3e9622f2eee38f0cc4930a0cf17 Author: kclem Date: Wed Apr 10 17:19:07 2019 -0400 v2.0.28 commit 098f5a68ba632987930fd70038af667e228b7a1f Author: kclem Date: Tue Apr 9 22:13:13 2019 -0400 update circleci path commit bb78ade66d20a826bbd8beee53711620869b86aa Author: kclem Date: Tue Apr 9 17:40:43 2019 -0400 circleci test path update2 commit e760c77bdd23fd087733c26eda86f15de6877bbf Author: kclem Date: Tue Apr 9 17:36:23 2019 -0400 circleci test path update commit 0e1b576959b09da72b101983d312842e06ea4c56 Author: kclem Date: Tue Apr 9 17:33:22 2019 -0400 circleci artifact storage commit 28c23d2172cf5c39faffd1ea498f6d771237567d Merge: c7da8466 e42b3a6b Author: kclem Date: Tue Apr 9 14:17:34 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit c7da84668556b897b43dd53eb2370c3404ab3fa2 Author: kclem Date: Tue Apr 9 14:17:23 2019 -0400 circleci testing for batch and pooled commit e42b3a6b4c11cab0ac9c29c378858706b7fd328e Author: Kendell Clement Date: Tue Apr 9 11:55:04 2019 -0400 Update badge links commit 6a435376fe43e6408507f1078518515f1f522ba8 Author: Kendell Clement Date: Tue Apr 9 11:42:21 2019 -0400 Got me some badges! commit f8c9713444472f5e3cef4fd7f7bce0704dd6a266 Author: kclem Date: Tue Apr 9 11:07:18 2019 -0400 circleci artifact update commit 8d4b825dcf01887db96b90640aafea564031a7e9 Author: kclem Date: Tue Apr 9 11:03:59 2019 -0400 circleci updates commit bfb3bc82abd22647554b80517554ef58e81d2090 Author: kclem Date: Tue Apr 9 10:51:53 2019 -0400 add expected results for circleci commit 8466e146d73a2c2149fdbcd67d4db656fe8c13e0 Author: kclem Date: Tue Apr 9 10:29:57 2019 -0400 circleci update commit 19c437975e57119531bcb0b9b2a6587a7d6b3ea7 Author: kclem Date: Tue Apr 9 10:14:42 2019 -0400 circleci update commit c665c19c97f4c6d4b45f40b3ac9522bf53ce05ba Author: kclem Date: Mon Apr 8 16:27:38 2019 -0400 circleci - use custom docker commit caa46ce2bc1de5e100bfd5db25179fb6b54f4d95 Author: kclem Date: Mon Apr 8 14:45:16 2019 -0400 python2 virtualenv commit a13e2fd2c9eea59652ec1ad7c6a9862a708bc33e Author: kclem Date: Mon Apr 8 13:54:32 2019 -0400 CircleCI testing commit 7257b54f77391da21532bf06ed84b1ce03d460e0 Author: Kendell Clement Date: Fri Apr 5 11:15:22 2019 -0400 Remove dependency on zip commit 7b694f507a61e424e994384cd765855d7f69dfb4 Author: kclem Date: Thu Apr 4 12:12:37 2019 -0400 add networkx requirement for py27 commit 099acbc8149dbbd5c99729ddc8e0928e3f890023 Author: kclem Date: Thu Apr 4 10:16:10 2019 -0400 Fixes for dockerhub commit 3a3bfbdd2373348901faac6fda468b70cb5ce725 Author: kclem Date: Thu Apr 4 10:09:06 2019 -0400 Bioconda submission fixes commit dff86e16812f2b9345ef86bd3185786f4f82d25a Author: kclem Date: Wed Apr 3 18:49:13 2019 -0400 More precise cleavage window and quant window plotting commit c8caf7fbdad415d4d3fb47cc5b9b0b76dd65deab Author: kclem Date: Wed Apr 3 17:49:54 2019 -0400 v2.0.27 add reports for pooled and WGS commit c68e3cb922d037c12a62c695c0ae1f6c148ace82 Author: kclem Date: Mon Mar 18 11:15:40 2019 -0400 batch info pickle, WGS bai location, meta mode commit 768c75c3a7864c365cf13fd65573859b9aa86ebe Author: kclem Date: Wed Mar 6 17:13:56 2019 -0500 v2.0.26 add report display name, remove paths from stored files, fix sgRNA plot, CRISPRessoPooled report HTML, add citation to report commit 58257b54fc440427e5437f8e7458fd5824020b6e Author: Kendell Clement Date: Fri Mar 1 16:55:27 2019 -0500 Update issue templates commit 50fb2d58f0d3777ba51d0f5a37e82cfc1a47ebff Author: kclem Date: Fri Mar 1 16:38:13 2019 -0500 Fix file naming and flash incompatibility commit eca34aebf86dc1b03c6107ee405cdf16898c8d51 Author: kclem Date: Wed Feb 20 17:26:42 2019 -0500 v2.0.25 Add inferring of guides commit 2ccec08691db17e6a2cfab8e310f5f29621319ca Author: kclem Date: Wed Feb 20 13:42:57 2019 -0500 quant window updates commit 21d558da1f8f5fed8f59de0eb154e5a8a505ff7a Author: kclem Date: Tue Feb 19 17:21:12 2019 -0500 add flash outies, fix quant window coords bug commit 22954e6d40ad2d237cb75c521a09f30c2b294066 Author: kclem Date: Tue Feb 19 11:17:30 2019 -0500 Update entrypoint for docker commit 0216329d8a8d1c072a269d14786820f943443153 Author: kclem Date: Wed Feb 13 16:40:39 2019 -0500 v2.0.24 update docker, setup.py commit e11b60fe1cf3c3e67e41964d45e843f28c2975a5 Merge: fb1687b8 d6a7b979 Author: kclem Date: Mon Feb 11 16:22:09 2019 -0500 v2.0.23 suppress plots, custom flash version commit fb1687b87de0cb7e5d1ab0acc2a2651d1be1fcec Author: kclem Date: Mon Feb 11 16:12:26 2019 -0500 v2.0.23 suppress plots, custom flash version commit d6a7b979e1ecd49b26c648f2007e1ecfa905ec14 Author: Kendell Clement Date: Fri Feb 1 12:20:50 2019 -0500 Update readme formatting commit 9e4c87c4f60edd82539b01b24f646962c0f06f4c Author: Kendell Clement Date: Thu Jan 24 17:13:28 2019 -0500 Add conda install instructions commit 4b2b06e52ee1aba4f3bdd0458c13813f2417fbf7 Author: kclem Date: Thu Jan 24 14:02:05 2019 -0500 add manifest.in commit 495e1f9829d6e72048b87a6972472c4242411286 Author: kclem Date: Wed Jan 23 11:05:44 2019 -0500 Change license location, license update commit 24c3b1b6ba66557b99469c0e12291fae2fcc800d Author: kclem Date: Wed Jan 23 11:01:44 2019 -0500 v2.0.22 commit 4a426bf715e37cbb8d619d54fc3a92385e9dcf1b Author: kclem Date: Tue Jan 22 15:25:13 2019 -0500 update batch amplicon naming commit f4fbe96b335c6e1a5b3dc252bf090e3d36ffb8cd Author: kclem Date: Tue Jan 22 12:44:00 2019 -0500 v2.0.21 detangled root location dependency from params commit 9d29737de257a2999f0763afd5f97ddafd6d2fe7 Author: kclem Date: Tue Jan 22 12:36:03 2019 -0500 Update CRISPRessoShared.py commit 573aa90cf70cd941c3e26361b2e656fbf34d8e5e Author: kclem Date: Fri Jan 18 18:10:47 2019 -0500 allow no cython commit 69811cf1eb58ac014a1ac34c56748aaffcead187 Author: kclem Date: Fri Jan 18 17:55:23 2019 -0500 require cython for installation commit 37ac0e6278c4825bb816c7cfe0a1fc3819abd516 Author: kclem Date: Fri Jan 18 17:44:19 2019 -0500 v2.0.20b prepare for bioconda integration commit 31c5ad02127366d0ace2849a6e996a86d690f4d1 Author: Kendell Clement Date: Tue Jan 15 17:22:53 2019 -0500 Update README.md commit 5b8cf82bfee3eeefcf2ba5bb31b4a60b25960df0 Author: Kendell Clement Date: Tue Jan 15 16:46:46 2019 -0500 Update README.md commit c932b8b4ad7c08d3fc94d5c57d0017001a78a42f Author: Kendell Clement Date: Tue Jan 15 16:40:59 2019 -0500 Update README.md commit 5cf5aa34b2ef97e38e45ee3394cd8b4aade50c6d Author: Kendell Clement Date: Fri Dec 21 15:03:50 2018 -0500 Add trimmomatic_command parameter commit 2ec374962c169c1a56cd48ff9080f7941433d016 Author: kclem Date: Thu Dec 20 18:30:01 2018 -0500 v2.0.19b HDR and WGS changes commit 78483624993c6fdb5ccc889a2a6f37036fb0f2c8 Author: kclem Date: Thu Dec 6 14:55:18 2018 -0500 add filtering for fastqs commit 17940c70b36d74d8b9134664b515f3b164c678b0 Author: kclem Date: Wed Nov 28 15:00:35 2018 -0500 v2.0.18b - fix bug with batch names, add param to suppress plots commit 038042e3cea983fc264460402d88e15766cc50b0 Author: kclem Date: Wed Nov 14 15:00:47 2018 -0500 Add router for docker commit 3ef96cc08a18f6eff9bb6115366e27304986ed27 Author: kclem Date: Wed Nov 14 14:52:41 2018 -0500 add Docker file commit 3e1cbf1dffc4dbc555942d45f91ad942237b81c3 Author: kclem Date: Wed Nov 14 14:29:44 2018 -0500 set white background for plots commit dd2cb254170b8363a7feb0427ed587d2093ebd3e Author: kclem Date: Wed Nov 14 11:13:02 2018 -0500 Set seaborn style commit dde6a675a5ba4e9ec100c29c2d59534aa39ef4fc Author: kclem Date: Tue Nov 13 16:13:55 2018 -0500 Fix line endings commit db0a67017f57c5a77bed9eb442cb7efa9df4b97a Author: kclem Date: Tue Nov 13 10:31:45 2018 -0500 v2.0.17b - bug with multiple references of different lengths commit 7f4afffa1485094aee4c0399a319a30c12ec473e Author: Kendell Clement Date: Tue Oct 23 17:22:07 2018 -0400 Update README.md commit 0843923b50d2db953600230ac23614f52591c4b4 Author: Kendell Clement Date: Tue Oct 23 16:59:48 2018 -0400 Update README.md commit 6f79084ce88502a40fe8b9b1839733ca224d14dd Author: Kendell Clement Date: Tue Oct 23 16:56:38 2018 -0400 Add files via upload commit 72d0b355b5d39164405b6311bda6231dbbdef371 Author: Kendell Clement Date: Tue Oct 23 15:44:26 2018 -0400 Update README.md commit 30fdc7fd5d73594b332efd546a1f4d85004a80d6 Author: Kendell Clement Date: Tue Oct 23 13:48:25 2018 -0400 Update README.md commit 5b11e51083ca87cbc1a8dff02b4634fe12176d29 Author: Kendell Clement Date: Tue Oct 23 13:42:11 2018 -0400 Update README.md commit 33d367310d702667d86202aba389a0fee4eba691 Author: Kendell Clement Date: Tue Oct 23 13:24:50 2018 -0400 Update README.md commit d6c647d32cdded7803f6b023949202c9486f5caa Author: Kendell Clement Date: Tue Oct 23 12:01:41 2018 -0400 Update README.md commit 5cb8fccf048a49b3798f6932c0b0db90569697fa Author: Kendell Clement Date: Mon Oct 22 17:42:41 2018 -0400 Update README.md commit 7b60f691450c932b5d32ff48218f187328d0726f Author: kclem Date: Tue Oct 16 15:27:34 2018 -0400 2.0.16b - batch mode report commit 6037c2945efdea6fecf67b037a70c30d4bc6696b Author: kclem Date: Fri Oct 12 18:14:27 2018 -0400 2.0.15 updates to pooled, adjust merging commit 326e1c9f370e1d6956ad565605309f37e214e927 Author: kclem Date: Thu Oct 4 10:28:07 2018 -0400 2.0.14b - adjusted flash overlap params, cannot take mult aln gap penlty commit 26ec80198dc06ca09eb525deaf387c330f701cac Author: kclem Date: Fri Sep 28 15:13:00 2018 -0400 default val for n_processes commit bec0d312629d006169495b241fb9ea0018380e62 Author: kclem Date: Tue Sep 25 10:55:21 2018 -0400 2.0.13b commit 51d1386856849af7f04be0ce587e97349c7149b8 Author: kclem Date: Wed Aug 22 18:18:23 2018 -0400 Produce report commit 017c409c19a6af99c09c97faff67c26ac902e157 Author: kclem Date: Mon May 14 16:44:32 2018 -0400 Initial Commit of files commit 4324c954cc4efa10fc01fc6d69f88253ef5a7483 Author: kclem Date: Mon May 14 16:31:29 2018 -0400 first commit commit 0c37209616db2848a623bb3fe84f17bf8429d402 Author: Samuel Nichols Date: Fri Jan 12 08:55:40 2024 -0700 Squashed commit of the following: commit 22fc03183a8070c30dfb74d5c23575ac19019855 Author: Samuel Nichols Date: Fri Jan 12 08:54:01 2024 -0700 Add guardrail partial commit e55f6b21972b578261bc5a864ce1d653d98f9e34 Author: Samuel Nichols Date: Mon Jan 8 07:50:59 2024 -0700 Functional guardrails, needs reports update commit 6e968e9699ed59a47d88191d03768e042d8b60a4 Merge: 32b49685 e948ce10 Author: Samuel Nichols Date: Mon Dec 18 13:34:36 2023 -0700 Merge branch 'guardrails-clean-history' of https://github.com/edilytics/CRISPResso2 into guardrails-clean-history commit 32b49685da320501dad2b0ebbb57887b66220ba8 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit 4e309cf6f732565d635de3d4c5d074ada3027e2d Author: Cole Lyman Date: Mon Dec 18 10:51:55 2023 -0700 Refactor to use CRISPRessoReports module commit e648dc087c0055bc5d2fca13c64071a371dea941 Author: Cole Lyman Date: Mon Dec 18 10:51:11 2023 -0700 Add CRISPRessoReports subtree commit e948ce107ebb0d1d99010ed12e937f34b5e607d4 Author: Samuel Nichols Date: Fri Dec 15 15:27:04 2023 -0700 Include guardrail functions commit d33c748871a625facfe8d792e29c77ab9779138f Author: Kendell Clement Date: Tue Nov 7 16:31:06 2023 -0700 Include parameter --assign_ambiguous_alignments_to_first_reference in readme commit a1435f7f491a6a61434f3051e39f39a4c9bf1edc Author: Kendell Clement Date: Wed Oct 11 17:17:30 2023 -0600 Enable quantification by sgRNA (#348) This PR includes: - storing the sgRNA-specific editing locations in the crispresso2_info object. Previously, each amplicon would record the indices of quantification windows across the guide, but not for individual guides. This stores the information for each guide in crispresso2_info['results']['refs'][reference_name]['sgRNA_include_idxs'] - a script (count_sgRNA_specific_edits.py) to parse through an allele table output from a completed CRISPResso run (`--write_detailed_allele_table` flag required) to count edits in each sgRNA separately. I don't have a good double-edited sample handy, but it can be run on the demo HDR data [hdr.fastq.gz](http://crispresso.pinellolab.org/static/demo/hdr.fastq.gz) using the command: ``` CRISPResso -r1 hdr.fastq.gz -a acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -e acatttgcttctgacacaactgtgttcactagcaacctcaaacagacaccatggtgcaCctgactccGgaggagaagtctgccgttactgcGctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcaggttggtatcaaggtta -c atggtgcatctgactcctgTggagaagtctgccgttactgccctgtggggcaaggtgaacgtggatgaagttggtggtgaggccctgggcag -g TGCACCATGGTGTCTGTTTG,GATGAAGTTGGTGGTGAGGCCC --write_detailed_allele_table -n hdr3 -p max -gn guide1,guide2 ``` ``` python CRISPResso2/scripts/count_sgRNA_specific_edits.py -f CRISPResso_on_hdr3 ``` This produces: ``` Processed 25000 alleles Reference: Reference (2391/23415 modified reads) UNMODIFIED: 21024 MODIFIED guide1: 2359 MODIFIED guide2: 32 Reference: HDR (856/1577 modified reads) UNMODIFIED: 721 MODIFIED guide1: 854 MODIFIED guide1 + guide2: 1 MODIFIED guide2: 1 ``` commit 2e3da02fdbed2fa8ae02a277763d65a502459827 Author: Cole Lyman Date: Tue Oct 10 15:29:08 2023 -0600 changed tuple to list for matplotlib change (#31) (#346) Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit cd3c332135fe4db0f9218e3d87263d5c65838ed9 Author: Kendell Clement Date: Sun Oct 1 01:54:46 2023 -0600 rename script to camel case commit 7c719d65fb36ac7654db9040f226564ea28fcab9 Author: Kendell Clement Date: Sun Oct 1 01:53:44 2023 -0600 Add new script for counting high quality bases commit f97cd2795e89464bcc9321ccfdbca3e6af2bcb4f Author: Kendell Clement Date: Thu Sep 14 15:15:30 2023 -0600 Prime editing alignment params (#336) Adds two parameters to control alignment of pegRNA components: --prime_editing_gap_open_penalty and --prime_editing_gap_extend_penalty. CRISPResso checks to see whether the pegRNA spacer and extension sequence are in the correct orientation, but sometimes they could align in the incorrect orientation with a higher score (e.g. via insertion of multiple gaps, whereas a single long gap would be preferred). Introducing these two parameters allows users to adjust the alignment parameters specifically for these prime-editing checks without adjusting the global alignment parameters which will be applied to reads that are aligned to the WT reference/prime-editing reference sequences. The new prime_editing_gap_open_penalty is set to -50, a higher gap open penalty than the default needleman_wunsch_gap_open penalty (-20). This commit breaks backward-reproducibility, but mostly in the checking of pegRNA component orientation - so previously some CRISPResso runs would have failed and produced an error, but now they will (hopefully) succeed. To achieve complete backward reproducibility, add the flag --prime_editing_gap_open_penalty -20 to runs. commit 64cbf36dae85cffa2c15e73f2a7ee8aa1077d917 Author: Cole Lyman Date: Thu Sep 7 16:43:30 2023 -0600 Fix samtools piping (#325) * Remove samtools pipe stderr to stdout Sometimes some of the libraries that samtools depends on don't have the correct version information, and as such samtools will report this to stderr when run. Because we pipe the output of samtools, we expect it to be valid SAM format, but when these library version messages are reported, it breaks CRISPRessoWGS. * Remove extra spacing at end of lines and add missing comma in WGS * Log stderr from samtools in CRISPRessoWGS commit 8feff4101f27406d9d88ace97d31a518276bff3f Author: Cole Lyman Date: Fri Sep 1 09:43:56 2023 -0600 Replace link to CRISPResso schematic with raw URL in README (#329) * Replace link to CRISPResso schematic with raw URL * Add new lines to the beginning of unordered lists commit 2e9e6bff5bcc536d5e2ba1440d1ab96d9d47efd6 Author: Kendell Clement Date: Thu Aug 10 00:52:12 2023 -0600 Try to unbreak CircleCI commit ae5b95246cb0f6d66c4cbfb50cf8f5a9626b0827 Author: Kendell Clement Date: Thu Aug 10 00:17:27 2023 -0600 Center command line text messages commit 4d9c71ecf2248c9bb1e10430178dc318b6621c8b Author: Kendell Clement Date: Thu Aug 10 00:17:07 2023 -0600 Fix bug in prime-editing scaffold-incorporation plotting If read is too short, scaffold incorporation detection will fail because it will check beyond the length of the read. commit 2b36a1a5c35e8a93516ce8baf464595615e0f402 Author: Kendell Clement Date: Wed Aug 9 15:29:48 2023 -0600 CRISPRessoPooled --compile_postrun_references bug fixes commit 3e04d1d402bcf95edd39fc7c8c9af61bb380f9db Author: Kendell Clement Date: Tue Aug 8 23:30:15 2023 -0600 Fix missing ' in Pooled --demultiplex_only_at_amplicons commit 06af527f9e2020c5cf251e7f1cec0b1eca1c1664 Author: Cole Lyman Date: Mon Jul 24 10:47:46 2023 -0600 Sort pandas dataframes by # of reads and sequences so that the order is consistent (#316) * Make sorting stable * Including c files * Sort by #Reads instead of %Reads to avoid floating point errors --------- Co-authored-by: Samuel Nichols commit de05533b3511a84f3b6b14fc2ef64db041613261 Author: Cole Lyman Date: Thu Jul 6 13:54:45 2023 -0600 Fix multiprocessing lambda pickling (#311) * Fix running plots in parallel The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. * Fix multiprocessing lambda pickling (#20) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Further fixes to pickling multiprocessing error (#21) * Refactor process_futures to be a dict This makes debugging much easier because you can associate the arguments to the future with the results. * Fix the pickling error when running in multiprocessing Only top-level functions (not lambdas) can be pickled to use in multiprocessing pools, thus the lambdas are converted to a regular function. * Use Counter instead of defaultdict in CRISPRessoCORE * Update process_futures to dict in Batch and Aggregate commit ebb016dff46c280dce8c3c09e8ac0e0cc25d4d74 Author: Kendell Clement Date: Mon Jul 3 17:12:09 2023 -0600 Enable CRISPRessoPooled multiprocessing when os allows multi-thread file append commit 7285da0e987b77b72c8885bb35940e0f50c146bd Author: Kendell Clement Date: Fri Jun 23 16:50:33 2023 -0600 Fix print bug for invalid fastq commit 9acdeac67441f9a1d55ac94b153bcb68fb89b92c Author: kclem Date: Wed Jun 21 16:03:48 2023 -0600 Slugify before creating filename - replaces invalid characters in batch names with _ commit f97e29c67de4c80b8d6b9cf334f363be4b514ade Author: Cole Lyman Date: Wed Jun 21 14:43:43 2023 -0600 Add verbosity argument to CRISPRessoAggregate (#18) fixes #306 (#307) * Add verbosity argument to CRISPRessoAggregate (#18) * Allow for amplicon and guide seqs to be some variant of NA in batch (#19) This was discovered when attempting to infer amplicon sequences in batch mode on the web interface, NAs were supplied for the amplicon sequences to the sub CRISPResso commands. commit 32e1e9797da5c3033cdc588e92f06b8813961953 Author: Mark Clement Date: Wed Jun 21 14:01:00 2023 -0600 Allow for interrogation of overlapping sgRNA sites commit 7248ba8c4deee125ad1ec12fdf1294a84d5f6f93 Author: Kendell Clement Date: Mon Jun 12 12:16:47 2023 -0600 Check input fastq file format Asserts input format of fastq files - including if gzipped files are missing the gz suffix. commit 83c8ab8f462e7d8c1d04c08c1a398b874f517251 Author: Kendell Clement Date: Mon Jun 5 13:41:55 2023 -0600 Fix CRISPRessoArgParser commit 14a2c8577f566e1b72d5f4e72cd6cd22079610be Author: Kendell Clement Date: Mon Jun 5 13:29:31 2023 -0600 Cosmetic updates for command-line use - version bump to 2.2.13 - If no args are provided, the command line version will print out an abbreviated help message - parameters can be excluded from CRISPRessoArgParser commit 1cd54bc1d03360c3d8121ba9e66b3589fe1cf252 Author: Cole Lyman Date: Thu May 11 14:31:47 2023 -0600 Fix multiprocessing error, don't start pool when only using single thread (#302) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload * Only start process pools when using multiple processes This is mainly to solve the issue when running on AWS Lambda, but this should improve single core performance overall. --------- Co-authored-by: Kendell Clement commit 92a705c939b370373a70cf6ae9f1616de33288b9 Author: Cole Lyman Date: Thu May 11 14:31:06 2023 -0600 Update `base_editor` parameters in README and add Plot Harness (#301) * Update README to have consistent use of `--base_editor_output` (#16) * Add files via upload --------- Co-authored-by: Kendell Clement commit 7d46c4490235df45c5546b1b470e4e6a99727031 Author: Cole Lyman Date: Wed May 10 15:41:33 2023 -0600 Clarify CRISPRessoWGS intended use (#303) * Update README to have consistent use of `--base_editor_output` (#16) * Add sample plotting jupyter notebook * Add clarifying info to CRISPRessoWGS description Clarify WGS usage commit 833a701787bb47674b3e921c38cac6189c775cf7 Author: Kendell Clement Date: Thu May 4 17:02:46 2023 -0400 Remove debug print statements commit 712eb2a11825e8d36f2870deb12b35486bd633fb Author: Kendell Clement Date: Thu May 4 16:40:07 2023 -0400 Allow dashes in filenames resolve #73 commit a439f094745b2b5e7f032f0777d4c67e6d6f93c5 Author: Kendell Clement Date: Sat Apr 22 23:41:58 2023 -0400 Raise exceptions from within futures in plot_pool commit 7e807a60de2a9d18bccd034b87106ceaf7153338 Author: Kendell Clement Date: Sat Apr 22 23:38:56 2023 -0400 Fix future pandas indexing warning Pandas error was "FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead" commit 304a92aa7a7ef8c705cb070dce25d9a2e5745ba9 Author: Cole Lyman Date: Thu Apr 20 13:59:27 2023 -0600 Remove debug print statements fixes #295 (#297) The format string option used here is only available in Python version >=3.8. commit 478c06f784603e96d20f96e91993fdcc4ac35c8a Author: Kendell Clement Date: Thu Apr 13 12:09:26 2023 -0400 Update plotCustomAllelePlot.py script for #292 (#293) Update type of 'max_rows' param to int Fix location of 'args' in crispresso2_info object commit bcdae39e05d530f4a4e78738c3b30f7664981919 Author: Kendell Clement Date: Mon Mar 27 13:18:34 2023 -0400 Update pooled parameter format commit 546446e36e7e68b527767d6c31ec341a49df2059 Author: Kendell Clement Date: Tue Feb 14 16:26:23 2023 -0500 Fix running plots in parallel (#286) The reason the plots were running slower before this change is because I was calling the plot function, not passing it to `submit`. So it was essentially running in serial, but worse because it was still spinning up/down the processes. Co-authored-by: Cole Lyman commit d75f32a2eb5aeaaee866c09e5655a3e27af8b1a1 Author: kclem Date: Fri Feb 10 15:45:15 2023 -0500 Fix #283 to avoid filename collisions Previously, amplicon names longer than 21bp were truncated, but the check for uniqueness wasn't working, so it would overwrite some plot files. This fixes the filename collision and enforces uniqueness in reference filename prefixes. Thanks @mbiokyle29 commit e577318006cd17b2725bd028e5e56634c6eb829a Author: kclem Date: Mon Feb 6 16:37:25 2023 -0500 Case-insensitive headers accepted in CRISPRessoPooled commit d34927620a4a6126a9988b3041e76f60728abbfe Author: Kendell Clement Date: Tue Jan 31 13:48:33 2023 -0500 Fix print statement in CORE commit ee88b7ed89c395f68225a50dea44a2ad69d5e9a5 Author: Kendell Clement Date: Tue Jan 31 13:22:51 2023 -0500 Version bump to 2.2.12 commit 1d4679c72d0c8b4154317c9aff5179217198e2d7 Author: Kendell Clement Date: Tue Jan 31 13:01:31 2023 -0500 Status Updates + Pooled Mixed Mode Update (#279) * Implement logging handler to overwrite the latest log status to file * Add StatusHandler to CRISPRessoCORE log This will take the latest log output and write it to a file (`status.txt`), the catch being that with each log the file is overwritten so that one can easily tell where CRISPResso currently is and what the error is (if any). These changes include some slight refactoring in order to accomodate any potential parameter exceptions. * Add StatusHandler to CRISPRessoBatch and refactor `logger.warn` to `warn` * Add StatusHandler to CRISPRessoPooled and a little refactoring * Implement `percent_complete` to the status log * Add StatusHandler to CRISPRessoAggregate log * Add StatusHandler to CRISPRessoCompare log * Add StatusHandler to CRISPRessoPooledWGSCompare log * Add StatusHandler to CRISPRessoWGS log * Rename `status.txt` to `CRISPResso_status.txt` * Modify status log names to match the tool they are generated from * Add percent_complete stages to CRISPRessoCORE These also include log statements of each plot that is being generated as well as fixing some variable name collisions with `ind`. * Format the percentage in the log to be 2 decimal places * Change all plotting logs from `info` to `debug` and simplify progress This refactors how the progress of the plots is calculated, making it much simplier. Before this change we would of had to keep track of the number of times `percent_complete` was output, but now it simply updates the percent complete after each amplicon is finished processing. Hopefully this will make things easier to mantain even though it will be a little less "accurate" (not sure how accurate the original implementation was...). * Implemented shared console log handler across all CRISPResso* calls This allows for easy changes to logging formatting, which was inspired by having to change the default logging level. The default logging level needs to be set at `logging.DEBUG` in order for the debug log statements to not be ignored for the running and status logs. * Add ability to set the verbosity level to each CRISPResso* tool This allows users to set a verbosity level between 1 and 4 using the `-v`/`--verbosity` CLI parameter. If the `--debug` flag is present, then the level will default to 4, being the most verbose. * Implement showing the last seen `percent_compelte` when none is provided * Keep track of and log when multiple parallel runs are completed These changes modify `CRISPRessoMultiProcessing.run_crispresso_cmds` such that we can now display when a run is completed. This potentially breaks how signals and interupts are handled with multiple runs happening, but this needs to be reviewed. * Add debug and percentage complete to CRISPRessoBatch * Add percent complete to CRISPRessoPooled * Add debug and percent_complete message to CRISPRessoAggregate * Add `percent_complete` to CRISPRessoCompare * Add `percent_complete` to CRISPRessoPooledWGSCompare * Add status and `percent_complete` to CRISPRessoMeta * Add `verbosity` arguments to CRISPRessoCompare and CRISPRessoPooledWGSCompare * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name * Fix bug to flow CRISPRessoPooled options to sub command * Make amplicon file args variable name clear * Update how parameters are set and retrieved from parameter object The refactor in the previous commit changed the type of the arguments to a dictionary which doesn't have the parameters as attributes, and this commit fixes that error. * Add note in output header for change in default CRISPRessoPooled In the next release (2.3.0) the `--demultiplex_only_at_amplicons` will be the default when running in mixed-mode. This is to allow for inexact alignments of the reads and the amplicons to the genome. For more context, see this issue https://github.com/pinellolab/CRISPResso2/issues/276 * Clarify the verbosity parameter help message * Separate out parameters to `normalize_name` in CRISPRessoCORE * Separate out parameters to `normalize_name` in CRISPRessoWGS * Separate out parameters to `normalize_name` in CRISPRessoPooled * Separate out parameters to `normalize_name` in CRISPRessoCompare * Fix bug in CRISPRessoPooled by replacing `database_id` with `normalize_name` * Refactor `run_crispresso_cmds` to not require a `logger` This commit implements the functionality to make the `logger` object optional by seeing which module called the `run_crispresso_cmds` function and obtaining the correct object from that module name. The function also immediately returns when no commands are passed to it. * Add amplicon name to plotting debug statements in CRISPRessoCORE --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman Co-authored-by: Samuel Nichols commit ff7eca76e6a3a08af4ac18ac4e88d20f2a06b1f9 Author: Kendell Clement Date: Thu Jan 26 15:27:27 2023 -0500 CRISPRessoPooled custom header fix (#278) * Fixing documentation to match pooled headers * Header removal bug fix change documentation to guide_seq * Update documentation and help feature for CRISPRessoPooled * Remove extra newlines from CRISPRessoPooled -h * Make variable names as clear as my firstborn child's name * Update one more variable name Co-authored-by: Samuel Nichols commit 104866e1080c973bb025d1a5ba59b19dca1658af Author: Cole Lyman Date: Thu Jan 5 14:00:26 2023 -0700 Fix deprecated numpy type names (fixes #269) (#270) In the most recent version of numpy (1.24) some of the types have been deprecated. This commit fixes these errors. commit 58a8e42df88b66fad6b4f6ad04a5b9d9d43d01b4 Author: Cole Lyman Date: Thu Jan 5 06:49:35 2023 -0700 Add snippet about installing CRISPResso2 via bioconda on Apple silicon (#274) I have suffered enough trying to debug my installation, so hopefully this helps someone else. Co-authored-by: Cole Lyman commit b9851e98104602eb78c2b384105267624295e9d3 Author: Cole Lyman Date: Thu Dec 22 13:30:23 2022 -0700 Fix bug when pooled bam is input (#265) This change checks to see if a bam file was input, and if so it doesn't try to remove any intermediate files because there aren't any. Co-authored-by: Cole Lyman commit b822612642043e75a19042941f69b457ce51f517 Author: Kendell Clement Date: Mon Dec 19 15:26:45 2022 -0500 Delete vscode settings commit b99aa624dec68ef7d19264340ce0cafa829625f4 Author: Kendell Clement Date: Mon Dec 19 13:29:14 2022 -0500 Clarify input param help for pooled bam commit 3fae1e8b821ec6b1890bff6561fa8fa67dc49a04 Author: Kendell Clement Date: Mon Dec 19 13:28:54 2022 -0500 Fix #235 - Cigar string is * if read unaligned Previously, the bam would set the cigar string to 0 if the read was unaligned. This breaks the sam->bam conversion and causes the errors in #235. commit c65ba07dc5a983453cdf7bb1e27005230dac6f1b Author: Cole Lyman Date: Thu Dec 8 13:48:17 2022 -0700 Add deprecation notice (#260) * Add FLASh and Trimmomatic deprecation notice to CLI output * Add Edilytics email address to CLI output commit 2a30e5a45f5350ee7c6435bce1cd4edc4d31668a Author: Kendell Clement Date: Tue Dec 6 12:16:19 2022 -0500 Format filterReadsOnSequencePresence script commit 9d764414edd88a46ad5e4f496e4f1c8d5d60ce3e Author: Kendell Clement Date: Fri Dec 2 22:12:54 2022 -0500 Clarify default CRISPRessoPooled settings for use_legacy_bowtie2_options_string commit 9ddea40f7f02b546941ddaa4c71fc5283075051a Author: kclem Date: Mon Nov 14 10:33:04 2022 -0500 Add check for prime editing extension sequence in prime edited sequence if the user specifies the prime_editing_override_prime_edited_ref_seq, it could not contain the extension seq (if they don't provide the extension seq in the appropriate orientation), so check that here. Extension sequence should be provided reverse-complement to the prime edited sequence. commit 152f2dd5001da7090641ee8a1326bde9f7e8104e Author: kclem Date: Wed Nov 9 11:53:41 2022 -0500 Version bump to 2.2.11a commit 9ed356e3a0c6c316d0860d121772f80ddca6de1d Author: kclem Date: Wed Nov 9 11:47:30 2022 -0500 Add param to override prime editing sequence checks CRISPResso checks that prime editing guides are provided in the proper orientation (e.g. pegRNA 3'->5', spacer sequence 5'->3') and checks these orientations by alignment. Sometimes, the alignment can be better in the opposite direction, and this parameter allows these checks to be overridden. Otherwise, these checks would halt the program and produce the output 'The prime editing pegRNA spacer sequence appears to be given in the 3\'->5\' order. The prime editing pegRNA spacer sequence (--prime_editing_pegRNA_spacer_seq) must be given in the RNA 5\'->3\' order.' commit 39dd80afb98a22b7edb6f801c363d86bb77eeb5b Author: kclem Date: Wed Nov 9 10:06:51 2022 -0500 Update filterReadsOnSequencePresence.py commit fe55526927e3fb6e17c9a8a6f59c7057bc1e14eb Author: Kendell Clement Date: Mon Nov 7 22:25:16 2022 -0500 Add script to filter input based on sequence presence commit 713e57a19c35180035ca35e11a5820065eda0198 Author: Kendell Clement Date: Tue Oct 18 16:02:26 2022 -0400 Allow spaces in read names for CRISPRessoWGS commit 39ce008bdddccdd8229c0ba185dce78bc2f66968 Author: Cole Lyman Date: Sat Oct 8 21:09:58 2022 -0600 Fix typo of CRISPResssoPlot when plotting nucleotide quilt (#250) commit 6a2b342c8503b7327c0a2414edfbd16912d60ca5 Author: Kendell Clement Date: Sat Oct 8 23:08:47 2022 -0400 Batch amplicon plots (#251) * Error out if HDR amplicon matches existing amplicon * Add check for amplicon sequence uniqueness * Fix bug with bam_input not having bam_output * Test for no returned lines in auto mode, version bump to 2.2.11 * Fix pandas deprecation of df.append commit 726b2b93d6e419a1b0aa6a968c97edc55b4cc5a8 Author: Kendell Clement Date: Thu Oct 6 16:32:02 2022 -0400 Fix CRISPRessoBatch plot pool bug when plots are suppressed commit 7e5049c4dfb88cbc87c91935a91d1f51120a10c2 Author: Cole Lyman Date: Wed Sep 21 21:04:51 2022 -0600 Fix batch quilt plot name (#249) This fixes an incorrectly named allele quilt plot input in CRISPRessoBatch. commit 1821ca5029c5a1485733f13ab3f2048b4f1fa04e Author: Kendell Clement Date: Thu Sep 15 15:49:08 2022 -0400 Version bump to 2.2.10 commit c5f79aebfc1ae209f4ee320df250eed89a02787c Author: Cole Lyman Date: Wed Sep 14 14:24:55 2022 -0600 Parallel plot refactor (#247) * Fix duplicate plotting in CRISPRessoBatch aggregate * Refactor mulltiprocessing plots in CRISPRessoBatch * Refactor multiprocessing plots in CRISPRessoCORE * Refactor multiprocessing plots for CRISPRessoAggregate commit 4ed5e24e6cc1dd8068e2391573ae2438acd32db2 Author: Kendell Clement Date: Tue Sep 13 14:12:11 2022 -0400 print files in curr dir if Aggregate can't find files commit ce25bc06f29988e7a10afd0b6a09ba0caf0950e0 Author: Kendell Clement Date: Mon Sep 12 10:32:57 2022 -0400 Spelling typo commit c15f01c75083403f17c58c121b2afe97e9f2a1ec Author: Kendell Clement Date: Tue Sep 6 17:49:52 2022 -0400 Add helper function to create alignment scoring matrix New scoring matrix can be created using CRISPResso2Align.make_matrix() commit c80f82838c5a228b79ad4484092877cfee08e02c Author: Cole Lyman Date: Mon Aug 22 18:28:33 2022 -0600 Add `zip_output` (#240) * Making zip of results * Zip command added, if zip is true place_report_in_output_folder is also true, zip removes all files while zipping * Adding --zip to compare and pooled/wgs compare * Add more formatting changes to CRISPRessoShared * Refactoring propagate_crispress_options so only one version exists * Zip added to arguments_to_ignore and warning added when changing arguments * Restore styling * Update README to include --zip * Rename --zip to --zip_output * Change --zip to --zip_output in CompareCORE and PooledWGSCompareCORE * Bug fix arg to args Co-authored-by: Samuel Nichols commit 5de3d7286d8e33c7cf4d3615fce715806e72f511 Author: Kendell Clement Date: Thu Aug 11 21:42:34 2022 -0400 Fix fix to aggregate for CRISPRessoWGS commit a2294c266f43b14969a5d6474076f31a77a57173 Author: Kendell Clement Date: Thu Aug 11 21:40:50 2022 -0400 Fix bug in aggregate for WGS commit 7ce3eb4abe4b8ceac933272ac9cb16a8bedf26a3 Author: Kendell Clement Date: Mon Aug 8 21:53:45 2022 -0400 Update CRISPRessoWGS to allow non-word characters in region names commit 040ac0033d6e250f4e3a412101874cf5e914e08a Author: kclem Date: Mon Aug 8 16:04:59 2022 -0400 Enable processing of cram files by CRISPRessoWGS Adds --reference to samtools view when viewing cram files commit cf112a0caba8789e28530cc09171285ec6ea9b4c Author: kclem Date: Mon Aug 8 14:55:46 2022 -0400 Auto amplicon detection for interleaved input Enables processing of interleaved fastq files for guess_guides and guess_amplicons, as well as get_most_frequent_reads. When interleaved input is present, the input is first separated into R1/R2 files, then processing is performed. commit 4ba524dc7b947feca8a0f743837844f9febc2171 Author: Cole Lyman Date: Thu Aug 4 11:32:11 2022 -0600 Potential fix for aggregate plots in Batch mode (#237) commit 6097a8a104d3f156ef7c08e196ac37e32bf04c71 Author: Kendell Clement Date: Thu Jul 21 22:45:48 2022 -0400 Fix pct_vectors in crispresso2_info json object commit 65a079d86d6f386793397398f839c46014b54543 Author: Kendell Clement Date: Wed Jul 20 23:46:37 2022 -0400 Fix more readme spelling bugs commit e817376ecd54cdea1f29e303ca25b9e7d1d38333 Author: Kendell Clement Date: Wed Jul 20 23:42:23 2022 -0400 Fix bug in readme spelling commit 49740ba1d66ed6d13a9e154b8b17bc8b5186581d Author: Kendell Clement Date: Wed Jul 20 16:10:09 2022 -0400 Fix loading of crispresso info from WGS and Pooled commit b68a43271115251b18e8955e285ccc18f549e8cd Author: Kendell Clement Date: Thu Jul 14 14:11:04 2022 -0400 Add plotly to dockerfile commit b0b7d41d697304d0d5fc93e3346c9de1b98ba41d Author: Kendell Clement Date: Thu Jul 14 14:10:00 2022 -0400 Fix #231 Allow N's in bam output (Try 2) commit c460b3e73fd06a230dbac2e37c86b833144ebf94 Author: Kendell Clement Date: Thu Jul 14 14:09:10 2022 -0400 Revert "Fix #231 Allow N's in bam output" This reverts commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3. commit 2f6ad1dbe05210af9ccc1b1f17783cd212a888d3 Author: Kendell Clement Date: Thu Jul 14 13:52:37 2022 -0400 Fix #231 Allow N's in bam output commit 0a2419e518dc9b3520058c3927f98b31cd51347e Author: Cole Lyman Date: Fri Jul 8 21:10:01 2022 -0600 Fix bug when name is provided instead of amplicon_name in pooled input file (#229) Also, raise an exception (instead of incorrectly executing) when there are not enough matched parameters in the pooled input file. commit cb58212379803788c04ca5793baaa760cbbeaa81 Author: Cole Lyman Date: Fri Jul 8 21:09:49 2022 -0600 Fix bug when comparing two samples with the same name. (#228) commit e8a796f5f451409cbafed4404dfba4b6b8a124ca Author: Kendell Clement Date: Thu Jun 23 21:30:23 2022 -0400 Version bump to 2.2.9 commit 632143ddedea48bab9229baeb4bf3ea4d1f658d6 Author: Cole Lyman Date: Mon Jun 20 19:53:14 2022 -0600 Don't run global frameshift plot when there are no reads (#226) When there are no reads (i.e. global_MODIFIED_FRAMESHIFT + global_MODIFIED_NON_FRAMESHIFT + global_NON_MODIFIED_NON_FRAMESHIFT == 0) there was a bug when trying to compute the pie chart, because all of the values in the pie chart are 0. This fix, will make sure that there is at least one read in order for the plot to bee constructed properly. commit 4bb06218e835d2624d53fd401542caef6f8a3a55 Author: kclem Date: Fri Jun 3 16:57:02 2022 -0400 Improvements for guide inference in 'auto' mode In 'auto' mode, a putative guide sequence is selected at the site of maximal editing. If the site of maximal editing happens near the end of the guide (e.g. base 0) many things will break (e.g. quantification windows, etc). This update excludes bases from being used to find the guide using the --exclude_bp_from_left and --exclude_bp_from_right parameters. At default, these parameters are 15bp, so the first and last 15bp would not be selected for the site of maximal editing and thus be the site of a guide sequence. In addition, the site of maximal editing must have 3x the magnitude over the background. commit 9d64de187835b2553ad2b4374d32edab27f83645 Author: Kendell Clement Date: Thu Jun 2 20:22:25 2022 -0400 Update README.md commit 6aafc5387986f5089ba55b68d128343d68052792 Author: Simon P Shen Date: Tue May 31 17:42:53 2022 -0400 directory in quotes in batch cmd (#222) Add quotes around output folder for folders that have spaces. commit 432f163ac68b9a650d1fd326171aadc505ee87f4 Author: Kendell Clement Date: Tue May 24 23:38:36 2022 -0400 CRISPRessoBatch fills NA values in batch settings NA values in CRISPRessoBatch are filled with the value from args - either the default value or the value from the command line args (if set) commit 6de774adbad3aa8cd99d07b0ba7692984b356cd4 Author: kclem Date: Mon May 23 14:18:02 2022 -0400 Fix file naming bug for HDR outputs In html file, figures 4e and 4f incorrectly referenced figure 4d. This fixes this bug. commit b88fec0668a4082a12ead3d26582e86d829dd7cc Author: Kendell Clement Date: Sat May 21 00:32:15 2022 -0400 For bam_output, fix bug that wrote unaligned lines twice commit 3564e77ebcdedb4b01cc01dcca18ba3221fac67c Author: Kendell Clement Date: Thu May 19 16:32:18 2022 -0400 Update README with CRISPRessoPooled headers and bam_output parameters commit bc08d81f17cb1929d1c37a1773cffcf36fb12fe2 Author: Kendell Clement Date: Thu May 19 16:11:30 2022 -0400 Add more links to tools commit 006c497a379ecd94b017a883a5db887861e1586a Author: Kendell Clement Date: Thu May 19 16:08:14 2022 -0400 Add links to tools commit dc8243373ad00d6bd467fc30c59942596ff0c5d6 Author: Kendell Clement Date: Mon May 16 21:38:06 2022 -0400 fastq_to_bam implementation (#219) commit e88b6833977c6b2768299e0b2e7af623e3a9ae7c Author: Kendell Clement Date: Sun May 8 02:14:13 2022 -0400 Fix bug for when guides don't agree in CRISPRessoAggregate commit 7eb763116a8c60603f1cd654645215767ee8eb52 Author: Kendell Clement Date: Thu May 5 03:28:21 2022 -0400 Fix bug for case of empty summary plots in report generation commit 0324fa67d14ed945f0c9531d9bcf73ebcf4ca042 Author: Kendell Clement Date: Thu May 5 03:28:02 2022 -0400 Create report for number of significant bases in CRISPRessoCompare commit e3c9d0026a9ee6732f3ed6bdcf2a824850d7e66a Author: Kendell Clement Date: Wed May 4 22:43:11 2022 -0400 Update pickle to json in readme and CRISPRessoPooledWGSCompare commit 1553f7977c12bf1091a20ca55b878bccfb739b61 Author: Kendell Clement Date: Wed May 4 18:10:04 2022 -0400 Merge pull request #4 from pinellolab/master (#218) commit bcecbfc047d294e26f381a6668e08cb4db24445c Merge: 15b0e05b bb13e007 Author: Kendell Clement Date: Wed May 4 18:06:37 2022 -0400 Merge branch 'master' into master commit bb13e007738d6e7a4909e01f03daff592f334f36 Merge: af4ab6e8 d0b41483 Author: Kendell Clement Date: Wed May 4 17:59:32 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 15b0e05b9e03bbec5236e58776ddf9aa2f93180e Author: Kendell Clement Date: Wed May 4 17:54:52 2022 -0400 2 flexible pooled input (#217) * Batch type coerce and r2 file check * Upgrade tabs for bootstrap5 * Update readme with additional pooled amplicon file headers Co-authored-by: Samuel Nichols commit d0b41483bee704940ba60c58289f412b04c71659 Author: Kendell Clement Date: Wed May 4 13:43:43 2022 -0400 Update README.md commit ce49fab5301cb73ba0daf6c765e350eb083c76f1 Merge: 5f909713 b913fcb4 Author: Kendell Clement Date: Wed May 4 13:40:30 2022 -0400 Merge pull request #3 from edilytics/2-flexible-pooled-input Add flexibility to CRISPRessoPooled amplicon input by allowing headers. Also, prime editing and quantification window coordinate parameters can be passed to CRISPRessoPooled. commit b913fcb402a8ba3106c3ff7913563a33d8d19fca Author: Kendell Clement Date: Wed May 4 13:38:25 2022 -0400 Update CRISPRessoPooledCORE.py Replace process to read header, increase flexibility for column order commit 945bf31f16530b7ce25b89095b2c7005bf146117 Merge: 7b8f6788 5f909713 Author: Kendell Clement Date: Wed May 4 12:45:24 2022 -0400 Merge branch 'master' into 2-flexible-pooled-input commit 5f9097133765736a7c2fe3c8e9b730845fed0b70 Author: Kendell Clement Date: Wed May 4 12:23:44 2022 -0400 Version bump to 2.2.8 commit c4a94ce0e06c6ebae13e128fbe6b708e635121c4 Author: Kendell Clement Date: Wed May 4 00:13:17 2022 -0400 Fix summary plot representation for multi reports *fixed old reference to make_multi_report which called old summary plot format * renamed summary_plot to summary_plots to reflect a dict with multiple plots commit 62900e9ae6fa37ce99a04f12a63ed5c912f75042 Author: Cole Lyman Date: Tue May 3 20:47:52 2022 -0600 Large aggregation (#192) * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add parameter `--suppress_batch_summary_plots` If many runs are run at the same time, batch summary plots may fail because they are too large for matplotlib. This parameter `--suppress_batch_summary_plots` allows individual runs to be plotted, but suppresses batch summary plots that may otherwise be too big. * Pep formatting cleanup * Add summary nucleotide plots to aggregate * Aggregate plots are paginated * Update CRISPRessoAggregateCORE.py Remove max sample limit for plotting * Add --max_samples_per_summary_plot to CRISPRessoAggregate Parameterize the max number of samples to plot on each page of reports. Additional PDFs will be created with this number of samples on them. * Add plotly function to plot an interactive heatmap * Fix deprecated numpy type to suppress warning * Add plotting of heatmaps to CRISPRessoAggregateCORE to summarize modification types These heatmaps are interactive (zoomable and panable) and show for each sample the percentage of insertions, substitutions, and deletions. * Add the heatmap summaries to the CRISPRessoAggregate report * Update Bootstrap to 5.1.3 This is mainly so that we can use the fullscreen modal functionality in this version. * Move the plotly heatmaps to a Bootstrap modal * Fix bug where plots were not filling up entire modal. I have tried countless different ways for this to work, and this is the best that I can come up with. After the modal is opened it triggers the plot to resize, and then for some reason you need to trigger the resize event. I think this is because a `div` changing size won't actually trigger the resizing of the plot (and neither will just calling `Plotly.Plots.resize`...?!). * Update the axis labels and add autosize to plotly heatmaps I'm pretty sure the autosize doesn't do anything, but it is there for good measure. * Abandon attempts to make plots fullscreen This includes removing the Bootstrap modal (two out of the three plots would resize properly and I couldn't figure out a way to have the plot displayed outside of the modal). I have left in some javascript to make the plot fullscreen, but I couldn't get the formatting quite right and the plot wasn't much bigger in the fullscreen version because there was a ton of space between the plot and the heatmap. If some brave soul would like to tackle it, feel free! * Rename and refactor how plot data is passed around I have consolidated how the plot data is passed around, so that now you can pass in only one dict with all of the information instead of 4 or 5 separate parameters. I also renamed the `heatmap_plot_*` to `allele_modification_heatmap_*`. * Implement the line plot version of the modification percentages This also includes correctly resizing the plot when the line plot tab is selected! * Change default `max_samples_per_summary_plot` to be 150 instead of 250 * Remove extra assignments of `this_number_samples` and suppress plot The plot that is suppressed is the large nucleotide quilt when there is a large number of samples. Is it okay to suppress this plot @kclem? * Implement parallel plotting in CRISPRessoAggregate * Fix sample indexing error and heatmap scaling for large number of samples * Add plotly requrement to setup.py * Remove space around vertical barcharts * Add scrollbar to long images in multiReport * Fill in default (empty) values to allele modification plots When not running CRISPRessoAggregate, default values for the `allele_modification_heatmap_plot` and `allele_modification_lin_plot` dictionaries will be set so that the template can be properly rendered. * Include CRISPRessoBatch in the refactor of how summary_plot dicts are handled * Update dockerfile for new docker * minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) * Allow for flexible parsing of quant window coordinates * CRISPRessoPooled debug flash command, fix pep formatting * Set flexiguide homology parameter type to int * Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values * Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. * Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. * Create allele modification heatmaps and line plots in CRISPRessoBatch * Add allele modification heatmaps and line plots to CRISPRessoBatch * Make all plots in CRISPRessoBatch run in parallel * Make `--suppress_batch_summary_plots` store true Also, only open and shutdown the process pool when necessary. * Add blank values for allele_modification entries when not present Co-authored-by: Kendell Clement Co-authored-by: dharjanto Co-authored-by: Samuel Nichols commit f67376fc9ab0e407d4086aa42fd1c77706ebc9c0 Author: Kendell Clement Date: Fri Apr 15 00:46:30 2022 -0400 Fix bug from old pandas for int cols Evidently old pandas versions throw an error if a column doesn't exist. This checks to see if the column exists before the values are set. commit b34fe2956ff88629809b2434878028723dfc4895 Author: Kendell Clement Date: Thu Apr 14 23:58:07 2022 -0400 Handle multiple qwcs in batch mode If multiple qwcs were provided in batch mode, a parsing error would occur. This fixes this bug. commit c94e3b9f2e301bda91e9c1e6f4ef794b33b5dbf0 Author: Samuel Nichols Date: Thu Apr 14 21:48:32 2022 -0600 Coerce ints in batch file checking (#200) * Batch type coerce and r2 file check * Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. * Coerce int values commit fc4542491bb86eb143db0044a848a56234403496 Author: Kendell Clement Date: Thu Apr 14 22:13:23 2022 -0400 Set flexiguide homology parameter type to int commit 23fe2aa8e26067d1bcf36bfafc67e023c7588d2f Author: Kendell Clement Date: Thu Apr 14 22:12:37 2022 -0400 CRISPRessoPooled debug flash command, fix pep formatting commit d292d33d8c1fa3bfd2cee656643fd47bcdab161d Author: Kendell Clement Date: Thu Apr 14 22:00:19 2022 -0400 Allow for flexible parsing of quant window coordinates commit e1667cb53a7ea6fbb33369c8530a78639ed423ec Author: dharjanto Date: Mon Apr 11 22:08:21 2022 -0400 minor bug fixes for plotCustomAllelePlot.py to work with Python3 (#212) commit 7b8f6788da18f6ab173fa3c3d10f4ab6bb2acc26 Author: Samuel Nichols Date: Fri Apr 8 10:21:00 2022 -0600 Update README commit 9bc24cd0474ed9f398dff64274d3181c4b2f8637 Author: Samuel Nichols Date: Tue Mar 29 11:25:09 2022 -0600 Using Amplicon_Name commit 88ac5d72074b3da63de035e02c911ce34cd29414 Merge: b6057a2d e5afa478 Author: Samuel Nichols Date: Mon Mar 28 22:32:09 2022 -0600 Merge remote-tracking branch 'origin/master' into 2-flexible-pooled-input commit b6057a2d54cb8637ff0900416de8e2de72213f76 Author: Samuel Nichols Date: Mon Mar 28 20:53:05 2022 -0600 Printing info statements for matched headers commit af4ab6e8507d7aa4b7b68f217a458e0d9c966f55 Merge: bbb7d6f0 51a943c3 Author: Cole Lyman Date: Fri Mar 25 09:44:13 2022 -0600 Merge branch 'pinellolab:master' into master commit 3c1eb012fc02563e3e963f17a62c7e932f5bcddc Author: Samuel Nichols Date: Thu Mar 24 12:31:43 2022 -0600 Debugging and column checking commit 0b47acbc592a6df6adf14641357b2104b76be691 Author: Samuel Nichols Date: Wed Mar 23 09:42:51 2022 -0600 New variables added to pooled commit a0ff3a44d6d19d7b37f91919b5c0180206f72d53 Author: Samuel Nichols Date: Mon Mar 21 09:32:28 2022 -0600 Read as string not bytes commit 710675fc3c0307e21103abd604315b47ff80a894 Author: Samuel Nichols Date: Wed Mar 16 13:51:30 2022 -0600 Adding command building for new options commit f386818a48e5c840bd567611e6f1320c8146cac7 Author: Samuel Nichols Date: Wed Mar 16 10:08:33 2022 -0600 Comment out df_template.iloc instance commit eb5e309da57c8b96cd760728ddbf67be05f30d1c Author: Samuel Nichols Date: Wed Mar 16 09:59:19 2022 -0600 Potential solution for flexible headers commit 51a943c3a8f8181963acc420e75a5e8ee103cf7c Author: Kendell Clement Date: Tue Mar 15 11:00:46 2022 -0400 CRISPRessoPooled pep formatting and fix CRISPRessoPooled doesn't re-count reads if it has been run once and the `aligned_pooled_bam` is provided as input pep code formatting changes commit bbb7d6f0907aa13518d20e7f470e7de518b825f4 Merge: ddbd39f0 5a10d638 Author: Kendell Clement Date: Tue Mar 15 10:23:38 2022 -0400 Merge branch 'master' of https://github.com/edilytics/CRISPResso2 commit 5a10d638c638f21f8a2934955e92ef7e117b889e Author: Kendell Clement Date: Sat Feb 26 14:21:57 2022 -0500 Move metadata for bam input and output commit e5afa4784d5330a1dc95c5deafcd9217edeac631 Author: Samuel Nichols Date: Wed Feb 16 10:20:24 2022 -0700 Coerce int values commit ede7d85b50055311908000578c76a1860ae9de4d Author: Samuel Nichols Date: Wed Feb 16 10:18:29 2022 -0700 Revert "Batch type coerce and r2 file check" This reverts commit f91736688ea9739cf3063e3601c52ad6da1116a4. commit f91736688ea9739cf3063e3601c52ad6da1116a4 Author: Samuel Nichols Date: Wed Feb 16 10:10:52 2022 -0700 Batch type coerce and r2 file check commit 7b4a310b0f8b64c00e02eca3d522ad50d39b43ae Author: Kendell Clement Date: Tue Feb 15 22:18:05 2022 -0500 Reiterate WGS region file is tab-separated Add note to WGS description that region file should be tab-separated. Closes #199 commit b8497542e388ad401d0815d426f27abc3201a76d Author: kclem Date: Fri Feb 11 15:07:14 2022 -0500 Extend x-axis to longest scaffold incorporation length commit ab7248947afade089809c74bfe6e9d5394e8f6dc Author: kclem Date: Wed Feb 9 17:05:11 2022 -0500 Fix prime editing indexing for plots commit ddbd39f06b262d5ebd2cc69e116c08b22b6bd84e Merge: a7ffd468 442a48c7 Author: Kendell Clement Date: Thu Jan 13 15:35:36 2022 -0500 Merge branch 'pinellolab:master' into master commit 442a48c7f4c62ec2ebc95fe268475e5e2a4b2f0c Author: Cole Lyman Date: Tue Jan 11 15:28:28 2022 -0700 Indel alignment fix (#182) * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. * Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. * Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. * Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. * Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. Co-authored-by: Kendell Clement commit a7ffd46822ce195d51ff4d3dba0f02fe9bc73c1e Author: Kendell Clement Date: Tue Jan 11 16:29:37 2022 -0500 Squashed commit of the following: commit 8564eb03f0d9e62abf4b7528baf5c2ae296be8f9 Merge: f6ef62c 07cc7d8 Author: Kendell Clement Date: Tue Jan 11 16:20:15 2022 -0500 Merge branch 'indel-alignment-fix' of https://github.com/edilytics/CRISPResso2 into indel-alignment-fix commit 07cc7d856ab3fcbbaa5381f17f29568192388887 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit f6ef62cfdf909adac1b10ea86555cd218f8b2a74 Author: Cole Lyman Date: Fri Dec 10 15:29:59 2021 -0700 Fix bug in `find_indels_substitutions` This bug occurred when there was a deletion at the end of a sequence, and was thus not properly accounted for. commit 7212f87f4be60057a6c848947ff6b5efde132a25 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d50b4e903b973c71a275e31d470b40e59280ee13 Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 4db066f7bc333b7662a9232ac732ebb33ac3ace8 Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 3b3a7417f5bbd6c2785a2af54a47e01d2e820451 Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9f5eff3d95b676b5ee2e23371a5604f600d34b2 Author: Cole Lyman Date: Fri Dec 10 15:26:17 2021 -0700 Add a unit test for `find_indels_substitutions` This unit test checks for deletions at the end of a sequence, which are inherently outside of the include_indx_set window. commit d4d45a918254ab19a7e7956e9e731389c6f36ecb Author: Cole Lyman Date: Fri Dec 10 15:03:22 2021 -0700 Fix a bug in `find_indels_substitutions` The bug that this commit fixes is when an insertion occurs at the edge of the include indexes. The trouble with this earlier was that it was using the `idx` to calculate the size of the insertion, but the `idx` wasn't being incremented anymore because it was outside of the include window. commit 13f00bb40239c83e6e5cf844561fdb7000d3d9ab Author: Cole Lyman Date: Fri Dec 10 15:01:39 2021 -0700 Add test case for `find_indels_substitutions` This test case is extracted from the CRISPRessoBatch integration test and provides an example where there is an insertion at the edge of the include index. commit 659ae34e8fd106f7ecc163b5bea0b5a80ab0283c Author: Cole Lyman Date: Fri Dec 10 11:37:07 2021 -0700 Fix bug in CRISPRessoCompare where sample names were not properly set This was a place where it was (partially) missed during the crispresso2_info object refactoring. commit e9990790a0081b765c1f54f4a9b18db522ab4693 Author: Kendell Clement Date: Wed Jan 5 16:48:17 2022 -0500 Allow for mixed-case prime editing input, fix #187 commit 4acdcddbef3e812655b7af9def772f0bb0dc30b5 Author: Kendell Clement Date: Wed Jan 5 16:37:22 2022 -0500 Update insertion annotation for fastq_output and bam_output Fix index issue for fastq_output, add reporting of inserted bases to bam_output. Index for fastq_output is increased because the insertion happens immediately after the insertion coordinate. commit 2f84dd02787abffa6d39efbc50c82c92d1c87528 Author: kclem Date: Fri Dec 10 16:39:41 2021 -0500 Fastq_output report inserted bases when the `--fastq_output` parameter is provided, the inserted bases will be written to the output fastq file. Previously, a string like "DEL= INS=78(1) SUB= " would indicate a 1bp insertion at site 78. This update outputs strings like "DEL= INS=78(1+G) SUB= " with a plus character followed by the inserted bases. commit ba742bbfa533a672321b27b5122d7ea6658c9014 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Implement algorithmic improvements for find_indels_substitutions" This reverts commit 13414833ef6cd5b232097f6ff0325a2b8e28b214. commit 5c8e1c5de6e800c820d9ec5ffe4809737f1211de Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Add test cases for find_indels_substitutions" This reverts commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808. commit d9a072368ca991ffde52aa1ee29e17f2758ed279 Author: Kendell Clement Date: Wed Dec 8 17:13:28 2021 -0500 Revert "Try some additional cython optimizations" This reverts commit 38b101d57384b9f7d664ac8719c1585f5f7142a5. commit 38b101d57384b9f7d664ac8719c1585f5f7142a5 Author: Cole Lyman Date: Fri Oct 8 14:42:49 2021 -0600 Try some additional cython optimizations commit 78e79f4b93d74ecb84161d3e2f34a8a3f523f808 Author: Cole Lyman Date: Fri Oct 8 14:15:11 2021 -0600 Add test cases for find_indels_substitutions commit 13414833ef6cd5b232097f6ff0325a2b8e28b214 Author: Cole Lyman Date: Fri Oct 8 11:50:25 2021 -0600 Implement algorithmic improvements for find_indels_substitutions These changes remove the need for iterating over the alignment multiple times. commit f4b6cfc03951215c3b9019dc47beb2913a5448ab Author: Kendell Clement Date: Fri Nov 19 00:10:05 2021 -0500 Fix CRISPRessoPooled calls to deprecated pands ix commit 751fbdb4ce432f11f3fb66c5a34f7db3d1d75dc1 Author: swrosati <40470095+swrosati@users.noreply.github.com> Date: Fri Nov 12 12:33:04 2021 -0600 Update ylabel_values -> y_label_values Typo causing error in certain modes. commit 76c80713b7b8209c9399d67f718d7ade20f20d91 Author: kclem Date: Wed Nov 10 11:37:16 2021 -0500 Update dockerfile for pyparsing commit ef15caee29380f58aaae392c897fabe47587486e Author: Kendell Clement Date: Wed Nov 10 00:45:51 2021 -0500 Fix int bug for n_reads in CRISPRessoPooled commit fc1ae890712ab2dc36d5ff36c81f64a10a2c2337 Author: Kendell Clement Date: Wed Nov 10 00:34:34 2021 -0500 Spelling fix and version bump to 2.2.7 commit 08369e294d6756f727167af50f14ff75cb4864b5 Author: Kendell Clement Date: Wed Nov 10 00:30:03 2021 -0500 Add bam input and selected demuxing for CRISPRessoPooled Adds features for providing aligned bams as input to CRISPRessoPooled and for a faster demultiplexing when amplicons and genome are provided. The parameters are: --aligned_pooled_bam: Path to aligned input for CRISPRessoPooled processing. If this parameter is specified, the alignments in the given bam will be used to demultiplex reads. If this parameter is not set (default), input reads provided by --fastq_r1 (and optionally --fastq_r2) will be aligned to the reference genome using bowtie2. If the input bam is given, the corresponding reference fasta must also be given to extract reference genomic sequences via the parameter --bowtie2_index. Note that the aligned reads are paired-end seqenced, they should already be merged into 1 read (e.g. via Flash) before alignment. --demultiplex_only_at_amplicons: If set, and an amplicon file (--amplicons_file) and reference sequence (--bowtie2_index) are provided, reads overlapping alignment positions of amplicons will be demultiplexed and assigned to that amplicon. If this flag is not set, the entire genome will be demultiplexed and reads with the same start and stop coordinates as an amplicon will be assigned to that amplicon. commit 0cdcfd7c4af834f3270e61b4db4b5f6d3da10d7c Author: Kendell Clement Date: Wed Nov 10 00:24:15 2021 -0500 Remove unused var for bam_index commit 3647ace73c95a2f44e45b6b42b4f1748d3a47f4c Author: kclem Date: Wed Nov 3 09:43:28 2021 -0400 Add --min-overlap param for flash in CRISPRessoPooled commit eea442a763e0c6c41da16a15d6e11ddf6d222dc8 Author: kclem Date: Mon Nov 1 11:25:55 2021 -0400 Remove deprecated pandas .ix from WGS commit 3e6c281c931756acd26acca96249b2fa1ad1db31 Author: Cole Lyman Date: Thu Oct 21 11:10:19 2021 -0600 Add unit tests These unit tests were in the other repo, but weren't moved over for some reason. commit 50e7cb21570ddb757904728800e31c2bcbd0d060 Author: Cole Lyman Date: Thu Oct 21 11:09:38 2021 -0600 Add Makefile to run tests This makes it easy to test the integrations cases because it will automatically install the package when needed. commit de91d51bcd9b725533a45c86ae72bc27dd5d3150 Author: Cole Lyman Date: Thu Oct 21 11:28:02 2021 -0600 Replace `np.int` with `int` Apparently `np.int` has been deprecated, so in places where precision isn't important `np.int` has been replaced with `int` according to the instructions from the warning. commit cabebbefb2646967dbeee80af08ac14156b1b53c Author: kclem Date: Wed Oct 20 12:33:49 2021 -0400 Convert cols to numeric for PE commit c2bdd96651eef5af38fb7bbc11d257a827ac080d Author: kclem Date: Wed Oct 20 12:00:43 2021 -0400 Make loggers module-specific Matplolib sometimes logs verbosely to info, so this stops using the root logger commit 8196b6a81f477ddcb0e34d61dfb54085de20c1a0 Author: Kendell Clement Date: Tue Oct 19 22:00:23 2021 -0400 Fix unicode error for bam read/write commit a923a7c2ef182238bd6b8aa77289bac487b7679b Author: Kendell Clement Date: Tue Oct 19 21:59:47 2021 -0400 All sub-crispresso processes are run with 1 process commit 75205b0d41423e4f62796e9674b0edbee68a11b9 Author: Kendell Clement Date: Mon Oct 18 23:03:59 2021 -0400 Fix #161 add params for prime editing guide analysis commit ecf23ef23e5701b232bba547a6d7d4b96f085f26 Author: kclem Date: Mon Oct 18 15:02:29 2021 -0400 Add param for plotting custom plots centered at point Add parameter to plotCustomAllelePlot script: --plot_center which if set, will produce plots centered at this point instead of sgRNAs. commit 71bf32ca789dde1d44cf41b9d3b702f12336e010 Author: Kendell Clement Date: Wed Oct 13 00:48:14 2021 -0400 Update README.md commit 53197e62706e37db54f7ed50c94f38a938955e59 Author: Kendell Clement Date: Tue Oct 12 15:43:08 2021 -0400 Fix #160 plotting for cut sites in plot 5 commit 90b43eaa03c0ea0fdee62a7b244204cad50056cc Author: Kendell Clement Date: Mon Oct 4 15:45:20 2021 -0400 Remove version checks for old seaborn and numpy, fix #155 commit 5e9dedf6bd7c7b3ef998bed811760a41b5313c6e Author: Kendell Clement Date: Tue Sep 28 21:16:54 2021 -0400 Optimize get_command_output, fix gzip binary bug commit 46953c39b769a4fa43b2ef0822dd92f07c4f1b9b Author: Kendell Clement Date: Wed Sep 22 01:58:36 2021 -0400 Update expected tests for batch commit 856354aadebc8410956e724b555224a55941a618 Author: Kendell Clement Date: Wed Sep 22 01:57:59 2021 -0400 Update CRISPRessoBatchCORE.py Move parsing of batch params to after assignment of n_processes so this value is applied to sub-processes commit f7f81747207cb279fd2c0d91986c7d4e7137fde4 Author: Kendell Clement Date: Wed Sep 22 00:23:04 2021 -0400 Fix #149 bug when read is much longer than the given reference commit 798eb4322899f70e2e2a3df0fdfad9b5598d3a41 Author: Kendell Clement Date: Tue Sep 21 17:43:38 2021 -0400 Fix #150 Open fastq.gz in 'rt' mode commit fa168843e6036c6d4ec4a5d7acb097c6a1532a98 Author: Kendell Clement Date: Fri Sep 17 12:33:12 2021 -0400 Remove requirement for python 2.7 commit 80fe12efbb0ee5c321262822b237fc8220a3f2c6 Author: Kendell Clement Date: Thu Sep 9 17:41:27 2021 -0400 Print batch summaries for amplicons with only one sample Previously, batch summaries were only printed for amplicons with more than one sample to save bits. This change produces batch summaries for amplicons with only one sample, and we shift responsibility to the user for purchasing carbon offsets to compensate for the generation and storage of these redundant bits. commit 43065310bf28e6a08780222250abd0d39ff6d0c5 Author: Kendell Clement Date: Thu Sep 9 17:38:58 2021 -0400 Add parameter 'assign_ambiguous_alignements_to_first_allele' For ambiguous alignments, setting this flag will force them to be assigned to the first (as provided by the references -a first and then -e second) amplicon. Thus, no reads will be discarded as 'ambiguous' and all reads will be counted once in the analysis. Version bump to 2.2.4 commit 11eaacd3bac9ebeabad69c1d5d92f1fd1a783a17 Author: Kendell Clement Date: Fri Sep 3 22:34:59 2021 -0400 push down crispresso2_info items commit 20272e1e95305888ca744ac56af2c08554eb9f1b Author: Kendell Clement Date: Fri Sep 3 22:32:24 2021 -0400 remove mention of pickle commit 3517285473d423dc32c6e3fdbac69eecf0caa7fc Author: Kendell Clement Date: Fri Sep 3 22:30:36 2021 -0400 minor pushdown of report name in crispresso2_info commit 8129494607488bf270567d40d43e01e043699f06 Author: Cole Lyman Date: Fri Sep 3 17:31:54 2021 -0400 fixup! Move region related objects to results in crispresso2_info commit 88a82d27e840c29b95b1602ae1594bf161108979 Author: Cole Lyman Date: Fri Sep 3 16:40:40 2021 -0400 Move some objects in CRISPRessoPooled crispresso2_info objects Move the objects: - 'final_data' -> 'results'/'final_data' - 'running_mode' -> 'running_info'/'running_mode' commit b702ab3e53e88170c7517591ce7a7050ccf1c2ba Author: Cole Lyman Date: Fri Sep 3 16:36:20 2021 -0400 Move some objects in CRISPRessoBatch crispresso2_info Move the following objects: - 'completed_batch_arr' -> 'results'/'completed_batch_arr' - 'batch_names_arr' -> 'results'/'batch_names_arr' - 'batch_input_names' -> 'results'/'batch_input_names' - 'nucleotide_frequency_summary_filename' -> 'results'/'nucleotide_frequency_filename' - 'nucleotide_percentage_summary_filename' -> 'results'/'nucleotide_percentage_filename' - 'modification_frequency_summary_filename' -> 'results'/'modification_frequency_summary_filename' - 'modification_percentage_summary_filename' -> 'results'/'modification_percentage_summary_filename' commit 2416c80e952cb58fa082ead5ef719ed7eab3eeb5 Author: Cole Lyman Date: Fri Sep 3 15:39:18 2021 -0400 Move nuc_quilt related objects to 'results'/'general_plots' to crispresso2_info The following objects have been moved: - 'window_nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'window_nuc_pct_quilt_plot_names' - 'nuc_pct_quilt_plot_names' -> 'results'/'general_plots'/'nuc_pct_quilt_plot_names' - 'window_nuc_conv_plot_names' -> 'results'/'general_plots'/'window_nuc_conv_plot_names' - 'nuc_conv_plot_names' -> 'results'/'general_plots'/'nuc_conv_plot_names' commit 37e354f94980d304e1cecf11fee95e61988dd3e3 Author: Cole Lyman Date: Fri Sep 3 14:58:36 2021 -0400 Move summary objects to 'results'/'general_plots' in crispresso2_info The following objects have been moved: - 'summary_plot_names' -> 'results'/'general_plots'/'summary_plot_names' - 'summary_plot_titles' -> 'results'/'general_plots'/'summary_plot_titles' - 'summary_plot_labels' -> 'results'/'general_plots'/'summary_plot_labels' - 'summary_plot_datas' -> 'results'/'general_plots'/'summary_plot_datas' - 'summary_plot_root' -> 'results'/'general_plots'/'summary_plot_root' - 'reads_summary_plot' -> 'results'/'general_plots'/'reads_summary_plot' - 'modification_summary_plot' -> 'results'/'general_plots'/'modification_summary_plot' commit 6df988151c602602470c944ccf561cd28f2dc30c Author: Cole Lyman Date: Fri Sep 3 14:04:12 2021 -0400 Move region related objects to results in crispresso2_info This includes: - 'regions' -> 'results'/'regions' - 'all_region_names' -> 'results'/'all_region_names' - 'good_region_names' -> 'results'/'good_region_names' - 'good_region_folders' -> 'results'/'good_region_folders' commit 053691311b158a2d3620670d20983d546a9c2c7b Author: Cole Lyman Date: Fri Sep 3 13:44:46 2021 -0400 Move 'samples_quantification_summary_filename' to 'results'/'alignment_stats'/'sample_quantification_summary_filename' in crispresso2_info commit eda3d50b6fafde4ea3145980cb2010417b799d88 Author: Cole Lyman Date: Fri Sep 3 13:27:42 2021 -0400 Move 'finished_steps' to 'running_info'/'finished_steps' in crispresso2_info commit 1da829c1c3cfd2bda9aaccca0aaa28c78a8f7271 Author: blasvicco@gmail.com Date: Mon Aug 30 17:48:02 2021 +0200 BugFix: database_id referenced before declared. commit c9321cc2cfcac34e4b6d8e1aa9fde9f25382e2de Author: Kendell Clement Date: Sat Aug 21 00:15:52 2021 -0400 Version bump to 2.2.2 for some reason the last release didn't pick up the last commits. commit 3aadab69ef1e3a44f12ee277a7767648e943a787 Author: Cole Lyman Date: Fri Aug 20 12:09:49 2021 -0400 Update filterFastqs so that the numpy arrays are writable commit 7ca129d2471aeebef2ba6d2dadf36265810f1a5c Author: Kendell Clement Date: Fri Aug 20 02:00:42 2021 -0400 Update CRISPRessoPlot.py Undo attempted matplotlib deprecation warning fix commit 8e8a65c0f29b6f99662ac1d272aa7199871f5cf5 Author: Kendell Clement Date: Fri Aug 20 01:26:50 2021 -0400 Plotting bug fixes Display and plotting fixes for batch report labels, and general plots, as well as a matplotlib deprecation fix commit 3f114752c5eca1554fcebf044b604b60a3eebeae Author: Kendell Clement Date: Fri Aug 20 00:06:38 2021 -0400 add batch summary of frameshift/splicing Closes #116 by adding frameshift/splice summary to batch Version bump to 2.2.1 Fix unicode error in slugify commit 5ad9fbc6645475885fc0f35bc26afd353a2b3d5f Author: Cole Lyman Date: Mon Aug 16 14:16:13 2021 -0400 Read plain text fastq as bytes in filterFastqs commit 6c20f28678469875392e3fc8946018b53a00256b Author: Kendell Clement Date: Mon Aug 16 13:35:12 2021 -0400 Remove test for seaborn version commit cec965feff65b4228d42784e498398b73b8caa49 Author: Cole Lyman Date: Mon Aug 16 12:16:04 2021 -0400 Fix filterFastqs This commit fixes the filterFastqs script by properly casting to strings (from bytes) in the appropriate places. It also replaces the deprecated `np.fromstring` function with `np.frombuffer`. commit 3c30e56a15fa15a3537c3d51e429c1ff8738d6fe Author: Cole Lyman Date: Thu Aug 12 09:57:50 2021 -0400 Add .gitignore file commit 2461e30770d5ae8a6686530981a4bf5c913e2f68 Author: Cole Lyman Date: Thu Aug 12 09:50:01 2021 -0400 Decode result from Popen from bytes to string This is done only where a string is necessary, the instances where this is not done are where the result is cast to a float or int (because they can accept bytes as input directly). commit 64ec8bacf9ae8cb0d3a9093ad12a916983f32207 Author: Cole Lyman Date: Sat Aug 7 13:49:05 2021 -0400 Refactor `results/refs` and `results/ref_names` crispresso2_info fields commit ebb33a7d691b524ed7ab92b5a870da09df045e00 Author: Cole Lyman Date: Fri Aug 6 20:32:03 2021 -0400 Refactor `results/general_plots` crispresso2_info fields commit 94768e209e2424394efb4d902aac4f0e4268aad3 Author: Cole Lyman Date: Fri Aug 6 14:58:25 2021 -0400 Refactor `results/alignment_stats` crispresso2_info fields commit a58b7a2b40bc99346a9ea8f6240034963b3d6b31 Author: Cole Lyman Date: Fri Aug 6 14:10:33 2021 -0400 Refactor `running_info` crispresso2_info fields commit a92ea4687e0cc69844c5d4a6250757540076ed59 Author: Kendell Clement Date: Thu Aug 5 14:28:29 2021 -0400 Version bump to 2.2.0 commit 20e9b6f228949886ebe592d4542aaddbb9fb123a Author: Cole Lyman Date: Thu Aug 5 11:52:22 2021 -0400 Remove argparse dependency Remove the argparse dependency from setup.py because argparse is a standard module in Python 3. commit ecf70ea4379e079e7669fc5ed8baabdcd11a8c61 Author: Kendell Clement Date: Fri Jul 30 01:23:38 2021 -0400 Update Dockerfile bowtie throws an error 'Can't locate Sys/Hostname.pm in @INC' This fixes it. commit 30fb34e17787858d4218eb0b0fcc1baf6259ee23 Author: Kendell Clement Date: Fri Jul 30 00:39:48 2021 -0400 Ignore python egg for copying, pin tbb for bowtie error https://github.com/BenLangmead/bowtie2/issues/336 commit c850abb672019a3684a544fccde9550f7eeb86f5 Author: Kendell Clement Date: Thu Jul 29 20:13:32 2021 -0400 Update config.yml commit eb70cb18d6910f418bb8052f45b58c05c397af43 Author: Kendell Clement Date: Thu Jul 29 17:47:06 2021 -0400 Update config.yml commit 01f079fd663027574115a0af974564d5b5efbe97 Author: Kendell Clement Date: Thu Jul 29 17:22:48 2021 -0400 Update CRISPResso2_router.py commit 2e3b5d42b8ea6705dbd2c2f59312b93390083da3 Author: Kendell Clement Date: Thu Jul 29 17:20:16 2021 -0400 Update Dockerfile commit 46d8c44f2dbe7a67531d3ccae6b0a3c51b12b9ca Author: Kendell Clement Date: Thu Jul 29 17:07:02 2021 -0400 Update Dockerfile commit 0999e82dd8e184a6b0aefa7a35fc9decaa1fc20d Author: Kendell Clement Date: Thu Jul 29 16:59:47 2021 -0400 Update Dockerfile commit 34b93cfeeb95d0a1375378996e3ca29e79fe5311 Author: Kendell Clement Date: Thu Jul 29 16:50:20 2021 -0400 Python 3 commit e040ac488b050d6f316d21196de002ef09383915 Author: Kendell Clement Date: Tue Jul 27 10:04:43 2021 -0400 Add testing file for batch, remove debug print lines commit aa7b9998c4660438f4303a57386f01617ddec1bb Author: Kendell Clement Date: Thu Jul 22 10:43:52 2021 -0400 Update Batch to allow for max processes usage commit 1566a1110215e32f338497040f1279c3581b6dcd Author: Kendell Clement Date: Wed Jul 21 01:02:24 2021 -0400 Fix reference to BadParameterException commit fea5017109ab56c955b8053eebad5f6f4218bc65 Author: Kendell Clement Date: Mon Jul 12 16:11:05 2021 -0400 Allow spaces in run names, print run name to report commit b8594354c96131ba33c9277fb719c95b1cf72f3d Author: Kendell Clement Date: Tue Jun 29 14:12:03 2021 -0400 Update CRISPRessoPooledCORE.py Fix debug statement commit 9bc9b9959cbc8faf9dc462669e333298a80a71d1 Author: Kendell Clement Date: Tue Jun 29 14:09:13 2021 -0400 Update CRISPRessoPooled for samtools sort status updates Redirect samtools sort status updates to log file. Version bump. Previously this would cause an error: `ERROR: list index out of range samtools view: writing to standard output failed: Broken pipe samtools view: error closing standard output: -1`. commit ad5b9d853ac3559e58a5ad569e7d3ac9c3d8a719 Author: Kendell Clement Date: Thu Jun 24 12:10:31 2021 -0400 Allow max processes for CRISPRessoPooledWGSCompare Users can provide -p max to use all processes available for comparisons commit 48d6c8724f541b285e5d6889723cc37fc99e5cfa Author: Kendell Clement Date: Wed Jun 23 20:53:51 2021 -0400 Create html report for CRISPRessoComparePooledWGS commit abe3054dd9b95f51cbf34e3c37e088f6beb8287c Author: Kendell Clement Date: Wed Jun 23 20:53:22 2021 -0400 Fix allele plot bug #103 If no regions are returned, max of a pandas dataframe returns an error because the df is empty commit 1ccb507858d92516e28286358d3aae9cce2cf7ea Author: Kendell Clement Date: Wed Jun 23 20:51:16 2021 -0400 Fix command prompt logo lines commit 293c249c0f380576121a123dec982e40409c977e Author: Kendell Clement Date: Sun Jun 20 00:26:34 2021 -0400 Update plotCustomAllelePlot.py Get rid of debug statements commit 947fbab70e0f4fa78cc43273b4fe2be5225043cc Author: Kendell Clement Date: Sun Jun 20 00:22:56 2021 -0400 Add plotCustomAllelePlot script for replotting allele plots commit 167a48659684ce1669da915ddb6995935c6a1aa6 Author: Kendell Clement Date: Wed May 26 11:16:51 2021 -0400 Update README with explicit instructions to activate conda environment commit af0b7e441d2a4f1e035fabfd353c58784f688371 Author: Kendell Clement Date: Sun May 23 00:22:00 2021 -0400 Update test results for more flexible pooled alignment The relaxing of bowtie parameters allowed more reads to align, which changed the expected test results for CRISPRessoPooled commit 0e08cd05c2f3279ac95a068be76f1d36a4b0224d Author: Kendell Clement Date: Sun May 23 00:06:21 2021 -0400 Fix double rows in Alleles_frequency_table due to read directionality Previous to this fix, forward and reverse reads having the same sequence would appear in different rows in the Alleles_frequency_table. This fix doesn't change counts or results, just combines the two rows in the Alleles_frequency_table. commit 7d2d761a3d4c25915be0d27b36f3b4a87068587d Author: Kendell Clement Date: Thu May 20 16:05:27 2021 -0400 CRISPRessoPooled - get rid of -k bowtie2 parameter The -k 1 parameter may not yield the best alignment. This commit gets rid of that parameter. commit 44dc9e75c660f4b3b683f1c80f5a964aa55e75bd Author: Kendell Clement Date: Wed May 19 22:52:46 2021 -0400 CRISPRessoPooled alignment more tolerant When given a genome file, CRISPRessoPooled aligns reads to the genome using the Bowtie2 aligner. The legacy parameters were somewhat strict. The new parameters reflect the 'default_min_aln_score' parameter in allowing for substantially more indels and mismatches than previous. The parameter `--use_legacy_bowtie2_options_string` has been added to use the legacy settings. Otherwise, the bowtie2 alignment settings will be calculated as follows: commit 84b870ce0d489295501aa57711edd6b18c011b92 Author: Kendell Clement Date: Tue May 4 10:22:22 2021 -0400 Raise exception if quantification_window_coordinates looks like an int Previously, if quantification_window_coordinates looks like an int when pandas parsed a batch file, it would throw an error trying to split the int. Now an exception will be raised. commit e25a6dbb13fcd0565f4c81e3ea42cfd115aa1bc0 Author: Kendell Clement Date: Mon Apr 26 16:19:00 2021 -0400 Fix bug if all reads are discarded If all reads are discarded, CRISPResso would fail because the df_alleles is empty. This adds discarded alleles to the df_alleles. commit 156d7d640355ad4755c6ce4cbab1b75bf31677c9 Author: Kendell Clement Date: Fri Apr 9 13:24:51 2021 -0400 CRISPRessoPooled - close active file in demux commit c5d9fcbbf5bd9b307c9050498d08e72dd98d42aa Author: Kendell Clement Date: Fri Apr 9 12:29:02 2021 -0400 Keep old awk command for speed for samples with <50 amplicons commit c7539661fc5bd29f4f6ee1e9c7795be932b53a8e Author: Cole Lyman Date: Fri Apr 9 12:18:45 2021 -0400 Alt pooled processing implementation (#87) These changes implement separating reads to their corresponding amplicons via Python instead of through awk. This is to get around the maximum number of open files that is limited on many operating systems. Co-authored-by: Kendell Clement commit e57d98738fa832766c78dc7eecfdb0588d92634d Author: Kendell Clement Date: Thu Apr 8 12:36:43 2021 -0400 Alt pooled processing implementation commit 4598226837cd4b726ae38f50958f977bde4794ca Author: Kendell Clement Date: Sun Mar 21 00:16:19 2021 -0400 HDR Updates - yw #82 Multiple amplicon names are resolved before adding the HDR amplicon -- unnamed amplicons are named Amplicon{i} for each amplicon. Plot 4g data (nuc pct table, mod pct table for all reads aligned to the first reference) is output and linked to from the plot display Ambiguous reads don't contribute to plot 4g data (which would otherwise lead to double counting and pct values > 1) commit d7915b2c561173238c6b0e82863f76a783db7c4d Author: Kendell Clement Date: Sat Mar 20 01:12:55 2021 -0400 Fix #72 bam_input error commit 32a9e98ffc6ceb1dd56e5e29c369176ed788c985 Author: Kendell Clement Date: Sat Mar 20 01:01:17 2021 -0400 Prime editing, fastq_out updates Prime editing input parameters are forced to be in the RNA 3'->5' direction. This makes sure that the scaffold incorporation happens on the correct side of the extension sequence. Errors are thrown if improper directionality is detected. Fastq_out now includes alignment scores and details for every run (it may be time to upgrade that SSD to hold these new fastq output files, but it makes debugging particular reads a lot easier!) Update to linked data for plot 3b in report commit c1b6ea2ecebd920ea12ab91f2d50e35c274f94fe Author: Kyle McChesney Date: Wed Mar 10 13:40:44 2021 -0700 fix missing import path on NaN commit 1b24ca351f4337e0a7bece7ef7addb4558f950d8 Author: Kendell Clement Date: Sat Feb 20 22:57:07 2021 -0500 Update links in readme to https - fix #79 commit d550fd1db8ce0e1d11ed77fd082c53a3d887bbc2 Author: Kendell Clement Date: Fri Feb 12 16:57:37 2021 -0500 Insertion quantification change + fix #76 Starting in version 2.1.0, insertion quantification has been changed to only include insertions completely contained by the quantification window. To use the legacy quantification method (i.e. include insertions directly adjacent to the quantification window) please use the parameter --use_legacy_insertion_quantification commit 798f661031236ee1aa5611f491ca135ec0432dc9 Author: Kendell Clement Date: Thu Jan 14 23:51:29 2021 -0500 Allow for int-looking chromosome names in WGS input file In CRISPRessoWGS, the region file contains a 'chr_id' column which is sometimes mis-recognized as ints when read by pandas if using the chromosome notation without 'chr' (e.g. 1,2,3 in stead of chr1,chr2,chr3). This bug fix forces chr_ids to be read as strs. commit 92c008673690a3cd32e33fe8cfcf07060cf6cb0f Author: Kendell Clement Date: Wed Dec 30 23:48:32 2020 -0500 Update README.md commit 8d590c5588d93ef0129512a5156fe3b45e2d3c4b Author: Kendell Clement Date: Wed Dec 30 23:46:36 2020 -0500 Introduction of CRISPRessoAggregate to aggregate stats across runs Adds CRISPRessoAggregate Adds start/end time to CRISPRessoBatch info Started removing pickle dependencies from Pooled and Report commit c8447246ada2758831a870f30ed99c496c18feb1 Author: Kendell Clement Date: Sat Dec 26 10:52:12 2020 -0500 Output alignment details for unaligned reads in fastq_out or bam_out commit dedc36cadc64e9302d88c3472652d2a4a2385d25 Author: Kendell Clement Date: Tue Nov 24 13:45:34 2020 -0500 Add scripts folder for one-off analyses commit 88b6e9c50c6d97da357534c67aaeba64c00ddc23 Author: Kendell Clement Date: Sat Nov 14 02:46:36 2020 -0500 Fix plot window cloning from Ref1 to HDR Plot window for sgRNA will be the same length after cloning even if the window is shorter or longer after comparing between ref1 and HDR. commit 7403a46e186c4b70ad606dbb0eebbbde684fac61 Author: Kendell Clement Date: Sat Nov 14 01:52:55 2020 -0500 Frameshift plot and HDR quant window updates Frameshift plots don't show 0-bp changes (these dwarf all other changes). The number of reads not shown are added to the legend. Addressed cloning quantification windows when bases are inserted in the clone-ee (previously these cloned bases would be ignored. Force HDR to clone all quantification windows from Ref 1 Fix #60 and #59 commit f3f9c122cc25ab62bdd7f30fb9d6ee4c9ab820a8 Author: Kendell Clement Date: Sat Nov 7 23:55:55 2020 -0500 Update histogram x-limits, caption, and data commit e9332f7eabed32e65430b44677c841dd0150ac5a Author: Kendell Clement Date: Sat Nov 7 02:26:59 2020 -0500 Fixes for when no reads align #63 commit 9fae60a57d99238e4f7a9ad2e992ae71cca49c60 Author: Kendell Clement Date: Sat Nov 7 02:08:59 2020 -0500 Standardize pie plot appearances commit f1738bd17b496a31d6f68a91ed7ae67a36f8f178 Author: Kendell Clement Date: Sat Nov 7 00:36:40 2020 -0500 plotting and pe fixes - Special bonus for y'all to keep you company during covid - axis ticks on most plots! - added parameter --plot_histogram_outliers to plot all insertion sizes in histogram - all insertion sizes are reported in .hist output files #64 - add HDR reference plot (may change this later to set ref1 to the longer reference of WT/HDR but for now it is always WT..) Allow reverse complement of extension seq if PE sequence is specified. commit 5a2d9271b3ea99342f22e59259149d30dc193e47 Merge: bd03668e 9812651c Author: Kendell Clement Date: Wed Oct 21 16:52:12 2020 -0400 Merge pull request #62 from matandro/patch-1 Fix a bug when generating compare plot commit 9812651c2aae1218393e8ce0d734812247cd59ab Author: Matan Drory Retwitzer Date: Wed Oct 21 10:45:01 2020 -0700 Fix a bug when generating compare plot Related to issue #61 This happens when N_ROWS < 1 which I assume has something to do with no results -> negative control commit bd03668ef1626a54e0b5bfbc010afacee60f45e3 Author: kclem Date: Fri Oct 2 17:39:52 2020 -0400 delete merging intermediate files commit 3c9b1344e19dee04b169c0860bc11304d6a594b5 Author: Kendell Clement Date: Wed Sep 30 23:51:08 2020 -0400 Version bump to v2.0.42 commit 2f8c6f9c7d31b276ff18000d579ef722f693c433 Author: Kendell Clement Date: Wed Sep 30 23:44:02 2020 -0400 Update new parameters, fix docker biuld problem commit f130270135b84e0904e0003858c263285834b6dd Author: Kendell Clement Date: Wed Sep 30 14:18:58 2020 -0400 Fix bug in read counting for interleaved fastqs commit faac4e1045e5b617b3743e328c0203ced27e3f31 Author: Kendell Clement Date: Thu Sep 24 22:37:50 2020 -0400 Fix bug in mode to write fastq out commit 388e8a97fc18648dd28e35063cb12680887395e2 Author: Kendell Clement Date: Wed Sep 23 23:49:44 2020 -0400 Update postrun reference output file Rename postrun references file to be more standardized with other output files. Output is now "CRISPResso2Pooled_postrun_references.txt" commit ee5be76e5e24e494abd93940622d51faa83a2371 Author: Kendell Clement Date: Wed Sep 23 23:32:34 2020 -0400 Multiple allele support for pooled mode Most common alleles for each pooled target are output if the flag '--compile_postrun_references' is provided. This writes alleles with frequncy defined by the parameter to --compile_postrun_reference_allele_cutoff This file can be manually edited to remove noisy alleles, and then used to run CRISPRessoPooled again but to provide alternate alleles to each CRISPResso run by using the parameter '--alternate_alleles'. This is particularly useful in cases where control experiments are available. The running pattern would be: 1) CRISPRessoPooled --compile_postrun_references {control} 2) CRISPRessoPooled --alternate_alleles {produced in step 1} {control} CRISPRessoPooled --alternate_alleles {produced in step 1} {experiment} commit fe5bee2ac7ca4a8dd165e2c7305a1c41a79b8d9d Author: Kendell Clement Date: Wed Sep 23 23:27:37 2020 -0400 Output + PE updates Add parameter '--fastq_output' to output a fastq file with annotations for each read. Also, the substitution frequency table is only produced in base editor mode -- in an effort to slim down CRISPResso2 output files. Also, especially for long Prime Editing insertions or deletions, the inference algorithm may incorrectly infer the prime-edited allele. I added the parameter '--prime_editing_override_prime_edited_ref_seq' which can be used to specify the prime-edited sequence manually. commit ca61c483a8a63fec2e85889f554648ea6f3a903c Author: Kendell Clement Date: Sat Sep 5 16:15:16 2020 -0400 update report caption for fig 10 commit 346c6f6e62097ea7690204cfe879ebb918fb244d Merge: e52cb734 44ccc5ec Author: kclem Date: Fri Sep 4 15:05:02 2020 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit e52cb73471f4538ecd98f943a38771f0d07a4476 Author: kclem Date: Fri Sep 4 15:04:48 2020 -0400 Update on help message for mark wildtype allele commit 44ccc5ec3ff6a57d265807b559010512f053d82a Author: Kendell Clement Date: Fri Sep 4 14:59:05 2020 -0400 Update to plotting quantification window plot vertically centered, pooled/wgs plots are not limited in height to allow analysis of large pools. commit ff675b12196593ceb8e8d8b58dfd6a8ca8865501 Author: Kendell Clement Date: Mon Jul 27 09:55:30 2020 -0400 Update parameters and description in README commit 25c6a1b45ef1e1c1243057b9b3ee06f638d55d26 Author: Kendell Clement Date: Sat Jul 25 00:29:14 2020 -0400 Update CRISPRessoBatchCORE.py If amplicon sequence is empty, auto mode is run for batches commit 2978a6ebc0cd977b36b4ea7e1965f5c43b46d325 Author: Kendell Clement Date: Thu Jul 23 08:46:01 2020 -0400 Removed WGS logging For some reason, piping output breaks multiprocessing blocking commit 8bc9214ba0a1b628b69a49a2cf306bf559e008b3 Author: Kendell Clement Date: Wed Jul 22 22:58:51 2020 -0400 Update CRISPRessoWGSCORE.py commit a9484fd481bfa8ad716c6326aba9c57f5271f885 Author: Kendell Clement Date: Wed Jul 22 14:24:38 2020 -0400 Add parameter discard_guide_positions_overhanging_amplicon_edge If run with param -- discard_guide_positions_overhanging_amplicon_edge, for guides that align to multiple positions, guide positions will be discarded if plotting around those regions would included bp that extend beyond the end of the amplicon. Normally this would cause CRISPResso to fail if plotting were requested beyond the end of an amplicon. commit 8d26edeed89e33451bd539c242f7ffaa560db3e6 Author: kclem Date: Wed Jul 22 13:58:10 2020 -0400 WGS doesn't print crispresso output to screen WGS printing error fixed commit 5ed03059d09aaf8a416c5fec553801eeca73355f Author: kclem Date: Tue Jul 21 00:00:40 2020 -0400 bam processing update of cached not-alignments commit cf593183d7b3d8200ec295ebcc653e518775046a Author: Kendell Clement Date: Thu Jul 16 21:49:21 2020 -0400 Fix naming of scaffold-incorporated reference links on plots commit 676ff623373b0cabf992017bd5eca255c4073d2a Author: Kendell Clement Date: Fri Jul 10 00:02:59 2020 -0400 Prime editing updates prime editing guides are shown as input on report output file names are produced without spaces commit 9de370148b2ada7210c10532247b3fa4b18a76de Author: Kendell Clement Date: Tue Jul 7 20:46:30 2020 -0400 Update batch for multiple quant window input commit 6ad1822f478d7f108b92f25378cc3ddf8dc3a2d3 Author: Kendell Clement Date: Wed Jul 1 16:26:38 2020 -0400 Bam processing + Prime Editing updates -Input can now be read from bam using the parameter `--bam_input` and (optionally) `--bam_chr_loc` to use the reads in the bam at this location as input. An output bam is produced with an additional soace-separated field prefixed by c2 (e.g. c2:Z:ALN=Inferred CLASS=Inferred_MODIFIED MODS=D47;I0;S0 DEL=56(47) INS= SUB= ALN_REF=TTGGCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGAAGTAGGGCCTTCGCGCACCTCATGGAATCCCTTCTGCAGCACCTGGATCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCT----------------------------------------------- ALN_SEQ=ACACCGGATGTTCCAATCAGTACGCAGAGAGTCGCCGTCTCCAAGGTGAAAGCGGA-----------------------------------------------TCGCTTTTCCGAGCTTCTGGCGGTCTCAAGCACTACCTACGTCAGCACCTGGGACCCCGCCACCGTGCGCCGGGCCTTGCAGTGGGCGCGCTACCTGCGCCACATCCATCGGCGCTTTGGTCGGCATGGCCCCATTCGCACGGCTCTGGAGCGGCGGCTGCACAACCAGTGGAGGCAAGAGGGCGGCTTTGGGC). Note that the alignment details (location, cigar string, etc) are not modified.. this may be done in the future). Bam file input cannot be trimmed or pre-processed with quality filtering. -Prime editing scaffold incorporation is now more accurate (looks for the scaffold sequence at the expected position directly after the extension sequence). A plot showing the number of bases matching the scaffold, as well as insertions after the extension sequence, and a data file with these numbers is produced. Added parameter `--prime_editing_pegRNA_scaffold_min_match_length` to define the minimum length required to classify a read as 'Scaffold-incorporated' -Renamed split_paired_end parameter to `--split_interleaved_input` for interleaved input -Auto mode now considers 5000 reads to detect amplicon sequences -Add new paramter `--annotate_wildtype_allele` to annotate wildtype alleles on the allele plots -Update output when reporting missing files -- only lists first 15 files in the current directory and directory of input parameter --reference https instead of http commit c0f2871befed86c4b314100584bf844f13d71d0e Author: Kendell Clement Date: Tue Jun 2 18:54:52 2020 -0400 Update CRISPRessoWGSCORE.py Remove debug print commit e5450b1cc6be518706969d27bda843cb7c16b082 Author: Kendell Clement Date: Tue Jun 2 18:53:20 2020 -0400 Updates for Pooled and WGS WGS gene annotations compatability fix and pooled gzip fix commit c2b286dbda3807752f34cdf6274e41d5640b408f Author: Kendell Clement Date: Tue May 12 00:33:36 2020 -0400 Fix docker bug, print version to Pooled + WGS commit 2a06cc18c3157e6c135dae7ce53adec94fe8a83f Author: kclem Date: Sun May 10 00:09:55 2020 -0400 Fix plotting bug if no sgRNAs given commit 1e3ca605e0c5ae65e8acc727667f952bc4c0d3ee Author: kclem Date: Sun May 10 00:00:32 2020 -0400 flexiguide fix commit 4e1d6b2b3424e725010e3e1a13522a7386228853 Author: Kendell Clement Date: Sat May 9 23:30:39 2020 -0400 Prime editing refinement and Pooled filesystem demand reduction - Prime editing parameters renamed, nicking guides match with flexibility - Prime editing extension seq is shown as a guide (with no cut site) - Prime editing summary plot included in report - Nucleotide plots are shaded when no changes from the reference sequence - sgRNA annotations are plotted on multiple lines if they overlap - N's don't count as substitutions - extended read analysis data available with --write_detailed_allele_table flag commit 8c584719b9771c01e53cfe409789b29a29fad665 Merge: f5069365 7624815e Author: Kendell Clement Date: Sat May 9 23:14:30 2020 -0400 Merge pull request #42 from ronaldhause/patch-1 ZipFile: set allowZip64=True to write larger allele frequency tables commit f5069365d37bd698ff3bf35e5c39a1b75e10dc1d Author: kclem Date: Sat May 9 12:04:10 2020 -0400 CRISPRessoPooled updates, fix #37 for too many files Demultiplexing in the case of amplicons + genome is parallelized to reduce sorting Only files with sufficient reads are demultiplexed and written Additional output file REPORT_READS_ALIGNED_TO_GENOME_ALL_DEPTHS.txt shows all alignment locations commit 7624815e3159d926d8a0710f674f44c616b68bd5 Author: Ron Hause Date: Sun May 3 22:50:07 2020 -0700 ZipFile: set allowZip64=True to write larger allele frequency tables Addresses terminating ERROR: Filesize would require ZIP64 extensions when trying to write compressed allele frequency tables > 2 GB commit 6af15a09033166b713df159a6ef850dde8867253 Author: kclem Date: Tue Apr 28 03:28:41 2020 -0400 int bug fixes commit 531753c0f5be89c255f65d876cba5e9bf00dd4a2 Author: Kendell Clement Date: Tue Apr 28 02:00:37 2020 -0400 Introduces support for prime editing, multiple window sizes and offsets, max processors commit 8e29c1e0966ebb2073d52a221ae77e56bb146431 Merge: adb0d8b7 039013ef Author: Kendell Clement Date: Fri Apr 24 13:06:46 2020 -0400 Merge pull request #41 from natecarlson/fix-name-error If the name column is called 'name', refer to it as 'name', not '#name'. commit 039013efe363603dfe89f10b857d58bb1ef8e8d9 Author: Nate Carlson Date: Fri Apr 24 09:46:21 2020 -0500 If the name column is called 'name', refer to it as 'name', not '#name'. commit adb0d8b791686846fa8522f46e064418cbfbdc1c Author: Kendell Clement Date: Mon Apr 20 16:26:32 2020 -0400 Update LICENSE.txt commit b514a68eea135e805333c67bf33bf0f1ea41a034 Author: Kendell Clement Date: Sun Apr 19 00:36:26 2020 -0400 Print CRISPResso command on multiprocessing fail commit 4bfadd08d30e1a8b926757c1af4a9c0c0dc0b484 Author: Kendell Clement Date: Sun Apr 19 00:03:58 2020 -0400 Index fix for crispresso multiprocessing Indexes were incremented for user enjoyment (1-based) but the more accurate approach is 0-based Also, no_rerun ignores changes to the flags 'debug' or 'n_processes' which shouldn't affect the outcome commit 867e692b326acd19ed5f291bd5699f5885b1d569 Author: kclem Date: Thu Apr 16 00:25:47 2020 -0400 Allele plot sgRNA labels stay on plot commit b0d89b4effc4242ec55ed4a3e20e8835a90e3588 Author: kclem Date: Wed Apr 15 23:58:46 2020 -0400 WGS fastq seqs are now uppercase, so guides match even in lowercase-masked genomes commit 8098f1f1dda6efdaf773b9f85622d79f32ac49c9 Author: kclem Date: Thu Apr 9 01:11:26 2020 -0400 Pooled multiprocessing updates commit 034ff2b5858dd29ab60017835695fad527a5213e Author: Kendell Clement Date: Tue Apr 7 02:16:39 2020 -0400 add n_processes param for pooled analysis commit 7fb477ff15c7f8d62d3acad4d14434942cd25bff Author: Kendell Clement Date: Tue Apr 7 00:36:47 2020 -0400 Pooled Set flag to skip reporting problematic regions commit 6c3aeff8b96d918ef2b3c518b73860d3f74480b8 Author: Kendell Clement Date: Mon Apr 6 19:56:56 2020 -0400 Pooled parallelization by chr Parallelized CRISPRessoPooled extraction to operate by chr Attempt to appease the dockerhub requrements -- require cython for compilation commit a66c4020f473d2dee80fa1257c04049b2ba6dbd3 Author: kclem Date: Fri Apr 3 13:41:38 2020 -0400 v2.0.33 plot updates allele plot sgRNAs that extend beyond plot are marked quantification window shading and right-side correction version commit 90e677f453ef971b478e4582120b17cbb572212c Author: Kendell Clement Date: Fri Apr 3 01:30:57 2020 -0400 Pooled bug fixes for regions with the same location and different names commit d795479d117e689a6679ffe463df413bff2f6a5a Author: Kendell Clement Date: Fri Apr 3 01:23:30 2020 -0400 Fix open error for docker commit 9aee866e5f65bf8df6495e81684651436a0b0a30 Author: Kendell Clement Date: Fri Apr 3 01:17:09 2020 -0400 Parallelization of Pooled and introduction of checkpoints for WGS and Pooled Alignment of amplicons is done in one bowtie2 call instead of one bowtiecall per amplicon Parallelize several time-intensive steps in Pooled (splitting by region, etc) --no_rerun flag will skip already-processed steps in Pooled and WGS commit fb1e7e25404bd80722824755c1b4ff4478449c48 Author: Kendell Clement Date: Thu Apr 2 01:34:52 2020 -0400 WGS updates - multiprocessing and no rerun WGS multiprocesses extraction of reads across multiple cores. WGS extraction doesn't occur if the --no_rerun flag is set and the files are all present. commit 45d4377f678fceeff83d60efa22dbd1078e6840e Author: Kendell Clement Date: Tue Mar 31 10:35:43 2020 -0400 Remove biopython for fastq parser Living life on the edge -- dropping the biopython fastq parser to remove biopython package dependency. This will discard the minimal error-checking provided by the biopython package. commit d52f69c9fa16ea38e919f9e3dc70fb6d53246610 Author: kclem Date: Tue Feb 25 17:05:08 2020 -0500 Pin dependency versions commit 86ff4bebcfcaa5c618ff124822ecb45de2b7c9ca Author: kclem Date: Tue Jan 28 16:17:57 2020 -0500 get rid of test for CRISPRessoCompare commit b7e96d6f4d1e0966cc0847b620349ee7fe21f43f Author: kclem Date: Tue Jan 28 15:26:20 2020 -0500 Fix problem with circleci testing Switch columns for output quantification files so that total reads is before the number of aligned reads commit ac976074c03967a386ed386d9ce3436c5fa1f2d5 Author: kclem Date: Fri Jan 24 14:02:14 2020 -0500 Version 2.0.32 - guide updates and other updates Introduced flexiguides - can match with flexibility to the reference sequence - useful for pooled screens of offtargets (parameter --fg or --flexiguide, with --fg or --flexiguide_homology to control how many mismatches) Flexiguide mismatches and other mismatches are shown on plots sgRNAs can be labeled (parameter -gn or --guide_name) sgRNA position shown on allele frequency plots detection of dsODN -- shown in plot 1d (parameter --dsODN) CRISPRessoPooled gene set input flexibility - more formats accepted plot 8 shown on html report commit ca9273377acbabf01f97f04a028d4fd87a09d6b5 Author: Kendell Clement Date: Fri Dec 6 15:14:38 2019 -0500 Fix docker bug #30 commit a3ae575f870ace49a459447e13288cfd50487e2f Author: kclem Date: Wed Oct 2 20:57:03 2019 -0400 remove debug statements commit 53d70eb83bad2aa8456b744dd29611a788f3dbbd Author: kclem Date: Tue Oct 1 15:41:39 2019 -0400 Fix #25 to accept bt2l bowtie2 index extension commit 039cc8e1b4a145e53cf06c1d52f102497928f3ac Author: kclem Date: Wed Sep 25 17:53:49 2019 -0400 v2.0.31 CRISPRessoPooled chr names fix, allele plot colors CRISPRessoPooled compatability with chr names with underscores (alternate scaffolds) Additional function to plot allele table with custom set of colors for a completed run commit c35d0151efcd70a6044399e16c841ee9d0ad0535 Merge: 491247e1 b1d1518a Author: kclem Date: Thu Sep 19 16:23:13 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 491247e1a07e2991a74341b4424c7c722a3db6a9 Author: kclem Date: Thu Sep 19 16:22:56 2019 -0400 updates to command line output, batch bug commit b1d1518a7ed4b72e92fd9d10586ccce653585630 Author: Kendell Clement Date: Fri Aug 23 11:26:53 2019 -0400 Update conda installation path commit cdfd78772de27bdb78a380e591d8857acd143562 Author: kclem Date: Tue Aug 13 11:24:58 2019 -0400 Use python 2.7 pandas commit 1417d1aa387d45f37953b93304a545e08943cc45 Author: kclem Date: Tue Jul 16 17:18:28 2019 -0400 force merge reads and fix #19 add optional param for force merging R1 and R2 in case they don't cover the entire amplicon fix labels for expected amplicon plots commit 60f90053ab65958871402b609d0b72b31021bc6b Author: kclem Date: Tue Jul 2 17:28:27 2019 -0400 v2.0.30 case-insensitive guides, fix #17 commit 702b9dce89e070d96064db314ac1c5155fedd42e Author: Kendell Clement Date: Tue Jul 2 16:18:17 2019 -0400 Update README.md commit 5a10eb9a9d24371d2bedf8b173f9c7401d7cbd53 Merge: bc7b3321 bc006c82 Author: kclem Date: Tue Jun 25 15:32:51 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit bc7b3321e1bb6b7ed0c8af48d609e86e55081f6b Author: kclem Date: Tue Jun 25 15:32:45 2019 -0400 plot window bug update commit bc006c826f338b9a1fcaec2286b506bdd2c548db Merge: 1f7171b8 feee2c2e Author: Kendell Clement Date: Thu Jun 20 00:56:45 2019 -0400 Merge pull request #16 from PEHGP/patch-1 args.trimmomatic_command for CRISPRessoPooled commit feee2c2eb20807933376f01a88102767ce5e42e4 Author: kuan <396777306@qq.com> Date: Thu Jun 20 09:44:32 2019 +0800 args.trimmomatic_command args.trimmomatic_command commit 1f7171b8eb2dcd63fada80fa6cac561e576f727c Merge: f6c9eed2 3d4a37a6 Author: kclem Date: Thu Jun 13 13:13:33 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit f6c9eed206b16fede7c88724c94d8d091f8657f3 Author: kclem Date: Thu Jun 13 13:13:20 2019 -0400 update pooled names commit 3d4a37a626f366f8f024ee7f62e017e8d68092b3 Author: Kendell Clement Date: Tue Jun 11 12:33:40 2019 -0400 Add web link to readme commit 89348066ede5c447db9f04b2337118a0624b8f63 Author: kclem Date: Tue Jun 11 11:14:55 2019 -0400 add batch percentage report commit 3bcd50d67c9b320c9f908eb2e3469143461e7df6 Author: kclem Date: Thu May 30 17:25:05 2019 -0400 document report param commit 79303435747a70b288b88bb076dda90b8d379f91 Author: kclem Date: Thu May 30 17:16:11 2019 -0400 output updates commit 9f14424f275f132a148308042db48c77cbda9b1d Author: kclem Date: Fri May 24 17:57:57 2019 -0400 plot updates, compare bug #12 commit 7f5482c411a3d56a73d7f0c66f0159b623653c2a Author: kclem Date: Tue May 21 17:47:08 2019 -0400 remove crispressocompare test condition (cuz has floats) commit 5e2f4094ec607f63981cd5aa2ee7f99ce1a20d12 Merge: aac9dfe7 5933d93c Author: kclem Date: Tue May 21 17:40:44 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit aac9dfe70292a90d6981b0859ff9ede8da7094a4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test changed test to something that won't be affected by float formatting commit 5933d93c5f1408f754895254306701b5b0eaadd4 Author: kclem Date: Tue May 21 17:18:46 2019 -0400 update circleci test commit 7d15cdfaa4a323f4baad96bf36c97fd455dd6684 Author: kclem Date: Tue May 21 17:11:09 2019 -0400 Add compiled c file commit 50134d64d33dc984ac81a066e7c370b160b861b3 Author: kclem Date: Tue May 21 16:58:57 2019 -0400 v2.0.28 Add report for CRISPRessoCompare Standardize naming conventions for files and plots from CRISPResso Add data links to CRISPRessoBatch report CRISPRessoBatch plots using the plot window around the cut site instead of only the quantification window If only one reference, 'Reference' is not shown in data plots or as a file prefix Set plotting indexes once for each guide (previously, they were specific to the amplicon) Base editing plots now plot for each guide (previously, they were one for each amplicon) commit 9e86bef89884a0e5980c7781cfcd56243fdd42f0 Author: kclem Date: Wed May 15 14:28:14 2019 -0400 Standardize concept of windows for quantification and plotting #11 commit dd63974334c94816efe5878e4aab1ec3c3e1a6c5 Author: kclem Date: Wed May 1 14:49:24 2019 -0400 python division bug.. <3<3 commit 4a4b88885ab2e034bdcbca9566aebe911c58f427 Author: kclem Date: Wed May 1 14:35:33 2019 -0400 fix bug for spaces in filepaths commit f7e0aee5d948abd876b5048c87cae59e265d0c6f Author: kclem Date: Fri Apr 26 11:56:51 2019 -0400 min merge size commit 12cc6a93c0b02be86575c6c7fe8b7569a96e0edd Author: kclem Date: Fri Apr 26 11:41:09 2019 -0400 Relax flash merging, add parameter for stringent flash merging (#7), remove debug statements commit 38bfb3174d8d37297587243cb9e04469e7e54a20 Author: Kendell Clement Date: Thu Apr 25 22:50:08 2019 -0400 Update CRISPRessoPooledCORE.py fix numpy -> string error commit 273fe3a1ab731a32c26efdcab58a502cd2162104 Author: kclem Date: Mon Apr 15 13:38:17 2019 -0400 Fix CRISPRessoWGS tests commit 2c1ec9f5eadaf62ac7df35535aa26965d993e96a Author: kclem Date: Mon Apr 15 13:15:03 2019 -0400 add tests for CRISPRessoWGS commit 6632700fbde6b7f5e49e402471fea88a0966d2c0 Author: kclem Date: Fri Apr 12 10:29:43 2019 -0400 update docker run message, enable local testing, remove networkx commit c31355e15afc49f6aa069968c94408f5f7b484c0 Author: kclem Date: Thu Apr 11 16:00:44 2019 -0400 ignore test directory for docker commit aad55c922fa9b3948e5b194b26da3a8a3ccc97d2 Author: kclem Date: Thu Apr 11 15:52:45 2019 -0400 fix plot label bug for 10a, update fig 2a data commit c40b2de4f21e69404de153b9e12dee6654568b67 Merge: 560b65e7 88990dbb Author: kclem Date: Wed Apr 10 17:30:41 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit 560b65e720a6dc5329203ecbc8c1b38b47aee219 Author: kclem Date: Wed Apr 10 17:30:34 2019 -0400 update tests for decimal shift for percentage in summary report commit 88990dbb29135d160879ed02dcac6874b0fed9f2 Author: Kendell Clement Date: Wed Apr 10 17:26:20 2019 -0400 Update README.md commit e4deb9211dadb3e9622f2eee38f0cc4930a0cf17 Author: kclem Date: Wed Apr 10 17:19:07 2019 -0400 v2.0.28 commit 098f5a68ba632987930fd70038af667e228b7a1f Author: kclem Date: Tue Apr 9 22:13:13 2019 -0400 update circleci path commit bb78ade66d20a826bbd8beee53711620869b86aa Author: kclem Date: Tue Apr 9 17:40:43 2019 -0400 circleci test path update2 commit e760c77bdd23fd087733c26eda86f15de6877bbf Author: kclem Date: Tue Apr 9 17:36:23 2019 -0400 circleci test path update commit 0e1b576959b09da72b101983d312842e06ea4c56 Author: kclem Date: Tue Apr 9 17:33:22 2019 -0400 circleci artifact storage commit 28c23d2172cf5c39faffd1ea498f6d771237567d Merge: c7da8466 e42b3a6b Author: kclem Date: Tue Apr 9 14:17:34 2019 -0400 Merge branch 'master' of github.com:pinellolab/CRISPResso2 commit c7da84668556b897b43dd53eb2370c3404ab3fa2 Author: kclem Date: Tue Apr 9 14:17:23 2019 -0400 circleci testing for batch and pooled commit e42b3a6b4c11cab0ac9c29c378858706b7fd328e Author: Kendell Clement Date: Tue Apr 9 11:55:04 2019 -0400 Update badge links commit 6a435376fe43e6408507f1078518515f1f522ba8 Author: Kendell Clement Date: Tue Apr 9 11:42:21 2019 -0400 Got me some badges! commit f8c9713444472f5e3cef4fd7f7bce0704dd6a266 Author: kclem Date: Tue Apr 9 11:07:18 2019 -0400 circleci artifact update commit 8d4b825dcf01887db96b90640aafea564031a7e9 Author: kclem Date: Tue Apr 9 11:03:59 2019 -0400 circleci updates commit bfb3bc82abd22647554b80517554ef58e81d2090 Author: kclem Date: Tue Apr 9 10:51:53 2019 -0400 add expected results for circleci commit 8466e146d73a2c2149fdbcd67d4db656fe8c13e0 Author: kclem Date: Tue Apr 9 10:29:57 2019 -0400 circleci update commit 19c437975e57119531bcb0b9b2a6587a7d6b3ea7 Author: kclem Date: Tue Apr 9 10:14:42 2019 -0400 circleci update commit c665c19c97f4c6d4b45f40b3ac9522bf53ce05ba Author: kclem Date: Mon Apr 8 16:27:38 2019 -0400 circleci - use custom docker commit caa46ce2bc1de5e100bfd5db25179fb6b54f4d95 Author: kclem Date: Mon Apr 8 14:45:16 2019 -0400 python2 virtualenv commit a13e2fd2c9eea59652ec1ad7c6a9862a708bc33e Author: kclem Date: Mon Apr 8 13:54:32 2019 -0400 CircleCI testing commit 7257b54f77391da21532bf06ed84b1ce03d460e0 Author: Kendell Clement Date: Fri Apr 5 11:15:22 2019 -0400 Remove dependency on zip commit 7b694f507a61e424e994384cd765855d7f69dfb4 Author: kclem Date: Thu Apr 4 12:12:37 2019 -0400 add networkx requirement for py27 commit 099acbc8149dbbd5c99729ddc8e0928e3f890023 Author: kclem Date: Thu Apr 4 10:16:10 2019 -0400 Fixes for dockerhub commit 3a3bfbdd2373348901faac6fda468b70cb5ce725 Author: kclem Date: Thu Apr 4 10:09:06 2019 -0400 Bioconda submission fixes commit dff86e16812f2b9345ef86bd3185786f4f82d25a Author: kclem Date: Wed Apr 3 18:49:13 2019 -0400 More precise cleavage window and quant window plotting commit c8caf7fbdad415d4d3fb47cc5b9b0b76dd65deab Author: kclem Date: Wed Apr 3 17:49:54 2019 -0400 v2.0.27 add reports for pooled and WGS commit c68e3cb922d037c12a62c695c0ae1f6c148ace82 Author: kclem Date: Mon Mar 18 11:15:40 2019 -0400 batch info pickle, WGS bai location, meta mode commit 768c75c3a7864c365cf13fd65573859b9aa86ebe Author: kclem Date: Wed Mar 6 17:13:56 2019 -0500 v2.0.26 add report display name, remove paths from stored files, fix sgRNA plot, CRISPRessoPooled report HTML, add citation to report commit 58257b54fc440427e5437f8e7458fd5824020b6e Author: Kendell Clement Date: Fri Mar 1 16:55:27 2019 -0500 Update issue templates commit 50fb2d58f0d3777ba51d0f5a37e82cfc1a47ebff Author: kclem Date: Fri Mar 1 16:38:13 2019 -0500 Fix file naming and flash incompatibility commit eca34aebf86dc1b03c6107ee405cdf16898c8d51 Author: kclem Date: Wed Feb 20 17:26:42 2019 -0500 v2.0.25 Add inferring of guides commit 2ccec08691db17e6a2cfab8e310f5f29621319ca Author: kclem Date: Wed Feb 20 13:42:57 2019 -0500 quant window updates commit 21d558da1f8f5fed8f59de0eb154e5a8a505ff7a Author: kclem Date: Tue Feb 19 17:21:12 2019 -0500 add flash outies, fix quant window coords bug commit 22954e6d40ad2d237cb75c521a09f30c2b294066 Author: kclem Date: Tue Feb 19 11:17:30 2019 -0500 Update entrypoint for docker commit 0216329d8a8d1c072a269d14786820f943443153 Author: kclem Date: Wed Feb 13 16:40:39 2019 -0500 v2.0.24 update docker, setup.py commit e11b60fe1cf3c3e67e41964d45e843f28c2975a5 Merge: fb1687b8 d6a7b979 Author: kclem Date: Mon Feb 11 16:22:09 2019 -0500 v2.0.23 suppress plots, custom flash version commit fb1687b87de0cb7e5d1ab0acc2a2651d1be1fcec Author: kclem Date: Mon Feb 11 16:12:26 2019 -0500 v2.0.23 suppress plots, custom flash version commit d6a7b979e1ecd49b26c648f2007e1ecfa905ec14 Author: Kendell Clement Date: Fri Feb 1 12:20:50 2019 -0500 Update readme formatting commit 9e4c87c4f60edd82539b01b24f646962c0f06f4c Author: Kendell Clement Date: Thu Jan 24 17:13:28 2019 -0500 Add conda install instructions commit 4b2b06e52ee1aba4f3bdd0458c13813f2417fbf7 Author: kclem Date: Thu Jan 24 14:02:05 2019 -0500 add manifest.in commit 495e1f9829d6e72048b87a6972472c4242411286 Author: kclem Date: Wed Jan 23 11:05:44 2019 -0500 Change license location, license update commit 24c3b1b6ba66557b99469c0e12291fae2fcc800d Author: kclem Date: Wed Jan 23 11:01:44 2019 -0500 v2.0.22 commit 4a426bf715e37cbb8d619d54fc3a92385e9dcf1b Author: kclem Date: Tue Jan 22 15:25:13 2019 -0500 update batch amplicon naming commit f4fbe96b335c6e1a5b3dc252bf090e3d36ffb8cd Author: kclem Date: Tue Jan 22 12:44:00 2019 -0500 v2.0.21 detangled root location dependency from params commit 9d29737de257a2999f0763afd5f97ddafd6d2fe7 Author: kclem Date: Tue Jan 22 12:36:03 2019 -0500 Update CRISPRessoShared.py commit 573aa90cf70cd941c3e26361b2e656fbf34d8e5e Author: kclem Date: Fri Jan 18 18:10:47 2019 -0500 allow no cython commit 69811cf1eb58ac014a1ac34c56748aaffcead187 Author: kclem Date: Fri Jan 18 17:55:23 2019 -0500 require cython for installation commit 37ac0e6278c4825bb816c7cfe0a1fc3819abd516 Author: kclem Date: Fri Jan 18 17:44:19 2019 -0500 v2.0.20b prepare for bioconda integration commit 31c5ad02127366d0ace2849a6e996a86d690f4d1 Author: Kendell Clement Date: Tue Jan 15 17:22:53 2019 -0500 Update README.md commit 5b8cf82bfee3eeefcf2ba5bb31b4a60b25960df0 Author: Kendell Clement Date: Tue Jan 15 16:46:46 2019 -0500 Update README.md commit c932b8b4ad7c08d3fc94d5c57d0017001a78a42f Author: Kendell Clement Date: Tue Jan 15 16:40:59 2019 -0500 Update README.md commit 5cf5aa34b2ef97e38e45ee3394cd8b4aade50c6d Author: Kendell Clement Date: Fri Dec 21 15:03:50 2018 -0500 Add trimmomatic_command parameter commit 2ec374962c169c1a56cd48ff9080f7941433d016 Author: kclem Date: Thu Dec 20 18:30:01 2018 -0500 v2.0.19b HDR and WGS changes commit 78483624993c6fdb5ccc889a2a6f37036fb0f2c8 Author: kclem Date: Thu Dec 6 14:55:18 2018 -0500 add filtering for fastqs commit 17940c70b36d74d8b9134664b515f3b164c678b0 Author: kclem Date: Wed Nov 28 15:00:35 2018 -0500 v2.0.18b - fix bug with batch names, add param to suppress plots commit 038042e3cea983fc264460402d88e15766cc50b0 Author: kclem Date: Wed Nov 14 15:00:47 2018 -0500 Add router for docker commit 3ef96cc08a18f6eff9bb6115366e27304986ed27 Author: kclem Date: Wed Nov 14 14:52:41 2018 -0500 add Docker file commit 3e1cbf1dffc4dbc555942d45f91ad942237b81c3 Author: kclem Date: Wed Nov 14 14:29:44 2018 -0500 set white background for plots commit dd2cb254170b8363a7feb0427ed587d2093ebd3e Author: kclem Date: Wed Nov 14 11:13:02 2018 -0500 Set seaborn style commit dde6a675a5ba4e9ec100c29c2d59534aa39ef4fc Author: kclem Date: Tue Nov 13 16:13:55 2018 -0500 Fix line endings commit db0a67017f57c5a77bed9eb442cb7efa9df4b97a Author: kclem Date: Tue Nov 13 10:31:45 2018 -0500 v2.0.17b - bug with multiple references of different lengths commit 7f4afffa1485094aee4c0399a319a30c12ec473e Author: Kendell Clement Date: Tue Oct 23 17:22:07 2018 -0400 Update README.md commit 0843923b50d2db953600230ac23614f52591c4b4 Author: Kendell Clement Date: Tue Oct 23 16:59:48 2018 -0400 Update README.md commit 6f79084ce88502a40fe8b9b1839733ca224d14dd Author: Kendell Clement Date: Tue Oct 23 16:56:38 2018 -0400 Add files via upload commit 72d0b355b5d39164405b6311bda6231dbbdef371 Author: Kendell Clement Date: Tue Oct 23 15:44:26 2018 -0400 Update README.md commit 30fdc7fd5d73594b332efd546a1f4d85004a80d6 Author: Kendell Clement Date: Tue Oct 23 13:48:25 2018 -0400 Update README.md commit 5b11e51083ca87cbc1a8dff02b4634fe12176d29 Author: Kendell Clement Date: Tue Oct 23 13:42:11 2018 -0400 Update README.md commit 33d367310d702667d86202aba389a0fee4eba691 Author: Kendell Clement Date: Tue Oct 23 13:24:50 2018 -0400 Update README.md commit d6c647d32cdded7803f6b023949202c9486f5caa Author: Kendell Clement Date: Tue Oct 23 12:01:41 2018 -0400 Update README.md commit 5cb8fccf048a49b3798f6932c0b0db90569697fa Author: Kendell Clement Date: Mon Oct 22 17:42:41 2018 -0400 Update README.md commit 7b60f691450c932b5d32ff48218f187328d0726f Author: kclem Date: Tue Oct 16 15:27:34 2018 -0400 2.0.16b - batch mode report commit 6037c2945efdea6fecf67b037a70c30d4bc6696b Author: kclem Date: Fri Oct 12 18:14:27 2018 -0400 2.0.15 updates to pooled, adjust merging commit 326e1c9f370e1d6956ad565605309f37e214e927 Author: kclem Date: Thu Oct 4 10:28:07 2018 -0400 2.0.14b - adjusted flash overlap params, cannot take mult aln gap penlty commit 26ec80198dc06ca09eb525deaf387c330f701cac Author: kclem Date: Fri Sep 28 15:13:00 2018 -0400 default val for n_processes commit bec0d312629d006169495b241fb9ea0018380e62 Author: kclem Date: Tue Sep 25 10:55:21 2018 -0400 2.0.13b commit 51d1386856849af7f04be0ce587e97349c7149b8 Author: kclem Date: Wed Aug 22 18:18:23 2018 -0400 Produce report commit 017c409c19a6af99c09c97faff67c26ac902e157 Author: kclem Date: Mon May 14 16:44:32 2018 -0400 Initial Commit of files commit 4324c954cc4efa10fc01fc6d69f88253ef5a7483 Author: kclem Date: Mon May 14 16:31:29 2018 -0400 first commit commit a433b57c059e5c3fbba6dc4dbd6c66a4843741ad Author: Cole Lyman Date: Thu Dec 14 16:05:24 2023 -0700 Add in buttons to reports and row for multiReport styling (#17) commit e1a3d81115255a3db4107150af9162d43d4a2d29 Author: Cole Lyman Date: Tue Dec 12 14:50:45 2023 -0700 Run metadata (#16) * Reports refactor (#37) * Changes necessary for selenium tests * Changes to make interleaved and pooled tests possible * PopulatePooled error * Changes for pooled testing * Changes for WGS selenium test file loading * Changes for WGS selenium tests. All tests functional. * Files for testing * Writing text for pooled * Add smallGenome.fa * Jinja partial for color picker and pip install in dockerfile * Names for color fields * Color picker input added to cmd_to_run * Adding color routes to other versions * Fix for responsiveness on cup and title * Adding color-picker partial to wgs and pooled * Tabs for different style options * New style menu with tabs * Style menu completed * Rough framework for style admin page * Working style FileAdmin, access button, and further partial refactoring * Checkbox for custom colors that shows and hides color selectors, box on home page for style folder * Optional save file * Updated Docker file and style_files.html * Changes to pooled and wgs, reset Dockerfile * Add style files to pooled and wgs * Adding style_files to partial * Debuging * Adding styling * Colors function refactored and working for all types * Remove style from Compare * Style file check * Style dropdown - allow save json only for admin * DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files * Restyle the color pickers These changes make it so that the colors are in a row instead of a column when the screen is wide enough. They are also responsive, so when the screen is small the color pickers will return to a column. * Add margins around style file elements * Move style folder inside of server folder * Refactor styles to be part of the database instead of files This allows for greater flexibility in displaying and updating them through the web interface. * Refactor `style_handler` to read the style from the database This completes the refactor to store style parameters using the database instead of storing the styles as files. * Fix error when the default style can't be read from the database The intended and proper behavior is that no style parameter is added to the command, and that is what happens now. * Restyle the colors in the admin view Add border around the colors in the admin view as well as moving them to be more vertically centered with the name. * Succesfully implemented selecting default style * Implement color pickers in style admin view * Refactor saving style files when there is no name specified * Remove style file card from admin index page * Add some default styles and rename the default to "Original" * Rename style_file references to style This is a final clean up of refactoring how the different style properties are stored. They are now stored in the database, so it doesn't make much sense to call them style files. * Implement creating styles from the admin panel * Move where the style files are stored in Docker * Implement Docker Compose for both production and development * Replace sub, ins, del with Substitution, Insertion, Deletion * Replace WSGI with Gunicorn for both development and production This allows us to hotreload the code when running the development version and also adds the Flask Debug Toolbar Extension which will be helpful in debugging. * Add a Makefile for commonly used Docker compose commands * Add reverse proxy to make Apache redirects work properly * Layout and report update * Bootstrap 5 changes * Working, missing custom label * Working file upload in partial. * Changes to submission.js for bootstrap 5 and load file upload partial * New submission.js template file * Jinja partials for all submissions * Move styling to main.css file * Add server file to render js * Commit before adding subtree * Removing reports found in subtree * Adding files * Web updates refactoring done * Add function to render template partials without using Flask * Use the `render_template` function for each report * Update path to template directory to include `CRISPRessoReports` * Update indentation in report.html and extract log params into partial * Added a few changes from the selenium-tests branch on C2Web * fig_reports and replacement * summaries partials and html updates * Fix error when rendering multi reports * Layout.html for C2WEB and CLI * Bootstrap 5 and partials changes * Path correction * Jinja choice loader * Subtree working * Centering issue and submit button fix * Styling and bootstrap changes * Spacing changes, submission_compared fix, and submission_wgs file upload fix * Remove extra files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca933c065a0f0765ce899eb537ab1ace41d43 * Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes * Removing CRISPRessoReport files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca933c065a0f0765ce899eb537ab1ace41d43 * Removing unecessary logic from submission_compare * Set up standalone Apache server container in Docker Compose * Add C2Web conda environment file * Make C2Web Docker image smaller (now 2.25 GB uncompressed) * Change style to styles * Fix for updating labels * Make C2Web Dockerfile multi-stage and make CRISPResso2 hot-reloadable These changes modify the C2Web Dockerfile to be multi-stage, so that there is a base stage (shared between dev and prod), a dev stage, and a prod stage. The dev stage doesn't install CRISPResso2, but binds a local copy of CRISPResso2 so that it can be hot reloaded. In the prod stage, this installs CRISPResso2 via conda. * Clean up Dockerfile and add CRISPResso2 dependencies to C2Web Docker These dependencies (plotly, seaborn-base, and matplotlib-base) are added so that they don't need to be added when CRISPResso2 is installed. * Final style fixes, color circles for style files * Update README.md with Docker compose details and update ignore files * Install CRISPResso2 in the build stage of Dockerfile * Removed docker-compose.prod.yml and created docker-compose.public.yml Also, updated the Makefile. Now, the default docker-compose.yml should be a suitable configuration for client facing production. The docker-compose.public.yml is a good configuration for the public facing site and the docker-compose.override.yml is a good configuration for development. * Share environment variables between web and celery and update README * Add spacing around body and footer tags * Replace spacing utilities classes with Bootstrap 5 versions * Resize images and fix filepath * Hide base editing if checkbox unchecked * Base editing partial * Add padding around pegRNA radio buttons and plot window size Also, add a margin around the JSON file upload box. * Increase size of Submit buttons * Fix hamburger menu and add -bs- to data-target and data-toggle * Fix plot window size spacing in Pooled * Fix the vertical span of the input labels in WGS, Pooled and Batch * Fix Pooled layout * Make input labels in the forms the same width * Remove ALLOW_USER_STYLE_UPLOAD parameter * Remove jinja loader from report_routes * Reformat style in submit_routes.py and update docs * Convert tabs to spaces in style_selection.html * Use string interpolation instead of concatenation in submission.js * Add an authentication check before exposing server_files in submission.js * Fix indentation and convert tabs to spaces in many templates * Clean up old files and comments * Replace tabs with spaces and reindent template files * Update README.md with git alias and subtree information * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 461ca93..1efad70 1efad70 Replace tabs with spaces and reindent template files e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle df896b0 Resize images and fix filepath 2f70855 Add spacing around body and footer tags 5cd6d27 Final style fixes, color circles for style files f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' 321815d Removing CRISPRessoReport files 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 04558fb Remove extra files 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix 980fdc4 Styling and bootstrap changes 9d40474 Centering issue and submit button fix aa5071c Subtree working db30843 Jinja choice loader 3b67ac0 Path correction 3aaca48 Bootstrap 5 and partials changes 8f5d8a1 Layout.html for C2WEB and CLI 290d829 Fix error when rendering multi reports 546397a summaries partials and html updates 858a751 fig_reports and replacement 073f1fe Added a few changes from the selenium-tests branch on C2Web 1061ebb Update indentation in report.html and extract log params into partial c3781e9 Update path to template directory to include `CRISPRessoReports` 84e0969 Use the `render_template` function for each report ee721b3 Add function to render template partials without using Flask 08fcd4e Web updates refactoring done 99c8e22 Adding files ef333f0 Removing reports found in subtree 1bae0df Commit before adding subtree 1fbb427 Add server file to render js d1d6fdf Move styling to main.css file 1241569 Jinja partials for all submissions 0534637 New submission.js template file c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial ecd03f6 Working file upload in partial. ce5d20f Working, missing custom label 6ba73e7 Bootstrap 5 changes e05d146 Layout and report update 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion ea44128 Move where the style files are stored in Docker 7f03e98 Implement creating styles from the admin panel 9b27a2e Rename style_file references to style a233d10 Add some default styles and rename the default to "Original" 43a8d29 Remove style file card from admin index page 1a8f332 Refactor saving style files when there is no name specified 64a7b1c Implement color pickers in style admin view 17c93c1 Succesfully implemented selecting default style fd79cdd Restyle the colors in the admin view 3cd94b8 Fix error when the default style can't be read from the database 5e626bd Refactor `style_handler` to read the style from the database 0f66d4a Refactor styles to be part of the database instead of files 6c7d3c8 Move style folder inside of server folder 9f71f21 Add margins around style file elements 2a28549 Restyle the color pickers 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files dc4f2c7 Style dropdown - allow save json only for admin 15e7483 Style file check 7bd0e91 Remove style from Compare 0ab45f5 Colors function refactored and working for all types 2e24f8b Adding styling d6621f1 Debuging ed00c82 Merge with master 5150f9b Adding style_files to partial 957a9ca Add style files to pooled and wgs 66dc2d3 Changes to pooled and wgs, reset Dockerfile fa6b1cf Updated Docker file and style_files.html ee0fcfc Optional save file 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder 0f26e2c Working style FileAdmin, access button, and further partial refactoring b3b70bd Rough framework for style admin page e4731d7 Style menu completed 1bb37bc New style menu with tabs 58f7e56 Tabs for different style options 3de893d Compare (#34) e66bef1 Update AWS EB instructions.docx 658a218 Fix bug when trying to send recovery password with bad email creds ee32e36 Adding color-picker partial to wgs and pooled 34ea688 Fix for responsiveness on cup and title f0c4d07 Adding color routes to other versions 110fe14 Color picker input added to cmd_to_run e732478 Names for color fields 2934631 Jinja partial for color picker and pip install in dockerfile 48bbf9c Cup animation (#33) 2905248 Selenium tests (#31) 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides 570e42a Don't remove commas from amplicons or guides 0d70425 Add smallGenome.fa fc33197 Writing text for pooled dccfcb3 Files for testing 4cea67c Changes for WGS selenium tests. All tests functional. ff05713 Changes for WGS selenium test file loading 495a98d Changes for pooled testing 0ad86a5 Merge pull request #30 from edilytics/pooled-upload-fix 127eb8f PopulatePooled error 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests 7847687 Add link to CRISPRessoWGS from profile page and change header 666f73b Remove example block from CRISPRessoWGS submission page 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled 8d979a4 Fix bug where files_to_delete was being replaced and standardize append 09e55fc Changes to make interleaved and pooled tests possible f89eca8 Changes necessary for selenium tests 3efe4f9 Clean up test files a696363 Merge pull request #28 from edilytics/s3 dcef708 Remove changes for CRISPRessoCompare e0c79cf Add demo config file for eb 03aba8e Update AWS EB instructions.docx a671c4e Set version to 2.6.3 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled da5b15b Timezone for history is displayed in user local timezone e11691f Update history to show time of previous run be675fb Update pooled with s3 4c7d429 Add data links to pooled report 353e88f Update admin portal landing page 712e828 Show run type in history 2802252 s3 and user updates efc3ed8 S3 error catching af68341 New S3 Validation f7d64e0 AWS validation before submission 8446093 Update s3 for batch and paired modes 0e7d327 S3_Upload function imrpvoed -JF b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 c991d52 added s3 user database model ab4aa54 add model for s3 bucket 853cda9 S3_Functionality improved -JF 2f060a6 Implemented front-end s3 browsing e082a5f stub out viewing method c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length c85a93f Merge pull request #15 from edilytics/wgs-interface 712270a Add support for CRISPRessoWGS deaacee Extract out function to get server files in submit_routes 151eb15 Update crispresso2_info object fields b2a974d Bump CRISPResso verion to 2.2.4 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 7f2dc1c Stop trimming json error messages, fix #11 d28c03b Update reporting logic to use the new CRISPResso2_info schema 03ee46f Bump CRISPResso version in Dockerfile and download release from Github 9151c5d Add CRISPRessoPooled report template 25a6e37 Merge pull request #6 from edilytics/pooled-interface b47d288 Check length of amplicons for hosted version, closes #4 54c28b6 Update submission file extension check 8fcadee Add a link to CRISPRessoPooled interface in user dashboard 7fd0283 Implement CRISPRessoPooled backend and report functionality 4063eb3 Modify submission.js to accept .txt and .tsv files b770323 Create template file for CRISPRessoPooled submission interface d4f2ed0 Merge pull request #5 from edilytics/flask-modularization 8527384 Convert some celery configurations settings to new format 962a209 Install less and vim in Dockerfile c693668 Read CRISPResso2_info from json files instead of pickle files a469e08 Move LoginManager to user_routes.py f62e67a Create db tables in init_db.py 0d85c90 Move login_required to user_routes 6f5e33e Reformatting of remaining __init__.py e615c0b Extract report routes out of __init__.py 20f2601 Extract user routes out from __init__.py 5582612 Extract status routes out from __init__.py 2406a10 Extract submit routes out from __init__.py b562fcd Extract celery tasks from __init__.py faa785d Extract views out from __init__.py ff44576 Extract model classes out from __init__.py 914498f Merge pull request #3 from edilytics/2to3 86ea7da Replace RabbitMQ with Redis adca9fb Upgrade celery to version 5.0.5 244ec33 Convert from Python 2 to Python 3 28b4f37 Refactor Docker image to use Python 3 via micromamba 2359800 Allow interleaved batches 428720b Add features: Allow admin init, server discovery depth 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon 5062365 Update README.md 51e02f4 Update README.md ac4a6d5 delete other images 4f3ad88 Update README.md fc0de1d Update README.md 08defa1 Update README.md 9604983 Trycatch pickle loads c1facd7 get rid of debug print of email d699d4d crispresso2.0.45 e7ff079 Update param descriptions 1f12d59 2.0.44 b81febe crispresso to 2.0.42 1a967a8 update report 178c56d 2.4 e41076d Job expiration 41d1a4c check progress on setinterval 756e488 server-side files ad19c3c Update to crispresso 2.0.40 prime editing e3a194a update errors and ignore email config 2efb0bb Update README.md 58844a6 initial commit 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 1efad706b7c147dd45601111d697534fa993c007 * Indentation and parenthesis * Semi-colon to README * Add targets for .env and clean in Makefile * Add flower support to the development version * Add support to map individual directories in Docker Compose * Fix typo in README * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 1efad70..13f7ae2 13f7ae2 Fix command used and parameters elements. Increase print width and height to 100% dcf7391 Adding styling for print-only and screen only git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 13f7ae215990b155ecf9b7659800f3790407ebc2 * Working in docker * Increase the size of the center column when printing * Remove some page breaks * Restore block statement * Spacing fix for empty page problem * Change gunicorn reload engine to poll This is because when running through x86 emulation (on M1 Mac), the inotify file system events are blocked. But reloading works with polling! * Pin SQLAlchemy version to 2.5.1 because newer versions don't work Also, this fixes the entrypoint for the dev build stage. * Make clarifications in README and fix more spelling * Add back in deleted report.html * Don't add styles to the database if they already exist * Upgrade to python 3.9 * Add report_data object to render templates * Reset subtree * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 69cb5e2 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 69cb5e235bd62abd34c779a9701ae77b77b319a7 * Fix region_name to be run_names * Update sizing on graphs * Create environmental variable for crispresso_version and load colors if version is above 2.2.12 * Make env variable use ARG * Add version check * Changes to work with Docker compose * Fix style selection database path * Add pe_ref_seq to WGS input and update to data-bs-toggle * Remove extra div that affected the height of prime edited ref seq * Fix gene annotation file label cut off on WGS * Move style function to after folder_id is declared * Docker compose pin versions and search C2 args for --config_files * Remove gtag code * Update CRISPResso to 2.2.14 * Specify platform in docker compose file * Move jinja_partials install from dev to build stage * Fix running when no default style is selected (or when selected style isn't available) * Add zip and unzip to prod step * Clean execution logs after run finishes This will clean the full paths from the execution logs so that those aren't exposed to the user. This also ensures that this only happens in one place in the code (instead of multiple). * Update path to favicon * Reorder conda channels, remove flask-sqlalchemy pin, and fix wtforms The latest version of wtforms has moved ColorInput from `wtforms.widgets.html5` to `wtforms.widgets`, which is reflected in this commit. * Fix S3 file upload by adding form field * Remove pyopenssl pin * Add create_styles and createUsers to app context in init_db.py * Fixed padding in S3 file upload * Removed commented out S3 upload code for CRISPRessoPooled * Add volume mount to Apache in dev version This allows any files that are edited in the static folder to hot reload. * Don't show root folder of S3 buckets because it leads to weird behavior * Fix old Bootstrap margin and padding utilities The new version of Bootstrap has replaced `ml-1` with `ms-1` and `mr-1` with `me-1`, etc. Instead of being "left" and "right", it is now "start" and "end" to account for right to left languages. * Fix the close button on the S3 modal * Fix positive quantification window in radio button label * Add another app context to init_db.py * Update IDs for jQuery examples and how radio buttons are selected Because of upgrading to Bootstrap 5, the way that labels and inputs needed to be formatted, the previous way of selecting a radio button input no longer worked. Now to select a radio button element programmatically, you can issue a `.click()` and it will be selected. * Remove extra report file * Replace deprecated padding utility classes in report * Remove duplicate id's and add a few aria labels * Add correct MIME type to submission.js file * Disable caching on submission pages to improve back button behavior * Add + to quantification window for pooled and WGS * Add S3 buckets to WGS and Compare * Remove S3 buckets from WGS Not going to implement this now, because it would be a significant effort to do it correctly. * Add note to S3 modal about large files being expensive * Don't show style selector when `--config_file` parameter isn't available * Install CRISPRessoPro in the dev environment * Fix error with app contexts and databases in unit tests * Implement handling duplicate style names when saving to db * Move Plotly JS import to reports templates and out of layout.html * Remove font installer, less, and vim dependencies --------- Co-authored-by: Cole Lyman Co-authored-by: Cole Lyman * added metadata to report partial * fixed logout button condition in banner * Fix loading favicon.ico, remove duplicate log_params, and fix README typos * Fix bug where `metadata` key is not found in `report_data` --------- Co-authored-by: Samuel Nichols Co-authored-by: Cole Lyman Co-authored-by: McKay commit 5083058ccaaf41426c389bd8e20f7ca6164429ec Author: Cole Lyman Date: Fri Sep 29 14:55:41 2023 -0600 Finishing touches on README commit 7da597a224a355d6cb4e16b2162f2dd7f503030a Author: Cole Lyman Date: Fri Sep 29 14:45:30 2023 -0600 Update README with tip about merge conflicts and remove `-s subtree` commit b4db4966247910f978c959015967cbb708fa81d8 Merge: 31c3bbe 8fafd56 Author: Cole Lyman Date: Fri Sep 29 14:36:36 2023 -0600 Merge pull request #15 from edilytics/cole/reports-readme-updates-reports Update README commit 8fafd5628fbab893388eec8f8476c61d793d5481 Merge: 4b6a5c3 31c3bbe Author: Cole Lyman Date: Fri Sep 29 14:35:58 2023 -0600 Merge branch 'master' into cole/reports-readme-updates-reports commit 4b6a5c32588686957babe0a555a90e99ac7b310f Author: Cole Lyman Date: Fri Sep 29 14:26:14 2023 -0600 Add sources and helpful links commit ff0a04f7c1a7cd9980a4b14d145499d13bfc4199 Author: Cole Lyman Date: Fri Sep 29 13:44:05 2023 -0600 Update README.md with instructions on how to develop on branches commit 31c3bbe4e804f8e4e2f9303de20290110bef221c Author: Cole Lyman Date: Fri Sep 29 13:31:38 2023 -0600 Add in the shortcut for `git merge` commit 3256a60b20b7206f84262373335f85a4c1e2e340 Author: Cole Lyman Date: Mon Sep 25 13:29:34 2023 -0600 Update the README.md with updates steps on pushing commit b1b2fd1eb9027b7d3852cd4c035dfb9fd55971b0 Author: Cole Lyman Date: Mon Sep 25 10:42:37 2023 -0600 Update README.md commit 5c714b96951b73ef0f881ddbce75e6e9d9d8a585 Author: Cole Lyman Date: Mon Sep 25 13:18:49 2023 -0600 Fix missing remote parameter in README.md commit 32325fd8b60aeae83dfa92a08a5365839879f6e5 Author: Cole Lyman Date: Fri Sep 22 16:00:30 2023 -0600 Further clarificatoin in the README.md commit 758a92c2ac3082279c98b3fc47b4ebfc06a96015 Author: Cole Lyman Date: Fri Sep 22 15:53:57 2023 -0600 Add instructins to README about how to bring in commits from subtree commit 0e54f0416923c3daf54b7331ec1abbcf0070ede1 Author: Cole Lyman Date: Fri Sep 22 15:16:03 2023 -0600 Add clarification to frequency of first few steps commit 9059230b9a6b0153c54cdc5516f9dfd33ee864a6 Author: Cole Lyman Date: Fri Sep 22 15:04:43 2023 -0600 Update README.md with new instructions on how to work with this repo commit 4f996f1523f52696bade121e96c2307d5ac64fab Author: Cole Lyman Date: Fri Sep 22 14:10:16 2023 -0600 Add README.md, __init__.py, and .github files/directory After the history rewrite these files were removed (because they were present in other repos) commit 3058e7d634ba559b80699bfdcb57c3158d5bd305 Author: Cole Lyman Date: Wed Sep 20 14:47:23 2023 -0600 Add proper link to CRISPResso website when on CLI commit 69c3e736c9dd34f1e330debe9bfe5cdbe0789a83 Merge: c5a678c 47a2625 Author: Samuel Nichols Date: Wed Sep 20 10:47:05 2023 -0600 Merge commit '47a2625280243f3da4a890b2c9f9d56ab404b0ca' into obscure_variables commit 47a2625280243f3da4a890b2c9f9d56ab404b0ca Author: Samuel Nichols Date: Wed Sep 20 09:31:17 2023 -0600 Guardrails fix (#13) commit 2cf6aedecf5d52b05f86a191577a07a5eae1af91 Author: Samuel Nichols Date: Mon Sep 18 15:09:11 2023 -0600 Origin/master (#12) * Remove gtag code * Update path to favicon * Replace deprecated padding utility classes in report * Move Plotly JS import to reports templates and out of layout.html * Update reports to use is_default_user to obscure variables --------- Co-authored-by: Cole Lyman commit 4d7ff013ab19913bf63e1da5d6ebe5ce4a998af2 Author: Samuel Nichols Date: Wed Aug 16 14:29:43 2023 -0600 Guardrails (#11) * Display guardrails * Check that guardrails dict exists * Add guardrails to report_data if it exists commit 6829bcb104bdf022ee5db273d8d094a4524a9d63 Author: Samuel Nichols Date: Tue Aug 15 14:56:50 2023 -0600 Guardrails (#10) * Display guardrails * Check that guardrails dict exists commit c5a678c5277448bff5dffb4b1cb4e17d601a7d51 Author: Samuel Nichols Date: Mon Aug 14 14:35:27 2023 -0600 Reports, add reports to packages, colors, ordered pandas sort (#28) * Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 461ca93..f41627e * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from f41627e..c9a09ec git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: c9a09ecd3daf015009a39cce80e5183cdc0eaf1c * Add CRISPRessoReports to packages * Colors only with pro commit c9a09ecd3daf015009a39cce80e5183cdc0eaf1c Merge: 69cb5e2 4adf34f Author: Samuel Nichols Date: Mon Aug 7 13:08:51 2023 -0600 Merge pull request #9 from edilytics/origin/master Origin/master commit 4adf34f59cd902ecf3ad5cbd1d7703c5401ca5b3 Author: Samuel Nichols Date: Mon Aug 7 13:05:35 2023 -0600 Update sizing on graphs commit 3f31714d83aba60f19d3d33c6add8fed174d8e2e Merge: 6d31590 69cb5e2 Author: Samuel Nichols Date: Thu Aug 3 13:14:43 2023 -0600 Merge commit 'b8c11d51d65ab8e0cbe0a37cfd00389aa84edbf8' as 'CRISPRessoWEB/CRISPRessoReports' commit 69cb5e235bd62abd34c779a9701ae77b77b319a7 Merge: ab74bc6 d66ed48 Author: Samuel Nichols Date: Thu Aug 3 13:09:21 2023 -0600 Merge pull request #8 from edilytics/web_report_refactor Cup script and if htmls statement commit d66ed48bfa140112abb111ad734145b3e96c6ec7 Author: Samuel Nichols Date: Thu Aug 3 13:01:28 2023 -0600 Cup script and if htmls statement commit ab74bc62ba6213b1537b540ef3311fba6e24980b Merge: 4c1c03b f41627e Author: Samuel Nichols Date: Thu Jul 27 13:07:46 2023 -0600 Merge pull request #7 from edilytics/fig_name_fix Fix 10f and 10g not showing up error, bootstrap 5 changes, and githubActions. commit f41627e83e2c162663c5b9c826ab1f706d5e837f Merge: 294c8ec 8dc60cd Author: Samuel Nichols Date: Thu Jul 27 12:25:33 2023 -0600 Merge remote-tracking branch 'origin/githubActions' into fig_name_fix commit 294c8ec82ee21c3be86b1c83bdce33c5bf798e5c Merge: bbb75d0 74f1450 Author: Samuel Nichols Date: Thu Jul 27 12:24:50 2023 -0600 Merge remote-tracking branch 'origin/Reports_refactor' into fig_name_fix commit bbb75d05f77015611aac9fe8e0501c2d4ebb280a Author: Samuel Nichols Date: Wed Jul 26 13:24:16 2023 -0600 Fix 10f and 10g not showing up error commit 8dc60cd2d96372ea278eaed6992fd63614356300 Author: Samuel Nichols Date: Tue Mar 21 14:14:40 2023 -0600 Pylint fixes: unused variables commit 4efa1d5243b1bc8f0165c4d8718ebd71466bd590 Author: Samuel Nichols Date: Tue Mar 21 13:07:35 2023 -0600 Dangerous defaults fix commit 749d244d3c3ffb5799fd175e12395e46eaada927 Author: Samuel Nichols Date: Tue Mar 21 12:31:48 2023 -0600 Pylint fixes commit 4c1c03b2688344f327cd3bd0df8a8c4a20b56317 Merge: 461ca93 a2a1b41 Author: Samuel Nichols Date: Mon Mar 6 10:00:39 2023 -0700 Merge pull request #6 from edilytics/print_styles Print styles commit a2a1b4149d93ec303beacb2dafd4a6782251d5f5 Author: Samuel Nichols Date: Mon Feb 27 14:24:00 2023 -0700 Remove borders when printing commit 78f3680bea3da1ee0cacd4e116530ebba76d63b1 Author: Samuel Nichols Date: Fri Feb 24 14:11:00 2023 -0700 Fix div issue, breakinpage at all points commit 7922efa4f2c5c46a1688d675aed420db4f8d9d85 Author: Samuel Nichols Date: Fri Feb 24 11:16:15 2023 -0700 Debugging for error commit 5d7576b8dca7ede0b6ad4469e3a3e57b985c62ce Author: Samuel Nichols Date: Fri Feb 17 13:15:44 2023 -0700 Spacing fix for empty page problem commit 785816bb266b6e6a13ac62a3d46ac442b7c032f4 Author: Samuel Nichols Date: Thu Feb 16 10:32:20 2023 -0700 Restore block statement commit fbdece1396f21843b07378c085232ae8b6d795fd Author: Samuel Nichols Date: Tue Feb 14 14:52:40 2023 -0700 Remove some page breaks commit 03151eaadd30f24fe921b573d550b02ba04597ed Author: Samuel Nichols Date: Tue Feb 14 13:31:24 2023 -0700 Increase the size of the center column when printing commit 4224800c16a9803de0a3bd93bb8bd776386c8f47 Author: Samuel Nichols Date: Tue Feb 14 12:58:05 2023 -0700 Working in docker commit 6d31590f66631f65658e72bf86e340b49c341f37 Merge: 85038b8 938f86d Author: Samuel Nichols Date: Tue Feb 14 10:52:52 2023 -0700 Switch reports branch commit 8d4ce1ffb445c1c116c6641f9a5aeb77e8ce2d50 Merge: e80a139 13f7ae2 Author: Samuel Nichols Date: Tue Feb 14 10:52:52 2023 -0700 Switch reports branch commit 938f86dfff7254fe53c9189c35460079931239e1 Author: Samuel Nichols Date: Tue Feb 14 10:51:45 2023 -0700 Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 1efad70..13f7ae2 13f7ae2 Fix command used and parameters elements. Increase print width and height to 100% dcf7391 Adding styling for print-only and screen only REVERT: 1efad70 Replace tabs with spaces and reindent template files REVERT: e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle REVERT: df896b0 Resize images and fix filepath REVERT: 2f70855 Add spacing around body and footer tags REVERT: 5cd6d27 Final style fixes, color circles for style files REVERT: f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' REVERT: 321815d Removing CRISPRessoReport files REVERT: 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes REVERT: 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 REVERT: 04558fb Remove extra files REVERT: 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix REVERT: 980fdc4 Styling and bootstrap changes REVERT: 9d40474 Centering issue and submit button fix REVERT: aa5071c Subtree working REVERT: db30843 Jinja choice loader REVERT: 3b67ac0 Path correction REVERT: 3aaca48 Bootstrap 5 and partials changes REVERT: 8f5d8a1 Layout.html for C2WEB and CLI REVERT: 290d829 Fix error when rendering multi reports REVERT: 546397a summaries partials and html updates REVERT: 858a751 fig_reports and replacement REVERT: 073f1fe Added a few changes from the selenium-tests branch on C2Web REVERT: 1061ebb Update indentation in report.html and extract log params into partial REVERT: c3781e9 Update path to template directory to include `CRISPRessoReports` REVERT: 84e0969 Use the `render_template` function for each report REVERT: ee721b3 Add function to render template partials without using Flask REVERT: 08fcd4e Web updates refactoring done REVERT: 99c8e22 Adding files REVERT: ef333f0 Removing reports found in subtree REVERT: 1bae0df Commit before adding subtree REVERT: 1fbb427 Add server file to render js REVERT: d1d6fdf Move styling to main.css file REVERT: 1241569 Jinja partials for all submissions REVERT: 0534637 New submission.js template file REVERT: c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial REVERT: ecd03f6 Working file upload in partial. REVERT: ce5d20f Working, missing custom label REVERT: 6ba73e7 Bootstrap 5 changes REVERT: e05d146 Layout and report update REVERT: 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion REVERT: ea44128 Move where the style files are stored in Docker REVERT: 7f03e98 Implement creating styles from the admin panel REVERT: 9b27a2e Rename style_file references to style REVERT: a233d10 Add some default styles and rename the default to "Original" REVERT: 43a8d29 Remove style file card from admin index page REVERT: 1a8f332 Refactor saving style files when there is no name specified REVERT: 64a7b1c Implement color pickers in style admin view REVERT: 17c93c1 Succesfully implemented selecting default style REVERT: fd79cdd Restyle the colors in the admin view REVERT: 3cd94b8 Fix error when the default style can't be read from the database REVERT: 5e626bd Refactor `style_handler` to read the style from the database REVERT: 0f66d4a Refactor styles to be part of the database instead of files REVERT: 6c7d3c8 Move style folder inside of server folder REVERT: 9f71f21 Add margins around style file elements REVERT: 2a28549 Restyle the color pickers REVERT: 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files REVERT: dc4f2c7 Style dropdown - allow save json only for admin REVERT: 15e7483 Style file check REVERT: 7bd0e91 Remove style from Compare REVERT: 0ab45f5 Colors function refactored and working for all types REVERT: 2e24f8b Adding styling REVERT: d6621f1 Debuging REVERT: ed00c82 Merge with master REVERT: 5150f9b Adding style_files to partial REVERT: 957a9ca Add style files to pooled and wgs REVERT: 66dc2d3 Changes to pooled and wgs, reset Dockerfile REVERT: fa6b1cf Updated Docker file and style_files.html REVERT: ee0fcfc Optional save file REVERT: 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder REVERT: 0f26e2c Working style FileAdmin, access button, and further partial refactoring REVERT: b3b70bd Rough framework for style admin page REVERT: e4731d7 Style menu completed REVERT: 1bb37bc New style menu with tabs REVERT: 58f7e56 Tabs for different style options REVERT: 3de893d Compare (#34) REVERT: e66bef1 Update AWS EB instructions.docx REVERT: 658a218 Fix bug when trying to send recovery password with bad email creds REVERT: ee32e36 Adding color-picker partial to wgs and pooled REVERT: 34ea688 Fix for responsiveness on cup and title REVERT: f0c4d07 Adding color routes to other versions REVERT: 110fe14 Color picker input added to cmd_to_run REVERT: e732478 Names for color fields REVERT: 2934631 Jinja partial for color picker and pip install in dockerfile REVERT: 48bbf9c Cup animation (#33) REVERT: 2905248 Selenium tests (#31) REVERT: 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides REVERT: 570e42a Don't remove commas from amplicons or guides REVERT: 0d70425 Add smallGenome.fa REVERT: fc33197 Writing text for pooled REVERT: dccfcb3 Files for testing REVERT: 4cea67c Changes for WGS selenium tests. All tests functional. REVERT: ff05713 Changes for WGS selenium test file loading REVERT: 495a98d Changes for pooled testing REVERT: 0ad86a5 Merge pull request #30 from edilytics/pooled-upload-fix REVERT: 127eb8f PopulatePooled error REVERT: 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests REVERT: 7847687 Add link to CRISPRessoWGS from profile page and change header REVERT: 666f73b Remove example block from CRISPRessoWGS submission page REVERT: 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled REVERT: 8d979a4 Fix bug where files_to_delete was being replaced and standardize append REVERT: 09e55fc Changes to make interleaved and pooled tests possible REVERT: f89eca8 Changes necessary for selenium tests REVERT: 3efe4f9 Clean up test files REVERT: a696363 Merge pull request #28 from edilytics/s3 REVERT: dcef708 Remove changes for CRISPRessoCompare REVERT: e0c79cf Add demo config file for eb REVERT: 03aba8e Update AWS EB instructions.docx REVERT: a671c4e Set version to 2.6.3 REVERT: 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled REVERT: da5b15b Timezone for history is displayed in user local timezone REVERT: e11691f Update history to show time of previous run REVERT: be675fb Update pooled with s3 REVERT: 4c7d429 Add data links to pooled report REVERT: 353e88f Update admin portal landing page REVERT: 712e828 Show run type in history REVERT: 2802252 s3 and user updates REVERT: efc3ed8 S3 error catching REVERT: af68341 New S3 Validation REVERT: f7d64e0 AWS validation before submission REVERT: 8446093 Update s3 for batch and paired modes REVERT: 0e7d327 S3_Upload function imrpvoed -JF REVERT: b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 REVERT: c991d52 added s3 user database model REVERT: ab4aa54 add model for s3 bucket REVERT: 853cda9 S3_Functionality improved -JF REVERT: 2f060a6 Implemented front-end s3 browsing REVERT: e082a5f stub out viewing method REVERT: c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length REVERT: c85a93f Merge pull request #15 from edilytics/wgs-interface REVERT: 712270a Add support for CRISPRessoWGS REVERT: deaacee Extract out function to get server files in submit_routes REVERT: 151eb15 Update crispresso2_info object fields REVERT: b2a974d Bump CRISPResso verion to 2.2.4 REVERT: 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 REVERT: 7f2dc1c Stop trimming json error messages, fix #11 REVERT: d28c03b Update reporting logic to use the new CRISPResso2_info schema REVERT: 03ee46f Bump CRISPResso version in Dockerfile and download release from Github REVERT: 9151c5d Add CRISPRessoPooled report template REVERT: 25a6e37 Merge pull request #6 from edilytics/pooled-interface REVERT: b47d288 Check length of amplicons for hosted version, closes #4 REVERT: 54c28b6 Update submission file extension check REVERT: 8fcadee Add a link to CRISPRessoPooled interface in user dashboard REVERT: 7fd0283 Implement CRISPRessoPooled backend and report functionality REVERT: 4063eb3 Modify submission.js to accept .txt and .tsv files REVERT: b770323 Create template file for CRISPRessoPooled submission interface REVERT: d4f2ed0 Merge pull request #5 from edilytics/flask-modularization REVERT: 8527384 Convert some celery configurations settings to new format REVERT: 962a209 Install less and vim in Dockerfile REVERT: c693668 Read CRISPResso2_info from json files instead of pickle files REVERT: a469e08 Move LoginManager to user_routes.py REVERT: f62e67a Create db tables in init_db.py REVERT: 0d85c90 Move login_required to user_routes REVERT: 6f5e33e Reformatting of remaining __init__.py REVERT: e615c0b Extract report routes out of __init__.py REVERT: 20f2601 Extract user routes out from __init__.py REVERT: 5582612 Extract status routes out from __init__.py REVERT: 2406a10 Extract submit routes out from __init__.py REVERT: b562fcd Extract celery tasks from __init__.py REVERT: faa785d Extract views out from __init__.py REVERT: ff44576 Extract model classes out from __init__.py REVERT: 914498f Merge pull request #3 from edilytics/2to3 REVERT: 86ea7da Replace RabbitMQ with Redis REVERT: adca9fb Upgrade celery to version 5.0.5 REVERT: 244ec33 Convert from Python 2 to Python 3 REVERT: 28b4f37 Refactor Docker image to use Python 3 via micromamba REVERT: 2359800 Allow interleaved batches REVERT: 428720b Add features: Allow admin init, server discovery depth REVERT: 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon REVERT: 5062365 Update README.md REVERT: 51e02f4 Update README.md REVERT: ac4a6d5 delete other images REVERT: 4f3ad88 Update README.md REVERT: fc0de1d Update README.md REVERT: 08defa1 Update README.md REVERT: 9604983 Trycatch pickle loads REVERT: c1facd7 get rid of debug print of email REVERT: d699d4d crispresso2.0.45 REVERT: e7ff079 Update param descriptions REVERT: 1f12d59 2.0.44 REVERT: b81febe crispresso to 2.0.42 REVERT: 1a967a8 update report REVERT: 178c56d 2.4 REVERT: e41076d Job expiration REVERT: 41d1a4c check progress on setinterval REVERT: 756e488 server-side files REVERT: ad19c3c Update to crispresso 2.0.40 prime editing REVERT: e3a194a update errors and ignore email config REVERT: 2efb0bb Update README.md REVERT: 58844a6 initial commit REVERT: 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 13f7ae215990b155ecf9b7659800f3790407ebc2 commit 13f7ae215990b155ecf9b7659800f3790407ebc2 Author: Samuel Nichols Date: Mon Feb 13 09:38:21 2023 -0700 Fix command used and parameters elements. Increase print width and height to 100% commit dcf739175ff8db098d63b9b000afd0bdb59e6dff Author: Samuel Nichols Date: Mon Feb 6 16:24:45 2023 -0700 Adding styling for print-only and screen only commit 74f1450207fcc45d074ad59683654fa100bde31a Author: Samuel Nichols Date: Tue Oct 4 10:48:36 2022 -0600 Load favicon from web server commit e80a13932ef1651db312ee4d0cf31290dbc8df6f Author: Samuel Nichols Date: Tue Oct 4 10:36:52 2022 -0600 Indentation and parenthesis commit 85038b83b2a156d9217fe71cc6d1d102bcb5e2bc Merge: cba4399 15c9fdf Author: Samuel Nichols Date: Tue Oct 4 10:32:20 2022 -0600 Merge commit '15c9fdf7d12d515e45d8bfaddfa3aebd0123c293' into Reports_refactor commit 15c9fdf7d12d515e45d8bfaddfa3aebd0123c293 Author: Samuel Nichols Date: Tue Oct 4 10:32:20 2022 -0600 Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 461ca93..1efad70 1efad70 Replace tabs with spaces and reindent template files e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle df896b0 Resize images and fix filepath 2f70855 Add spacing around body and footer tags 5cd6d27 Final style fixes, color circles for style files f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' 321815d Removing CRISPRessoReport files 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 04558fb Remove extra files 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix 980fdc4 Styling and bootstrap changes 9d40474 Centering issue and submit button fix aa5071c Subtree working db30843 Jinja choice loader 3b67ac0 Path correction 3aaca48 Bootstrap 5 and partials changes 8f5d8a1 Layout.html for C2WEB and CLI 290d829 Fix error when rendering multi reports 546397a summaries partials and html updates 858a751 fig_reports and replacement 073f1fe Added a few changes from the selenium-tests branch on C2Web 1061ebb Update indentation in report.html and extract log params into partial c3781e9 Update path to template directory to include `CRISPRessoReports` 84e0969 Use the `render_template` function for each report ee721b3 Add function to render template partials without using Flask 08fcd4e Web updates refactoring done 99c8e22 Adding files ef333f0 Removing reports found in subtree 1bae0df Commit before adding subtree 1fbb427 Add server file to render js d1d6fdf Move styling to main.css file 1241569 Jinja partials for all submissions 0534637 New submission.js template file c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial ecd03f6 Working file upload in partial. ce5d20f Working, missing custom label 6ba73e7 Bootstrap 5 changes e05d146 Layout and report update 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion ea44128 Move where the style files are stored in Docker 7f03e98 Implement creating styles from the admin panel 9b27a2e Rename style_file references to style a233d10 Add some default styles and rename the default to "Original" 43a8d29 Remove style file card from admin index page 1a8f332 Refactor saving style files when there is no name specified 64a7b1c Implement color pickers in style admin view 17c93c1 Succesfully implemented selecting default style fd79cdd Restyle the colors in the admin view 3cd94b8 Fix error when the default style can't be read from the database 5e626bd Refactor `style_handler` to read the style from the database 0f66d4a Refactor styles to be part of the database instead of files 6c7d3c8 Move style folder inside of server folder 9f71f21 Add margins around style file elements 2a28549 Restyle the color pickers 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files dc4f2c7 Style dropdown - allow save json only for admin 15e7483 Style file check 7bd0e91 Remove style from Compare 0ab45f5 Colors function refactored and working for all types 2e24f8b Adding styling d6621f1 Debuging ed00c82 Merge with master 5150f9b Adding style_files to partial 957a9ca Add style files to pooled and wgs 66dc2d3 Changes to pooled and wgs, reset Dockerfile fa6b1cf Updated Docker file and style_files.html ee0fcfc Optional save file 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder 0f26e2c Working style FileAdmin, access button, and further partial refactoring b3b70bd Rough framework for style admin page e4731d7 Style menu completed 1bb37bc New style menu with tabs 58f7e56 Tabs for different style options 3de893d Compare (#34) e66bef1 Update AWS EB instructions.docx 658a218 Fix bug when trying to send recovery password with bad email creds ee32e36 Adding color-picker partial to wgs and pooled 34ea688 Fix for responsiveness on cup and title f0c4d07 Adding color routes to other versions 110fe14 Color picker input added to cmd_to_run e732478 Names for color fields 2934631 Jinja partial for color picker and pip install in dockerfile 48bbf9c Cup animation (#33) 2905248 Selenium tests (#31) 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides 570e42a Don't remove commas from amplicons or guides 0d70425 Add smallGenome.fa fc33197 Writing text for pooled dccfcb3 Files for testing 4cea67c Changes for WGS selenium tests. All tests functional. ff05713 Changes for WGS selenium test file loading 495a98d Changes for pooled testing 0ad86a5 Merge pull request #30 from edilytics/pooled-upload-fix 127eb8f PopulatePooled error 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests 7847687 Add link to CRISPRessoWGS from profile page and change header 666f73b Remove example block from CRISPRessoWGS submission page 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled 8d979a4 Fix bug where files_to_delete was being replaced and standardize append 09e55fc Changes to make interleaved and pooled tests possible f89eca8 Changes necessary for selenium tests 3efe4f9 Clean up test files a696363 Merge pull request #28 from edilytics/s3 dcef708 Remove changes for CRISPRessoCompare e0c79cf Add demo config file for eb 03aba8e Update AWS EB instructions.docx a671c4e Set version to 2.6.3 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled da5b15b Timezone for history is displayed in user local timezone e11691f Update history to show time of previous run be675fb Update pooled with s3 4c7d429 Add data links to pooled report 353e88f Update admin portal landing page 712e828 Show run type in history 2802252 s3 and user updates efc3ed8 S3 error catching af68341 New S3 Validation f7d64e0 AWS validation before submission 8446093 Update s3 for batch and paired modes 0e7d327 S3_Upload function imrpvoed -JF b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 c991d52 added s3 user database model ab4aa54 add model for s3 bucket 853cda9 S3_Functionality improved -JF 2f060a6 Implemented front-end s3 browsing e082a5f stub out viewing method c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length c85a93f Merge pull request #15 from edilytics/wgs-interface 712270a Add support for CRISPRessoWGS deaacee Extract out function to get server files in submit_routes 151eb15 Update crispresso2_info object fields b2a974d Bump CRISPResso verion to 2.2.4 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 7f2dc1c Stop trimming json error messages, fix #11 d28c03b Update reporting logic to use the new CRISPResso2_info schema 03ee46f Bump CRISPResso version in Dockerfile and download release from Github 9151c5d Add CRISPRessoPooled report template 25a6e37 Merge pull request #6 from edilytics/pooled-interface b47d288 Check length of amplicons for hosted version, closes #4 54c28b6 Update submission file extension check 8fcadee Add a link to CRISPRessoPooled interface in user dashboard 7fd0283 Implement CRISPRessoPooled backend and report functionality 4063eb3 Modify submission.js to accept .txt and .tsv files b770323 Create template file for CRISPRessoPooled submission interface d4f2ed0 Merge pull request #5 from edilytics/flask-modularization 8527384 Convert some celery configurations settings to new format 962a209 Install less and vim in Dockerfile c693668 Read CRISPResso2_info from json files instead of pickle files a469e08 Move LoginManager to user_routes.py f62e67a Create db tables in init_db.py 0d85c90 Move login_required to user_routes 6f5e33e Reformatting of remaining __init__.py e615c0b Extract report routes out of __init__.py 20f2601 Extract user routes out from __init__.py 5582612 Extract status routes out from __init__.py 2406a10 Extract submit routes out from __init__.py b562fcd Extract celery tasks from __init__.py faa785d Extract views out from __init__.py ff44576 Extract model classes out from __init__.py 914498f Merge pull request #3 from edilytics/2to3 86ea7da Replace RabbitMQ with Redis adca9fb Upgrade celery to version 5.0.5 244ec33 Convert from Python 2 to Python 3 28b4f37 Refactor Docker image to use Python 3 via micromamba 2359800 Allow interleaved batches 428720b Add features: Allow admin init, server discovery depth 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon 5062365 Update README.md 51e02f4 Update README.md ac4a6d5 delete other images 4f3ad88 Update README.md fc0de1d Update README.md 08defa1 Update README.md 9604983 Trycatch pickle loads c1facd7 get rid of debug print of email d699d4d crispresso2.0.45 e7ff079 Update param descriptions 1f12d59 2.0.44 b81febe crispresso to 2.0.42 1a967a8 update report 178c56d 2.4 e41076d Job expiration 41d1a4c check progress on setinterval 756e488 server-side files ad19c3c Update to crispresso 2.0.40 prime editing e3a194a update errors and ignore email config 2efb0bb Update README.md 58844a6 initial commit 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 1efad706b7c147dd45601111d697534fa993c007 commit 1efad706b7c147dd45601111d697534fa993c007 Author: Cole Lyman Date: Mon Oct 3 15:21:07 2022 -0600 Replace tabs with spaces and reindent template files commit e7ef285dbc92b309b674c8c3f54b88cb8e07a2a0 Author: Samuel Nichols Date: Wed Sep 28 11:31:42 2022 -0600 Fix hamburger menu and add -bs- to data-target and data-toggle commit df896b0f4630e5fd282fb3ac414c47039ce0db34 Author: Samuel Nichols Date: Wed Sep 28 10:41:37 2022 -0600 Resize images and fix filepath commit 2f70855b57de16584ee0fe85f4f30c1a0d13cf97 Author: Cole Lyman Date: Wed Sep 28 10:21:16 2022 -0600 Add spacing around body and footer tags commit 5cd6d2707e6c95a20939531f1be770be4169625b Author: Samuel Nichols Date: Fri Sep 23 11:48:32 2022 -0600 Final style fixes, color circles for style files commit cba43990501386df65d38d88e34a8d7176a9c5e6 Merge: 321815d e7de9b7 Author: Samuel Nichols Date: Tue Sep 13 11:02:06 2022 -0600 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' commit e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5 Author: Samuel Nichols Date: Tue Sep 13 11:02:06 2022 -0600 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca933c065a0f0765ce899eb537ab1ace41d43 commit f8d7d92393f1e3293990ab841c350861fc6e49d3 Merge: 321815d 461ca93 Author: Samuel Nichols Date: Tue Sep 13 11:02:06 2022 -0600 Merge commit '90392b44c4bf86da0940887f85401072f4190428' as 'CRISPRessoWEB/CRISPRessoReports' commit 321815d0c3599408afb6b1506f793275ff673f42 Author: Samuel Nichols Date: Tue Sep 13 11:01:52 2022 -0600 Removing CRISPRessoReport files commit 84174e6393ac68874698e910e1ba3d5daf098e3e Author: Samuel Nichols Date: Sat Sep 10 13:02:11 2022 -0600 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 7d9b4e5 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 7d9b4e54795c7459c917da50797c0fc5e30f8de7 commit aa5071c0b112f2a30457587408cc8f688a9390ca Author: Samuel Nichols Date: Fri Sep 9 10:34:52 2022 -0600 Subtree working commit db30843e77f9f66f96ec91b3e02058e0ddbaa111 Author: Samuel Nichols Date: Thu Sep 8 10:27:28 2022 -0600 Jinja choice loader commit 3b67ac0944c3f9c264603d4b3958d06d98d80030 Author: Samuel Nichols Date: Thu Sep 8 10:24:58 2022 -0600 Path correction commit 3aaca4875ffe6eddbcdc6e81b37eb07522c7311d Author: Samuel Nichols Date: Wed Aug 24 18:13:20 2022 -0600 Bootstrap 5 and partials changes commit 8f5d8a1136cbe16c5daf78da24655374e01b4f5d Author: Samuel Nichols Date: Mon Aug 22 10:21:10 2022 -0600 Layout.html for C2WEB and CLI commit 290d829f3b20f4186694a3895bb31200ef4a6d8f Author: Cole Lyman Date: Thu Jul 7 09:11:02 2022 -0600 Fix error when rendering multi reports commit 546397a73d59288205b1a5772cf83a8f513b7e5b Author: Samuel Nichols Date: Thu Jul 7 10:11:15 2022 -0400 summaries partials and html updates commit 858a7517d24caee7e3333594477d9aa0dad56ac4 Author: Samuel Nichols Date: Wed Jul 6 11:27:46 2022 -0400 fig_reports and replacement commit 073f1fe5c55941cc5a759c63782783bbbfb2a5a8 Author: Cole Lyman Date: Wed Jul 6 15:02:30 2022 -0600 Added a few changes from the selenium-tests branch on C2Web commit 1061ebb3de0f0c0a4773213f5f0b5aa97f28fc79 Author: Cole Lyman Date: Thu Jun 30 00:25:14 2022 -0600 Update indentation in report.html and extract log params into partial commit c3781e9c71e20f8b11bfa51d709df1f50bb95e6b Author: Cole Lyman Date: Thu Jun 30 00:05:40 2022 -0600 Update path to template directory to include `CRISPRessoReports` commit 84e0969968218bf74478a62754370937f1bc7312 Author: Cole Lyman Date: Wed Jun 29 23:42:26 2022 -0600 Use the `render_template` function for each report commit ee721b37a97155da72d5cddba01bfded58461748 Author: Cole Lyman Date: Wed Jun 29 23:19:05 2022 -0600 Add function to render template partials without using Flask commit 08fcd4eda347886e66c4d2a170eb03b00a02781a Author: Samuel Nichols Date: Tue May 17 10:08:08 2022 -0600 Web updates refactoring done commit 99c8e2243b7c37f2284189984e37a26742a21b3b Author: Samuel Nichols Date: Fri May 6 10:59:05 2022 -0600 Adding files commit 461ca933c065a0f0765ce899eb537ab1ace41d43 Author: Cole Lyman Date: Fri Sep 2 14:53:30 2022 -0600 Fix tab layout in report by removing extra closing div tags commit 9f7380b0c997c4ac634a5bf1ed6903046013f77d Author: Cole Lyman Date: Fri Sep 2 14:51:00 2022 -0600 Fix footer on web pages commit 68936690a00725a6c700a291d8091f5429de5dd6 Author: Samuel Nichols Date: Fri Sep 2 10:30:42 2022 -0600 Testing commit 06f1ee74220f6b233462ea6c3e2bca374eb633b4 Merge: 04cc7d9 c956fbd Author: Cole Lyman Date: Wed Aug 17 15:06:24 2022 -0600 Merge commit 'a92f38b66455728992f58a506edbd14bd6ef0e8b' as 'CRISPResso2/CRISPRessoReports' commit c956fbdc7964706ac90a3c228619d2b39b3511da Merge: 2f6086e a952878 Author: Cole Lyman Date: Wed Aug 17 14:51:21 2022 -0600 Merge pull request #1 from edilytics/jinja-partials Jinja partials commit a952878c37ae1e27ac5b3890c5e60633c26c6a4b Merge: e7b1888 ca3b6a1 Author: Cole Lyman Date: Wed Aug 17 14:50:42 2022 -0600 Merge pull request #2 from edilytics/selenium-tests Added a few changes from the selenium-tests branch on C2Web commit ca3b6a1ed8233e3784db4e9a4f1cc1d7dfb2d8d8 Merge: 1fd7df3 e7b1888 Author: Cole Lyman Date: Wed Aug 17 14:50:31 2022 -0600 Merge branch 'jinja-partials' into selenium-tests commit e7b1888d639b7048bc62a6363f49e993e2a11bb0 Merge: 904b05a 3673981 Author: Samuel Nichols Date: Thu Jul 7 11:55:08 2022 -0400 Merge fix, working templates for all multi reports commit 04cc7d9781325a2b53b2c8e4721db71db905e297 Merge: 95bc1f6 a511ec6 Author: Cole Lyman Date: Thu Jul 7 09:15:47 2022 -0600 Merge commit 'a511ec62614b38b8783863a593611be23367c3d6' into reports commit 367398122714d854d3fa6dd81b6ecea08c5e77ca Merge: d279edd a511ec6 Author: Cole Lyman Date: Thu Jul 7 09:15:47 2022 -0600 Merge commit 'a511ec62614b38b8783863a593611be23367c3d6' into reports commit a511ec62614b38b8783863a593611be23367c3d6 Author: Cole Lyman Date: Thu Jul 7 09:11:02 2022 -0600 Fix error when rendering multi reports commit 904b05a6fd7752e751e5817e780f2b380e13e697 Author: Samuel Nichols Date: Thu Jul 7 10:11:15 2022 -0400 summaries partials and html updates commit 1fd7df3a020dbdf08fde0d7e72ed25bc8a7abd7c Author: Cole Lyman Date: Wed Jul 6 15:02:30 2022 -0600 Added a few changes from the selenium-tests branch on C2Web commit d279eddb2168ff9edf0269080948b92c7803af7f Author: Samuel Nichols Date: Wed Jul 6 11:27:46 2022 -0400 fig_reports and replacement commit 95bc1f6dac7393959c0359ce37b0e6cd49e55805 Merge: 652ccf1 d6ca6f9 Author: Cole Lyman Date: Thu Jun 30 00:27:13 2022 -0600 Merge commit 'd6ca6f9ff3b79b9519237da0a8dcc32dd8dead73' into reports commit d6ca6f9ff3b79b9519237da0a8dcc32dd8dead73 Author: Cole Lyman Date: Thu Jun 30 00:25:14 2022 -0600 Update indentation in report.html and extract log params into partial commit 652ccf1574a62d39664f0bd6b4171f24722611e7 Merge: d6d7a54 ef18bb9 Author: Cole Lyman Date: Thu Jun 30 00:10:48 2022 -0600 Merge commit 'ef18bb9779f446d8be53aabcd966063bc186b32a' into reports commit ef18bb9779f446d8be53aabcd966063bc186b32a Author: Cole Lyman Date: Thu Jun 30 00:05:40 2022 -0600 Update path to template directory to include `CRISPRessoReports` commit d6d7a5496fd9b015ae4ac6fd8d0659ae25a82a91 Author: Cole Lyman Date: Wed Jun 29 23:43:33 2022 -0600 Add 'CRISPResso2/CRISPRessoReports/' from commit '2c25436b458d830df70aaaf19da170c5815de93f' git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-mainline: e8a796f5f451409cbafed4404dfba4b6b8a124ca git-subtree-split: 2c25436b458d830df70aaaf19da170c5815de93f commit 2c25436b458d830df70aaaf19da170c5815de93f Author: Cole Lyman Date: Wed Jun 29 23:42:26 2022 -0600 Use the `render_template` function for each report commit 293c0c937f39a19363dc567737230580b7f68fa4 Author: Cole Lyman Date: Wed Jun 29 23:19:05 2022 -0600 Add function to render template partials without using Flask commit 2f6086ef1daa56d48fd23761ea08cf3c7fa6a059 Author: Samuel Nichols Date: Tue May 17 10:08:08 2022 -0600 Web updates refactoring done commit 79d25cac42da71bb77f2bea0b565f6319830e015 Author: Samuel Nichols Date: Fri May 6 10:59:05 2022 -0600 Adding files --- CRISPResso2/CRISPRessoReports/.gitattributes | 2 + .../CRISPRessoReports/CRISPRessoReport.py | 17 +++--- CRISPResso2/CRISPRessoReports/README.md | 2 +- .../templates/batchReport.html | 54 ++----------------- .../CRISPRessoReports/templates/layout.html | 19 ++++--- .../templates/pooledReport.html | 5 +- .../CRISPRessoReports/templates/report.html | 14 +++-- .../shared/partials/fig_summaries.html | 30 +++++------ .../shared/partials/guardrail_warnings.html | 8 +-- 9 files changed, 55 insertions(+), 96 deletions(-) create mode 100644 CRISPResso2/CRISPRessoReports/.gitattributes diff --git a/CRISPResso2/CRISPRessoReports/.gitattributes b/CRISPResso2/CRISPRessoReports/.gitattributes new file mode 100644 index 00000000..d68b3d4c --- /dev/null +++ b/CRISPResso2/CRISPRessoReports/.gitattributes @@ -0,0 +1,2 @@ +*.html text eol=lf +*.md text eol=lf diff --git a/CRISPResso2/CRISPRessoReports/CRISPRessoReport.py b/CRISPResso2/CRISPRessoReports/CRISPRessoReport.py index b655b9d9..c4c17fd7 100644 --- a/CRISPResso2/CRISPRessoReports/CRISPRessoReport.py +++ b/CRISPResso2/CRISPRessoReports/CRISPRessoReport.py @@ -5,27 +5,22 @@ ''' import os -from jinja2 import Environment, FileSystemLoader, ChoiceLoader +from jinja2 import Environment, FileSystemLoader, PackageLoader, ChoiceLoader from jinja_partials import generate_render_partial, render_partial from CRISPResso2 import CRISPRessoShared if CRISPRessoShared.is_C2Pro_installed(): - import CRISPRessoPro + from CRISPRessoPro import __version__ as CRISPRessoProVersion C2PRO_INSTALLED = True else: C2PRO_INSTALLED = False -def get_jinja_loader(root): - if C2PRO_INSTALLED: - return Environment( - loader=ChoiceLoader([ - FileSystemLoader(os.path.join(root, 'CRISPRessoReports', 'templates')), - FileSystemLoader(os.path.join(os.path.dirname(CRISPRessoPro.__file__), 'templates')), - ]), - ) +def get_jinja_loader(_ROOT): + if CRISPRessoShared.is_C2Pro_installed(): + return Environment(loader=ChoiceLoader([FileSystemLoader(os.path.join(_ROOT, 'CRISPRessoReports', 'templates')), PackageLoader('CRISPRessoPro', 'templates')])) else: - return Environment(loader=FileSystemLoader(os.path.join(root, 'CRISPRessoReports', 'templates'))) + return Environment(loader=FileSystemLoader(os.path.join(_ROOT, 'CRISPRessoReports', 'templates'))) def render_template(template_name, jinja2_env, **data): diff --git a/CRISPResso2/CRISPRessoReports/README.md b/CRISPResso2/CRISPRessoReports/README.md index 39487ca6..df083f58 100644 --- a/CRISPResso2/CRISPRessoReports/README.md +++ b/CRISPResso2/CRISPRessoReports/README.md @@ -6,7 +6,7 @@ Take care when committing into these files as not to mix unrelated git histories ## How do I work with this repo? -Step 1 only needs to be done once per cloned repo, the other steps will need to be dne more frequently. +Step 1 only needs to be done once per cloned repo, the other steps will need to be done more frequently. 1. Add the remote to this repo to the "parent" repo (i.e. `CRISPResso2` or `C2Web`). **You should only have to do this once.** diff --git a/CRISPResso2/CRISPRessoReports/templates/batchReport.html b/CRISPResso2/CRISPRessoReports/templates/batchReport.html index fe4a0200..a3f6ddc9 100644 --- a/CRISPResso2/CRISPRessoReports/templates/batchReport.html +++ b/CRISPResso2/CRISPRessoReports/templates/batchReport.html @@ -1,6 +1,5 @@ {% extends "layout.html" %} {% block head %} - +{% if C2PRO_INSTALLED %} + +{% endif %} + {% endblock %} {% block content %} @@ -196,55 +199,6 @@
{{allele_modification_heatmap_plot_titles[heatmap_plot_name]}}
}); {% endfor %} - {# - - - - #} {% endif %}
diff --git a/CRISPResso2/CRISPRessoReports/templates/layout.html b/CRISPResso2/CRISPRessoReports/templates/layout.html index f7a616a4..90fc2b93 100644 --- a/CRISPResso2/CRISPRessoReports/templates/layout.html +++ b/CRISPResso2/CRISPRessoReports/templates/layout.html @@ -62,10 +62,20 @@ {% if is_web %} + + + {% else %} {% endif %} + + + + + + + {% block head %}{% endblock %} @@ -209,14 +219,7 @@

Analysis of genome editing ou {% endif %} - - - - - - - - {% block foot %}{% endblock %} + {% block foot %}{% endblock %} diff --git a/CRISPResso2/CRISPRessoReports/templates/pooledReport.html b/CRISPResso2/CRISPRessoReports/templates/pooledReport.html index ea95254f..3ceb129d 100644 --- a/CRISPResso2/CRISPRessoReports/templates/pooledReport.html +++ b/CRISPResso2/CRISPRessoReports/templates/pooledReport.html @@ -40,9 +40,11 @@ } } + {% if C2PRO_INSTALLED %} {% endif %} + {% endblock %} {% block content %} @@ -97,6 +99,3 @@

{{report_data['titles'][plot_name]}}
{% endblock %} - -{% block foot %} -{% endblock %} diff --git a/CRISPResso2/CRISPRessoReports/templates/report.html b/CRISPResso2/CRISPRessoReports/templates/report.html index a62c45ad..dd6db09f 100644 --- a/CRISPResso2/CRISPRessoReports/templates/report.html +++ b/CRISPResso2/CRISPRessoReports/templates/report.html @@ -73,9 +73,11 @@ } } + {% if C2PRO_INSTALLED %} {% endif %} + {% endblock %} {% block content %} @@ -88,7 +90,7 @@ {% if report_data['report_display_name'] != '' %}
{{report_data['report_display_name']}}
{% endif %} - {{ render_partial('shared/partials/guardrail_warnings.html', report_data=report_data) }} + {{ render_partial('shared/partials/guardrail_warnings.html', report_data=report_data) | safe}}
CRISPResso2 run information
  • @@ -179,7 +181,7 @@
    Global splicing analysis
    {# end of global coding sequence analysis #} {# start hdr summary #} - {% if report_data['figures']['locs'][report_data.amplicons[0]]['plot_4g'] or 'plot_4g' in report_data['figures']['htmls'][report_data.amplicons[0]] %} + {% if report_data['figures']['locs'][report_data.amplicons[0]]['plot_4g'] or 'plot_4g' in report_data['figures']['htmls'][amplicon_name] %}
    {% if report_data.amplicons|length == 1 %} {# if only one amplicon #} @@ -189,7 +191,11 @@
    HDR summary report (all reads aligned to {{report_data.amplicons[0]}})
    {% endif %}
    - {{ render_partial('shared/partials/fig_reports.html', report_data=report_data, fig_name='plot_4g', amplicon_name=report_data.amplicons[0])}} + {% if 'plot_4g' in report_data['figures']['htmls'][amplicon_name] %} + {{ report_data['figures']['htmls'][amplicon_name]['plot_4g']|safe }} + {% else %} + {{ render_partial('shared/partials/fig_reports.html', report_data=report_data, fig_name='plot_4g', amplicon_name=report_data.amplicons[0])}} + {% endif %}
    {% endif %} @@ -722,7 +728,7 @@
    Base editing for {{amplicon_name}}
    {% if C2PRO_INSTALLED %} - + {{ render_partial('partials/batch_d3.html', nucleotide_quilt_slugs=(report_data['nuc_quilt_names'])) }} {% endif %} diff --git a/CRISPResso2/CRISPRessoReports/templates/shared/partials/fig_summaries.html b/CRISPResso2/CRISPRessoReports/templates/shared/partials/fig_summaries.html index 63d40dea..fe208736 100644 --- a/CRISPResso2/CRISPRessoReports/templates/shared/partials/fig_summaries.html +++ b/CRISPResso2/CRISPRessoReports/templates/shared/partials/fig_summaries.html @@ -1,15 +1,15 @@ -
    - {% if report_data['htmls'] and report_data['htmls'][plot_name]%} - {{report_data['htmls'][plot_name]|safe}} - {% else %} - {% if plot_name in ['Nucleotide_conversion_map', 'Nucleotide_percentage_quilt'] %} - - {% else %} - - {% endif %} - {% endif %} - - {% for (data_label,data_path) in report_data['datas'][plot_name] %} -

    Data: {{data_label}}

    - {% endfor %} -
    +
    + {% if report_data['htmls'] and report_data['htmls'][plot_name]%} + {{report_data['htmls'][plot_name]|safe}} + {% else %} + {% if plot_name in ['Nucleotide_conversion_map', 'Nucleotide_percentage_quilt'] %} + + {% else %} + + {% endif %} + {% endif %} + + {% for (data_label,data_path) in report_data['datas'][plot_name] %} +

    Data: {{data_label}}

    + {% endfor %} +
    diff --git a/CRISPResso2/CRISPRessoReports/templates/shared/partials/guardrail_warnings.html b/CRISPResso2/CRISPRessoReports/templates/shared/partials/guardrail_warnings.html index 2e7f6cad..49b74978 100644 --- a/CRISPResso2/CRISPRessoReports/templates/shared/partials/guardrail_warnings.html +++ b/CRISPResso2/CRISPRessoReports/templates/shared/partials/guardrail_warnings.html @@ -1,5 +1,5 @@ -{% if 'guardrails_htmls' in report_data['run_data']['results'] and report_data['run_data']['results']['guardrails_htmls']|length > 0 %} - {% for message in report_data['run_data']['results']['guardrails_htmls'] %} - {{message}} - {% endfor %} +{% if 'guardrails_htmls' in report_data['run_data']['results'] and report_data['run_data']['results']['guardrails_htmls']|length > 0 %} + {% for message in report_data['run_data']['results']['guardrails_htmls'] %} + {{message | safe}} + {% endfor %} {% endif %} \ No newline at end of file