Failed batch runs #33
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 7d9b4e5..e18807d * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from e18807d..e9da7bf git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: e9da7bff794058e1fcdb3dc9ced79871c6a30e18 * Add CRISPRessoReports to packages * Colors only with pro
…port, and displaying with HTML and Javascript
Colelyman
requested changes
Jan 11, 2024
There was a problem hiding this comment.
This looks great, one issue that needs to be fixed before we merge it in, is that when I run CRISPRessoCompare it results in this error:
Traceback (most recent call last):
File "/Users/cole/code/edilytics/CRISPResso2/CRISPResso2/CRISPRessoCompareCORE.py", line 443, in main
CRISPRessoReport.make_compare_report_from_folder(report_name, crispresso2_info, OUTPUT_DIRECTORY, _ROOT)
File "/Users/cole/code/edilytics/CRISPResso2/CRISPResso2/CRISPRessoReports/CRISPRessoReport.py", line 328, in make_compare_report_from_folder
make_multi_report_from_folder(crispresso2_info, names_arr, output_title, crispresso_report_file, folder, _ROOT, 'compare')
File "/Users/cole/code/edilytics/CRISPResso2/CRISPResso2/CRISPRessoReports/CRISPRessoReport.py", line 382, in make_multi_report_from_folder
failed_runs = crispresso2_info['results']['failed_batch_arr']
KeyError: 'failed_batch_arr'
You can reproduce this error by running make compare-test from the tests repo.
* Fix showing link to report in CLI (only show in web) * Remove styling of jumbotron The p-5 added some weird space at the top of the container, the rounded-3 did not make a difference (because there is no background), and the h-100 also did not make a difference. * Remove extra spaces at end of the line * Remove color legend from figure caption in plot 4f * Refactor fig_reports.html partial to reduce duplication * Add opacity to custom colors on allele quilt plot * Remove extra spaces * Change default color of deletion It looked too similar to `N` and was difficult to tell apart. * Refactor plot 10c, refactor displaying of figures This commit adds flexbox to the plots, this was mainly for plots 10b and 10c because their alignment was off. * Add more plots to get the correct percentages for width * Remove setting the height of the plots * Check for failed batch info before retrieving it in `make_multi_report_from_folder` * Fix extraneous whitespace in `fig_reports` partial * Only load certain resources when on web mode * Move jQuery import to bottom of the page to improve performance * Extract out report footer buttons to partial * Fix too many closing divs in batchReport.html * Refactor failed runs to be a partial * Move the failed run JS to the partial This has the benefit of keeping the relevant code close, and also prevents the error that we were running into before where `chevronIcon` wasn't found when there were no failed runs (because the element wasn't there). * Remove `report_name` id because it probably has spaces * Move existing Plotly plots to batchReport from multiReport * Fix typo in fig 11c and resize it to 40%
Colelyman
approved these changes
Jan 22, 2024
Colelyman
added a commit
that referenced
this pull request
Jan 22, 2024
* Reports, add reports to packages, colors, ordered pandas sort (#28) * Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 7d9b4e5..e18807d * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from e18807d..e9da7bf git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: e9da7bff794058e1fcdb3dc9ced79871c6a30e18 * Add CRISPRessoReports to packages * Colors only with pro * changed tuple to list for matplotlib change (#31) * wgs and batch failed runs implementation * Added failed run functionality including shared function, edits to Report, and displaying with HTML and Javascript * Merge CRISPRessoReports master into failed-batch-runs * Cole's failed-batch-runs review and changes (#36) * Fix showing link to report in CLI (only show in web) * Remove styling of jumbotron The p-5 added some weird space at the top of the container, the rounded-3 did not make a difference (because there is no background), and the h-100 also did not make a difference. * Remove extra spaces at end of the line * Remove color legend from figure caption in plot 4f * Refactor fig_reports.html partial to reduce duplication * Add opacity to custom colors on allele quilt plot * Remove extra spaces * Change default color of deletion It looked too similar to `N` and was difficult to tell apart. * Refactor plot 10c, refactor displaying of figures This commit adds flexbox to the plots, this was mainly for plots 10b and 10c because their alignment was off. * Add more plots to get the correct percentages for width * Remove setting the height of the plots * Check for failed batch info before retrieving it in `make_multi_report_from_folder` * Fix extraneous whitespace in `fig_reports` partial * Only load certain resources when on web mode * Move jQuery import to bottom of the page to improve performance * Extract out report footer buttons to partial * Fix too many closing divs in batchReport.html * Refactor failed runs to be a partial * Move the failed run JS to the partial This has the benefit of keeping the relevant code close, and also prevents the error that we were running into before where `chevronIcon` wasn't found when there were no failed runs (because the element wasn't there). * Remove `report_name` id because it probably has spaces * Move existing Plotly plots to batchReport from multiReport * Fix typo in fig 11c and resize it to 40% --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]>
Colelyman
pushed a commit
that referenced
this pull request
Mar 22, 2024
…7ae2 13f7ae2 Fix command used and parameters elements. Increase print width and height to 100% dcf7391 Adding styling for print-only and screen only REVERT: 1efad70 Replace tabs with spaces and reindent template files REVERT: e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle REVERT: df896b0 Resize images and fix filepath REVERT: 2f70855 Add spacing around body and footer tags REVERT: 5cd6d27 Final style fixes, color circles for style files REVERT: f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' REVERT: 321815d Removing CRISPRessoReport files REVERT: 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes REVERT: 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 REVERT: 04558fb Remove extra files REVERT: 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix REVERT: 980fdc4 Styling and bootstrap changes REVERT: 9d40474 Centering issue and submit button fix REVERT: aa5071c Subtree working REVERT: db30843 Jinja choice loader REVERT: 3b67ac0 Path correction REVERT: 3aaca48 Bootstrap 5 and partials changes REVERT: 8f5d8a1 Layout.html for C2WEB and CLI REVERT: 290d829 Fix error when rendering multi reports REVERT: 546397a summaries partials and html updates REVERT: 858a751 fig_reports and replacement REVERT: 073f1fe Added a few changes from the selenium-tests branch on C2Web REVERT: 1061ebb Update indentation in report.html and extract log params into partial REVERT: c3781e9 Update path to template directory to include `CRISPRessoReports` REVERT: 84e0969 Use the `render_template` function for each report REVERT: ee721b3 Add function to render template partials without using Flask REVERT: 08fcd4e Web updates refactoring done REVERT: 99c8e22 Adding files REVERT: ef333f0 Removing reports found in subtree REVERT: 1bae0df Commit before adding subtree REVERT: 1fbb427 Add server file to render js REVERT: d1d6fdf Move styling to main.css file REVERT: 1241569 Jinja partials for all submissions REVERT: 0534637 New submission.js template file REVERT: c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial REVERT: ecd03f6 Working file upload in partial. REVERT: ce5d20f Working, missing custom label REVERT: 6ba73e7 Bootstrap 5 changes REVERT: e05d146 Layout and report update REVERT: 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion REVERT: ea44128 Move where the style files are stored in Docker REVERT: 7f03e98 Implement creating styles from the admin panel REVERT: 9b27a2e Rename style_file references to style REVERT: a233d10 Add some default styles and rename the default to "Original" REVERT: 43a8d29 Remove style file card from admin index page REVERT: 1a8f332 Refactor saving style files when there is no name specified REVERT: 64a7b1c Implement color pickers in style admin view REVERT: 17c93c1 Succesfully implemented selecting default style REVERT: fd79cdd Restyle the colors in the admin view REVERT: 3cd94b8 Fix error when the default style can't be read from the database REVERT: 5e626bd Refactor `style_handler` to read the style from the database REVERT: 0f66d4a Refactor styles to be part of the database instead of files REVERT: 6c7d3c8 Move style folder inside of server folder REVERT: 9f71f21 Add margins around style file elements REVERT: 2a28549 Restyle the color pickers REVERT: 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files REVERT: dc4f2c7 Style dropdown - allow save json only for admin REVERT: 15e7483 Style file check REVERT: 7bd0e91 Remove style from Compare REVERT: 0ab45f5 Colors function refactored and working for all types REVERT: 2e24f8b Adding styling REVERT: d6621f1 Debuging REVERT: ed00c82 Merge with master REVERT: 5150f9b Adding style_files to partial REVERT: 957a9ca Add style files to pooled and wgs REVERT: 66dc2d3 Changes to pooled and wgs, reset Dockerfile REVERT: fa6b1cf Updated Docker file and style_files.html REVERT: ee0fcfc Optional save file REVERT: 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder REVERT: 0f26e2c Working style FileAdmin, access button, and further partial refactoring REVERT: b3b70bd Rough framework for style admin page REVERT: e4731d7 Style menu completed REVERT: 1bb37bc New style menu with tabs REVERT: 58f7e56 Tabs for different style options REVERT: 3de893d Compare (#34) REVERT: e66bef1 Update AWS EB instructions.docx REVERT: 658a218 Fix bug when trying to send recovery password with bad email creds REVERT: ee32e36 Adding color-picker partial to wgs and pooled REVERT: 34ea688 Fix for responsiveness on cup and title REVERT: f0c4d07 Adding color routes to other versions REVERT: 110fe14 Color picker input added to cmd_to_run REVERT: e732478 Names for color fields REVERT: 2934631 Jinja partial for color picker and pip install in dockerfile REVERT: 48bbf9c Cup animation (#33) REVERT: 2905248 Selenium tests (#31) REVERT: 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides REVERT: 570e42a Don't remove commas from amplicons or guides REVERT: 0d70425 Add smallGenome.fa REVERT: fc33197 Writing text for pooled REVERT: dccfcb3 Files for testing REVERT: 4cea67c Changes for WGS selenium tests. All tests functional. REVERT: ff05713 Changes for WGS selenium test file loading REVERT: 495a98d Changes for pooled testing REVERT: 0ad86a5 Merge pull request pinellolab#30 from edilytics/pooled-upload-fix REVERT: 127eb8f PopulatePooled error REVERT: 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests REVERT: 7847687 Add link to CRISPRessoWGS from profile page and change header REVERT: 666f73b Remove example block from CRISPRessoWGS submission page REVERT: 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled REVERT: 8d979a4 Fix bug where files_to_delete was being replaced and standardize append REVERT: 09e55fc Changes to make interleaved and pooled tests possible REVERT: f89eca8 Changes necessary for selenium tests REVERT: 3efe4f9 Clean up test files REVERT: a696363 Merge pull request #28 from edilytics/s3 REVERT: dcef708 Remove changes for CRISPRessoCompare REVERT: e0c79cf Add demo config file for eb REVERT: 03aba8e Update AWS EB instructions.docx REVERT: a671c4e Set version to 2.6.3 REVERT: 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled REVERT: da5b15b Timezone for history is displayed in user local timezone REVERT: e11691f Update history to show time of previous run REVERT: be675fb Update pooled with s3 REVERT: 4c7d429 Add data links to pooled report REVERT: 353e88f Update admin portal landing page REVERT: 712e828 Show run type in history REVERT: 2802252 s3 and user updates REVERT: efc3ed8 S3 error catching REVERT: af68341 New S3 Validation REVERT: f7d64e0 AWS validation before submission REVERT: 8446093 Update s3 for batch and paired modes REVERT: 0e7d327 S3_Upload function imrpvoed -JF REVERT: b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 REVERT: c991d52 added s3 user database model REVERT: ab4aa54 add model for s3 bucket REVERT: 853cda9 S3_Functionality improved -JF REVERT: 2f060a6 Implemented front-end s3 browsing REVERT: e082a5f stub out viewing method REVERT: c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length REVERT: c85a93f Merge pull request #15 from edilytics/wgs-interface REVERT: 712270a Add support for CRISPRessoWGS REVERT: deaacee Extract out function to get server files in submit_routes REVERT: 151eb15 Update crispresso2_info object fields REVERT: b2a974d Bump CRISPResso verion to 2.2.4 REVERT: 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 REVERT: 7f2dc1c Stop trimming json error messages, fix #11 REVERT: d28c03b Update reporting logic to use the new CRISPResso2_info schema REVERT: 03ee46f Bump CRISPResso version in Dockerfile and download release from Github REVERT: 9151c5d Add CRISPRessoPooled report template REVERT: 25a6e37 Merge pull request #6 from edilytics/pooled-interface REVERT: b47d288 Check length of amplicons for hosted version, closes #4 REVERT: 54c28b6 Update submission file extension check REVERT: 8fcadee Add a link to CRISPRessoPooled interface in user dashboard REVERT: 7fd0283 Implement CRISPRessoPooled backend and report functionality REVERT: 4063eb3 Modify submission.js to accept .txt and .tsv files REVERT: b770323 Create template file for CRISPRessoPooled submission interface REVERT: d4f2ed0 Merge pull request #5 from edilytics/flask-modularization REVERT: 8527384 Convert some celery configurations settings to new format REVERT: 962a209 Install less and vim in Dockerfile REVERT: c693668 Read CRISPResso2_info from json files instead of pickle files REVERT: a469e08 Move LoginManager to user_routes.py REVERT: f62e67a Create db tables in init_db.py REVERT: 0d85c90 Move login_required to user_routes REVERT: 6f5e33e Reformatting of remaining __init__.py REVERT: e615c0b Extract report routes out of __init__.py REVERT: 20f2601 Extract user routes out from __init__.py REVERT: 5582612 Extract status routes out from __init__.py REVERT: 2406a10 Extract submit routes out from __init__.py REVERT: b562fcd Extract celery tasks from __init__.py REVERT: faa785d Extract views out from __init__.py REVERT: ff44576 Extract model classes out from __init__.py REVERT: 914498f Merge pull request #3 from edilytics/2to3 REVERT: 86ea7da Replace RabbitMQ with Redis REVERT: adca9fb Upgrade celery to version 5.0.5 REVERT: 244ec33 Convert from Python 2 to Python 3 REVERT: 28b4f37 Refactor Docker image to use Python 3 via micromamba REVERT: 2359800 Allow interleaved batches REVERT: 428720b Add features: Allow admin init, server discovery depth REVERT: 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon REVERT: 5062365 Update README.md REVERT: 51e02f4 Update README.md REVERT: ac4a6d5 delete other images REVERT: 4f3ad88 Update README.md REVERT: fc0de1d Update README.md REVERT: 08defa1 Update README.md REVERT: 9604983 Trycatch pickle loads REVERT: c1facd7 get rid of debug print of email REVERT: d699d4d crispresso2.0.45 REVERT: e7ff079 Update param descriptions REVERT: 1f12d59 2.0.44 REVERT: b81febe crispresso to 2.0.42 REVERT: 1a967a8 update report REVERT: 178c56d 2.4 REVERT: e41076d Job expiration REVERT: 41d1a4c check progress on setinterval REVERT: 756e488 server-side files REVERT: ad19c3c Update to crispresso 2.0.40 prime editing REVERT: e3a194a update errors and ignore email config REVERT: 2efb0bb Update README.md REVERT: 58844a6 initial commit REVERT: 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 13f7ae2
Colelyman
added a commit
that referenced
this pull request
Mar 22, 2024
* Reports refactor (#37) * Changes necessary for selenium tests * Changes to make interleaved and pooled tests possible * PopulatePooled error * Changes for pooled testing * Changes for WGS selenium test file loading * Changes for WGS selenium tests. All tests functional. * Files for testing * Writing text for pooled * Add smallGenome.fa * Jinja partial for color picker and pip install in dockerfile * Names for color fields * Color picker input added to cmd_to_run * Adding color routes to other versions * Fix for responsiveness on cup and title * Adding color-picker partial to wgs and pooled * Tabs for different style options * New style menu with tabs * Style menu completed * Rough framework for style admin page * Working style FileAdmin, access button, and further partial refactoring * Checkbox for custom colors that shows and hides color selectors, box on home page for style folder * Optional save file * Updated Docker file and style_files.html * Changes to pooled and wgs, reset Dockerfile * Add style files to pooled and wgs * Adding style_files to partial * Debuging * Adding styling * Colors function refactored and working for all types * Remove style from Compare * Style file check * Style dropdown - allow save json only for admin * DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files * Restyle the color pickers These changes make it so that the colors are in a row instead of a column when the screen is wide enough. They are also responsive, so when the screen is small the color pickers will return to a column. * Add margins around style file elements * Move style folder inside of server folder * Refactor styles to be part of the database instead of files This allows for greater flexibility in displaying and updating them through the web interface. * Refactor `style_handler` to read the style from the database This completes the refactor to store style parameters using the database instead of storing the styles as files. * Fix error when the default style can't be read from the database The intended and proper behavior is that no style parameter is added to the command, and that is what happens now. * Restyle the colors in the admin view Add border around the colors in the admin view as well as moving them to be more vertically centered with the name. * Succesfully implemented selecting default style * Implement color pickers in style admin view * Refactor saving style files when there is no name specified * Remove style file card from admin index page * Add some default styles and rename the default to "Original" * Rename style_file references to style This is a final clean up of refactoring how the different style properties are stored. They are now stored in the database, so it doesn't make much sense to call them style files. * Implement creating styles from the admin panel * Move where the style files are stored in Docker * Implement Docker Compose for both production and development * Replace sub, ins, del with Substitution, Insertion, Deletion * Replace WSGI with Gunicorn for both development and production This allows us to hotreload the code when running the development version and also adds the Flask Debug Toolbar Extension which will be helpful in debugging. * Add a Makefile for commonly used Docker compose commands * Add reverse proxy to make Apache redirects work properly * Layout and report update * Bootstrap 5 changes * Working, missing custom label * Working file upload in partial. * Changes to submission.js for bootstrap 5 and load file upload partial * New submission.js template file * Jinja partials for all submissions * Move styling to main.css file * Add server file to render js * Commit before adding subtree * Removing reports found in subtree * Adding files * Web updates refactoring done * Add function to render template partials without using Flask * Use the `render_template` function for each report * Update path to template directory to include `CRISPRessoReports` * Update indentation in report.html and extract log params into partial * Added a few changes from the selenium-tests branch on C2Web * fig_reports and replacement * summaries partials and html updates * Fix error when rendering multi reports * Layout.html for C2WEB and CLI * Bootstrap 5 and partials changes * Path correction * Jinja choice loader * Subtree working * Centering issue and submit button fix * Styling and bootstrap changes * Spacing changes, submission_compared fix, and submission_wgs file upload fix * Remove extra files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca93 * Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes * Removing CRISPRessoReport files * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 461ca93 * Removing unecessary logic from submission_compare * Set up standalone Apache server container in Docker Compose * Add C2Web conda environment file * Make C2Web Docker image smaller (now 2.25 GB uncompressed) * Change style to styles * Fix for updating labels * Make C2Web Dockerfile multi-stage and make CRISPResso2 hot-reloadable These changes modify the C2Web Dockerfile to be multi-stage, so that there is a base stage (shared between dev and prod), a dev stage, and a prod stage. The dev stage doesn't install CRISPResso2, but binds a local copy of CRISPResso2 so that it can be hot reloaded. In the prod stage, this installs CRISPResso2 via conda. * Clean up Dockerfile and add CRISPResso2 dependencies to C2Web Docker These dependencies (plotly, seaborn-base, and matplotlib-base) are added so that they don't need to be added when CRISPResso2 is installed. * Final style fixes, color circles for style files * Update README.md with Docker compose details and update ignore files * Install CRISPResso2 in the build stage of Dockerfile * Removed docker-compose.prod.yml and created docker-compose.public.yml Also, updated the Makefile. Now, the default docker-compose.yml should be a suitable configuration for client facing production. The docker-compose.public.yml is a good configuration for the public facing site and the docker-compose.override.yml is a good configuration for development. * Share environment variables between web and celery and update README * Add spacing around body and footer tags * Replace spacing utilities classes with Bootstrap 5 versions * Resize images and fix filepath * Hide base editing if checkbox unchecked * Base editing partial * Add padding around pegRNA radio buttons and plot window size Also, add a margin around the JSON file upload box. * Increase size of Submit buttons * Fix hamburger menu and add -bs- to data-target and data-toggle * Fix plot window size spacing in Pooled * Fix the vertical span of the input labels in WGS, Pooled and Batch * Fix Pooled layout * Make input labels in the forms the same width * Remove ALLOW_USER_STYLE_UPLOAD parameter * Remove jinja loader from report_routes * Reformat style in submit_routes.py and update docs * Convert tabs to spaces in style_selection.html * Use string interpolation instead of concatenation in submission.js * Add an authentication check before exposing server_files in submission.js * Fix indentation and convert tabs to spaces in many templates * Clean up old files and comments * Replace tabs with spaces and reindent template files * Update README.md with git alias and subtree information * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 461ca93..1efad70 1efad70 Replace tabs with spaces and reindent template files e7ef285 Fix hamburger menu and add -bs- to data-target and data-toggle df896b0 Resize images and fix filepath 2f70855 Add spacing around body and footer tags 5cd6d27 Final style fixes, color circles for style files f8d7d92 Merge commit 'e7de9b7745a71bbc9fedf2c8fc6396fcc898f2c5' as 'CRISPRessoWEB/CRISPRessoReports' 321815d Removing CRISPRessoReport files 17d9ead Radio buttons, center buttons and inputs (login, register, new password), new div name for style dropdown fixes 84174e6 Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 461ca93 04558fb Remove extra files 8e3a590 Spacing changes, submission_compared fix, and submission_wgs file upload fix 980fdc4 Styling and bootstrap changes 9d40474 Centering issue and submit button fix aa5071c Subtree working db30843 Jinja choice loader 3b67ac0 Path correction 3aaca48 Bootstrap 5 and partials changes 8f5d8a1 Layout.html for C2WEB and CLI 290d829 Fix error when rendering multi reports 546397a summaries partials and html updates 858a751 fig_reports and replacement 073f1fe Added a few changes from the selenium-tests branch on C2Web 1061ebb Update indentation in report.html and extract log params into partial c3781e9 Update path to template directory to include `CRISPRessoReports` 84e0969 Use the `render_template` function for each report ee721b3 Add function to render template partials without using Flask 08fcd4e Web updates refactoring done 99c8e22 Adding files ef333f0 Removing reports found in subtree 1bae0df Commit before adding subtree 1fbb427 Add server file to render js d1d6fdf Move styling to main.css file 1241569 Jinja partials for all submissions 0534637 New submission.js template file c5406d1 Changes to submission.js for bootstrap 5 and load file upload partial ecd03f6 Working file upload in partial. ce5d20f Working, missing custom label 6ba73e7 Bootstrap 5 changes e05d146 Layout and report update 517e9f8 Replace sub, ins, del with Substitution, Insertion, Deletion ea44128 Move where the style files are stored in Docker 7f03e98 Implement creating styles from the admin panel 9b27a2e Rename style_file references to style a233d10 Add some default styles and rename the default to "Original" 43a8d29 Remove style file card from admin index page 1a8f332 Refactor saving style files when there is no name specified 64a7b1c Implement color pickers in style admin view 17c93c1 Succesfully implemented selecting default style fd79cdd Restyle the colors in the admin view 3cd94b8 Fix error when the default style can't be read from the database 5e626bd Refactor `style_handler` to read the style from the database 0f66d4a Refactor styles to be part of the database instead of files 6c7d3c8 Move style folder inside of server folder 9f71f21 Add margins around style file elements 2a28549 Restyle the color pickers 2c82c08 DEFAULTUSER can't see style_dropdown and variable for ALLOW_USER_STYLE_UPLOAD for users to upload style files dc4f2c7 Style dropdown - allow save json only for admin 15e7483 Style file check 7bd0e91 Remove style from Compare 0ab45f5 Colors function refactored and working for all types 2e24f8b Adding styling d6621f1 Debuging ed00c82 Merge with master 5150f9b Adding style_files to partial 957a9ca Add style files to pooled and wgs 66dc2d3 Changes to pooled and wgs, reset Dockerfile fa6b1cf Updated Docker file and style_files.html ee0fcfc Optional save file 229e21d Checkbox for custom colors that shows and hides color selectors, box on home page for style folder 0f26e2c Working style FileAdmin, access button, and further partial refactoring b3b70bd Rough framework for style admin page e4731d7 Style menu completed 1bb37bc New style menu with tabs 58f7e56 Tabs for different style options 3de893d Compare (#34) e66bef1 Update AWS EB instructions.docx 658a218 Fix bug when trying to send recovery password with bad email creds ee32e36 Adding color-picker partial to wgs and pooled 34ea688 Fix for responsiveness on cup and title f0c4d07 Adding color routes to other versions 110fe14 Color picker input added to cmd_to_run e732478 Names for color fields 2934631 Jinja partial for color picker and pip install in dockerfile 48bbf9c Cup animation (#33) 2905248 Selenium tests (#31) 5641fd3 Merge pull request #32 from edilytics/multi-amplicon-guides 570e42a Don't remove commas from amplicons or guides 0d70425 Add smallGenome.fa fc33197 Writing text for pooled dccfcb3 Files for testing 4cea67c Changes for WGS selenium tests. All tests functional. ff05713 Changes for WGS selenium test file loading 495a98d Changes for pooled testing 0ad86a5 Merge pull request pinellolab#30 from edilytics/pooled-upload-fix 127eb8f PopulatePooled error 30ff7a7 Merge remote-tracking branch 'origin/pooled-upload-fix' into selenium_tests 7847687 Add link to CRISPRessoWGS from profile page and change header 666f73b Remove example block from CRISPRessoWGS submission page 27fcc13 Fix bug where amplicon file isn't being uploaded properly in CRISPRessoPooled 8d979a4 Fix bug where files_to_delete was being replaced and standardize append 09e55fc Changes to make interleaved and pooled tests possible f89eca8 Changes necessary for selenium tests 3efe4f9 Clean up test files a696363 Merge pull request #28 from edilytics/s3 dcef708 Remove changes for CRISPRessoCompare e0c79cf Add demo config file for eb 03aba8e Update AWS EB instructions.docx a671c4e Set version to 2.6.3 3bb3a8d Pull out s3 javascript for use in crispresso and crispressopooled da5b15b Timezone for history is displayed in user local timezone e11691f Update history to show time of previous run be675fb Update pooled with s3 4c7d429 Add data links to pooled report 353e88f Update admin portal landing page 712e828 Show run type in history 2802252 s3 and user updates efc3ed8 S3 error catching af68341 New S3 Validation f7d64e0 AWS validation before submission 8446093 Update s3 for batch and paired modes 0e7d327 S3_Upload function imrpvoed -JF b48e0dc Merge branch 's3' of https://github.com/edilytics/C2Web into s3 c991d52 added s3 user database model ab4aa54 add model for s3 bucket 853cda9 S3_Functionality improved -JF 2f060a6 Implemented front-end s3 browsing e082a5f stub out viewing method c5b6d13 Merge pull request #7 from edilytics/check-amplicon-length c85a93f Merge pull request #15 from edilytics/wgs-interface 712270a Add support for CRISPRessoWGS deaacee Extract out function to get server files in submit_routes 151eb15 Update crispresso2_info object fields b2a974d Bump CRISPResso verion to 2.2.4 58ae313 Merge pull request #10 from edilytics/update-to-crispresso-2.2.2 7f2dc1c Stop trimming json error messages, fix #11 d28c03b Update reporting logic to use the new CRISPResso2_info schema 03ee46f Bump CRISPResso version in Dockerfile and download release from Github 9151c5d Add CRISPRessoPooled report template 25a6e37 Merge pull request #6 from edilytics/pooled-interface b47d288 Check length of amplicons for hosted version, closes #4 54c28b6 Update submission file extension check 8fcadee Add a link to CRISPRessoPooled interface in user dashboard 7fd0283 Implement CRISPRessoPooled backend and report functionality 4063eb3 Modify submission.js to accept .txt and .tsv files b770323 Create template file for CRISPRessoPooled submission interface d4f2ed0 Merge pull request #5 from edilytics/flask-modularization 8527384 Convert some celery configurations settings to new format 962a209 Install less and vim in Dockerfile c693668 Read CRISPResso2_info from json files instead of pickle files a469e08 Move LoginManager to user_routes.py f62e67a Create db tables in init_db.py 0d85c90 Move login_required to user_routes 6f5e33e Reformatting of remaining __init__.py e615c0b Extract report routes out of __init__.py 20f2601 Extract user routes out from __init__.py 5582612 Extract status routes out from __init__.py 2406a10 Extract submit routes out from __init__.py b562fcd Extract celery tasks from __init__.py faa785d Extract views out from __init__.py ff44576 Extract model classes out from __init__.py 914498f Merge pull request #3 from edilytics/2to3 86ea7da Replace RabbitMQ with Redis adca9fb Upgrade celery to version 5.0.5 244ec33 Convert from Python 2 to Python 3 28b4f37 Refactor Docker image to use Python 3 via micromamba 2359800 Allow interleaved batches 428720b Add features: Allow admin init, server discovery depth 11df5d8 Client and server-side checks for invalid characters on sgRNA and amplicon 5062365 Update README.md 51e02f4 Update README.md ac4a6d5 delete other images 4f3ad88 Update README.md fc0de1d Update README.md 08defa1 Update README.md 9604983 Trycatch pickle loads c1facd7 get rid of debug print of email d699d4d crispresso2.0.45 e7ff079 Update param descriptions 1f12d59 2.0.44 b81febe crispresso to 2.0.42 1a967a8 update report 178c56d 2.4 e41076d Job expiration 41d1a4c check progress on setinterval 756e488 server-side files ad19c3c Update to crispresso 2.0.40 prime editing e3a194a update errors and ignore email config 2efb0bb Update README.md 58844a6 initial commit 8ff1878 Initial commit git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 1efad70 * Indentation and parenthesis * Semi-colon to README * Add targets for .env and clean in Makefile * Add flower support to the development version * Add support to map individual directories in Docker Compose * Fix typo in README * Squashed 'CRISPRessoWEB/CRISPRessoReports/' changes from 1efad70..13f7ae2 13f7ae2 Fix command used and parameters elements. Increase print width and height to 100% dcf7391 Adding styling for print-only and screen only git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 13f7ae2 * Working in docker * Increase the size of the center column when printing * Remove some page breaks * Restore block statement * Spacing fix for empty page problem * Change gunicorn reload engine to poll This is because when running through x86 emulation (on M1 Mac), the inotify file system events are blocked. But reloading works with polling! * Pin SQLAlchemy version to 2.5.1 because newer versions don't work Also, this fixes the entrypoint for the dev build stage. * Make clarifications in README and fix more spelling * Add back in deleted report.html * Don't add styles to the database if they already exist * Upgrade to python 3.9 * Add report_data object to render templates * Reset subtree * Squashed 'CRISPRessoWEB/CRISPRessoReports/' content from commit 69cb5e2 git-subtree-dir: CRISPRessoWEB/CRISPRessoReports git-subtree-split: 69cb5e2 * Fix region_name to be run_names * Update sizing on graphs * Create environmental variable for crispresso_version and load colors if version is above 2.2.12 * Make env variable use ARG * Add version check * Changes to work with Docker compose * Fix style selection database path * Add pe_ref_seq to WGS input and update to data-bs-toggle * Remove extra div that affected the height of prime edited ref seq * Fix gene annotation file label cut off on WGS * Move style function to after folder_id is declared * Docker compose pin versions and search C2 args for --config_files * Remove gtag code * Update CRISPResso to 2.2.14 * Specify platform in docker compose file * Move jinja_partials install from dev to build stage * Fix running when no default style is selected (or when selected style isn't available) * Add zip and unzip to prod step * Clean execution logs after run finishes This will clean the full paths from the execution logs so that those aren't exposed to the user. This also ensures that this only happens in one place in the code (instead of multiple). * Update path to favicon * Reorder conda channels, remove flask-sqlalchemy pin, and fix wtforms The latest version of wtforms has moved ColorInput from `wtforms.widgets.html5` to `wtforms.widgets`, which is reflected in this commit. * Fix S3 file upload by adding form field * Remove pyopenssl pin * Add create_styles and createUsers to app context in init_db.py * Fixed padding in S3 file upload * Removed commented out S3 upload code for CRISPRessoPooled * Add volume mount to Apache in dev version This allows any files that are edited in the static folder to hot reload. * Don't show root folder of S3 buckets because it leads to weird behavior * Fix old Bootstrap margin and padding utilities The new version of Bootstrap has replaced `ml-1` with `ms-1` and `mr-1` with `me-1`, etc. Instead of being "left" and "right", it is now "start" and "end" to account for right to left languages. * Fix the close button on the S3 modal * Fix positive quantification window in radio button label * Add another app context to init_db.py * Update IDs for jQuery examples and how radio buttons are selected Because of upgrading to Bootstrap 5, the way that labels and inputs needed to be formatted, the previous way of selecting a radio button input no longer worked. Now to select a radio button element programmatically, you can issue a `.click()` and it will be selected. * Remove extra report file * Replace deprecated padding utility classes in report * Remove duplicate id's and add a few aria labels * Add correct MIME type to submission.js file * Disable caching on submission pages to improve back button behavior * Add + to quantification window for pooled and WGS * Add S3 buckets to WGS and Compare * Remove S3 buckets from WGS Not going to implement this now, because it would be a significant effort to do it correctly. * Add note to S3 modal about large files being expensive * Don't show style selector when `--config_file` parameter isn't available * Install CRISPRessoPro in the dev environment * Fix error with app contexts and databases in unit tests * Implement handling duplicate style names when saving to db * Move Plotly JS import to reports templates and out of layout.html * Remove font installer, less, and vim dependencies --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * added metadata to report partial * fixed logout button condition in banner * Fix loading favicon.ico, remove duplicate log_params, and fix README typos * Fix bug where `metadata` key is not found in `report_data` --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: McKay <[email protected]>
mbowcut2
added a commit
that referenced
this pull request
Jun 19, 2024
* Reports, add reports to packages, colors, ordered pandas sort (#28) * Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 7d9b4e5..e18807d * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from e18807d..e9da7bf git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: e9da7bff794058e1fcdb3dc9ced79871c6a30e18 * Add CRISPRessoReports to packages * Colors only with pro * changed tuple to list for matplotlib change (#31) * wgs and batch failed runs implementation * Added failed run functionality including shared function, edits to Report, and displaying with HTML and Javascript * Merge CRISPRessoReports master into failed-batch-runs * Cole's failed-batch-runs review and changes (#36) * Fix showing link to report in CLI (only show in web) * Remove styling of jumbotron The p-5 added some weird space at the top of the container, the rounded-3 did not make a difference (because there is no background), and the h-100 also did not make a difference. * Remove extra spaces at end of the line * Remove color legend from figure caption in plot 4f * Refactor fig_reports.html partial to reduce duplication * Add opacity to custom colors on allele quilt plot * Remove extra spaces * Change default color of deletion It looked too similar to `N` and was difficult to tell apart. * Refactor plot 10c, refactor displaying of figures This commit adds flexbox to the plots, this was mainly for plots 10b and 10c because their alignment was off. * Add more plots to get the correct percentages for width * Remove setting the height of the plots * Check for failed batch info before retrieving it in `make_multi_report_from_folder` * Fix extraneous whitespace in `fig_reports` partial * Only load certain resources when on web mode * Move jQuery import to bottom of the page to improve performance * Extract out report footer buttons to partial * Fix too many closing divs in batchReport.html * Refactor failed runs to be a partial * Move the failed run JS to the partial This has the benefit of keeping the relevant code close, and also prevents the error that we were running into before where `chevronIcon` wasn't found when there were no failed runs (because the element wasn't there). * Remove `report_name` id because it probably has spaces * Move existing Plotly plots to batchReport from multiReport * Fix typo in fig 11c and resize it to 40% --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]>
mbowcut2
added a commit
that referenced
this pull request
Nov 8, 2024
* Reports, add reports to packages, colors, ordered pandas sort (#28) * Sort by #Reads instead of %Reads to avoid floating point errors * Fix x-axis spacing on some reports * Add break to header matching loop to prevent match statements being printed after failure * Check all headers and only error if there are unmatched values * Fix indent * Remove missing_header variable * Fix tick marks * Squashed 'CRISPResso2/CRISPRessoReports/' changes from 7d9b4e5..e18807d * X-axis tick fix on fig 6a * Fix function name from styles to config * Squashed 'CRISPResso2/CRISPRessoReports/' changes from e18807d..e9da7bf git-subtree-dir: CRISPResso2/CRISPRessoReports git-subtree-split: e9da7bff794058e1fcdb3dc9ced79871c6a30e18 * Add CRISPRessoReports to packages * Colors only with pro * changed tuple to list for matplotlib change (#31) * wgs and batch failed runs implementation * Added failed run functionality including shared function, edits to Report, and displaying with HTML and Javascript * Merge CRISPRessoReports master into failed-batch-runs * Cole's failed-batch-runs review and changes (#36) * Fix showing link to report in CLI (only show in web) * Remove styling of jumbotron The p-5 added some weird space at the top of the container, the rounded-3 did not make a difference (because there is no background), and the h-100 also did not make a difference. * Remove extra spaces at end of the line * Remove color legend from figure caption in plot 4f * Refactor fig_reports.html partial to reduce duplication * Add opacity to custom colors on allele quilt plot * Remove extra spaces * Change default color of deletion It looked too similar to `N` and was difficult to tell apart. * Refactor plot 10c, refactor displaying of figures This commit adds flexbox to the plots, this was mainly for plots 10b and 10c because their alignment was off. * Add more plots to get the correct percentages for width * Remove setting the height of the plots * Check for failed batch info before retrieving it in `make_multi_report_from_folder` * Fix extraneous whitespace in `fig_reports` partial * Only load certain resources when on web mode * Move jQuery import to bottom of the page to improve performance * Extract out report footer buttons to partial * Fix too many closing divs in batchReport.html * Refactor failed runs to be a partial * Move the failed run JS to the partial This has the benefit of keeping the relevant code close, and also prevents the error that we were running into before where `chevronIcon` wasn't found when there were no failed runs (because the element wasn't there). * Remove `report_name` id because it probably has spaces * Move existing Plotly plots to batchReport from multiReport * Fix typo in fig 11c and resize it to 40% --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, if the --skip_failed flag was set to true, it would result in the following bad UX:
In CRISPRessoBatch, WGS, and Pooled, failed files/regions/amplicons would simply disappear from the resulting HTML report.
With the completion of this project, these failed runs will now be displayed along with an error message.
This was accomplished by creating a CRISPRessoShared function that checks the output folder and retrieves the error message from CRISPResso_status.txt, enriches the CRISPRessoReport package with this information to then conditionally render HTML along with some JavaScript to display information about the failed runs.