Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New version of SweFreq with PostgreSQL #557

Merged
merged 390 commits into from
May 9, 2019
Merged

New version of SweFreq with PostgreSQL #557

merged 390 commits into from
May 9, 2019

Conversation

talavis
Copy link
Contributor

@talavis talavis commented May 9, 2019

Describe the pull request:

  • Bug fix
  • Functional change
  • New feature
  • Code cleanup
  • Build system change
  • Documentation change
  • Language translation

Pull request long description:

General

  • Database migrated from mysql and mongodb to postgresql
    • New database schemas
  • Import scripts for datasets and references to the new database
  • Support for registered access for Dataset VCF files
  • Simplified setup for development enviroment
  • Beacon migrated to "external" elixir beacon
  • Beacon UI changes
  • Target python version is now 3.6
  • A lof of bug fixes

Browser

  • Browser migrated to postgresql
  • Browser UI changes, e.g. dynamic updates to number of shown variants
  • Tests for most of the browser backend
  • Support for showing the same variant in other datasets on the browser variant page
  • Download will now download the visible variants, not all variants for the gene/region/etc
  • A lof of bug fixes

talavis added 30 commits March 20, 2019 16:16
@talavis
fix slight error
e6b3cd5
@talavis
more tests added, complete coverage in lookups; also some fixes due t…
7d73335 
…o problems found during testing
@talavis
More tests for utils, including some fixing in utils
e4841f3
@talavis
incorrect indentation of a return statement
309bcdb
@talavis
100% coverage, all passing in utils
2325da3
@talavis
pylint fixes in lookups
b14ba70
@talavis
pylint fixes in utils
cc2fe67
@talavis
type hints added
baaf982
@talavis
migrate the download function
3c0f904
@talavis
some fixes for tests, documentation additions
6dbccd4
@talavis
Api routes updated to support dataset version as well.
400bc17 
Thanks to @kusalananda for help with regex.
@talavis
updated get_gene_by_name for new schema
7bc68c2
@talavis
should fix get_variants_in_gene for new db schema
c3ae8e4
@talavis
should fix get_variants_in_transcripts for new db schema, also some o…
40ffcf1 
…ther fixes in the get_variants_in functions
@talavis
a couple of small fixes
2d7c4f5
@talavis
remove when db is fixed
c407e9f
@talavis
adding genes and transcripts to variants after db schema update
e9faf47
@talavis
seems to work with new db schema, but speed is not optimal
e48d54c
@talavis
decreased logging
19f1698
@talavis
rsid fixing; they can't be '.' with new db
85644cb
@talavis
converting tests to chromosome 22
428b12e
@talavis
fix for finding other names; return the db.Gene part
314eb20
@talavis
handling a few cases of missing data observed during testing
dccc240
@talavis
fix incorrect naming of returned genes/transcripts
887baed
@talavis
further migration of test to chromosome 22
f833739
@talavis
lookups updated for testing only chr22
862f0c8
@talavis
pgsql updated for testing only chr22
667a8a5
@talavis
utils updated to test chr 22
133b6d6
@talavis
rsid != '.' in postgres
816f842
@talavis
increased coverage
9a9e518
MalinAhlberg and others added 28 commits April 25, 2019 08:24
@MalinAhlberg
Use "create or replace" for beacon sql
abf7853
@talavis
access_level is now uppercase; None becomes PRIVATE
9072394
@talavis
Merge pull request #541 from NBISweden/feature/compute_callcount
3395599 
Feature/compute callcount
@MalinAhlberg
Get accesslevel from datasets
28dea19
@MalinAhlberg
Use beacon_access in beacon_dataset_table
c55c7f3
@MalinAhlberg
Show all available dataset versions in beacon
b8ddbd0
@MalinAhlberg
Use view to filter out beacon datasets
bec96e2
@talavis
Merge pull request #548 from NBISweden/feature/access_level
a662559 
Use new access levels in beacon
@talavis
Merge pull request #547 from NBISweden/feature/sql-enum-upper
65d0d26 
Make access_level uppercase with PRIVATE instead of None
@talavis
Remove var_call_ref from database.
940c0ee 
Retrieve the value from data.reference_sets.reference_build instead.
@viklund
Merge pull request #549 from NBISweden/feature/drop-var-call-ref
f716868 
Remove var_call_ref from database.
@talavis
Change beacon to use the elixir beacon instead of the simple internal…
cc485c7 
… one.
@talavis
remove beacon
0d2c571
@talavis
do not import beacon
2518ccb
@talavis
remove debugging message
3b898de
@talavis
Add empty category and flags field in case of empty VEP.
83d1240
@viklund
Merge pull request #551 from NBISweden/bugfix/vep
a834839 
Add empty category and flags field in case of empty VEP
@viklund
Merge pull request #550 from NBISweden/feature/beacon-requests
7ef2c91 
Make the frontend beacon use the elixir beacon
@talavis
Make beacon searches specific to dataset and remove reference selecti…
de997c5 
…on from the the frontend.
@talavis
Add example to beacon page
7a2d9d1
@norling
Merge pull request #552 from NBISweden/feature/beacon-requests
898219c 
Make beacon search specific, remove reference selection
@talavis
prevent position from being < 0
c0a2c6b
@talavis
Merge pull request #553 from NBISweden/bugfix/beacon-fin
e94a615 
Prevent position from being < 0 in the beacon UI
@talavis
Update the sFTP password generator to use the digest function of Post…
51dad58 
…greSQL.
@talavis
Remove sFTP tab from admin for dataset as it's not working.
a556670
@norling
Merge pull request #554 from NBISweden/feature/sftp-fixing
07cea86 
Fix sFTP for Postgres
@talavis
Better region parsing.
ddd6572 
Can now handle regions in the format chrom:pos and chrom:pos1-pos2.

Also changed behaviour for when to return errors.
@viklund
Merge pull request #556 from NBISweden/feature/range-only-start
ff7db7f 
Make it possible to request ranges as single position
viklund
viklund approved these changes May 9, 2019
View reviewed changes
Copy link
Member

@viklund viklund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dev page is working. I think that is enough for now.

@viklund viklund merged commit 2446814 into master May 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@viklund viklund viklund approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@talavis @viklund @MalinAhlberg @norling