Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Valuset builder #298

Merged
merged 7 commits into from
Sep 24, 2024
Merged

Valuset builder #298

merged 7 commits into from
Sep 24, 2024

Conversation

dogversioning
Copy link
Contributor

This makes the following changes:

  • Removed map of deprecated 'statistics' module to 'builders' (breaking)
    • Added a builder for constructing clinical valusets from the valueset study
    • Light restructuring of PSM builder due to structure change
  • Adjusted CTAS from parquet creation for better cross-DB compatibility

Checklist

  • Consider if documentation in docs/ needs to be updated
    • If you've changed the structure of a table, you may need to run generate-md
    • If you've added/removed core study fields that not in US Core, update our list of those in core-study-details.md
  • Consider if tests should be added
  • Update template repo if there are changes to study configuration in manifest.toml

This makes the following changes:

- Removed map of deprecated 'statistics' module to 'builders' (breaking)
  - Added a builder for constructing clinical valusets
  - Light restructuring of PSM builder due to structure change
- Adjusted CTAS from parquet creation for better cross-DB compatibility
docs/builders/valueset.md Show resolved Hide resolved
expansion_rules_path = (
pathlib.Path(__file__).resolve().parents[0] / "expansion_rules.tsv"
)
# TODO: get this table to write its parquet to the valueset_data folder
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just seeing this note to myself on review - i'll look into this

cumulus_library/builders/valueset_builder.py Show resolved Hide resolved
Copy link
Contributor

@mikix mikix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK mostly ignored the builder python and test code for my own sanity - but I seemed to have lots of opinion on docs 👀

@@ -19,8 +19,7 @@
log_utils,
study_manifest,
)
from cumulus_library.builders import protected_table_builder
from cumulus_library.statistics import psm
from cumulus_library.builders import protected_table_builder, psm_builder, valueset_builder
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: your own life might be better if you re-exported all the individual builder classes in builders/__init__.py, so that you could do the following:

from cumulus_library import builders
builder = builders.PsmBuilder()

This also avoids some churn as you add/remove/rename builders.

(I've wound up on this pattern in the ETL anyway - all the cumulus_etl.* modules are toplevel concepts like "loaders" or "nlp" and then those modules are little fiefdoms that export their internals out to the top level. Helps me mentally map the code structure too, to keep semantic meaning one level deep. I don't always hit that mark, but I try 😄)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm slowly coming around to this view of things - i may do it in a separate PR for my own sanity.

docs/builders/valueset.md Outdated Show resolved Hide resolved
docs/builders/valueset.md Outdated Show resolved Hide resolved
docs/builders/valueset.md Show resolved Hide resolved
docs/builders/valueset.md Outdated Show resolved Hide resolved
docs/builders/valueset.md Outdated Show resolved Hide resolved
docs/builders/valueset.md Outdated Show resolved Hide resolved
docs/builders/valueset.md Outdated Show resolved Hide resolved
docs/builders/valueset.md Outdated Show resolved Hide resolved
cumulus_library/builders/valueset/expansion_rules.tsv Outdated Show resolved Hide resolved
Copy link

github-actions bot commented Sep 23, 2024

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
2240 2228 99% 90% 🟢

New Files

File Coverage Status
cumulus_library/builders/valueset/additional_rules_builder.py 100% 🟢
cumulus_library/builders/valueset/rxnorm_valueset_builder.py 100% 🟢
cumulus_library/builders/valueset/static_builder.py 100% 🟢
cumulus_library/builders/valueset/umls.py 100% 🟢
cumulus_library/builders/valueset/valueset_utils.py 100% 🟢
cumulus_library/builders/valueset/vsac.py 100% 🟢
cumulus_library/builders/valueset_builder.py 100% 🟢
TOTAL 100% 🟢

Modified Files

File Coverage Status
cumulus_library/init.py 100% 🟢
cumulus_library/actions/builder.py 100% 🟢
cumulus_library/actions/importer.py 100% 🟢
cumulus_library/builders/counts.py 100% 🟢
cumulus_library/databases/duckdb.py 100% 🟢
cumulus_library/study_manifest.py 100% 🟢
TOTAL 100% 🟢

updated for commit: 4d01bad by action🐍

@dogversioning dogversioning merged commit b1a484c into main Sep 24, 2024
3 checks passed
@dogversioning dogversioning deleted the mg/valueset branch September 24, 2024 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants