Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update plausible unit concept ID list #570

Closed
katy-sadowski opened this issue Oct 15, 2024 · 5 comments
Closed

Update plausible unit concept ID list #570

katy-sadowski opened this issue Oct 15, 2024 · 5 comments
Assignees
Labels
check bug/enhancement DQ check SQL logic has a bug or needs refinement

Comments

@katy-sadowski
Copy link
Collaborator

katy-sadowski commented Oct 15, 2024

Update all 3 concept-level threshold files to include some additional plausible unit concepts:

  • The number and percent of records for a given CONCEPT_ID 3009201 (THYROTROPIN [UNITS/VOLUME] IN SERUM OR PLASMA) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (44777578,8719,9040,9093)). (Threshold=5%).
  • The number and percent of records for a given CONCEPT_ID 3010813 (LEUKOCYTES [#/VOLUME] IN BLOOD) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (44777588,8848,8961,9444)). (Threshold=5%).
  • The number and percent of records for a given CONCEPT_ID 3007461 (PLATELETS [#/VOLUME] IN BLOOD) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8848,8961,9444)). (Threshold=5%).
@katy-sadowski katy-sadowski added the check bug/enhancement DQ check SQL logic has a bug or needs refinement label Oct 15, 2024
@katy-sadowski
Copy link
Collaborator Author

The number and percent of records for a given CONCEPT_ID 3010813 (LEUKOCYTES [#/VOLUME] IN BLOOD) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (44777588,8848,8961,9444)). (Threshold=5%).

Is missing https://athena.ohdsi.org/search-terms/terms/8647

@katy-sadowski
Copy link
Collaborator Author

The number and percent of records for a given CONCEPT_ID 3007461 (PLATELETS [#/VOLUME] IN BLOOD) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8848,8961,9444)). (Threshold=5%).

Is missing https://athena.ohdsi.org/search-terms/terms/32706

@adelmestri
Copy link

adelmestri commented Nov 27, 2024

Some more issues to be solved on UNIT_CONCEPT_ID:
Table = ‘Measurement’, category = ‘Plausibility’, Status = ‘Fail’

  1. The number and percent of records for a given CONCEPT_ID 4148615 (NEUTROPHIL COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8784,8848,8961)). (Threshold=5%).” UNIT_CONCEPT_ID = 9444 = ‘1 billion per liter’ should be added to the list (8784,8848,8961).
    In fact
    8848 = thousand per microliter = 10^3 per microliter
    1 billion = 10^9
    1 liter = 10^6 microliter
    1 billion per liter = thousand per microliter

  2. The number and percent of records for a given CONCEPT_ID 4254663 (LYMPHOCYTE COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8848)). (Threshold=5%).
    UNIT_CONCEPT_ID = 9444 = ‘1 billion per liter’ should be added to the list for the same reason as point above

  3. The number and percent of records for a given CONCEPT_ID 4298431 (WHITE BLOOD CELL COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8848,8961)). (Threshold=5%).
    UNIT_CONCEPT_ID = 9444 = ‘1 billion per liter’ should be added to the list for the same reason as point above

  4. The number and percent of records for a given CONCEPT_ID 4030871 (RED BLOOD CELL COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8734,8815,8931,9444)). (Threshold=5%).
    UNIT_CONCEPT_ID = 9445 = 'billion per milliliter' should be added:
    8734 = 'trillion per liter'
    Billion = trillion/10^3
    Milliliter = liter/10^3

  5. The number and percent of records for a given CONCEPT_ID 4197602 (SERUM TSH MEASUREMENT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8719,9040,9093)). (Threshold=5%).
    8719 = ‘milliunit per milliliter’
    9040 = ‘milli-international unit per liter’
    9093 = ‘micro-international unit per milliliter’

The following should be added to the list of acceptable UNIT_CONCEPT_ID:
44777578 = ‘milliunit per liter’
8750 = ‘microunit per liter’
8923 = ‘international unit per liter’

@katy-sadowski
Copy link
Collaborator Author

More:

  • The number and percent of records for a given CONCEPT_ID 4184637 (HEMOGLOBIN A1C MEASUREMENT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8554,8632,8737)). (Threshold=5%). - add 9579 - millimole per mole
  • The number and percent of records for a given CONCEPT_ID 37393850 (MCHC - MEAN CORPUSCULAR HAEMOGLOBIN CONCENTRATION) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8636,8713)). (Threshold=5%). - add 8554 - percent and 8753 millimole per liter
  • The number and percent of records for a given CONCEPT_ID 4151358 (HEMATOCRIT DETERMINATION) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (44777604,8554)). (Threshold=5%). - add 8523 - ratio
  • The number and percent of records for a given CONCEPT_ID 4197602 (SERUM TSH MEASUREMENT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8719,9040,9093)). (Threshold=5%). -add 44777578 - milliunit per liter; 44777583 - micro-international unit per liter; 8750 - microunit per liter
  • The number and percent of records for a given CONCEPT_ID 3034639 (HEMOGLOBIN A1C [MASS/VOLUME] IN BLOOD) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8713,8840)). (Threshold=5%). - add 9579 - millimole per mole and 8923 - international unit per liter
  • The number and percent of records for a given CONCEPT_ID 4030871 (RED BLOOD CELL COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8734,8815,8931,9444)). (Threshold=5%). - add 9445 - billion per milliliter
  • The number and percent of records for a given CONCEPT_ID 3000034 (MICROALBUMIN [MASS/VOLUME] IN URINE) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8576,8723,8751,8840,8859)). (Threshold=5%). - add 8636 - gram per liter
  • The number and percent of records for a given CONCEPT_ID 3016723 (CREATININE [MASS/VOLUME] IN SERUM OR PLASMA) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8840)). (Threshold=5%). - add 8749 - micromole per liter
  • The number and percent of records for a given CONCEPT_ID 3024128 (BILIRUBIN.TOTAL [MASS/VOLUME] IN SERUM OR PLASMA) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8840)). (Threshold=5%). - add 8749 - micromole per liter
  • add 8753 - millimole per liter for:
    • The number and percent of records for a given CONCEPT_ID 3003338 (MCHC [MASS/VOLUME]) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8713)). (Threshold=5%).
    • The number and percent of records for a given CONCEPT_ID 3022192 (TRIGLYCERIDE [MASS/VOLUME] IN SERUM OR PLASMA) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8840)). (Threshold=5%).
    • The number and percent of records for a given CONCEPT_ID 3004501 (GLUCOSE [MASS/VOLUME] IN SERUM OR PLASMA) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8840)). (Threshold=5%).
    • The number and percent of records for a given CONCEPT_ID 4191837 (CALCULATED LDL CHOLESTEROL LEVEL) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8840)). (Threshold=5%).
  • The number and percent of records for a given CONCEPT_ID 3037556 (URATE [MASS/VOLUME] IN SERUM OR PLASMA) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8840)). (Threshold=5%). - add 8923 - international unit per liter
  • add 9444 - billion per liter to:
    • The number and percent of records for a given CONCEPT_ID 40487382 (TOTAL LYMPHOCYTE COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8784,8848,8961)). (Threshold=5%).
    • The number and percent of records for a given CONCEPT_ID 4298431 (WHITE BLOOD CELL COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8848,8961)). (Threshold=5%).
    • The number and percent of records for a given CONCEPT_ID 4254663 (LYMPHOCYTE COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8848)). (Threshold=5%).
    • The number and percent of records for a given CONCEPT_ID 4148615 (NEUTROPHIL COUNT) with implausible units (i.e., UNIT_CONCEPT_ID NOT IN (8784,8848,8961)). (Threshold=5%).

@katy-sadowski
Copy link
Collaborator Author

These were fixed in release 2.6.2 :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
check bug/enhancement DQ check SQL logic has a bug or needs refinement
Projects
Status: Merged into develop branch
Development

No branches or pull requests

3 participants