Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scoring: broaden use of "knowledge level" and "agent type" into scoring function #715

Open
andrewsu opened this issue Aug 28, 2023 · 1 comment

Comments

@andrewsu
Copy link
Member

As defined in this document, knowledge_level can take one of the following values:

knowledge_assertion
logical_entailment
prediction
statistical_association
observation
not_provided

and agent_type can take the following values:

manual_agent
automated_agent
 — data_analysis_pipeline
 — computational_model
 — text_mining_agent
 — image_processing_agent
manual_validation_of_ automated_agent
not_provided

These are currently provided in the infores catalog (e.g., these lines for AGRKB), though the vast majority of agent_type is currently not_provided. In the near future, these edge properties will become part of the Biolink Model itself (see this branch)

This information should be very useful for scoring. Currently, we hard-code a list of text-mined resources. This issue will track the expansion of this effort. As far as I can see, the sequence of steps will be something like this:

  • update of Biolink Model
  • update our x-bte specification to include these fields
  • update SmartAPI annotations to use these fields
  • update BTE to consume this information
  • update scoring analysis to weight the different values appropriately.

This will be useful because we do have many resources that combine edges of with very different provenance. For example, the drug indications from ChEML in turn draw from many resources, like DailyMed and ClinicalTrials.gov. It appears that the edges based on Daily Med (e.g., acetaminophen - treats - back pain are much more reliable than text-mined edges based on clinicalTrials.gov (e.g., acetaminophen - treats - cleft palate). See indications in ChEBML record for acetaminophen.

@colleenXu
Copy link
Collaborator

Update on knowledge_level / agent_type:

  • we know these values "ahead of time" when they're in the x-bte annotation. These fields have been added to the x-bte annotation used by Dev/CI instances (see this comment).
  • But for Text-Mining/Multiomics/TRAPI APIs, we should get these values in their TRAPI edge's attributes after parsing their responses. And at the moment, there may be cases where there's no KL/AT edge-attribute info because this feature is still being rolled out.

We may want to flag / gracefully handle when an edgeg doesn't have any KL/AT info...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants