-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
investigate TRAPI validation errors showing in ARAX UI #587
Comments
How to use the TRAPI validator:
|
In Andrew's examples, there are 4 categories of errors raised by the validator. I'm posting a set of 3 comments that cover these error categories. Investigation Part 1: "invalid qualifiers" My guess is that the validator is flagging edges where the However, it looks like we are modeling this exactly how we were told by Sierra: put the biolink prefix on qualified-predicate values and no prefixes on the enum values (Translator Slack discussion, notes, and related Translator Slack discussion) From Example 1 (this is JSON)
From the notebook (this is python dict, printed)https://github.com/colleenXu/RegistryMetadataDev/blob/main/ValidatingBTE_Response.ipynb
IN-PROGRESS: Translator Slack discussion with Sierra and Richard B |
I was thinking that "activity_or_abundance" wasn't a valid value. Is that a valid value? |
Where does one find a list of allowed values? Not finding it here: |
Investigation Part 2: "unknown edge attribute-type-id" and "missing primary knowledge source" Both of these are only on edges from Text-Mining KP. To investigate, I ran a query earlier today, where BTE would only use Text-Mining KP edges.
STATUS: I've informed their team through Translator Slack. It'll be their responsibility to handle this. |
Thanks. FYI https://arax.ncats.io/beta is now running the TRAPI validator version 3.4.9 |
Investigation Part 3: "edge attribute_type_id not curie" This still needs discussion within our team. My notes on this specific error
My commentary on the larger issues
|
Note that with the ARAX-beta update that just happened, Andrew's examples now give have different validation-reports that have more issues. I haven't analyzed these yet. |
A more recent example: ARAX link with validation codes dictionary Validation json
Analysis of errorsThere's 1 type of error, where an Edge attribute's type_id isn't a curie and biolink-model/TRAPI/Translator expects it to be...
Analysis of warningsThere are multiple warnings, however these are either not related to BTE or are ongoing issues...
|
it's more of an node attribute and reasoner-validator considers it not a valid edge-attribute type_id
Regarding Investigation Part 3 / attribute_type_id and value issues...
|
Just a note for future reference. Given how much other stuff we have on our plate, I'm only concerned with TRAPI validation errors at the moment. Validation warnings can be put on the back burner unless there's something specific we collectively agree at a prioritization meeting that we want to address... |
From today's group meeting:
Our understanding is that this basically == "green checkmark in ARAX UI" for creative-mode queries going through UI/ARS. Unclear if sending queries to BTE through ARAX UI will also trigger the validation This "green checkmark in ARAX UI" is now a goal. We agreed to start paying more attention to that part of the ARAX UI, bringing up issues we see, and addressing them |
I think the answer is probably that it will not. Just because of how the system has evolved, validation happens during fetching of pre-existing PKs from the ARS. but not queries initiated through our UI. But not completely certain in this answer. |
One known issue is edge-attributes not being curies / biolink-model association-slot terms (Discussed in the past, here as "Part 3" and here) I'd like to run new queries through the UI/ARS, and see if these are still showing up as a validation error. And if we still need to address it...
|
Analyzing the latest ARAX-UI+reasoner-validator validation of BTE responsesquery: creative-mode MVP1, what treats acanthosis nigricans
Using ARAX-UI-beta: parent PK and BTE's response PK Versioning: Analyzing ErrorsThere were no critical ("red") errors. There were some lower-level/ordinary ("orange") errors: click here to expandEdge has a value that is not a CURIE for attribute_type_id (Knowledge Graph Edge Attribute)To-do: described in my previous comment. We already know about this issue. full text
Qualifier_type_id for edge has unresolved qualifier_value (Knowledge Graph Edge Qualifier)
activity_or_abundance is a valid value for an To-do: Follow up with Sierra (biolink-model) and Richard B (reasoner-validator). There appears to be errors in biolink-model's
This has been previously discussed in this issue. Edge has unknown attribute_type_id (Knowledge Graph Edge Attribute)
To-do: Follow-up with Edgar (Text-Mining). This is coming from their KP, and the issue seems to be that biolink-model 3.5.0 doesn't have the association-slot/edge-attribute "semmed_agreement_count"... Analyzing WarningsThese all come from biolink-model or NodeNorm behavior...and I don't think code changes are needed for BTE. It's a less-urgent to-do to let those tools' devs know. click to expandNode identifiers found unmapped to target categories for node (Query Graph Node)
I don't think this is something to worry about. This is a mismatch between what a user may assign a QNode category to be vs. what the categories of the QNode IDs are in NodeNorm. The origin is: biolink-model doesn't list MONDO as an accepted prefix for DiseaseOrPhenotypicFeature (in fact no prefixes are given). Node identifier found unmapped to target categories for node (Knowledge Graph Node)
I don't think this is something to worry about. The origin is: biolink-model doesn't list CHEMBL.COMPOUND as an accepted prefix for ChemicalEntity Node has a abstract or mixin category (Knowledge Graph Node)
I don't think this is something to worry about. This ID is The origin is:
Edge has an attribute_type_id that is not an association slot (Knowledge Graph Edge Attribute)
Both evidence_type and support_graphs exist in biolink-model, but aren't association-slots. We can remove the evidence_type edge-attribute (see previous comment)... but the |
[DONE: 2023-07-20 late night] As discussed in the earlier comment, I'm commenting out / adjusting the fields BTE queries for and the response-mapping. This will address a pervasive TRAPI validation error: Covered in these commits: |
chemical-affects-gene-association aspect qualifiers were set to the "Part" enum, when they should probably be set to the "Aspect" Enum. This was identified in biothings/biothings_explorer#587 (comment) (under Analyzing Errors, Qualifier_type_id for edge has unresolved qualifier_value (Knowledge Graph Edge Qualifier)
Seeing what validation issues still exist after adjusting x-bte annotationRan the same creative-mode MVP1 query from my post last week. I think our responses are basically good! Using ARAX-UI-dev: parent PK and BTE's response PK Versioning: same as before ( Analyzing ErrorsThere were still no critical ("red") errors. Of the lower-level/ordinary ("orange") errors: these are all addressed or communicated to the other teams responsible
trapi validation error
Analyzing Warningsthese are all addressed or communicated to the other teams responsible
|
Marking as |
Update on what's going on now:
|
Moving to on-hold because by now, it's very much a "wait for Translator to update biolink-model / reasoner-validator"... |
The latest validator version 3.8.0 configured to validate against Biolink model 3.5.3 is now deployed at https://arax.ci.transltr.io/ |
With my n=1 spot check at https://arax.ci.transltr.io/?r=a13266eb-fd85-4bd6-95b9-6510832b35e8, BTE is returning a green check. @colleenXu unless you had additional testing you wanted to do, feel free to close this issue and check BTE off on NCATSTranslator/Feedback#379. Thanks all! |
Okay, going to close this issue. Yay! |
Note that I see validation issues when BTE uses the The use of this predicate was previously brought up in biolink/biolink-model#1171 |
An updated reasoner-validator has been deployed to the beta version of the ARAX UI. BTE (and most teams) are showing errors. Example links are below, more info can be found by mousing over and clicking on the TRAPI 1.3 column:
Example 1: https://arax.ncats.io/beta/?r=30fa40cd-e74d-4532-ada1-9914c87db20a
TRAPI validator error
BTE response: https://arax.ncats.io/api/arax/v1.3/response/cba4c3db-1318-4937-b755-1ba8655fe773
Example 2: https://arax.ncats.io/beta/?r=61f8c292-39d3-4f8c-9831-ad26c6a793b6
More info on the validation errors are available at https://ncatstranslator.github.io/reasoner-validator/validation_codes_dictionary.html
The text was updated successfully, but these errors were encountered: