part 1: review/add Translator resources to BTE's API list

[andrewsu]

Long ago we made a principled decision that we would use an API allow list to define what resources BTE would include. We should revisit that decision to make sure we still agree that's the right choice. And if so, we should review resources that are not currently being called by BTE for possible inclusion in our API allow list.

[colleenXu]

UPDATE:

I made a minimal list of APIs BTE isn't using right now. This includes TRAPI KPs and APIs with x-bte that are also tagged translator.

We decided to...

Try adding 5 TRAPI KPs that I was able to retrieve edges from thru BTE in manual testing: biothings/bte-server#43

• automat robokop (remove all other automats in favor of using this to access all resources)
• multiomics clinical trials (TRAPI from Plover) - see migrate to Multiomics CTKP, remove other clinicaltrials.gov-based x-bte operations #861 (comment)
• multiomics drug approvals (TRAPI from Plover)
• multiomics microbiome kp (TRAPI from Plover)
• multiomics kp (TRAPI from Plover)

Considering, see Part 2...

• genetics provider: meta_knowledge_graph response edges' knowledge_types is empty array. Sent Translator Slack message in reasonerapi channel. Aragorn doesn't have issues with it.
• Knowledge Collaboratory: ARAX team reports that this API always errors out, takes 30s to respond with 500 error Knowledge Collaboratory KP mostly responding with 500 error NCATSTranslator/Feedback#946
• OpenPredict: ARAX team reports that this API often errors out, takes 60s to respond with 504 error OpenPredict KP often responds with 504 error in Test NCATSTranslator/Feedback#947
• MolePro: Queries to it timeout often, and adds wait time. And we may be querying it more with the record limit, parallel features.

Wait on...

• 2 TRAPI KPs that I had issues with during manual testing:
  • RTX-KG2: previously identified problems with BTE's retrieved edges. Haven't discovered cause yet. Aragorn doesn't have issues with it.
  • text-mined co-occurrence: uncompatible TRAPI version (1.4) + meta_knowledge_graph response edges' knowledge_types is null. Sent Translator Slack message in reasonerapi channel
• 1 TRAPI KP that ARAX + Aragorn have had issues with:
  • SPOKE: ARAX team reports that this API often errors out within 1s SPOKE often responds with 500 error in Test NCATSTranslator/Feedback#948

Don't add (for now): The in-limbo APIs with x-bte

• PharmGKB Evaluate annotation of PharmGKB API #556
• RaMP RaMP: process of x-bte annotation + adding to BTE  #705

[colleenXu]

Marc Duby has reported that Genetics KP CI has a fix for knowledge_types (Translator Slack). I haven't checked yet.

I'm assuming adding this KP would be for the next sprint, not this one...

[tokebe]

Yeah, I don't think we want to rush adding new KPs...

[colleenXu]

This issue is to track the 5 KPs we did add to BTE #865 (comment)

• automat robokop (remove all other automats in favor of using this to access all resources)
• multiomics clinical trials (TRAPI from Plover) - see migrate to Multiomics CTKP, remove other clinicaltrials.gov-based x-bte operations #861 (comment)
• multiomics drug approvals (TRAPI from Plover)
• multiomics microbiome kp (TRAPI from Plover)
• multiomics kp (TRAPI from Plover)

[colleenXu]

Given urgent issues with BTE timeouts: going to remove MolePro. Queries to it timeout often, and adds wait time. And we may be querying it more with the record limit, parallel features. biothings/bte-server#49

Already adjusted main comment

[tokebe]

Related changes deployed to Prod as of 11/13