Skip to content

Commit

Permalink
Restructured navigation and revised text to enhance usability
Browse files Browse the repository at this point in the history
  • Loading branch information
RichardBruskiewich committed Sep 11, 2024
1 parent c8cfa8f commit aa0511b
Show file tree
Hide file tree
Showing 10 changed files with 103 additions and 110 deletions.
4 changes: 1 addition & 3 deletions docs/architecture/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
![image](../img/translator-banner.jpg)

# Translator Architecture

The Translator system is comprised of five main components, shown in the diagram below
The Translator platform consists of five main components, shown in the diagram below
([Fecho et al. 2022](../index.md#references)). Knowledge Providers (KPs) contribute domain-specific, high-value
information abstracted from one or more underlying ‘knowledge sources’. Autonomous Relay Agents (ARAs) build upon the
knowledge contributed by KPs by way of reasoning and inference across KPs. The Autonomous Relay System (ARS) functions
Expand Down
8 changes: 1 addition & 7 deletions docs/architecture/ui.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
# Translator User Interface

The Translator User Interface (UI) is under development as a Web interface to the Translator system that receives scored results and evidence, confidence, and provenance from the ARS and exposes results to users. This section doesn't cover the use of the UI, but rather, technical details facilitating its further development and maintenance.

* Andy Crouse _et al_ to elaborate.

## Tutorials

* [Translator User Interface tutorials](../development-guide/tutorials/index.md)
The primary Translator User Interface (UI) is under development as a [web portal](https://ui.transltr.io/) to the Translator system that receives scored results and evidence, confidence, and provenance from the ARS and exposes results to users. This section doesn't cover the use of the UI, but rather, technical details facilitating its further development and maintenance.
36 changes: 16 additions & 20 deletions docs/development-guide/HelloTranslator.ipynb
Original file line number Diff line number Diff line change
@@ -1,19 +1,10 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "spatial-chrome",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import requests\n",
"import json"
]
"metadata": {},
"cell_type": "markdown",
"source": "Translator data is accessed through web services which comply with the syntactic standards of the [Translator Reasoner Application Interface (\"TRAPI\")](https://github.com/NCATSTranslator/ReasonerAPI) and semantic standards of the [Biolink Model](https://github.com/biolink/biolink-model). One primary TRAPI compliant service available to query Translator is the Autonomous Relay System (\"ARS\").",
"id": "1f63a930adb212ce"
},
{
"cell_type": "markdown",
Expand All @@ -24,27 +15,30 @@
}
},
"source": [
"TRAPI Documentation: https://github.com/NCATSTranslator/ReasonerAPI\n",
"\n",
"Most TRAPI documents contain a `message` key. Within that `message` are a `query_graph` denoting the user query,\n",
"a `knowledge_graph` consisting of the union of all nodes and edges that match the `query_graph` pattern, and a list of `results` that bind `query_graph` elements to `knowledge_graph` elements.\n",
"\n",
"The following message contains only a `query_graph`. This query graph consists of 3 nodes connected together in a line. Two of the nodes (`n00` and `n02`) have specified identifiers, while the middle node of the line does not. Rather the middle node has a list of `categories` that are acceptable.\n",
"Most TRAPI documents contain a `message` key. Within that `message` are a `query_graph` denoting the user query, a `knowledge_graph` consisting of the union of all nodes and edges that match the `query_graph` pattern, and a list of `results` that bind `query_graph` elements to `knowledge_graph` elements.\n",
"\n",
"The following message contains only a `query_graph`. This query graph consists of 3 nodes connected together in a line. Two of the nodes (`n00` and `n02`) have specified identifiers, while the middle node of the line does not. Rather the middle node has a list of Biolink Model `categories` that are acceptable. The relationship between the nodes is specified using a valid Biolink Model `predicate`.\n",
"\n",
"This query asks \"Find me a Biological Process or Activity, or a Gene, or a Pathway that is related to both `PUBCHEM.COMPOUND:644073` (Buprenorphine) and `HP:0000217` (Drymouth)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "pressing-genesis",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"jupyter": {
"is_executing": true
}
},
"outputs": [],
"source": [
"import requests\n",
"import json\n",
"\n",
"query={\n",
" \"message\": {\n",
" \"query_graph\": {\n",
Expand Down Expand Up @@ -76,7 +70,9 @@
" }\n",
" }\n",
" }\n"
]
],
"outputs": [],
"execution_count": null
},
{
"cell_type": "markdown",
Expand Down
43 changes: 17 additions & 26 deletions docs/development-guide/index.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,31 @@
![image](../img/translator-banner.jpg)

# Guide for Developers

This section of the Technical Documentation provides several resources for developers designing, implementing and maintaining Translator software.
This section of the Technical Documentation provides several resources for developers designing, implementing, testing and maintaining Translator software.

- [Quick Start](quickstart.md)
- **Graph Modeling Paradigms:**
- [Core knowledge graph principles](../architecture/biolink/knowledge_graphs.md)
- [Key Biolink Model components](https://biolink.github.io/biolink-model/guidelines/understanding-the-model.html)
- [Getting started with Biolink](https://biolink.github.io/biolink-model/guidelines/using-the-modeling-language.html)
- [Working with the model (tutorial)](https://biolink.github.io/biolink-model/guidelines/working-with-the-model.html)
- [Developer Cookbook](cookbook.md)
- [Tutorials](tutorials/index.md)
- [Testing Resources](../architecture/sri/testing/index.md)
- [Frequently Asked Questions (FAQ)](../faq.md)
## Overview of Primary Translator standards
- [Introduction to Knowledge Graphs and the Biolink Model](../architecture/biolink/knowledge_graphs.md)
- [Translator networking protocol: the Translator Reasoner Application Programming Interface (TRAPI)](../architecture/sri/trapi.md)
- [Workflows](../architecture/workflows.md)

The actual deployment of the system is further documented in the [Deployment Guide](../deployment-guide/index.md).
## Coding against Translator
- [“Hello Translator” Jupyter Notebook](https://github.com/NCATSTranslator/TranslatorTechnicalDocumentation/blob/master/docs/development-guide/HelloTranslator.ipynb) providing a quick (Python) programmatic introduction to Translator interfaces and knowledge graph data.
- [Tutorials](tutorials/index.md)
- [Guidance for Testing](../architecture/sri/testing/index.md)

An overview of the Translator Architecture is provided in the [Architecture overview page](index.md).
Briefly, the system is composed of the following components:
An overview of the Translator platform is provided in the [Architecture overview page](index.md). Briefly, the system is composed of the following components:

1. [User Interface (UI)](../architecture/ui.md)
2. [Autonomous Relay System (ARS)](../architecture/ars_usage.md)
3. [Workflow Runner](../architecture/workflows.md)
4. [Translator Reasoner Application Programming Interface (TRAPI)](../architecture/sri/trapi.md)
5. [Translator SmartAPI Registry](../architecture/registry.md)
6. [Autonomous Relay Agents (ARAs)](../architecture/ara/index.md)
7. [Knowledge Providers (KPs)](../architecture/kp/index.md)
3. [Autonomous Relay Agents (ARAs)](../architecture/ara/index.md)
4. [Knowledge Providers (KPs)](../architecture/kp/index.md)
5. [Workflow Runner](../architecture/workflows.md)
6. [Translator SmartAPI Registry](../architecture/registry.md)

Tying together the above components are the activities, standards and service components of the Translator
Tying together the above components are the activities, aforementioed primary standards and service components of the Translator
[Standards and Reference Implementation ("SRI")](../architecture/sri/index.md) project team.

Additional architecture details are stored in the
[Translator Architecture repository](https://github.com/NCATSTranslator/TranslatorArchitecture).

## Joining the Translator Community as a Developer [draft]

* What are the mandatory technical requirements of having your stuff in Translator (Minimal standards; QA)
* How do keep quality assurance / gatekeeping in place; rubber-stamping; governance; vetting process for new internal KP's etc. - Governance issues?
## Getting Involved
- [Consortium Development Guidelines](tutorials/consortium_development_guidelines.md)
2 changes: 1 addition & 1 deletion docs/development-guide/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Quick Start

A [“Hello Translator” Jupyter Notebook](https://github.com/NCATSTranslator/TranslatorTechnicalDocumentation/blob/master/docs/development-guide/HelloTranslator.ipynb) is available to help orient newcomers to software development against the Translator platform.
A [“Hello Translator” Jupyter Notebook](https://github.com/NCATSTranslator/TranslatorTechnicalDocumentation/blob/master/docs/development-guide/HelloTranslator.ipynb) is a quick (Python) programmatic introduction to Translator interfaces and knowledge graph data.
21 changes: 21 additions & 0 deletions docs/development-guide/tutorials/component_builders_roadmap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# A Translator Component Builder's Road Map

The Phase 2 - Development projects of Translator created a cluster of KP and ARA components, queried by the ARS. Knowledge coverage was constrained to a priority set of knowledge sources highlighted by Phase 2 teams, but obviously, don't completely reflect every knowledge sources nor knowledge graph processing strategy imaginable. The need to integrate such novel resources into Translator suggest the need to develop and integrate novel KP, ARA or utility components in the platform.

In addition, future development of Translator (e.g. Phase 3 - Performance, starting _circa_ December 2024) will likely draw in new teams and their developers, interested in contributing to the platform, hence, needing a clear starting point to get up to speed on component development.

Even with Phase 2 project teams, already familiar with the platform, the maintenance of Translator components will sometimes run up against the challenge of complexity of the Translator platform thus, highlighting an ongoing need to refresh knowledge about platform standards, available software and best practices relating to those components.

The purpose of this document is to provide a concrete one-stop, step-by-step road map about component design and implementation which serves all the above development needs.

# Resources

1. Turnkey Options for Knowledge Provider creation:
2. [BioThings Service Provider](../../architecture/kp/service-provider.md)
3. [PLOVER](https://github.com/RTXteam/PloverDB)
[Plater](https://github.com/TranslatorSRI/Plater)
5. [Biolink Model](https://biolink.github.io/biolink-model/working-with-the-model/)
6. [Implementing TRAPI](https://github.com/NCATSTranslator/ReasonerAPI/tree/master/ImplementationGuidance)
7. [Specifying Workflows](workflows.md)
8. [Deploying a Translator Component](../../deployment-guide/index.md)
9. [Registering a TRAPI service](../../architecture/registry.md#adding-an-api-to-the-translator-smartapi-registry)
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Developer Cookbook (Richard, Tim, Chris B)
# Consortium Development Guidelines

1. Project Management
1. Project Management in Translator (Chris B)
1. Project Management in Translator
1. Project milestones
2. Github in Translator
1. Setting up repositories
Expand All @@ -10,21 +10,21 @@
3. Access Policies
2. Forking code
3. Issue tracking
4. How to submit a PR - Richard
4. How to submit a project GitHub Pull Request (PR)
5. Publishing releases
6. Publishing modules (pypi)
2. Coding process - Tim, Chris B
1. Test Driven development
2. Documentation Requirements
1. Licensing
3. Describe how to test, deploy & version artifacts - Tim, Chris B, Mark W.
3. Describe how to test, deploy & version artifacts
1. Versioning artifacts
2. Component registration: Translator Smart API Registry
1. SmartAPI translator extension metadata
3. Deployment environments:
1. x-maturity: dev, prod, etc.
4. Continuous Integration Dev Ops
4. Continuous Integration
5. Packaging
1. Docker
6. Deployment Process
6. Deployment Process (DevOps)
1. ITRB
21 changes: 4 additions & 17 deletions docs/development-guide/tutorials/index.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,6 @@
![image](../../img/translator-banner.jpg)

# Tutorials

1. [Biolink Model](https://biolink.github.io/biolink-model/working-with-the-model/)
2. [Implementing TRAPI](https://github.com/NCATSTranslator/ReasonerAPI/tree/master/ImplementationGuidance)
3. [Registering a TRAPI service](../../architecture/registry.md#adding-an-api-to-the-translator-smartapi-registry)
4. [Workflows](workflows.md)


# Purpose of this page (work-in-progress)
* Brainstorm outline all the specific necessary technical and sociological steps required to "plug into" the Translator ecosystem?
* Add link outs to videos (existing?) for various stuff

## Additional Tutorial topics for composition?

1. KP-starter kit
3. Standing Up a KP/ARA
5. Workflow runner (video)
1. [What are knowledge graphs, why do we use them and how do we build them?](../../architecture/biolink/knowledge_graphs.md)
2. So you are a Data Scientist wanting to directly play with Translator knowledge graphs? You can try out the available ["Hello Translator" Jupyter Notebook](../HelloTranslator.ipynb) or learn more about [how to access the Autonomous Relay System ("ARS")](https://docs.google.com/document/d/1_a4gE_lY-2oZTrdFMtaZ_pxqNgd-x_1ZYI7hRGfFjng/edit?pli=1#heading=h.pp72dzm2iy0h)
3. So you want to build (or maintain/extend) a new (or existing) Translator Component? [Where do you start and what do you need to know?](component_builders_roadmap.md)
4. How do I directly engage with the [Consortium Development Process](consortium_development_guidelines.md)?
14 changes: 10 additions & 4 deletions docs/faq.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
# What I want to know is...

### What are knowledge graphs, why do we use them and how do we build them?
Check out our [introduction to knowledge graphs](architecture/biolink/knowledge_graphs.md)?

### How can I start playing with Translator?
Translator is generally accessible to researchers via the [Translator Web portal](https://ui.transltr.io/). Python-savvy data scientists can try out the ["Hello Translator" Jupyter Notebook](development-guide/HelloTranslator.ipynb).

### How can I develop software to use and/or contribute to Translator?
We composed a [set of tutorials](development-guide/tutorials/index.md) for you.

### How do I find an identifier for a biomedical concept/entity?
CURIE's (Compact Uniform Resource Identifiers) are used in TRAPI messages to identify particular biomedical objects and concepts.
Here are two ways to convert from natural language to a CURIE:

1. The [SRI Name Resolver](https://name-resolution-sri.renci.org/docs), specifically the `/lookup` endpoint, can take natural language and find a CURIE associated with it.
2. The [ARAX UI](https://arax.transltr.io/) has a `Synonyms` feature on the left bar under the heading `Tools`. In it, you can type a term (with autocomplete) and find equivalent CURIES. This approach uses both the SRI Name Resolver and additional information derived from RTX-KG2.


### How do I compute the Normalized Google Distance (NGD) between two concepts
### How do I compute the Normalized Google Distance (NGD) between two concepts?
The endpoint [https://arax.transltr.io/api/arax/v1.4/ui/#/PubmedMeshNgd/pubmed_mesh_ngd](https://araxtransltr.io/api/arax/v1.4/ui/#/PubmedMeshNgd/pubmed_mesh_ngd) takes two natural language inputs (terms occurring in UMLS) and returns the NGD between them.


### How can I quickly view a TRAPI response?
The ARAX UI [https://arax.transltr.io/](https://arax.transltr.io/) can take a TRAPI response <img src="https://github.com/user-attachments/assets/780025a6-9907-4802-87ba-238896c002a5" width="150"> and render the results. This is mainly helpful for debugging and development use. You can also enter a PK in the ID field <img src="https://github.com/user-attachments/assets/c3cdda80-9790-4324-99a8-057bb9a84202" width="150">.

52 changes: 26 additions & 26 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,33 +11,33 @@ nav:

- Development Guide:
- Overview: development-guide/index.md
- Knowledge Graph Principles: architecture/biolink/knowledge_graphs.md
- Translator Architecture: architecture/index.md
- Quick Start: development-guide/quickstart.md
- User Interface: architecture/ui.md
- Autonomous Relay System: architecture/ars_usage.md
- Workflow Runner: architecture/workflows.md
- Translator Registry: architecture/registry.md
- Autonomous Relay Agents:
- Overview: architecture/ara/index.md
- ARAX: architecture/ara/arax.md
- BTE: architecture/ara/bte.md
- Knowledge Providers:
- Overview: architecture/kp/index.md
- RTX-KG2: architecture/kp/rtx-kg2.md
- Service Provider: architecture/kp/service-provider.md
- Standards and Reference Implementation:
- Overview: architecture/sri/index.md
- Biolink Model: "https://biolink.github.io/biolink-model/"
- Translator Reasoner API (TRAPI): architecture/sri/trapi.md
- Evidence, Provenance & Confidence Metadata: architecture/sri/epc.md
- Testing:
- Overview: architecture/sri/testing/index.md
- SRI Testing: architecture/sri/testing/sri_testing.md
- Tutorials:
- Overview: development-guide/tutorials/index.md
- Workflow Tutorial: development-guide/tutorials/workflows.md
- Cookbook: development-guide/cookbook.md
- Tutorials: development-guide/tutorials/index.md
- Knowledge Graph Principles: architecture/biolink/knowledge_graphs.md
- Translator Architecture:
- Overview: architecture/index.md
- User Interface: architecture/ui.md
- Autonomous Relay System: architecture/ars_usage.md
- Workflow Runner: architecture/workflows.md
- Translator Registry: architecture/registry.md
- Autonomous Relay Agents:
- Overview: architecture/ara/index.md
- ARAX: architecture/ara/arax.md
- BTE: architecture/ara/bte.md
- Knowledge Providers:
- Overview: architecture/kp/index.md
- RTX-KG2: architecture/kp/rtx-kg2.md
- Service Provider: architecture/kp/service-provider.md
- Standards and Reference Implementation:
- Overview: architecture/sri/index.md
- Biolink Model: "https://biolink.github.io/biolink-model/"
- Translator Reasoner API (TRAPI): architecture/sri/trapi.md
- Evidence, Provenance & Confidence Metadata: architecture/sri/epc.md
- Testing:
- Overview: architecture/sri/testing/index.md
- SRI Testing: architecture/sri/testing/sri_testing.md

- Consortium Development Guidelines: development-guide/tutorials/consortium_development_guidelines.md
- Deployment Guide:
- Introduction: deployment-guide/index.md
- Continuous Integration for Translator: deployment-guide/ci.md
Expand Down

0 comments on commit aa0511b

Please sign in to comment.