This software is written to support the goals of Alliance of Genome Resources
These instructions will get you a copy of the project up and running on your local machine for development and testing. This includes both the default configuration and instructions on how to customize your environment. This configuration allows user to have flexible development environment.
- Prerequisites
- Installing
- Configuration
- Running the Development Enviroment
- Running the Production Enviroment
- API Usage
- Running the tests
- NodeJS
- Python 2.7
- Python Package Index (pip)
- Virtualenv
- Elasticsearch
Starting with Python 2.7.9, pip is included by default with the Python binary installers.
make sure the executables npm, node, pip, and virtualenv are accessible from /usr/local/bin (MAC, Linux)
if not create symbolic links as needed
Make sure /usr/local/bin is in your PATH (MAC, Linux)Create a virtualenv for isolating the python dependencies:
> mkdir -p ~/.virtualenvs/agr
> # The prototype currently requires Python2
> # Assuming virtualenv and python2 are in your PATH
> virtualenv -p python2 ~/.virtualenvs/agr- Webapp requires (NodeJS)
- API requires (python, pip, virtualenv, Elasticsearch)
- Indexer requires (python, pip, virtualenv, Elasticsearch)
Local ES hosted INDEX see elasticsearch setup for more info
To run a full install of all sub directories run the following:
> git clone https://github.com/alliance-genome/agr.git
> cd agr
agr> source ~/.virtualenvs/agr/bin/activate
(agr) agr> make -C webapp install
(agr) agr> make -C webapp build
(agr) agr> make -C api install
(agr) agr> make -C indexer install- API_URL A url pointing to the server that is hosting the /api endpoint for the frontend
- DEV_SERVER_UI_PORT Used to specify the local port that the WEBPACK server will be listening on
These parameters will default to host: "localhost" and port "2992". If you want to override them, you can do so through setting the environment variables. One such configuration a UI developer might be want to use is something along the lines of:
(agr) agr> cd webapp
(agr) agr/webapp> export API_URL=http://dev.alliancegenome.org
(agr) agr/webapp> export DEV_SERVER_UI_PORT=12345After running this set of commands the webpack-dev-server will be hosting the website at the URL http://localhost:12345 and will be pulling data from the API running on dev.alliancegenome.org.
The following variables can be used to customize your development environment:
- "PRODUCTION" Controls the start up of the flask server. If set to "false", debugging will be turned on and the server will crash upon first encountered error. If set to "true" the server runs the WSGIServer.
- "ES_AWS" controls whether or not SSL will be used on the connection to ES_HOST.
- "ES_HOST" points the flask server to the running Elastic Search instance.
- "ES_INDEX" indicates the index the flask server will be using to get its data.
- "API_PASSWORD" Not used at the moment, but will be the password controlling access to the writable API's
(agr) agr> cd api
(agr) agr/api> export PRODUCTION=true
(agr) agr/api> export ES_AWS=true
(agr) agr/api> export ES_HOST="search-es1-oyqxarxm2djn35dfodzniituhe.us-west-2.es.amazonaws.com"
(agr) agr/api> export ES_INDEX=es_username
(agr) agr/api> export API_PASSWORD="api_password"The defaults for these variables are set to:
- PRODUCTION = false
- ES_AWS = false
- ES_HOST = "http://localhost:9200"
- ES_INDEX = searchable_items_blue
- API_PASSWORD = "api_password"
- ES_AWS controls whether or not SSL will be used on the connection to ES_HOST.
- ES_HOST points the indexer to the running ElasticSearch instance.
- ES_INDEX indicates which index the indexer will be using to put documents into.
(agr) agr> cd indexer
(agr) agr/indexer> export ES_AWS=true
(agr) agr/indexer> export ES_HOST="search-es1-oyqxarxm2djn35dfodzniituhe.us-west-2.es.amazonaws.com"
(agr) agr/indexer> export ES_INDEX=es_usernameThe defaults for these params assume ElasticSearch is running on "localhost".
There is no configuration for ElasticSearch please see the elasticsearch setup for more info
All of the following steps are not necessary, if developing only one part of the system. The frontend (webapp) can be pointed to a different API server and the webpack dev server will run on its own without the API, indexer, or ES running.
Simularly, if only the API needs to be developed then one can start the API and set the ES_HOST variable to the location of an instance that already has data and there is no need to run the webapp, indexer, or ElasticSearch instance.
If developing the indexer, go through the local setup of the ElasticSearch and point the indexer to run against the ElasticSearch server running on localhost, without having to run the API or webapp.
In a seperate terminal window run the following commands:
(agr) agr> make -C webapp run
npm start
> [email protected] start /Users/oblod/git/agr/webapp
> webpack-dev-server --history-api-fallback --hot --inline
[HPM] Proxy created: /api -> http://localhost:5000
http://localhost:2992/
webpack result is served from /assets/
content is served from dist
404s will fallback to /index.html
The react/require-extension rule is deprecated. Please use the import/extensions rule from eslint-plugin-import instead.
The react/wrap-multilines rule is deprecated. Please use the react/jsx-wrap-multilines rule instead.Now the server is running on port 2992 and is redirecting "/api" with proxy to localhost:5000 which should be hosted by the flask server.
In a seperate terminal window run the following set of commands:
(agr) agr> make -C api run
PRODUCTION= API_PASSWORD='api_password' ES_HOST='127.0.0.1:9200' ES_AWS= ES_INDEX='searchable_items_blue' python src/server.py
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger pin code: 326-018-460The flask server is now running on localhost:5000, serving /api requests.
In a seperate terminal window run the following commands:
(agr) agr> make -C indexer index
cd src && ES_HOST='127.0.0.1:9200' ES_AWS= ES_INDEX='searchable_items_blue' python fetch_save_index.py
ES_HOST: 127.0.0.1:9200
ES_INDEX: searchable_items_blue
ES_AWS:
...
...
...
>After the indexer has run, the Elastic search instance at localhost:9200/searchable_items_blue will be loaded with data that is now available to the API.
For running a local ElasticSearch instance see the elasticsearch setup for more info.
In a production style running the application. Skip the section for running the webapp and use nginx to server the webapp/dict folder as the root of the site. The nginx configuration is as follows:
server {
listen 80;
root webapp/dist;
index index.html;
server_name hostname.alliancegenome.org;
error_page 404 =200 /index.html;
location /api {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://localhost:5000;
}
}Setup the hostname to whatever hostname you are wanting to use. Error page is set to allways return the index.html page and a 200 status. Change the /api url to point to the running flask server.
Once you have the API up and running, with it pointed at a valid ES instance that has all loaded data, you should be able to run the following command and get the same results.
> curl http://dev.alliancegenome.org/api/gene/MGI:98341
{
"category": "gene",
"crossReferences": [ { ... }],
"dataProvider": "MGI",
"dateProduced": "2017-02-10T08:53:14-05:00",
"description": null,
...
"geneLiteratureUrl": "http://www.informatics.jax.org/reference/marker/MGI:98341?typeFilter=Literature",
"geneSynopsis": null,
"geneSynopsisUrl": "https://en.wikipedia.org/wiki/SnRNP70",
"gene_biological_process": [ ... ],
"soTermName": "protein_coding_gene",
"species": "Mus musculus",
"symbol": "Snrnp70",
"taxonId": "10090"
}Plans to make the API documentation available at /api.
The first three examples of running tests, run tests on the code in the directory that is being tested. The "api tests", does integration tests across the system (webapp, api, elasticsearch)
(agr) agr> make -C webapp test
(agr) agr> make -C api test-py
(agr) agr> make -C indexer test
# If the full system is up and running then the following can be run
(agr) agr> make -C api tests