api

Official API for the NIMBUS Voice Assistant accessible via HTTP REST protocol.

🚧 This API is still in-development, so expect the endpoints to be constantly changing until finalized.

GitHub Actions Status

Documentation

TODO: INSERT LINK TO API DOCUMENTATION
TODO: USE http://readthedocs.org to host the docs live on the internet
TODO: CREATE API DOCS 😅

Dev Environment Setup

Prerequisites

Python 3.6.9+ (we're running 3.8 on gce)
pip 9.0.1+ (pip 20.1.1 is whats on gce as of this writing, but whatever version you have with your python install is probably fine)
git (can you run git --version?) (we're using 2.25.1 on gce, but any version will almost certainly work)
Access to database credentials/api keys (contact us)

If you're running Windows, we recommend (it is not required) setting up the Windows subsystem for Linux.

Database Configuration

**There are THREE ways to get your database config setup. If you're a member of CSAI, there's a script to generate a config file - contact a club member to get access to it. **

Create a file called config.json that should include at least the following details of a MySQL database:

{
    ...
    "mysql": {
        "host": "HOSTNAME",
        "port": "PORT e.g. 3306",
        "user": "USERNAME",
        "password": "PASSWORD",
        "database": "DATABASE",
        ...
    }
    ...
}

You can also use config_SAMPLE.json as a reference (easier!)

Contact anyone on the Data Team to get connection details for the Nimbus database

Install pip packages

Note: If you don't have pip installed for python2, you may have to call pip instead of pip3)

pip3 install -r requirements.txt

Download spaCy model

There are two spaCy models usable for nimbus predictions. These are (from spaCy's docs) General-purpose pretrained models to predict named entities, part-of-speech tags and syntactic dependencies. Can be used out-of-the-box and fine-tuned on more specific data. We use these to classify a question someone asks Nimbus, so we can retrieve an answer from our database.See here for an idea of the difference between lg and sm.

python3 -m spacy download en_core_web_sm

or

python3 -m spacy download en_core_web_lg

Get API keys for terminal instance

git clone https://github.com/calpoly-csai/nimbus-config-secrets.git

cd nimbus-config-secrets 

source .export_env_vars

You should now have everything you need to run the API server.

Run the API server

python3 flask_api.py

Opens localhost:PORT
PORT is changed in gunicorn_config.py

Run in the background

python3 flask_api.py&

Run your first API request

curl --location --request POST 'localhost:8080/ask' --header 'Content-Type: application/json' --data-raw '{ "question": "What are the prereqs for CSC 480?" }'

Sends Query to Database through localhost:8080/ask

Run the tests

pytest

(you may have to run the following if you have python 2.7 installed as well):

python3 -m pytest

Python PEP8 Style Standards

Run the format script to automatically make our code look nice

./format.sh

Sometimes the format script is not enough, so run lint to manually style our code

./lint.sh

Some common issues and solutions

`/bin/bash^M: bad interpreter: No such file or directory`:

Note: This issue can occur (for me, at least) in files other than just format.sh. These instructions should fix this problem in any file.

snekiam@P1:api$ ./format.sh
-bash: ./format.sh: /bin/bash^M: bad interpreter: No such file or directory

This can happen when you're running bash on windows sometimes. Linux doesn't recognize carriage returns (^M) the same way windows does. You may have this error in more than one file, but it can be fixed by running dos2unix like this:

snekiam@P1:api$ dos2unix format.sh
dos2unix: converting file format.sh to Unix format...
snekiam@P1:api$ ./format.sh
(output of format.sh, which can be long so I haven't included it here)

Deployment

What we use

A Linux server (e.g. Ubuntu 20.04 LTS) with open firewall at tcp:5000 for Flask, tcp:80 for http and tcp:443 for https and tcp:22 for ssh and tcp:3306 for mysql

See this documentation of the database deployment process

We're using docker to deploy - at some point, it'll be easy for you to run the Nimbus API in a local docker container, but at the moment it requires some things (like SSH keys and SSL private keys) which shouldn't be required for a development environment.