TL;DR

The first and foremost choice should be using a reproducible environment like docker. However, if you cannot do that for any reason, this might help you.

If you don’t care about your Python version, check the up-to-date section below. However, if you are specifically looking for Python 3.6.5, follow the rest.

In this project, one can set up a FastAPI application template with SocketIO and Celery that is compatible with Python 3.6.5 (Ubuntu 18.04 LTS). This setup is intended for air-gap servers that do not have access to internet. So the FastAPI static files are served from the host as well.

License

A template back-end with FastAPI [1], SocketIO [2] and Celery [3] for air-gap servers running python 3.6.5+.

Original author: Pedram Ashofteh Ardakani <[email protected]> Contributing author(s):

Copyright(c) 2022

The main structure is heavily inspired from Michael Yin’s [4] article “The Definitive Guide to Celery and FastAPI” [5]. However, I’ve integrated SocketIO and prepared this template for air-gap servers. It is worthy of note that the FastAPI docs are served in the /static directory as instructed by Sebastián Ramírez (aka tiangolo) [6].

[1] https://github.com/tiangolo/fastapi

[2] https://github.com/miguelgrinberg/python-socketio

[3] https://github.com/celery/celery

[4] https://testdriven.io/authors/yin/

[5] https://testdriven.io/courses/fastapi-celery/app-factory/

[6] https://fastapi.tiangolo.com/advanced/extending-openapi/#self-hosting-javascript-and-css-for-docs

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 3 or later as published by the Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; see the file COPYING.LIB. If not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

Prepare the environment

First, if you are specifically looking for python 3.6.5 download the requirements with the requirements.txt file on your local machine. This file contains the software versions that are compatible to python 3.6.5.

Otherwise, if you want to install everything on your own machine or just use the most-recent version of the software, follow the up-to-date version.

Python 3.6.5 specific software for offline use

$ mkdir to-upload
$ cd to-upload
$ python3 -m pip download -r requirements.txt

Up-to-date software for current version use

$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ python3 -m pip install pydantic fastapi python-socketio \
                                 uvicorn[standard] redis celery

Uploading to server

Now upload them to the server using (e.g. using scp or rsync):

$ rsync -rvuz --progress * USERNAME@SERVERADDRESS:/path/to/requirements

SSH to server and create a virtual environment if you need to:

$ ssh USERNAME@SERVERADDRESS
(server) $ cd /path/to/your/project
(server) $ python3 -m venv .venv

You can now activate the virtual environment and install all the requirements:

(server) $ source /path/to/your/project/.venv/bin/activate
(.venv)  $ cd /path/to/requirements
(.venv)  $ python3 -m pip install *

If you get errors here, read them carefully. Sometimes you need to download a package with another version. But if everything installed correctly, go to the next step.

Test the program

In developement

Just use the dev.sh script to start up and stop.sh to stop all the programs.

$ ./run.sh
$ ./stop.sh

In production

Upload your program to the server, activate the virtual environment, and run the program as is:

$ rsync -rvuz /path/to/your/program USERNAME@SERVERADDRESS:/path/to/your/project
$ ssh USERNAME@SERVERADDRESS
(server) $ cd /path/to/your/project
(server) $ ./run.sh

Look out for any errors. If everything goes smoothly, you’d be able to see an output like below and connect to the SocketIO and FastAPI routes.

INFO:     Started server process [10082]
INFO:     Waiting for application start up.
INFO:     Application start up complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

SocketIO

If you don’t already have a socketio client installed on your local machine, you can do so with the following command:

python3 -m pip install python-socketio[client]

Now you can follow the test:

import socketio

sio = socketio.Client()

# Enter SERVERADDRESS here. But if running locally:
sio.connect("127.0.0.1:8000")

sio.emit("echo", "Hello There!")

Now the uvicorn program should give you an output like this:

INFO:     Started server process [10082]
INFO:     Waiting for application start-up.
INFO:     Application start-up complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     172.18.1.1:45222 - "GET /socket.io/?transport=polling&EIO=4&t=1656757289.22189 HTTP/1.1" 200 OK
INFO:     ('172.18.1.1', 45230) - "WebSocket /socket.io/" [accepted]
on connect: jKStnNxPP-VqEqUxAAAB
Extended socket heard session id <jKStnNxPP-VqEqUxAAAB> say: 'Hello There!'

Please note that the first time SocketIO connects, it will try for long polling and then it will try to upgrade the connection to websocket. If it fails, it will let you know.

FastAPI

You can simply open up your browser and look up the server URLs. Here, the uvicorn is serving on my localhost:

$ curl http://127.0.0.1:8000
{"message":"hello world"}

Now, you can simply check the default swagger-ui docs generated with your browser:

$ firefox http://127.0.0.1:8000/docs

You should be able to see the documentation now. uvicorn should report a similar output:

INFO:     127.0.0.1:42358 - "GET /docs HTTP/1.1" 200 OK
INFO:     127.0.0.1:55888 - "GET /static/swagger-ui-bundle.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:42358 - "GET /static/swagger-ui.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:55888 - "GET /openapi.json HTTP/1.1" 200 OK

This means that the static files are served successfully.

Updating endpoints, etc.

Adding socketio events

You can simply append your socketio endpoints to the following file: /PATH/TO/APP/socketio_utils/extended.py

Adding FastAPI routes

You can simply append your FastAPI routes to the following file: /PATH/TO/APP/routes/__init__.py

Adding Celery tasks

You can simply append your Celery tasks to the following file: /PATH/TO/APP/tasks/__init__.py

Using as a service

Starting as service for the first time

Modify the contents of the uvicorn.service, celery.service, and project.env as needed. Then copy the .service files to /etc/systemd/system/ directory. Finally enable and start the services using:

sudo systemctl enable celery.service uvicorn.service=
sudo systemctl start celery.service uvicorn.service=

Modifying the services

Whenever you modify the .service files and run the following command for them to take effect before restarting:

sudo systemctl daemon-reload
sudo systemctl restart uvicorn.service celery.service

Checking status

You might want to check the service status or see the logs and outputs for yourself. For that, just use the following command:

sudo systemctl status uvicorn.service

and/or:

sudo systemctl status celery.service