Skip to content

Commit

Permalink
12: Add basic CI with linters
Browse files Browse the repository at this point in the history
We want PRs to have an explicit indicator for linter errors in the branch/PR. It will allow us to see the state of the branch without explicitly running linters locally if we don't want it.

In the scope of this task we need to add a basic CI pipeline with linters and status indicator on GitHub.

Steps to do:

	- add `Dockerfile` for the application (we don't want to add `ENTRYPOINT` in it for now, since we suppose that it is better to have container entrypoints inside the `docker-compose` file)
	- add `docker-compose` file
	Compose file should have following services:
	- `forum123-build` (will be used to build application)
	- `forum123-mypy` (to run `mypy`)
	- `forum123-flake8` (to run `flake8`)
	- `forum123-pylint` (to run `pylint`)
	We want them to be separated on different services to be able to run build in one job, and then run all linters in a parallel way in three different jobs (parallel execution will be implemented sometimes later, now we need only to have different services in compose file).
	- setup CI pipeline using GitHub Actions

Also we want to put this configuration in a separate folder like `envs/dev` to indicate that this is only for development purposes. And when we will need to add some production infrastructure it will go into `envs/prod`.
Seems like using different folders is more convenient than having a bunch of Dockerfile's and docker-compose files with suffixes like `.dev` and `.prod`.

We decided to put mypy stubs in `requirements-dev.txt`, because we had a troubles with installing types on CI. In our case we had to run `mypy` twice - first time, to populate `mypy`'s cache and define what types are missing, and second time to install types and check for errors. The cause of this isssue is that github always run its jobs in a new clean containers, so each new `mypy` run will not have `mypy`'s cache from the previous run. Therefore we had to run `mypy` twice for every job to have type stubs installed. More about related problems with missing type stubs on CI from other people you can read here: python/mypy#10600

Worth to mention that now we're not going to make this pipeline optimized by performance. We are planning to add caching of intermediate docker layers later in the scope of another task. For now we just need this pipeline to work and nothing more.
  • Loading branch information
Alexey Tsivunin committed Jan 6, 2023
1 parent 61f072c commit d711eac
Show file tree
Hide file tree
Showing 5 changed files with 71 additions and 1 deletion.
24 changes: 24 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
on: push
jobs:
build:
name: build
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v3

- name: Build linters and application images
run: |
docker-compose -f envs/dev/docker-compose.yml build forum123-build
docker-compose -f envs/dev/docker-compose.yml build forum123-mypy
docker-compose -f envs/dev/docker-compose.yml build forum123-flake8
docker-compose -f envs/dev/docker-compose.yml build forum123-pylint
- name: Run mypy
run: docker-compose -f envs/dev/docker-compose.yml run forum123-mypy

- name: Run flake8
run: docker-compose -f envs/dev/docker-compose.yml run forum123-flake8

- name: Run pylint
run: docker-compose -f envs/dev/docker-compose.yml run forum123-pylint
20 changes: 20 additions & 0 deletions envs/dev/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
FROM python:3.10-slim

WORKDIR /forum123

# Setup and activate python3.10 virtual environment. It will allow us to avoid this warning from pip:
# WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system
# package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
RUN python3.10 -m venv .venv
ENV PATH="/forum123/.venv/bin:$PATH"

COPY requirements.txt requirements-dev.txt ./

RUN apt-get update \
&& pip install --upgrade pip \
&& pip install -r requirements.txt \
-r requirements-dev.txt

COPY . ./

ENV FLASK_APP=forum123
20 changes: 20 additions & 0 deletions envs/dev/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
version: "3"

services:

forum123-build: &forum123-build
build:
context: ../.. # path from the current file to the project root directory
dockerfile: envs/dev/Dockerfile # path from the project root directory to the Dockerfile

forum123-mypy:
<<: *forum123-build
entrypoint: mypy

forum123-flake8:
<<: *forum123-build
entrypoint: flake8

forum123-pylint:
<<: *forum123-build
entrypoint: pylint src
6 changes: 6 additions & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Development dependencies. Please keep this list in alphabetical order.
# Note that mypy type stubs listed in a separate section below.

darglint==1.8.1 # Checks whether a docstring's description matches the actual function/method implementation
dlint==0.12.0 # Tool for encouraging best coding practices and helping ensure Python code is secure.
Expand Down Expand Up @@ -28,3 +29,8 @@ flake8-type-checking==1.0.3 # Plugin for managing type-checking imports & forwa
flake8==3.9.2
mypy==0.991
pylint==2.15.9


# Mypy types. Please keep this list in alphabetical order.

types-cryptography==3.3.23.2
2 changes: 1 addition & 1 deletion src/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@

@bp.route("/")
@bp.route("/index")
def index() -> str: # pylint: disable=unused-variable
def index() -> int: # pylint: disable=unused-variable
"""Use this view function to check whether Flask is installed properly."""
return "Hello, wordl!"

0 comments on commit d711eac

Please sign in to comment.