Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-8605] Added docker compose #8689

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions IMAGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -416,3 +416,22 @@ signals). This entrypoint works as follows:

* If first argument is equal to "bash" - you are dropped in bash shell.
* If there are any arguments they are passed to "airflow" command

Docker Compose
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this documentation to docs/*.rst? Otherwise this documentation will not be available to end users.

--------------

Docker Compose can be used to run Airflow installation. The provided Compose (docker-compose.yml) defines
the necessary services to run Airflow. It also starts an ephemeral container to initialize the database
and creates a user.

This starts the Compose.

.. code-block::

docker-compose -f docker-compose.yml up -d

This stops the Compose.

.. code-block::

docker-compose -f docker-compose.yml stop
78 changes: 78 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
version: "3.7"
x-airflow-environment: &airflow-environment
AIRFLOW__CORE__EXECUTOR: CeleryExecutor
AIRFLOW__WEBSERVER__RBAC: "True"
AIRFLOW__CORE__LOAD_EXAMPLES: "False"
AIRFLOW__CELERY__BROKER_URL: "redis://:@redis:6379/0"
AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres:5432/airflow

services:
postgres:
image: postgres:11.5
environment:
POSTGRES_USER: airflow
POSTGRES_DB: airflow
POSTGRES_PASSWORD: airflow
redis:
image: redis:5
environment:
REDIS_HOST: redis
REDIS_PORT: 6379
ports:
- 6379:6379
init:
image: apache/airflow:1.10.10
environment:
<<: *airflow-environment
depends_on:
- redis
- postgres
volumes:
- ./dags:/opt/airflow/dags
entrypoint: /bin/bash
command: >
-c "airflow list_users || (airflow initdb
&& airflow create_user --role Admin --username airflow --password airflow -e [email protected] -f airflow -l airflow)"
restart: on-failure
webserver:
image: apache/airflow:1.10.10
ports:
- 8080:8080
environment:
<<: *airflow-environment
depends_on:
- init
volumes:
- ./dags:/opt/airflow/dags
command: "webserver"
restart: always
flower:
image: apache/airflow:1.10.10
ports:
- 5555:5555
environment:
<<: *airflow-environment
depends_on:
- redis
command: flower
restart: always
scheduler:
image: apache/airflow:1.10.10
environment:
<<: *airflow-environment
depends_on:
- webserver
volumes:
- ./dags:/opt/airflow/dags
command: scheduler
restart: always
worker:
image: apache/airflow:1.10.10
environment:
<<: *airflow-environment
depends_on:
- scheduler
volumes:
- ./dags:/opt/airflow/dags
command: worker
restart: always