Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breeze adding exec command #23052

Merged
merged 1 commit into from
Apr 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions BREEZE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,14 @@ Those are commands mostly used by contributors:
* Initialize local virtualenv with ``./scripts/tools/initialize_virtualenv.py`` command
* Run static checks with autocomplete support ``breeze static-checks`` command
* Run test specified with ``./breeze-legacy tests`` command
* Join running interactive shell with ``./breeze-legacy exec`` command
* Build CI docker image with ``breeze build-image`` command
* Cleanup breeze with ``breeze cleanup`` command
Bowrna marked this conversation as resolved.
Show resolved Hide resolved
* Run static checks with autocomplete support ``breeze static-checks`` command
* Run test specified with ``./breeze-legacy tests`` command

Additional management tasks:

* Join running interactive shell with ``breeze exec`` command
* Stop running interactive environment with ``breeze stop`` command
* Execute arbitrary docker-compose command with ``./breeze-legacy docker-compose`` command

Expand Down Expand Up @@ -580,8 +587,7 @@ capability of creating multiple virtual terminals and multiplex between them. Mo
found at `tmux GitHub wiki page <https://github.com/tmux/tmux/wiki>`_ . Tmux has several useful shortcuts
that allow you to split the terminals, open new tabs etc - it's pretty useful to learn it.

Here is the part of Breeze video which is relevant (note that it refers to the old ``./breeze-legacy``
command but it is very similar to current ``breeze`` command):
Here is the part of Breeze video which is relevant:

.. raw:: html

Expand All @@ -595,12 +601,11 @@ command but it is very similar to current ``breeze`` command):

Another way is to exec into Breeze terminal from the host's terminal. Often you can
have multiple terminals in the host (Linux/MacOS/WSL2 on Windows) and you can simply use those terminals
to enter the running container. It's as easy as launching ``./breeze-legacy exec`` while you already started the
to enter the running container. It's as easy as launching ``breeze exec`` while you already started the
Breeze environment. You will be dropped into bash and environment variables will be read in the same
way as when you enter the environment. You can do it multiple times and open as many terminals as you need.

Here is the part of Breeze video which is relevant (note that it refers to the old ``./breeze-legacy``
command and it is not yet available in the current ``breeze`` command):
Here is the part of Breeze video which is relevant:

.. raw:: html

Expand All @@ -612,6 +617,12 @@ command and it is not yet available in the current ``breeze`` command):
</div>


Those are all available flags of ``exec`` command:

.. image:: ./images/breeze/output-exec.svg
:width: 100%
:alt: Breeze exec

Additional tools
----------------

Expand Down
35 changes: 34 additions & 1 deletion dev/breeze/src/airflow_breeze/commands/developer_commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
get_available_packages,
)
from airflow_breeze.pre_commit_ids import PRE_COMMIT_LIST
from airflow_breeze.shell.enter_shell import enter_shell
from airflow_breeze.shell.enter_shell import enter_shell, find_airflow_container
from airflow_breeze.shell.shell_params import ShellParams
from airflow_breeze.utils.confirm import set_forced_answer
from airflow_breeze.utils.console import get_console
Expand All @@ -66,6 +66,7 @@
"commands": [
"shell",
"start-airflow",
"exec",
"stop",
"build-docs",
"static-checks",
Expand Down Expand Up @@ -144,6 +145,9 @@
],
},
],
"breeze exec": [
Bowrna marked this conversation as resolved.
Show resolved Hide resolved
{"name": "Drops in the interactive shell of active airflow container"},
],
"breeze stop": [
{
"name": "Stop flags",
Expand Down Expand Up @@ -478,3 +482,32 @@ def args_doc_builder(self) -> List[str]:
for single_filter in self.package_filter:
doc_args.extend(["--package-filter", single_filter])
return doc_args


@main.command(name='exec', help='Joins the interactive shell of running airflow container')
@option_verbose
@option_dry_run
@click.argument('exec_args', nargs=-1, type=click.UNPROCESSED)
def exec(verbose: bool, dry_run: bool, exec_args: Tuple):
container_running = find_airflow_container(verbose, dry_run)
if container_running:
cmd_to_run = [
"docker",
"exec",
"-it",
container_running,
"/opt/airflow/scripts/docker/entrypoint_exec.sh",
]
if exec_args:
cmd_to_run.extend(exec_args)
process = run_command(
cmd_to_run,
verbose=verbose,
dry_run=dry_run,
check=False,
no_output_dump_on_exception=False,
text=True,
)
if not process:
sys.exit(1)
sys.exit(process.returncode)
42 changes: 41 additions & 1 deletion dev/breeze/src/airflow_breeze/shell/enter_shell.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import subprocess
import sys
from pathlib import Path
from typing import Dict, Union
from typing import Dict, Optional, Union

from airflow_breeze import global_constants
from airflow_breeze.build_image.ci.build_ci_image import build_ci_image
Expand Down Expand Up @@ -137,3 +137,43 @@ def run_shell_with_build_image_checks(
if cmd_added is not None:
cmd.extend(['-c', cmd_added])
return run_command(cmd, verbose=verbose, dry_run=dry_run, env=env_variables, text=True)


def stop_exec_on_error(returncode: int):
get_console().print('\n[error]ERROR in finding the airflow docker-compose process id[/]\n')
sys.exit(returncode)


def find_airflow_container(verbose, dry_run) -> Optional[str]:
exec_shell_params = ShellParams(verbose=verbose, dry_run=dry_run)
check_docker_resources(verbose, exec_shell_params.airflow_image_name, dry_run)
exec_shell_params.print_badge_info()
env_variables = construct_env_variables_docker_compose_command(exec_shell_params)
cmd = ['docker-compose', 'ps', '--all', '--filter', 'status=running', 'airflow']
docker_compose_ps_command = run_command(
cmd,
verbose=verbose,
dry_run=dry_run,
text=True,
capture_output=True,
env=env_variables,
# print output if run in verbose mode to better diagnose problems.
no_output_dump_on_exception=not verbose,
)
if dry_run:
return "CONTAINER_ID"
if docker_compose_ps_command.returncode != 0:
stop_exec_on_error(docker_compose_ps_command.returncode)
return None

output = docker_compose_ps_command.stdout
container_info = output.strip().split('\n')
if container_info:
container_running = container_info[-1].split(' ')[0]
if container_running.startswith('-'):
# On docker-compose v1 we get '--------' as output here
stop_exec_on_error(docker_compose_ps_command.returncode)
return container_running
else:
stop_exec_on_error(1)
return None
3 changes: 2 additions & 1 deletion images/breeze/output-commands.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
133 changes: 133 additions & 0 deletions images/breeze/output-exec.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.