Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: brightsparklabs/appcli
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 1.3.4
Choose a base ref
...
head repository: brightsparklabs/appcli
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 1.3.5
Choose a head ref
Loading
4 changes: 3 additions & 1 deletion .github/workflows/build_and_publish_wheel.yml
Original file line number Diff line number Diff line change
@@ -27,4 +27,6 @@ jobs:
- name: Check python
run: make check
- name: Build and publish wheel to PYPI
run: make publish-wheel
run: |
export PYPI_TOKEN
make publish-wheel
32 changes: 29 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -5,10 +5,36 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

The changelog is applicable from version `1.0.0` onwards.

---

## [Unreleased]

### Added

### Fixed

---

## [1.3.5] - (21/01/2022)

### Added

- Enable custom commands to run `exec` commands on service containers via the orchestrator
- Allow tasks to be run in detached mode with flag `-d/--detach`.
- Renaming the launcher script to create only 1 hidden file: `.<timestamp>_<app_name>_<app_version>`.
- [#118](https://github.com/brightsparklabs/appcli/issues/118) Added `version` command to fetch version of app managed by appcli.
- [#144](https://github.com/brightsparklabs/appcli/issues/144) Added `--lines/-n` option to the `logs` commands for orchestrators. This is the `n` number of lines from the end to start the tail.
- [#147](https://github.com/brightsparklabs/appcli/issues/147) Remove ':' character from backup filenames, to allow tools like `tar` to work more easily with the unmodified filename.
- [#165](https://github.com/brightsparklabs/appcli/issues/165) Added ability to start/shutdown/restart multiple services at a time with the `service` command.

### Fixed

- Adjust logging header formatting misalignment
- Fixed issue where applications with non-shell-safe `app_name` weren't able to be installed or run.
- Fix Dockerfile issues identified by [Hadolint](https://github.com/hadolint/hadolint).
- Minor fix to README example python script.

---

## [1.3.4] - (14/05/2021)
@@ -17,7 +43,7 @@ The changelog is applicable from version `1.0.0` onwards.

- Stack settings file is no longer overwritten to the default when running `migrate` command.
- [#130](https://github.com/brightsparklabs/appcli/issues/130) Added back in `docker` binary to the appcli Docker image, so tasks can now be run again.
- [#115](https://github.com/brightsparklabs/appcli/issues/115) Fixed DockerSwarm orchestrator with the addition of the `docker` binary
- [#115](https://github.com/brightsparklabs/appcli/issues/115) Fixed DockerSwarm orchestrator with the addition of the `docker` binary.

---

@@ -37,7 +63,7 @@ The changelog is applicable from version `1.0.0` onwards.

### Fixed

- Fixed [#89](https://github.com/brightsparklabs/appcli/issues/89): `configure get` on a boolean returns `None` if setting is `false`
- Fixed [#89](https://github.com/brightsparklabs/appcli/issues/89): `configure get` on a boolean returns `None` if setting is `false`.

---

@@ -88,7 +114,7 @@ any python referring to the library will need to use full module path references
### Added

- The launcher now supports `NO_INTERACTIVE` environment variable to disable interactive mode.
- Expose the `APP_NAME` environment variable to the launcher container
- Expose the `APP_NAME` environment variable to the launcher container.

### Fixed

8 changes: 4 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -6,13 +6,13 @@
# www.brightsparklabs.com
##

FROM alpine AS docker-binary-download
FROM alpine:3.15.0 AS docker-binary-download

WORKDIR /tmp

# Download and extract the static docker binary
RUN \
wget https://download.docker.com/linux/static/stable/x86_64/docker-20.10.6.tgz \
wget -q https://download.docker.com/linux/static/stable/x86_64/docker-20.10.6.tgz \
&& tar xf docker-20.10.6.tgz

FROM python:3.8.2-slim-buster
@@ -32,8 +32,8 @@ RUN \
# prepare for docker install
&& apt-get update \
&& apt-get -y install --no-install-recommends \
git \
vim-tiny \
git=1:2.20.1-2+deb10u3 \
vim-tiny=2:8.1.0875-5 \
&& apt-get -y autoremove \
&& apt-get -y clean \
&& rm -rf /var/lib/apt/lists/*
108 changes: 98 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -28,6 +28,13 @@ The library exposes the following environment variables to the `docker-compose.y
`production` or `staging`. This allows multiple instances of the application to run on the same
Docker daemon. Defaults to `production`.

Note: the `APP_NAME` variable is derived from the `app_name` passed in to the `Configuration` object in the
main python entrypoint to the application. In order for the application to work, the `app_name` is forced to conform
with the shell variable name standard: `[a-zA-Z_][a-zA-Z_0-9]*`. Any characters that do not fit this regex will be
replaced with `_`. Once the replacement is done, the entire field is capitalised. See:
(https://unix.stackexchange.com/questions/428880/list-of-acceptable-initial-characters-for-a-bash-variable)
(https://linuxhint.com/bash-variable-name-rules-legal-illegal/)

The `docker-compose.yml` can be templated by renaming to `docker-compose.yml.j2`, and setting
variables within the `settings.yml` file as described in the Installation section.

@@ -80,7 +87,7 @@ python3 implicit namespaced packages.
seed_app_configuration_file=Path(BASE_DIR, 'resources/settings.yml'),
stack_configuration_file=Path(BASE_DIR, 'resources/stack-settings.yml'),
baseline_templates_dir=Path(BASE_DIR, 'resources/templates/baseline'),
configurable_templates_dir=Path(BASE_DIR, 'resource/templates/configurable'),
configurable_templates_dir=Path(BASE_DIR, 'resources/templates/configurable'),
orchestrator=DockerComposeOrchestrator(
docker_compose_file = Path('docker-compose.yml'),
docker_compose_override_directory = Path('docker-compose.override.d/'),
@@ -102,6 +109,80 @@ python3 implicit namespaced packages.
if __name__ == '__main__':
main()

#### Custom Commands

You can specify some custom top-level commands by adding click commands or command groups to the configuration object.
Assuming 'web' is the name of the service in the docker-compose.yml file which you wish to exec against, we can create
three custom commands in the following example:

- `myapp ls-root` which lists the contents of the root directory within the `web` service container and prints it out.
- `myapp ls-root-to-file` which lists the contents of the root directory within the `web` service container and dumps to file within the container.
- `myapp tee-file` which takes some text and `tee`s it into another file the `web` service container.

```python

def get_ls_root_command(orchestrator: DockerComposeOrchestrator):
@click.command(
help="List files in the root directory",
)
@click.pass_context
def ls_root(ctx: click.Context):

# Equivalent command within the container:
# `ls -alh`
cli_context: CliContext = ctx.obj
output: CompletedProcess = orchestrator.exec(cli_context, "web", ["ls", "-alh", "/"])
print(output.stdout.decode())

return ls_root

def get_tee_file_command(orchestrator: DockerComposeOrchestrator):
@click.command(
help="Tee some text into a file",
)
@click.pass_context
def tee_file(ctx: click.Context):

# Equivalent command within the container:
# `echo "Some data to tee into the custom file" | tee /ls-root.txt`
cli_context: CliContext = ctx.obj
output: CompletedProcess = orchestrator.exec(cli_context, "web", ["tee", "/my_custom_file.txt"], stdin_input="Some data to tee into the custom file")

return tee_file

def get_ls_root_to_file_command(orchestrator: DockerComposeOrchestrator):
@click.command(
help="List files in the root directory and tee to file",
)
@click.pass_context
def ls_root_to_file(ctx: click.Context):

# Equivalent command within the container:
# `ls -alh | tee /ls-root.txt`
cli_context: CliContext = ctx.obj
output: CompletedProcess = orchestrator.exec(cli_context, "web", ["ls", "-alh", "/"])
data = output.stdout.decode()
orchestrator.exec(cli_context, "web", ["tee", "/ls-root.txt"], stdin_input=data)

return ls_root_to_file

def main():
orchestrator = DockerComposeOrchestrator(Path("docker-compose.yml"))
configuration = Configuration(
app_name="appcli_nginx",
docker_image="thomas-anderson-bsl/appcli-nginx",
seed_app_configuration_file=Path(BASE_DIR, "resources/settings.yml"),
stack_configuration_file=Path(BASE_DIR, "resources/stack-settings.yml"),
baseline_templates_dir=Path(BASE_DIR, "resources/templates/baseline"),
configurable_templates_dir=Path(BASE_DIR, "resources/templates/configurable"),
orchestrator=orchestrator,
custom_commands={get_tee_file_command(orchestrator),get_ls_root_command(orchestrator),get_ls_root_to_file_command(orchestrator)}
)
cli = create_cli(configuration)
cli()

```

### Build configuration template directories

- Store any Jinja2 variable definitions you wish to use in your configuration
@@ -390,6 +471,7 @@ To be used in conjunction with your application `./myapp <command>` e.g. `./myap
| restore | Restore a backup of application data and configuration. |
| service | Lifecycle management commands for application services. |
| task | Commands for application tasks. |
| version | Fetches the version of the app being managed with appcli. |
| view-backups | View a list of locally-available backups. |

### Options
@@ -512,12 +594,12 @@ Runs application services. These are the long-running services which should only

usage: `./myapp service [OPTIONS] COMMAND [ARGS]`

| Command | Description |
| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| logs | Prints logs from all services. |
| shutdown | Shuts down the system. If a service name is provided, shuts down the single service only. |
| start | Starts the system. If a service name is provided, starts the single service only. |
| restart | Restarts service(s) (`shutdown` followed by `start`). Optionally run a `configure apply` during the restart with the `--apply` flag. If a service name is provided, restarts the single service only. |
| Command | Description |
| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| logs | Prints logs from all services. |
| shutdown | Shuts down the system. If one or more service names are provided, shuts down the specified service(s) only. |
| start | Starts the system. If one or more service names are provided, starts the specified service(s) only. |
| restart | Restarts service(s) (`shutdown` followed by `start`). Optionally run a `configure apply` during the restart with the `--apply` flag. If one or more service names are provided, restarts the specified service(s) only. |

| Option | Description |
| ------ | ------------------------------- |
@@ -529,14 +611,20 @@ Runs application tasks. These are short-lived services which should exit when th

usage: `./myapp task [OPTIONS] COMMAND [ARGS]`

| Command | Description |
| ------- | ---------------------------------- |
| run | Runs a specified application task. |
| Command | Description |
| ------- | -------------------------------------------------------------------------------------------- |
| run | Runs a specified application task. Optionally run in the background with `-d/--detach` flag. |

| Option | Description |
| ------ | ------------------------------- |
| --help | Show the help message and exit. |

#### Command: `version`

Fetches the version of the app being managed with appcli.

usage: `./myapp version`

#### Command: `view-backups`

View a list of all backups in the configured backup folder.
14 changes: 11 additions & 3 deletions appcli/backup_manager/backup_manager.py
Original file line number Diff line number Diff line change
@@ -291,10 +291,18 @@ def __create_backup_filename(self, app_name: str, backup_name: str) -> str:
Returns:
The formatted .tgz filename.
"""
now: datetime = datetime.datetime.now(datetime.timezone.utc).replace(
microsecond=0
# datetime's ISO format includes the ':' separator for the `hours:minutes:seconds`.
# Since we're using this format in the filename of the backup, the backup filename
# will include the ':' character.
# Tools like `tar` (by default) expects files with ':' in the name to be a remote
# resouces. To avoid this issue, we remove all ':'.
now: str = (
datetime.datetime.now(datetime.timezone.utc)
.replace(microsecond=0)
.isoformat()
.replace(":", "")
)
return f"{app_name.upper()}_{backup_name.upper()}_{now.isoformat()}.tgz"
return f"{app_name.upper()}_{backup_name.upper()}_{now}.tgz"

def __rolling_backup_deletion(self, backup_dir: Path):
"""Delete old backups, will only keep the most recent backups.
2 changes: 2 additions & 0 deletions appcli/cli_builder.py
Original file line number Diff line number Diff line change
@@ -30,6 +30,7 @@
from appcli.commands.migrate_cli import MigrateCli
from appcli.commands.service_cli import ServiceCli
from appcli.commands.task_cli import TaskCli
from appcli.commands.version_cli import VersionCli
from appcli.functions import error_and_exit, extract_valid_environment_variable_names
from appcli.logger import enable_debug_logging, logger
from appcli.models.cli_context import CliContext
@@ -72,6 +73,7 @@ def create_cli(configuration: Configuration, desired_environment: Dict[str, str]
ServiceCli,
TaskCli,
BackupManagerCli,
VersionCli,
):
commands = cli_class(configuration).commands
default_commands.update(**commands)
Loading