Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Makefile to automate common command-line tasks #436

Draft
wants to merge 9 commits into
base: dev
Choose a base branch
from
Draft

Conversation

Raruto
Copy link
Contributor

@Raruto Raruto commented Jan 3, 2023

List of changes:

Added files:

  • Makefile *
  • Makefile.assets.mk
  • Makefile.tasks.mk
  • Makefile.semver.mk (not included in VCS, remotely downloaded) *
  • Makefile.venv.mk (not included in VCS, remotely downloaded) *

* additional implementation notes: g3w-suite/makefiles

Deprecated files:

  • ci_scripts/build_suite.sh replaced by: make g3w-admin/core/static/bower_components

  • ci_scripts/run_env_development.sh replaced by: make run-docker

  • ci_scripts/setup_suite.sh replaced by: make setup-suite

  • pavement.py replaced by: make install and make runserver

  • run_docker_tests.sh replaced by: make tests

Refer to the Makefile.assets.mk and Makefile.tasks.mk files for more information about these deprecated files

TODO:

  • find out how to remove gulpfile.js dependency (ref: make build-assets)
  • figure out what happened to the following file: ci_scripts/Dockerfile.local-develpment-ubuntu-2004.dockerfile (ref: make build-docker and make run-docker)
  • update / remove ci_scripts/README.md
  • add / update contributing section within README.md
  • do some additional in-depth testing on all the docker related scripts (eg. make tests)
  • update .github/workflows/test_runner.yml file in order to use make tasks
  • write a make killserver task in order to replace the paver kill task (from pavement.py) or deprecate the background parameter inside the make runserver [ foreground = "set any value to disable background" ]
  • do some in-depth tests on all tasks related to the creation and use of the virtual environment (ref. Makefile.venv.mk)
  • make use of virtual enviroment binaries instead of the global ones within Makefile.tasks.mk (eg. adding export PATH := $(VENV_BIN):$(PATH) before declaring PYTHON and PIP variables ? ref: httpie/Makefile#L13-L22)

Closes: #431

Related to: #419, #434, #380, #373

## Files added:

- add new file `Makefile`
- add new file `Makefile.assets.mk`
- add new file `Makefile.server.mk`
- add remote file [`Makefile.semver.mk`](https://github.com/g3w-suite/makefiles/Makefile.semver.mk)
- add remote file [`Makefile.venv.mk`](https://github.com/g3w-suite/makefiles/Makefile.venv.mk)

## Deprecated files

- `ci_scripts/build_suite.sh` (ref: `make g3w-admin/core/static/bower_components`)
- `ci_scripts/run_env_development.sh` (ref: `make run-docker`)
- `ci_scripts/setup_suite.sh` (ref: `make setup-suite`)
- `pavement.py` (ref. `make install` and `make runserver`)
- `run_docker_tests.sh` (ref: `make tests`)

## TODO:

- find out how to remove `gulpfile.js` dependency (ref: `make build-assets`)
- figure out what happened to the following file: `ci_scripts/Dockerfile.local-develpment-ubuntu-2004.dockerfile` (ref: `make build-docker` and `make run-docker`)
- update / remove `ci_scripts/README.md`
- add / update contributing section within `README.md`
- do some additional in-depth testing on all the docker related scripts (eg. `make tests`)
@wlorenzetti wlorenzetti added the refactoring Anything which could result in a API change label Jan 4, 2023
@Raruto Raruto marked this pull request as draft January 4, 2023 08:06
@Raruto Raruto added this to the v3.6 milestone Jan 4, 2023
@Raruto Raruto added the feature New feature or request label Jan 4, 2023
make tests \
q=${{ matrix.qgis_version }} \
mode=test \
DOCKER_COMPOSE="docker-compose -f docker-compose.${{ matrix.qgis_version }}.yml"
Copy link
Contributor Author

@Raruto Raruto Jan 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the automated tests are run on the Ubuntu 20.04 runner image:

ci_tests:
runs-on: ubuntu-20.04

As per actions/runner-images/images/linux/Ubuntu2004-Readme.md, the docker compose (v2) is installed, so why we can't just use that instead of docker-compose (v1)?

Docker Amazon ECR Credential Helper 0.6.0
Docker Compose v1 1.29.2
Docker Compose v2 2.14.0+azure-1
Docker-Buildx 0.9.1
Docker-Moby Client 20.10.21+azure-2
Docker-Moby Server 20.10.21+azure-2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker Compose v1 1.29.2
Docker Compose v2 2.14.0+azure-1

This happens because some of these packets are available for GitHub Actions or Azure DevOps (that's why Docker Compose v2 2.14.0+azure-1 is only available for Azure V1 pipelines)

Ref: runner-images#faqs


runner-images/images/linux/scripts/installers/docker-compose.sh

#!/bin/bash -e
################################################################################
##  File:  docker-compose.sh
##  Desc:  Installs Docker Compose
################################################################################

# Install docker-compose v1 from releases
URL="https://github.com/docker/compose/releases/download/1.29.2/docker-compose-Linux-x86_64"
curl -L $URL -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

invoke_tests "Tools" "Docker-compose v1"

runner-images/images/win/scripts/Installers/Install-Docker.ps1

Write-Host "Install-Package Docker-Compose v1"
Choco-Install -PackageName docker-compose

Write-Host "Install-Package Docker-Compose v2"
$dockerComposev2Url = "https://github.com/docker/compose/releases/latest/download/docker-compose-windows-x86_64.exe"
$cliPluginsDir = "C:\ProgramData\docker\cli-plugins"
New-Item -Path $cliPluginsDir -ItemType Directory
Start-DownloadWithRetry -Url $dockerComposev2Url -Name docker-compose.exe -DownloadPath $cliPluginsDir

@wlorenzetti wlorenzetti modified the milestones: v3.6, v3.7 May 24, 2023
@wlorenzetti wlorenzetti removed this from the v3.7 milestone Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request refactoring Anything which could result in a API change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a Makefile to automate common command-line tasks
2 participants