Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable skipping pre-commit hook repos during development dependency update workflow and action #158

Merged
merged 3 commits into from
Oct 22, 2024
+240 −29
Merged

Enable skipping pre-commit hook repos during development dependency update workflow and action #158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1cc4182
feat: Add the ability to skip updating a pre-commit repo as a part of…
nfelt14 Oct 22, 2024
eab7fae
chore: Fixed things found during PR review
nfelt14 Oct 22, 2024
019f8d2
chore: revert breaking change, it isn't needed
nfelt14 Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/_reusable-update-python-and-pre-commit-dependencies.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ on:
required: false
type: boolean
default: false
pre-commit-repo-update-skip-list:
description: A comma-separated list of pre-commit repo urls to skip updates
for (only applicable when `update-pre-commit=true`).
required: false
type: string
default: ''
pre-commit-hook-skip-list:
description: A comma-separated list of pre-commit hooks to skip (only applicable
when `run-pre-commit=true`).
Expand Down Expand Up @@ -89,6 +95,7 @@ jobs:
dependency-dict: ${{ inputs.dependency-dict }}
update-pre-commit: ${{ inputs.update-pre-commit }}
run-pre-commit: ${{ inputs.run-pre-commit }}
pre-commit-repo-update-skip-list: ${{ inputs.pre-commit-repo-update-skip-list }}
pre-commit-hook-skip-list: ${{ inputs.pre-commit-hook-skip-list }}
export-dependency-groups: ${{ inputs.export-dependency-groups }}
- if: ${{ !endsWith(github.repository, '/python-package-ci-cd') }} # Run the public action when this is run outside the python-package-ci-cd repository
Expand All @@ -97,6 +104,7 @@ jobs:
dependency-dict: ${{ inputs.dependency-dict }}
update-pre-commit: ${{ inputs.update-pre-commit }}
run-pre-commit: ${{ inputs.run-pre-commit }}
pre-commit-repo-update-skip-list: ${{ inputs.pre-commit-repo-update-skip-list }}
pre-commit-hook-skip-list: ${{ inputs.pre-commit-hook-skip-list }}
export-dependency-groups: ${{ inputs.export-dependency-groups }}
- uses: stefanzweifel/git-auto-commit-action@8621497c8c39c72f3e2a999a26b4ca1b5058a842 # v5.0.1
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-actions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ jobs:
update-pre-commit: true
run-pre-commit: true
dependency-dict: '{"dev": ["pyright"]}'
pre-commit-repo-update-skip-list: https://github.com/pre-commit/pre-commit-hooks,https://github.com/executablebooks/mdformat
pre-commit-hook-skip-list: remove-tabs,forbid-tabs,check-readthedocs,check-dependabot,check-github-actions,check-github-workflows,commitizen,blacken-docs,yamlfix,hadolint,mdformat,markdown-link-check,check-poetry,toml-sort-fix,pyright,poetry-audit,ruff,ruff-format,docformatter,renovate-config-validator,actionlint
export-dependency-groups: |
actions-update_development_dependencies:actions/update_development_dependencies,
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ Valid subsections within a version are:

Things to be included in the next release go here.

### Added

- Added a new `pre-commit-repo-update-skip-list` input parameter to the `update_development_dependencies` action and the `_reusable-update-python-and-pre-commit-dependencies.yml` workflow to allow users to skip updating specific `pre-commit` hooks.

### Changed

- Bumped dependency versions.
Expand Down
5 changes: 5 additions & 0 deletions actions/update_development_dependencies/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ inputs:
the update-pre-commit input to `true`.
required: false
default: 'false'
pre-commit-repo-update-skip-list:
description: A comma-separated list of pre-commit repo urls to skip updates for
(only applicable when `update-pre-commit=true`).
required: false
default: ''
pre-commit-hook-skip-list:
description: A comma-separated list of pre-commit hooks to skip (only applicable
when `run-pre-commit=true`).
Expand Down
49 changes: 43 additions & 6 deletions actions/update_development_dependencies/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@

from pathlib import Path

import yaml

from pypi_simple import PyPISimple
from yamlfix import fix_files # pyright: ignore[reportUnknownVariableType]

Expand Down Expand Up @@ -126,21 +128,51 @@ def update_poetry_dependencies(
)


def update_pre_commit_dependencies(python_executable: str, repository_root_directory: Path) -> None:
def get_pre_commit_repos(repository_root_directory: Path) -> list[str]:
"""Get the list of repo urls from the .pre-commit-config.yaml file.

Args:
repository_root_directory: The root directory of the repository.

Returns:
The list of repo urls.
"""
pre_commit_file_data = yaml.safe_load(
(repository_root_directory / ".pre-commit-config.yaml").read_text()
)
repo_list: list[str] = []
for repo in pre_commit_file_data.get("repos", []):
if (repo_url := str(repo["repo"])) == "local":
continue # skip local repos, they don't need to be updated
repo_list.append(repo_url)
return repo_list


def update_pre_commit_dependencies(
python_executable: str,
repository_root_directory: Path,
pre_commit_repo_update_skip_list: list[str],
) -> None:
"""Update the pre-commit dependencies in the .pre-commit-config.yaml file.

This function will also fix the formatting of the yaml file using the `yamlfix` package.

Args:
python_executable: The path to the python executable to use.
repository_root_directory: The root directory of the repository.
pre_commit_repo_update_skip_list: A list of pre-commit repo urls to skip updating.
"""
run_cmd_in_subprocess(
f"git config --global --add safe.directory "
f'"{repository_root_directory.resolve().as_posix()}"'
)
# Update pre-commit config file
run_cmd_in_subprocess(f'"{python_executable}" -m pre_commit autoupdate --freeze')
# Update every hook in the pre-commit config file, skipping any hooks in the skip list
for repo in get_pre_commit_repos(repository_root_directory):
if repo in pre_commit_repo_update_skip_list:
continue
run_cmd_in_subprocess(
f'"{python_executable}" -m pre_commit autoupdate --freeze --repo {repo}'
)

# Fix the formatting of the pre-commit config file
with warnings.catch_warnings():
Expand Down Expand Up @@ -194,7 +226,10 @@ def main() -> None:
export_dependency_groups = [
x.strip() for x in os.environ["INPUT_EXPORT-DEPENDENCY-GROUPS"].split(",") if x
]
pre_commit_hook_skip_list = os.environ["INPUT_PRE-COMMIT-HOOK-SKIP-LIST"]
pre_commit_hook_run_skip_list = os.environ["INPUT_PRE-COMMIT-HOOK-SKIP-LIST"]
pre_commit_repo_update_skip_list = [
x.strip() for x in os.environ["INPUT_PRE-COMMIT-REPO-UPDATE-SKIP-LIST"].split(",") if x
]
install_dependencies = os.environ["INPUT_INSTALL-DEPENDENCIES"].lower() in _ENV_VAR_TRUE_VALUES
run_pre_commit = os.environ["INPUT_RUN-PRE-COMMIT"].lower() in _ENV_VAR_TRUE_VALUES
update_pre_commit = os.environ["INPUT_UPDATE-PRE-COMMIT"].lower() in _ENV_VAR_TRUE_VALUES
Expand All @@ -208,14 +243,16 @@ def main() -> None:
python_executable, repo_root_path, dependency_dict, lock_only=not install_dependencies
)
if update_pre_commit or run_pre_commit:
update_pre_commit_dependencies(python_executable, repo_root_path)
update_pre_commit_dependencies(
python_executable, repo_root_path, pre_commit_repo_update_skip_list
)
if export_dependency_groups:
export_requirements_files(python_executable, export_dependency_groups)
if run_pre_commit:
# Run the pre-commit hooks, ignore any errors since they are
# just being run to auto-fix files.
with contextlib.suppress(subprocess.CalledProcessError):
os.environ["SKIP"] = pre_commit_hook_skip_list
os.environ["SKIP"] = pre_commit_hook_run_skip_list
run_cmd_in_subprocess(f'"{python_executable}" -m pre_commit run --all-files')


Expand Down
20 changes: 11 additions & 9 deletions actions/update_development_dependencies/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,16 @@ This action enables updating Python development dependencies using the

## Inputs

| Input variable | Necessity | Description | Default |
| --------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `repo-root` | optional | The root directory of the repository. | `.` |
| `install-dependencies` | optional | A boolean indicating if packages should be installed via poetry (this is not usually needed). | `false` |
| `dependency-dict` | optional | Specify a valid JSON dictionary of dependency groups to update, where each key is a dependency group name, and each value is a list of dependencies to update within that group, e.g. `'{"dev": ["pylint", "ruff"], "tests": ["ruff"]}'`. Use an empty string, e.g. `""`, for dependencies located in the default group' | `{}` |
| `update-pre-commit` | optional | A boolean indicating if the pre-commit hooks should be updated. | `false` |
| `run-pre-commit` | optional | A boolean indicating to run the pre-commit hooks to perform auto-fixing after updating the dependencies. Setting this input to `true` will also set the update-pre-commit input to `true`. | `false` |
| `pre-commit-hook-skip-list` | optional | A comma-separated list of pre-commit hooks to skip (only applicable when `run-pre-commit=true`). | `""` |
| `export-dependency-groups` | optional | A comma-separated list of dependency groups to export to a `requirements.txt` file. The format is `group1,group2:custom-path/to/test/folder`. | `""` |
| Input variable | Necessity | Description | Default |
| ---------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `repo-root` | optional | The root directory of the repository. | `.` |
| `install-dependencies` | optional | A boolean indicating if packages should be installed via poetry (this is not usually needed). | `false` |
| `dependency-dict` | optional | Specify a valid JSON dictionary of dependency groups to update, where each key is a dependency group name, and each value is a list of dependencies to update within that group, e.g. `'{"dev": ["pylint", "ruff"], "tests": ["ruff"]}'`. Use an empty string, e.g. `""`, for dependencies located in the default group' | `{}` |
| `update-pre-commit` | optional | A boolean indicating if the pre-commit hooks should be updated. | `false` |
| `run-pre-commit` | optional | A boolean indicating to run the pre-commit hooks to perform auto-fixing after updating the dependencies. Setting this input to `true` will also set the update-pre-commit input to `true`. | `false` |
| `pre-commit-repo-update-skip-list` | optional | A comma-separated list of pre-commit repo urls to skip updates for (only applicable when `update-pre-commit=true`). | `""` |
| `pre-commit-hook-skip-list` | optional | A comma-separated list of pre-commit hooks to skip (only applicable when `run-pre-commit=true`). | `""` |
| `export-dependency-groups` | optional | A comma-separated list of dependency groups to export to a `requirements.txt` file. The format is `group1,group2:custom-path/to/test/folder`. | `""` |

## Example

Expand All @@ -53,6 +54,7 @@ jobs:
dependency-dict: '{"dev": ["pylint", "ruff"], "tests": ["ruff"]}' # optional, but without it nothing will get updated by Poetry
update-pre-commit: true # optional
run-pre-commit: true # optional
pre-commit-repo-update-skip-list: 'https://github.com/pre-commit/pre-commit-hooks' # optional
pre-commit-hook-skip-list: 'pylint' # optional, hooks that don't auto-fix things can (and probably should be) skipped
export-dependency-groups: 'docs,tests:custom-path/to/test/folder' # optional

Expand Down
Loading
Loading