autorefine

Warning

Work in Progress. 🚧

Hit the 👁 Watch button to know when this project is ready to be tried out!

Mechanize the code quality improvement process to operate on a large scale.

The project is aimed to provide a framework for the process of refactoring Python projects from the very beginning to the very end:

• autorefine analyze → analyze the project with linters and type checkers,
• autorefine roadmap → create a refactoring roadmap (like CERT-Polska/malduck#110, but with more details available as a specialized document),
• autorefine plan → create a delivery plan (like in the issue above) to split the refactoring process into a sequence of iterations,
• autorefine autopilot → apply automatic fixes and commit them in reviewable and git-bisectable chunks with meaningful descriptions,
• autorefine ci → typically run tests and other tools that determine whether the project runs safe,
• autorefine audit → run pip-audit and try to auto-fix in dependency specs,
• autorefine troubleshoot → using the configured ci task repeatedly when running git bisect to find out what fix went wrong, describe the problem,
• leave the rest for manual work,
• autorefine finalize → create a PR (like CERT-Polska/malduck#111) to finalize the current iteration of refactoring with a detailed description of the changes and a link to the roadmap, delivery plan and all commits.

Created to accomplish jaraco/skeleton#98 and for personal use to raise awareness of best practices across the Python community globally.

Refactoring Workflow

autorefine will be a CLI tool to help you refactor a project in a few simple steps:

1. Fork a repository you want to refactor.

2. Create a new branch for refactoring.

3. Run autorefine configure to run a wizard that will help you configure autorefine, configure audits (autorefine will typically detect doctest, pytest and other tools that determine whether the project runs safe) and refactor the project for a specific choice of linting tools.

4. Run autorefine analyze to analyze the project with linters and type checkers. autorefine now knows what needs to be fixed in the current HEAD.

5. Run autorefine roadmap to create a refactoring roadmap. You will have a checklist of all the things that need to be fixed in the project automatically and manually.

6. Run autorefine plan to create a delivery plan and set up iterations of the refactoring process. For example, you can start off by a PR that aims to modernize the codebase and does not change the implementation (fixes in the roadmap will be grouped and you will be able to select which ones you want to apply in this iteration), that would be the first iteration, and then apply some implementation-changing autofixes, such as replacing f"'{x}'" with f"{x!r}"—that could be the second iteration. Having a delivery plan will make autorefine create a PR for each iteration, so that you can review the changes and merge them separately.

7. Run autorefine autopilot to apply automatic fixes and commit them in reviewable and git-bisectable chunks with meaningful descriptions.

8. Run autorefine ci to prevent future regressions.

9. Run autorefine audit to address supply chain security concerns.

10. Run autorefine troubleshoot to repeatedly call autorefine audit within git bisect to find out what fix went wrong, and get a detailed description of the problem and suggestions on how to fix it.

11. Manually fix the problems that cannot be fixed automatically. You will find them in the roadmap (autorefine roadmap).

12. Run autorefine finalize to finalize the current iteration and create a PR with a detailed description of the changes and a link to the roadmap, delivery plan and all commits of the current refactor iteration.

Planned Tooling

The project is planned to be a wrapper around the following well-tested & recognized tools:

• MonkeyType for automatic type annotations generation,
• Fixit for automatic fixes that require static analysis and scope analysis and planning of manual refactoring steps,
• Ruff for automatic fixes and planning of manual refactoring steps,
• Static type checking:
  • mypy,
  • pyre,
  • pytype,
• diff-cover for coverage reports on fixes,
• smokeshow for hosting detailed refactoring roadmaps and delivery plans,
• pandas for collecting tasks and creating markdown tables,
• GitHub CLI for creating tickets.

Non-MVP Ideas

• autorefine makeci to create a dedicated CI pipeline (which will, by default, run autorefine analyze and autorefine audit on every PR) for your Git hosting provider (GitHub, GitLab, Bitbucket, etc.),
• autorefine template [TEMPLATE_NAME] to configure Quality Assurance and CI/CD tooling (Ruff, tox, towncrier, Sphinx, etc.) by smart copying what is seen a specified repository (e.g. autorefine template jaraco/skeleton+jaraco.develop could be used to copy tox.ini, GitHub Actions etc.)—feature for projects that don't intend to use a skeleton.
• autorefine breakdown to create separate tickets for each fix in the roadmap in your workflow management tool (GitHub Projects, Jira, ClickUp, Asana, Trello, etc.).

Get Inspired

• https://instagram-engineering.com/static-analysis-at-scale-an-instagram-story-8f498ab71a0c

For Enterprise

	Available as part of the Tidelift Subscription. This project and the maintainers of thousands of other packages are working with Tidelift to deliver one enterprise subscription that covers all of the open source you use. Learn more here.

To report a security vulnerability, please use the Tidelift security contact.
Tidelift will coordinate the fix and disclosure.

Installation

To use this globally as a CLI tool only, simply install it with pipx:

pipx install autorefine

But you might also simply install it with pip to access the library API:

pip install autorefine

If you use Poetry, then you might want to run:

poetry add autorefine

For Contributors

Note

If you use Windows, it is highly recommended to complete the installation in the way presented below through WSL2.

1. Fork the autorefine repository on GitHub.

2. Install Poetry.
   Poetry is an amazing tool for managing dependencies & virtual environments, building packages and publishing them. You might use pipx to install it globally (recommended):

   pipx install poetry

   _{If you encounter any problems, refer to the official documentation for the most up-to-date installation instructions.}

   Be sure to have Python 3.8 installed—if you use pyenv, simply run:

   pyenv install 3.8

3. Clone your fork locally and install dependencies.

   git clone https://github.com/your-username/autorefine path/to/autorefine
   cd path/to/autorefine
   poetry env use $(cat .python-version)
   poetry install

   Next up, simply activate the virtual environment and install pre-commit hooks:

   poetry shell
   pre-commit install

For more information on how to contribute, check out CONTRIBUTING.md.
Always happy to accept contributions! ❤️

Legal Info

© Copyright by Bartosz Sławecki (@bswck).
This software is licensed under the terms of GPL-3.0 License.