Migrate local and CI linting and formatting to trunk.io

[surovic]

Trunk is a superlinter that supports both local and CI linting and source formatting tasks. AFAIK it supports all the languages and markups that we use. The PR migrates all of our linter configuration to be compatible with trunk and also replaces flake8 with ruff as it subsumes it and is more performant.

The one caveat is that we now effectively only lint python v3.8. mypy is set to that version and ruff will only lint issues and suggest fixes that are compatible with python 3.8. and up.

The linters also only check changed code for issues. This means developers should not be harassed about issues they didn't cause.

The last side-effect of this PR is that the top level Makefile has been rewritten and now supports:

• make lint - locally runs trunk check, which is equivalent to the CI linting workflow.
• make format - locally runs trunk fmt, which formats changes made in the repo.
• make docker - build the trailofbits/polytracker docker image.
• make test - runs the pytest /polytracker/tests command in trailofbits/polytracker docker image container.
• make clean - deletes all trailofbits/polytracker* docker images.

[hbrodin]

This looks good to me. Nice that you did some cleanup of the readme and scripts. It looks much cleaner. With that said, I'd like for someone else to have a second opinion on this before proceeding. I have never used trunk.io before so I couldn't really tell if there are better ways of achieving what we do here.

The one thing that comes to mind is that if we now test/lint using python 3.8 we should mention that in the readme. I couldn't find anything about that.

[kaoudis]

The linting multiple python versions was a legacy thing that I kept because we were already implicitly doing it! I don't think it added any value. Thanks! ✅

[kaoudis]

We should probably still run tests with a couple Python versions though just in case.

[surovic]

We should probably still run tests with a couple Python versions though just in case.

Hmm, should we really? We essentially dictate the python version in the docker image build.

[kaoudis]

We should probably still run tests with a couple Python versions though just in case.

Hmm, should we really? We essentially dictate the python version in the docker image build.

My reasoning here is that we have in the past allowed the use of different Python versions with Polytracker. If someone out there is using it with a prior Python version, it would be good to not arbitrarily take that support away from them. However, we only support Polytracker usage in the container, to the best of my knowledge, and as long as we're clear there's one true way to run Polytracker in the README, I don't see any reason to continue running builds or tests under any Pythons other than the Python version we prefer to build and run with.