Auto-build Docker images before on-merge if setup.py was changed

[muellerzr]

What does this PR do?

This PR introduces a new workflow that will check if the setup.py was modified during a pull request merge. If so, it will trigger the docker images to be rebuilt before running the on-merge tests.

It also changes self-push to be ran on a workflow_run, specifically the new check-dependencies job. This new job also maintains the same "on-merge" check the previous job had, when it comes to determining if it should be ran and when.

Finally, build-docker-images is now also ran on a workflow_call, so that check-dependencies can trigger it.

This is the same as done in Accelerate recently, with the only difference being additional file filters huggingface/accelerate#424

Why is this needed?

A frustration I've noticed over the last few months in this repo is the main tests runners are failing for an entire day, due to a new dependency introduced. This solves this problem, since the issue roots from the docker images being used.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ydshieh @LysandreJik

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ydshieh]

Do not merge for now. The push CI in transformers is somehow tricky, after the recent change in #17369.
I will review tomorrow, but I think some changes have to be made.

[ydshieh]

@muellerzr Thank you for the PR.

The most direct approach would be

Integrate the check check-for-setup and build-docker-containers in
https://github.com/huggingface/transformers/blob/main/.github/workflows/self-push-caller.yml
(before the job run_push_ci)

Otherwise (if you really want to keep the logic you have), the following block

  workflow_run:
    workflows: ["Check for dependency modification"]
    branches: ["main"]
    types: [completed]

should go in self-push-caller.yml.

The main point is to run the actual push CI tests on another branch (push-ci), otherwise there will be more than 256 job results shown in the commit history page.

I would prefer the most direct approach (the first one).

[muellerzr]

@ydshieh I believe I addressed what you wanted, let me know if otherwise 😄

[ydshieh]

LGTM, but you can rebase on main first and double check if there are some redundant blocks in the final commit before merge.

Thank you for this work!

There are some changes recently (the 2 push workflow files)