[Infra] Remove the third party action from CI

[acezen]

Proposed changes

Apache project only allows to use github or apache's actions, try to replace the third-party action in CI.

[SemyonSinchenko]

@acezen, do you need help here with all this python stuff?

[acezen]

@acezen, do you need help here with all this python stuff?

yeah, Apache only allows project to use github/apache's actions in CI, can you help me how to replace the poetry related action in CI?

[SemyonSinchenko]

Of course. I will do it in the evening.

[acezen]

Of course. I will do it in the evening.

that's great! thanks Sem.

[SemyonSinchenko]

@acezen This way is working for me:

    - name: Install Poetry
      run: |
          yes | sudo python3 -m pip install poetry --quiet
          cd pyspark
          poetry env use python3

I tried it in my fork and it works

I see that you already made a lot of changes, so we can merge your current work and I will fix broken pipelines, or you can just copy-paste the way how I install poetry and it should work.

[acezen]

@acezen This way is working for me:

    - name: Install Poetry
      run: |
          yes | sudo python3 -m pip install poetry --quiet
          cd pyspark
          poetry env use python3

I tried it in my fork and it works

I see that you already made a lot of changes, so we can merge your current work and I will fix broken pipelines, or you can just copy-paste the way how I install poetry and it should work.

Thanks Sem, I will try to copy-paste to check the CI.

[acezen]

Hi, @SemyonSinchenko, it works! Can you take a review for the changes?

[SemyonSinchenko]

I think we still need to switch to latest Ubuntu, because 20.04 will stop getting security updates in the middle of 2024. I would suggest latest or 22.04.

[SemyonSinchenko]

LGTM

[acezen]

for

It seems that the CI still run a ubuntu-20.04, we can merge this change first, and change to ubuntu-latest in another PR.