Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate CI to Github Actions #1159

Merged

Conversation

tobias-urdin
Copy link
Contributor

This PR's is to move away from Travis CI since it's not credit based and we run out of credits after just a few CI runs.
However this also includes a lot of other fixes and changes that benefits the stability and testing of Gnocchi.

The migration plan to move to Github Actions from Travis CI is to (that this PR partly does) continue using Docker but for now build on each job run until we change DockerHub to read Dockerfile from main repo. We can then add "container" directly to our jobs in the workflow and use a hosted container image, and thus retire the ci-tools repo.

This PR will:

  • Move Dockerfile from ci-tools repo to main repo (inside our gnocchi-container GitHub action)
  • Updates s3rver version to 3.7.0 and uses default credentials which hopefully stabilizes the S3 jobs - hence the update to the s3rver default credentials (username and password is S3RVER)
  • Migrate Docker container to Ubuntu 20.04 (defaults to Python 3.8 but we also run 3.6)
  • Because of above this means we are testing newer version of Ceph, Redis etc that is installed from distro packages
  • Add testing for Python 3.8 but keeps Python 3.6 testing
  • Adds back Ceph testing but only for Python 3.8 - this is because the rados module (python3-rados dist package) that we import from global site-packages is only for Python 3.8, we can build a Ubuntu 18.04 image with python3-rados for Python 3.6 if we want to test that but can be a follow up.
  • This fixes so that the deploy pipeline (that is the only thing left) in Travis CI uses Python 3
  • Finally this contains some minor fixes to documentation, old comments, deprecation warnings and things to make it easier to run locally.

This PR has a hard dependency on jd/pifpaf#150 to make Ceph working.

There are some follow up patches that can be done after this which is not covered in this patch.

  • When DockerHub container image builds is updated we can remove our custom GitHub gnocchi-container action and instead use the "container" directive directly in the workflow which will pull the image from DockerHub instead of rebuilding on each job run
  • Add Python 3.6 Ceph jobs again
  • Move away the deploy pipeline from Travis CI into a GitHub actions that is executed when a new release is created

This removes the functional testing that was
used in Travis CI.

This still leaves the deploy pipeline and fixes
the before_deploy script so that python3 is installed
so the pypi upload works.

It adds the "os" key to the Travis CI config which was
a warning. The "user" key in the deploy pipeline is deprecated
and should be "username".
If we want to be able to bump s3rver to a newer
version we must use the default credentials since
that "S3" user can use signing.
This action takes the Dockerfile from ci-tools
and updates it to run on Ubuntu 20.04

The action creates the container on each run but
after we have migrated away DockerHub to this Dockerfile
and archived ci-tools repo we can use the "container"
directive on workflows to pull the image from there
instead of building it on each run.
Adds the configuration the Gnocchi workflow which moves
the docs, pep8 and functional testing jobs to GitHub Actions.
This adds Python 3.8 and Ubuntu Focal support and
pins pifpaf until [1] is merged and released.

Fixes some warnings on whitelist_externals that
wasn't correct.

[1] jd/pifpaf#150
This sets the s3rver username and password for the
S3 functional testing jobs, required since we have
bumped s3rver version.
@tobias-urdin tobias-urdin force-pushed the feature/fix-migrate-and-update-ci branch from 7214224 to c51a587 Compare July 4, 2021 19:57
@tobias-urdin
Copy link
Contributor Author

@jd see above, all commits should have appropriate comments as well. I tested the workflow on my fork so should be working, but you might need to allow the workflow inside the gnocchi project settings (I don't have permissions I think).

@tobias-urdin tobias-urdin requested a review from jd July 4, 2021 20:05
@tobias-urdin
Copy link
Contributor Author

I should also mention that it only migrates the master branch to Github Actions, the stable branches needs to be migrated in a follow up.

This updates the Mergify config so that it checks
the GitHub Actions. We also remove the backport
labels for the branches that we recently renamed
as EOL.
@tobias-urdin tobias-urdin force-pushed the feature/fix-migrate-and-update-ci branch from c51a587 to e47b926 Compare July 4, 2021 20:25
@openstacker
Copy link

Nice work. Thanks.

@jd
Copy link
Member

jd commented Jul 5, 2021

GitHub Actions is enabled on this project.

@jd
Copy link
Member

jd commented Jul 5, 2021

@tobias-urdin That looks good to me, I'm ok to merge it. You can still fix it as you go.

@tobias-urdin tobias-urdin merged commit 8ed14c7 into gnocchixyz:master Jul 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants