Skip to content
This repository has been archived by the owner on May 6, 2022. It is now read-only.

We are not able to release Service Catalog because of timeout #2726

Closed
mszostok opened this issue Oct 7, 2019 · 3 comments
Closed

We are not able to release Service Catalog because of timeout #2726

mszostok opened this issue Oct 7, 2019 · 3 comments
Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. release

Comments

@mszostok
Copy link
Contributor

mszostok commented Oct 7, 2019

Description

Currently, the Service Catalog release is done by Travis CI. There is one job called Deploy. In that stage, we are building and pushing:

  • Service Catalog images for all arch (amd64 arm arm64 ppc64le s390x)
  • svcat CLI for all arch (win, linux, darwin)

Release history with time consumption:

Travis builds are timeouted when a job on a public repository takes longer than 50 minutes.
source: https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts

Problem

As we can see we were always really close to job timeout. The build process also depends on network connection because we are downloading a docker image for each arch, then building our binaries and pushing result images to quey.
In version 0.3.0 we are building additionally migration and cleaner binaries which also affected building time.

As a result, the new Service Catalog version cannot be released because we are hitting timeout:

The job exceeded the maximum time limit for jobs, and has been terminated.

Solution

The Travis job timeout cannot be customized. We are also limmited by the open-source global queue and sometimes we need to wait for jobs to be started.

As "hot-fix" we should parallelize our deploy stage by arch type. In the long term, we should move our release pipeline to Prow infra where we have more control.

@mszostok
Copy link
Contributor Author

mszostok commented Oct 9, 2019

Partially resolved by the: #2727

Still, we need to move our release pipeline under Prow infra.

@mszostok
Copy link
Contributor Author

mszostok commented Oct 9, 2019

/lifecycle frozen

@k8s-ci-robot k8s-ci-robot added the lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. label Oct 9, 2019
@mrbobbytables
Copy link

This project is being archived, closing open issues and PRs.
Please see this PR for more information: kubernetes/community#6632

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. release
Projects
None yet
Development

No branches or pull requests

3 participants