major deploy tidy

[casperdcl]

• remove duplication of PyPI build (was repeated in build stage and deploy stage)
• remove duplication of Snapcraft build (was repeated in build stage and deploy stage)
• fix Snap stable channel not updating #3152 missing snap stable <- master deployment
• remove deploy stage in favour of conditional deployments on build stages (as per binary packages)
• skip package deployment (but still build) on cron jobs & pull requests
• references
  • https://docs.travis-ci.com/user/deployment#conditional-releases-with-on
  • https://docs.travis-ci.com/user/environment-variables

[casperdcl]

@efiop I think this is mergeable (the wonderful thing being there's no way to fully test without merging into master and tagging)

[efiop]

@casperdcl Need to be careful about it, I will take a look at it a bit later today. Thank you 🙂

[casperdcl]

Not sure about this osx logic - is it still required?

[efiop]

@casperdcl Good catch! Not required anymore, it was a leftover from the older days.

[ghost]

what's the reason behind this check, @casperdcl ?

[casperdcl]

travis' deploy.file might not exist. This is a convenient way to check

[efiop]

@casperdcl Still don't understand what it is for. What is deploy.file?

[efiop]

Got it. It is because of globs passed to it.

[efiop]

@casperdcl snap job now takes 19!!! minutes, which is more than 2x longer than other build jobs. Let's remove it and only deploy snap on tags, I don't think anyone needs dev branch snaps anyway.

[casperdcl]

That looks like a one-off mistake. Looking at the travis log the job should've taken about 330 seconds (similar to all previous builds). Will try restarting.

[casperdcl]

There we go... back down to 6min

[efiop]

Thanks! 🙂

[casperdcl]

Ah I was about to push a little bit of tidying after the reviews. Will do in a new PR (incl fixing any issues if there are any)

[efiop]

@casperdcl Please use WIP prefix to make it clear if a PR is being worked on or not. 🙂

[casperdcl]

well the 2 review conversations were not yet marked as resolved...

[casperdcl]

anyway happily seems to have all worked perfectly. Opened a very tiny #3173.

[efiop]

@casperdcl Yeah, I should've been more careful with that. WIP is appreciated too though 😄 Thanks!

[efiop]

@casperdcl Ok, what I didn't realize is that we broke the staging. Previously we:

1. checked patch formatting
2. ran tests
3. tried to build all needed release packages
4. built and deployed irreversible packages (e.g. pypi that we can't delete and upload that same version again)

Now we've lost 4 and if one package fails, pypi will be deployed anyways. So if one of the packages finds out that we are missing some import/dependency pypi will have that broken version already available for everyone to install. It is not a giant problem for now, but most likely I will change it back in the future. Please, let's be careful about these next time.

[casperdcl]

@efiop the new steps are:

1. check formatting
2. concurrently run tests
3. concurrently build & deploy

The old method was:

1. check formatting
2. concurrently run tests
3. concurrently build
4. concurrently rebuild & deploy

i.e. I intentionally dropped step 3.

[efiop]

@casperdcl Yep, I understand that. Didn't read your description carefully enough when reviewing. 🙂

[casperdcl]

@efiop do you mean you prefer having an extra step? It's slower that way (due to rebuilding for deployment), involves duplication in .travis.yml (so harder to maintain/error prone), and shouldn't be required as the tests are meant to check that everything works.

In fact on occasion one deployment may fail (e.g. due to PyPI servers being temporarily down) which should not affect the other deployments. We can just restart the failing deployment manually.

[efiop]

@casperdcl It being slower is not an issue, as it is only done during a release. Duplication is not nice, I agree, especially with snap, which has quite a bit more involved config than pypi that we used to have.

In fact on occasion one deployment may fail (e.g. due to PyPI servers being temporarily down) which should not affect the other deployments. We can just restart the failing deployment manually.

My concern is that one package will break and while we are fixing it, all other ones will be already released, which might cause a confusion of someone not being able to find a specific type of package.But thinking about it, it is not a big deal these days as the release is not fully automated and neither it will be, because we have to wait manual confirmation from homebrew/conda/choco guys anyway. So let's just keep it as is, it does look nicer now, no doubt about it 🙂 Thank you!