👷📦🚀 Enable Travis CI to publish dists to PYPI #262

webknjaz · 2018-06-10T22:31:27Z

Resolves #222

@Mariatta you still need to update secret and probably add @brettcannon's account for blurb.

Some hacks present are trying to work around limitations described in travis-ci/dpl#822

webknjaz · 2018-06-10T22:36:36Z

.travis.yml

+    # This happenes when other CI (AppVeyor etc.) has already uploaded
+    # the very same dist (usually sdist).
+    skip-existing: true
+    user: Mariatta  # TODO: consider having a separate "uploader user"


@Mariatta you might want to set up a separate account with "Maintainer" level privileges in PYPI to use instead of your own, which is more dangerous if it gets compomised.

Recently, we've started replacing our flows to use a deployment-dedicated user in aio-libs as well: https://github.com/aio-libs/aiohttp/blob/master/.travis.yml#L84

@Mariatta I've just realized that blurb is deployed by @brettcannon and your account is not linked to that dist. Will you fix this by having an account with access to publish both or should I separate deployment step to have different deployer users?

webknjaz · 2018-06-10T22:37:35Z

.travis.yml

+      # Encrypt with `travis encrypt -r python/core-workflow --org` while using travis-ci.org;
+      # change to `travis encrypt -r python/core-workflow --api-endpoint 'https://api.travis-ci.com/'`
+      # upon switch to __free__ travis-ci.com:
+      secure: PLACE_YOUR_ENCRYPTED_PASSWORD_HERE


@Mariatta feel free to update the PR branch with a secret generated for your account as per instructions in comments above.

webknjaz · 2018-06-11T11:53:55Z

@Mariatta I've fixed conditionals.

Demo:

push build: https://travis-ci.com/webknjaz/core-workflow/builds/75899792
tag blurb-v0.0.3: https://travis-ci.com/webknjaz/core-workflow/builds/75899834
tag cherry-picker-v1.2.19: https://travis-ci.com/webknjaz/core-workflow/builds/75899965

It should be ready for merging (you may add PYPI creds before or after merge).

webknjaz · 2018-06-22T11:05:01Z

@Mariatta I'm going to update this to exclude non-related builds when releasing a certain tool. This will include conditions: v1 and moving stuff off the matrix to explicit jobs.include.

Ref: travis-ci/travis-conditions#1

Mariatta · 2018-07-09T03:03:05Z

tag cherry-picker-v1.2.19: https://travis-ci.com/webknjaz/core-workflow/builds/75899965

That build failed. Any clue?

Just to confirm how this will work, when I'm ready to release to PyPI, just create a tag like cherry-picker-vX.Y.Z and travis will run?

webknjaz · 2018-07-10T09:33:50Z

@Mariatta oh it's fine.

HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://upload.pypi.org/legacy/

I didn't put any valid PYPI credentials there. That's why this happened. It will work once you add credentials as per https://github.com/python/core-workflow/pull/262/files#diff-354f30a63fb0907d4ad57269548329e3R54.
(You can push those to my branch bound to this PR via UI or Git)

webknjaz · 2018-07-10T16:17:01Z

@Mariatta I've made jobs appearence more granular for release mode.

webknjaz · 2018-07-12T16:26:09Z

@Mariatta just in case it gets lost in diff page, we need to solve this as well: #262 (comment)

webknjaz · 2018-07-17T08:52:49Z

@Mariatta do you need any additional help with this?

@Mariatta

Resolves python#222 @Mariatta still needs to update secret

Mariatta · 2018-08-01T03:29:26Z

@webknjaz I've asked @brettcannon to create the bot account for deployment to PyPI. Not sure if he's gotten around doing it yet.

webknjaz · 2018-08-01T08:24:11Z

@Mariatta alternatively you could create a shared acc yourself and then you can add it to one project with minimal privileges and @brettcannon would add it to other. If this helps :)

brettcannon · 2018-08-03T19:07:08Z

I just created the core-workflow account.

brettcannon · 2018-08-03T19:07:37Z

https://pypi.org/user/core-workflow/

Mariatta · 2018-08-03T19:24:07Z

.travis.yml

+      # Encrypt with `travis encrypt -r python/core-workflow --org` while using travis-ci.org;
+      # change to `travis encrypt -r python/core-workflow --api-endpoint 'https://api.travis-ci.com/'`
+      # upon switch to __free__ travis-ci.com:
+      secure: PLACE_YOUR_ENCRYPTED_PASSWORD_HERE


Thanks @brettcannon for creating the core-workflow PyPI user. I've added it as maintainer to cherry_picker in PyPI.
Do you mind creating the encrypted password and add it here?

Sent the password to Mariatta securely as I'm heading out for a short vacation once I'm done with my GitHub notifications 😄

Mariatta · 2018-08-03T19:24:21Z

.travis.yml

+    # This happenes when other CI (AppVeyor etc.) has already uploaded
+    # the very same dist (usually sdist).
+    skip-existing: true
+    user: &pypi-user Mariatta  # TODO: consider having a separate "uploader user"


This has to be core-workflow now.

May I expect that you will update this branch yourself?

If not - please share the encrypted string and I'll do it myself :)

I've updated. At least I think I did it correctly ...

Looks fine. Sometimes it doesn't work because of corrupted secret value, but we'll only see this during actual deployment in this repo. Re-encryption usually helps.

Mariatta · 2018-08-03T19:26:02Z

.travis.yml

+    env:
+      TARGET_PKG: blurb
+    deploy:
+      <<: *deployment-config


I don't know if blurb maintainers (I'm not one of them) want the same capability or not. Maybe we need to remove this portion.

I don't really like removing stuff. Instead, we can add smth likeif: false to disable this. Deal?

webknjaz · 2018-08-03T22:12:55Z

.travis.yml

+- &run-if-tagged
+  if: tag IS present
+- &run-if-cherry-picker
+  if: tag =~ ^cherry\-picker\-v\d+\.\d+\.\d+$


@Mariatta I'm a bit concerned about restricting this to vX.Y.Z format, while in real life there might be some extensions (dev/a/b). Should I extend this regexp to take those into account?

I think we can leave the regex as is (handling vX.Y.Z) and when the need arise we can adjust it later.

Okay, I just thought of a common use case for testing pre-releases when you are unsure whether build artifact (dist) is correct.

webknjaz · 2018-08-04T15:22:04Z

I believe that you can merge this now.

webknjaz · 2018-08-04T15:24:54Z

I would then test releasing mechanizm, but since it's a change, which does not affect the software, I'd add .post0. It looks like a right thing to do.

Mariatta

Thanks!!! 🌮

Mariatta · 2018-08-04T15:27:53Z

Once we have updated changelog for cherry_picker (open issue #279), I can try to cut a release. But I'm about to head to the beach now 🏖 ...

webknjaz · 2018-08-04T15:28:18Z

🎉

webknjaz · 2018-08-04T15:29:28Z

Yeah, actually me too: just arrived to Cyprus a few hours ago 🌴

the-knights-who-say-ni added the CLA signed label Jun 10, 2018

webknjaz force-pushed the feature/travis-cd-to-pypi branch from 076b715 to e6113fc Compare June 10, 2018 22:31

webknjaz mentioned this pull request Jun 10, 2018

Automatically deploy cherry_picker (and/or blurb) to PyPI #222

Closed

webknjaz commented Jun 10, 2018

View reviewed changes

webknjaz force-pushed the feature/travis-cd-to-pypi branch 4 times, most recently from 076f934 to 91b4aab Compare June 11, 2018 11:49

webknjaz force-pushed the feature/travis-cd-to-pypi branch from f0c6de3 to 6886155 Compare July 10, 2018 16:00

webknjaz added 10 commits July 17, 2018 10:55

👷📦🚀 Enable Travis CI to publish dists to PYPI

354abdf

Resolves python#222 @Mariatta still needs to update secret

Move job conditions to mixtures

998af76

Make test matrix explicit

ce8dddc

Test only related project when releasing to PYPI

998785a

Opt-in for better conditions support

9848285

Uppercase OR in if-clauses

58180db

Add project markers via env vars

970d7d5

Move creation of hacky stub to before_deploy

64daaa1

Split deployment configs for blurb / cherry-picker

23ee67b

Make better use of env var for prj distinction

931f0c3

webknjaz force-pushed the feature/travis-cd-to-pypi branch from 1a2d21c to 931f0c3 Compare July 17, 2018 09:02

webknjaz mentioned this pull request Jul 31, 2018

Mypy 0.620 and generic type aliases aio-libs/multidict#272

Merged

Mariatta reviewed Aug 3, 2018

View reviewed changes

Mariatta and others added 5 commits August 3, 2018 14:53

Add core-workflow encrypted password

fddf8dc

PYPI -> PyPI

70dfcc4

temporary disable publishing blurb

1e560b0

Drop unnecessary deploy customizations for blurb

681c591

Drop unnecessary TODO comments

318f920

webknjaz commented Aug 3, 2018

View reviewed changes

Mariatta approved these changes Aug 4, 2018

View reviewed changes

Mariatta merged commit 084b51e into python:master Aug 4, 2018

webknjaz mentioned this pull request Dec 7, 2018

Allow git + ssh based uploads pypi/warehouse#2315

Closed

webknjaz mentioned this pull request May 31, 2019

Create .travis.yml python/cherry-picker#2

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

👷📦🚀 Enable Travis CI to publish dists to PYPI #262

👷📦🚀 Enable Travis CI to publish dists to PYPI #262

webknjaz commented Jun 10, 2018 •

edited

Loading

webknjaz Jun 10, 2018

webknjaz Jul 12, 2018

webknjaz Jun 10, 2018

webknjaz commented Jun 11, 2018

webknjaz commented Jun 22, 2018 •

edited

Loading

Mariatta commented Jul 9, 2018

webknjaz commented Jul 10, 2018

webknjaz commented Jul 10, 2018

webknjaz commented Jul 12, 2018

webknjaz commented Jul 17, 2018

Mariatta commented Aug 1, 2018

webknjaz commented Aug 1, 2018

brettcannon commented Aug 3, 2018

brettcannon commented Aug 3, 2018

Mariatta Aug 3, 2018

brettcannon Aug 3, 2018

Mariatta Aug 3, 2018

webknjaz Aug 3, 2018

webknjaz Aug 3, 2018

Mariatta Aug 3, 2018

webknjaz Aug 3, 2018

Mariatta Aug 3, 2018

webknjaz Aug 3, 2018

Mariatta Aug 3, 2018

webknjaz Aug 3, 2018

Mariatta Aug 4, 2018

webknjaz Aug 4, 2018

webknjaz commented Aug 4, 2018

webknjaz commented Aug 4, 2018

Mariatta left a comment •

edited

Loading

Mariatta commented Aug 4, 2018

webknjaz commented Aug 4, 2018

webknjaz commented Aug 4, 2018

👷📦🚀 Enable Travis CI to publish dists to PYPI #262

👷📦🚀 Enable Travis CI to publish dists to PYPI #262

Conversation

webknjaz commented Jun 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webknjaz commented Jun 11, 2018

webknjaz commented Jun 22, 2018 • edited Loading

Mariatta commented Jul 9, 2018

webknjaz commented Jul 10, 2018

webknjaz commented Jul 10, 2018

webknjaz commented Jul 12, 2018

webknjaz commented Jul 17, 2018

Mariatta commented Aug 1, 2018

webknjaz commented Aug 1, 2018

brettcannon commented Aug 3, 2018

brettcannon commented Aug 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webknjaz commented Aug 4, 2018

webknjaz commented Aug 4, 2018

Mariatta left a comment • edited Loading

Choose a reason for hiding this comment

Mariatta commented Aug 4, 2018

webknjaz commented Aug 4, 2018

webknjaz commented Aug 4, 2018

webknjaz commented Jun 10, 2018 •

edited

Loading

webknjaz commented Jun 22, 2018 •

edited

Loading

Mariatta left a comment •

edited

Loading