RLS: 0.21.0 #17285

TomAugspurger · 2017-08-18T19:20:28Z

For 0.21.0, October 2017

Pre-Release

Open a pre-release issue and mention related parties. (A few weeks before release) with this template.
Clear the bug queue
Run an ASV: `cd asv_bench; asv continuous -f 1.1 $(git describe --abbrev=0)..HEAD``
Review deprecations from previous releases here
Review experimental features from previous releases here
Clean up release.rst (contributors and stats will be done later)
Clean up whatsnew.rst
Run a full doc build and make sure everything is clean
```
./make.py clean; ./make.py; ./make.py zip_html;  ./make.py latex_forced
```
- Check the output, check whatsnew and release.rst for formatting errors.
run build_dist.sh, uninstall cython and make a test install of the tarball to ensure
cython is not required for installation from tarball/pypi (now also part of travis scripts, but check).
Ensure you have SSH access to pandas.pydata.org (@TomAugspurger or Andy Terrel should be able to get you an account).
Ensure you have upload rights to PyPI (Wes or Jeff)
Ensure you have commit rights on github.com/macPython/pandas-wheels/ and github.com/conda-forge/pandas-feedstock

Release Candidate

These are items that must be done for an RC. See the respective sections for details.

The Release

Final Pre-Release

To be done just before tagging the commit. This is optional for release-candidates.

Call a hold on master in the pre-release issue before you start.
Finalize the documentation by running the
- Run scripts/announce.py and add to release.rst
- Update the release date in release.rst and whatsnew/<version>.rst
Commit the doc changes

Tag the Release

Create an empty commit: git commit --allow-empty -m 'RLS: v0.21.0'
- For Release-Candidates, it's -m 'RLS: v0.21.0.rc1'
Create the tag: git tag -a v0.21.0 -m "Version 0.21.0"

Test the Release

Some local sanity tests before pushing, making the release official

Build the release
- Conda:

# build a conda package
conda build ./conda.recipe --numpy 1.11 --python 2.7 -q --output

# fresh conda env
conda create -n pandas_0.18.2_test python=2.7 pandas nose

# build and install into that env
conda install -n pandas_0.18.2_test pandas --use-local

# switch to that env & test
source activate pandas_0.18.2_test
import pandas as pd
pd.test()

Pip

# build the wheel
python setup.py bdist wheel

# activate another environment
workon pandas-test
pip uninstall -y pandas
pip install <path/to/wheel>
python -c "import pandas; pandas.test()"

Build the docs. Check the version number and release notes

Push the Release

This is where things become final. No going back once the tagged commit is pushed

Push the release
- Major release: git push upstream master --follow-tags
- Maintenance: git push upstream 0.20.x --follow-tags

Build Source Distribution

python setup.py sdist --formats=gztar (TODO: update build_dist.sh to handle maintenance branches)

Release on Github

Draft a new release here
If this is an RC, check the box saying it's a pre-release
Upload the pandas-<version>.tar.gz from the previous step as a "binary"
Generate the SHA256 for the release with openssl dgst -sha256 dist/pandas-<version>.tar.gz
We use the SHA from our .tar.gz since the one auto-generated by Github isn't stable if the release page
is later modified.

Build Binary Distributions

Windows wheels are built automatically by Christoph Gohlke.

Open a PR for Mac and Linux wheels to update the BUILD_COMMIT here
Final Release Only Open a PR for the pandas-feedstock
- Update the version
- Update the SHA256
- Set the build number 5to 0

Download Binary Distributions

Conda-Forge is handled automatically. We handle wheels.

Windows wheels from here
Mac / Linux wheels with python scripts/fetch_wheels.py, from here

Upload to PyPI

Upload the source and wheels simultaneously. Uploading just the source may break some users'
workflows, if they trigger an update but don't have a C-complier

Assuming you've downloaded everything to dist: twine upload dist/*
You may want to hide the release until the docs are up and announcement is sent out

Build and Upload the Docs

Build and upload

cd docs;
./make.py clean; ./make.py; ./make.py zip_html;  ./make.py latex_forced
python make.py upload 0.20.2

Update the Website

clone the repo: https://github.com/pandas-dev/pandas-website.git
Update index.rst to have the most recent release, and the prior major release.
Update _tehems/pydata/layout.html to have the most recent minor release of each major version
commit and push
build: python make.py html
upload: python upload.py # TODO: verify this works
Verify it updated (maybe in an incognito browser to avoid caching)

Announce

Make sure everything is in place
- pypi binaries for windows, mac wheels, source
- docs built and published
- website updated
Send an ANN to the mailing list. include links to both release notes and whatsnew.
send announcements to [email protected], [email protected], [email protected], [email protected], [email protected]

** Final Version**

** Release Candidates **

Start the next release cycle

Only after a major release

When finishing a major release we have a few extra steps to ensure that
the development version is always ahead of the backports version.

Create a backports branch off the release commmit, e.g. git checkout -b 0.22.x master
Push that to upstream: git push -u upstream 0.22.x
Switch back to master and make a new commit to start the next cycle
- git checkout master
- git commit --allow-empty -m 'DEV: Start 0.22 cycle'
- git tag -a v0.22.0.dev0 -m 'Version 0.22.0 start'
- git push upstream master --follow-tags
Ensure that a whatsnew file is available for the next major and minor releases

Finish

Close the Milestone on github
Close the RLS issue

The text was updated successfully, but these errors were encountered:

jreback · 2017-09-23T21:18:30Z

I had to add moto to the deps, but looks like should be ok now: https://travis-ci.org/MacPython/pandas-wheels/builds/279039055

jreback · 2017-09-23T21:19:20Z

filter for 0.21.0. issues: https://github.com/pandas-dev/pandas/issues?q=is%3Aopen+is%3Aissue+milestone%3A0.21.0

bunch o deprecations remain. I am going to do a few, but anyone who can help out would be great!

jreback · 2017-09-27T11:47:08Z

We have quite a number of PRs that are marked for 0.21 that need to go in. so we should delay at least thru next week before RC1.

TomAugspurger · 2017-09-27T12:57:58Z

Sounds good.

…

On Wed, Sep 27, 2017 at 6:47 AM, Jeff Reback ***@***.***> wrote: We have quite a number of PRs that are marked for 0.21 that need to go in. so we should delay at least thru next week before RC1. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17285 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIizc2Gewm4PqiUI6HYWqlDWjSIPlks5smjXBgaJpZM4O78lR> .

TomAugspurger · 2017-10-10T15:45:12Z

Which of the remaining issues in https://github.com/pandas-dev/pandas/milestone/46 do we want to block the RC?

DEPR: Deprecate the convert parameter completely #17831
Bug: On Python 3 to_csv() encoding defaults to ascii if the dataframe contains special characters. #17097 (waiting on CI to finish)

Anything else?

TomAugspurger · 2017-10-10T15:45:20Z

cc @jorisvandenbossche

jorisvandenbossche · 2017-10-10T15:51:14Z

I would like to have #17728 in.

I think #17833 would be good to do as well and not too much work I think.

gfyoung · 2017-10-10T16:08:43Z

@jorisvandenbossche : I think we should also try to adjust the read_csv deprecations if we can before the release as well.

jorisvandenbossche · 2017-10-10T16:35:32Z

Do we have a idea of the timeline? (when has the release manager time?)
@TomAugspurger you were planning to do the relaese? (or @jreback, or me?)

Another thing we should think about is when we start branching. For the previous releases we only branched some time after the final release, but that means we should hold off merging clean-up PRs for a while (like IMO all the cython lib restructuring ones). We could also do the branching earlier this time.

TomAugspurger · 2017-10-10T16:43:35Z

I'm able to do the release. Anytime this week works for me.

jreback · 2017-10-10T22:22:40Z

we need the reindex fixes in before you do the RC1. I think hold off on anything that is not in the final until after actual release. Should tag nothing else as 0.21.0 while we are doing the RC.

TomAugspurger · 2017-10-10T22:38:19Z

I think hold off on anything that is not in the final until after actual release.

Agreed.

Should tag nothing else as 0.21.0 while we are doing the RC.

You're saying not merge anything into master? That's OK by me, although branching 0.21.x and branching isn't a problem either.

jreback · 2017-10-13T01:55:49Z

yeah just wanted to clear the queue. We should only mark/merge things for 0.21.0 at this point that are either bug fixes or don't impact the rc (e.g. realatively small things). we can branch on release I think (sure you can do it earlier, but then you have to port any changes, no big deal really, but might be cleaner w/o branching).

FYI call the next branch 0.22dev (as we don't then commit to a 1.0 wholesale :>)

TomAugspurger · 2017-10-13T01:57:04Z

Sounds good (I'm doing release stuff now).

Was about to open an issue but maybe you know. Should I be able to pip install the sdist without Cython in the environment I'm installing into? I'll post the error message in a sec (trying it again).

TomAugspurger · 2017-10-13T02:09:25Z

Nvm, seems to be working fine. Not sure what I did earlier.

jreback · 2017-10-13T02:37:46Z

we actually do test this (that’s the build_test in ci)

TomAugspurger · 2017-10-23T10:48:04Z

How are things looking for a release this week? Have any blockers come through that I've missed?

jreback · 2017-10-23T11:27:43Z

looks ok to me, nothing has come up. though that actually means that no-one is actually testing it!

@jorisvandenbossche

TomAugspurger · 2017-10-23T11:33:53Z

I'll have a bit of time to dig into #17861 today.

jreback · 2017-10-24T10:21:46Z

btw, when you tag, do so for 0.22.0-dev (and FYI, I think we had an issue about the actual format of the tag itself)

jreback · 2017-10-25T10:43:07Z

should add create the new doc templates as well (0.22 and 0.21.1)

TomAugspurger · 2017-10-26T18:01:27Z

Added doc templates in #17992

I have PRs #17982 and #17966 to fix the two outstanding PRs.

Joris has #17954

I'm going to kick off a run of the benchmarks on our benchmark server to get the PRs merged this morning.

Then I think we're on track for a release tomorrow.

TomAugspurger · 2017-10-27T11:52:09Z

OK, I believe everything is in. I'll start the release process in a little bit.

jorisvandenbossche · 2017-10-27T12:08:54Z

Can you give a ping at the moment you are doing the actual RLS commit ?
(was still doing some doc clean-up, but not necessarily that critical)

TomAugspurger · 2017-10-27T12:11:57Z

No rush, I'm still doing a doc build + local testing, and I'm about to go offline for ~1 hour. I'll wait to do the release commit until I hear from you.

jorisvandenbossche · 2017-10-27T12:17:17Z

There is still an "allowed failures" failing: TestCompression.test_compression_warns_when_decompress_caches_zlib. But it seems this is just because it causes a deprecation warning that some test is failing.

jorisvandenbossche · 2017-10-27T12:20:36Z

BTW, offline testing of the doc build is not a good idea in my experience :-) (you get a lot of errors for things that need internet access, so you don't really see the real errors)

TomAugspurger · 2017-10-27T12:23:10Z

@jorisvandenbossche fix for the numpy dev warnings at #17987

I think our next point release will be before numpy's next release, so we should be OK waiting.

jorisvandenbossche · 2017-10-27T12:51:50Z

I am working on this one: #18001

TomAugspurger · 2017-10-27T15:22:47Z

OK #18001 is merged. Tagging.

TomAugspurger · 2017-10-27T18:30:37Z

Alrighty I pushed a 0.21.x branch and tagged 0.22.0.dev0. Should be good to merge to master.

I noticed that matplotlib has a backport bot. I'll reach out to the Jupyter people that wrote it to see if we can use it too :)

TomAugspurger · 2017-10-27T18:32:08Z

Travis OSX jobs seem to be taking a while, so I may not be able to send out the announce email until later tonight or tomorrow.

TomAugspurger · 2017-10-28T18:55:57Z

Conda forge finished, so those will be trickling in over the next day or so.

Pushed to PyPi and updated the website.

Thanks everyone!

jreback · 2017-10-28T19:18:58Z

thanks @TomAugspurger and @jorisvandenbossche awesome job!

jreback · 2017-10-28T19:19:51Z

@TomAugspurger maybe update the release checklist with your checkboxes (here)? (or maybe better is a link to this issue). So that next time can just copy paste them.

TomAugspurger · 2017-10-28T19:35:01Z

The wiki should be up to date with everything that's here. I tried to remember to change it in both places if I had to modify a step.

…

On Sat, Oct 28, 2017 at 2:19 PM, Jeff Reback ***@***.***> wrote: @TomAugspurger <https://github.com/tomaugspurger> maybe update the release checklist with your checkboxes (here)? (or maybe better is a link to this issue). So that next time can just copy paste them. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17285 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHImNao0KompZCETxr0Ku_XpMznOH7ks5sw35bgaJpZM4O78lR> .

jorisvandenbossche · 2017-10-28T19:37:59Z

Thanks a lot for the release! Great job!

TomAugspurger added the Release label Aug 18, 2017

TomAugspurger added this to the 0.21.0 milestone Aug 18, 2017

TomAugspurger closed this as completed Oct 28, 2017

RLS: 0.21.0 #17285

RLS: 0.21.0 #17285

Comments

TomAugspurger commented Aug 18, 2017 • edited Loading

Pre-Release

Release Candidate

The Release

Final Pre-Release

Tag the Release

Test the Release

Push the Release

Build Source Distribution

Release on Github

Build Binary Distributions

Download Binary Distributions

Upload to PyPI

Build and Upload the Docs

Update the Website

Announce

Start the next release cycle

Finish

jreback commented Sep 23, 2017

jreback commented Sep 23, 2017

jreback commented Sep 27, 2017

TomAugspurger commented Sep 27, 2017 via email

TomAugspurger commented Oct 10, 2017

TomAugspurger commented Oct 10, 2017

jorisvandenbossche commented Oct 10, 2017 • edited Loading

gfyoung commented Oct 10, 2017

jorisvandenbossche commented Oct 10, 2017

TomAugspurger commented Oct 10, 2017

jreback commented Oct 10, 2017

TomAugspurger commented Oct 10, 2017 • edited Loading

jreback commented Oct 13, 2017

TomAugspurger commented Oct 13, 2017 • edited Loading

TomAugspurger commented Oct 13, 2017

jreback commented Oct 13, 2017

TomAugspurger commented Oct 23, 2017

jreback commented Oct 23, 2017

TomAugspurger commented Oct 23, 2017

jreback commented Oct 24, 2017

jreback commented Oct 25, 2017

TomAugspurger commented Oct 26, 2017

TomAugspurger commented Oct 27, 2017

jorisvandenbossche commented Oct 27, 2017

TomAugspurger commented Oct 27, 2017

jorisvandenbossche commented Oct 27, 2017

jorisvandenbossche commented Oct 27, 2017 • edited Loading

TomAugspurger commented Oct 27, 2017

jorisvandenbossche commented Oct 27, 2017

TomAugspurger commented Oct 27, 2017

TomAugspurger commented Oct 27, 2017

TomAugspurger commented Oct 27, 2017

TomAugspurger commented Oct 28, 2017

jreback commented Oct 28, 2017

jreback commented Oct 28, 2017

TomAugspurger commented Oct 28, 2017 via email

jorisvandenbossche commented Oct 28, 2017

TomAugspurger commented Aug 18, 2017 •

edited

Loading

jorisvandenbossche commented Oct 10, 2017 •

edited

Loading

TomAugspurger commented Oct 10, 2017 •

edited

Loading

TomAugspurger commented Oct 13, 2017 •

edited

Loading

jorisvandenbossche commented Oct 27, 2017 •

edited

Loading