Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Next Release? #677

Closed
13 tasks done
esc opened this issue Apr 3, 2014 · 133 comments
Closed
13 tasks done

Next Release? #677

esc opened this issue Apr 3, 2014 · 133 comments
Milestone

Comments

@esc
Copy link
Contributor

esc commented Apr 3, 2014

How does everyone feel about cutting a new release? I think it's high time, there are many new features and several bugs that have been fixed. The questions that nag me is how to release? I couldn't find any maintainers notes anywhere and there are many small details to be taken care of. Let me sketch some bullet-points from my previous experience releasing software, these can serve as a skeleton to be expanded.

  • mark any pull-requests / issues that should or need to be taken care of
  • take care of these pull-requests
  • update the release notes
  • update the changelog and commit to master
  • update the version number and and commit to master 0.9.x
  • tag the release (ideally using a signed Git tag)
  • push the master
  • push the release tag
  • bump version on master to next development version and push
  • upload to PyPI
  • generate documentation and upload (to where?)
  • new website (https://github.com/graphite-project/graphite-project.github.io/issues)
  • announce release (twitter?)

Some things to note:

  • Only critical bug fixes and no new features will be merged to master as soon as we start cutting the release
  • We can of course do the "release candidate" thing, where you cut two or three release candidates which you make available for testing for a week or so, that can help to stabilize the release.
  • Do we want to release only graphite-web or also ceres/whisper/carbon?
  • Lastly, the release procedure and all the things that were done will be written down and committed after the release to help with future releases.
@brutasse
Copy link
Member

brutasse commented Apr 3, 2014

👍

It'd be nice to have a release branch, like with 0.9.X. That way you create the branch and can do all release-related things there (RC, etc) without having to freeze master. When fixing things, fix in master and backport to the release branch. I fear that some issues will reappear with the release as some fixes only made it to 0.9.X and not master.

Regarding the docs, they are built automatically on readthedocs: http://graphite.readthedocs.org/ (someone with enough permissions please replace the project URL with that, the repo home on GitHub still points to wikidot). So IMO no action is required here. Just mark the 0.10.0 release notes as final when cutting the release.

Carbon & whisper releases are needed as well, with some pull-requests cleanup on these projects. Ceres has never been released so far so I don't know what the status is.

@esc
Copy link
Contributor Author

esc commented Apr 3, 2014

I would prefer to prep and cut the release from master. Release branches are useful if you want to continue development during a release time. Also, I think it is simpler to just use master, no need to backport or forwardport.

Regarding the project url, I have no ownership over th organization. I think it needs to be someone who does have that. @obfuscurity would you do use the honours? :)

@obfuscurity
Copy link
Member

Yes, I think we all agreed some time ago to focus on cutting our next major release (0.10) from master. I'd like to see any glaring fixes applied to 0.9.x but we need to put that cow to pasture, so to speak.

I'm happy to tag a pre-release candidate but I feel like we need to comb through the issues and PRs for any glaring fixes that should be included in a RC.

@esc esc modified the milestone: 0.10.0 Apr 3, 2014
@esc
Copy link
Contributor Author

esc commented Apr 10, 2014

@obfuscurity I agree that as many glaring things should be applied as possible. However, there are really a ton of PRs and Issues and sometimes it is better to release (ship) something that is adequate, rather than wait until it is perfect.

For sorting issues, I would suggest to use the 0.10 milestone to flag anything that should absolutely go into 0.10:

https://github.com/graphite-project/graphite-web/issues?milestone=2&page=1&state=open

So currently there are 8 issues remaining. Does that mean: a) there are only 8 left to go? or b) that we still need to sift through the remaining 278 issues and 72 PRs?

In the Numpy project the releases usually start with a request on the mailinglist, for anyone to speak up about issues that they thing should absolutely be resolved before the next release. This is nice, because it doesn't require the maintainers to sift through each and every one of the issues, which in the case of graphite(-web) is quite a daunting task.

So, perhaps we can find a common ground where we say, the following issues will be fixed within a pre-defined time-period that we can commit to, and everything else will be sorted later. Also, for any major issues, security, grave functionality impediments, we can always push maintenance releases of the form 0.10.X.

@timbunce
Copy link

I strongly agree that getting into a regular release cycle is more important that worrying too much about combing the backlog.

Some suggestions that I've seen work elsewhere:

  • Work out your branching strategy (e.g. separate development and production branches) and corresponding version numbering strategy
  • Get someone who's done it before (ideally) to make a release and document all the steps carefully.
  • Plan to make one development release a month, plus production bug fix releases if needed.
  • Ask for a list of volunteers to act as a 'release manager' for just one of the upcoming development releases. They'd just need to follow the script. There's less pressure since they're only development releases and their only commitment is for a single release.

What I've seen happen before is that each volunteer will polish and automate the process a little.
Soon you'll have a release process that takes very little effort plus a pool of people with experience making releases.

The key point is that working on smoothing out the release process is an investment that will be more than paid back. Once momentum is restored to the project more people will be motivated to contribute and the backlog will be unblocked.

@steve-dave
Copy link
Member

At this stage I won't comment on how to structure releases. There are a lot of good ideas here but I'm new to the project and wouldn't feel confident in suggesting how to go forward in that respect.

However, I would like to see at least one more 0.9.x release. As far as I know it is considered the "stable" release, but currently Graphite things just don't work if 0.9.x is checked out from all 3 repos. I have opened a number of PRs across the 3 repos that get the branch to the point where it does run and it would be great if they could be merged and then a new, possibly final, 0.9.x release tagged. I have one more 0.9.x branch in the works that doesn't have a PR here, but it is only a backport of a performance improvement so it wouldn't matter so much if it wasn't accepted here - my main interest is in getting 0.9.x across all three repos running out of the box.

It's entirely possible that master is almost ready for a 0.10.x, but I'm not sure that it would be ready to be the new "stable" release as it has quite a bit of new functionality in it that may not be complete or well tested. For this reason, IMHO, it would be best to keep 0.9.x going at least there is a new "stable".

Thanks!

@obfuscurity
Copy link
Member

I would love to see a 0.9.x release come out soon and then have someone step up to take over the project leadership. Sadly I don't have much time to work on the project these days and I don't expect that to change anytime soon. Personally, I see projects like graphite-api and cyanite as the future of the Graphite community, but clearly a lot of folks would like to see something official come out of the official project for their short-term upgrade path.

Is anyone willing to help massage the next release into shape?

@steve-dave
Copy link
Member

I am happy to be involved in the next (which obviously may be the final) release(s) of 0.9.x, but I'm not familiar enough with master to know if I have the bandwidth to be (heavily) involved in 0.10.x. I have a number of PRs pending for 0.9.x (various fixes and backports to get the branch back to running out of the box) that could be the makings of 0.9.13-pre and I'd be happy investigate bug reports relating to the pre-release once it has been tagged. Of course I'd be happy to help review any other PRs that it is thought should be included as well, but perhaps anything other than bugfixes could be postponed till at least 0.9.14?

@SEJeff
Copy link
Member

SEJeff commented Jun 19, 2014

I'll have from Saturday through Thursday evenings completely uninterrupted to go through and merge a lot of PRs. I'll make a special point to look through all of your outstanding ones. Sorry but we've been busy with real life things unfortunately.

@steve-dave
Copy link
Member

Thanks Jeff. Totally understand the real life side of it, was more offering help than expecting anything. Once again, if I can help, let me know.

@esc
Copy link
Contributor Author

esc commented Jun 20, 2014

I gather that it isn't clear if we should release a 0.9.x maintenance release or/and a 0.10.x release. We have been using the current master (plus some additional feature branches) in production since April w/o any issues. What are you guys using in production?

The problem with having a 0.9.x and a 0.10.x simultaneously that it is a bit of a split brain situation, and having to supply two branches with fixes and updates is somewhat of a maintenance nightmare.

As you can see here:

zsh» git log --oneline master..0.9.x | wc -l
200
zsh» git log --oneline 0.9.x..master | wc -l
641

the two branches have diverged quite a
lot. Note however that this does not take into account any commits that have been cherry picked.

If we do decide to to go for a 0.9.x, for which I do sense some momentum, we should use the 0.9.13 milestone, for which there are already some issues:

https://github.com/graphite-project/graphite-web/issues?milestone=4&page=1&state=open

@esc
Copy link
Contributor Author

esc commented Jun 20, 2014

BTW: tests for 0.9.x are currently passing:

https://travis-ci.org/graphite-project/graphite-web/builds/19817617

Also, we should favor bugfixes only for 0.9.x as @steve-dave suggested and additionally reject anything that isn't tested in one way or another.

@brutasse
Copy link
Member

Some fixes even made it to 0.9.x but not to master. I tried to resync things once but gave up… +1 for bugfix-only branches.

@steve-dave
Copy link
Member

I agree it's quite concerning how far the branches have diverged. I still feel that 0.9.x should be maintained until 0.10.x is released and becomes the official stable, perhaps put a line in the sand at 0.10.1 or 0.10.2? Any fixes that are applicable to both branches should be applied to both branches until 0.9.x is final (certainly not just to 0.9.x) - I'm not sure the preferred procedure for this but I have just been opening a PR for each branch rather than leaving it to the person that merges to have to port it. If someone else doesn't get to it first I will review the fixes that have been applied to 0.9.x and port them to master where applicable.

@drawks
Copy link
Member

drawks commented Jun 20, 2014

The repo organization aspect of this is really the most difficult part to sort out in my opinion. I'm not sure how to dig out of the current mess, but the end result should look like:

  1. Active development is all based against Master and no feature branches should be maintained in the official git project.
  2. Major versions should cut be cut into separate maint. branches for continued support in the form of bugfixes and once they are ~2 majors behind they should either be abandoned or be kept around only for critical security fixes.
  3. All PRs for merge into master should be squashed into single commits and should include a concise description of their intended behavior AND a unit test to match. Only PRs which are pure refactors of code which is already covered by existing tests should be accepted without tests.
  4. Some sort of automated/scripted procedure should be adopted for cutting new versions which should handle the work of updating all relevant version strings AND ideally updating a CHANGELOG document which would be initially synthesized from commit messages, but should be massaged into something readable by whoever is cutting the release.
  5. All of the above along with any suggested workflow should be codified in a CONTRIBUTING file at the base of the project to provide guidance for contributors. Additionally we might consider adding a FAQ section to this document to address questions which have been asked and answered before i.e. Why isn't this project PEP8 style compliant?

edit: This is all just a suggestion from my point of view and one that I've made before in other channels. None of the above should be construed as me dictating new policy or assuming a role of leadership in the project; I've got too much other stuff going on to bite off a bigger chunk of graphite.

@torkelo
Copy link
Contributor

torkelo commented Jun 22, 2014

I would like to help any way I can. Mostly I would like to create a new home page for graphite on a domain like graphite.org (or similar), with updated install guides, links to read the docs, download links, links to graphite-api etc. And usage screenshots & diagrams to show of what graphite can do.

@obfuscurity
Copy link
Member

I had intended to build up something similar to Sensu's website for Graphite but never got around to it. I have the graphiteapp.org domain registered if anyone wants to use that.

@esc
Copy link
Contributor Author

esc commented Jun 23, 2014

I agree that cleaning up the repo structure in the long run is something to aim at. Also having a nice a website under a good domain-name is something to aspire to. However, given the recent dilemma we are in, I would like to focus on getting a release out the door that fulfills the minimum requirements -- something like a minimum viable release.

From what I gather here, maybe a 0.9.13 release would be a good interim solution, but I still don't have an idea what you guys use in production. Anyway, to move towards that release, we will only merge bug-fixes that have tests.

@steve-dave you said you had some pending pull requests? I would like to take up your offer of looking into those for the 0.9.x release. Perhaps you could point the people with appropriate privileges that are reading this thread: (@esc, @obfuscurity, @SEJeff, @drawks. others?) to any bugfixes you would like to have included. Of course this also goes for anyone else reading this thread, who has commits that they feel need to be applied to 0.9.x

From a maintainers / release managers point of view, here is a small checklist for evaluating if a commit is suitable:

  • [] can the commit be merged / cherry-picked cleanly to 0.9.x?
  • [] is it a bug-fix?
  • [] does it have tests?
  • [] releationship to master? Is it backported from master? Does it need to be applied there?

If you encounter a commit which you can not deal with immediately, please set the 0.9.13 milestone, to make sure we have a list of things that need to be done:

https://github.com/graphite-project/graphite-web/issues?milestone=4&page=1&state=open

Also, again, what do you guys use in production?

Also does anyone feel like we need, or would prefer to have a mailinglist for this project. I sometimes miss being able to answer in-line when discussing with github issues?

@steve-dave
Copy link
Member

@esc FWIW we are currently running the 0.9.x branch in production with a handful of patches that I have created pull requests for to be considered for merging.

Would you like the 0.9.13 candidate pull requests highlighted here in this thread, tag those with permissions in the actual PRs, or highlight them in some other way? I apologise if this is obvious, but I am new to both this project and Github. Also, should I include carbon and whisper PRs or are they handled by different people?

@esc
Copy link
Contributor Author

esc commented Jun 23, 2014

No problem, by highlighting, i meant, just writing the name, e.g. @esc in the issue. That way you will draw mine (our) attention to this pull-request so that it can be considered.

Regarding whisper and carbon, I realized earlier today, that the current 0.9.x branch of graphite-web actually needs a whisper from its master. This would indicate that maybe new releases of both projects would also be required.

@esc
Copy link
Contributor Author

esc commented Jun 23, 2014

@steve-dave not sure yet who or how whisper and carbon will be handled.

@steve-dave
Copy link
Member

@esc Is there a deadline or target date? I've lost a couple of days to miscellaneous issues and haven't been able to go through the PRs yet...

@esc
Copy link
Contributor Author

esc commented Jun 25, 2014

No, as long as we keep momentum, it will be ready when it is ready. Importantly, motion towards the release should not stagnate.

@obfuscurity
Copy link
Member

Our docs builds have been failing on RTD since Aug 13. @gwaldo is this something you can help take a look at?

@gwaldo
Copy link
Member

gwaldo commented Oct 13, 2015

Sure, I'll try to take a look today.

On Sun, Oct 11, 2015 at 11:19 PM, Jason Dixon [email protected]
wrote:

Our docs builds have been failing on RTD since Aug 13
https://readthedocs.org/projects/graphite/builds/3236334/. @gwaldo
https://github.com/gwaldo is this something you can help take a look at?


Reply to this email directly or view it on GitHub
#677 (comment)
.

@gwaldo
Copy link
Member

gwaldo commented Oct 13, 2015

Looking into this for a little bit, the most plain errors are /opt/graphite can't be created and that django isn't being imported.

Looking at the last build to succeed, it looks like a very different style of information being presented.

I just signed up for a readthedocs.org account. Does anything need to be done to add me to the RTD for Graphite so that I can see what options are available to manage it or engage with the RTD team?

@obfuscurity
Copy link
Member

@gwaldo Added you as a project admin in RTD.

@obfuscurity
Copy link
Member

Note that aside from the RTD docs build problem, graphite-project/whisper#136 is our last showstopper bug before the next 0.9.x release.

@obfuscurity
Copy link
Member

Anyone have time to look at that RTD docs build failure for 0.9.x yet? That's the only blocker at this point.

@deniszh
Copy link
Member

deniszh commented Oct 25, 2015

It looks like I found something, but I do not know how to fix it properly. I'm absolute novice in RTD, so...
NFI

It seems that pip on RTD now honors setup.cfg settings, if it's located in same directory - that's why when RTD tries to run
...pip install --exists-action=w -rdocs/requirements.txt it tries to write to '/opt/graphite' (because we still have

[install]
prefix=/opt/graphite

in setup.cfg for 0.9.x - and fails.
@obfuscurity tried to fix that with 8539ebb, but problem that installing docs dep's step running before running pip setup.py install...
I found no ways how we can change pip behaviour, so, we need to change setup.py for 0.9.x to similar setup as we have in master. I'll create PR for that.
Please test.

@obfuscurity
Copy link
Member

its_fine

@torkelo
Copy link
Contributor

torkelo commented Oct 26, 2015

yay!

@obfuscurity
Copy link
Member

I don't think there's anything holding up the 0.9.14 release except for tagging and building pypi packages. I made some small improvements to the docs the other day to address what I saw was a gap for the installation process (specifically, setting up the db). We still intend to rollout a new website, but it's not ready yet, and I don't want it to hold up the new release any longer.

Aside from those things, anything else I'm overlooking?

@JeanFred
Copy link
Member

JeanFred commented Nov 5, 2015

Can’t think of anything @obfuscurity :)

@bmhatfield
Copy link
Member

Let 'er rip!

@gwaldo
Copy link
Member

gwaldo commented Nov 5, 2015

There is a significant gut-and-rebuild of the docs that I want to do, but there's no reason to hold this up. I'll be happy to see this release.

@obfuscurity
Copy link
Member

I've bumped the setup.py versions, tagged the 0.9.14 release, built new source distributions, and uploaded the releases to PyPi for graphite-web, carbon, and whisper. There needs to be a formal writeup somewhere (or better yet, automated), but for the time being here are the steps I performed:

  • bump the version in setup.py
  • git tag the new version
  • push both to origin
  • launched a new Synthesize vagrant (this clones and checks out the 0.9.14 release)
  • inside the checkout for each, sudo python setup.py sdist
  • extract the PKG_INFO file from each tarball
  • login to https://pypi.python.org/pypi as myself (project admin)
  • for each project, submit PKG_INFO to https://pypi.python.org/pypi?%3Aaction=submit_form
  • upload source distribution tarball for each release ("files")

@obfuscurity
Copy link
Member

I would appreciate testing and feedback of the 0.9.14 pypi packages.

P.S. Note that we have recently resumed work on the new website. With this will come the shutdown of the old wikidot site, improved docs (including "quickstarts"), and other goodies. But I felt it was in the project's best interest to not hold back 0.9.14 any longer. We will aim to ship the new site ASAP, but if it happened to coincide with a new release from master, I think that would be cool too. 😉

P.P.S. Thanks to everyone's hard work and persistence for helping make this release a reality.

@obfuscurity
Copy link
Member

P.P.P.S. I've also hidden the 0.9.13-pre1 packages on PyPi.

@jraby
Copy link

jraby commented Nov 7, 2015

Congrats!

I'll test this out next week.
On Nov 7, 2015 1:19 AM, "Jason Dixon" [email protected] wrote:

P.P.P.S. I've also hidden the 0.9.13-pre1 packages on PyPi.


Reply to this email directly or view it on GitHub
#677 (comment)
.

@bmhatfield
Copy link
Member

We're also looking to bring our install up to current as well, so I'll be upgrading over the next couple weeks as well.

@obfuscurity
Copy link
Member

Moving the discussion on fixture migrations over to #938.

@obfuscurity
Copy link
Member

Note that I'm going to hide the pypi packages for 0.9.14 until a suitable upgrade path is identified.

@obfuscurity
Copy link
Member

Disregard, it appears that the syncdb process for upgrading works fine. I'll update the docs.

@obfuscurity
Copy link
Member

Updated the release notes for upgrading the database and including the release date (today). Re-tagged the release and generated a new source dist. Had to switch formats from tar.gz to zip for PyPI to allow the re-uploaded file (didn't realize this was a thing until now). Tested that pip install still works as expected.

@obfuscurity
Copy link
Member

Ladies and gentlemen, Graphite 0.9.14 is officially released.

@obfuscurity
Copy link
Member

Announced on Twitter.

screen shot 2015-11-07 at 9 41 09 pm

@obfuscurity
Copy link
Member

For the sake of closing this issue out (oh dear god yes), I'm going to mark the "website" task complete even though it's clearly not. Trust me when I say we're working actively on this and I expect it to be out within the next 2-4 weeks.

@esc
Copy link
Contributor Author

esc commented Nov 8, 2015

Nice!

On 8 November 2015 03:43:36 CET, Jason Dixon [email protected] wrote:

Closed #677.


Reply to this email directly or view it on GitHub:
#677 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests