splitting version numbers for targets #3010
Comments
|
I do think you should reserve the first three numbers for base Antlr. Given your project cadence, 4.x -> 5.x basically means "we wrote a new book", so you need the other two numbers for normal software engineering purposes. To version the runtimes separately, the conventions that spring to mind are:
Maybe the least confusing, but the least conventional, would be to give each runtime their own letter codes, so it would be 4.9.0-js1 for patchlevel 1 in the JavaScript runtime, and 4.9.0-swift1 for patchlevel 1 in the Swift runtime. Whichever you prefer, I think people are generally used to the idea that the fourth number onwards is a bit subtle, so I don't think having skew between runtimes in the fourth number is a big problem. I do think that 4.9.x being different between runtimes would be more likely to confuse people. |
|
Why not simply do a quick follow up release? If everything is automated it should not mean much extra work for you. If not then it is about time to make everything in the release process automatic. That's the general approach these days in the software industry anyway: release quick, release often. |
|
As the PHP target is maintained under another repo to avoid these issues,
feel free to decide whatever makes sense to you guys.
…On Sat, Dec 19, 2020 at 07:01 Mike Lischke ***@***.***> wrote:
Why not simply do a quick follow up release? If everything is automated it
should not mean much extra work for you. If not then it is about time to
make everything in the release process automatic. That's the general
approach these days in the software industry anyway: release quick, release
often.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3010 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAHGHPDEQ6OBYP74PMMQRE3SVR2YRANCNFSM4VBXPYUA>
.
|
|
@mike-lischke unfortunately not everything is automated. I have to go through and update a lot of stuff manually (api docs, website, examples, ...) and the process seems to break every time I do it. maven continues to confuse the crap out of me. there is the issue with builds failing randomly due to timeout. There is the issue that I have to wait for appveyor to give me a decent build just so I can publish C#. I have to publish quite a few of the targets manually. granted I try to write all this down, but it definitely has a high transaction cost is my point 👍 |
|
@ewanmellor I absolutely suggested 4.9.0-js1 type version numbers but @ericvergnaud found out that NPM and a bunch of other target publishing stuff will have a problem with anything that isn't major.minor.patch :( this is a really tough problem if you ask me. I can't think of a great solution |
|
Yes, that's true, NPM is going to need the third digit to change for the package management to do its job properly. It sounds like you're going to need to move the middle digit for every platform release, there's really no other way. |
|
A somewhat cumbersome way of limiting the damage could be to agree on a convention, where multi-target patch numbers increase by 10, whereas target-specific ones increase by 1. |
|
@parrt Most of the problems seem to stem from the runtimes, as they require environments and knowledge you don't have (why should you?). ANTLR4 itself doesn't depend on any runtime, but it makes sense, of course, to bundle the Java runtime with it. The individual runtimes are separate and can be released on their own. This would require to set up ANTLR4 repos for each runtime where only the tests specific for the particular runtime are executed. You would then remove the runtimes from the main repo and release both runtimes + tool separately. They are 2 very different pieces anyway. The only tricky part here is to have a shared source of all tests, so that you don't have to maintain them in several places (maybe a git submodule?). Your task would only be to take care for the tool + Java target. Target authors have to take care for their language and can publish new releases at any time, independently. Version numbers would not correspond between tool and targets. Also, target authors could be allowed to merge patches on their own and don't need your intervention. Github supports releases for a given repo and that would be your source of the runtimes when you want to publish them on the ANTLR4 homepage. But people can, at any time download a previously released version on a brand new one, even if you didn't do the full release yet. How does that sound to you? |
|
@mike-lischke Yep. I considered splitting apart the repository into multiple, one per target, but it's kind of drastic. And, at the beginning, I made the decision to put them all into one for ease of testing and to keep the runtimes and the tool consistent. Ha ha :) @ericvergnaud not a bad idea. What if we took it further and made a digit for the runtime, but that would limit us to 10 runtimes. or maybe we use hexadecimal digits so we get 16 targets total? not sure npm or other tools could handle the hexadecimal though. I am strongly motivated to delegate control of the targets to the respective maintainers, but I'm concerned about tool-target version matching for consistency. We also have to modify all of the testing infrastructure, but that is workable if we assume that all target repositories follow a naming scheme and are at the same level as the antlr repo. The git submodule thing really was driving me crazy for the PHP runtime; I literally am not smart enough to use git it to its fullest (or maven for that matter. haha). |
|
I'm pretty sure hex version numbers is going to be a non-starter. Git submodules shouldn't be a problem. It's definitely a model you have to sit down and learn, but it's a rational model once you get it. Maven is just unalloyed garbage, but then all the Java build systems are bad, so it's not like switching is going to make you happy. The independent repos idea scares me a bit in this case, given how intertwined all the tests stuff is. You could do it, but I think you'll have to consider antlr-runtime-test-framework as a real package with a real interface with real dependents, so that each of the runtimes could consume it without constantly breaking. |
|
I feel that splitting patch versions doesn't mean that we have to separate the repos. It will be a major effort to update the testing framework, and require more effort to coordinate minor version upgrade if we separate repos and ownership. Dart runtime can be released independently already. Just bump the patch version in pubspec.yaml and publish a release. Though that might cause confusion for users who sees inconsistent versions. So I don't see how separating repos can make it easier.
Right now major + minor version is used to check codegen and runtime compatibility. I think it still makes sense to make a release if the new codegen doesn't work for the old runtime. The 4.9 release for Dart probably isn't necessary under that criteria haha. |
|
@lingyv-li seems what cannot be automated is publishing from a CI, and the problem mentioned is pretty much the same for any target (I also publish some other stuff to GitHub and Maven using a local script which reads non-committed files containing credentials). |
|
Yes, it cannot be fully automated especially the dart publish command literally opens a web page to login to the Google account. In npm for example, you can use access tokens which can be stored securely in the CI. I imagine the ideal state to be using tools like semantic-release or auto, which can make the version bump and release fully automated in CI with PR labels. Though that's not possible right now at least for Dart runtime, but we should try to get as close as we can. |
|
Hiya. Ok, it seems like we are converging on not splitting the repository, so at least that some progress. :) The human script that I use an update is here for releasing: https://github.com/antlr/antlr4/blob/master/doc/releasing-antlr.md A lot of it has to do with publishing the targets, which I guess is what we are trying to separate out. All I know is that every time I try to go through script, that something is different and I have all sorts of problems with different websites and CI and using maven properly and updating submodules etc.. As for submodules, I'm afraid I'm not willing to put the effort in the learn that crap. git is already shit in my mind and I don't want any more pollution. ;) Especially something I will never use again or elsewhere. Maybe you guys can have some ideas after looking at that release script, but it still leaves the primary problem of version numbers. |
|
Speaking of CI and automation as we were, it looks like the Antlr CI is still using travis.org, which is being shut down in 8 days: https://mailchi.mp/3d439eeb1098/travis-ciorg-is-moving-to-travis-cicom?e=%5BUNIQID%5D |
|
Migrating to travis.com actually... And from the message I got there this should happen automatically. |
|
Maybe it makes sense to migrate to GitHub Actions. |
|
I think we are already migrated to the .com: |
|
wow. I can't seem to add antlr org to travis-ci.com. I must be getting dumber by the week. it seems that I can add my parrt account but not the antlr organization, at least not properly. I think it said it queued removal of some old stuff so I will let this sit for a day. @KvanTTT Where do the github actions run? Do we have to pay for it? |
|
I expect Travis themselves would migrate accounts?
Worth checking with them if that’s the case?
… Le 26 déc. 2020 à 04:26, Terence Parr ***@***.***> a écrit :
wow. I can't seem to add antlr org to travis-ci.com. I must be getting dumber by the week.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#3010 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJHARR2B27O25CZ6ACDSWTYO3ANCNFSM4VBXPYUA>.
|
|
I also had trouble when I opened my Github account in travis.com. Not all of my repos are listed there and instead they list now all the repos from the organizations I'm in, which is definitely not what I want. So I removed everything from there and now I'm waiting that travis.org is automatically migrated to travis.com. |
They work on GitHub servers and available on the "Actions" tab in a repository. It's free, @teverett has already integrated GitHub actions into grammars repository: https://github.com/antlr/grammars-v4/actions. He knows better how to integrate. |
|
@mike-lischke ok, that's what I'll do though I think I messed that up by disconnecting too much stuff between github and travis. :( |
|
Ah. I think i figured it out. shitty interface on github. antlr/antlr4 is hooked up to travis-ci.com now. |
|
Grammars-v4 is now using GH Actions.
… Terence Parr ***@***.***>
December 26, 2020 at 11:03 AM
Ah. I think i figured it out. shitty interface on github. antlr/antlr4
is hooked up to travis-ci.com now.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3010 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJLKZVF6JBBEZGWOYFSLXLSWYQOJANCNFSM4VBXPYUA>.
|
|
@teverett so can it do the complex builds we do for antlr tool with testing etc...? I see travis-ci.org still connected to the grammars-v4 repo, btw. this looks interesting: https://github.com/antlr/grammars-v4/actions Oh and i see this: https://github.com/antlr/grammars-v4/blob/master/.github/workflows/main.yml boy that sure looks like our travis yml file. Can you tell us more how well it works? Compare to travis? |
|
I honestly don't know. Grammars-v4 was easy to port b/c it's basically
"run this maven build". I do end up paying GH once in a while, less
than $5 CDN for build capacity for grammars-v4.
… Terence Parr ***@***.***>
December 27, 2020 at 10:39 AM
@teverett <https://github.com/teverett> so can it do the complex
builds we do for antlr tool with testing etc...? I see travis-ci.org
still connected to the grammars-v4 repo, btw. this looks interesting:
https://github.com/antlr/grammars-v4/actions
Oh and i see this:
https://github.com/antlr/grammars-v4/blob/master/.github/workflows/main.yml
boy that sure looks like our travis yml file. Can you tell us more how
well it works? Compare to travis?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3010 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJLKZWK5MI6X4IFXURZ2XDSW5WL3ANCNFSM4VBXPYUA>.
|
|
Ok, thanks Tom. |
I also think that submodules using is a good idea. On one hand, they solve the problem of runtimes splitting, on another hand, they can be synchronized easily. I use UI for git (GitExtensions) and submodules representation is very straightforward there. Maybe using a console is a bit confusing if use submodules. |
|
You haven't pushed the most recent commit in the submodule. |
|
hahah. you sure? I could swear that I did do that. It also says that I am detached now when I go back into that directory. shit. $ git status |
|
Ok, so let's just add one more line to my human script which is to |
|
If the submodule isn't checked out to an particular branch (i.e. it is a "detached HEAD") then you need to tell it which branch to push to. Assuming that the correct branch is master, you probably want If you're going to be working on the submodule regularly, I'd check out the submodule onto the correct development branch (or master if that's right). |
|
at this point I'm going to remove it if I can find a way to do it safely. ok, I have unhooked it using the GUI (Fork, actually a very nice tool). |
|
I just realized that removing the submodule is going to break the testing of PHP as part of travis. On the other hand, it also has a build sequence that triggers upon push to its repository: https://travis-ci.com/github/antlr/antlr-php-runtime/jobs/467888291 |
|
Looks like swift is failing at travis: https://travis-ci.com/github/antlr/antlr4/jobs/467890059 (They seem to be working on my local mac though) |
Looks like it needs a version number bump somewhere. |
|
dang. ok. thanks. It's in there: Maybe that build was old? Nope. current. also weird it works locally. ah. i see one small booboo and will try again. |
|
Yes, I see your change, that's the right one I think. |
|
Crap. PHP tests are still attempted (on appveyor). trying to add php back and clone repo during install. heh, swift is looking better https://travis-ci.com/github/antlr/antlr4/builds/211707047 |
|
Travis: "Builds have been temporarily disabled for public repositories due to a negative credit balance. Please go to the Plan page to replenish your credit balance or alter your Consume paid credits for OSS setting." clicking on the links gives site error. hahaha. Damn. ok, we have now consumed all credits with a few hours playing: "-2810 available credits (purchase date: January 03, 2021) You have used 12810 of 10000 credits" |
|
Looks like travis is now out. github actions? anybody want to take a crack at that? @KvanTTT ?? |
|
I can try to integrate next week. |
|
https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing: "1 minute of mac build time costs 50 credits" So the free plan now gives 2.5 test runs per month for our Swift tests alone. Wow. It didn't even get a chance to run our extended test suite, which currently has macOS tests for .NET and C++ at approximately the same length. Even if you disabled the macOS tests entirely and just ran the Linux ones, I think you're looking at 7 or 8 full test runs a month before the free plan caps out. |
|
Does this mean free Travis is going away, effectively? That would be a bummer. |
|
It’s not going away, but the limitations make it unusable for ANTLR.
… Le 4 janv. 2021 à 15:31, Mike Lischke ***@***.***> a écrit :
Does this mean free Travis is going away, effectively? That would be a bummer.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#3010 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJBZ4VUL26QD42IJGWTSYFVGLANCNFSM4VBXPYUA>.
|
|
I'll have to look at github actions (costs money). Can we get away with just AppVeyor? |
|
I'm reading this on migrating to github actions: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/migrating-from-travis-ci-to-github-actions |
|
The problem I have with GitHub actions is the long history of Microsoft trapping people into a free product and then suddenly making it expensive…
(appreciate that’s exactly what just happened with Travis CI on Mac)
I think the cost of migrating our CI from Travis to any CI will be small, rather I would suggest we focus on identifying the right one.
Travis, Github, AppVeyor, Gitlab all offer Linux/Wintel/MacOS builds (and there’s probably others out there)
I’d suggest, once 4.9.1 is out that we list them all and do a qui pros-and-cons. Happy to drive that effort.
… Le 5 janv. 2021 à 07:07, Terence Parr ***@***.***> a écrit :
I'm reading this on migrating to github actions: ***@***.***/actions/learn-github-actions/migrating-from-travis-ci-to-github-actions ***@***.***/actions/learn-github-actions/migrating-from-travis-ci-to-github-actions>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#3010 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJAB4YJNCB37P5N6UFTSYJCZXANCNFSM4VBXPYUA>.
|
|
@ericvergnaud it would be great if you could leave that discussion after I push out 4.9.1 thanks |
|
Ok, finishing up maven release and will update website etc... soon. closing this. |
|
Anybody know npm? |
|
Have you run "npm install" first? |
|
hahah. I just figure that out and added it to the documentation after a web search. ugh. thanks Mike! |
|
folks, we have 4.9.1 out the door :) thanks for the help!! |
I don't think that we should be biased against any vendor. In this case, we should migrate to GitLab or another service because GitHub is also under Microsoft. In the end, any company tries to earn more money and there is no "Corporation of Good" in the modern world (even Google is not such a corporation). By the way, GitHub private repositories became free after GitHub had been acquired by Microsoft. If Microsoft or any other company became paid or expensive, we could switch to another service. Also, I think we can try to ask for some advanced resources for free because ANTLR is a valuable project. |
|
@parrt congrats on 4.9.1. the test plugin and grammars-v4 are updated. |
Hiya: @pboyer, @mike-lischke, @janyou, @ewanmellor, @hanjoes, @ericvergnaud, @lingyv-li, @marcospassos
Eric has raised the point that it would be nice to be able to make quick patches to the various runtimes; e.g., there is a stopping bug now in the JavaScript target. He proposes something along these lines:
This is in optimal as people have criticized me in the past for bumping, say, 4.6 to 4.7 for some minor changes. It also has the problem that 4.9.x will not mean the same thing in two different targets possibly, as each target will now have their own version number.
Rather than break up all of the targets into separate repositories or similar, can you guys think of a better solution? Any suggestions? The goal here is to allow more rapid target releases, and independent of me having to do a major release of the tool.
The text was updated successfully, but these errors were encountered: