chore(dev/release): decouple version numbers #1762

lidavidm · 2024-04-25T03:32:31Z

Start refactoring our release process so that we can theoretically handle different version numbers in different language subprojects.

Fixes #1490.

kou · 2024-04-25T03:52:56Z

dev/release/versions.env

+VERSION_GLIB="1.0.0"
+VERSION_JAVA="0.12.0"
+# Because C++/Go/Python are effectively tied at the hip, they share a single
+# version number. Also covers Conda/Linux packages.
+# TODO: untie this so that we can have independent version #s for different drivers (at least for Postgres/SQLite)
+VERSION_NATIVE="1.0.0"


GLib/Ruby can also use VERSION_NATIVE.

lidavidm · 2024-04-25T06:36:32Z

Demo release: https://github.com/lidavidm/arrow-adbc/releases/tag/apache-arrow-adbc-2024.05-rc0

lidavidm · 2024-04-25T06:37:14Z

Python used the wrong version number.
Need to validate what version number the Linux packages used.

lidavidm · 2024-04-25T06:50:20Z

Linux packages look right, let's see about Python...

lidavidm · 2024-04-25T06:57:21Z

It appears our milestone assigner should still work

lidavidm@debian ~/C/arrow-adbc (versioning)> printf "maint-2024.05\nmaint-0.11.0" | grep -o '[0-9.]*$' | sort --version-sort
0.11.0
2024.05

lidavidm · 2024-04-25T08:49:24Z

@kou I did an initial pass over the release scripts...the main thing now will be having to bump all the versions by hand post-release but I think that's OK for now

kou

+1

kou · 2024-04-25T09:09:26Z

.github/workflows/packaging.yml

          tar xf apache-arrow-adbc-${{ steps.info.outputs.VERSION }}.tar.gz
+
+          source ./apache-arrow-adbc-${{ steps.info.outputs.VERSION }}/dev/release/versions.env
+
          mv apache-arrow-adbc-${{ steps.info.outputs.VERSION }} adbc
-          mv apache-arrow-adbc-${{ steps.info.outputs.VERSION }}.tar.gz adbc/ci/linux-packages/


It's not related to this PR but we can remove echo "VERSION=${VERSION}" >> "$GITHUB_OUTPUT" and always use ${VERSION} instead of ${{ steps.info.outputs.VERSION }}.

ok, let's see if I did that right...

kou · 2024-04-25T09:12:20Z

.github/workflows/packaging.yml

          mv apache-arrow-adbc-${{ steps.info.outputs.VERSION }} adbc
-          mv apache-arrow-adbc-${{ steps.info.outputs.VERSION }}.tar.gz adbc/ci/linux-packages/
+          # Need to align the file path with the version number (and not the release)


Ah, we use apache-arrow-adbc-YYYY.MM tag after this PR?
(VERSION is YYYY.MM and VERSION_NATIVE is 1.0.0, right?)

Yup. Though I'm undecided if it should be MM or just a counter.

ah, well, if we add the day below then it will be a date and that sounds good to me.

dev/release/utils-prepare.sh

kou · 2024-04-25T09:21:20Z

dev/release/versions.env

+
+# The release as a whole has a date-based identifier.  This is used to
+# identify tags, branches, and so on.
+RELEASE="2024.05"


We may want to add date too. (e.g. 2024.05.29)
We may want to publish multiple versions in a month.

dev/release/03-source.sh

kou · 2024-04-25T09:29:53Z

.github/workflows/packaging.yml

-          mv apache-arrow-adbc-${{ steps.info.outputs.VERSION }}.tar.gz adbc/ci/linux-packages/
+          # Need to align the file path with the version number (and not the release)
+          cp -r adbc apache-arrow-adbc-${VERSION_NATIVE}
+          tar czf adbc/ci/linux-packages/apache-arrow-adbc-${VERSION_NATIVE}.tar.gz apache-arrow-adbc-${VERSION_NATIVE}


I can update ci/linux-packages/ to use apache-arrow-adbc-${RELEASE}.tar.gz instead of apache-arrow-adbc-${VERSION_NATIVE}.tar.gz.
Should I do this in this branch or in a separated PR?

I don't mind if you want to push to this branch!

OK! I'll push to this branch!

Fixes apache#1490.

Co-authored-by: Sutou Kouhei <[email protected]>

Because we use apache-arrow-adbc-YYYY.MM.DD.

kou · 2024-04-25T20:36:00Z

dev/release/versions.env

+
+# The release as a whole has a date-based identifier.  This is used to
+# identify tags, branches, and so on.
+RELEASE="2024.05.06"


Hmm. It may be better that we omit preceding 0: 2024.5.6
This is not a semantic versioning but semver https://pypi.org/project/semver/ can't parse 2024.05.06 format:

>>> import semver >>> semver.VersionInfo.parse('1.02.03') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3/dist-packages/semver.py", line 656, in parse raise ValueError("%s is not valid SemVer string" % version) ValueError: 1.02.03 is not valid SemVer string

(Sorry. The above transcript uses old semver, 2.10.2.)

Ok! I'll take another look this afternoon

What do you think of just using a counter in that case then? (2024.1, 2024.2, 2024.3)

What do you think of just using a counter in that case then? (2024.1, 2024.2, 2024.3)

It will work too!

It may be confused but we may be able to omit the 2024. part: 1, 2, 3

We need to choose the next identifier when we publish a new release. If we use date-based identifier, it may be difficult. For example, we want to publish a next release in this year but it may be slipped to the next year.

(Should we choose the next identifier when we publish the next release?)

Yeah, I think we can always adjust the identifier at publication time so it will be OK if we have to change it. We can always make the scripts a bit more flexible in that case too so we can update the version at any time.

But yeah, in that case I think having the identifier just be 1, 2, 3, makes more sense.

I suppose there might be a little confusion for our first release. But after that we will have release = 2, version number = 1.1.0/0.13.0 which should be OK.

Ok, I went with "12.0.0" since this will be our 12th release.

Ah...the version in the docs will look a little funny. I suppose that's OK for now. We can always label the docs with a date or something later.

Can we use "12" instead of "12.0.0" because we will always use ".0.0" for the part.

kou · 2024-04-26T04:34:58Z

dev/release/versions.env

+# specific language governing permissions and limitations
+# under the License.
+
+# The release as a whole has a date-based identifier.  This is used to


Could you update the "date-based" part?

lidavidm · 2024-04-26T04:40:15Z

It appears the Debian build is still having issues?

https://github.com/lidavidm/arrow-adbc/actions/runs/8842860533/job/24282183949

kou · 2024-04-26T04:51:36Z

Hmm. "1.0.0" not "12.0.0" as the version:

https://github.com/lidavidm/arrow-adbc/actions/runs/8842860533/job/24282184256#step:9:2483

(expected one of apache-arrow-adbc_1.0.0.orig.tar.gz, apache-arrow-adbc_1.0.0.orig.tar.bz2,

I'll take a look at it.

dev/release/versions.env

Co-authored-by: Sutou Kouhei <[email protected]>

kou · 2024-04-26T04:55:59Z

https://github.com/lidavidm/arrow-adbc/blob/ab50f798307221e854ac0611dece70e814e3cebe/.github/workflows/packaging.yml#L300-L302

This is related.
This workflow assumes that we update versions in ci/linux-packages/{debian/changelog,yum/apache-arrow-spec.in} before tagging.

kou · 2024-04-26T04:58:20Z

Here: https://github.com/lidavidm/arrow-adbc/blob/ab50f798307221e854ac0611dece70e814e3cebe/dev/release/utils-prepare.sh#L115-L118

Could you run dev/release/01-prepare.sh before tagging?

lidavidm · 2024-04-26T05:00:45Z

Hmm, I did run 01-prepare.sh. I'll try again.

lidavidm · 2024-04-26T05:03:43Z

Trying again: https://github.com/lidavidm/arrow-adbc/actions/runs/8843345685

It looks like debian/changelog was updated: lidavidm@31d283e#diff-79fdeab51b21b35cb8b6b6544ca232fd4919a6b0e32f3551bbd94a044454b7e6

lidavidm · 2024-04-26T05:15:57Z

Hmm, no luck.

kou · 2024-04-26T05:16:57Z

It should be 12-1 not 1.0.0-1.

lidavidm · 2024-04-26T05:17:45Z

Don't we want 1.0.0 as the actual version number of the package?

kou · 2024-04-26T05:18:14Z

diff --git a/dev/release/utils-prepare.sh b/dev/release/utils-prepare.sh
index dbbc531e5..acd5b7732 100644
--- a/dev/release/utils-prepare.sh
+++ b/dev/release/utils-prepare.sh
@@ -23,6 +23,7 @@ update_versions() {
 
   local conda_version="${VERSION_NATIVE}"
   local csharp_version="${VERSION_CSHARP}"
+  local linux_version="${RELEASE}"
   local rust_version="${VERSION_CSHARP}"
   case ${type} in
     release)
@@ -30,7 +31,6 @@ update_versions() {
       local docs_version="${RELEASE}"
       local glib_version="${VERSION_NATIVE}"
       local java_version="${VERSION_JAVA}"
-      local linux_version="${VERSION_NATIVE}"
       local py_version="${VERSION_NATIVE}"
       local r_version="${VERSION_R}"
       ;;
@@ -39,7 +39,6 @@ update_versions() {
       local docs_version="${RELEASE} (dev)"
       local glib_version="${VERSION_NATIVE}-SNAPSHOT"
       local java_version="${VERSION_JAVA}-SNAPSHOT"
-      local linux_version="${VERSION_NATIVE}-SNAPSHOT"
       local py_version="${VERSION_NATIVE}dev"
       local r_version="${VERSION_R}.9000"
       ;;

lidavidm · 2024-04-26T05:19:00Z

I'll give it a shot.

lidavidm · 2024-04-26T05:19:44Z

https://github.com/lidavidm/arrow-adbc/actions/runs/8843504428

kou · 2024-04-26T05:20:34Z

Ah, we will use RELEASE as package version because .deb/.rpm use source archive's version (apache-arrow-adbc-${RELEASE}.tar.gz) as their package version.

lidavidm · 2024-04-26T05:20:51Z

Ok, sounds good!

lidavidm · 2024-04-26T06:13:15Z

Looks like it passed. That was quicker than I expected. Thanks for the help @kou!

02-sign.sh worked: https://github.com/lidavidm/arrow-adbc/releases/tag/apache-arrow-adbc-12-rc0

I just pushed some other fixes after grepping for uses of $version or ${version} that were left over.

I expect during the release I will have to adjust some things on the fly, but that's OK.

dev/release/utils-prepare.sh

Co-authored-by: Sutou Kouhei <[email protected]>

Start refactoring our release process so that we can theoretically handle different version numbers in different language subprojects. Fixes apache#1490. --------- Co-authored-by: Sutou Kouhei <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]>

github-actions bot added this to the ADBC Libraries 1.0.0 milestone Apr 25, 2024

lidavidm force-pushed the versioning branch from 051d886 to b72ee85 Compare April 25, 2024 03:41

kou reviewed Apr 25, 2024

View reviewed changes

lidavidm force-pushed the versioning branch from b72ee85 to 837ca16 Compare April 25, 2024 03:59

lidavidm mentioned this pull request Apr 25, 2024

feat: 1.0.0 libraries release (tracking issue) #1490

Closed

26 tasks

lidavidm force-pushed the versioning branch 4 times, most recently from 443a254 to ddd5b80 Compare April 25, 2024 05:14

lidavidm force-pushed the versioning branch from ddd5b80 to 1f8ed5a Compare April 25, 2024 06:50

lidavidm force-pushed the versioning branch from 1f8ed5a to 5399217 Compare April 25, 2024 08:44

kou approved these changes Apr 25, 2024

View reviewed changes

lidavidm and others added 4 commits April 25, 2024 06:20

chore(dev/release): decouple version numbers

761cde1

Fixes apache#1490.

Update dev/release/03-source.sh

20eb207

Co-authored-by: Sutou Kouhei <[email protected]>

Update dev/release/utils-prepare.sh

f7b08ac

Co-authored-by: Sutou Kouhei <[email protected]>

remove GITHUB_OUTPUT

cdfc124

lidavidm force-pushed the versioning branch from 94f2489 to cdfc124 Compare April 25, 2024 10:20

lidavidm and others added 3 commits April 25, 2024 06:20

put day in release

1dd0936

linux-packages: Use YYYY.MM.DD as source version

78c8449

Because we use apache-arrow-adbc-YYYY.MM.DD.

Use RELEASE as version

4b4e231

kou reviewed Apr 25, 2024

View reviewed changes

kou and others added 4 commits April 26, 2024 05:43

mv is needed

865bbcf

Remove unused ids

9aa9e53

Use VERSION

b9073a5

set release to just be a counter

418e88b

kou reviewed Apr 26, 2024

View reviewed changes

update

82efd4f

kou reviewed Apr 26, 2024

View reviewed changes

dev/release/versions.env Outdated Show resolved Hide resolved

lidavidm and others added 2 commits April 26, 2024 00:52

Update dev/release/versions.env

4925741

Co-authored-by: Sutou Kouhei <[email protected]>

update

ab50f79

give this a try

aff0e18

lidavidm added 3 commits April 26, 2024 02:05

Fix 03-source

4e1ca2f

More fixes

151b774

More fixes

a13cd67

kou reviewed Apr 26, 2024

View reviewed changes

dev/release/utils-prepare.sh Outdated Show resolved Hide resolved

Update dev/release/utils-prepare.sh

05c806b

Co-authored-by: Sutou Kouhei <[email protected]>

cocoa-xu mentioned this pull request Apr 26, 2024

fix(go/adbc/driver/flightsql): should use ctx.Err().Error() #1769

Merged

lidavidm merged commit 7402f9a into apache:main Apr 29, 2024
55 checks passed

lidavidm deleted the versioning branch April 29, 2024 02:14

chore(dev/release): decouple version numbers #1762

chore(dev/release): decouple version numbers #1762

Conversation

lidavidm commented Apr 25, 2024

Choose a reason for hiding this comment

lidavidm commented Apr 25, 2024

lidavidm commented Apr 25, 2024

lidavidm commented Apr 25, 2024

lidavidm commented Apr 25, 2024

lidavidm commented Apr 25, 2024

kou left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lidavidm commented Apr 26, 2024

kou commented Apr 26, 2024

kou commented Apr 26, 2024

kou commented Apr 26, 2024

lidavidm commented Apr 26, 2024

lidavidm commented Apr 26, 2024

lidavidm commented Apr 26, 2024

kou commented Apr 26, 2024

lidavidm commented Apr 26, 2024

kou commented Apr 26, 2024

lidavidm commented Apr 26, 2024

lidavidm commented Apr 26, 2024

kou commented Apr 26, 2024 • edited Loading

lidavidm commented Apr 26, 2024

lidavidm commented Apr 26, 2024

kou commented Apr 26, 2024 •

edited

Loading