HOWTO-RELEASE

	Notes on Preparing a GDAL Source Release
	========================================


Prerequisites:

1) Check that the release is ready to go as far as ABI (binary compatibility)
   is concerned. This can be checked by comparing the installed headers of the
   candidate release with the installed headers of the previous release
   (diff -ur $(OLD_INSTALL_DIR)/include $(NEW_INSTALL_DIR)/include). The API
   is defined as all functions and classes exported by the CPL_DLL keyword.

  - For major and minor releases, there must be no function signature change
    for the C API. Only new functions are allowed.

  - For major releases, the allowed changes in C++ API should (or must?) be
    such that user calling C++ code can still compile against new headers
    without modification (existing methods can become virtual, default
    arguments can be added, new methods or members can be added)

  - For minor releases (1.6.1 versus 1.6.0), the C++ ABI stability must be
    preserved : no method signature change, no addition of virtual methods, no
    new members. Only non-virtual methods can be added.

    It may also be helpful to check:
      https://abi-laboratory.pro/tracker/timeline/gdal/

2) Delete existing fix_typos directory and run scripts/fix_typos.sh

Process :

1) "make completion" to regenerate scripts/gdal-bash-completion.sh
   if new command line switches have been added. scripts/completionFinder.py
   must also be edited before if new utilities/scripts are added/removed.

2) Update the release date, and number information in gcore/gdal_version.h.in
   (*NOT* gdal_version.h which is a generated file)
   Note: the format of GDAL_RELEASE_DATE should be YYYYMMDD.

3) Update two instances of year in CITATION file to the current year.

4) Update "version" and "date-released" in CITATION.cff (Note: the DOI does
   *not* need to be updated as it is a generic one)

5) Update the VERSION file (if not already done)

6) Update the version information in the following files:
    - ./swig/python/gdal-utils/osgeo_utils/__init__.py (gdal-utils python package)
    - ./swig/python/README.rst (libgdal)

7) Update the GDAL_SOVERSION number at top of gdal.cmake according to the
   directions given in the comment preceding it.

8) Prepare release overview in the NEWS.md file. For example, to get all changes
   from v3.4.0 to current HEAD
     git log --reverse -v v3.4.0..HEAD . ":(exclude)autotest" ":(exclude)doc" ":(exclude).github"

   For a feature release, e.g 3.6.0, you can use scripts/generate_log_feature_release.py to generate a log without commit that went in the previous 3.5.x bugfix releases with:
     python3 scripts/generate_log_feature_release.py

  - commit new version to NEWS.md file.

  - for bugfixes releases, forward port additions of NEWS.md to master

9) Update doc/source/about_no_title.rst, doc/source/download.rst and doc/source/download_past.rst
   to advertise the new release and link to the release notes. Do that in a dedicated commit.

   (It is necessary to do that at that stage, so that the ReadTheDocs "stable" branch, based on the latest tag, proposes to download it)

10) If this is a feature release (e.g 3.1), prepare a branch.

   git checkout master
   git pull origin master
   git checkout -b release/3.1
   git push origin release/3.1

11) Tag the release with a RC suffix:

   git checkout release/3.1
   git pull origin release/3.1
   git tag -a -m "Create tag v3.1.0RC1" v3.1.0RC1
   git push origin v3.1.0RC1

12) Prepare archives

12.0) Ensure you have the following prerequisites (beyond normal build deps):

   If make is not GNU make, e.g., export MAKE=gmake

   doxygen

   python available as "python", or e.g. export PYTHON=python3.9

   python3 available as "python3" in path (export PYTHON=python3.9
   partially works, but some scripts under doc have hardcoded python3)

   sphinx-build in path, or e.g. export SPHINXBUILD=sphinx-build-3.9

   breathe extension for sphinx

   swig 4 in path as "swig"

   md5sum (GNU version) in path (not a POSIX requirement)

12.1) Create the source distributions using the mkgdaldist.sh script.
   The argument should be the version number (i.e. 1.4.2). As our process involves
   doing betas or RCs, use the -rc option so that the filenames include this
   information (after promotion to official release, filename renaming will have
   to be done)

   % ./mkgdaldist.sh 3.1.0 -tag v3.1.0RC1 -rc rc1

   For a beta version:

   % ./mkgdaldist.sh 3.1.0beta1 -tag v3.1.0beta1

   For a version intended for testing packaging (and not for
   distribution):

   % ./mkgdaldist.sh 3.4.80 -tag master

   It might be useful to compare the content of the generated tarball, with
   a previous release (diff -Nur gdal-3.0.1 gdal-3.0.2).
   This is more easily doable for a bugfix release.

12.2) Create a snapshot of the documentation (only for feature releases)

    1. Refresh
       https://download.osgeo.org/gdal/for_doc/javadoc.zip
       by building the Java bindings, with -DGDAL_JAVA_GENERATE_JAVADOC=ON.
       javadoc.zip is in the swig/java subdirectory of the build directory.
    2. cd doc
    3. ./build_doc_snapshot 310
       This generates gdal310doc.zip

12.3) Publish the resulting files on download.osgeo.org,
    in /osgeo/download/gdal/X.Y.Z (where X.Y.Z is the version number)

    with ~/.ssh/config containing:

   Host osgeo7-*
        ProxyCommand ssh rouault@download.osgeo.org -W $(sed -e "s/^osgeo7-//;s/$/.lxd/" <<< "%h"):%p


13) Announce the release candidate availability to gdal-dev@
    Example: https://lists.osgeo.org/pipermail/gdal-dev/2019-June/050509.html

14) If new RC is needed, update gcore/gdal_version.h.in with the new date
    and update NEWS,and go to 7)

    If no new RC is needed, after a few days, raise a motion to gdal-dev@ for approval

15) Once the vote has positively completed,

15.1) Tag the release as final:

    git checkout v3.1.0RC1
    git tag -a -m "Create tag v3.1.0" v3.1.0
    git push origin v3.1.0

15.2) Log on download.osgeo.org, go to /osgeo/download/gdal/X.Y.Z

      Remove the RC suffixes for the final RC, like:
      mv gdal-3.8.3rc1.tar.xz gdal-3.8.3.tar.xz
      mv gdal-3.8.3rc1.tar.xz.md5 gdal-3.8.3.tar.xz.md5
      mv gdal-3.8.3rc1.tar.xz.sig gdal-3.8.3.tar.xz.sig
      mv gdal-3.8.3rc1.tar.gz gdal-3.8.3.tar.gz
      mv gdal-3.8.3rc1.tar.gz.md5 gdal-3.8.3.tar.gz.md5
      mv gdal-3.8.3rc1.tar.gz.sig gdal-3.8.3.tar.gz.sig
      mv gdal383rc1.zip gdal383.zip
      mv gdal383rc1.zip.md5 gdal383.zip.md5
      mv gdal383rc1.zip.sig gdal383.zip.sig
      mv gdalautotest-3.8.3rc1.tar.gz gdalautotest-3.8.3.tar.gz
      mv gdalautotest-3.8.3rc1.zip gdalautotest-3.8.3.zip

      And edit the .md5 file to remove the reference to RC in the names they contain
      vim gdal-3.8.3.tar.gz.md5
      vim gdal-3.8.3.tar.xz.md5
      vim gdal383.zip.md5

      Check that everything is fine:
      md5sum -c *.md5

15.3) In /osgeo/download/gdal, add a symlink from X.Y.Z to CURRENT (except for stable releases in a "old" branch).

  % ln -sf X.Y.Z CURRENT

16) Update GitHub to close the release milestone.
    Then create a new milestone for the next release.

17) Upload the new Python bindings to Pypi (requires upload rights to
the GDAL package by one of the current owners : HowardB/FrankW/EvenR)
( procedure taken from http://peterdowns.com/posts/first-time-with-pypi.html )

17.1) Prerequisite:
    a) Install twine https://pypi.org/project/twine/

    b) Create a $HOME/.pypirc file :

    [distutils] # this tells distutils what package indexes you can push to
    index-servers = pypi
                    pypitest

    [pypi] # authentication details for live PyPI
    repository: https://upload.pypi.org/legacy/
    username: yourlogin
    password: yourpassword

    [pypitest] # authentication details for test PyPI
    repository: https://test.pypi.org/legacy/
    username: yourlogin
    password: yourpassword

17.2) create gdal sdist and upload to pypi:

    a) Create a build directory if not already done. Go to it and run cmake

    b) cd $BUILD_DIR/swig/python

    c) python3 setup.py sdist

    d) Check the output (this assumes setuptools >= 65.3.0, otherwise the filename is GDAL-*.gz):
        twine check dist/gdal-*.gz

    e) For trial :
        twine upload dist/gdal-*.gz -r pypitest

    f) For real :
        twine upload dist/gdal-*.gz

17.3) create gdal-utils wheel and upload to pypi:

    a) cd $BUILD_DIR/swig/python/gdal-utils

    b) python3 setup.py bdist_wheel

    c) Check the output:
        twine check dist/gdal_utils-*.whl

    c) For trial :
        twine upload dist/gdal_utils-*.whl -r pypitest

    d) For real :
        twine upload dist/gdal_utils-*.whl

18) Generate signed maven artifacts with GPG for Java bindings.
    This step is required in order to deploy the
    maven artifacts to the central Maven repository. Before this step can
    proceed you must set up a signing key as described here:

      http://central.sonatype.org/pages/working-with-pgp-signatures.html

 a) Each developer can use their own signing key although it is usually best to
    avoid constantly using a different key for releases as users will need to
    import the public key in order to verify the artifacts.

    Here are a quick set of steps to generate a signing key key.

       gpg --gen-key
       gpg --list-keys (and note the key id)
       gpg --keyserver hkp://pool.sks-keyservers.net --send-keys <key id>

    See the above link for more details.

  b) Run CMake with the following variables:

       1. GPG_KEY: the identifier for the signing keys
       2. GPG_PASS: the passphrase for the signing key (optional)

  c) Build the bindings: make java_binding

    Upon success you should see a file named "swig/java/build/maven/bundle.jar"
    that contains all the maven artifacts with signatures. This file is what
    will be uploaded to maven central. See the next step.

19) Deploy maven artifacts to Maven central.

    NOTE: Before you can deploy to maven central you must set up an account
    in Sonatype JIRA. That can be done here:

        https://issues.sonatype.org/secure/Signup!default.jspa

    Once you have an account set up you must be associated with the gdal
    project. Create a ticket here asking to be associated with the project:

        https://issues.sonatype.org/browse/OSSRH

    The entire deployment process is described in detail here:

       http://central.sonatype.org/pages/manual-staging-bundle-creation-and-deployment.html
       http://central.sonatype.org/pages/releasing-the-deployment.html

    The following steps summarize the process.

a) Log into the Sonatype repository manager at https://oss.sonatype.org. Use the
   same credentials as your Sonatype JIRA account.

b) Once log in select "Staging Upload" on the left hand side.

c) Select "Artifact Bundle" under "Upload Mode" and then choose the "bundle.jar"
   created in the previous Step 22. Finally "Upload Bundle" to start the upload.

d) When the upload has been completed you will be notified that a staging
   repository has been created. Note the name of the repository. It should look
   something like "orggdal-100x".

e) From the left hand menu navigate to "Staging Repositories". In the search
   box look for the staging repository name you noted from the previous section.
   Or just search for "gdal". It should be obvious which repository is the
   current one.

f) Select the staging repository. If all is well You should see the option to
   "Release" (Located as a button near the top of the page). If not it means
   there was an issue with the bundle. Consult the "Activity" tab at the bottom
   of the page to find out why.

e) Click the "Release" button and that is it! The release should be available in
   Maven Central shortly. You can verify this by going to search.maven.org and
   searching for "gdal".

18) Create a release on GitHub
    Go to https://github.com/OSGeo/gdal/releases/new
    And create a release for the vX.Y.Z tag
    Attach the following files:
    - gdal-X.Y.Z.tar.gz
    - gdal-X.Y.Z.tar.gz.sig
    - gdal-X.Y.Z.tar.gz.md5
    - gdalXYZ.zip
    - gdalXYZ.zip.sig
    - gdalXYZ.zip.md5
    - gdalautotest-X.Y.Z.zip

20) Regenerate Docker images

    e.g
        ./docker/build-all.sh --with-multi-arch --release --tag 3.3.1 --gdal v3.3.1 --proj 8.1.0 --platform linux/arm64,linux/amd64 --push

    and update docker/README.md with the latest release

21) Announce release to :
    - major release: gdal-dev@lists.osgeo.org, gdal-announce@lists.osgeo.org, news_item@osgeo.org.
    - bugfix release: gdal-dev@lists.osgeo.org, gdal-announce@lists.osgeo.org

   Example:
    - bugfix release: https://lists.osgeo.org/pipermail/gdal-dev/2019-July/050543.html
    - feature release: https://lists.osgeo.org/pipermail/gdal-dev/2020-May/052117.html

Note: gdal-announce@ is moderated. Make sure that your email address is approved
('mod' tick disabled in https://lists.osgeo.org/mailman/admin/gdal-announce/members),
or your message manually approved, with an administrator of the list.

22) For a feature release: after it has been done, in the master branch, update
    the VERSION to the next one and in gcore/gdal_version.h.in, update
    GDAL_VERSION_MAJOR/_MINOR, GDAL_RELEASE_NAME (with a dev suffix, like "3.7.0dev"),
    and set GDAL_RELEASE_DATE to a date like {YEAR}9999.
    Update swig/python/gdal-utils/osgeo_utils/__init__.py to something like (3, 6, 99, 0)
    if master is 3.7.0dev

23) For bugfixes releases, forward port to master changes done in
    doc/source/about_no_title.rst, doc/source/download.rst and doc/source/download_past.rst

24) For a feature release, enable a new version in the ReadTheDocs administration panel.
    a) Go to https://readthedocs.org/projects/gdal/versions/
    b) In the "Activate a version" tab, enter "release/X.Y" in the text entry and click on the Filter button
    c) Click on the Activate button
    d) Go to https://readthedocs.org/projects/gdal/, and in the "Compile a version" drop-down list,
       select the newt "release-X.Y" label and click on the "Compile a version" button