Kyuubi Release Guide

Introduction

The Apache Kyuubi (Incubating) project periodically declares and publishes releases. A release is one or more packages of the project artifact(s) that are approved for general public distribution and use. They may come with various degrees of caveat regarding their perceived quality and potential for change, such as “alpha”, “beta”, “incubating”, “stable”, etc.

The Kyuubi community treats releases with great importance. They are a public face of the project and most users interact with the project only through the releases. Releases are signed off by the entire Kyuubi community in a public vote.

Each release is executed by a Release Manager, who is selected among the Kyuubi committers. This document describes the process that the Release Manager follows to perform a release. Any changes to this process should be discussed and adopted on the dev mailing list.

Please remember that publishing software has legal consequences. This guide complements the foundation-wide Product Release Policy and Release Distribution Policy.

Overview

The release process consists of several steps:

  1. Decide to release
  2. Prepare for the release
  3. Cut branch iff for major release
  4. Build a release candidate
  5. Vote on the release candidate
  6. If necessary, fix any issues and go back to step 3.
  7. Finalize the release
  8. Promote the release

Decide to release

Deciding to release and selecting a Release Manager is the first step of the release process. This is a consensus-based decision of the entire community.

Anybody can propose a release on the dev mailing list, giving a solid argument and nominating a committer as the Release Manager (including themselves). There’s no formal process, no vote requirements, and no timing requirements. Any objections should be resolved by consensus before starting the release.

In general, the community prefers to have a rotating set of 1-2 Release Managers. Keeping a small core set of managers allows enough people to build expertise in this area and improve processes over time, without Release Managers needing to re-learn the processes for each release. That said, if you are a committer interested in serving the community in this way, please reach out to the community on the dev mailing list.

Checklist to proceed to the next step

  1. Community agrees to release
  2. Community selects a Release Manager

Prepare for the release

Before your first release, you should perform one-time configuration steps. This will set up your security keys for signing the release and access to various release repositories.

One-time setup instructions

ASF authentication

The environments ASF_USERNAME and ASF_PASSWORD have been used in several places and several times in the release process, you can either one-time set up them in ~/.bashrc or ~/.zshrc, or export them in terminal everytime.

  1. export ASF_USERNAME=<your apache username>
  2. export ASF_PASSWORD=<your apache password>

Subversion

Besides on git, svn is also required for Apache release, please refer to https://www.apache.org/dev/version-control.html#https-svn for details.

GPG Key

You need to have a GPG key to sign the release artifacts. Please be aware of the ASF-wide release signing guidelines. If you don’t have a GPG key associated with your Apache account, please create one according to the guidelines.

Determine your Apache GPG Key and Key ID, as follows:

  1. gpg --list-keys --keyid-format SHORT

This will list your GPG keys. One of these should reflect your Apache account, for example:

  1. pub   rsa4096 2021-08-30 [SC]
  2.       8FC8075E1FDC303276C676EE8001952629BCC75D
  3. uid           [ultimate] Cheng Pan <chengpan@apache.org>
  4. sub   rsa4096 2021-08-30 [E]

Here, the key ID is the 8-digit hex string in the pub line: 29BCC75D.

To export the PGP public key, using:

  1. gpg --armor --export 29BCC75D

The last step is to update the KEYS file with your code signing key https://www.apache.org/dev/openpgp.html#export-public-key

  1. svn checkout --depth=files "https://dist.apache.org/repos/dist/dev/incubator/kyuubi" work/svn-kyuubi
  2. (gpg --list-sigs "${ASF_USERNAME}@apache.org" && gpg --export --armor "${ASF_USERNAME}@apache.org") >> KEYS
  3. svn commit --username "${ASF_USERNAME}" --password "${ASF_PASSWORD}" --message "Update KEYS" work/svn-kyuubi

Cut branch iff for major release

Kyuubi use version pattern {MAJOR_VERSION}.{MINOR_VERSION}.{PATCH_VERSION}[-{OPTIONAL_SUFFIX}], e.g. 1.3.0-incubating. Major Release means MAJOR_VERSION or MINOR_VERSION changed, and Patch Release means PATCH_VERSION changed.

The main step towards preparing a major release is to create a release branch. This is done via standard Git branching mechanism and should be announced to the community once the branch is created.

The release branch pattern is branch-{MAJOR_VERSION}.{MINOR_VERSION}, e.g. branch-1.3.

After cutting release branch, don’t forget bump version in master branch.

Build a release candidate

  1. Set environment variables.
  1. export RELEASE_VERSION=<release version, e.g. 1.3.0-incubating>
  2. export RELEASE_RC_NO=<RC number, e.g. 0>
  1. Bump version.
  1. build/mvn versions:set -DgenerateBackupPoms=false \
  2.   -DnewVersion="${RELEASE_VERSION}" \
  3.   -Pkubernetes,kyuubi-extension-spark-3-1,spark-block-cleaner,tpcds
  5. git commit -am "[RELEASE] Bump ${RELEASE_VERSION}"
  1. Create a git tag for the release candidate.

The tag pattern is v${RELEASE_VERSION}-rc${RELEASE_RC_NO}, e.g. v1.3.0-incubating-rc0

  1. Package the release binaries & sources, and upload them to the Apache staging SVN repo. Publish jars to the Apache staging Maven repo.
  1. build/release/release.sh publish

Vote on the release candidate

The release voting takes place on the Apache Kyuubi (Incubating) developers list (the (P)PMC is voting).

  • If possible, attach a draft of the release notes with the email.
  • Recommend represent voting closing time in UTC format.
  • Make sure the email is in text format and the links are correct

Once the vote is done, you should also send out a summary email with the totals, with a subject that looks something like [VOTE][RESULT] ….

Finalize the Release

Be Careful!

THIS STEP IS IRREVERSIBLE so make sure you selected the correct staging repository. Once you move the artifacts into the release folder, they cannot be removed.

After the vote passes, to upload the binaries to Apache mirrors, you move the binaries from dev directory (this should be where they are voted) to release directory. This “moving” is the only way you can add stuff to the actual release directory. (Note: only (P)PMC members can move to release directory)

Move the sub-directory in “dev” to the corresponding directory in “release”. If you’ve added your signing key to the KEYS file, also update the release copy.

  1. build/release/release.sh finalize

Verify that the resources are present in https://www.apache.org/dist/incubator/kyuubi/. It may take a while for them to be visible. This will be mirrored throughout the Apache network.

For Maven Central Repository, you can Release from the Apache Nexus Repository Manager. Log in, open Staging Repositories, find the one voted on, select and click Release and confirm. If successful, it should show up under https://repository.apache.org/content/repositories/releases/org/apache/kyuubi/ and the same under https://repository.apache.org/content/groups/maven-staging-group/org/apache/kyuubi/ (look for the correct release version). After some time this will be sync’d to Maven Central automatically.

Promote the release

Update Website

TODO

Create an Announcement

Once everything is working create an announcement on the website and then send an e-mail to the mailing list.

Enjoy an adult beverage of your choice, and congratulations on making a Kyuubi release.