Java Product Release Process

This document describes the process that the Shibboleth team uses to package up and release a new version of our Java software products.

Release Environment

All our Java products use Maven to perform the actual build of release artifacts. Those artifacts are then uploaded to our Nexus installation at https://build.shibboleth.net/nexus.

The Javadoc artifacts are now managed somewhat separately but essentially as a follow-on step to the release, and no longer relies on the Maven “site” plugin and are built much faster and more reliably than before.

Translation Import

Before releasing a project that contains i18n artifacts, please check that the latest files from the corresponding folder tree in the java-idp-translations repository are imported into the project’s repository. See also MessagesTranslation.

Initial Configuration

Prior to performing any releases a few one-time configurations need to be done.

Ensure that you have the proper data in your ~/.m2/settings.xml file.
Generate an SSH key for both Git and Site deploys.

Generate a PGP key and get it signed by the other developers. To avoid use of SHA-1 use the following in your gpg.conf file:

personal-digest-preferences SHA512
cert-digest-algo SHA512
default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed

Send the public SSH and PGP keys to committers email list with a request to be allowed to perform releases.

A systems admin will then:

Log in to Nexus and go the Security -> Users -> Add -> External User Role Mappingscreen
1. Enter the user's LDAP uid as their user name (same userid they use for SVN)
2. In the Role Management section, add the role Nexus Deployment and save the user account
Add an account for you on the server, set your SSH key as an authorized key, and allow you to su to the Shibboleth website user.
1. Add your key to the 'javasites' user, so that you can perform site releases.
Add your GPG key to the KEYS file on the download site

Parent POM Adjustment

In most cases, a major or minor release will involve tagging a new version of the parent POM and updating all projects to depend on it. The parent should not be updated for patch releases, this can cause Maven import confusion given that dependent projects (e.g. opensaml) will be referencing the older parent - see JPAR-175. Generally, for a given release, all projects should reference the same parent. If library updates are required for a patch release, you will need to override the dependencies in the appropriate project POMs. When in doubt, ask.

Ensure that your working copy is in sync with the project repository (i.e., pull down any updates and commit any local changes).
If not done, tag the parent POM to freeze the parent so it can be referenced by the subsequent steps.
1. The tagging process for this is identical (bar the project and versions) to the release process described in the section below.
Run the command java-parent-project/bin/lock-version-monolithic.sh (for a monolithic project} or java-parent-project/bin/lock-version-multimodule.sh (for a multi-module project). The arguments are the parent POM tag number, and the path to the project.
```
# Lock java-shib-shared against tag version 17 of V5 parent
java-parent-project/bin/lock-version-multimodule.sh 17 java-shib-shared
```
After verifying the project's build using the newly referenced parent POM, commit the project changes.

Release Process

Ensure you are using the correct Java version e.g. Amazon Corretto 17.
1. If you are building the IdP, make sure you set MAVEN_HOME to the bin directory of your maven installation so the distribution-dependency-test runs correctly in idp-installer.

Ensure that your working copy is in sync with the project repository (i.e., pull down any updates and commit any local changes). It can be useful to do this from scratch from a temporary folder, possibly inside a clean docker container. For example, into the temp directory, using a fresh maven repo location.

# incase you get issues connecting to the GPG agent.
export GPG_TTY=$(tty) 
# somewhere to store a fresh maven repo. If you want to preserve the transient release repo for archiving, etc, 
# use a permanent location for REPO, with the version in the name.
export REPO=/tmp/repo
cd /tmp
git clone git@git.shibboleth.net:<project>
cd <project>
git checkout main

Check that a non-snapshot version of maven-dist-enforcer and maven-dist-enforcer-data are used.
Check the .check-m2 file is in the projects distribution project.
Check the project builds, all the tests pass, and the artifacts are produced correctly (signed in their target directories):
1. ```
mvn -Dmaven.repo.local=$REPO -Prelease,sign clean verify
```
2. OPTIONALLY: If you want to manually inspect the signature report on the contents of the populated m2 directory, you need to use the install phase e.g.
  1. mvn -Dmaven.repo.local=$REPO -Prelease,sign clean install then vi target/m2SignatureReport.txt . This is automatically checked on deploy, so is not required.
Set the project version number in the POM to the release version number using the command below. Also, ensure you update versions of any dependencies on our own projects that should be shipped with the release.
```
mvn -Dmaven.repo.local=$REPO -DgenerateBackupPoms=false -DnewVersion=4.Y.Z versions:set
```

Once satisfied, tag the release using a name equal to the version number of the release (e.g., 4.1.4 etc.) and ensure the POM at the tip of the branch being released NEVER contains a non-SNAPSHOT version number.

# Be sure there are SNAPSHOTs in the right places. So from the root directory of the project:
find . -name 'pom.xml' -exec grep SNAPSHOT {} \;

# Check status
git status
# Confirm changes manually
git diff
# Assume working copy is on branch 'main', with ONLY tag-specific changes pending
# (e.g. non-SNAPSHOT version)
# Add the changes
git add -A
# Confirm changes have been added
git status
# Commit the changes to be tagged to the HEAD of the branch
git commit -m "Update files to be tagged for release"

# Create a signed, annotated tag from HEAD
git tag -s -m "Tag 4.Y.Z release" 4.Y.Z

Once tagged, set the project version number in the POM to the next SNAPSHOT increment.

# Edit pom.xml to update the version to X.Y.Z+1-SNAPSHOT (in the example, 4.1.5-SNAPSHOT).
mvn -Dmaven.repo.local=$REPO  versions:set -DnewVersion=4.Y.Z+1-SNAPSHOT -DgenerateBackupPoms=false

Add and commit the changes you made, and then push the tag.

# Commit the pom.xml change.
git add -A
git commit -m "Bump version after release"

# Push the new commits.
git push
# Do not push the tag.  Signature checking is more restrictive for non SNAPSHOT builds

Checkout or export the tag to build the final release:

git checkout 4.Y.Z
mvn -Dmaven.repo.local=$REPO -DskipTests -Prelease,sign clean deploy

After the build has been completed push the tag

# Push the new tag (no easy backout when the tag is pushed!)
git push origin 4.Y.Z

At this stage, DO NOT make any changes to fix errors without increasing the version again. This commits you to a release and you cannot go back.
Mark the version as released in Jira.

Post Product Release

Once all the projects have been released, unlock the first-party dependencies in each project by manually editing their POM files and updating any first-party project dependencies to their next SNAPSHOT increment.

cd /tmp
git clone git@git.shibboleth.net:<project>
cd <project>
git checkout main
vi pom.xml
# Updated project dependencies to next SNAPSHOT increment. 
git add -A
git commit -m "Unlock first-party dependencies"

Javadoc Release Process

So far as I know, this process does not require a separate repo, but it does assume you have variables defined for $MAVEN and $REPO to run, and as usual you have to be logged into the server for the Nexus deploy step to work.

Additionally, it will assume your SSH user name is $USER (from your shell) but may be overridden by setting $USERNAME.

A script can be run from the root of each project (where the individual project’s parent POM lives) to:

Clean, and then build the “aggregate” javadoc for that project, which should show up in the target/ folder.
Sign and upload the javadoc artifact to Nexus.
SSH to the server and execute a loopback curl command to unpack the “latest” release version of the docs for a project to the right place and delete the SNAPSHOT javadoc from that version.

The script pulls all of the relevant settings from the pom.xml file in the directory from which it is run and will display them, and offer invididual “default-safe” prompts at each step to allow you to abort.

$ cd java-shib-shared
$ ../java-parent-project/bin/build-and-deploy-javadoc.sh

Note that while the script is largely idempotent, once an artifact is uploaded to Nexus, it won’t allow you to overwrite that, so once that step is done, you can’t start over without intervention.

Final Steps

For top-level projects (IdP): Place the product distribution in the download section of the website by:

Log in to the Shibboleth webserver

Copy and rename the distribution archives and PGP signatures from the Nexus work directory to a temporary location e.g. your home directory

VERSION=4.X.Y

cp -v /home/nexus/sonatype-work/nexus/storage/releases/net/shibboleth/idp/idp-distribution/$VERSION/idp-distribution-$VERSION.tar.gz shibboleth-identity-provider-$VERSION.tar.gz
cp -v /home/nexus/sonatype-work/nexus/storage/releases/net/shibboleth/idp/idp-distribution/$VERSION/idp-distribution-$VERSION.tar.gz.asc shibboleth-identity-provider-$VERSION.tar.gz.asc
cp -v /home/nexus/sonatype-work/nexus/storage/releases/net/shibboleth/idp/idp-distribution/$VERSION/idp-distribution-$VERSION.zip shibboleth-identity-provider-$VERSION.zip
cp -v /home/nexus/sonatype-work/nexus/storage/releases/net/shibboleth/idp/idp-distribution/$VERSION/idp-distribution-$VERSION.zip.asc shibboleth-identity-provider-$VERSION.zip.asc

Create SHA256 checksums for both the zip and tar file

sha256sum shibboleth-identity-provider-$VERSION.tar.gz > shibboleth-identity-provider-$VERSION.tar.gz.sha256
sha256sum shibboleth-identity-provider-$VERSION.zip > shibboleth-identity-provider-$VERSION.zip.sha256
# Confirm the checksums are correct
sha256sum -c shibboleth-identity-provider-$VERSION.tar.gz.sha256
sha256sum -c shibboleth-identity-provider-$VERSION.zip.sha256

Create a suitable version-named directory on the download site and copy in the distribution, signatures, and checksums:

sudo mkdir -v -p /home/shibwww/html/downloads/identity-provider/.$VERSION
sudo cp -v -p shibboleth-identity-provider-$VERSION* /home/shibwww/html/downloads/identity-provider/.$VERSION/
sudo chown -v -R shibwww:shibwww /home/shibwww/html/downloads/identity-provider/.$VERSION

Copy in the Windows installers (which are created separately) - note these are being sources from Rod’s home directory, so the path needs checking.

sudo sh -c "cp -v /home/rdw/shibboleth-identity-provider-$VERSION-x64/shibboleth-identity-provider-$VERSION-x64.msi* /home/shibwww/html/downloads/identity-provider/.$VERSION/"
sudo chown -R shibwww:shibwww /home/shibwww/html/downloads/identity-provider/.$VERSION/
sudo chmod -R g-w /home/shibwww/html/downloads/identity-provider/.$VERSION/

Adjust the latest sym-link to point to the new version.

For top-level projects:
1. Update the relevant product's ReleaseNotes topic with the new release, the date, a link to a JIRA issue filter that produces the appropriate list of issues, and any important information about issues addressed, advisories, special requirements during upgrades, etc.
2. Send an announcement to announce@shibboleth.net. Optionally sign the email with your PGP key.
3. Update the "latest stable version" mentioned on the front page of the wiki space for the relevant product.
For releases that include one or more Security Advisories:
1. Finalize each advisory and fill in the date, double check any footnotes and URLs for correctness, and finalize the filename of the advisory based on the date and correct the self-referential link to the advisory in the text. See previous advisories for examples.
2. Sign the advisory in text form with a PGP key that's in the project's PGP_KEYS file.
3. Place the signed advisory on the server in /home/shibwww/html/community/advisories in a file named secadv_YYYYMMDD.txt and ensure appropriate ownership and 644 file mode.
4. Send the PGP-signed advisory to announce@shibboleth.net (see previous examples in the archive).
5. Update the relevant SecurityAdvisories page in the wiki space for an IdP advisory. The page includes a tabular display of all the advisories and links, and a table listing releases and how the advisories affect them.

Example using Docker

The attached script has been used to build a specific project inside the Shibboleth Docker Environment

Docker itself is started (this is on Windows) by this command

docker run -i -t --rm --name  shibboleth-build --hostname amazon11 --volume=%userprofile%\shibboleth-build-docker\user:/home/user ianayoung/shibboleth-build-docker:amazon11

The %userprofile%\shibboleth-build-docker\user directory was configured as noted above and the the docker Docker Environment Documentation

Windows Installer

This is described separately.

Releasing the “enforcer” plugins.

Our experience is that thinking through the implications of the build order of the enforcer plugins (java-mvn-enforcer, java-mvn-enforcer-data, java-mvn-enforcer-parent) is extra-ordinarily confusing. Although all that follows can be derived from first principals it is ‘very challenging’ and the following is offered as guidance so as to to avoid brain-melt

The three project (enforce, enforcer-data, enforcer-parent) are build in the same way as any other java project (with no javadoc)
If they do need to be release, then the release of enforcer and enforcer-data has to be
before the java-parent project.
In order to reduce circularity the enforcer projects have their own parent. This is a subset of the contents of the java-parent project.
This enforcer-parent-project does have references to enforcer and enforcer-data versions, and because circularity remains it is important to follow the guidance below
- The enforcer and enforcer-data project must NEVER have a parent with a SNAPSHOT version.
- By extension, this means that the enforcer parent project is release AFTER the enforcer or enforcer-data project

A typical build would be

Release java-mvn-enforcer-data (shall we say version ‘d' and parent version 'p’)
Release java-parent
Release the rest of the Java stack
Soon (but not immediately) release enforcer-parent version p+1
- Update any dependencies from java-parent (including the latest enforcer and enforcer data)
- Do a standard release of the parent project
- Update enforcer and enforcer-data to be parented by version p+1
- IMMEDIATELY run a complete (m2 checking) build (of each project).
  There should not be an issue (because there is nothing in enforcer-parent that is not in java-parent) but if there is you need to
  - Revert the parent of enforcer data to be version 'p'
  - Fix the problem,
  - Re-release enforcer-data
  - Point enforcer-parent to this version and start again with the release of parent version p+2