Upgrading

Owned by Scott Cantor
Last updated: Jul 09, 2024
6 min read

You must install the new version on top of your previous installation. This is not only safe but essential to properly maintain a working system.

In the event that you refuse to adhere to this approach, please be aware that you MUST copy back all previously existing files in the conf directory that were present before, not just the ones you may have changed. Failure to do so is the primary cause of issues after upgrading when the proper approach is not followed.

In effect, make sure that you capture all of the files in your configuration management system, not just changed files. Once you install once, you own and effectively have changed all of the files because future upgrades will make that assumption. This will increase the likelihood, but not guarantee, that an improper upgrade "from scratch" will work, but it is still not a supported approach and may lead to outright failure at any time.

Before You Begin

It is generally safe to upgrade directly to the latest “newer” version as long as that’s only a single major version later than you start from. It is not required to upgrade to, for example, 5.0 before 5.1, etc. However, skipping does of course mean taking into account all of the accumulated ReleaseNotes covering all the intervening versions.

Upgrading from V3 or Earlier

This is unsupported; you must upgrade to the latest V4 release first. We do NOT support upgrades from V3 or earlier, primarily to eliminate extra compatibility logic in the installer and to reduce the variance of the starting point.

Upgrading from V4

The following are important prerequisites for ugrading from V4:

  1. If you are upgrading from a pre-V5 release, you must upgrade to the latest V4 release first and remove deprecation warnings. Note that some of these warnings may only be visible in your servlet container’s log. This is due to the order of operations; the IdP’s logging service cannot be started until the global Spring context is running, and some of the configuration triggering warnings may be in that root context.

    1. There is one set of warnings that are an exception to this: the undocumented “SAML2NameID” and “SAML1NameID” attribute encoders produce warnings indicating they were removed from the next major version:

      DEPRECATED xsi:type 'SAML2NameID', (file [/opt/shibboleth-idp/conf/attribute-resolver.xml]): This will be removed in the next major version of this software; replacement is (none)

      In fact they were not, and their use (while still unsupported and undocumented) will produce a less aggressive “at risk” warning going forward. There are no explicit plans to remove them, but their use is incompatible with the vast majority of SAML software and should not be used in new deployments.

  2. You must ensure that your versions of Java and servlet container meet the system requirements for V5. This is more complex than usual because there are relatively few servlet containers right now that support both Java and Jakarta Servlet specifications, and V4 requires the former while V5 requires the latter. At a minimum, it’s a good idea to upgrade a V4 system to Java 17 first, likely also installing one of our scripting engine plugins (Nashorn or Rhino).
    If using Jetty it should be possible to use essentially the same configuration between Jetty 10 and 11, so if using Jetty 9 you should definitely migrate a V4 system to Jetty 10 before the V5 upgrade, at least in development/testing.
    Note that the jetty installed by the V4 windows installer is Jetty 10.x.

  3. Finally, remove the system directory before upgrading (the installer will block on this issue). In V4, this directory was used for compatibility with older systems that had overridden the web.xml file, but we are removing this stopgap. In the event your V4 IdP fails to start with the system directory removed, you will need to either remove your overriden copy of web.xml, or at least freshen it by updating the very top of the file and correct the contextConfigLocation context parameter:

    <!-- Spring application context files. Files are loaded in the order they appear with subsequent files overwriting same named beans in previous files. --> <context-param> <param-name>contextConfigLocation</param-name> <param-value>classpath*:/META-INF/net.shibboleth.idp/preconfig.xml,classpath:/net/shibboleth/idp/conf/global-system.xml,classpath*:/META-INF/net.shibboleth.idp/postconfig.xml</param-value> </context-param>

Further Note Regarding web.xml

There are a large number of simplications in V5 to reduce the amount of content in web.xml and to limit the need to change it for upgrades. For this release, it may be prudent to consider rebasing your version on the delivered file. At a minimum, you should expect some warnings after upgrading to note that much of what was left behind from V4 is now deprecated or unused.

General Upgrade Information

It's a good idea to review the Installation material to refamiliarize yourself with the general process. Most importantly, review the ReleaseNotes carefully for any important changes you might need to account for or make. In general, you should not expect upgrades to require changes ahead of time, but security issues or other major bugs might occasionally require special care.

The upgrade process is designed to be very safe, and will never (barring exigent need and clear documentation) overwrite any configuration files, views/templates, properties, etc. that are already in place.

By design, the idp.home/edit-webapp directory can be used to preserve changes across upgrades, but if you modify an existing file, principally web.xml, (as opposed to adding files), you should always compare your changed versions to the upgraded files to understand if any changes are important, though we will make every effort to highlight any such changes in the ReleaseNotes.

Be aware that rolling upgrades (that result in online servers on different versions) are generally guaranteed to work only for patch upgrades (changes in the final digit). Minor upgrades may sometimes include internal changes to storage formats or other implementation details that could prevent certain features from working interoperably between versions. It is best to either plan for a relatively fast rolling upgrade within a maintenance window, or plan for a short period of downtime. (Having said that, in practice these kinds of breaking changes are rare are most of the time rolling upgrades do in fact work.)

Note on Plugins

Generally it’s best to update all installed plugins to the latest supported versions for your current IdP version before you actually upgrade the IdP, as there may be minor adjustments to them to account for changes to the later IdP release. At times you may notice more warnings in the log if you upgrade the IdP first. With major upgrades, it is likely if not inevitable that most of them will require further updating.

Windows Upgrade

The installer available for Microsoft Windows handles upgrades from older releases.

In V5 the installation/update of the IdP is separated from the Installation/update of Jetty-base. A V4 to V5 update involves updating the IdP and then installing jetty-base.

Non-Windows Upgrade

  1. Download the latest Identity Provider software package (the zip file has Windows line endings, the tarball Unix line endings).

  2. Unpack the archive you downloaded to a convenient location. It will not be needed after installation and none of its content should be used outside the scope of the installation process.

  3. Change into the newly created distribution directory, shibboleth-identity-provider-VERSION

  4. You should take a backup of the idp.home directory prior to the upgrade in case anything goes wrong. Remember that there are files in this directory tree with highly sensitive information.

  5. Run either ./bin/install.sh (on non-Windows systems) or .\bin\install.bat (on Windows systems).

    • Make sure to specify the same installation directory you used originally (the idp.home directory). This will cause the installer to perform an upgrade. Using a different installation directory will essentially perform a new installation and this is not a supported mechanism for doing upgrades.

  6. After reviewing any necessary further changes, rebuild the warfile with any edits you've applied by running either idp.home/bin/build.sh or idp.home\bin\build.bat

  7. Re-deploy the new IdP warfile, located in idp.home/war/idp.war

Background

The rules for upgrades (which in turn drive your upgrade procedure) are derived from the Java Product Version Policy. This means that (except in exceptional circumstances):

  • It will be possible to move between patch versions of the same major and minor version (e.g.. from 5.3.2 to 5.3.1 or 5.3.3)

  • It will be possible to move to more recent minor versions of the same major version (e.g. from 5.3.2 to 5.4.1, but not from 5.4.1 to 5.3.2)

Exceptions would be:

  • Use of reserved ('impl') class names (as per the Java Product Version Policy) in configuration.

  • Use of reserved ('impl') classes by extensions.

  • Very rarely if required for security reasons.

In addition, major version releases (V3 to V4, or V5 to V6) will be compatibly upgraded subject to the following constraints:

  • The starting installation is the very latest release of the previous major release (at time of writing this is V4.3.1 for a V5.0.0 upgrade).

  • The installation starts and operates with no deprecation warnings.