The Shibboleth IdP V4 software will leave support on September 1, 2024.

ReleaseNotes

Owned by Scott Cantor
Last updated: Sept 15, 2023

Upgrades MUST be done in place!

If you are upgrading, PLEASE follow the directions on the Upgrading page and do NOT install the new version separately and then attempt to apply your old configuration files. You MUST upgrade "in place" by using an installation directory that contains a copy of the previously working configuration.

Failure to do so will result in a non-working system in the majority of cases because there are absolutely essential differences between the output of the installer in the two cases to prevent your configuration from being altered in unexpected ways.

This problem is exacerbated by the extent of the internal changes made to V4.1 and the results of combining old and new files without an extreme amount of effort will be unpleasant. It is urgent that people heed this warning and attempt to modernize settings (if at all) only after upgrading and verifying the behavior of the system.

Upgrade Plugins First

Less critically than the above warning, it’s advisable to upgrade any plugins you may have installed (meaning official plugins using the machinery introduced with V4.1 of the IdP) before upgrading the IdP itself, as this may prevent some extra warning noise in the log (or outright incompatibility issues noted by the installer). Upgrade and test those plugin features, and once those are validated, you can proceed to upgrade the IdP itself.

Please review these release notes before upgrading your system. You should review the notes for all the versions subsequent to the one you're running prior to upgrade, including referring back to older V3 notes.

Also be aware of the following issues regarding container or Java compatibility:

Known Bugs

These are known issues in the code that are likely to affect deployers and have yet to be fixed or that require some intervention because of how upgrades work.

  • https://shibboleth.atlassian.net/browse/IDP-1820

    • This is a change required to web.xml for systems that have been upgraded from prior to V3.2 and have been maintaining their own copy of web.xml since then. The fix needed for this is documented below under V4.1.0 under Breaking Changes → Deployment Descriptor Issue.

  • https://shibboleth.atlassian.net/browse/IDP-1625

    • This issue breaks the Cancel option in the Duo login flow when the CSRF feature is enabled. The fix is documented in the issue and can be applied directly to the duo.vm template. Since it involves a template, the fix is manual.

4.3.1.2 (Windows only) (September 15, 2023)

This is a service release of the Windows installation package which updates Jetty to 10.0.16, addressing security advisories issued by the Jetty Project.

4.3.1.1 (Windows only) (April 19, 2023)

This is a service release of the Windows installation package which updates Jetty to 10.0.15, addressing a pair of security advisories issued by the Jetty Project.

4.3.1 (March 30, 2023)

This is a patch release to address pair of regressions (one serious).

https://shibboleth.atlassian.net/browse/IDP-2084

https://shibboleth.atlassian.net/browse/IDP-1685

There are no other changes in this release.

4.3.0 (January 18, 2023)

Getting issues...

This is a small(ish) feature release expected to wrap up work on the V4 branch. It contains some additional deprecation and at-risk warnings for features either scheduled for removal in V5 or being considered for removal in a future version. Please check your warning log regularly for these warnings to ensure that you aren’t caught off guard or have the opportunity to express any concerns regarding any at risk features.

Changes to Existing Behavior

When proxying (i.e., using the SAML login flow feature), the IdP will now include a <Scoping> element in the request containing the original requesting SP's entityID. This is a requirement of the standard and was a bug by omission. In the event that you wish to suppress this behavior either out of concern for privacy or due to bugs on the part of other actors in the system, the ignoreScoping profile setting will suppress the generation of (and the evaluation of) the element altogether, though this is contrary to the standard.

A change was made to the feature allowing the IdP to auto-search for property files to load at startup. Prior to this version, if any directories under the search tree were unreadable due to access control, IdP startup would fail. With this release, the code is now trapping that error and startup continues, allowing this to be done deliberately to hide content from the IdP. Since this error can only be logged to the container’s log (it happens prior to the IdP starting up), it may be less obvious that it’s happening if it’s by mistake.

Deployers should ensure their containers are logging appropriately in some way (this is outside the scope of the IdP itself, though our Jetty examples do include it) and be aware there could be helpful information there.

Miscellaneous Changes

The most impactful change is a harbinger of a necessary adjustment that will be coming with V5.0 impacting any scripts or components that require injection of the HttpServletRequest or HttpServletResponse via the shibboleth.HttpServletRequest and shibboleth.HttpServletResponse beans, which are now deprecated in favor of a new pair of beans, shibboleth.HttpServletRequestSupplier and shibboleth.HttpServletResponseSupplier.

For the time being, uses of those beans will produce deprecation warnings, but they will be removed in V5.0. Remediating the warning requires adding the “Supplier” suffix to the end of the bean names at the point of injection, and then adding a call to get() in between the original reference to the bean and the servlet method called.

For example, this script line relying on the “custom” extension point to access the user agent string:

var ua = custom.getHeader("User-Agent");

becomes this (the only change is the additional indirection through the get() method):

var ua = custom.get().getHeader("User-Agent");

The quickest way to remediate the issue is to grep for the old bean names, adjust them whereever they’re spotted, and then track back to where they’re used (usually via scripts as in the above example) and add the indirection method.

See also https://shibboleth.atlassian.net/wiki/spaces/IDP4/pages/3139993601

New Features

New Properties

New Beans

4.2.1.1 (July 6, 2022)

This is a service release of the Windows installation package which addresses a Jetty logging issue and updates Jetty to 10.0.11 to address a memory leak.

4.2.1 (April 18, 2022)

This is a fixed package to address a regression that caused the spymemcached library to be omitted. Only users of the memcache storage option would be impacted.

4.2.0 (April 15, 2022)

This is a relatively “light” feature release that mainly exists to support API changes needed for some of our plugins, which have been updated along with it and require this version to use.

Getting issues...

Changes to Existing Behavior

A regression in all 4.x releases was identified in the way the Attribute-Based Subject C14N feature handles scoped attribute values such that only the value portion would be returned and used, ignoring the scope. For example, an eduPersonPrincipalName of “foo@example.org” would be returned as “foo”. This was easily fixed, but MAY impact existing behavior if the “broken” behavior were relied on. This can be remedied by adjusting the configuration to transform the scoped value back into an unscoped one but is something that could alter behavior following an upgrade until it’s addressed.

The support for auto-loading files that end in “.properties” was apparently broken in such a manner that the V4.1 releases may have excluded any file also named idp.properties regardless of location. The new version was fixed to properly handle all paths as unique regardless of filename and load such files, so you should NOT store any “backup” files or copies of your property files inside the conf tree. Note that the new release also logs any duplicate properties it detects for debugging. This will normally be sent to the container’s logging sink because it happens before the IdP is “started” enough to log things itself.

A long-standing design flaw was fixed so that deployers can configure inbound SAML profile interceptor flows without having to “repeat” a built-in interceptor flow that implements some of the inbound security handling logic. We are unaware of widespread (or any?) use of the inbound profile interceptor hook, but if it’s in use, there are possibly unneeded duplicate references to a flow ID such as “security/saml2-sso” or similarly-named flows that need to be removed. The duplicate reference will cause message replay failures because the flow will be run twice.

The CAS implementation was modified to better support consent by storing consented attribute IDs inside service ticket state for recovery during validation. This allows consent to be enabled and honored during the first hop of ticket validation, even if the consent is stored client-side or was granted only for a single request.

A bug dating back to the earliest Shibboleth releases was corrected to prevent the IdP from issuing duplicate SAML Attributes with the same fully-qualified name. The IdP now detects this and merges the values such that a single Attribute is produced with all of the values. This seems to have been a behavior people had somehow depended on, though doing so would depend on bugs in an SP in the first place since the result of processing 2 Attributes with 1 value and 1 Attribute with 2 values cannot be different in a compliant SP implementation.

User Interface Changes

The original view templates and stylesheets have been replaced in this release with a freshened set that are intended to be more modern and accessible. The look and feel is still assumed to be inadequate for any use other than testing, but may provide a better baseline to adjust. The name/location of the “common” stylesheet is adjustable with a message property.

The new placeholder stylesheet and logos have different filenames from earlier versions, so are installed automatically on new or upgraded installs. This is necessary so that installing modules that copy in new Velocity templates will result in at least a usable configuration. Older views that already exist will never be overwritten by upgrades, so they will continue to reference whatever stylesheets were used before.

See the https://shibboleth.atlassian.net/wiki/spaces/IDP4/pages/2934734849 topic for more information.

Another “UI-adjacent” change is adding support for post-authentication interceptor flows into the Administrative flow layer, in particular the Hello World flow. This allows easier demoing and testing of features like consent or expiring password warnings within the IdP itself.

Logout Changes

This release contains a few new options and optimizations to improve logout behavior and quiet noise in the logs, and are worth a review if you operate an IdP with a lot of SPs that do not support logout.

It includes an automatic behavioral change that tracks the endpoint used to deliver an assertion when starting a session, and uses that URL when selecting a logout endpoint to use if there are multiple endpoints spanning different virtual hosts or paths. The endpoint selected will contain the longest matching sequence of characters starting from the beginning of the URL(s). This approach is notably more compatible with Shibboleth SPs that are virtually hosted with a single entityID.

Another automatic change eliminates attempts to issue logout requests to SAML 2.0 SPs whose metadata contains no logout endpoints. This should reduce the extra noise of EndpointResolutionFailed events in the log and improve performance.

A new property named idp.logout.assumeAsync can be enabled to handle SPs that can issue logout requests but do not properly handle inbound logout requests or responses. Enabling the option allows an IdP administrator who controls the SP's metadata to remove the broken logout endpoints from the metadata without preventing the handling of logout requests because of "unable to respond" failures.

A new property named idp.logout.propagationHidden can be enabled to hide the list of services and logout status during logout propagation. Enabling this will require other template changes to properly report the logout to the user but allows the logout propagation to be hidden without editing style sheets or changing system files.

A new bean named shibboleth.PlaintextNameIDFormats can be defined to suppress encryption of particularly <NameID> formats that are not sensitive enough to require encryption (e.g., transient IDs when query support is not enabled). This further optimizes performance when lots of logouts are propagated at once.

Session Enhancements

The bean shibboleth.SessionAddressLookupStrategy can be used to source the “address” associated with an IdP session in an arbitrary way, allowing a wider variety of techniques for managing session affinity with clients. The IdP no longer requires that the address be in the form of an IP address, though it remains aware of that as a default.

Support for adjusting cookie names used by the IdP has been extended to cover most/all use cases, including the session cache. This is NOT intended to encourage any dependency on the names of the cookies, but rather to support ongoing hackery around the web for changing cookie behavior based on name, notably the Google invention of a __Host- prefix to prevent domain-scoped cookies from being fed into sensitive services. See https://shibboleth.atlassian.net/wiki/spaces/IDP4/pages/3140714497 for examples.

Miscellaneous Changes

Display name and descriptive information associated with attributes used on the consent view is now determined in a just-in-time fashion. This reduces the processing needed for those deployments (or specific attributes) which do not require consent. This change should be irrelevant unless you are using an externally-developed feature using the old (and now deprecated) APIs. Legacy behavior can be re-established by using the idp.service.attribute.resolver.suppressDisplayInfo property.

The startup sequence now logs any duplicate properties it finds to help with debugging. This is generally only visible in the container log prior to IdP “startup” so requires proper logging configuration at that layer.

New Properties

New Beans

New Messages

4.1.7 (April 18, 2022)

This is a patch release to update to Spring Framework 5.3.19 in response to the RCE vulnerability reported against Spring MVC.

4.1.6 (March 31, 2022)

This is a patch release to update to Spring Framework 5.3.18 in response to the RCE vulnerability reported against Spring MVC.

4.1.5 (January 19, 2022)

Getting issues...

This is a patch release primarily to update logback, the logging solution we provide by default. A few other dependencies were also updated to address open CVEs, including Spring.

The new logback version was issued in response to the log4shell debacle, and includes some hardening of features and the removal of other features to reduce the attack surface, though logback itself did not have any meaningful vulnerability that wouldn’t require compromise of its configuration file to begin with.

While it’s possible some deployers may have made use of the removed features, we’d rather err on the side of caution and the removed features will eventually be impossible to support anyway in order to keep up with any subsequent logback versions. Those with a real need for any of the removed features could choose to switch to an alternate logging solution, though some custom development will usually be required for that.

Please note that the IdP does not, and will never, require write access to its configuration. Many potential vulnerabilities may rely on elevation of privilege by leveraging one bug to make a change to a configuration file that in turn exposes the system to an RCE. It is a good idea to make sure the configuration is owned by a different account than the one used to run the IdP’s JVM, and so make the configuration read-only. Just making the files read-only is not sufficient since ownership is enough to allow the mode to be changed back. This is not new, and is very general advice, but it’s worth remembering.

The version of Jetty provided with the Windows installer has been refreshed to the latest 9.4 revision. Note that some earlier versions dating back to 9.4.39 and possibly some later revisions exhibited a bug on Linux observed to cause low priority CPU exhaustion (noticeable but not such that higher priority activity would be interrupted). This issue resolved itself with the version included in this release. It is unknown to what extent this issue may or may not impact Windows.

4.1.4 (July 27, 2021)

This is a patch release that supersedes V4.1.3 and fixes a regression introduced into that version.

4.1.3 (July 23, 2021)

Getting issues...

This is a patch release, primarily addressing https://shibboleth.atlassian.net/browse/IDP-1833, which prevented the new Duo plugin from functioning on Windows.

Unfortunately it contains a regression that breaks the SAML proxying support, so please refer to the V4.1.4 release for the Duo fix, now available.

4.1.2 (May 27, 2021)

Getting issues...

This is a patch release addressing some bugs and removing some dependencies that are no longer required for the core IdP to run.

Please note that there is no V4.1.1. An issue with dependency versions identified during the release process led to that version being abandoned and replaced immediately by this version.

Also note that the OIDC plugins have all been updated in conjunction with this release (the installer should note the need to update them). The Jira filter above includes all of the issues addressed in both the IdP and the updated plugins.

If you use the X509 login flow, the workaround required to address the regression in V4.1 is no longer required, so you should remove edit-webapp/WEB-INF/classes/net/shibboleth/idp/flows/authn/x509-authn-beans.xml from your system prior to upgrading, or do so and rebuild the war after.

4.1.0.1 (Windows only) (April 5, 2021)

This is a service release of the Windows installation package which updates Jetty to 9.4.39, addressing several security advisories issued by the Jetty Project.

4.1.0 (March 24, 2021)

Getting issues...

This is a significant new feature release that includes a larger than usual number of new configuration options, but these are backward-compatible and mostly simplify things for new deployers. The new Plugin and Module layers are a key addition in support of an initial set of add-on features described in the IdP Plugins wiki space.

A section has been added to the Upgrading page specific to this release. Note especially that previous versions of the OIDC OP extension are not compatible with this release, nor will many of the older OIDC configuration settings work with the new plugin without some (usually small) alterations. See OIDC OP Upgrading for details.

More specifics can be found among the various topics, but you may wish to review the new material on ModuleConfiguration and consider disabling any modules associated with unused features. This will remove some unchanged files outright and rename others so that you can find and remove them. A new install of V4.1 only enables the "idp.authn.Password" and "idp.admin.Hello" modules (which support the Password login flow and the new Hello World flow respectively).

Note that while much of the documentation includes sections of material labeled "V4.1+", this doesn't mean the V4.0 material is "non-functional" in an upgraded system. It's just easier to document approaches that are "standard" in each version without constantly noting this. Some of the V4.1 "tabs" in the various topics will also contain advice or information on how to "modernize" a configuration to replace unnecessary XML with property definitions and eliminate additional logging.

Breaking Changes

Deployment Descriptor Issue

Older installations of the IdP dating back to V3.0 or V3.1 that have customized web.xml (identified by the presence of edit-webapp/WEB-INF/web.xml) may have an incompatibility that is not accounted for during the upgrade process. The issue will prevent the IdP from starting up successfully and will be logged in the Java container as an error to load the global.xml file. The trigger for this issue is the following outdated content in web.xml:

Old web.xml Content
<param-name>contextClass</param-name> <param-value>net.shibboleth.ext.spring.context.DeferPlaceholderFileSystemXmlWebApplicationContext</param-value>

To correct this issue, the classname must be changed to net.shibboleth.ext.spring.context.DelimiterAwareApplicationContext and the warfile rebuilt to pick up the change. You may of course wish to compare your copy to the new default version (in dist/webapp/WEB-INF/web.xml) for other useful changes, but this specific change is the only known incompatibility.

Extension Point Change

As part of making the system more plugin-friendly, one very infrequent extension point was broken, contrary to our usual compatibility rules for a minor release. Registration of custom PrincipalSerializer beans as part of more advanced authentication extensions will no longer be recognized and the shibboleth.DefaultPrincipalSerializers and shibboleth.PrincipalSerializers list beans are no longer recognized as of V4.1.

The replacement for this mechanism is a new auto-wired service interface, PrincipalService. Updated information on this mechanism can be found on the Authentication development documentation page, but this is very unlikely to impact anyone. Upgraded systems should not break outright, but will fail to locate and use any custom serializers registered using the old hook.

Changes to Defaults

The following defaults have changed for new installs but will not change for properly upgraded systems (generally these are set through explicitly uncommented properties or beans that have to be present to enable a behavior):

  • The new idp.bindings.inMetadataOrder property defaults to true, but is set to false in new installs. The primary purpose and impact is discussed below under "Restoration of Logout Behavior".

  • The Attribute Release consent flow is no longer enabled by default, due to the lack of a deployable storage option out of the box and the the ease with which it can be enabled.

Elimination of system Files

With this release, we have eliminated the requirement for a "system" directory in the installation tree in favor of embedding the resources into the system's libraries. Because of historical references to files in this directory in the deployer-editable web.xml file, it's not possible to remove it entirely from an upgraded system to maintain compatibility, but new installs will not include it. A future release may require modifications to web.xml to allow for complete elimination of the directory.

As part of this change, you may also wish to remove a stale, though harmless, reference to "%{idp.home}/system/messages/messages" from the shibboleth.MessageSourceResources bean in services.xml

New Property Files and Property Lookup

Many common settings that used to be configurable only with XML files and Spring configuration are now exposed as properties (if and only if the original beans don't exist, for compatibility). There are new property files in the various subdirectories (conf/authn/authn.properties, conf/admin/admin.properties, conf/c14n/subject-c14n.properties) that contain examples of these settings.

While new installs automatically take advantage of these properties, (properly) upgraded systems will not without other adjustments. The linked topics discuss how to "enable" their use in those cases for those that want to take advantage of them.

Also take note that with this release, the process of "finding" property files has been improved. In older versions, and when the idp.searchForProperties property is false/missing, the set of property files loaded is solely controlled by the idp.additionalProperties setting. New installs by default, or systems that explicitly enable the idp.searchForProperties property, will locate and load all files with the ".properties" extension in the conf tree. The explicit property can still be used for directly loading files from alternate locations.

Note that there are no plans to actually break or remove support for the older ways of configuring many features; the property additions are purely for simplification for newer deployers or for those who wish to take advantage.

Auto-Wired Components and Logging

To better accommodate plugins, the system now relies on Spring to automatically locate and consume beans of certain types at runtime instead of requiring lists of beans in many places in the user-editable configuration.

As a result, upgraded systems will initially output a number of INFO log messages at startup such as:

Log example
2021-03-11 13:51:22,113 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: intercept/context-check 2021-03-11 13:51:22,113 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: intercept/terms-of-use 2021-03-11 13:51:22,113 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: intercept/attribute-release 2021-03-11 13:51:22,215 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: http://shibboleth.net/ns/profiles/status 2021-03-11 13:51:22,215 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: http://shibboleth.net/ns/profiles/reload-service-configuration 2021-03-11 13:51:22,216 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: http://shibboleth.net/ns/profiles/reload-metadata 2021-03-11 13:51:22,216 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: http://shibboleth.net/ns/profiles/resolvertest 2021-03-11 13:51:22,218 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: http://shibboleth.net/ns/profiles/metrics 2021-03-11 13:51:22,218 - - INFO [net.shibboleth.ext.spring.util.IdentifiedComponentManager:89] - Replacing auto-wired component: http://shibboleth.net/ns/profiles/storage

Similar messages will appear for the authentication flow IDs and the authentication type comparison rules. These reflect the fact that your existing configuration explicitly contains definitions for the various "flow descriptors" and other beans present in various files that are now either optional or reduced in size.

These definitions can be removed if desired, eliminating the messages, but you should refer to the "Upgrading from V4.0 and Earlier" section under the new V4.1+ tabs on various pages (AuthenticationConfiguration, ProfileInterceptConfiguration, AdministrativeConfiguration) for help with that to make sure you don't lose any customizations you might have made. It's entirely non-essential and the log messages are there simply to provide accurate information about whether the internal versions of these beans are used or not.

Later on, if you need to explicitly override one of the beans that is auto-wired, the message will simply act as a verification that this is happening (i.e., if you don't see it, you know you didn't manage to successfully override the system's version).

Restoration of Logout Behavior

Adding SOAP Logout support had an unintentional side effect of causing the SOAP binding to be selected at times instead of the more advisable approach, required by the standard, of favoring front-channel bindings. This problem of binding selection dates back to V2 and was caused by the IdP choosing endpoints in metadata order. This has been addressed with a new property, idp.bindings.inMetadataOrder, that can be set to false to produce a more desirable binding selection order that favors Redirect and POST when supported. It defaults to true for compatibility, but is set to false for new installs.

IPv6 Client Address Sanitization

After waiting for a fix upstream in Jetty that got reversed in a patch, we have applied a global fix to our code to ensure that IPv6 client addresses are consistently exposed without the use of brackets around the address (brackets should be confined to "address-as-host" scenarios, not bare addresses). This corrects a number of problems, but may result in changes to log files and could impact scripts that were not coded defensively to handle either format.

Removed Objects and Files

Some of the changes in this release have resulted in the elimination of certain beans and even whole files in many cases.

The conf/session-manager.xml file is no longer created on new installs and in the vast majority of cases can be removed outright. If you adjusted the list of client storage service beans in that file (e.g., if you're not using client storage), you could just move that overridden list to global.xml and then remove the file. The other beans that were in the file are now superfluous.

Several other XML files are no longer created on new installs but are more likely to contain customizations and require more deliberate work to remove, which is discussed above and in various places in the documentation.

In addition, there are many "optional" files that are no longer installed by default, including some XML files and views, because they are part of Modules that enable optional features. Turning on some features in the configuration will require enabling the corresponding module, or things will not function correctly in ways that will vary from failure to start the IdP, to exceptions at runtime, to log messages about missing files.

New Flows

New Objects

Renamed Objects

New Properties

There are a lot of new properties because of the aforementioned changes, but a few miscellaneous settings are mentioned here:

4.0.1.2 (February 2, 2021)

This is a service release of the Windows installation package which deals with a compatibility issue with the container packages (Jetty and Procrun) when upgrading from 3.4.8.  It is unneeded by anyone maintaining their own Java container, which continues to be strongly recommended.

4.0.1.1 (July 22, 2020)

This is a service release of the Windows installation package in order to address a security issue in Jetty, which has been updated to version 9.4.30. It is unneeded by anyone maintaining their own Java container, which continues to be strongly recommended.

4.0.1 (June 3, 2020)

Getting issues...

This is a patch release to address bugs identified and to shore up some missing LDAP features that arose due to the loss of JNDI support, such as SASL authentication and referral handling. The LDAP features are an exception to our usual policy on new features in a patch release and configuration options are denoted with the 4.0.1 designation where applicable.

A bug was corrected that was causing metadata resolver crashes on invalid metadata containing empty elements. Previous releases inadvertently accepted and ignored these elements. The patch adds warnings about the invalid content but corrects the crashing bug.

Jetty has been updated for the Windows installer to version 9.4.28 and an issue corrected with the use of wildcard certificates. The full fix for this in the case of back-channel deployment involves an update to the plugin we supply for certificate validation bypass, and this plugin has been renamed to reflect that it is designed solely for use with Jetty 9.4. The Jetty94 page has been updated to reflect this. The older plugin and back-channel XML configuration remains compatible for now, but will fail in the unlikely even a wildcard or multi-SAN certificate were used.

Changes to Defaults

  • The transcoding rule shipped with 4.0.0 for the "telephoneNumber" attribute had the SAML 1 and SAML 2 names swapped, so upgraded systems will carry this mistake until corrected by hand.

Known Issues

  • A bug was introduced into the default web.xml file in the interminable attempts to fix warnings around the default method constraints. The element <authn-constraint/> should have been <auth-constraint/>. This may cause warnings or result in the rule being ignored.

  • In both 4.0.1 and 4.0.0, an inadvertent change has caused the attribute filtering engine to alter the order of attribute values produced by the resolver. The order in fact varies by server and over time. While this is not a bug per se because attribute values are unordered by that point, it does cause unpredictable hashing values when comparing values for attribute release consent, which causes re-prompting. This will be fixed in V4.1.

4.0.0 (March 11, 2020)

This is the first release of the fourth-generation Identity Provider software. The key documentation links are located on the IDP4 space Home page, such as SystemRequirements, Installation, and Upgrading material. Note the new SystemRequirements as they have substantially changed with regard to Java and container versions.

This release should interoperate with all previous releases of Shibboleth and other software that supports the same standards.

As a major upgrade, the list of issues fixed and features added is numerous. Many significant changes are highlighted below, with particularly noteworthy issues highlighted in note or warning boxes.

There are API additions, changes, and removals, and any third-party extensions will need to be checked for code compatibility and recompiled for use with this version, and any custom scripts created by deployers should be carefully tested and reviewed.

Changes to Defaults

The following defaults have changed for both new installs and upgrades and should be reviewed for possible incompatibilities:

  • The idp.cookie.secure property now defaults to true if not set by the deployer. In the unlikely event that HTTP compatibility is a requirement, the property would need to be adjusted. (Note that the container's session cookies are controlled by policy in web.xml and while we have also changed that default, only upgraders without their own web.xml customizations would be exposed to that change.)

  • The default LDAP provider is now UnboundID. See the LDAP section below for more information.

  • Some legacy code that was used to maintain the V2-style formatting of the %T audit field (the timestamp of the record) was removed and the log now outputs all date/time fields using the supplied formatter or a default. The format of the %T field may therefore change for some deployers.

    • Related, the shibboleth.AuditDateTimeFormat bean is interpreted as a pattern by the Java DateTimeFormatter class, rather than the deprecated joda-time version, and there are some differences in those classes that may result in changes to your logs. If you require identical output, review is warranted.

  • A side effect of the attribute-related changes noted below is that SAML 2.0 AttributeEncoders with no friendlyName attribute default to populating FriendlyName with the attribute's ID. At present there is no way to restore the original behavior other than to convert to use of the AttributeRegistryConfiguration to express the rule.

The following defaults have changed for new installs but will not change for properly upgraded systems (generally these are set through explicitly uncommented properties or beans that have to be present to enable a behavior):

  • The default XML Encryption algorithm is now AES128-GCM rather than AES128-CBC. See GCMEncryption for much more on this, and do note that newly installed systems will fail to interoperate with a large number of SAML SPs without modern algorithm support.

  • HTML Local Storage is now enabled by default when the client-side session storage feature is used (which has been and remains the default option as well).

    • In addition the logout-enabling idp.session.trackSPSessions and idp.session.secondaryServiceIndex properties are also enabled by default. They can be disabled by deployers that do not wish to try and support logout propagation.

  • The set of profiles enabled by default no longer includes any SAML 1.1 support, or SAML 2.0 Attribute Query.

  • The default trust engines used at runtime now omit PKIX support. Support for only the Metadata Interoperability Profile is now the default behavior with no attempt at a fallback.

  • A few ReloadableServices that were configured by default to fail-fast have changed to non-fail-fast behavior.

  • Audit output has been changed to include a much more useful default format and various improvements to how specific fields are logged to shrink repetitive and unnecessary constant values.

Removal of Deprecated Features

A number of previously deprecated features, settings, APIs, etc. have been removed. The use of these deprecated bits should have produced a standard warning either at startup or during use in the most recent releases of the software, and most of them are now non-functional. Eliminating warnings before upgrading is strongly advised.

Date/Time APIs

The entire code base has been converted from the joda-time Date/Time API to the new Date/Time API introduced in Java 8. This change is widespread in the code, with most uses of long/Long types to represent timestamps and durations converted into Instant and Duration classes.

The change is mostly invisible to deployers because of the widespread support for ISO Duration notation in the configuration. There are a couple of areas of direct impact that are now deprecated and will be removed from a future version:

It was also observed that V3 may have been improperly interpreting timestamps without the trailing 'Z' indicator as compliant UTC values, which was not a correct assumption. Using such values is a violation of the SAML specification in any event, and V4 does not handle such values and will reject them in most cases as expired or in the future depending on the default time zone of the system relative to UTC.

Predicate/Function APIs

The entire code base has been converted from the Guava Predicate/Function API to the new functional API interfaces introduced in Java 8. This change is widespread in the code, but we have supplied migration aids via our numerous utility/parent beans and a support class that converts between the legacy Guava Predicate.apply() signature and the new standard Predicate.test() signature.

We expect most custom code and scripts based on Guava Predicates will work unmodified, but may produce warnings when the apply() method is called to allow code cleanup and prevent future problems.

One place we missed is near the bottom of the views/login.vm file. If you happen to be using the "not-quite-deprecated" Extended Flows feature, there's a call to "apply" in the template that needs to be manually changed to "test". This also applies to new installs prior to V4.1.0 which will contain the fixed template.

Beware of Guava's "compose" method

Legacy use of Guava Functions may work for the most part, but there is a known issue in Spring's wiring behavior that breaks the use of Guava's "compose" helper. If you have custom bean wiring anywhere that contains a reference like this, you will likely have to update it to avoid problems:

<bean class="com.google.common.base.Functions" factory-method="compose">

Using that syntax will often result in the two constructor arguments to that bean getting swapped/reversed, which reverses the order of the composition, usually producing bad output at best, and a crash or even more catastrophic outcome at worst.

Unfortunately there's no easy fix for this because of the way Guava accommodates the conversion to Java 8's Function interface. As a result, the best thing you can do is simply scan for this and convert the above syntax to:

<bean parent="shibboleth.Functions.Compose">

That is functionally equivalent and should avoid the re-ordering behavior we're seeing with Spring.

Profile Configuration APIs

A lot of cleanup was performed to eliminate inconsistent method names in the APIs used for configuring and overriding profile behavior in relying-party.xml using overrides. In a small number of advanced cases, people may encounter errors in their configuration due to type incompatibilities, which may require reviewing the appropriate class Javadoc. But as a rule of thumb, all the property names now follow these conventions:

  • p:something takes a literal value such as a boolean, integer, long, String, Duration, etc.

  • p:somethingPredicate takes a Java Predicate when the equivalent literal would be a boolean

  • p:somethingLookupStrategy takes a Java Function when the equivalent literal would be other than a boolean

As an example, the V3 property signAssertions was Predicate-valued, while the V4 version takes a literal boolean and signAssertionsPredicate takes a Predicate. There are a number of such cases but not many with common use.

LDAP Changes

The default LDAP implementation is now the UnboundID library and JNDI is no longer officially supported. There will be no JNDI support at all in a future version and we expect to migrate to a native implementation being built within a future version of Ldaptive at that time.

This release includes configuration updates to facilitate that change, particularly in the LDAPAuthnConfiguration and to a lesser degree in the LDAPConnector. Deployers using this feature should review and update their configurations when possible to modernize them and minimize the need for future changes. In many cases, the ldap-authn-config.xml file can be replaced outright with no impact, along with an additional change noted in the LDAPAuthnConfiguration page under "Note Regarding Upgrades".

Most previously necessary <LDAPProperty> elements can be replaced by various new (or recently added) settings that configure the equivalent behavior in a non-JNDI fashion. For example, specifying binary attributes in the LDAPConnector is now possible directly via the <BinaryAttributes> element added in a V3.4 patch. If you encounter properties that have no obvious replacement, seek assistance.

The ldaptive JAAS module has an incompatible change in its handling of timeout settings. Specifying values in milliseconds will result in a DateTime parsing exception. The values must be converted to ISO syntax (e.g., 2000 → PT2S).

In addition, various LDAP pooling properties prefixed by "idp.pool.LDAP." (such as idp.pool.LDAP.validatePeriod) previously defaulted to numeric values expressed in seconds (e.g. 300 == 5 minutes). These are now being interpreted as milliseconds, which causes overly frequent pool validation if the defaults are used with an unmodified V3 ldap-authn-config.xml file. This is discussed further on the LDAPAuthnConfiguration page.

Finally, the LDAPConnector's maxResultSize option's behavior is different from V3 in that the connector will raise an error if the size is exceeded rather than ignoring this condition.

Password Flow Extension

The Password login flow has undergone significant internal reimplementation to support a much higher degree of flexibility. The redesign has resulted in the elimination of one existing extension mechanism that we don't believe was heavily used, described in the V3 documentation under the heading Custom Back-Ends.

A simpler extension point is now provided for implementation of such custom validators, and converting is not a complex task.

A major new addition to V4 is the AttributeRegistry service that replaces, or at least supplements, the AttributeEncoder mechanism for translating attributes. While upgraded systems should operate unchanged, deployers are advised to review this new feature and may achieve substantial simplification of their configuration by taking advantage of it. Many existing encoders may be removable.

Note that installing from scratch and then applying a legacy configuration will in most cases result in duplicate Attributes appearing in SAML messages due to the overlap between the existing encoders and the new rules. The upgrade process is designed to prevent that by excluding the new rules from the default configuration. Of course, the inverse is also true: if you want the new default rules to apply, you must make that transition after upgrading.

A consequence of this new service is that adjusting AttributeEncoders requires reloading "shibboleth.AttributeRegistryService" and not the “shibboleth.AttributeResolverService” service as in previous releases.

In the unlikely event that you have any attribute display name or description metadata present for an attribute definition but do not have an encoder, the compatibility support is not going to attach that display metadata to the resulting attribute. This does not seem like a likely scenario, but it's a change.

The rules for attribute IDs are tightened in this release to preclude whitespace and single quotes. With advance warning this set may be tightened further if warranted. We strongly advise sticking to simple ASCII characters.

The class hierarchy around IdPAttributeValue has been simplified and the use of generics eliminated. Some scripts may be impacted by changes to method names. There is no longer a universal getValue method exposed, but getNativeValue generally returns the equivalent result save for scoped values, where a Pair is returned. This change prevents unintentional "lossy" conversions to String for non-String values.

The removal of the deprecated <SourceAttribute> element from the TemplateAttributeDefinition syntax creates a rare scenario that can break existing behavior in a small set of cases if an InputDataConnector is supplied with the allAttributes option set. Using this syntax requires that all inputs either have the same number of values (or no values). Instead of using the <SourceAttribute> element in conjunction with allAttributes, simply enumerate the attributes to be sourced from the connector.

Metadata Changes

Due to additional "up front" processing of some SAML metadata content, certain invalid SAML constructs that used to be ignored are no longer being tolerated during the initial load of a MetadataProvider, in particular the appearance of empty XML elements such as <md:ServiceName/>,  <md:ServiceDescription/>, and all the child elements within the <md:Organization> element.

Such elements are invalid according to the standard and the IdP's behavior is correct, but the initial release had a bug that results in a NullPointerException instead of ignoring the element. The crash was corrected in V4.0.1. Deployers should not in general rely on the IdP to be tolerant of invalid SAML constructs.

Logout Changes

The Single Logout feature now includes some additional control points, properties, and changes to the default views that allow more control over how logout proceeds, whether users are able to cancel/abort them, and other requested "flow of control" enhancements. These are discussed in the LogoutConfiguration topic.

In addition, SOAP logout for SAML has been implemented for the first time, and unfortunately this change can cause some impact to existing deployments because of a bug related to how bindings are chosen, which has historically been in metadata order. This can cause SOAP to be selected in preference to front-channel bindings, which is an error. Control over this has been added for a future release. This section will be updated with more information as various aspects of controlling the SOAP logout behavior are investigated.

New Deprecations

Due to a Spring bug that causes problems with configuration reloading, we have deprecated the original set of HttpClient parent beans and any inheritance from those beans in custom HttpClient configurations. We have introduced new parent beans without predefined settings that were causing the problems, and adjusted the HttpClientConfiguration documentation in various ways to refer to them.

Internal components that rely on a default HttpClient configuration now rely on a system bean definition intended only for internal use, configured as before via the services.properties file.

If you have beans that are inheriting from the deprecated versions, a warning will be issued. They will be removed from V5. This extends to several properties that were used to configure the caching versions of the clients, which will be removed.

CSRF Prevention

The IdP includes a new Cross-Site Request Forgery (CSRF) Protection feature that is enabled by default for new installs. The idp.csrf.enabled property is a general on/off switch if problems are encountered or upgraders wish to enable it. The feature can also be easily incorporated in custom webflow views with minimal effort.

Extensible Installer

Mostly of interest to packagers, the installer is now built with Java and has explicit extension points. See Programming the V4 Installer.

Noteworthy New Features

New Objects

New Properties