ReleaseNotes

Please review these release notes before upgrading your system. You should review the notes for all the versions subsequent to the one you're running prior to upgrade, including referring back to older V4 notes.

Known Bugs

https://shibboleth.atlassian.net/browse/OSJ-392

• This issues also affects direct import of Azure/Intra AD metadata.

• Until fixed in the next update, the only workaround for this is to revert the new feature by setting opensaml.config.xml.unmarshall.strictMode = false in a properties file and restarting.

https://shibboleth.atlassian.net/browse/IDP-2223

• When plugins are upgraded, their modules will add “idpnew” files per usual but the version label added to the filenames will be the current IdP version and not the plugin’s version. The API changes to fix this are done but the plugins in general will remain compatible with IdP V5.0 and so the bug will persist until they break compatibility with that version.

Note to Developers

If you have extension code, plugins, etc., there are numerous changes to the API (principally at the lowest levels) that require in most cases a reimport of the missing classes. Deleting any missing imports and re-importing them via your IDE will resolve a large number of issues, provided that you have updated your POM appropriately. See https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3251470353 for an explanation of the layout of the V5 library stack and POM import templates.

5.1.2 (April 15, 2024)

This is a patch release to address a Spring Framework advisory, a repeat (actually third time) of the same URL parsing bug that resulted in the previous update and impacts the CAS protocol support feature. It also includes an update to the ldaptive LDAP implementation to address a character encoding issue and a fix for an earlier issue around decryption while proxying that was not fully addressed originally.

5.1.1 (March 20, 2024)

This is a patch release that updates Spring Framework to address a security issue that impacts the IdP’s CAS protocol support that was identified the day after the original V5.1.0 release.

This release also fixes a logout regression introduced in V5.1.0 and rolls back the planned warning on the use of hyphens in IdPAttribute names, as this warning occurs under some conditions outside the deployer’s control. It may be re-introduced in the future under more controlled conditions, so use of hyphens remains discouraged.

5.1.0 (March 13, 2024)

This is a minor upgrade containing some new features and some changed defaults (more than are sometimes typical with our minor updates). Please review this information carefully when upgrading.

Changes to Defaults or Behaviors

The following defaults have changed for both new installs and upgrades and should be reviewed for possible incompatibilities:

• In keeping with best practices, we have changed the default behavior of our HTTP clients to limit TLS versions to 1.2 and 1.3. While Java’s own defaults automatically limit this, those don’t apply to the outbound connections made with the Apache HttpClient library. In the unlikely case that you run into a scenario requiring TLS < 1.2, you can supply a different set of allowed protocols via an HttpClientSecurityParameters bean and inject it wherever required (limiting it to that specific case). To allow for this, we have changed the internal handling of many settings such that they no longer prevent the use of such a parameters object but will merge its settings into the final set of parameters used.

• The idp.csp property default has been changed to include additional Content Security Policy (CSP) rules that are not expected to impact deployers but provide some additional protection against injected content. You can examine the default idp.properties file (created alongside when upgrading) and review the topic for more details.

• When not overridden, the internal templates used for the views rendered for the SAML POST (and related) binding include new CSP additions by default. These can be disabled by setting the new idp.encoders.cspEnabled property to false. This is discussed in the topic.

• While not a breaking change, the IdP now warns when hyphens appear in an attribute ID. This is discouraged due to the difficulty in using the attribute ID in contexts like scripting. This is most likely to be noticed by those with existing deployments of the OIDC OP plugin, which used them in the examples for the subject ID claim, but this is easily corrected (and will be adjusted in the next plugin release’s defaults).

• Some internally necessary changes have resulted in a breaking change to a configuration syntax that was never intended to work, but accidentally did, namely using a String-based collection or value to set the defaultAuthenticationMethods profile configuration option in relying-party.xml. That is, this worked by accident in previous releases, including V5.0.0:

  <bean parent="SAML2.SSO" p:defaultAuthenticationMethods="#{{ 'https://example.org/custom/ac' }}"

  Unfortunately, that no longer works as a shorthand, and was not in fact intended to work. In the case of SAML 2.0, this is the proper syntax:

  <bean parent="SAML2.SSO"> <property name="defaultAuthenticationMethods"> <bean parent="shibboleth.AuthnContextClassRefPrincipal" c:_0="https://example.org/custom/ac" /> </property> </bean>

  To see specific examples of this setting and the related issues around using it, see the profile configuration documentation (in the Authentication) sections, such as . Note that the exact type of Principal object needed will usually vary in other profiles.

Other Relevant Changes

The views/logout-complete.vm template was altered (for new installs only) to generalize a “completion” indicator so that the OIDC OP plugin’s upcoming logout support will be automatically compatible with it. In the absence of the change, the plugin’s logout flow will not complete fully in certain cases, though much of the important work will still happen. The change is as follows:

Old logout-complete.vm:

New logout-complete.vm:

Note that this change will not work on older IdP versions < 5.1 as it relies on an API addition to support the change.

New Features

The limited Content Security Policy (CSP) support in previous versions has been significantly extended to allow more control over the use of Javascript in views. See the topic.

The error handling view, when used by the MVC layer outside of Spring Web Flow (e.g., using the back button) has been extended to populate more of the “standard” variables available to the view under other conditions.

Some new variables have been added to the Password login view, see and a new approach to error message handling in that view is now provided.

Internal support for adding new view rendering technologies at runtime via plugins was introduced, with the first example being Thymeleaf. (We do not have an estimated release date for that plugin as we are so far unable to verify the keys needed to verify the artifacts.)

A feature to add arbitrary request headers to the logging MDC was added. See the topic.

The tool has a new “unfiltered” option to include unfiltered attributes in the dump.

Transcoding registry service rules now support the use of SAML metadata “tags” to specify overridden names to use when encoding SAML (or other protocol) Attributes into responses. This is an alternative to the use of a relyingParties attribute to limit encoding rules to specific SPs.

Some login flows now provide support for saving a username into a sealed cookie, and recovering the cookie value to pre-populate the username in some cases. Currently this support is limited to the Password flow but may be leveraged by custom flows with additional effort.

A new <EntityRegex> element is supported for defining regular expression-based conditions in several metadata filters.

A new variable, shibboleth.AttributeHelper , is available in some intercept flows and the Hello World flow. It is also available in many error views (where it may be less useful). This provides a simpler means of accessing any resolved attribute values. See for more details.

New Objects

• shibboleth.CSPDigester

• shibboleth.CSPNonce

• shibboleth.warning.CookieCondition

• shibboleth.StorageAwarePersistentCookieManager

• shibboleth.MDCHeaders

• shibboleth.authn.entityIDForDiscoveryStrategy

• shibboleth.BiConditions.TRUE

• shibboleth.BiConditions.FALSE

New Properties

• idp.encoders.cspEnabled

• idp.admin.entityID

• idp.cookie.storageService

• idp.authn.usernameCookieName

• idp.authn.usernamePrecedence

• idp.authn.Password.errorMessageFunction

• idp.authn.Password.genericMessageID

• idp.authn.SAML.convertUnknownPrincipals

• idp.views.velocity.activationCondition

• idp.views.velocity.order

• idp.views.jsp.order

5.0.0 (Sep 13th, 2023)

This is the first release of the fifth-generation Identity Provider software. The key documentation links are located on the IDP5 space Home page, such as SystemRequirements, Installation, and Upgrading material. Note the new SystemRequirements as they have substantially changed with regard to Java and servlet container versions.

This release should interoperate with all previous releases of Shibboleth and other software that supports the same standards.

As a major upgrade, the list of issues fixed and features added is numerous. Many significant changes are highlighted below, with particularly noteworthy issues highlighted in note or warning boxes.

General Notes

Upgrades to V5 and subsequent releases rely on an expanded use of the Module feature to track the configuration files and changes to them. Where previous releases included a copy of the “default” configuration in the dist folder, V5 tracks every user-modifiable file as part of a Module such that new versions of modified files will be created during upgrades. Also a change, the file suffix used will be of the form “.idpnew-majorminorpatch”, including a version suffix allowing identification of the version that created it. As before, if a file is replaced outright for exigent reasons, the old version will have an “.idpsave” extension.

The V5 release includes a significant amount of code refactoring that will impact deployers in largely mechanical ways. That is, where necessary to relocate classes we have provided as much compatibility as we can to support and warn on historical usage, but where post-upgrade changes are mandatory they should largely be “grep, replace” operations changing various class names from A to B.

In most cases, these names may be located in scripts that can be easily adjusted. Those with Java-based customizations will of course need to revise the version(s) of any dependencies to reference to V5 “stack” of libraries and that should flag any incompatibilities. Generally deleting all imports and reimporting the various classes used will be the quickest way to update things.

Notable Class Changes

Changes to Defaults

The following defaults have changed for both new installs and upgrades and should be reviewed for possible incompatibilities:

• The idp.hsts property default has been changed from “off” to 1 year, as we assume all IdPs have long since moved to TLS endpoints.

• Several default cookie names and paths have been adjusted to leverage Google’s non-standard trick of prefixing them with “__Host-” to prevent domain injection of important cookies. The path also had to be adjusted to the server root, as a requirement of that trick. There are related changes to web.xml described separately below.

• The idp.xml.strictEntityResolution property, used to prevent any attempts while parsing XML to locate remotely hosted schemas, has been changed to “true” by default. This may impact deployers who are relying on schemaLocation hints in files that reference unknown schema locations. By design we support particular HTTP URLs that reference all our schemas to allow for editor validation, while routing all access to the schemas by the software through local copies. Using other URLs will be blocked and trigger a warning.

• The XML processing code in OpenSAML has been enhanced to support a more strict form of processing that rejects unexpected/unknown XML Attributes, Elements, and even stray characters inside elements. Older versions tended to ignore them. In the vast majority of cases, rejecting such content is desirable but it is possible to turn off this processing mode by setting the property opensaml.config.xml.unmarshall.strictMode to “false”. Note that this setting appears to reject otherwise valid metadata produced by Microsoft products (see Known Bugs).

• The deprecated joda-time library has been removed from the distribution. Any customizations that rely on this library either must be removed or the library re-added for local inclusion. The most likely historical use of it would be in the configuration of the expiring-password interceptor or other features reliant on date-based predicates.

• Spring 6 has implemented a 10,000 character limit on the size of SpEL expressions to counter various denial of service vectors. This primarily impacts the use of some of our properties, as they are injected into Spring expressions to process them and/or allow dynamic behavior. We would, of course, not advise any approach that would lead to the use of such large properties and will review any areas of configuration that need additional flexibility as they’re reported to us.

• The path/name of the low-level metrics providing timing data regarding dynamic metadata resolution have changed to avoid exposing internal class names and better align to the expanded metrics available in this release. If scripts depend on these JSON structure paths and can’t be updated easily, the base name of the various metrics can be adjusted via the metricsBaseName setting added to the schema.

• A new feature of the MFA login flow is the ability to better control the value of the timestamp that is associated with the results and eventually communicated to SPs. Prior to V5, the timestamp defaulted to the latest timestamp found on one of the “embedded” results being combined by the flow (e.g., when combining the results of the Password and Duo features). V5 defaults to using the earliest timestamp instead, and a property (idp.authn.MFA.useLatestTimestamp) has been added to control this.

• Various APIs have been converted from reliance on Spring’s Resource interface to our “equivalent” non-Spring Resource interface due to inconsistencies in the way resources were “derived” from strings in the configuration. In most cases, the locations of files tend to be handled with properties or Strings and this is compatible; in unusual cases where actual Spring Resource class types were defined and injected into some beans, this may break and require adjustments to the configuration.

• The internal default CAS ticket service has been switched from the original server-side storage version to the encrypted/encoded service that embeds the required state into the ticket. Deployers whose configurations include the shibboleth.CASTicketService bean will see no change as this bean overrides the internal default. The main impact of this change is that ticket size increases (affecting some broken clients) and that multi-hop proxying is unsupported. If the latter is needed, reverting to the in-memory version is required.

The following defaults have changed for new installs but will not change for properly upgraded systems (generally these are set through explicitly uncommented properties or beans that have to be present to enable a behavior):

• None at present

Installation and Installation Commands

The following should be noted:

• The install command is only available inside the distribution, and all installation (including property based installation) should be done using this script.

• Many options can now be given to the installer as command line options. Nonetheless, is still possible although some properties are no longer available. Properties cannot be injected on the command line.

• All user visible or modifiable files are controlled by one of three new modules (idp.Core, idp.CommandLine, and idp.EditWebApp). This means that while local changes will almost always be retained, you will be able to see a side by side version of each file.

• Only batch files appropriate to the operating system (.sh for Unix and .bat for Windows) are now populated on first installation. The “wrong” type of file will generally be left behind from older installs and can be removed by hand.

• A new command update is available in bin. By default this will check to see whether the IdP can be upgraded. In addition it can download and check the signature on the upgraded distribution. Additionally the IdP itself checks for available updates when it starts, and logs the result. This can be disabled with the idp.updateCheck.enable property.

• The Windows msi installer has been substantially rewritten and is now split between a relatively standard IdP Installer and an installer for Jetty and our jettty-base deployment files.

• When enumerating all installed plugins and during various other operations, the plugin installer will now warn if a required module is unavailable.

• If the plugin installer changes the plugin state (by installing, removing or updating a plugin) it will output the resulting plugin state when it completes.

Changes to web.xml

We have heavily revised the content of the deployment descriptor file with an eye on reducing the need for deployers to override it, and reducing future conflicts if it is overridden. Much of the original boilerplate in the file is now gone and the content is now more configurable via Spring (via properties, etc.) and is auto-registering itself at runtime using newer servlet API features.

We have attempted to preserve as much compatibility as possible with V4-compatible versions of the file, but if you have overridden this file via edit-webapp, examining the new version in dist/webapp/WEB-INF/web.xml and modernizing the overridden version (or even better, eliminating it) is strongly encouraged.

Note that it is possible in most servlet containers to override context parameters using special container-specific hooks, usually in the context deployment fragment that specifies the location of the idp.war file. The new version of web.xml contains a few context parameters that can be used to disable the RemoteUser and X509 servlets if they are not in use, and it is advisable to override those parameters using the container rather than by overriding web.xml.

Profile Configuration API Changes

A few properties on the various profile configuration beans used in relying-party.xml have been moved out of some shared interfaces and classes and on to specific instances of these beans where specifically appropriate. We do not expect this to impact deployers. For example, this change makes it impossible to set signAssertions as part of the SAML Logout profile configuration (because there are no assertions to sign). A number of such incongruities were fixed in cases where a property should never have been set due to it being non-functional.

If the system fails to run correctly and indicates the relevant service’s configuration isn’t valid and/or the log is full of issues around the various profile configuration beans in that file, reviewing any unusual settings against the documentation is a good idea.

General Configuration Changes

Prior releases warned, but did not fail, on the presence of dependency or failover elements inside a Static DataConnector. The schema has been updated to actually preclude them since they are unsupported.

Some features in older versions tended to assume the presence of relatively short XML files specific to those features, such as the various <flow>-authn-config.xml files. These files are often now optional and superfluous. In such cases, when a feature no longer relies on any required files, no Module is necessary to create or remove them, and so a number of Modules present in V4 no longer exist in V5, in particular a number of the simpler login flows such as External, RemoteUser, etc. This does not mean the features have been removed or changed, or that existing files won’t continue to work. Generally the specific documentation on a given feature will note that a particular older file may be evaluated for removal.

HttpClient Changes

The Apache HttpClient library has been updated to V5. Most of the changes to this library have been accomodated by our internal Spring declarations and we do not expect most deployers to run into compatibility issues, but testing any custom client definitions is advised. Notably, the deprecated legacy beans that used to be provided for custom client definitions have been removed, per the documentation.

One breaking change involves HTTP Basic Authentication. While it was possible to make that work in earlier releases, the APIs used do not allow for pre-emptive authentication to avoid extra round trips, which is a fairly major issue with IdP use cases. We have added new APIs and parent beans to facilitate this in a supported and hopefully forwardly-compatible way, per the documentation.

LDAP Changes

The ldaptive library has been upgraded from V1.3 to V2.x, which relies on an internal LDAP protocol implementation rather than UnboundID (or JNDI) as in older versions.

Note that over the course of the last several releases and particularly during the transition to V4.1, the ldap-authn-config.xml file has become incompatible with the updated implementation and will probably not function properly in an upgraded V5 system. In most cases, simpler uses of LDAP are automatically handled with the modernized ldap.properties and password-authn-config.xml files, and advanced cases can usually be handled in simpler ways, but you are best off starting from scratch in a sense after upgrading and modernizing the relevant files and removing the now-removed one. Members are welcome to ask for help if they have more unusual requirements that can’t be met easily.

Connection Strategy

General improvements have been made to quarantine unresponsive URLs so they are considered inactive for a period before being tried again. Improvements have been made to the ROUND_ROBIN connection strategy to make it stateful and bring it’s behavior inline with user expectations.

Lost Connections

The behavior in V1.3 for lost connections was lazy reinitialization, a connection wasn’t known to be lost until it was used. In V2.x when connections are lost they immediately attempt to reconnect. This feature may produce unwanted noise in logs but is generally beneficial. Note that it can be disabled.

New Properties

• Added startTLSTimeout bean property to authentication and resolver configs. Allows separation of responseTimeout, which may need to accommodate long searches, from startTLS which should occur quickly.

• Added autoReconnect bean property to control whether lost connections automatically reconnect.

Logout Changes

An explicit workaround for Internet Explorer in the reporting of logout status (which itself is not a suggested approach anymore) was removed due to its reliance on an unsupported Java library to detect the browser. Given that IE is no longer supported by Microsoft, we removed the unsupported detection logic.

Removed Features

A number of previously deprecated features, settings, APIs, etc. have been removed. The use of these deprecated bits should have produced a standard warning either at startup or during use in the most recent releases of the software, and most of them are now non-functional or outright breaking. Eliminating warnings before upgrading is strongly advised.

The following non-exhaustive list of previously deprecated features have been removed:

• Support for SAML assertion delegation (virtually undocumented for several releases)

• The Password login flow’s Extended Flow feature (supplanted as far back as V3.3 with the MFA feature)

• The “legacy” authn/Duo login flow reliant on the Duo “iframe” integration style, which Duo has replaced and is retiring in 2024. See the DuoOIDC plugin for the replacement feature.

• The bean facades for HttpServletRequest/Response (replaced with new “Supplier” beans in V4.3, per )

• Legacy V2 compatibility scripting support in the Attribute Resolver (any references to edu.internet2… package names will break)

• Support for the exportAllAttributes setting in DataConnectors

• The EntityRoleWhiteList MetadataFilter type name has been removed, supplanted with the EntityRole type name

• The MappedAttributeInMetadata filter policy type name has been removed, supplanted with the AttributeInMetadata type name

Removed Objects

The following beans are no longer defined, so any references to them will break. Checking your files for them is a good sanity check.

• shibboleth.HttpServletRequest

• shibboleth.HttpServletResponse

  • Per

• shibboleth.NonCachingHttpClient

• shibboleth.MemoryCachingHttpClient

• shibboleth.FileCachingHttpClient

  • Removal of old, deprecated HttpClient beans

• shibboleth.NonFailFastValidator

  • Replaced by the failFastInitialize setting on various DataConnectors

• shibboleth.OutgoingIDWSFSSOSBindings

• shibboleth.Encoders.IDWSFSOAPEncoder

• shibboleth.Binding.LibertySOAP

• shibboleth.Conditions.IssuingDelegatedAssertion

• shibboleth.Conditions.AllowedSAMLPresenters

  • Part of the removed support for SAML delegation

• shibboleth.authn.Password.PrincipalOverride

• shibboleth.authn.Password.ExtendedFlows

• shibboleth.authn.Password.ExtendedFlowParameters

  • Part of the removed “Extended Flow” feature of the Password login flow

• shibboleth.RelyingPartyConfigurationResolver

  • Old facade for RelyingPartyConfiguration service, not generally used

Removed XML Settings

The following settings have been removed from various schemas:

• From all DataConnectors:

  • springResources

  • springResourcesRef

  • factoryPostProcessorsRef

  • postProcessorsRef

  • exportAllAttributes

• From all AttributeDefinitions, DataConnectors, and AttributeFilter rules:

  • profileContextStrategyRef, contextStrategyRef, profileContextRef, etc.

• From all AttributeFilter rules and the Mapped AttributeDefinition:

  • ignoreCase (long replaced by the inverse, caseSensitive)

• From StoredId DataConnector:

  • failFast

• From RelationalDatabase DataConnector:

  • readOnly

• From LDAP DataConnector:

  • searchExecutorRef (replaced by searchOperationRef)

Removed Properties

• idp.service.attribute.resolver.suppressDisplayInfo

• idp.httpclient.memorycaching.maxCacheEntries

• idp.httpclient.memorycaching.maxCacheEntrySize

• idp.httpclient.filecaching.maxCacheEntries

• idp.httpclient.filecaching.maxCacheEntrySize

• idp.httpclient.filecaching.cacheDirectory

• idp.duo.*

• idp.authn.duo.*

Newly Deprecated Objects

• Liberty.SSOS

• Liberty.SSOS.MDDriven

• shibboleth.SameSiteCookieFilter (referenced in older web.xml)

• shibboleth.ResponseHeaderFilter (referenced in older web.xml)

New Objects

• shibboleth.Conditions.EntityAttributes

• shibboleth.Conditions.MappedEntityAttributes

• shibboleth.TagCandidate

• shibboleth.TagLookupString

• shibboleth.TagLookupBoolean

• shibboleth.TagLookupInteger

• shibboleth.TagLookupLong

• shibboleth.TagLookupDouble

• shibboleth.TagLookupDuration

• shibboleth.TagLookupList

• shibboleth.TagLookupSet

• shibboleth.TagLookupBean

• shibboleth.HttpHost

• shibboleth.BasicAuthCredentials

• shibboleth.UserFacingPrefixPredicate

• shibboleth.authn.discoveryRequestFunction

New Properties

• opensaml.config.xml.unmarshall.strictMode

• idp.updateCheck.enable

• idp.cookie.bufferingCondition

• idp.responseHeaderCondition

• idp.encoding.forceUTF8

• idp.logging.MDC.enabled

• idp.logging.MDC.createSession

• idp.authn.SAML.NameIDLookupStrategy

• idp.authn.MFA.useLatestTimestamp

• idp.session.persistent