ReleaseNotes

Please review these release notes before upgrading your system. You should review the notes for all the versions subsequent to the one you're running prior to upgrade, including referring back to older V4 notes.

Known Bugs

https://shibboleth.atlassian.net/browse/OSJ-392

• Until fixed, the only workaround for this is to revert the new feature by setting opensaml.config.xml.unmarshall.strictMode = false in a properties file and restarting.

Note to Developers

If you have extension code, plugins, etc., there are numerous changes to the API (principally at the lowest levels) that require in most cases a reimport of the missing classes. Deleting any missing imports and re-importing them via your IDE will resolve a large number of issues, provided that you have updated your POM appropriately. See https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3251470353 for an explanation of the layout of the V5 library stack and POM import templates.

5.1.0 (Unreleased)

This is a minor upgrade containing some new features and some changed defaults.

Changes to Defaults

The following defaults have changed for both new installs and upgrades and should be reviewed for possible incompatibilities:

• The idp.csp property default has been changed to include additional CSP rules that are not expected to impact deployers but provide some additional protection against injected content. You can examine the default idp.properties file (created alongside when upgrading) and review the https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199510790 topic for more details.

• When not overridden, the internal templates used for the views rendered for the SAML POST (and related) binding include new CSP additions by default. These can be disabled by setting the new idp.encoders.cspEnabled property to false. This is discussed in the https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199510790 topic.

New Features

Content Security Policy support has been extended to allow tighter control over use of Javascript in views. See the https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199510790 topic.

A feature to add arbitrary request headers to the logging MDC was added. See the https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199509659 topic.

The https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199511404 tool has a new “unfiltered” option to include unfiltered attributes in the dump.

New Objects

• shibboleth.CSPDigester

• shibboleth.CSPNonce

• shibboleth.warning.CookieCondition

• shibboleth.MDCHeaders

• shibboleth.authn.entityIDForDiscoveryStrategy

New Properties

• idp.encoders.cspEnabled

• idp.admin.entityID

5.0.0 (Sep 13th, 2023)

This is the first release of the fifth-generation Identity Provider software. The key documentation links are located on the IDP5 space Home page, such as SystemRequirements, Installation, and Upgrading material. Note the new SystemRequirements as they have substantially changed with regard to Java and servlet container versions.

This release should interoperate with all previous releases of Shibboleth and other software that supports the same standards.

As a major upgrade, the list of issues fixed and features added is numerous. Many significant changes are highlighted below, with particularly noteworthy issues highlighted in note or warning boxes.

General Notes

Upgrades to V5 and subsequent releases rely on an expanded use of the Module feature to track the configuration files and changes to them. Where previous releases included a copy of the “default” configuration in the dist folder, V5 tracks every user-modifiable file as part of a Module such that new versions of modified files will be created during upgrades. Also a change, the file suffix used will be of the form “.idpnew-majorminorpatch”, including a version suffix allowing identification of the version that created it. As before, if a file is replaced outright for exigent reasons, the old version will have an “.idpsave” extension.

The V5 release includes a significant amount of code refactoring that will impact deployers in largely mechanical ways. That is, where necessary to relocate classes we have provided as much compatibility as we can to support and warn on historical usage, but where post-upgrade changes are mandatory they should largely be “grep, replace” operations changing various class names from A to B.

In most cases, these names may be located in scripts that can be easily adjusted. Those with Java-based customizations will of course need to revise the version(s) of any dependencies to reference to V5 “stack” of libraries and that should flag any incompatibilities. Generally deleting all imports and reimporting the various classes used will be the quickest way to update things.

Notable Class Changes

Changes to Defaults

The following defaults have changed for both new installs and upgrades and should be reviewed for possible incompatibilities:

• The idp.hsts property default has been changed from “off” to 1 year, as we assume all IdPs have long since moved to TLS endpoints.

• Several default cookie names and paths have been adjusted to leverage Google’s non-standard trick of prefixing them with “__Host-” to prevent domain injection of important cookies. The path also had to be adjusted to the server root, as a requirement of that trick. There are related changes to web.xml described separately below.

• The idp.xml.strictEntityResolution property, used to prevent any attempts while parsing XML to locate remotely hosted schemas, has been changed to “true” by default. This may impact deployers who are relying on schemaLocation hints in files that reference unknown schema locations. By design we support particular HTTP URLs that reference all our schemas to allow for editor validation, while routing all access to the schemas by the software through local copies. Using other URLs will be blocked and trigger a warning.

• The XML processing code in OpenSAML has been enhanced to support a more strict form of processing that rejects unexpected/unknown XML Attributes, Elements, and even stray characters inside elements. Older versions tended to ignore them. In the vast majority of cases, rejecting such content is desirable but it is possible to turn off this processing mode by setting the property opensaml.config.xml.unmarshall.strictMode to “false”. Note that this setting appears to reject otherwise valid metadata produced by Microsoft products (see Known Bugs).

• The deprecated joda-time library has been removed from the distribution. Any customizations that rely on this library either must be removed or the library re-added for local inclusion. The most likely historical use of it would be in the configuration of the expiring-password interceptor or other features reliant on date-based predicates.

• Spring 6 has implemented a 10,000 character limit on the size of SpEL expressions to counter various denial of service vectors. This primarily impacts the use of some of our properties, as they are injected into Spring expressions to process them and/or allow dynamic behavior. We would, of course, not advise any approach that would lead to the use of such large properties and will review any areas of configuration that need additional flexibility as they’re reported to us.

• The path/name of the low-level metrics providing timing data regarding dynamic metadata resolution have changed to avoid exposing internal class names and better align to the expanded metrics available in this release. If scripts depend on these JSON structure paths and can’t be updated easily, the base name of the various metrics can be adjusted via the metricsBaseName setting added to the https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199506698 schema.

• A new feature of the MFA login flow is the ability to better control the value of the timestamp that is associated with the results and eventually communicated to SPs. Prior to V5, the timestamp defaulted to the latest timestamp found on one of the “embedded” results being combined by the flow (e.g., when combining the results of the Password and Duo features). V5 defaults to using the earliest timestamp instead, and a property (idp.authn.MFA.useLatestTimestamp) has been added to control this.

• Various APIs have been converted from reliance on Spring’s Resource interface to our “equivalent” non-Spring Resource interface due to inconsistencies in the way resources were “derived” from strings in the configuration. In most cases, the locations of files tend to be handled with properties or Strings and this is compatible; in unusual cases where actual Spring Resource class types were defined and injected into some beans, this may break and require adjustments to the configuration.

• The internal default CAS ticket service has been switched from the original server-side storage version to the encrypted/encoded service that embeds the required state into the ticket. Deployers whose configurations include the shibboleth.CASTicketService bean will see no change as this bean overrides the internal default. The main impact of this change is that ticket size increases (affecting some broken clients) and that multi-hop proxying is unsupported. If the latter is needed, reverting to the in-memory version is required.

The following defaults have changed for new installs but will not change for properly upgraded systems (generally these are set through explicitly uncommented properties or beans that have to be present to enable a behavior):

• None at present

Installation and Installation Commands

The following should be noted:

• The install command is only available inside the distribution, and all installation (including property based installation) should be done using this script.

• Many options can now be given to the installer as command line options. Nonetheless, https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500711 is still possible although some properties are no longer available. Properties cannot be injected on the command line.

• All user visible or modifiable files are controlled by one of three new modules (idp.Core, idp.CommandLine, and idp.EditWebApp). This means that while local changes will almost always be retained, you will be able to see a side by side version of each file.

• Only batch files appropriate to the operating system (.sh for Unix and .bat for Windows) are now populated on first installation. The “wrong” type of file will generally be left behind from older installs and can be removed by hand.

• A new command update is available in bin. By default this will check to see whether the IdP can be upgraded. In addition it can download and check the signature on the upgraded distribution. Additionally the IdP itself checks for available updates when it starts, and logs the result. This can be disabled with the idp.updateCheck.enable property.

• The Windows msi installer has been substantially rewritten and is now split between a relatively standard IdP Installer and an installer for Jetty and our jettty-base deployment files.

• When enumerating all installed plugins and during various other operations, the plugin installer will now warn if a required module is unavailable.

• If the plugin installer changes the plugin state (by installing, removing or updating a plugin) it will output the resulting plugin state when it completes.

Changes to web.xml

We have heavily revised the content of the deployment descriptor file with an eye on reducing the need for deployers to override it, and reducing future conflicts if it is overridden. Much of the original boilerplate in the file is now gone and the content is now more configurable via Spring (via properties, etc.) and is auto-registering itself at runtime using newer servlet API features.

We have attempted to preserve as much compatibility as possible with V4-compatible versions of the file, but if you have overridden this file via edit-webapp, examining the new version in dist/webapp/WEB-INF/web.xml and modernizing the overridden version (or even better, eliminating it) is strongly encouraged.

Note that it is possible in most servlet containers to override context parameters using special container-specific hooks, usually in the context deployment fragment that specifies the location of the idp.war file. The new version of web.xml contains a few context parameters that can be used to disable the RemoteUser and X509 servlets if they are not in use, and it is advisable to override those parameters using the container rather than by overriding web.xml.

Profile Configuration API Changes

A few properties on the various profile configuration beans used in relying-party.xml have been moved out of some shared interfaces and classes and on to specific instances of these beans where specifically appropriate. We do not expect this to impact deployers. For example, this change makes it impossible to set signAssertions as part of the SAML Logout profile configuration (because there are no assertions to sign). A number of such incongruities were fixed in cases where a property should never have been set due to it being non-functional.

If the system fails to run correctly and indicates the relevant service’s configuration isn’t valid and/or the log is full of issues around the various profile configuration beans in that file, reviewing any unusual settings against the documentation is a good idea.

General Configuration Changes

Prior releases warned, but did not fail, on the presence of dependency or failover elements inside a Static DataConnector. The schema has been updated to actually preclude them since they are unsupported.

Some features in older versions tended to assume the presence of relatively short XML files specific to those features, such as the various <flow>-authn-config.xml files. These files are often now optional and superfluous. In such cases, when a feature no longer relies on any required files, no Module is necessary to create or remove them, and so a number of Modules present in V4 no longer exist in V5, in particular a number of the simpler login flows such as External, RemoteUser, etc. This does not mean the features have been removed or changed, or that existing files won’t continue to work. Generally the specific documentation on a given feature will note that a particular older file may be evaluated for removal.

HttpClient Changes

The Apache HttpClient library has been updated to V5. Most of the changes to this library have been accomodated by our internal Spring declarations and we do not expect most deployers to run into compatibility issues, but testing any custom client definitions is advised. Notably, the deprecated legacy beans that used to be provided for custom client definitions have been removed, per the https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199510442 documentation.

One breaking change involves HTTP Basic Authentication. While it was possible to make that work in earlier releases, the APIs used do not allow for pre-emptive authentication to avoid extra round trips, which is a fairly major issue with IdP use cases. We have added new APIs and parent beans to facilitate this in a supported and hopefully forwardly-compatible way, per the https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199510442 documentation.

LDAP Changes

The ldaptive library has been upgraded from v1.3 to v2.x.

Connection Strategy

General improvements have been made to quarantine unresponsive URLs so they are considered inactive for a period before being tried again. Improvements have been made to the ROUND_ROBIN connection strategy to make it stateful and bring it’s behavior inline with user expectations.

Lost Connections

The behavior in v1.3 for lost connections was lazy reinitialization, a connection wasn’t known to be lost until it was used. In v2.x when connections are lost they immediately attempt to reconnect. This feature may produce unwanted noise in logs but is generally beneficial. Note that it can be disabled.

New Properties

• Added startTLSTimeout bean property to authentication and resolver configs. Allows separation of responseTimeout, which may need to accommodate long searches, from startTLS which should occur quickly.

• Added autoReconnect bean property to control whether lost connections automatically reconnect.

Logout Changes

An explicit workaround for Internet Explorer in the reporting of logout status (which itself is not a suggested approach anymore) was removed due to its reliance on an unsupported Java library to detect the browser. Given that IE is no longer supported by Microsoft, we removed the unsupported detection logic.

Removed Features

A number of previously deprecated features, settings, APIs, etc. have been removed. The use of these deprecated bits should have produced a standard warning either at startup or during use in the most recent releases of the software, and most of them are now non-functional or outright breaking. Eliminating warnings before upgrading is strongly advised.

The following non-exhaustive list of previously deprecated features have been removed:

• Support for SAML assertion delegation (virtually undocumented for several releases)

• The Password login flow’s Extended Flow feature (supplanted as far back as V3.3 with the MFA feature)

• The “legacy” authn/Duo login flow reliant on the Duo “iframe” integration style, which Duo has replaced and is retiring in 2024. See the DuoOIDC plugin for the replacement feature.

• The original bean facades for HttpServletRequest/Response (replaced with new “Supplier” beans in V4.3, per https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500962)

• Legacy V2 compatibility scripting support in the Attribute Resolver (any references to edu.internet2… package names will break)

• Support for the exportAllAttributes setting in DataConnectors

• The EntityRoleWhiteList MetadataFilter type name has been removed, supplanted with the EntityRole type name

• The MappedAttributeInMetadata filter policy type name has been removed, supplanted with the AttributeInMetadata type name

Removed Objects

The following beans are no longer defined, so any references to them will break. Checking your files for them is a good sanity check.

• shibboleth.HttpServletRequest

• shibboleth.HttpServletResponse

  • Per https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500962

• shibboleth.NonCachingHttpClient

• shibboleth.MemoryCachingHttpClient

• shibboleth.FileCachingHttpClient

  • Removal of old, deprecated HttpClient beans

• shibboleth.NonFailFastValidator

  • Replaced by the failFastInitialize setting on various DataConnectors

• shibboleth.OutgoingIDWSFSSOSBindings

• shibboleth.Encoders.IDWSFSOAPEncoder

• shibboleth.Binding.LibertySOAP

• shibboleth.Conditions.IssuingDelegatedAssertion

• shibboleth.Conditions.AllowedSAMLPresenters

  • Part of the removed support for SAML delegation

• shibboleth.authn.Password.PrincipalOverride

• shibboleth.authn.Password.ExtendedFlows

• shibboleth.authn.Password.ExtendedFlowParameters

  • Part of the removed “Extended Flow” feature of the Password login flow

• shibboleth.RelyingPartyConfigurationResolver

  • Old facade for RelyingPartyConfiguration service, not generally used

Removed XML Settings

The following settings have been removed from various schemas:

• From all DataConnectors:

  • springResources

  • springResourcesRef

  • factoryPostProcessorsRef

  • postProcessorsRef

  • exportAllAttributes

• From all AttributeDefinitions, DataConnectors, and AttributeFilter rules:

  • profileContextStrategyRef, contextStrategyRef, profileContextRef, etc.

• From all AttributeFilter rules and the Mapped AttributeDefinition:

  • ignoreCase (long replaced by the inverse, caseSensitive)

• From StoredId DataConnector:

  • failFast

• From RelationalDatabase DataConnector:

  • readOnly

• From LDAP DataConnector:

  • searchExecutorRef (replaced by searchOperationRef)

Removed Properties

• idp.service.attribute.resolver.suppressDisplayInfo

• idp.httpclient.memorycaching.maxCacheEntries

• idp.httpclient.memorycaching.maxCacheEntrySize

• idp.httpclient.filecaching.maxCacheEntries

• idp.httpclient.filecaching.maxCacheEntrySize

• idp.httpclient.filecaching.cacheDirectory

• idp.duo.*

• idp.authn.duo.*

Newly Deprecated Objects

• Liberty.SSOS

• Liberty.SSOS.MDDriven

• shibboleth.SameSiteCookieFilter (referenced in older web.xml)

• shibboleth.ResponseHeaderFilter (referenced in older web.xml)

New Objects

• shibboleth.Conditions.EntityAttributes

• shibboleth.Conditions.MappedEntityAttributes

• shibboleth.TagCandidate

• shibboleth.TagLookupString

• shibboleth.TagLookupBoolean

• shibboleth.TagLookupInteger

• shibboleth.TagLookupLong

• shibboleth.TagLookupDouble

• shibboleth.TagLookupDuration

• shibboleth.TagLookupList

• shibboleth.TagLookupSet

• shibboleth.TagLookupBean

• shibboleth.HttpHost

• shibboleth.BasicAuthCredentials

• shibboleth.UserFacingPrefixPredicate

• shibboleth.authn.discoveryRequestFunction

New Properties

• opensaml.config.xml.unmarshall.strictMode

• idp.updateCheck.enable

• idp.cookie.bufferingCondition

• idp.responseHeaderCondition

• idp.encoding.forceUTF8

• idp.logging.MDC.enabled

• idp.logging.MDC.createSession

• idp.authn.SAML.NameIDLookupStrategy

• idp.authn.MFA.useLatestTimestamp

• idp.session.persistent