Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Warning

Upgrades MUST be done in place!

If you are upgrading from V3 or V4.0, PLEASE follow the directions on the Upgrading page and do NOT install V4.1 the new version separately and then attempt to apply your old configuration files. You MUST upgrade "in place" by using an installation directory that contains a copy of the previously working configuration.

Failure to do so will result in a non-working system in the majority of cases because there are absolutely essential differences between the output of the installer in the two cases to prevent your configuration from being altered in unexpected ways.

This problem is exacerbated by the extent of the internal changes made to V4.1 and the results of combining old and new files without an extreme amount of effort will be unpleasant. It is urgent that people heed this warning and attempt to modernize settings (if at all) only after upgrading and verifying the behavior of the system.

Please review these release notes before upgrading your system. You should review the notes for all the versions subsequent to the one you're running prior to upgrade, including referring back to older V3 notes.

...

Note

Upgrade Plugins First

Less critically than the above warning, it’s advisable to upgrade any plugins you may have installed (meaning official plugins using the machinery introduced with V4.1 of the IdP) before upgrading the IdP itself, as this may prevent some extra warning noise in the log (or outright incompatibility issues noted by the installer). Upgrade and test those plugin features, and once those are validated, you can proceed to upgrade the IdP itself.

Please review these release notes before upgrading your system. You should review the notes for all the versions subsequent to the one you're running prior to upgrade, including referring back to older V3 notes.

Also be aware of the following issues regarding container or Java compatibility:

Known Bugs

These are known issues in the code that are likely to affect deployers and have yet to be fixed or that require some intervention because of how upgrades work.

  • https://shibboleth.atlassian.net/browse/

    Jira Legacy
    serverSystem Jira
    serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506
    keyIDP-1820

    • This is a change required to web.xml for systems that have been upgraded from prior to V3.2 and have been maintaining their own copy of web.xml since then. The fix needed for this is documented below under V4.1.0 under Breaking Changes → Deployment Descriptor Issue.

    https://shibboleth.atlassian.net/browse/
  • Jira Legacy
    serverSystem Jira
    serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506
    keyIDP-1625

    • This issue breaks the Cancel option in the Duo login flow when the CSRF feature is enabled. The fix is documented in the issue and can be applied directly to the duo.vm template. Since it involves a template, the fix is manual.

...

...

4.

...

3.

...

3 (April 15th, 2024)

This is a patch release to address a Spring Framework advisory, a repeat (actually third time) of the same URL parsing bug that resulted in the previous update and impacts the CAS protocol support feature.

4.3.2 (March 21, 2024)

Jira Legacy
serverSystem Jira
jqlQueryfilter=10065
counttrue
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506

...

This

...

It includes an automatic behavioral change that tracks the endpoint used to deliver an assertion when starting a session, and uses that URL when selecting a logout endpoint to use if there are multiple endpoints spanning different virtual hosts or paths. The endpoint selected will contain the longest matching sequence of characters starting from the beginning of the URL(s). This approach is notably more compatible with Shibboleth SPs that are virtually hosted with a single entityID.

Another automatic change eliminates attempts to issue logout requests to SAML 2.0 SPs whose metadata contains no logout endpoints. This should reduce the extra noise of EndpointResolutionFailed events in the log and improve performance.

...

Miscellaneous Changes

...

New Properties

...

New Beans

...

New Messages

...

is a patch release to roll up bug fixes and update dependencies on the old branch of the software, and intended to be the final V4 release barring additional security issues requiring further releases. It addresses the Spring Framework bug noted in an advisory.

4.3.1.4 (Windows only) (November 3, 2023)

This is a service release of the Windows installation package which updates Jetty to 10.0.18, addressing a potential memory leak that might impact some deployers.

4.3.1.3 (Windows only) (October 11, 2023)

This is a service release of the Windows installation package which updates Jetty to 10.0.17, addressing security advisories issued by the Jetty Project. These vulnerabilities only impact use of HTTP/2, which is not supported by our delivered Jetty installation, but we are updating out of caution.

4.3.1.2 (Windows only) (September 15, 2023)

This is a service release of the Windows installation package which updates Jetty to 10.0.16, addressing security advisories issued by the Jetty Project.

4.3.1.1 (Windows only) (April 19, 2023)

This is a service release of the Windows installation package which updates Jetty to 10.0.15, addressing a pair of security advisories issued by the Jetty Project.

4.3.1 (March 30, 2023)

This is a patch release to address pair of regressions (one serious).

Jira Legacy
serverSystem Jira
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506
keyIDP-2084

Jira Legacy
serverSystem Jira
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506
keyIDP-1685

There are no other changes in this release.

4.3.0 (January 18, 2023)

Jira Legacy
serverSystem JIRA
jqlQueryfilter=10058
counttrue
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506

This is a small(ish) feature release expected to wrap up work on the V4 branch. It contains some additional deprecation and at-risk warnings for features either scheduled for removal in V5 or being considered for removal in a future version. Please check your warning log regularly for these warnings to ensure that you aren’t caught off guard or have the opportunity to express any concerns regarding any at risk features.

Changes to Existing Behavior

When proxying (i.e., using the SAML login flow feature), the IdP will now include a <Scoping> element in the request containing the original requesting SP's entityID. This is a requirement of the standard and was a bug by omission. In the event that you wish to suppress this behavior either out of concern for privacy or due to bugs on the part of other actors in the system, the ignoreScoping profile setting will suppress the generation of (and the evaluation of) the element altogether, though this is contrary to the standard.

A change was made to the feature allowing the IdP to auto-search for property files to load at startup. Prior to this version, if any directories under the search tree were unreadable due to access control, IdP startup would fail. With this release, the code is now trapping that error and startup continues, allowing this to be done deliberately to hide content from the IdP. Since this error can only be logged to the container’s log (it happens prior to the IdP starting up), it may be less obvious that it’s happening if it’s by mistake.

Deployers should ensure their containers are logging appropriately in some way (this is outside the scope of the IdP itself, though our Jetty examples do include it) and be aware there could be helpful information there.

Miscellaneous Changes

The most impactful change is a harbinger of a necessary adjustment that will be coming with V5.0 impacting any scripts or components that require injection of the HttpServletRequest or HttpServletResponse via the shibboleth.HttpServletRequest and shibboleth.HttpServletResponse beans, which are now deprecated in favor of a new pair of beans, shibboleth.HttpServletRequestSupplier and shibboleth.HttpServletResponseSupplier.

For the time being, uses of those beans will produce deprecation warnings, but they will be removed in V5.0. Remediating the warning requires adding the “Supplier” suffix to the end of the bean names at the point of injection, and then adding a call to get() in between the original reference to the bean and the servlet method called.

For example, this script line relying on the “custom” extension point to access the user agent string:

Code Block
var ua = custom.getHeader("User-Agent");

becomes this (the only change is the additional indirection through the get() method):

Code Block
var ua = custom.get().getHeader("User-Agent");

The quickest way to remediate the issue is to grep for the old bean names, adjust them whereever they’re spotted, and then track back to where they’re used (usually via scripts as in the above example) and add the indirection method.

See also Moving to Suppliers for accessing HttpServlet Objects

New Features

New Properties

New Beans

...

4.2.1.1 (July 6, 2022)

This is a service release of the Windows installation package which addresses a Jetty logging issue and updates Jetty to 10.0.11 to address a memory leak.

4.2.1 (April 18, 2022)

This is a fixed package to address a regression that caused the spymemcached library to be omitted. Only users of the memcache storage option would be impacted.

4.2.0 (April 15, 2022)

This is a relatively “light” feature release that mainly exists to support API changes needed for some of our plugins, which have been updated along with it and require this version to use.

Jira Legacy
serverSystem JIRA
jqlQueryfilter=10035
counttrue
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506

Changes to Existing Behavior

A regression in all 4.x releases was identified in the way the Attribute-Based Subject C14N feature handles scoped attribute values such that only the value portion would be returned and used, ignoring the scope. For example, an eduPersonPrincipalName of “foo@example.org” would be returned as “foo”. This was easily fixed, but MAY impact existing behavior if the “broken” behavior were relied on. This can be remedied by adjusting the configuration to transform the scoped value back into an unscoped one but is something that could alter behavior following an upgrade until it’s addressed.

The support for auto-loading files that end in “.properties” was apparently broken in such a manner that the V4.1 releases may have excluded any file also named idp.properties regardless of location. The new version was fixed to properly handle all paths as unique regardless of filename and load such files, so you should NOT store any “backup” files or copies of your property files inside the conf tree. Note that the new release also logs any duplicate properties it detects for debugging. This will normally be sent to the container’s logging sink because it happens before the IdP is “started” enough to log things itself.

A long-standing design flaw was fixed so that deployers can configure inbound SAML profile interceptor flows without having to “repeat” a built-in interceptor flow that implements some of the inbound security handling logic. We are unaware of widespread (or any?) use of the inbound profile interceptor hook, but if it’s in use, there are possibly unneeded duplicate references to a flow ID such as “security/saml2-sso” or similarly-named flows that need to be removed. The duplicate reference will cause message replay failures because the flow will be run twice.

The CAS implementation was modified to better support consent by storing consented attribute IDs inside service ticket state for recovery during validation. This allows consent to be enabled and honored during the first hop of ticket validation, even if the consent is stored client-side or was granted only for a single request.

A bug dating back to the earliest Shibboleth releases was corrected to prevent the IdP from issuing duplicate SAML Attributes with the same fully-qualified name. The IdP now detects this and merges the values such that a single Attribute is produced with all of the values. This seems to have been a behavior people had somehow depended on, though doing so would depend on bugs in an SP in the first place since the result of processing 2 Attributes with 1 value and 1 Attribute with 2 values cannot be different in a compliant SP implementation.

User Interface Changes

The original view templates and stylesheets have been replaced in this release with a freshened set that are intended to be more modern and accessible. The look and feel is still assumed to be inadequate for any use other than testing, but may provide a better baseline to adjust. The name/location of the “common” stylesheet is adjustable with a message property.

The new placeholder stylesheet and logos have different filenames from earlier versions, so are installed automatically on new or upgraded installs. This is necessary so that installing modules that copy in new Velocity templates will result in at least a usable configuration. Older views that already exist will never be overwritten by upgrades, so they will continue to reference whatever stylesheets were used before.

See the UserInterface topic for more information.

Another “UI-adjacent” change is adding support for post-authentication interceptor flows into the Administrative flow layer, in particular the Hello World flow. This allows easier demoing and testing of features like consent or expiring password warnings within the IdP itself.

Logout Changes

This release contains a few new options and optimizations to improve logout behavior and quiet noise in the logs, and are worth a review if you operate an IdP with a lot of SPs that do not support logout.

It includes an automatic behavioral change that tracks the endpoint used to deliver an assertion when starting a session, and uses that URL when selecting a logout endpoint to use if there are multiple endpoints spanning different virtual hosts or paths. The endpoint selected will contain the longest matching sequence of characters starting from the beginning of the URL(s). This approach is notably more compatible with Shibboleth SPs that are virtually hosted with a single entityID.

Another automatic change eliminates attempts to issue logout requests to SAML 2.0 SPs whose metadata contains no logout endpoints. This should reduce the extra noise of EndpointResolutionFailed events in the log and improve performance.

A new property named idp.logout.assumeAsync can be enabled to handle SPs that can issue logout requests but do not properly handle inbound logout requests or responses. Enabling the option allows an IdP administrator who controls the SP's metadata to remove the broken logout endpoints from the metadata without preventing the handling of logout requests because of "unable to respond" failures.

A new property named idp.logout.propagationHidden can be enabled to hide the list of services and logout status during logout propagation. Enabling this will require other template changes to properly report the logout to the user but allows the logout propagation to be hidden without editing style sheets or changing system files.

A new bean named shibboleth.PlaintextNameIDFormats can be defined to suppress encryption of particularly <NameID> formats that are not sensitive enough to require encryption (e.g., transient IDs when query support is not enabled). This further optimizes performance when lots of logouts are propagated at once.

Session Enhancements

The bean shibboleth.SessionAddressLookupStrategy can be used to source the “address” associated with an IdP session in an arbitrary way, allowing a wider variety of techniques for managing session affinity with clients. The IdP no longer requires that the address be in the form of an IP address, though it remains aware of that as a default.

Support for adjusting cookie names used by the IdP has been extended to cover most/all use cases, including the session cache. This is NOT intended to encourage any dependency on the names of the cookies, but rather to support ongoing hackery around the web for changing cookie behavior based on name, notably the Google invention of a __Host- prefix to prevent domain-scoped cookies from being fed into sensitive services. See Cookie Prefixing for examples.

Miscellaneous Changes

Display name and descriptive information associated with attributes used on the consent view is now determined in a just-in-time fashion. This reduces the processing needed for those deployments (or specific attributes) which do not require consent. This change should be irrelevant unless you are using an externally-developed feature using the old (and now deprecated) APIs. Legacy behavior can be re-established by using the idp.service.attribute.resolver.suppressDisplayInfo property.

The startup sequence now logs any duplicate properties it finds to help with debugging. This is generally only visible in the container log prior to IdP “startup” so requires proper logging configuration at that layer.

New Properties

New Beans

New Messages

...

4.1.7 (April 18, 2022)

This is a patch release to update to Spring Framework 5.3.19 in response to the RCE vulnerability reported against Spring MVC.

4.1.6 (March 31, 2022)

This is a patch release to update to Spring Framework 5.3.18 in response to the RCE vulnerability reported against Spring MVC.

4.1.5 (January 19, 2022)

Jira Legacy
serverSystem JIRA
jqlQueryfilter=10043
counttrue
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506

This is a patch release primarily to update logback, the logging solution we provide by default. A few other dependencies were also updated to address open CVEs, including Spring.

The new logback version was issued in response to the log4shell debacle, and includes some hardening of features and the removal of other features to reduce the attack surface, though logback itself did not have any meaningful vulnerability that wouldn’t require compromise of its configuration file to begin with.

While it’s possible some deployers may have made use of the removed features, we’d rather err on the side of caution and the removed features will eventually be impossible to support anyway in order to keep up with any subsequent logback versions. Those with a real need for any of the removed features could choose to switch to an alternate logging solution, though some custom development will usually be required for that.

Please note that the IdP does not, and will never, require write access to its configuration. Many potential vulnerabilities may rely on elevation of privilege by leveraging one bug to make a change to a configuration file that in turn exposes the system to an RCE. It is a good idea to make sure the configuration is owned by a different account than the one used to run the IdP’s JVM, and so make the configuration read-only. Just making the files read-only is not sufficient since ownership is enough to allow the mode to be changed back. This is not new, and is very general advice, but it’s worth remembering.

The version of Jetty provided with the Windows installer has been refreshed to the latest 9.4 revision. Note that some earlier versions dating back to 9.4.39 and possibly some later revisions exhibited a bug on Linux observed to cause low priority CPU exhaustion (noticeable but not such that higher priority activity would be interrupted). This issue resolved itself with the version included in this release. It is unknown to what extent this issue may or may not impact Windows.

4.1.4 (July 27, 2021)

This is a patch release that supersedes V4.1.3 and fixes a regression introduced into that version.

...

Jira Legacy
serverSystem JIRA
jqlQueryfilter=10034 counttrue10034
counttrue
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506

This is a patch release, primarily addressing

Jira Legacy
serverSystem Jira
serverIdf52c7d31-6eab-3f0e-93c3-231b5754d506
This is a patch release, primarily addressing https://shibboleth.atlassian.net/browse/
keyIDP-1833
, which prevented the new Duo plugin from functioning on Windows.

...

If you use the X509 login flow, the workaround required to address the regression in V4.1 is no longer required, so you should remove edit-webapp/WEB-INF/classes/net/shibboleth/idp/flows/authn/x509-authn-beans.xml from your system prior to upgrading, or do so and rebuild the war after.

4.1.0.1 (Windows only) (April 5, 2021)

This is a service release of the Windows installation package which updates Jetty to 9.4.39, addressing several security advisories issued by the Jetty Project.

...

This is a significant new feature release that includes a larger than usual number of new configuration options, but these are backward-compatible and mostly simplify things for new deployers. The new Plugin and Module layers are a key addition in support of an initial set of add-on features described in the IdP Plugins wiki space.

A section has been added to the Upgrading page specific to this release. Note especially that previous versions of the OIDC OP extension are not compatible with this release, nor will many of the older OIDC configuration settings work with the new plugin without some (usually small) alterations. See OIDC OP Upgrading for details.

...

Also take note that with this release, the process of "finding" property files has been improved. In older versions, and when the idp.searchForProperties property is false/missing, the set of property files loaded is solely controlled by the idp.additionalProperties setting. New installs by default, or systems that explicitly enable the idp.searchForProperties property, will locate and load all files with the ".properties" extension in the conf tree. The explicit property can still be used for directly loading files from alternate locations.

Note that there are no plans to actually break or remove support for the older ways of configuring many features; the property additions are purely for simplification for newer deployers or for those who wish to take advantage.

Auto-Wired Components and Logging

To better accomodate accommodate plugins, the system now relies on Spring to automatically locate and consume beans of certain types at runtime instead of requiring lists of beans in many places in the user-editable configuration.

...

In addition, there are many "optional" files that are no longer installed by default, inclusing including some XML files and views, because they are part of Modules that enable optional features. Turning on some features in the configuration will require enabling the corresponding module, or things will not function correctly in ways that will vary from failure to start the IdP, to exceptions at runtime, to log messages about missing files.

...

New Properties

There are a lot of new properties because of the aforementioned changes, but a few miscellaneous settings are mentioned here:

...

This is the first release of the fourth-generation Identity Provider software. The key documentation links are located on the IDP4 space Home page, such as SystemRequirements, Installation, and Upgrading material. Note the new SystemRequirements as they have substantially changed with regard to Java and container versions.

...

  • The idp.cookie.secure property now defaults to true if not set by the deployer. In the unlikely event that HTTP compatibility is a requirement, the property would need to be adjusted. (Note that the container's session cookies are controlled by policy in web.xml and while we have also changed that default, only upgraders without their own web.xml customizations would be exposed to that change.)

  • The default LDAP provider is now UnboundID. See the LDAP section below for more information.

  • Some legacy code that was used to maintain the V2-style formatting of the %T audit field (the timestamp of the record) was removed and the log now outputs all date/time fields using the supplied formatter or a default. The format of the %T field may therefore change for some deployers.

    • Related, the shibboleth.AuditDateTimeFormat bean is interpreted as a pattern by the Java DateTimeFormatter class, rather than the deprecated joda-time version, and there are some differences in those classes that may result in changes to your logs. If you require identical output, review is warranted.

  • A side effect of the attribute-related changes noted below is that SAML 2.0 AttributeEncoders with no friendlyName attribute default to populating FriendlyName with the attribute's ID. At present there is no way to restore the original behavior other than to convert to use of the AttributeRegistryConfiguration to express the rule.

...

Note
Beware of Guava's "compose" method

Legacy use of Guava Functions may work for the most part, but there is a known issue in Spring's wiring behavior that breaks the use of Guava's "compose" helper. If you have custom bean wiring anywhere that contains a reference like this, you will likely have to update it to avoid problems:

<bean class="com.google.common.base.Functions" factory-method="compose">

Using that syntax will often result in the two constructor arguments to that bean getting swapped/reversed, which reverses the order of the composition, usually producing bad output at best, and a crash or even more catastrophic outcome at worst.

Unfortunately there's no easy fix for this because of the way Guava accomodates accommodates the conversion to Java 8's Function interface. As a result, the best thing you can do is simply scan for this and convert the above syntax to:

<bean parent="shibboleth.Functions.Compose">

That is functionally equivalent and should avoid the re-ordering behavior we're seeing with Spring.

...

The ldaptive JAAS module has an incompatible change in its handling of timeout settings. Specifying values in millseconds milliseconds will result in a DateTime parsing exception. The values must be converted to ISO syntax (e.g., 2000 → PT2S).

In addition, various LDAP pooling properties prefixed by "idp.pool.LDAP." (such as idp.pool.LDAP.validatePeriod) previously defaulted to numeric values expressed in seconds (e.g. 300 == 5 minutes). These are now being interpreted as milliseconds, which causes overly frequent pool validation if the defaults are used with an unmodified V3 ldap-authn-config.xml file. This is dicussed discussed further on the LDAPAuthnConfiguration page.

Finally, the LDAPConnector's maxResultSize option's behavior is different from V3 in that the connector will raise an error if the size is exceeeded exceeded rather than ignoring this condition.

...

A major new addition to V4 is the AttributeRegistry service that replaces, or at least supplements, the AttributeEncoder mechanism for translating attributes. While upgraded systems should operate unchanged, deployers are advised to review this new feature and may achieve substantial simplification of their configuration by taking advantage of it. Many existing encoders may be removeableremovable.

Note

Note that installing from scratch and then applying a legacy configuration will in most cases result in duplicate Attributes appearing in SAML messages due to the overlap between the existing encoders and the new rules. The upgrade process is designed to prevent that by excluding the new rules from the default configuration. Of course, the inverse is also true: if you want the new default rules to apply, you must make that transition after upgrading.

A consequence of this new service is that adjusting AttributeEncoders requires reloading "shibboleth.AttributeRegistryService" and not the resolver “shibboleth.AttributeResolverService” service as in previous releases.

...

Such elements are invalid according to the standard and the IdP's behavior is correct, but the initial release had a bug that results in a NullPointerException instead of ignoring the element. The crash was corrected in V4.0.1. Deployers should not in general rely on the IdP to be tolerant of invalid SAML constructs.

Logout Changes

The Single Logout feature now includes some additional control points, properties, and changes to the default views that allow more control over how logout proceeds, whether users are able to cancel/abort them, and other requested "flow of control" enhancements. These are discussed in the LogoutConfiguration topic.

...

Due to a Spring bug that causes problems with configuration reloading, we have deprecated the original set of HttpClient parent beans and any inheritence inheritance from those beans in custom HttpClient configurations. We have introduced new parent beans without predefined settings that were causing the problems, and adjusted the HttpClientConfiguration documentation in various ways to refer to them.

...

New Properties