MigratingFromV3

The V4 SP is not directly upgradable from V3 or earlier, because of the radical changes to the design. The configuration of the old SP is generally “split” between the new Agent configuration, which is radically pared back, and the new Hub’s configuration, which is Spring-based and reuses much of the IdP’s existing configuration while adding a new component/service for configuring Agents and some of their behavior.

In the majority of cases, migrating an older SP deployment to the new Agent is relatively simple, while migrating to the new Hub is less so. Migrating an Agent is discussed here, while this topic discusses how to create a Hub configuration in consideration of an older SP deployment.

It is suggested that deployers review the entry-level material about deploying the Hub, particularly the GettingStarted and SAMLBasics topics (as the legacy SP inherently was a SAML deployment, and so understanding the basics of setup will help with understanding what migration will entail).

It is not so much a migration as creating a Hub deployment to emulate the behavior of the older system. There is much more variability in this than moving to the new Agent, other than obvious considerations around the externally visible aspects that are relevant to IdP(s) that were supporting the older SP. It is absolutely possible to deploy the Hub and new Agents to act in a manner entirely identical to an older SP such that it is a transparent migration. But it is not necessarily always the best choice to do so, apart from the fact that there is no reason to ever change an SP’s entityID, short of selling off a domain.

The following sections are organized around different parts of the older system and its configuration, and at the end notes the parts of the configuration that are either gone, ignorable entirely, or are strictly an Agent consideration and so addressed there.

Impact on Application Code

The Hub does not directly impact most application behavior anymore than “shibd” ever did (despite misconceptions that it did), so this is not really a huge consideration, with the major exception that the Hub is entirely responsible for producing and delivering the Attributes that an Agent will receive when it establishes sessions. While the Agent can in fact rename any Attributes it gets and so maintain a locally-defined interface to its applications, it cannot create data where none exists.

Thus, migration must take into consideration exactly what Attributes a legacy SP was extracting or producing for its use and ensure that the Hub will provide a superset of that to its Agents (or coordinate a change to that contract as appropriate).

In most other respects, the Hub does not concern itself with details about its Agents at the level of specific content, and does not provide any support for authorization rules. That, and other settings that are expected to require “content-aware agility” are confined to the Agents.

URL Awareness

The Hub does not in fact know or (by default) limit what URLs an Agent operates under. While an SP’s metadata (in both SAML and OpenID Connect cases) will generally need to contain some of those specifics, the Hub doesn’t consume or produce that metadata and so in most cases it does not know the information, nor does the Hub’s confguration need to change to reflect any changes to it.

There is one specific exception to this that is fairly limited: the legacy SP eventually gained support for self-assigning its own SAML entityID on the basis of the URL being accessed, which allowed for easier deployment of virtually hosted web farms if use of a single entityID was not preferred. The Hub does not implement this exact feature, but because it is responsible for the determination of the entityID under which it operates for a given transaction, it has the flexibility to derive the value with an awareness of the particular URL in use to emulate the previous behavior. (We will provide code for this out of the box, but it is not present just yet.)

Logging

The Hub of course has the same logging capabilities as the IdP software and indeed because it operates within the IdP as a set of plugins and reuses many of the same Java classes, it is impossible to isolate it from the IdP’s logging entirely. This is one of many reasons that operating the Hub inside an actual IdP is not precisely expected or recommended (and not really one of the bigger reasons).

The diagnostic logging of the Hub does include new MDC fields that capture the Agent and Application making requests, to extend the logging format generically to include this information.

The Hub does (or will) have its own Audit log stream, though it is based on the same code and machinery as the IdP’s Audit logging. This replaces the legacy SP’s little-used transaction log features, and the Agent has no comparable audit logging.

Remoting

The Hub is the “receiving” end of the requests issued by the Agent (as “shibd” was) but it is now operating as a web service within a Java servlet container as the IdP does. When co-located with an Agent, the Hub’s container might operate on localhost only on any port desired and TLS would be optional.

Obviously when serving remote Agents, TLS is required/assumed, and the Hub authenticates Agents by means of an ID and secret using HTTP Basic Authentication (something “shibd” did not support).

In either case, none of the existing settings from V3 are directly relevant and the existing <Listener>, etc. elements can be ignored, with the rare exception that the legacy SP did “unofficially” support IP-based restrictions when TCP was used. The Hub supports restrictions on access globally via the sp.agent.authn.allowedAddressRanges property in conf/sp/sp.properties or on a per-Agent basis in conf/sp/agents.xml via the p:allowedAddressRangesbean property that can be attached to a given Agent bean. In most cases, this would be a “new” constraint, not something carried over.

Session Cache and Persistence

While the Agent formally handles all aspects of managing sessions, the Hub supports the use of existing and future StorageService implementations in Java, as a replacement for the legacy SP’s support for the now-removed C++ version of that interface. The most common of these are probably the memcache support built-in to the IdP, and the JDBC plugin, both of which are fairly direct replacements for the older memcache and ODBC support that have generally not been all that robust.

There is no direct migration of the older configurations apart from the obvious similarities involved, but if a legacy SP were using one of them, it would be expected that the Hub would have to be outfitted with an equivalent bean (typically in conf/global.xml) to provide the same “back-end”, and there will usually be some additional new configuration needed to connect the Hub to it for a particular purpose, the most common of which would be for storing and retrieving Agent Sessions.

The legacy elements typically used to configure alternative persistence strategies for various features include the <StorageService>, <ReplayCache>, <ArtifactMap>, and <SessionCache> elements. The handling of the replay cache is actually governed by existing IdP configuration for that purpose.

Attribute Configuration

Most of the legacy configuration for extracting, filtering, and mapping Attributes is part of the Hub (reusing extant IdP services and configuration), so the legacy <AttributeExtractor>, <AttributeResolver>, and <AttributeFilter> elements and the referenced XML files deal with settings that have to be translated across to Hub deployment. Unfortunately with the exception of the filtering layer, the configuration of the legacy SP is not directly translatable other than by understanding what the intentions and use cases were in configuring various SP features and translating those.

To directly speak to the individual pieces of the legacy configuration:

• The extraction and mapping of Attributes via attribute-map.xml is implemented with the Hub’s use of the IdP’s AttributeRegistryConfiguration, which supports decoding SAML <Attribute> and <NameID> elements, OIDC claims, etc. Obviously the syntax is entirely distinct but the basic features are similar to define a mapping between the “wire” details and the internal ID and how to decode the values. The SP included some alternative “extraction” features, some of which exist in different form now, but in general the Hub has much more sophisticated features for this, including the usual IdP scripting capabilities.

• The legacy filtering layer (attribute-policy.xml) is implemented with the Hub’s use of the IdP’sAttributeFilterConfiguration. The Hub’s SAML plugin provides an example filter policy under conf/examples that emulates a significant portion of the legacy SP’s default policy. The SAML plugin also supports a property called sp.saml.acceptUnfilteredAttributes that allows default acceptance of all data without a specific policy applied, emulating the legacy SP’s wildcard feature. In the rare case that custom policies were created, the syntax of the older SP’s filtering layer is very similar to the IdP’s, as it was derived from that original schema and code.

• The legacy SP’s support for issuing SAML queries when using SAML 1.1 is irrelevant here because support for SAML 1.1 is not provided.

• Use of the legacy SP’sSimpleAggregationAttributeResolverfeature is replaced by the addition to the Java codebase of a SAML AttributeQuery DataConnector (docs TBD). In general, the IdP’sAttributeResolverConfigurationis a much more capable layer able to supplement data from many sources seamlessly and apply transforms of various sorts.

One difference between the Hub’s defaults and the legacy SP’s defaults is that the SP tended to use shorter/abbreviated names for the mapped data than the Hub uses. Obviously this is subject to change if desired, and can be quite easily massaged around using the Agent’s feature of remapping those names (see below).

In deploying the Hub, a major responsibility is to ultimately define exactly what IdPAttributes can be delivered to the Agent(s) and under what naming conventions, and ideally in doing so provide a sample agent.ini file for its Agents to facilitate connecting to the Hub and enumerating the IdPAttributes that are provided in the [attribute-mappings] section.

For example, if the Hub intends to supply these IdPAttributes to its Agents (either all or a subset perhaps):

• eduPersonPrincipalName

• eduPersonScopedAffiliation

• givenName

• sn

• displayName

• mail

It could produce a sample configuration for its Agents that emulates the legacy SP a bit more closely:

These mappings not only document the contract with the Hub, but also assist with allowing Agents to safely deploy Header-based attribute publication if it insists on doing so. Once in the Agent’s hands of course, those mappings can be altered to anything it wishes.

Metadata Configuration

Functionally this is a direct migration as SAML use at least will require the same source of Metadata as the legacy SP used. While the configuration is at least similar and the legacy SP’s feature set is mostly a subset of the IdP’s features, it is not a directly equivalent XML configuration. The IdP’s MetadataConfigurationmust be altered to connect to the same sources with the same protections applied.

One notable difference is that the IdP does not support the notion of a JSON-based “discovery” feed to be consumed by the EDS software, or any other SP features related to that. We have considered a few options for helping to plug that gap but would ultimately encourage federations to supply such a feed themselves and/or for deployers to consider migrating to tools such as Seamless Access, as it does a superior job, though at the cost of a globally-shared domain.

ApplicationDefaults / ApplicationOverride

Most of the settings controlled with the legacy SP’s <ApplicationDefaults>/etc. elements were used to control default SAML behavior and/or attach different components (such as Metadata or Attribute confiigurations) to different content. These elements acted as the defaults for what was possible to override per-IdP with the <RelyingParty> element (see below).

The Hub continues to support Application and Application Override constructs that are expressed as Java beans associated with particular Agent beans in the AgentResolverService configuration and that topic goes into detail about how that works. As has been true for a very long time, there are almost no scenarios in which an SP should ever make use of the <ApplicationOverride> element/feature today, and the new design makes that even less necessary and even more ill-advised. So the best thing to do as part of any migration is to undo such a thing or consider how to undo it in the process of migrating.

However, one can maintain overrides and use the same Application ID(s) to allow existing <RequestMap> or Apache configurations to connect to the Hub with the expected results. (That is, if you have an Application Override called “custom”, you can define it in the Hub with the same ID for that Agent, and the Agent should be able to make requests that will properly connect to it.)

The settings applied with these elements are either Agent-relevant (in which case they would now appear as ContentSettingsin the Agent), or Hub-relevant, in which case they would need to be considered by the Hub deployer during a migration. An example of the former would be the REMOTE_USER setting, while an example of the latter would be almost anything else, particularly entityID, and much of the content contained within the numerous XML elements found within the old <Sessions> element. The entityID setting can be carried over globally in conf/sp/sp.properties via sp.issuer, or can be set on a specific Agent or Application bean asp:issuer. It’s also not always SAML-specific (thus the change in name).

Most of the children of the <Sessions> element apart from the generic <Handler> element will contain settings that if not defaulted will usually require applying as properties to relevant places in the sp.properties, saml.properties, or agents.xml files in the Hub. More specific information about the various legacy Handlers is found in the next section.

One of the old features (though not recommended) was the ability to define dedicated sources of Metadata or dedicated Attribute mappings or filter policies wthin an <ApplicationOverride>. While it is possible to construct additional instances of the various services for Metadata Resolution, Attribute Resolution, Attribute Filtering, etc. and to attach them to Application beans in the agents.xml file, we do not at present have a practical way to do so in the Spring configuration. We also don’t recommend doing it as there are almost always better ways to address whatever the use case might be. In particular we have, or will have, various built-in conditions you can wire into those features that can limit their use on a per-Agent or per-Application basis to allow reuse of the single global instances of those services.

Handlers

The legacy SP defined (or auto-defined) various handlers using the elements inside the <Sessions> element. While the generic <Handler> element is pretty exclusively used for features that are either gone, or of interest only to Agents, most of the others are automatically built-in to the Hub. The discussion below is split into the more modern (V2.4+ or so) legacy configuration, and the much older configuration that came before that.

One key distinction is that the legacy configuration dealt with the paths to associate with specific handlers. That aspect of the configuration is solely an Agent consideration; the Hub does not know or care what the locations of these handlers in the Agent are, and to the extent it ever needs to know, the Agent provides the information when making requests to the Hub.

The Hub controls which SSO “profiles” can be used based on the beans defined in the agents.xml file’s shibboleth.sp.DefaultProfileConfigurations bean or by installing profiles on a finer-grained basis. (This is analagous to how the IdP enables profiles in its relying-party.xml file.)

Newer SP Syntax (<SSO>, <Logout>, etc.)

With the more modern configuration introduced with V2.4 of the SP, much of the SSO handler configuration was automatically wired up by the software and this is an approximation of how the Hub works now internally.

The most important settings and the ones that are essentially always needed would be the <SSO> element’s entityIDsetting, which directed the SP as to which single IdP to use, or the discoveryURLsetting that define use of an IdP Discovery service when multiple IdPs are used. (It is still an option for an Agent to define an entityID to use as a content setting or inside its handler configuration, so that specific requests will map to a particular IdP; this is likely to be a common/ongoing exception to the “no SAML awareness in the Agent” rule.)

In many cases, these values can be set globally, and will apply across all supported protocols, via properties in conf/sp/sp.properties:

• sp.defaultAuthority

• sp.discoveryURL

While logout has not yet been implemented, it is more likely than not that the old <Logout>element won’t be relevant to the Hub.

Older SP Syntax (<SessionInitiator>, <AssertionConsumerService>, <LogoutInitiator>, etc.)

The prehistoric approach of definiing individual handlers for all of the SP’s protocol features is superfluous, as many of the more advanced and little used features have been removed from this version. In particular, the Agent no longer supports custom “chains” of handlers, though the Hub has a built-in notion of chaining of Session Initiator and Token Consumer flows that requires little or no explicit configuration.

Most of the complexity from the old version had to do with a number of specialized approaches to IdP Discovery. All of those mechanisms are essentially encapsulated in the Hub’s support for redirecting to a standardized Discovery Service, which is then free to perform whatever trickery and magic is desired, without burdening the SP software.

RelyingParty Configuration

The legacy SP includes a feature somewhat modeled on the V2 IdP’s earliest concept of a RelyingParty override configuration to allow settings to be tailored to (in the SP’s case) specific IdPs. With a few exceptions, the settings here tend to be SAML-specific or sometimes very technical in nature. This is a responsibility fully devolved to the Hub in the new design and most of the settings, if they are supported at all, will be found on various Java interfaces exposed in the Hub’s new Agent Resolver service, and the various Application and profile configuration beans in it.

There are a few notable settings that are of concern to the Agent and not the Hub (e.g., control over population of REMOTE_USER), but it’s very rare to see them defined specifically to an IdP using this feature.

In most cases, the existence of a <RelyingParty> element in the legacy SP would correspond to a requirement for a RelyingParty override in the Hub’s configuration and/or a custom profile bean to supply a particular setting. You may also want to gravitate, at least with SAML, toward a metadata-driven approach, as per the IdP’s MetadataDrivenConfigurationfeatures, which is equally usable via tags added to IdP metadata.

One notable use case that’s likely more common than others would be the use of this feature to override the SP’s self-assigned SAML entityID (or OIDC issuer value) for specific IdPs, and that directly translates into Hub configuration using a RelyingParty override with a p:issuer property attached (or can be implemented via a lookup function in more advanced cases).

Credential Configuration

The <CredentialResolver> element in the legacy SP specifies signing and encryption keys and certificates (the latter being of more importance most of the time). This is entirely a Hub consideration now as it brokers all such operations for its Agent(s).

For a co-located Hub serving an Agent, this is a fairly simple exercise of copying the old keypair files over into the Hub, which is mentioned in the introductory documentation. This can circumvent the need to update the Agent’s corresponding SP metadata if maintaining complete external compatibility is desired.

Where this becomes much more of an issue is when a Hub is deployed to serve many discrete Agents. While it is physically possible to import by hand a multitude of individual keypairs and define them directly in XML so they can then be “attached” to particular Agent definitions in the Hub, work is still underway in developing effective mechanisms for this. However, it is not self-evidently a great idea to do all that work. It’s perfectly allowed in SAML or OIDC for many logically discrete SP/RPs to operate with a single set of keys, though it seems to be less “accepted” for RPs to share client secrets.

For one thing, there is no risk encapsulation; a compromise of any of them will likely compromise all of them. And for another, we think the majority of heavily scaled up Hub deployments are likely to be internal to enterprises and involve a relatively small audience of IdPs (often one). In such cases, control over SAML Metadata is likely to be somewhat or totally closely held by IdP operators and it should be more attractive to simply plan for a migration of those SP deployments to new keys (and perhaps new entityIDs). A side benefit is the opportunity to consolidate all SAML and OIDC response endpoints to a single Agent location rather than the legacy SP’s “endpoint per SAML binding” requirement.

Other Configuration Sections

The rest of the configuration not mentioned above is either only relevant to the Agent’s configuration or is often just redundant or superfluous now.

The following elements will tend to be necessary at least in part to construct the proper Agent configuration:

• <ISAPI>

• <RequestMap>

• <Sessions>

• <Handler>(the generic element tends to refer to Agent-only functionality that isn’t part of the Hub)

• <Errors>

The following elements can generally be ignored entirely (along with any additional XML files referenced if applicable):

• <InProcess> or <OutOfProcess>

• <Extensions>

• <TrustEngine>

• <SecurityPolicyProvider>

• <ProtocolProvider>