SP4 Prerelease Testing

This page provides a starting point for those willing/interested in testing the new generation of the Shibboleth Service Provider, which will eventually replace the “to-be-sunset” V3 software. It is similar in many ways and radically different in others. Those already using the SP may find themselves wanting to look at alternative solutions, while those who dismissed earlier versions may want to reassess the new version’s capabilities. The end of this page includes a brief list of some of the pros/cons for existing and new adopters to consider.

For the time being, our plan is to tag the Agent source code (the web server modules), and provide the source code and Windows/Linux packages clearly labeled with prerelease version information in the usual ways we provide the existing SP software on our site and the mirrors.

The Hub is implemented as a set of plugins that install into the Shibboleth IdP software and they require V5.2.0+ as a baseline. They do not require or even expect that the system is actually operating as a Shibboleth IdP, it is a software “shell” with services and configuration the SP Hub needs to o perate, alone with the additional components the plugins provide.

We are not initially planning to produce tagged and official prerelease versions of these plugins, as it will complicate our development process and create extra work for the team, while also making it impossible for testers to obtain bug fixes without a lot of extra releases. Since the bulk of the newer code lives in these plugins, they are more likely to contain bugs that will inhibit testing than the Agent is at this point. A table below notes what Hub plugin versions to install for a given Agent release until the software reaches a permanently supported ongoing state closer to the official release.

The IdP’s plugin feature supports the installation of snapshot versions (per Plugin Testing) and we have example commands below demonstrating how to install them.

The documentation for both Agent and Hub are in a prerelease state, with the former farther along than the latter. Of course feedback on the documentation is also encouraged.

The software configuration out of the box relies on the following defaults:

• Agent and Hub co-existing on a single host communicating over 127.0.0.1 on port 8080. Altering this requires some simple adjustments but also a set of credentials for the Agent to use (per ConnectingToHub and SecuringAgentConnections).

• Sessions default to filesystem storage; this may change in a future release depending on feedback. See SessionCacheSettings for additional options.

• Agent logging is routed to syslog (which can be accessed via journalctl) and the Windows Event Log depending on platform. A subset of information is also sent to the Apache log in that scenario but not all.

The Agent installation is via the “normal” process that will be used for the official release. See Installation and Upgrades; this is largely equivalent to installing older versions excepting that “shibd” no longer exists as the Hub is a separate piece of software.

The Agent Windows package can be found alongside the source code (links below), while the RPM packages are available from the usual mirrors in the “shibboleth-sp” package. This is different from, and will not upgrade, the currently supported 3.x version, which is called “shibboleth” alone.

Note that by design the new SP installs to the same “root” location but different subdirectories from older versions so that it can physically co-exist with it. Installing the new version will not remove or upgrade the older version and this is intentional. Officially we intend that the two versions can co-exist to some degree but are not intended to remain in place together for any length of time, merely as an aid to transitioning.

Assuming a “from-scratch” install, you will need to at least gain basic familiarity with installing the IdP since it is a requirement to operate the Hub. V5.2.0 or later is required. Initially you don’t need to worry too much about the IdP’s configuration, as only a subset of it is necessary for basic SP testing. In particular, the prompt for an entityID and scope to use for initial installation are ignored by the SP as those are IdP-only settings.

We recommend, particularly for testing this software, the use of the IdP’s new-ish Jetty plugin, which provides a simple way to bootstrap a Jetty container for use by the IdP that is “compatible” with the assumptions made when testing the Agent and Hub on the same host. You can of course adjust properties, issue a certificate, etc. to enable the Hub to support remote use by Agents.

If using the Jetty plugin, then the sequence of steps you will follow are:

1. Install the IdP, without much regard for any settings you are asked for except the installation location.

2. Install the Jetty plugin, download Jetty, and follow the Jetty plugin’s documentation to get a basic working IdP.

3. Finally, proceed to the Hub plugins as described below.

Because the Hub plugins are not formally released yet, some special steps and options are required to install them, per Plugin Testing. To save you time, we have includes some of the necessary commands here. These commands will depend on exactly which versions of the plugins are being installed, as they are built and uploaded by our CI system each evening.

First, download the snapshot trust keyring.

Then, install each plugin needed/desired for testing, obtaining the URL to the latest tar.gz file from the following directories, using the appropriate version required for the Agent being tested, initially 0.0.1-SNAPSHOT.

Only one of the files in the release directory at any time will have the tar.gz extension as only a single SNAPSHOT of each version is retained.

Given the URL to a plugin snapshot, the command to use for installation is:

Plugin Location Roots:

• Core shibd Plugin (always required)

  • https://build.shibboleth.net/maven/snapshots/net/shibboleth/sp/sp-dist/

• SAML shibd Plugin (required for SAML 2.0 support)

  • https://build.shibboleth.net/maven/snapshots/net/shibboleth/sp/sp-saml-dist/

• OIDC shibd Plugin (required for OIDC support)

  • https://build.shibboleth.net/maven/snapshots/net/shibboleth/sp/sp-oidc-dist/

  • This plugin also relies on an unreleased version of the OIDC Common plugin (the SNAPSHOT version for this will most likely remain fixed):

    • https://build.shibboleth.net/maven/snapshots/net/shibboleth/oidc/oidc-common-dist/3.4.0-SNAPSHOT/

    • While this version MAY successfully work with the officially released OP and RP plugins for the IdP, that is not a currently supported goal of this testing.

You should feel free to update to later nightly versions of a given SNAPSHOT version, and they should cleanly install over top of the existing software. Make sure the IdP isn’t running when you do this.

The table below contains each prerelease version made available for testing, with a link to the Agent source code and Windows installer, and an indication of which SNAPSHOT version of the Hub plugins are compatible with that Agent release.

If we intentionally break compatibility with the Agent, we will update the SNAPSHOT version of each plugin to reflect that and leave the older versions available for continued use until the Agent is subsequently released with the necessary changes.

Members of the Shibboleth Consortium should feel free to leverage Slack or Jira for help as desired while testing, or feel free to use the development mailing list if preferred.

Non-members are welcome to test the software, but please use the open-to-all-subscribers development mailing list for discussion, questions, etc. during this prerelease period.

While bug reporting via Jira is fine, at this stage we’re happy to file bugs arising as the result of discussion on other’s behalf, to avoid an inundation of false bug reports arising from confusion, documentation mistakes, etc.

Informally, this is what’s left undone at this stage of development. The Agent and SAML support are farther along in maturity than the OIDC support. In addition to what’s left to implement, there is a non-final list of changes from V3 that will be formalized and moved into the Agent documentation later.

That said, the “big” ticket items not done yet:

• Agent

  • Any real load testing at all

  • Some form of cookie-backed session implementation

  • Hub-mediated SAML/OIDC Single Logout support (logout is just “local” to the Agent right now)

  • Administrative logout / session revocation

  • Agent hosting of OIDC RP metadata, keysets, etc.

• Hub General

  • Audit logging – will be based on IdP features in this area but not implemented yet for SP operations

• Hub SAML

  • Single logout, per above

  • Any (direct) help with SP metadata production

  • ECP

• Hub OpenID Connect

  • Numerous esoteric features, including use of tokens to refresh claims

  • Single logout, per above

  • Metadata, keyset, and other configuration endpoints

  • Additional options for controling requests to OP without SAML analogs

  • OID Federation Support

As an informal aid to those considering whether to invest time into testing this software, we have prepared a short list of the pros and cons as we see them relative to the current software and to similar solutions.

• Fully compatble with most existing applications using the legacy SP, modulo a small number of very esoteric features.

• Much smaller native code footprint and much simpler build with no “unusual” dependencies not readily available on supported platforms or that lack proper maintainance. This version is designed with long term sustainability in mind.

• Much simpler Agent configuration with no exposure to SAML or OpenID at all for Agent deployers in “standard” SSO use cases.

• Far less XML in the configuration (none required in the case of Apache)

• Better clustering options, particularly as we get closer to release.

• Significant feature evolution possible without updates to Agent code through Hub plugin updates.

• Focus on scaling enterprise deployment while retaining support for multi-lateral federation. The Hub model can potentially be shared by a large number of Agents, in many cases with relatively minimal ongoing configuration. Has many advantages of a proxy with the additional advantage of not being one from an external perspective.

• Seamless support for both SAML and OpenID in one Apache/IIS solution with extensibility to other protocols in the future.

• Simple framework for integration into other web servers application environments by the project, or by third parties if they’re willing. Agents without a requirement for managing sessions can be very lightweight.

• Dependent on Shibboleth IdP as a plugin “Hub” for hosting the Java code that offloads work from the Agent and provides most of the advantages above.

• Hub configuration is in some cases potentially more complex than the old SP’s configuration, though somewhat cleaner. Depends a lot on one’s exposure to the Shibboleth IdP and one’s opinion of it.

• For a single Agent, the Hub is a bit more work than shibd was, not less, and more heavyweight being in Java and requiring a servlet container. The advantages only begin to appear in a shared Hub deployment.

• Some, though not an overwhelming amount of, new Java code to support SAML and OpenID relying party implementation, so code maturity will be less than the old SP for a while.

• A shared Hub introduces network overhead and failure modes to Agent deployment; never forget the distributed computing fallacy. A network can never match the reliability of not needing one.