This page is out of date and replaced by Plugin Installation


As discussions have continued the shape of the proposed solution has changed.  Rather that rewrite the document I have added sections.  It doesn't make for great readability, but it does make for easier re-review.

During the life of IdP V4 we want to put the framework in place to allow us move away from a monolithic approach where everything anybody might want is available to a slightly more modular approach. where deployers can select what services they want. 

Examples might be

This documents explores the area.



Two key things have resulted from discussion so far

1) The plugin name is critical

This name must be

The suggestion is to use the reverse DNS notation that the author controls hence net.shibboleth.plugins.scripting.rhino  could be used by the Shibboleth project because it owns the *  domain tree.

2) Our thoughts on trust and signing have developed.

See below

After-installation end-state

For a product called 

Driving the Installation

The install package is shipped as .tgz (Linux line ending) or .zip (Windows).  The distributiin must be accompanied with an armoured detached PGP signatures.

The format of an unpacked package is 

Example package layout

DirectoryInstallation notes
bin/libnot allowed (it is refreshed by the installation)
bootstrapNOT COPIED
Contains the bootstrap information for the plugin installer
dist not allowed
docrecommended (for licenses)
Note: changes to this file will not be installed and so the name should change with changes to ensure that the customer site has up to date licensing documentation
logsnot disallowed
metadatanot disallowed
systemnot allowed
wildcard copied to dist/webapp-pluginId

The Service Interface

The installation is driven from within the (unzip'd) package itself via the service interface.  So the installer just needs to unpack, add edit-webapp/WEB_INF/lib  (in the package) to the classpath and fire up service API to see what it has to do. 

The Interface will have methods to return (at least) the following 

String getPluginId()Used as per above to uniquify file and property names
List<String> getAdditionalPropertyFiles() 

0 or more property file names to add to 

Not supported in currently

List<Path> getFilePathsToCopy() 0 or more files to add to the distribution
  • Install will fail if files are going where they shouldn’t (see above)
  • The install will NOT  do a wildcard copy
  • Webapp is special cased, and wildcard copied, so it must not be mentioned
List<Pair<URL, Path>> getExternalFilePathsToCopy() 0 or more files and their location.  The <Path> is relative to idp-home .  The user will be prompted to download these (leap of faith).  The alternative for them is to download this by hand.
List<URL> getUpdateURLs() 1 or more locations where a property file can be located.  The properties will include those with a name derived from the plugin Id.
List<Pair<Path, List<String>>> getPropertyMerges() 

A list of property files and paired with a list of properties to merge. So in the example above if would be

[ { “,                            // File
    [ “idp.service.attribute.filter.resources” , ]  // List of properties in that file
  } , 

Not currently supported

int getMajorVersion() 
int getMinorVersion() 
int getPatchVersion() 

Bootstrap region

The plugin ID and the initial keystore are included as text (so no suspect jar need be on the classpath) in a directory with is part of the package (but not installed) called "bootstrap".  This contains two files


Uninstallation cannot occur because of any changes to the configuration will render the system unusable.  As part of an upgrade the webapp parts of the previous distribution will be removed but nothing else.


Doing an update consists of collecting the new version, uninstalling the old version, installing the new one.


This has many different aspects.  The four most important are:

1) Jar versions. 

Jars can now be sourced from four different types of place

  1. The IdP.
  2. This plugin
  3. Other plugins
  4. Edit-webapp

If each supplies a jar which provides the same class, precisely which class will be used will be random.

I do not believe that it is sensible to check for this. so the best we can do is provide mitigation, probably via some orthogonal configuration.  This might (for instance) be a series of regexps which would make up a black list of jars not to copy.  How this is communicated is tbd.

2) IdP to Module versioning 

There is a new IdP version.  Will the modules work with it?

3) Module to IdP versioning

This Module has a new version.  Can it be installed against the IdP?

Which of course doesn't help the "How do I update my IdP" story unless the plugin writers allow some spread of supported versions so allow staggered upgrade.

4) Module to Module versioning

Out of scope however we should make the following recommendations (tbd whether this is a SHOULD or a MUST 

The Update Property file

The file is is located at the URLs pointed to by getUpdateURLs().  Multiple locations allow for redundancy.  The property names are derived from the plugin id , with the specific version appended where this is relevant.  All three digits of the version information should be present and dot-separated.

Property NameProperty Value DescriptionExample
<pluginid>.versions A list of versions in arbitrary order.  All three digits of the version must be present net.shibboleth.plugin.totp=1.0.0 2.0.0 2.1.0 2.1.1 2.1.2 

Space separated list of directories where this version is to be found.

Multiple values allow redundancy. 


The file name.  the following four suffixes will be added during download

  • .tar.gz  (Linux line endings)
  • .zip (Windows line endings)
  • .tar.gz.asc (signature)
  • .zip.asc

The maximum (exclusive) IdP version supported.

Trailing version and ppatch levels of zero are inferred


The minimum (inclusive) IdP version supported.

Trailing version and ppatch levels of zero are inferred.


Values are:

  • Current
  • OutOfDate
  • Unsupported
  • Secadv
  • Withdrawn,

Rules for version comparison

Detecting the need to update

This is all done from within the IdP in an admin flow. 

In a later release we might chose to have an external program - this would build a classpath from idp.home/dist/webapp-*/WEB_INF/lib 


# plugin update <pluginid>

Example of being crash proof

  1. Unpackage to idp.home/ 
  2. Rename (atomic) to idp.home/
  3. Uninstall old (from idp.home/ 
  4. Install from idp.home/
  5. Commit changed by deleting idp.home/

Initial Install

The initial install is made fraught by the whole "bootstrapping trust" thing.  You have an artefact which you don't trust which is the only place you can do to find out it.   For the initial bootstrapping of trust we do not want to run any code which we do not trust.  The trick is therefore to

Blocking IdP upgrades

handwaving ahead.

Given a putative IdP version "it can be seen that" one can test whether the installed plugins are compatible.  This leads us to two options

Behind all this is a feeling that there is a "turtles all the way down" solution by which the IdP also exports most of the Service Interface and this can therefore be used to drive update detection and testing and even "download and update" the IdP.  

IdP Updates Insert

IdP update needs to be modfiied to preserve the dist\edit-webapp-pluginID  folders.


Nothing special.  We should have no interest in deploying special windows installation technology.  Windows user can just do what everyone else does.  Anyone smart enough to want a plugin probably wants to (a) not be running Windows and (b) owning their jetty install.


My ideas here are not well formed yet, but there seem to be at least three different areas

Of course there are two axes of "one off testing" and regression/unit testing,  I think I'd sooner concentrate on the latter and leave the former as ad hoc testing as part of the development process.

Project Process Issues

Again, this is an area where my ideas are not well formed yet.  Some observations

Licensing considerations

These come in two flavors

Plugin licensing

The plugins need to obey relevant licensing restrictions.  The plugin architecture allow the optional install time download of files which can help this

Plugin Installer licensing

Building the plugin will require some new dependencies.  We need to be aware of them

Further documentation