As discussions have continued the shape of the proposed solution has changed. Rather that rewrite the document I have added sections. It doesn't make for great readability, but it does make for easier re-review. |
During the life of IdP V4 we want to put the framework in place to allow us move away from a monolithic approach where everything anybody might want is available to a slightly more modular approach. where deployers can select what services they want.
Examples might be
This documents explores the area.
Two key things have resulted from discussion so far
This name must be
The suggestion is to use the reverse DNS notation that the author controls hence net.shibboleth.plugins.scripting.rhino
could be used by the Shibboleth project because it owns the *.shibboleth.net
domain tree.
See below
For a product called net.shibboleth.foo
dist/webapp-net.shibboleth.foo
dist/webapp-*
(in a random order). This is this done after the population from dist/webapp
(as now) and before edit-webapp
.post-edit-webapp
folder to our distribution to allow us to force the jars we want.preConfig.xml
and postConfig.xml
to setup global beanscredentials/plugins-trust/net.shibboleth.foo/truststore.asc
which contains the public PGP keys that the deployer has accepted for use with this plugin. See PluginTrustidp.additionalProperties
List properties can be edited. For example
idp.service.attribute.filter.resources = shibboleth.AttributeFilterResources (or) #idp.service.attribute.filter.resources = shibboleth.AttributeFilterResources |
becomes
idp.service.attribute.filter.resources = BEAN.idp.service.attribute.filter.resources.net.shibboleth.foo idp.service.attribute.filter.resources.OLD.net.shibboleth.foo = shibboleth.AttributeFilterResources |
And the installation is required (via preConfig/postConfig) to define BEAN.idp.service.attribute.filter.resources.net.shibboleth.foo
which is the list merge of what it needs plus %{idp.service.attribute.filter.resources.OLD.net.shibboleth.foo}
net.shibboleth.ext.spring.factory.CombiningListFactoryBean
The install package is shipped as .tgz (Linux line ending) or .zip (Windows). The distributiin must be accompanied with an armoured detached PGP signatures.
The format of an unpacked package is
Directory | Installation notes |
---|---|
bin | optional. |
bin/lib | not allowed (it is refreshed by the installation) |
bootstrap | NOT COPIED Contains the bootstrap information for the plugin installer |
conf conf/* | optional. |
dist | not allowed |
credentials | deprecated |
doc | recommended (for licenses) Note: changes to this file will not be installed and so the name should change with changes to ensure that the customer site has up to date licensing documentation |
flows/* | optional |
logs | not disallowed |
messages | optional |
metadata | not disallowed |
system | not allowed |
view | optional |
webapp | Required wildcard copied to dist/webapp-pluginId |
The installation is driven from within the (unzip'd) package itself via the service interface. So the installer just needs to unpack, add edit-webapp/WEB_INF/lib
(in the package) to the classpath and fire up service API to see what it has to do.
The Interface will have methods to return (at least) the following
Method | Description | ||
---|---|---|---|
String getPluginId() | Used as per above to uniquify file and property names | ||
List<String> getAdditionalPropertyFiles() | 0 or more property file names to add to
| ||
List<Path> getFilePathsToCopy() | 0 or more files to add to the distribution
| ||
List<Pair<URL, Path>> getExternalFilePathsToCopy() | 0 or more files and their location. The <Path> is relative to idp-home . The user will be prompted to download these (leap of faith). The alternative for them is to download this by hand. | ||
List<URL> getUpdateURLs() | 1 or more locations where a property file can be located. The properties will include those with a name derived from the plugin Id. | ||
List<Pair<Path, List<String>>> getPropertyMerges() | A list of property files and paired with a list of properties to merge. So in the example above if would be
| ||
int getMajorVersion() | |||
int getMinorVersion() | |||
int getPatchVersion() |
The plugin ID and the initial keystore are included as text (so no suspect jar need be on the classpath) in a directory with is part of the package (but not installed) called "bootstrap". This contains two files
id.pr
operty
a file with a single property in it pluginid
keys.txt
the signing keys. This is optional. See PluginTrustUninstallation cannot occur because of any changes to the configuration will render the system unusable. As part of an upgrade the webapp parts of the previous distribution will be removed but nothing else.
Doing an update consists of collecting the new version, uninstalling the old version, installing the new one.
This has many different aspects. The four most important are:
Jars can now be sourced from four different types of place
If each supplies a jar which provides the same class, precisely which class will be used will be random.
I do not believe that it is sensible to check for this. so the best we can do is provide mitigation, probably via some orthogonal configuration. This might (for instance) be a series of regexps which would make up a black list of jars not to copy. How this is communicated is tbd.
There is a new IdP version. Will the modules work with it?
This Module has a new version. Can it be installed against the IdP?
We know the IdP version (it is written during install and upgrade to dist):
#Version file written at 2020-03-11T09:59:08.023Z #Wed Mar 11 09:59:08 GMT 2020 idp.installed.version=4.0.0 idp.previous.installed.version=3 |
Which of course doesn't help the "How do I update my IdP" story unless the plugin writers allow some spread of supported versions so allow staggered upgrade.
Out of scope however we should make the following recommendations (tbd whether this is a SHOULD or a MUST
The file is is located at the URLs pointed to by getUpdateURLs()
. Multiple locations allow for redundancy. The property names are derived from the plugin id , with the specific version appended where this is relevant. All three digits of the version information should be present and dot-separated.
Property Name | Property Value Description | Example |
---|---|---|
<pluginid>.versions | A list of versions in arbitrary order. All three digits of the version must be present | net.shibboleth.plugin.totp=1.0.0 2.0.0 2.1.0 2.1.1 2.1.2 |
<pluginid>.downloadURL.<version> | Space separated list of directories where this version is to be found. Multiple values allow redundancy. | net.shibboleth.idp.plugin.rhino.downloadURL.1.1.0=\ |
<pluginid>.baseName.<version> | The file name. the following four suffixes will be added during download
| net.shibboleth.idp.plugin.rhino.baseName.1.1.0=\ |
<pluginid>.idpVersionMax.<version> | The maximum (exclusive) IdP version supported. Trailing version and ppatch levels of zero are inferred | net.shibboleth.idp.plugin.rhino.idpVersionMax.1.1.0=5 |
<pluginid>.idpVersionMin.<version> | The minimum (inclusive) IdP version supported. Trailing version and ppatch levels of zero are inferred. | net.shibboleth.idp.plugin.rhino.idpVersionMin.1.1.0=4.1.1 |
<pluginid>.supportLevel.<version> | Values are:
| net.shibboleth.idp.plugin.rhino.supportLevel.1.1.0=Current |
This is all done from within the IdP in an admin flow.
version.matrix.properties
file.In a later release we might chose to have an external program - this would build a classpath from idp.home/dist/webapp-*
/WEB_INF/lib
# plugin update <pluginid> |
URL+/version/package.tgz
and URL+/package.jar.tgz.sig
(or zip equivalents)idp.home/unpackage-net.shibboleth.foo
idp.home/unpackage-net.shibboleth.foo
idp.home/edit-webapp-net.shibboleth.foo
idp.home/unpackage-net.shibboleth.foo
idp.home/unpackage-net.shibboleth.foo
dist-net.shibboleth.foo
The initial install is made fraught by the whole "bootstrapping trust" thing. You have an artefact which you don't trust which is the only place you can do to find out it. For the initial bootstrapping of trust we do not want to run any code which we do not trust. The trick is therefore to
handwaving ahead. |
Given a putative IdP version "it can be seen that" one can test whether the installed plugins are compatible. This leads us to two options
Behind all this is a feeling that there is a "turtles all the way down" solution by which the IdP also exports most of the Service Interface and this can therefore be used to drive update detection and testing and even "download and update" the IdP.
IdP Updates Insert
IdP update needs to be modfiied to preserve the dist\edit-webapp-pluginID
folders.
Nothing special. We should have no interest in deploying special windows installation technology. Windows user can just do what everyone else does. Anyone smart enough to want a plugin probably wants to (a) not be running Windows and (b) owning their jetty install.
My ideas here are not well formed yet, but there seem to be at least three different areas
Of course there are two axes of "one off testing" and regression/unit testing, I think I'd sooner concentrate on the latter and leave the former as ad hoc testing as part of the development process.
Again, this is an area where my ideas are not well formed yet. Some observations
These come in two flavors
The plugins need to obey relevant licensing restrictions. The plugin architecture allow the optional install time download of files which can help this
Building the plugin will require some new dependencies. We need to be aware of them