Version 0.10.0 (next feature release)
Release date: TBA
For a complete list of issues addressed in this release, see https://issues.shibboleth.net/jira/issues/?filter=12070
This is a major pre-1.0 feature release.
Dependency Changes
MDA-217: This release is built with Java 11, and requires a Java 11 or later execution environment. (Previously MDA-189 targeted the release to Java 8)
JPAR-125: Updated from V4.3 to V5.x of Spring Framework.
Packaging Changes
In version 0.10.0, the packaging of the Metadata Aggregator has been completely revised:
MDA-266: In prior versions, a single distribution package in
.zip
format was provided bundling a simple command-line interface, the MDA framework and all required dependencies, with a name likeaggregator-cli-0.9.2-bin.zip
. In version 0.10.0, two distribution packages are available:mda-distribution-0.10.0.tar.gz
, intended for non-Windows systems, andmda-distribution-0.10.0.zip
, intended for Windows systems.
Each package includes, as before, the MDA framework, the simple command-line interface and all required dependencies. Additional text files such as license material and examples follow Windows line endings for the .zip
distribution and Unix line endings for the .tar.gz
distribution. Both distributions include invocation scripts for both Linux (mda.sh
) and Windows (mda.bat
) systems following the natural line-ending conventions for their intended platforms.
By preference, you should use the .zip
distribution only on Windows systems. If you are migrating from 0.9.x on a non-Windows system, you should switch to the .tar.gz
distribution instead. However, the .zip
distribution still includes the Linux invocation script with Unix-style line endings so it will still work. This behaviour is not guaranteed for future releases.
MDA-266: Maven artifact IDs for the JAR artifacts have changed:
aggregator-pipeline
is nowmda-framework
aggregator-cli
is nowmda-cli
The group ID remains net.shibboleth.metadata
.
MDA-158: The packaging of the RSA key list resources introduced in version 0.9.0 has been changed. Previously included in the
aggregator-pipeline
artifact, these resources have now been moved into a separatemda-keylists-rsa
artifact. The resource names have not changed. This means that if your application does not use these resources, it may decrease in size by around 13MB. Applications making use of the blacklist resources may need to add a dependency onmda-keylists-rsa
.MDA-181: A Maven BOM (Bill Of Materials) artifact has been made available. This makes it easier for projects using the Shibboleth MDA as a dependency to acquire a consistent set of managed dependencies without using the Shibboleth parent POM. You can include the MDA BOM in your Maven project like this:
<dependencyManagement> <dependencies> <dependency> <groupId>net.shibboleth.metadata</groupId> <artifactId>mda-bom</artifactId> <version>0.10.0</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
JPAR-113: The packaged
mda-
module JAR files, as well as a number of underlying library JARs have been given automatic module names as described in Java Modularity, for future use on the Java module path under Java 9 and later releases. Note that this version of the Shibboleth Metadata Aggregator has not been tested on, and is therefore not guaranteed to work on, the Java module path.
API Removals
One aim of this release is to prepare for a stable 1.x release series. As part of this, a number of elements have been removed from the API:
MDA-192: The
ancestorEntity
method has been removed fromAbstractDOMTraversalStage
; a protectederrorPrefix
method has replaced it in order to allow sub-classes to replicate this or similar behaviour. A newAbstractSAMLTraversalStage
class has been added to incorporate the specific old behaviour.MDA-223: A number of constant fields have been removed from the
XMLSignatureSigningStage
andXMLDSIGSupport
classes and therefore the API, as they are now part of the base Java API. For example,XMLSignatureSigningStage.ALGO_ID_SIGNATURE_RSA_SHA256
is replaced by Java'sSignatureMethod.RSA_SHA256
.MDA-249: The decorated collection class
ItemCollectionWithMetadata
has been removed. There is no replacement for this functionality, but it was only sporadically supported in the framework and has no real-world use as far as we are aware.MDA-250: The
ItemMetadataSupport
class has been removed. TheaddAll
methods provided by this class more simply implemented in calling classes by direct operation on an item'sgetItemMetadata()
result. Note, in particular, thatClassToInstanceMultiMap
provides aputAll
variant which can be passed anotherClassToInstanceMultiMap
of the same type. This leads to statements such asnew.getItemMetadata().putAll(old.getItemMetadata());
as the optimal way to copy one item's item metadata collection into another item.JSE-28: This release bundles a new version of the Shibboleth
spring-extensions
project, which removes support for SVN-based resources.
API Additions
MDA-52: A new stage
ElementsStrippingStage
has been added to allow stripping a number of different elements (all from the same namespace) from a DOM document. The stage may be operated in a blacklisting or whitelisting mode, with blacklisting the default. LikeElementStrippingStage
, anelementNamespace
property determines the namespace in question, and all elements in other namespaces are ignored.MDA-56: A new stage
EntityAttributeAddingStage
has been added to add entity attributes to the metadata for SAML entities. This is configured usingattributeName
,attributeNameFormat
andattributeValue
properties, withattributeName
andattributeNameFormat
defaulting to the values required to add an entity category attribute. The stage is based on a newContainer
framework which attempts to generate reasonably well formatted XML for nested container elements, and handles the insertion of the required parent containers (Extensions
,EntityAttributes
,Attribute
) when they are not already present.MDA-116: The
ScriptletStage
has a newvariableName
property controlling the name of the variable in the script context to which the item list is assigned. This defaults to"items"
as before but may be overridden for languages or scripts that require a different value. For example, settingvariableName
to"$items"
allows use of the stage with Ruby.MDA-138: A new
SAMLStringElementCheckingStage
can be used to check the general rule in SAML that elements with string values must have at least one non-whitespace character.MDA-160: The
EntityAttributesFilteringStage
has been extended with a newrecordingRemovals
property, defaulting to false. IfrecordingRemovals
is set totrue
, each removed entity attribute is recorded as aWarningStatus
in the item's item metadata, indicating the name and value of the entity attribute removed. This can then be processed by subsequent stages, such as aStatusMetadataLoggingStage
.MDA-173: A new
DiscoFeedCollectionSerializer
can be used in conjunction with the existingSerializationStage
to generate a JSON discovery feed compatible with the Shibboleth Embedded Discovery Service (EDS), as an alternative to the Shibboleth SP's ability to generate such a feed at its/DiscoFeed
endpoint. A configuration example is available. The following properties may be set; all arefalse
by default:prettyPrinting
creates more human-readable output using white space, this is the Shibboleth SP's default but is disabled by default for theDiscoFeedCollectionSerializer
.includingLegacyDisplayNames
is equivalent to the SP'slegacyOrgNames
attribute.includingEntityAttributes
is equivalent to the SP'stagsInFeed
attribute.
MDA-177: An entity attribute matcher
AssuranceCertificationMatcher
has been added to allow simpler matching of entity attributes containing assurance certifications, such as that used by the SIRTFI framework.MDA-178: A bean definition resource has been added to simplify access to each bean class in the aggregator-pipeline artifact. In XML configuration, this can be accessed by
<import resource="classpath:net/shibboleth/metadata/beans.xml"/>
. One abstract bean is defined for each available bean class, named after the class's simple name prefixed by "mda.
". After including this resource, for example,class="net.shibboleth.metadata.dom.XMLSignatureValidationStage"
can be replaced byparent="mda.XMLSignatureValidationStage"
; this definition will also include theinit-method
anddestroy-method
properties for the bean when appropriate.MDA-179: The simple command-line interface now includes a
--version
option to request the printing of the framework version number.MDA-184: A new utility class
RegexFileFilter
has been added to support one of the common use cases of theDOMFilesystemSourceStage
, where only certain files should be processed from a directory, based on their names.MDA-187: A new
IPHintValidationStage
has been added to allow validation of<mdui:IPHint>
elements in SAML metadata. ThecheckingNetworks
property (defaulttrue
) requires that the value represents a network and not a host, thus faulting values such as "127.0.0.1/24".MDA-193: To make using the
Validator
framework more straightforward, the newValidatorSequence
class abstracts the concept of a sequence ofValidators
which can be maintained and applied as a group. Existing classes requiring this behaviour have been refactored to take advantage ofValidatorSequence
.MDA-199: A new
X509ROCAValidator
component allows RSA public keys in X.509 certificates to be checked for vulnerability to ROCA (the Return of Coppersmith's Attack, also known as CVE-2017-15361).MDA-200: The
BaseValidator
abstract class has been extended to add anaddErrorMessage
method and amessage
property, which acts as a format string forErrorStatus
item metadata generated throughaddErrorMessage
.MDA-201: New
AcceptAllValidator
andRejectAllValidator
components have been added. Both always returnAction.DONE
so that they can be used to terminate a sequence of validators.AcceptAllValidator
has no other functionality;RejectAllValidator
uses itsmessage
property to format an appropriateErrorStatus
for theItem
on which validation is being performed.MDA-202: Four new validator components (
AcceptStringValueValidator
,RejectStringValueValidator
,AcceptStringRegexValidator
andRejectStringRegexValidator)
have been added to matchString
values. All four returnAction.DONE
if the match occurs and will therefore terminate a sequence of validators;Action.CONTINUE
is returned otherwise. TheReject
forms also add a formattedErrorStatus
on matching.MDA-214: A new
X509DSADetector
component allows DSA keys in metadata to be rejected, or merely warned about.MDA-229: A new
StringElementValidationStage
component allows the validation of the string contents of designated DOM elements. UsesetElementNames
(set theelementNames
property) to specify collection of element names to validate. A second method (setElementName
, or set theelementName
property) provides a shortcut for the common single-element case.MDA-234: A new
ItemOrderingStage
allows the collection of entities to be re-ordered according to a suppliedItemOrderingStrategy<T>
(the default strategy does not change the ordering).JSPT-73: This release bundles a new version of the Shibboleth
java-support
package, which implements a newFixedStringIdentifierGenerationStrategy
for use when it is not necessary to use differentID
attribute values for different documents.
API Changes
MDA-166: The
ItemSerializer
andItemCollectionSerializer
interfaces now allow serializers to throwIOException
when appropriate. The providedDOMItemSerializer
will throw anIOException
wrapping aTransformerException
if the latter is thrown during XML serialization. Previously, this condition would only have resulted in logging at ERROR level.MDA-167: The
ItemIdTransformStage
now transforms identifiers using a collection ofFunction
objects rather than of the similarConverter
provided by the Spring framework. This also affects the type of theMDQueryMD5ItemIdTransformer
andMDQuerySHA1ItemIdTransformer
classes. This change will not affect existing configurations if only those classes are in use. This matches the use ofFunction
elsewhere in the API.MDA-169: The
SAMLMetadataSupport.getDescriptorExtensions
method has been renamed togetDescriptorExtension
to reflect the fact that it returns a single result.MDA-170: The
PKCS11PrivateKeyFactoryBean
temporarily introduced in v0.9.2 has, as indicated for that release, moved to thespring-extensions
module; configurations will need to be adjusted accordingly:
Old class name:net.shibboleth.metadata.util.PKCS11PrivateKeyFactoryBean
New class name:net.shibboleth.ext.spring.factory.PKCS11PrivateKeyFactoryBean
MDA-171: The
SAMLMetadataSupport.getDescriptorExtension
method's parameters must now be non-null
; their annotations have been changed to@Nonnull
to correspond with this. In previous releases, they were annotated as@Nullable
and passingnull
would result in the method returningnull
.MDA-175: The
ItemOrderingStrategy
interface defined by theEntitiesDescriptorAssemblerStage
now allows the ordering strategy to throw aStageProcessingException
if, for example, the items presented are invalid in some way and can not be ordered. Such an exception will be propagated upwards to the caller of the stage'sexecute
method.MDA-179: The
Version
class'sgetMicroVersion
method has been renamed togetPatchVersion
to align with current (semantic versioning) terminology.MDA-182: Several classes exposed as part of the API for building custom stages have been reworked to simplify implementation of other stages and to correspond to current naming conventions:
BaseStage
has been renamed toAbstractStage
BaseIteratingStage
has been renamed toAbstractFilteringStage
A new
AbstractIteratingStage
allows the simpler construction of stages which process each Item independently
MDA-188: The
AbstractDOMTraversalStage
framework has been generalised to allow the use of custom context objects specific to the particular traversal, rather than relying on sometimes tortured uses of theClassToInstanceMultiMap
to carry everything. This is a breaking change, but will only affect writers of stages derived fromAbstractDOMTraversalStage
:Context objects must implement the
DOMTraversalContext
interface. This no longer includes thegetStash
method (returning aClassToInstanceMultiMap
but does add a newend()
method to be called at the end of the traversal.A basic implementation of
SimpleDOMTraversalContext
is provided without any data fields. This can be used in many cases where custom storage is not required in the context; for an example, seeAbstractElementVisitingStage
.More complex cases can extend
SimpleDOMTraversalContext
to include additional fields and method. For a very straightforward example, seeCRDetectionStage
. A more complex example, including use of theend()
method fromDOMTraversalContext
, can be found inElementsStrippingStage
.
MDA-198: In previous releases, the three X.509 validation component (
X509RSAExponentValidator
,X509RSAKeyLengthValidator
andX509RSAOpenSSLBlacklistValidator
) all set a default ID related to their names (e.g.,RSAKeyLength
). This default ID setting behaviour has been removed. This may have two effects on configurations which do not explicitly set the component ID:If a configuration did not set the component ID, initializing the component will now fail with a
ComponentInitializationException
.Configurations that implicitly set the component ID to a defaulted Spring component ID using
IdentifiableBeanPostProcessor
may give different results, as the Spring component ID may now appear in status objects replacing the previous default.
MDA-191: The stages
PullUpCacheDurationStage
,PullUpValidUntilStage
,SetCacheDurationStage
,SetValidUntilStage
andValidateValidUntilStage
now use theInstant
andDuration
classes (introduced in Java 8) in their APIs rather than usinglong
values representing milliseconds as in previous releases.This aligns the metadata aggregator with other Shibboleth projects based on the Java 11 platform. If you use the Java language to configure these stages, you will need to re-code appropriately; in most cases, this can be done quickly using
Instant.fromEpochMilli()
andDuration.ofMillis()
, but we recommend adopting the modernjava.time
classes throughout.The
DurationToLongConverter
is no longer included as part of thejava-support
dependency. If you were using it as part of an XML configuration to specify durations in ISO 8601 format (e.g., "PT15M") then you should replace references toDurationToLongConverter
with references to the newStringToDurationConverter
.
MDA-206: The
PipelineDemultiplexerStage
'swaitingForPipelines
property previously defaulted tofalse
, which could result in unexpected behaviour if the stage was invoked a second time without arranging to synchronise execution with the called pipelines. As a result, most deployments setwaitingForPipelines
totrue
so that the called pipelines will all complete before control is passed on from thePipelineDemultiplexerStage
; this behaviour is now the default.MDA-219: In previous releases, the
DOMResourceSourceStage
tested for the existence of the suppliedResource
as part of component initialization, potentially resulting in aComponentInitializationException
duringinitialize()
. Unfortunately, this could also cause large remote resources to be fetched unnecessarily, and in addition did not guarantee that theResource
would be accessible during normal operation. This check has been removed: now, aStageProcessingException
will be thrown during stage execution instead, in the same way as was already the case for other I/O errors related to theResource
.MDA-222: The contract for bean properties representing collections has changed:
Property setters for collection properties are now annotated as
@Nonnull
@NonnullElements
@Unmodifiable
.Previously, some setters allowed a null value to act in place of an empty collection. This usage will now result in most cases in a
NullPointerException
.Previously, some setters filtered null values out of provided collections. Again, this usage will now result in most cases in a
NullPointerException
.Setters now guarantee not to modify the passed collection. This was previously true in practice in most cases, but is now guaranteed.
Most property getters for collection properties are also now annotated as
@Nonnull
@NonnullElements
@Unmodifiable
.In exceptional cases, getters may be annotated as
@NonnullAfterInit
instead of@Nonnull
. This is only done when an "empty collection" default is inappropriate for the property and would normally be accompanied by@NotEmpty
on both the setter and the getter.
MDA-232: Previously, an invalid XPath expression provided to an
XPathFilteringStage
went largely unremarked: if the expression failed to compile, no filtering would be performed. If the expression could not be evaluated (for example, because it referenced a$variable
) errors would be logged, and all items would be filtered out. Now:An XPath expression which fails to compile will result in a
ComponentInitializationException
when the stage is initialized.An XPath expression which fails to evaluate will result in a
StageProcessingException
when the stage is executed.
MDA-234: The
EntitiesDescriptorAssemblerStage
previously allowed entities to be reordered by setting a property to an instance of anItemOrderingStrategy
sub-interface that was specific toItem<Element>
. This interface has been extracted to thepipeline
package and expressed as a generic, allowing it to be reused for the newItemOrderingStage
. If you are using theorderingStrategy
property, your implementation ofItemOrderingStrategy
will need to be retargeted at the new definition.MDA-238: All uses of the Guava functional interfaces (
Function
,Predicate
andSupplier
) have been replaced with their Java 8 equivalents.This aligns the metadata aggregator with other Shibboleth projects based on the Java 11 platform.
This change should be largely transparent, but if you use Java configuration you should note that
Predicate
's method name has changed fromapply
totest
.
MDA-239: The collection of items passed to an
ItemCollectionSerializer
is now explicitly annotated as@NonnullElements
. This constraint was previously assumed by implementations.MDA-240: The
QName
constants used as part of the API ofEntityRoleFilterStage
have been moved to theSAMLMetadataSupport
class. This will affect configurations which use those fields directly, but not configurations which construct their own equivalentQName
s.MDA-242: The entire framework has been reviewed for thread safety. The major highlights of this are:
Implementations of
Item
are no longer expected to be@ThreadSafe
and in fact are expected to be@NotThreadSafe
. In particular, theDOMElementItem
implementation has always had mutable but unsynchronized contents both in the DOM data it wraps and in theClassIndexedMultiMap
that is used to store the item'sItemMetadata
(although theItemMetadata
objects themselves are immutable and therefore thread-safe). Find more about this topic and its implications in the Javadoc for Item.All classes that are part of configurations (pipelines, stages, validators, serializers, strategy functors etc.) are now
@ThreadSafe
or@Immutable
.These revisions have resulted in a number of minor changes to the API, such as the removal of some getters provided for non-property values and the movement of some non-API classes to "impl" sub-packages.
MDA-247: The
ComponentInfo
class is now@Immutable
as required by allItemMetadata
subclasses. Callers need to migrate to a new constructor, as the existing ones relied on the presence of setter methods; all setter methods have been removed.MDA-248: The
Item
andItemMetadata
interfaces no longer extendSerializable
. This dated back to early design work related to state management in the Shibboleth V3 identity provider. The IdP later took a different direction, but the unnecessary constraint remained in the MDA until now. The effect of this removal is mainly to simplify implementations, and remove the need for extension classes to include aserialVersionUID
variable. The inability to use Java serialization on these classes is not expected to be noticed in practice.MDA-253: The signature of the
Stage
interface'sexecute
method has changed fromCollection<Item<T>>
toList<Item<T>>
to signify that the ordering of the items is significant, as it is for almost all use cases. This has always been the case in practice as the underlying implementation provided bySimpleItemCollectionFactory
is anArrayList
. This should not significantly affect existing deployments, but will affect downstream packages whereStage
s are defined.MDA-254:
ItemIdentificationStrategy
is now a generic interface, parameterised by the type ofItem
to be identified. This makes it possible to write type-safe implementations of the interface specialised for specific types ofItem
such asItemIdentificationStrategy<Element>
to processItem<Element>
.MDA-257: The contract implemented between
AbstractItemMetadataSelectionStage
and its subclasses (ItemMetadataFilterStage
,ItemMetadataTerminationStage
,StatusMetadataLoggingStage
) has changed to add a type bound for the selection criteria, and to pass aClassToInstanceMultiMap
of that bound back to the subclasses. If you have a custom subclass ofAbstractItemMetadataSelectionStage
you will need to track this change. It should not affect existing deployments unlessStatusMetadataLoggingStage
has wrongly been passed selection criteria which were not subclasses ofStatusMetadata
.
Improvements
MDA-183: the
compromised-1024.txt
andcompromised-2048.txt
resources have been extended with keys shipped with some releases of the Jetty container.
Bug Fixes
MDA-179: The
Version
class is now functional, rather than throwing aNullPointerException
when used.MDA-196: Setting the
XMLSignatureSigningStage
'sincludeX509SubjectName
property totrue
caused aClassCastException
. It now behaves as intended, resulting in an<X509SubjectName>
element being added to the signature's<KeyInfo>
's<X509Data>
element.MDA-216: signatures generated by the
XMLSignatureSigningStage
under Java 11 are now consistent with signatures generated under earlier versions of Java.MDA-220:
EntityFilterStage
handled the case of whitelisting incorrectly when the collection of entity IDs to whitelist was empty. The stage now correctly removes all items from the collection, rather than removing none of them.MDA-224:
XMLSignatureSigningStage
threw an IndexOutOfBoundsException if theincludeKeyValue
property was set totrue
without either setting thepublicKey
orcertificates
properties; the stage now just omits theKeyValue
from the signature as ifincludeKeyValue
had been set tofalse
.MDA-226: The
idTransformers
property setter onItemIdTransformStage
incorrectly accumulated values from multiple calls instead of acting as a normal collection setter; this has been corrected.MDA-243: The getter method for the
ContactPersonFilterStage
'sdesignatedTypes
property was misspelled. It is nowgetDesignatedTypes
.