Version 0.10.x

Owned by Ian Young
Last updated: Aug 05, 2024
22 min read

Version 0.10.0 (current stable release)

Release date: 2024-05-16

For a complete list of issues addressed in this release, see Metadata Aggregator | Releases | 0.10.0Released.

This is a major pre-1.0 feature release. The intention is to stabilise the API as much as possible in advance of the 1.0.0 release of the product. There are many changes to the API in this release, and you should expect to make changes in your deployment when migrating from previous releases. Where possible, deprecation warnings are logged to allow gradual migration. Some of those deprecations (to be determined) will be resolved by removing the old APIs in v1.0.0; others will be retained past that point.

This release introduces a standard bean definition resource which both simplifies XML configuration and assists in migration from older versions of the API. Deployers are strongly recommended to adopt this approach if using XML configuration. See Standard bean definition resource for full details.

Dependency Changes

  • MDA-274: This release is built with Java 17, and requires a Java 17 or later execution environment. The distribution is now bundled with Spring Framework V6.x, up from V4.x, and with the new refactored Shibboleth shared dependencies replacing java-support and spring-extensions. This aligns the MDA with V5.x of the IdP product.

Packaging Changes

In version 0.10.0, the packaging of the Metadata Aggregator has been completely revised:

  • MDA-266: In prior versions, a single distribution package in .zip format was provided bundling a simple command-line interface, the MDA framework and all required dependencies, with a name like aggregator-cli-0.9.2-bin.zip. In version 0.10.0, two distribution packages are available:

    • mda-distribution-0.10.0.tar.gz, intended for non-Windows systems, and

    • mda-distribution-0.10.0.zip, intended for Windows systems.

Each package includes, as before, the MDA framework, the simple command-line interface and all required dependencies. Additional text files such as license material and examples follow Windows line endings for the .zip distribution and Unix line endings for the .tar.gz distribution. Both distributions include invocation scripts for both Linux (mda.sh) and Windows (mda.bat) systems following the natural line-ending conventions for their intended platforms.

By preference, you should use the .zip distribution only on Windows systems. If you are migrating from 0.9.x on a non-Windows system, you should switch to the .tar.gz distribution instead. However, the .zip distribution still includes the Linux invocation script with Unix-style line endings so it will still work. This behaviour is not guaranteed for future releases.

  • MDA-266: Maven artifact IDs for the JAR artifacts have changed:

    • aggregator-pipeline is now mda-framework

    • aggregator-cli is now mda-cli

The group ID remains net.shibboleth.metadata.

  • MDA-158: The packaging of the RSA key list resources introduced in version 0.9.0 has been changed. Previously included in the aggregator-pipeline artifact, these resources have now been moved into two separate artifacts: mda-keylists-rsa for most use cases and mda-keylists-rsa-legacy for legacy applications in which shorter RSA keys (with a modulus of less than 2048 bits) are in use. The resource names have also changed to prevent split package issues in the Java module system. This means that if your application does not use these resources, it may decrease in size by around 13MB. Applications making use of the key list resources may need to add a dependency on mda-keylists-rsa or, in rare cases, mda-keylists-rsa-legacy. The resources provided are:

    • In mda-keylists-rsa:

      • net/shibboleth/metadata/keylists/rsa/compromised-2048.txt

      • net/shibboleth/metadata/keylists/rsa/compromised-4096.txt (new in 0.10.0)

      • net/shibboleth/metadata/keylists/rsa/debian-2048.txt

      • net/shibboleth/metadata/keylists/rsa/debian-4096.txt

    • In mda-keylists-rsa-legacy:

      • net/shibboleth/metadata/keylists/rsa/legacy/compromised-1024.txt

      • net/shibboleth/metadata/keylists/rsa/legacy/debian-1024.txt

      • net/shibboleth/metadata/keylists/rsa/legacy/debian-512.txt

  • MDA-290: The packaged mda-module JAR files are now sealed. This means that classes within these artifacts may not be overridden by other resources on the classpath.

  • JPAR-113: The packaged mda-module JAR files, as well as a number of underlying library JARs have been given automatic module names as described in Java Modularity, for future potential use on the Java module path. Note that this version of the Shibboleth Metadata Aggregator has not been tested on, and is therefore not guaranteed to work on, the Java module path.

API Removals

One aim of this release is to prepare for a stable 1.x release series. As part of this, a number of elements have been removed from the API:

  • MDA-192: The ancestorEntity method has been removed from AbstractDOMTraversalStage; a protected errorPrefix method has replaced it in order to allow sub-classes to replicate this or similar behaviour. A new AbstractSAMLTraversalStage class has been added to incorporate the specific old behaviour.

  • MDA-203: A number of classes which were not originally intended to be part of the API have been moved to implementation packages whose names include “.impl". You should not access these classes, as they may be changed, renamed or removed in any future version of the product. The classes include:

    • FutureSupport

    • PipelineCallable

    • XMLSignatureValidator

  • MDA-223: A number of constant fields have been removed from the XMLSignatureSigningStage and XMLDSIGSupport classes and therefore the API, as they are now part of the base Java API. For example, XMLSignatureSigningStage.ALGO_ID_SIGNATURE_RSA_SHA256 is replaced by Java's SignatureMethod.RSA_SHA256.

  • MDA-249: The decorated collection class ItemCollectionWithMetadata has been removed. There is no replacement for this functionality, but it was only sporadically supported in the framework and has no real-world use as far as we are aware.

  • MDA-250: The ItemMetadataSupport class has been removed. The addAll methods provided by this class are more simply implemented in calling classes by direct operation on an item's getItemMetadata() result. Note, in particular, that ClassToInstanceMultiMap provides a putAll variant which can be passed another ClassToInstanceMultiMap of the same type. This leads to statements such as new.getItemMetadata().putAll(old.getItemMetadata()); as the optimal way to copy one item's item metadata collection into another item.

  • MDA-259: The FutureSupport.futureNow method has been removed. Use the JDK’s CompletableFuture class instead.

  • MDA-281: The ComponentInfo class has been removed, along with the supporting logic. This was intended to be used for performance monitoring, but presented performance issues of its own. The progress logging facility added in MDA-282 (see below) addresses many of these use cases in a much more convenient way.

  • MDA-303: The SAMLMetadataSupport.addDescriptorExtension method has been removed. The Container framework can be used as a replacement.

  • JSE-28: This release bundles a new version of the Shibboleth support library for Spring, which removes support for SVN-based resources.

API Additions

  • MDA-52: A new stage ElementsStrippingStage has been added to allow stripping a number of different elements (all from the same namespace) from a DOM document. The stage may be operated in a blacklisting or whitelisting mode, with blacklisting the default. Like ElementStrippingStage, an elementNamespace property determines the namespace in question, and all elements in other namespaces are ignored.

  • MDA-56: A new stage EntityAttributeAddingStage has been added to add entity attributes to the metadata for SAML entities. This is configured using attributeName, attributeNameFormat and attributeValue properties, with attributeName and attributeNameFormat defaulting to the values required to add an entity category attribute. The stage is based on a new Container framework which attempts to generate reasonably well formatted XML for nested container elements, and handles the insertion of the required parent containers (Extensions, EntityAttributes, Attribute) when they are not already present.

  • MDA-116: The ScriptletStage has a new variableName property controlling the name of the variable in the script context to which the item list is assigned. This defaults to "items" as before but may be overridden for languages or scripts that require a different value. For example, setting variableName to "$items" allows use of the stage with Ruby.

  • MDA-138: A new SAMLStringElementCheckingStage can be used to check the general rule in SAML that elements with string values must have at least one non-whitespace character.

  • MDA-160: The EntityAttributesFilteringStage has been extended with a new recordingRemovals property, defaulting to false. If recordingRemovals is set to true, each removed entity attribute is recorded as a WarningStatus in the item's item metadata, indicating the name and value of the entity attribute removed. This can then be processed by subsequent stages, such as a StatusMetadataLoggingStage.

  • MDA-173: A new DiscoFeedCollectionSerializer can be used in conjunction with the existing SerializationStage to generate a JSON discovery feed compatible with the Shibboleth Embedded Discovery Service (EDS), as an alternative to the Shibboleth SP's ability to generate such a feed at its /DiscoFeed endpoint. A configuration example is available. The following properties may be set; all are false by default:

    • prettyPrinting creates more human-readable output using white space, this is the Shibboleth SP's default but is disabled by default for the DiscoFeedCollectionSerializer.

    • includingLegacyDisplayNames is equivalent to the SP's legacyOrgNames attribute.

    • includingEntityAttributes is equivalent to the SP's tagsInFeed attribute.

  • MDA-177: An entity attribute matcher AssuranceCertificationMatcher has been added to allow simpler matching of entity attributes containing assurance certifications, such as that used by the SIRTFI framework.

  • MDA-178, MDA-294: A Standard bean definition resource has been added to simplify access to each bean class in the mda-framework artifact. In XML configuration, this can be accessed by <import resource="classpath:net/shibboleth/metadata/beans.xml"/>. One abstract bean is defined for each available bean class, named after the class's simple name prefixed by "mda.". After including this resource, for example, class="net.shibboleth.metadata.dom.XMLSignatureValidationStage" can be replaced by parent="mda.XMLSignatureValidationStage"; this definition will also include the init-method and destroy-method properties for the bean when appropriate. This resource also implements MDA-277 to provide migration assistance from old releases of the MDA.

  • MDA-179: The simple command-line interface now includes a --version option to request the printing of the framework version number.

  • MDA-184: A new utility class RegexFileFilter has been added to support one of the common use cases of the DOMFilesystemSourceStage, where only certain files should be processed from a directory, based on their names.

  • MDA-187: A new IPHintValidationStage  has been added to allow validation of <mdui:IPHint> elements in SAML metadata. The checkingNetworks property (default true ) requires that the value represents a network and not a host, thus faulting values such as "127.0.0.1/24".

  • MDA-193: To make using the Validator framework more straightforward, the new ValidatorSequence class abstracts the concept of  a sequence of Validators which can be maintained and applied as a group. Existing classes requiring this behaviour have been refactored to take advantage of ValidatorSequence.

  • MDA-199: A new X509ROCAValidator component allows RSA public keys in X.509 certificates to be checked for vulnerability to ROCA (the Return of Coppersmith's Attack, also known as CVE-2017-15361).

  • MDA-200: The BaseValidator abstract class has been extended to add an addErrorMessage method and a message property, which acts as a format string for ErrorStatus item metadata generated through addErrorMessage.

  • MDA-201: New AcceptAllValidator and RejectAllValidator components have been added. Both always return Action.DONE so that they can be used to terminate a sequence of validators. AcceptAllValidator has no other functionality; RejectAllValidator uses its message property to format an appropriate ErrorStatus for the Item on which validation is being performed.

  • MDA-202: Four new validator components (AcceptStringValueValidator, RejectStringValueValidator, AcceptStringRegexValidator and RejectStringRegexValidator) have been added to match String values. All four return Action.DONE if the match occurs and will therefore terminate a sequence of validators; Action.CONTINUE is returned otherwise. The Reject forms also add a formatted ErrorStatus on matching.

  • MDA-214: A new X509DSADetector component allows DSA keys in metadata to be rejected, or merely warned about.

  • MDA-229: A new StringElementValidationStage component allows the validation of the string contents of designated DOM elements. Use setElementNames (set the elementNames property) to specify collection of element names to validate. A second method (setElementName, or set the elementName property) provides a shortcut for the common single-element case.

  • MDA-231: A new StringAttributeValidationStage component allows the validation of the string contents of designated DOM attributes. Its API mirrors that of StringElementValidationStage with the addition of a setAttributeNames method (an attributeNames property) to specify a collection of attributes to validate on the selected elements. setAttributeName (an attributeName property) provides a shortcut for the common single-attribute case.

  • MDA-233: The Validator interface has been extended with a second validate method with an additional String valueContext parameter. This provides a way for a caller to provide context about the validation so that error messages can be more helpful: for example, a failed validation of a particular component in a URL can refer to the entire URL as well as the rejected component value. Both validate methods are provided with default bodies, so that existing Validator implementations work without change: new implementations must implement at least one of the methods, but this can be either one. Within the aggregator framework, this functionality has been retrofitted to many existing classes so that the valueContext is propagated through a validator hierarchy, and the new Validator<URL> classes make use of the value in reporting (see MDA-299 below).

  • MDA-234: A new ItemOrderingStage allows the collection of entities to be re-ordered according to a supplied ItemOrderingStrategy<T> (the default strategy does not change the ordering).

  • MDA-282: CompositeStage and SimplePipeline now support a boolean loggingProgress property which, if enabled, causes instances to log progress through the configured stages at the INFO level. Execution of each stage is surrounded by an indication of the number of items being processed as well as the elapsed time for processing, and a final composite elapsed time is logged at the end.
    This feature is enabled for all instances of CompositeStage and SimplePipeline regardless of their loggingProgress property if the system property net.shibboleth.metadata.loggingAllProgress is set to true. This is intended to allow an initial overall view of processing time before digging in to specific instances of CompositeStage later.

  • MDA-287: A new ScopeValidationStage component has been added to allow the checking of <shibmd:Scope> elements in SAML metadata. The stage accepts two lists of Validator<String>, one for scopes where regex="false" or absent, and another for scopes where regex="true". Additional validator components AsLiteralTailStringValidator, AsDomainNameStringValidator, RejectDomainNameNotUnderPublicSuffixValidator and RejectDomainNamePublicSuffixValidator have also been added to allow the construction of the kind of complex validator policy required in this area. An example of the kind of policy detail achievable with this mechanism (which has been used for some years in the InCommon federation and the UK federation) can be found in this test case.

  • MDA-288: A new DuplicateEntityInAggregateCheckingStage allows checking of SAML metadata aggregates to ensure that they do not include entities with duplicate entityID values.

  • MDA-289: A new FixedStringIdentifierGenerationStrategy had been added for use when it is not necessary to use different ID attribute values for different documents.

  • MDA-299: A new AsURLStringValidator allows validating string values (such as DOM attributes using StringAttributeValidationStage) as URLs. Additional validators may then be applied to the resulting URL: HTTPSOnlyURLValidator, EmptyPortURLValidator and MissingHostURLValidator are provided for common use cases.

  • MDA-301: The BaseAsValidator class now includes detail from an IllegalArgumentException resulting from a failed conversion in the resulting error message. The extended addErrorMessage method used to implement this is also available for use by other BaseValidator subclasses.

API Changes

  • MDA-166: The ItemSerializer and ItemCollectionSerializer interfaces now allow serializers to throw IOException when appropriate. The provided DOMItemSerializer will throw an IOException wrapping a TransformerException if the latter is thrown during XML serialization. Previously, this condition would only have resulted in logging at ERROR level.

  • MDA-167: The ItemIdTransformStage now transforms identifiers using a collection of Function objects rather than of the similar Converter provided by the Spring framework. This also affects the type of the MDQueryMD5ItemIdTransformer and MDQuerySHA1ItemIdTransformer classes. This change will not affect existing configurations if only those classes are in use. This matches the use of Function elsewhere in the API.

  • MDA-169: The SAMLMetadataSupport.getDescriptorExtensions method has been renamed to getDescriptorExtension to reflect the fact that it returns a single result.

  • MDA-170: The PKCS11PrivateKeyFactoryBean temporarily introduced in v0.9.2 has, as indicated for that release, moved to the spring-extensions module; configurations will need to be adjusted accordingly:
    Old class name: net.shibboleth.metadata.util.PKCS11PrivateKeyFactoryBean
    New class name: net.shibboleth.ext.spring.factory.PKCS11PrivateKeyFactoryBean

  • MDA-171: The SAMLMetadataSupport.getDescriptorExtension method's parameters must now be non-null; their annotations have been changed to @Nonnull to correspond with this. In previous releases, they were annotated as @Nullable and passing null would result in the method returning null.

  • MDA-175: The ItemOrderingStrategy interface defined by the EntitiesDescriptorAssemblerStage now allows the ordering strategy to throw a StageProcessingException if, for example, the items presented are invalid in some way and can not be ordered. Such an exception will be propagated upwards to the caller of the stage's execute method.

  • MDA-179: The Version class's getMicroVersion method has been renamed to getPatchVersion to align with current (semantic versioning) terminology.

  • MDA-182: Several classes exposed as part of the API for building custom stages have been reworked to simplify implementation of other stages and to correspond to current naming conventions:

    • BaseStage has been renamed to AbstractStage

    • BaseIteratingStage has been renamed to AbstractFilteringStage

    • A new AbstractIteratingStage allows the simpler construction of stages which process each Item independently

  • MDA-188: The AbstractDOMTraversalStage framework has been generalised to allow the use of custom context objects specific to the particular traversal, rather than relying on sometimes tortured uses of the ClassToInstanceMultiMap to carry everything. This is a breaking change, but will only affect writers of stages derived from AbstractDOMTraversalStage:

    • Context objects must implement the DOMTraversalContext interface. This no longer includes the getStash method (returning a ClassToInstanceMultiMap but does add a new end() method to be called at the end of the traversal.

    • A basic implementation of SimpleDOMTraversalContext is provided without any data fields. This can be used in many cases where custom storage is not required in the context; for an example, see AbstractElementVisitingStage.

    • More complex cases can extend SimpleDOMTraversalContext to include additional fields and method. For a very straightforward example, see CRDetectionStage. A more complex example, including use of the end() method from DOMTraversalContext, can be found in ElementsStrippingStage.

  • MDA-198: In previous releases, the three X.509 validation component (X509RSAExponentValidator, X509RSAKeyLengthValidator and X509RSAOpenSSLBlacklistValidator) all set a default ID related to their names (e.g., RSAKeyLength). This default ID setting behaviour has been removed. This may have two effects on configurations which do not explicitly set the component ID:

    • If a configuration did not set the component ID, initializing the component will now fail with a ComponentInitializationException.

    • Configurations that implicitly set the component ID to a defaulted Spring component ID using IdentifiableBeanPostProcessor may give different results, as the Spring component ID may now appear in status objects replacing the previous default.

  • MDA-191: The stages PullUpCacheDurationStage, PullUpValidUntilStage, SetCacheDurationStage, SetValidUntilStage and ValidateValidUntilStage now use the Instant and Duration classes (introduced in Java 8) in their APIs rather than using long values representing milliseconds as in previous releases.

    • This aligns the metadata aggregator with other Shibboleth projects based on the Java 11 platform. If you use the Java language to configure these stages, you will need to re-code appropriately; in most cases, this can be done quickly using Instant.fromEpochMilli() and Duration.ofMillis(), but we recommend adopting the modern java.time classes throughout.

    • The DurationToLongConverter is no longer included as part of the java-support dependency. If you were using it as part of an XML configuration to specify durations in ISO 8601 format (e.g., "PT15M") then you should replace references to DurationToLongConverter with references to the new StringToDurationConverter.

  • MDA-206: The PipelineDemultiplexerStage's waitingForPipelines property previously defaulted to false, which could result in unexpected behaviour if the stage was invoked a second time without arranging to synchronise execution with the called pipelines. As a result, most deployments set waitingForPipelines to true so that the called pipelines will all complete before control is passed on from the PipelineDemultiplexerStage; this behaviour is now the default.

  • MDA-208: The enum type used to specify the SHA digest variant to be used in XMLSignatureSigningStage has been renamed from ShaVariant to SHAVariant to match our general conventions. If you refer to this type directly, your code will need to be changed. The bean property shaVariant has also been renamed to SHAVariant; in this case, the old property name is supported in this version with compile-time and run-time deprecation warnings; this migration support will be removed in V1.0.

  • MDA-219: In previous releases, the DOMResourceSourceStage tested for the existence of the supplied Resource as part of component initialization, potentially resulting in a ComponentInitializationException during initialize(). Unfortunately, this could also cause large remote resources to be fetched unnecessarily, and in addition did not guarantee that the Resource  would be accessible during normal operation. This check has been removed: now, a StageProcessingException will be thrown during stage execution instead, in the same way as was already the case for other I/O errors related to the Resource.

  • MDA-222: The contract for bean properties representing collections has changed:

    • Property setters for collection properties are now annotated as @Nonnull @NonnullElements @Unmodifiable.

      • Previously, some setters allowed a null value to act in place of an empty collection. This usage will now result in most cases in a NullPointerException.

      • Previously, some setters filtered null values out of provided collections. Again, this usage will now result in most cases in a NullPointerException.

      • Setters now guarantee not to modify the passed collection. This was previously true in practice in most cases, but is now guaranteed.

    • Most property getters for collection properties are also now annotated as @Nonnull @NonnullElements @Unmodifiable.

      • In exceptional cases, getters may be annotated as @NonnullAfterInit instead of @Nonnull. This is only done when an "empty collection" default is inappropriate for the property and would normally be accompanied by @NotEmpty on both the setter and the getter.

  • MDA-232: Previously, an invalid XPath expression provided to an XPathFilteringStage went largely unremarked: if the expression failed to compile, no filtering would be performed. If the expression could not be evaluated (for example, because it referenced a $variable) errors would be logged, and all items would be filtered out. Now:

    • An XPath expression which fails to compile will result in a ComponentInitializationException  when the stage is initialized.

    • An XPath expression which fails to evaluate will result in a StageProcessingException when the stage is executed.

  • MDA-234: The EntitiesDescriptorAssemblerStage previously allowed entities to be reordered by setting a property to an instance of an ItemOrderingStrategy sub-interface that was specific to Item<Element>. This interface has been extracted to the pipeline  package and expressed as a generic, allowing it to be reused for the new ItemOrderingStage. If you are using the orderingStrategy  property, your implementation of ItemOrderingStrategy will need to be retargeted at the new definition.

  • MDA-238: All uses of the Guava functional interfaces (Function, Predicate and Supplier) have been replaced with their Java 8 equivalents.

    • This aligns the metadata aggregator with other Shibboleth projects based on the Java 17 platform.

    • This change should be largely transparent, but if you use Java configuration you should note that Predicate's method name has changed from apply to test.

  • MDA-239: The collection of items passed to an ItemCollectionSerializer is now explicitly annotated as @NonnullElements. This constraint was previously assumed by implementations.

  • MDA-240: The QName constants used as part of the API of EntityRoleFilterStage have been moved to the SAMLMetadataSupport class. This will affect configurations which use those fields directly, but not configurations which construct their own equivalent QNames.

  • MDA-242: The entire framework has been reviewed for thread safety. The major highlights of this are:

    • Implementations of Item are no longer expected to be @ThreadSafe and in fact are expected to be @NotThreadSafe. In particular, the DOMElementItem implementation has always had mutable but unsynchronized contents both in the DOM data it wraps and in the ClassIndexedMultiMap that is used to store the item's ItemMetadata (although the ItemMetadata objects themselves are immutable and therefore thread-safe). Find more about this topic and its implications in the Javadoc for Item.

    • All classes that are part of configurations (pipelines, stages, validators, serializers, strategy functors etc.) are now @ThreadSafe or @Immutable.

    • These revisions have resulted in a number of minor changes to the API, such as the removal of some getters provided for non-property values and the movement of some non-API classes to "impl" sub-packages.

  • MDA-245: A number of APIs have been renamed with more inclusive and in many cases more obvious terminology. For example, the terms “blacklist” and “whitelist” are no longer used and have been replaced with terms such as keeping/removing or “keylist”. The majority of these changes are in lightly used parts of the API and in all cases some form of migration assistance has been provided for users of previous versions of the MDA. Some of this assistance requires the use of the standard bean definition resource; all such accommodations will be removed in version 1.0.0. Classes with affected methods are: EntityAttributeFilteringStage, EntityRegistrationAuthorityFilterStage, ContactPersonFilterStage, EntityFilterStage, EntityRoleFilterStage, ElementsStrippingStage, NamespacesStrippingStage and XMLSignatureValidationStage. The X509RSAOpenSSLBlacklistValidator has affected methods and has also been renamed as X509RSAOpenSSLKeylistValidator.

  • MDA-248: The Item and ItemMetadata interfaces no longer extend Serializable. This dated back to early design work related to state management in the Shibboleth V3 identity provider. The IdP later took a different direction, but the unnecessary constraint remained in the MDA until now. The effect of this removal is mainly to simplify implementations, and remove the need for extension classes to include a serialVersionUID variable. The inability to use Java serialization on these classes is not expected to be noticed in practice.

  • MDA-253: The signature of the Stage interface's execute method has changed from Collection<Item<T>> to List<Item<T>> to signify that the ordering of the items is significant, as it is for almost all use cases. This has always been the case in practice as the underlying implementation provided by SimpleItemCollectionFactory is an ArrayList. This should not significantly affect existing deployments, but will affect downstream packages where Stages are defined.

  • MDA-254: ItemIdentificationStrategy is now a generic interface, parameterised by the type of Item to be identified. This makes it possible to write type-safe implementations of the interface specialised for specific types of Item such as ItemIdentificationStrategy<Element> to process Item<Element>.

  • MDA-255: The default value of the noChildrenAProcessingError property for the EntitiesDescriptorAssemblerStage has been changed from false to true, and a StageProcessingException will now be thrown by default if an empty item collection is processed. Note that the behaviour of the stage in the case of noChildrenAProcessingError=false condition has not changed: it will still not generate a schema-invalid EntitiesDescriptor item, instead returning an empty item collection. Changing the default property is intended to make this behaviour less likely to pass unnoticed.

  • MDA-257: The contract implemented between AbstractItemMetadataSelectionStage and its subclasses (ItemMetadataFilterStage, ItemMetadataTerminationStage, StatusMetadataLoggingStage) has changed to add a type bound for the selection criteria, and to pass a ClassToInstanceMultiMap of that bound back to the subclasses. If you have a custom subclass of AbstractItemMetadataSelectionStage you will need to track this change. It should not affect existing deployments unless StatusMetadataLoggingStage has wrongly been passed selection criteria which were not subclasses of StatusMetadata.

  • MDA-258: The GenerateIdStage was an outlier in which a one-argument constructor was used instead of a property to set the IdentifierGenerationStrategy to use. The one-argument constructor is now deprecated, and will be removed in 1.0.0. Instead, as with all other stages, use the zero-argument constructor and set the generator property if required (using the new setGenerator/getGenerator methods). The default generator remains a Type4UUIDIdentifierGenerationStrategy.

  • MDA-272: The CRDetectionStage has moved from the net.shibboleth.metadata.dom package to net.shibboleth.metadata.dom.saml.

  • MDA-275: The PipelineDemultiplexerStage, PipelineMergeStage and SplitMergeStage classes have the ability to run multiple child pipelines, potentially concurrently. The API of these classes has changed from allowing an executorService property to be set with an external Java ExecutorService to an executor property which is instead an instance of the simpler Executor type. Existing configurations should still function, as ExecutorService is a subtype of Executor; however, the old accessor methods are deprecated and will be removed in MDA 1.0.0.
    This change to the API has allowed the default execution environment for the child pipelines to be improved: in previous versions of the MDA, this was a Java-defined single thread ExecutorService which created a non-daemon thread on which the pipelines would run for each instance of these classes. These non-daemon threads were permanently leaked regardless of whether the caller provided their own ExecutorService, and could prevent JVM exit. By default, these classes now use a simple Executor which runs the pipelines sequentially on the calling thread and creates no new threads of its own.
    The simple command line utility provided with the MDA previously always called System.exit even on non-exceptional termination, which terminated the JVM regardless of the presence of non-daemon threads. This hid the erroneous JVM exit behaviour in most situations. The command line utility no longer performs a System.exit call except on abnormal termination of the configuration; therefore, if you use these classes and provide your own Executor, you must manage its lifecycle so that it is correctly shut down with the application context or the application may no longer terminate correctly.

  • MDA-276: The CompositeStage and SimplePipeline classes have been unified, and are now interchangeable. CompositeStage now implements the Pipeline interface and can be used in any context where SimplePipeline was previously required; SimplePipeline is now essentially an alias for CompositeStage. To achieve this, the setComposedStages and getComposedStages methods have been deprecated in favour of setStages and getStages (in XML configuration, the composedStages property becomes just stages).

  • MDA-286: Testing APIs available for use by downstream projects have been moved into sub-packages with names ending in .testing.

  • MDA-291, MDA-292: StatusMetadata is now an abstract sealed class with ErrorStatus, InfoStatus and WarningStatus being the only permitted subclasses. Those subclasses are non-sealed and may be subclassed.

  • MDA-293: The utility class IdentifiableBeanPostProcessor has moved to a new package as a result of the dependency changes within the Shibboleth software stack. The standard bean definition resource now includes an abstract parent bean mda.IdentifiableBeanPostProcessor as the preferred upgrade path for this component.

  • MDA-295: The PipelineDemultiplexerStage has been changed to avoid the use of the Pair primitive. Instead of having a pipelineAndSelectionStrategies property a caller would set to a List<Pair> where each Pair held a Pipeline and a corresponding Predicate, the stage now has a pipelinesAndStrategies property which takes a List<PipelineAndStrategy> whose elements are a new record type holding the same components as the old Pair. A new mda.PipelineAndStrategy parent bean is provided for convenience in XML configuration.
    The old property definition is supported in this version with compile-time and run-time deprecation warnings; this migration support will be removed in V1.0.
    An example XML configuration might now look like this:

    <bean id="exampleDemux" parent="mda.PipelineDemultiplexerStage"> <property name="pipelinesAndStrategies"> <list> <bean parent="mda.PipelineAndStrategy"> <constructor-arg ref="examplePipeline"/> <constructor-arg ref="exampleSelector"/> </bean> </list> </property> </bean>

Improvements

  • MDA-183: the compromised-1024.txt and compromised-2048.txt resources have been extended with keys shipped with some releases of the Jetty container.

  • MDA-213: The definition of “white space” used by ElementWhitespaceTrimmingStage has been broadened to include all characters Java regards as whitespace; previously only ASCII control characters were trimmed.

  • MDA-260: The default logging level used by the simple command-line utility now includes INFO logging for all components in net.shibboleth.metadata, not just the CLI itself. The --verbose and --quiet levels are unchanged.

  • MDA-270: Error statuses added by X509ValidationStage are now more informative in the case where Java rejects the certificate entirely. Instead of "could not convert X509Certficate data" the message will now include the JDK’s reason, for example “X.509 certificate: Empty issuer DN not allowed in X509Certificates”.

  • MDA-278: The performance of EmptyContainerStrippingStage has been greatly improved in the case where large numbers of elements are being removed.

  • MDA-300: The “compromised” key list resources in the mda-keylists-rsa and mda-keylists-rsa-legacy modules now incorporate the standard RSA test keys from RFC 9500.

Bug Fixes

  • MDA-179: The Version class is now functional, rather than throwing a NullPointerException when used.

  • MDA-196: Setting the XMLSignatureSigningStage's includeX509SubjectName property to true caused a ClassCastException. It now behaves as intended, resulting in an <X509SubjectName> element being added to the signature's  <KeyInfo>'s <X509Data> element.

  • MDA-216: signatures generated by the XMLSignatureSigningStage under recent versions of Java are now consistent with signatures generated under earlier versions of Java: specifically, they no longer include explicit CR (ASCII 13) characters even on systems where lines are terminated by LF characters only.

  • MDA-220: EntityFilterStage handled the case of whitelisting incorrectly when the collection of entity IDs to whitelist was empty. The stage now correctly removes all items from the collection, rather than removing none of them.

  • MDA-224: XMLSignatureSigningStage threw an IndexOutOfBoundsException if the includeKeyValue property was set to true without either setting the publicKey or certificates properties; the stage now just omits the KeyValue from the signature as if includeKeyValue had been set to false.

  • MDA-226: The idTransformers property setter on ItemIdTransformStage incorrectly accumulated values from multiple calls instead of acting as a normal collection setter; this has been corrected.

  • MDA-243: The getter method for the ContactPersonFilterStage's designatedTypes property was misspelled. It is now getDesignatedTypes.

  • MDA-269: The MDQueryMD5ItemIdTransformer, MDQuerySHA1ItemIdTransformer, SHA1StringTransformer and X509RSAOpenSSLBlacklistValidator classes had a dependency on the default Charset, which in Java versions prior to Java 18 can vary across platforms, resulting in unstable results for strings containing non-ASCII characters. These classes now use UTF-8 exclusively when converting strings, matching the behaviour introduced by JEP 400 in Java 18.

  • MDA-273: CRDetectionStage was intended to indicate the SAML entity in which a &#13; appeared, as a way of helping to narrow this down in large aggregates. This functionality was broken but now appears: the error message is now prefixed with the entity’s ID attribute if present, or failing that its entityID.

  • MDA-280: Previously, EntityDescriptorItemIdPopulationStage would throw a ConstraintViolation at run time if asked to extract the entityID from an entity lacking that attribute, or for which the attribute was the empty string. This would result in failure of the entire pipeline. The stage now ignores such an entity, in the same way as it previously ignored items which were not EntityDescriptors at all. To detect such malformed entities in a controlled fashion, schema validation (and, optionally, removal of schema-invalid entities) is the recommended approach.