V5CodeReorganization

Notes on possible directions for reorganizing code in the “5th generation” timeframe (i.e., when we ship a Spring 6, Java 17, Jakarta EE 9 version).

Current State

The “stack” of Maven projects (excluding IdP plugins and the idwsfconsumer artifact expected to go away in V5):

• java-support

  • spring-extensions

    • java-mda

    • java-identity-provider (also depends on java-opensaml obviously)

  • java-opensaml

    • xmlsectool (just -core, -security-api and -xmlsec-api)

Problems

Both java-support and spring-extensions are grab-bag single module projects with optional dependencies and generic names that don’t tie them clearly to the project.

The addition of the SP to the Java domain implies a likely need for additional sharing of components between the IdP and SP (less likely the MDA). Some of these components are larger and more coherent than others, leaving another grab bag that is at a higher layer and likely in some cases to depend on OpenSAML (anything that doesn’t could likely move lower but adds to the incoherence there.

Properly factoring every unique shared component into its own full project is possibly too many components to efficiently manage, particularly during releases.

Component Inventory

Currently in java-support:

• Custom annotations

• Codec support (baseN, HTML encoding)

• Collection classes and support

• Component APIs (the initializable, destroyable, identifiable, etc. stuff)

• (tentatively but unlikely to stay) the DDF code ported over for the SP to use

• HTTP client supporting code (lots of security bits in OpenSAML though)

  • Requires 3 of the 9 <optional> dependencies

• Predicate and Function support and implementations

• “Net” stuff, itself a giant grab bag, the Jakarta dependencies are mainly here

• “Primitive” support, another grab bag

• The core resolver and criterion classes used throughout OpenSAML and the IdP

• The Resource class that abstracts away from Spring’s

• Some base classes for Scripted stuff

• “Security” stuff, another grab bag including Access Control and DataSealer

  • Requires 2 of the 9 <optional> dependencies

• The core classes for reloadable services without the Spring additions

• Some Velocity supporting code

  • Requires 1 of the 9 <optional> dependencies

• XML foundational stuff, including the ParserPool layer

The remaining <optional> dependencies are marked as 2 for logging and one for command lines.

Currently in spring-extensions:

• Spring-aware CLI classes

• Spring converters and some other low-level Spring additions

• Spring context classes used by the IdP at least, and SP later

• Spring error handling code

• Spring factory beans, including HTTP clients and credentials

• Spring resource additions and helpers

• Spring-aware reloadable service classes

• “Utilities”, full of all sorts of stuff including the main support class for building ApplicationContexts

• The port of Spring’s old Velocity support with bug fixes we need (no sign Velocity is really ever going to accept those)

Future State

We still need virtually all of the existing code “somewhere”. These chunks of code might be large enough or at least prominent enough to warrant their own projects if we’re prepared to manage that many projects:

• HTTP client supporting code

• The DataSealer

• Velocity support

• XML stuff

There are a lot of candidate components that may end up in common between the IdP and SP. Most of them probably will depend on OpenSAML, a few don’t. The full list will evolve based on development so is speculative at best at this stage.

• Small components that should be shared at least through common base classes we can override to deal with differences or regressions that arise:

  • LoggingService → interface fits in java-support today

  • LogbackLoggingService → implementation fits in spring-extensions today

  • Reloadable AccessControlService → implementation fits in spring-extensions today

  • ReloadableServiceGaugeSet → probably needs to be moved down

  • Property-aware ApplicationContextInitializer → fits in spring-extensions today

  • OpenSAMLConfigBean → probably could migrate to OpenSAML (it’s not Spring-aware, just a bean)

  • Some CLI classes might have some applicability to the SP, are located in idp-core

  • XMLObject implementations for Shibboleth metadata extensions

  • Plugin/module supporting classes

• Larger components that might warrant a new, full-fledged, multi-module project:

  • Metadata classes (Spring parsing, supporting classes)

  • Attribute classes, transcoding/registry, resolver, filter

    • Arguably separating the modules as the IdP does now may not even be beneficial, but probably not worth collapsing either?

    • includes XML Schemas and configuration formats

  • Relying party layer

    • This is probably not going to be directly applicable but there are overlaps with the SP and the way we model proxying that might warrant it

  • Session layer

    • Clear overlaps to some extent between IdP and SP, though likely differences too

  • NameID consumption / Subject C14N

    • Definitely used by proxy functionality so may be somehow adaptable to SP

Proposals in Flight

Discussed again on 6/3 call. Made initial plans to work on reorganizing projects as follows:

1. Create a new top-level “shared/utility” repo with java-support and spring-extensions moved into subdirectories of a new multi-module project, with history rewritten via reposurgeon to reflect those locations. Any commit signatures will be removed. Tags and branches will be renamed for uniqueness. Code can be moved around and new modules created at that point. Some of the code may then be migrated into new top-level repos if desired.

   1. See https://shibboleth.atlassian.net/wiki/spaces/~65887685/pages/2991489026

2. Begin to create new top-level projects for large IdP subsystems we need for the SP by cloning the IdP repository and removing content. History can be cleaned up after that point to eliminate the deleted content.

   1. Created java-shib-metadata and java-shib-attribute and have completed rebasing the IdP on them.

   2. Some code also moved down into a new opensaml-spring module, with attendant loss of history.