Dependency analysis

From time to time it becomes advisable to run an analysis over the pom files to ensure that each sub-projects dependencies are listed and that any needless dependencies are removed.  Maven provides a tool to help with this, but it needs to be treated with circumspection.

1. Ensure that the (entire) project builds cleanly

2. Run dependency analysis from the parent directory using the command 'mvn dependency:analyze'.  The key part of the log will look like

...

1. this for each subproject:

   mvn dependency:analyze output

   Code Block

   [WARNING] Used undeclared dependencies found: [WARNING] org.opensaml:opensaml-profile-api:jar:3.0-SNAPSHOT:compile [WARNING] com.google.guava:guava:jar:14.0.1:compile [WARNING] joda-time:joda-time:jar:2.2:compile [WARNING] Unused declared dependencies found: [WARNING] net.shibboleth.idp:idp-profile-api:jar:3.0-SNAPSHOT:compile [WARNING] org.springframework.webflow:spring-webflow:jar:2.3.2.RELEASE:compile [WARNING] org.glassfish:javax.json:jar:1.0.2:runtime [WARNING] org.codehaus.janino:janino:jar:2.6.1:compile [WARNING] ch.qos.logback:logback-classic:jar:1.0.11:

...

1. runtime [WARNING]

...

1. javax.mail:mail:jar:1.4.7:compile [WARNING] org.springframework:spring-context-support:jar:3.2.2.RELEASE:compile [WARNING] javax.servlet:servlet-api:jar:2.5:provided [WARNING] org.springframework:spring-test:jar:3.2.2.RELEASE:test [WARNING] net.shibboleth.utilities:java-support:test-jar:tests:2.0-SNAPSHOT:test [WARNING] org.slf4j:jcl-over-slf4j:jar:1.7.5:

...

1. runtime [WARNING] org.slf4j:jul-to-slf4j:jar:1.7.5:

...

1. runtime [WARNING] org.slf4j:log4j-over-slf4j:jar:1.7.5:

...

1. runtime [WARNING] xmlunit:xmlunit:jar:1.4:test

2. If in doubt, you can normally ignore any "unused declared" dependencies which have either the runtime or test scope when performing a review, as long as everything still works. If you use the command mvn -DignoreNonCompile dependency:analyze, these will be filtered out from the log.

3. Move any commonly "Used undeclared dependency" into the parent pom (in this case this was guava which occurred as

...

1. missing in more than 50% of the projects).

2. Iterate over each subproject (its is probably easiest to do this in the order in the log file - that is from most dependent to least)

   1. Add any missing "Used undeclared dependency"s to the subproject's pom. Note that

...

1. 1. sometimes dependencies are specified as required for compile when they are only actually required for test; this should be fixed by changing their scope.

   2. Carefully remove any "Unused declared dependency"s.  This list will contain modules which are not listed in the pom file and have been inherited, either from the parent pom, or from some dependent module.  Care is needed because

      1. Maven can only analyze static

...

1. 1. 1. dependencies, run time loaded dependencies (such as injecting opensaml-impl into the test scope) need to be listed to allow the test to run, equally several modules (e,g, janino, mail) are needed by the logging providers.

      2. Removing an unused dependency may well break "downstream" poms which require this module to be present.  You may wish to 'skip forward' and add the missing module to these modules at the same time that you remove the explicit dependency.

...

1. 1. 1. 
         Eclipse is quite good at warning that this has happened, but you need to keep it refreshed if editing outside eclipse.

   2. Run a build and test inside the subproject

2. Once all subprojects have been completed do a project level build and test - use of 'mvn site' rather than 'mvn install' can be helpful

3. Eyeball all the [changed] poms:

   • (Advisable) Ensure that the ordering of the dependencies is consistent

...

1. • ; the order currently used is:

     • Project supplied, versioned dependencies 

       Code Block
       language html/xml
       <dependency> <groupId>${project.groupId}</groupId> <artifactId>idp-authn-api</artifactId> <version>${project.version}</version> </dependency>

     • Project supplied, unversioned dependencies: 

       Code Block
       language html/xml
       <dependency> <groupId>net.shibboleth.ext</groupId> <artifactId>spring-extensions</artifactId> </dependency>

...

1. • • Unversioned, non-project dependencies: 

       Code Block
       language html/xml
       <dependency> <groupId>org.apache.velocity</groupId> <artifactId>velocity</artifactId> </dependency>

   • Check for -impl modules being included in -api and -impl modules (except for test scope)

2. Check in.  It is advisable to provoke a jenkins build at this stage "just in case".

Which Scope Should It Be?

• provided dependencies should be used only for APIs which are supplied by the run-time container.

• compile dependencies should be used for all other modules whose APIs are referenced by the source code.

• If a compile dependency is used by optional code in a project like java-support, the dependency should be compile but marked as optional. Client projects which require the optional functionality must manually include the missing transitive dependency with runtime scope if they do not already require it at compile scope.

• If a module is required to be present at run time but is not referenced by the source code, it is a runtime scope dependency.

• If a module is not required by the main source code but is required for testing, it is a test scope dependency.

Dependency updates

Dependencies can be updated when new versions become available during development. It is also sensible to perform a check for dependency updates of which we are not otherwise aware prior to a release.

To review the available dependency updates in a particular context (either the parent POM or a subordinate project which defines its own dependencies not managed by the parent project) the command of interest is:

mvn versions:display-dependency-updates

Beta and alpha release suggested as upgrades should be ignored (although there are sometimes other releases between current and the beta which may warrant an upgrade). One way of cutting down this clutter is to constrain the updates the versionsplugin will show you. For example, to show only patch updates (assuming semantic versioning) you can say:

mvn versions:display-dependency-updates -DallowAnyUpdates=false -DallowMajorUpdates=false -DallowMinorUpdates=false

Note that these options are hierarchical: allowAnyUpdates must be disabled before allowMajorUpdates has any effect, and so forth.

Test scopes can usually be upgraded easily, equally minor revisions.   

Obviously critical subsystems (XML handling, beans, encoders) need further discussion.

Dependencies for the Java 17 platform

The anchor product for the Java 17 platform is Identity Provider v5.0. This is based on Jakarta Servlet 5 and therefore Jakarta EE 9. As EE 10 and Servlet 6 already exist, care needs to be taken to ensure a consistent set of dependencies. The relevant components of Jakarta EE 9 are as follows:

• Jakarta Activation 2.0

• Jakarta Annotations 2.0

• JSON Binding (JSON-B) 2.0

• JSON Processing (JSON-P) 2.0

• Jakarta Mail 2.0

• Jakarta RESTful Web Services 3.0

• Jakarta Persistence 3.0

• Jakarta Server Pages 3.0

• Jakarta Servlet 5.0

• Jakarta XML Binding (JAXB) 3.0 (note that we no longer use this in the IdP, as we no longer use Hibernate)

As of 2023-08-22, the following exceptions can be noted:

Dependencies for the Java 11 platform

The anchor product for the Java 11 platform is Identity Provider v4.x. This is based on Servlet 3.1 and therefore Java EE 7.

As of 2021-02-14, for the Java 11 platform, the following exceptions can be noted:

Plugin Maintenance

The simplest way to find what is out of date is to run 

mvn versions:display-plugin-updates

Then edit the requisite pom file (which will usually be the parent pom).  Do a complete build  prior to checking in. Use of 'mvn site' rather than 'mvn install' can be helpful in heading off Jenkins issues.