Supply Chain Defence for the Shibboleth Java Products

Owned by Rod Widdowson
Last updated: Sept 08, 2023
5 min read

This page outlines the Shibboleth Project’s approach to defending our, and our customers', supply chains. Whilst we do not view what we are doing as being particularly extensive or “paranoid”, we have found it useful to have somewhere which explains what we do and (more importantly) why.

This document is an attempt to describe the issues we are trying to defend against, our threat models, our defence process and why we need support from teams providing upstream tools and libraries.

Threat Models

We are specifically interested in defending against two supply chain attacks:

Deployer Supply Chain

This is an attack on our users' supply chain by malware shipped as part of a Shibboleth distribution, where by the term “user” we mean someone who is deploying the Shibboleth products to provide middleware services to their users. Consider that a Shibboleth IdP is intimately attached to an enterprise Identity management system and possibly to an enterprise databases and that it is at liberty to provide trusted access via a webserver to all your users; it is easy to see that there is scope for doing a great deal of mischief.

Typically, Shibboleth distribution contains supporting libraries from outwith the project, and any of them can be a vector for injected malware. Our defence against this is to try to ensure that all third party software that we ship as part of our packages has not been tampered with since it was verified by an entity (preferably a person) that can be verified as being a member of the library’s development team. Obviously “tampered” extends to meaning “invalidly submitted on behalf of”.

Developer Supply Chain

We use many tools when building the Shibboleth projects and these tools themselves can be used to attack our supply chain by inserting malware into the artifacts (jar files) that they are used to produce. This would be a more advanced attack — inserting a malware packet that itself creates a malware packet in one of our distributables — but it also has a much wider attack surface. Many, many hundreds of jar files are involved in our builds (as opposed to a few tens of jars shipped).

Again, our defence is to ensure that all tools that have been used to build a Shibboleth project has not been tampered with since it was verified by an entity (preferably a person) that can be verified as being a member of the tool’s development team.

Out of Scope

As mentioned above we do not view our checking as being advanced and in particular we do not believe that we can defend against advanced and well resourced “nation state” attacks. We simple seek to ensure that we do not present low-hanging fruit.

Historical Perspective

In the earlier days of the Internet it was common to “just trust” software that was downloaded; malware had not been commercialized and in general the risks were low.

When the Shibboleth project started using Maven to build its Java projects (around the time of IdP V3), the world had changed enough that some defence seemed advisable. Initially we configured our build environment such that, during a release build, all third part products and tools could only be downloaded from servers that we owned and populated. Individual developers would review a jar file before it was uploaded (and of course once it was uploaded its contents could not be changed).

The world has again moved on and it is no longer practical to do this. Rather we are constrained to use the centralized servers to source our third party libraries. Our base assumption is that downloaded binaries (from whatever source) cannot be trusted per se, no matter where they are downloaded from. For instance there are jars with the same names as those provided by the Shibboleth project up on one of these servers and the project has never been involved in doing this. To reiterate: we assume that a file and its checksum can be tampered with; we need a way to validate the contents using some trust mechanism which is independent of the distribution mechanism. Digital cryptographic signing is a usual mechanism to enable this.

To reiterate, we are not making a judgement about the products we use, we are simply ensuring that what we use is the same as what the development team produced..

What do we do?

Our defence against these threats is to check the veracity of every jar that we ship, and every jar that we download. We do this using the GPG signatures that are usually (but not always) available alongside the artifact in the centralized repository. The Shibboleth project has developed an extensive library of signing certificates and these are stored on a per (Maven) group basis: just because someone is trusted to signed something in (say) org.apache.ant doesn’t render them trusted to sign things for com.microsoft.sqlserver.

As part of our build, we check the signatures for the jars that we ship using the certificates in this library, thus addressing the first threat model. At the end of the build the entire library of jar files implicated in the build process is similarly checked, addressing the second threat model. All our Java products are built in an isolated build environment, starting de novo for each build and so we can feel slightly assured that nothing has slipped by.

In order for this to keep working it is essential that we continue to maintain the library of signing certificates. The certificates used by our dependencies change frequently and we need to add the new certificates to our library having first ensured that they are trustworthy. This last phrase is key: it does not suffice to download a certificate from a keyserver and start using it. Anyone can upload a signing certificate and so its existence on a keyserver does not mean we can draw any conclusions as to its source and trust worthiness.

We validate each key in one or more of the following ways:

  • Finding the key published on a website that can be attributed to the project. For example, the Shibboleth project publishes its keys in the download area. This is considered the gold standard.

  • As a variation of the above, some projects “publish” their keys in their GitHub repository.

  • Finding that the same key has been used to sign a tag in the project’s source repository. This is tenuous but acceptable.

  • It has (rarely) been possible to add a key because of the GPG “web of trust” framework (maybe a key has been signed by a trusted key, or perhaps by a key that has also signed a trusted key).

  • We can accept a key that is already acceptable to a related project (for instance two sub groups of the org.apache.maven parent)

If none of these are available we contact the project owner asking for them to publish the key somewhere (as per the above), or failing that to acknowledge the key in email (although this is cumbersome and extremely second best).

It is important to understand that as a product team, making your signing certificate available is really important. Whilst is is inconvenient, without having a signing certificate available in a trust worthy location you are rendering your products open to being hijacked and misused. Further an important part of your user community will not be able to use your latest versions leaving you open to heightened support costs.

Under the rules the Shibboleth project use, when faced with the situation of not being able to trust a jar file we will generally refuse to use the affected jar files. At the time of writing there are several projects where we do not use their latest versions because of this issue.