August 2023 Update

Created by Scott Cantor
Last updated: Sept 11, 2023
4 min read

It’s been a couple of months since the last update for the usual summer slowdown reasons, but work has been ongoing to get through the IdP backlog, get the installers in shape, and complete some internal testing so we could get a beta released, which we did last week. Paul Caskey kindly worked on a TAP container build for the beta as well.

Reiterating some of the basics:

  • You need Java 17 and Jetty 11 or Tomcat 10.1 to run V5. Jetty 11 is an essentially compatible migration from Jetty 10, save for the logback library versions it uses (if the logback module is enabled).

  • Java 17 can be used with the V4 IdP to prepare for eventual use with V5, so if you’re not on Java 17 yet, getting there is a good step to take.

  • Java 17 lacks a built-in Javascript engine, so as most deployers tend to have some scripting somewhere installing one of our scripting plugins is generally required. We have left this as an add-on simply to ensure flexibility and prevent any “core” dependency on any Javascript engine going forward.

Notably, it can be a little bit awkward right now to test unreleased plugins. We have plans to improve that, but for now, we’ve signed all of them, provided a trust store file to use, and the --noCheck option bypasses the usual version support enforcement that otherwise blocks things.

To provide some perspective on the impact of the upgrade, it took (calendar-wise) basically a couple of weeks at most to port all of the project’s official plugins over to work with V5. While they all required changes, the smaller plugins took perhaps an hour of work to port and the largest ones still took a pretty small amount of time. My own upgrade to a snapshot prior to the beta release identified one breaking issue (Basic Authentication support in the HttpClient that I used in my HTTP data connectors) that required some new code to deal with. Everything else basically ran with warnings (which I haven’t bothered to clean up yet), so the total time spent that I could attribute to the upgrade itself was on the order of a few hours.

Given the scope of changes we have made, this is an achievement, and a validation of the somewhat extended timeline we’ve spent getting to this point. Working on compatibility tricks has taken up a non-trivial portion of the work. In light of the experiences so far, we do not expect a long beta cycle, and I expect a final release certainly no later than the end of September, hopefully a bit sooner if we can release it ahead of the Internet2 TechExchange conference. Mostly we are chasing dependencies and fixing bugs at this point.

A couple of the changes to the Windows installation process deserve mention:

  • The Windows installer now directly unpacks the distribution and invokes the standard installer. The advantage of this, aside from less duplication, is that it’s now possible to mix use of the standard installation process and the Windows installer over time (or move from one to the other) more or less interchangeably.

  • Secondly, the option of Jetty installation has been moved to a separate package so that it can be maintained independently without having to produce full IdP installer releases. It’s possible this might evolve in the future so as to isolate virtually all of the IdP-specific aspects of this and leave mostly just a functional Jetty installer for Windows. Ideally maybe even something the Jetty project might take over, though that’s probably a pipe dream at this stage. Somewhat more believable might be a shell-based installer for other platforms that produces a similar sort of starting point.

Another thing that will be noticeable if testing this upgrade is that you will have many more “extra” files created as comparison points, as all of the configuration files are now “known” to the IdP’s module system and so backup versions get created for them. They are also versioned backups so that multiple upgrades in sequence of different versions don’t overwrite those files and can be compared over time for those so inclined. A future version will hopefully provide some kind of tool along the lines of RPM’s ability to run diffs of the configuration changes after an upgrade.

Substantial effort over the last month also went into a redesign of the metadata generation plugin that was released earlier as somewhat of an experiment. The new version is an attempt to produce something similar to the metagen shell script that has been, somewhat oddly, included in the SP distribution for some years now. The idea of metadata generation fits the IdP better than the SP, and porting it to a Java-based design was the best way I saw to produce something portable and that we could maintain. It is Velocity-based, so allows for customizable templates to be used to drive the generation process, but has a ton of options that extend to producing a lot of different metadata constructs. It will be a work in progress for a while, but is at least capable enough for an initial release. The documentation on it is forthcoming (it is not at all compatible with the original 1.0 release of the plugin). It is hoped that it will help the deployers like myself that are still command-line-centric and need a way to script metadata for local and cloud services, as I have done for many years.

Obviously we’ll be focused on releasing everything over the next month or so, including all of the plugins we have, so it’s a lot of stuff to get out there. There are a few bug fixes and features expected in the OP plugin release so it’s not solely just a port to V5, but it is mostly that.

Once the dust clears, the project will be engaging with the Board and our membership about what comes next. Likely a range of options and budgeting levels will be considered while we factor in all of the many projects people have inquired about over the last few years, and the SP will of course be a big part of that conversation.