Example 4.1 Upgrade

This is a record of the changes made to "modernize" a system upgraded from 3.0 through 4.0.1, and now on to 4.1 to provide some concrete examples of the new appoaches in 4.1. There aren't radical differences, just some XML → property changes but some of the changes are a little subtle.

After doing the usual install on top of my tree to upgrade to 4.1, I started the IdP back up and verified that everything worked as before. The only additional warning in the log was because the EntityRoleWhitelist filter was deprecated/renamed, so that's a simple EntityRoleWhitelist → EntityRole conversion in metadata-providers.xml.

At that point, I could certainly stop and that's fine for a lot of people, maybe most, but the point here is to run through some of the cleanup I can do to reduce the number of files and convert some XML to properties.

Module Cleanup

The first thing to do is to use the new module system to check for modules that are enabled (which is likely most of them) and that you may want to disable. DIsabling them will remove some files and rename others with ".idpsave" extensions, after which it's a simple matter to remove them since you shouldn't need them if you're disabling a feature you're not using.

My example (obviously which modules somebody doesn't need will depend on what they use):

If you don't know what a module does, run "module -i <name>" to find out. By definition you don't need any modules that are not already enabled as they're either new features or things you weren't using before.

This is a good place to do a second "restart" checkpoint and make sure things are working.

Configuration Changes

Next up is to remove session-managers.xml, which is likely only modified if you emptied the client storage services list if you're not using them. I am so the internal default matches that list and I just removed the file. If I had changed the bean I would have just moved it to global.xml.

Before changing anything else, the next change is to adjust how properties are found by pruning back the idp.additionalProperties list to limit it to files outside the conf tree and to add in the new setting idp.searchForProperties = true so that all the new property files and all the old ones are loaded for me. At this point, I had/have two files I need to continue to point to directly, /secure/password.properties and /secure/duo.properties

I did my own work to isolate secrets and passwords before the IdP defaulted to that a bit, and I put all of my original Duo settings in a similar place. I'll rework that later in this walkthrough but for now I'm just leaving it.

Another restart checkpoint. Restarting here should be safe because most of the new properties are just ignored in favor of the existing XML bean definitions I have in place so they're loaded but not doing anything yet.

The rest of the changes are more involved and depend more on local changes made or not made. They're grouped below into various topics.

Admin Changes

Now I'll move on to "admin" cleanup, which is the easiest. I didn't have any meaningful customizations to admin/general-admin.xml, so all I had to do is just remove it and restart, which activates the (pretty unimportant) settings in admin/admin.properties and gets rid of the INFO messages regarding my admin descriptors overriding the auto-wired versions that are internal now. If I had made any changes to the XML, I would have had to adjust some properties to match, or in extreme cases may have had to leave one or more of the admin flow beans in place (or move the individual beans to global.xml if I cared to get rid of the file).

After restarting, I verified my admin functions are working fine, such as the status page, reloading services, aacli, etc.

Subject C14N Changes

This is another minor area, and is just cosmetically improved. There are no modules defined for this layer because all of the existing per-method configuration files have been supplanted by the use of properties and a suggestion to define (or move) any more complex beans needed in conf/c14n/subject-c14n.xml.

The only method I use is the attribute-driven c14n flow. Most of those settings are in conf/c14n/subject-c14n.properties now, except for the custom bean I defined for obtaining the input Principal name for the resolver to use, which I moved to conf/c14n/subject-c14n.xml as suggested. I removed simple-subject-c14n.xml and x500-subject-c14n-config.xml, as well as attribute-sourced-subject-c14n-config.xml since none of them are in use by me anymore.

After restarting, I verified my logins are working fine including unusual username inputs that would exercise that logic.

Intercept Changes

Next step is the "intercept" cleanup. In my case, I have an extension that implements some additional interceptors used with Amazon AWS, so I can't initially just dump the intercept/profile-intercept.xml file. Later on I'll enhance the extension into a plugin and internally define Spring beans in the jar so I can get rid of these definitions but for now I can just get rid of the old list of flows and move my AWS flow descriptor beans outside the list. Most people won't have any custom flows like this and can just delete the file.

One note: some old features don't rely on defined configuration files anymore and so the "old" file exists but isn't associated with a module you can disable to remove it. The external-intercept-config.xml file is an example of that, so if you don't use that flow (and you probably don't), deleting the file by hand is necessary if you want to clean it up.

Most of the old interceptors do have modules and many also do rely on XML configuration so I'm left with a few XML files in there but less than before.

Restarting again I verified my interceptors all work as before.

Authentication Changes

Before tackling the "authn" area, I did some Duo cleanup to match more of the default layout of settings. I copied my duo.properties file back into conf/authn and adjusted things so my secret keys were moved into my password.properties file, and then removed the reference to duo.properties from idp.properties since it will be auto-located now. This is all cosmetic really but it aligns things to the defaults better and by putting duo.properties in the "expected" place, the system accurately reports that the idp.authn.Duo module is enabled since it determines that based on file presence.

The authn cleanup is by far the majority of the work because the most existing customizations to flow settings will usually be here, and in particular for a properly configured IdP doing anything but simple authentication there will be "supportedPrincipals" collections to transfer from general-authn.xml into authn.properties.

A pre-task here is to move some authn-related properties that used to be in idp.properties into authn.properties, just to avoid confusion. They're all commented out in the new file to start with to avoid any breakage but it's easy to move them.

There are basically three stages of real work here, to adjust and reduce content in authn-comparison.xml, to migrate general-authn.xml changes into properties, and finally to migrate flow-specific configuration if desired in some cases.

The first two are kind of intertwined because the final goal is to be able to delete general-authn.xml, so the first thing I did is to move a couple of beans I was using into authn-comparison.xml (shibboleth.AuthenticationPrincipalWeightMap and shibboleth.PrincipalSymbolics). Then I verified that I had made no other changes to the original authn-comparison.xml file because I never attempted to support any of the "inexact" matching options in SAML. The majority of people are in that state, which means that much of the original content can be deleted now that the default behavior is wired up internally. You can just compare to the dist copy to see what's left in it by default. This eliminates more INFO messages about overriding auto-wired beans.

Then I went back into general-authn.xml and removed all flow descriptor beans of methods I do not use, leaving me with the three I do use, Password, Duo, and MFA. At that point, I know I've made changes to the "supportedPrincipals" but I want to verify exactly what else I changed so I can make any simple edits to authn.properties first.

The particular rule of thumb in authn.properties is that most of the settings actually in the file commented out are sort of geared around identifying settings that are typically most likely to already have been adjusted in the XML. That's not 100% true, but it's generally the case. After review, I identified only one setting beyond the principal collections that I had changed, which was to toggle Duo to support nonBrowser use.

So combining that one small thing with the principal settings, my (relevant) uncommented authn.properties values needed (for now) are:

Initial authn.properties Changes

Easier review and adjusting of these lists vs. the XML syntax is one of the benefits. Note that unfortunately the syntax requires comma-delimited values, which is easy to miss and likely to be a common source of errors.

At this point, I can delete general-authn.xml as superfluous.

I restart again for another checkpoint, and my logs at startup contain no more INFO messages that mention replacement of auto-wired components.

Lastly, I can migrate some actual method-specific authentication options out of the XML and get rid of some older files.

In the case of MFA, there's nothing I can really change, as all of that configuration remains XML-driven, so nothing to do there.

In the case of Duo, I don't have any XML settings (I used to, but my system now relies on just a pair of property-driven Duo integrations so I don't have any odd configuration right now). SInce I don't have anything in duo-authn-config.xml, I can delete it. The Duo module actually knows this file is optional so the module still reports as enabled even when that file is missing.

The Password flow is where I do have some simplifications I can make. I don't use JAAS anymore, so I can immediately get rid of my old jaas-authn-config.xml file. The ldap-authn-config.xml file is unchanged from the 4.0 version and the definitions in that file were all actually property-driven anyway, so the internal definitions of the beans work fine for me and I can get rid of ldap-authn-config.xml as well (once I stop importing it). The Kerberos settings in krb5-authn-config.xml are needed but are almost always simple property conversions, in my case setting the service principal name and keytab path in authn.properties. Once I do that, there's nothing in the XML file I need and can delete it.

I'm left with my password-authn-config.xml file and I still need that but can remove a lot of property-based material from it. I can get rid of all the imports of the old method-specific files I just removed. I can get rid of a bunch of TRUE/FALSE Boolean beans in the file as all of those are properties now. I have to toggle one of those in authn.properties to true (I lowercase the username on the way in) but the rest are defaulted. Again, nothing really important, just less clutter in the file.