Santuario 2.0.7 changes Base64 output
Description
Environment
Activity
Scott Cantor October 27, 2020 at 12:35 PM
For my own sanity, since I had blocked all this from my memory, none of this applies when the system property is set, right? The IdP is just producing run-on lines?
Ian Young October 27, 2020 at 10:11 AM
Completed, commit 713b50865cc4a5df1d7a205f066eb28ca5223a41.
Need to document, and check on the -multi job.
Ian Young October 26, 2020 at 6:12 PMEdited
Mirabile dictu, the add-CRs thing doesn't apply to EC key values, just RSA and DSA.
Ian Young October 26, 2020 at 5:19 PMEdited
Unpinning the version of Santuario in use reveals that the current version (2.1.4) differs from 2.0.6 in the following ways:
As we discovered in MDA-216, CRs are present in the Java string contents of
ds:SignatureValueandds:X509CertificateNot noticed in (and perhaps we need to circle back on that, although it may be using the JDK's API and not xmlsec's) CRs are also present now in
ds:Modulus,ds:G,ds: P}} and {{ds:Y(for DSA signatures).For
ds:Modulus,ds:G,ds: P}} and {{ds:Ythe values have changed from<x>NLvalueNL</x>to<x>value</x>for no apparent reason.
The CRs are poison for a couple of different reasons:
They mean people can't compare old with new to gain confidence in a new version of xmlsectool being a drop-in upgrade.
We know that some processors (ADFS?) just can't deal with them, once they are encoded as
& # 1 3 ;in the output document (or, in some versions of the JDK, apparently& # x D ;because consistency is for wimps).
The change to element formatting is less significant:
They mean people can't compare old with new to gain confidence in a new version of xmlsectool being a drop-in upgrade.
They break some of our tests.
Having been round this several times in my mind, my conclusion is that the right way to handle this is to force the output to look like it did before, by a targeted intervention on the generated signature element, based on:
A list of elements within the signature for which CRs will be stripped (although arguably this should just be "every element with only text content").
A separate list of elements for which the NL, value, NL formatting should be ensured.
A number of elements will appear in both lists.
My unit test for this (ignoring the other tests that are also failing with the latest xmlsec) checks for any CRs within the signature so that we'll gets a heads-up if something in the JDK breaks this again. I'll try and get it to test one of each credential type (RSA, DSA, EC) just to be on the safe side.
Ian Young October 1, 2020 at 9:51 AM
The other alternative for XMLSecTool is to do what I've done for the MDA, which is to manipulate the signature to remove the problematic CRs. I feel it's important for deployers to be able to validate a new version as much as possible by seeing identical output when that's possible.
Version 2.0.7 of the Santuario XML security library changes the way multi-line Base64 representations (such as public keys) are expressed in generated XML, by forcing CRLF line endings within the text content. This is irrespective of whether any other text content is CRLF terminated, or whether the document is ultimately serialised using Unix or Windows conventions.
This is due to the following change:
https://issues.apache.org/jira/browse/SANTUARIO-446
I think that change is misguided, but there's not much we can do at this stage.
The result is one test failure, and potentially inconsistent line termination in output files which will be problematic for end users. To avoid this, we should (for now) pin the version of Santuario in use to 2.0.6 and review for the next release.
There are more changes in this area in even more recent versions of Santuario, so it is possible that this problem will go away again by the time we want to make a new release based on the Shibbolth V4 (Java 8) stack.