Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device.
Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device. Atlassian cookies and tracking notice, (opens new window)
The audit logging feature provides a detailed record of every transaction handled by the IdP to allow tracing of user activity, statistical analysis of usage, legal record keeping, etc. It is highly extensible, allowing the format to be customized, selection of fields to log on a per-profile basis, and the addition of custom audit fields using additional Java code or scriptlets without modifying core code in most cases. Unlike the diagnostic logs, the audit log is meant to be machine-readable.
The primary audit stream deals with the request/response profile handling of the IdP itself (this also includes logging the requests and responses associated with the SAML proxying support). This is routed by default through a logger named "Shibboleth-Audit" to a file called idp-audit.log.
Another audit record stream exists for logging decisions made by users regarding attribute release and terms of use acceptance and is routed by default through a logger named "Shibboleth-Consent-Audit" to a file called idp-consent-audit.log.
There is also separate stream of authentication audit events for each supported login flow. By default this feature is not enabled, but if done, these records appear in the same audit stream as the primary auditing. It is straightforward (and usually advisable) to route these records by category to a separate file or destination since the record format is not the same as the primary records.
Finally, the primary audit log supports the ability to create multiple log records in different formats or with different fields at the same time for the same events (and optionally route them to different categories for output to different destinations). The authentication auditing does not support this feature for simplicity (and because of the relative paucity of fields).
General Configuration
As with the diagnostic logs, the actual logging is handled by the Logback implementation and so the portion of configuration that deals with how the logs are written and managed is in the conf/logback.xml file through the manipulation of the two <logger> elements and related content for the "Shibboleth-Audit" and "Shibboleth-Consent-Audit" categories.
However, there is minimal formatting applied there, just direct output of the audit message. The configuration of the audit format(s) to use for particular transactions is handled separately, in conf/audit.xml, to allow customizing of the audit fields themselves, using formatting tokens that are themselves configured within the IdP and are extensible. You can think of it like a second logging abstraction on top of the underlying one to avoid reliance on Logback-specific extensions.
There are really two halves to the auditing system: field extraction and log output. While suitable defaults are provided, both halves can be changed using conf/audit.xml (or conf/intercept/consent-intercept-config.xml for the “Shibboleth-Consent-Audit” category). Extending field extraction typically requires additional Java code or scripts to pull new data out of the request state and make it available to the log output stage, but there are a lot of built-in fields.
Field extraction is the process of populating a labeled field in an AuditContext object that is built up over the life of a transaction. The context is just a map of field names to collections of values and the field names are just conventions built-in to the software to provide a standard set of useful fields to audit. Fields are extracted at various points through the life of a transaction so that adequate detail can be exposed about the request, the processing of the request, and the response. These extraction points are associated with collections of field extraction beans that do the actual work to pull data out of the state of the transaction and store it for output. The built-in extraction points and beans are as follows (empty beans are not provided to reduce clutter, but you can add them, per the example below): shibboleth.FlowStartAuditExtractors Extractors that run immediately after the flow starts operating and the ability to add fields is possible shibboleth.PostDecodeAuditExtractors Extractors focused on the input message, runs after decoding the input message shibboleth.PostLookupAuditExtractors Runs after establishing the sender's identity, supplements the input side with information about the relying party shibboleth.PostAssertionAuditExtractors Extractors focused on authentication, session, and attribute information, runs after those processes and after the initial creation of the outgoing assertion (if relevant to the request) shibboleth.PostResponseAuditExtractors Extractors focused on the outgoing message, runs after construction of the message shibboleth.LogoutRequestAuditExtractors Extractors focused on information specific to a SAML LogoutRequest message, runs after decoding of the message shibboleth.LogoutAuditExtractors Extractors focused on information in a non-SAML logout shibboleth.ErrorViewAuditExtractors Extracts subject, session, and attribute information, runs in the event that a local error is generated shibboleth.consent.PreConsentAuditExtractors Extracts basic information about the consent step, runs before a decision is made Empty map beans are declared in conf/intercept/consent-intercept-config.xml shibboleth.consent.ConsentAuditExtractors Extracts information about the decision, runs after a decision is made Empty map beans are declared in conf/intercept/consent-intercept-config.xml shibboleth.PostRequestAuditExtractors Extractors for an AuthnRequest built for proxied SAML SSO shibboleth.PostInboundResponseAuditExtractors Extractors for a Response from proxied SAML SSO shibboleth.PostInboundAssertionAuditExtractors Extractors for Assertions from proxied SAML SSO Each entry in these maps is a key containing the string label to associate (the name of the field), and a value containing a Function<ProfileRequestContext,Object> instance. Functions can return either single objects that get converted to a string for logging, or a Collection of objects that are converted to strings and separated by commas within the logged field. In this manner, you can add your own fields by implementing a function (or writing a script) to return any data you can find in the context tree and associating it with a custom field label by adding it to one of the maps named above. A similar extractor map exists for each login flow in the IdP, allowing different auditing behavior per-flow. All these beans have names based on the underlying login flow (e.g. shibboleth.authn.Password.AuditExtractors, etc.)
A skeletal example follows to demonstrate how to create a map to contain a custom field extraction rule: Adding custom fields to the audit log via audit.xml <!-- Adds a function to extract a custom field called MYFIELD after the final response has been generated. -->
<bean id="shibboleth.PostResponseAuditExtractors" parent="shibboleth.DefaultPostResponseAuditExtractors">
<property name="sourceMap">
<map merge="true">
<entry key="MYFIELD" value-ref="MyExtractionFunctionBean" />
</map>
</property>
</bean> A real world example: if you're logging fields containing a URL, you may need to escape characters in a URL because of the delimiter you choose to use in the log format. You can do this using a script: Encoding 'pipe' character appearing in CAS service URLs <bean id="shibboleth.CASValidationAuditExtractors" parent="shibboleth.DefaultCASValidationAuditExtractors" lazy-init="true">
<property name="sourceMap">
<map merge="true">
<entry>
<key>
<util:constant static-field="net.shibboleth.idp.cas.protocol.CASAuditFields.SERVICE_URL"/>
</key>
<bean parent="shibboleth.ContextFunctions.Scripted" factory-method="inlineScript">
<constructor-arg>
<value>
<![CDATA[
var serviceLookupFunctionClass = Java.type("net.shibboleth.idp.cas.audit.impl.ServiceLookupFunction");
var serviceLookupFunction = new serviceLookupFunctionClass();
var service = serviceLookupFunction.apply(input);
if (service != null) {
var serviceNew = service.replace(/\|/g, '%7C');
}
else {
var serviceNew = null;
}
serviceNew;
]]>
</value>
</constructor-arg>
</bean>
</entry>
</map>
</property>
</bean> A rather more convoluted example is to dump the values or arbitrary attributes into the log. To do this we define an abstract bean to emit the attribute values (in an arbitrary format) for any attribute (most of this is really error and null handling): Output Attribute Values <bean id="AttributeValueExtraction" parent="shibboleth.ContextFunctions.Scripted" factory-method="inlineScript" abstract="true">
<constructor-arg>
<value>
<![CDATA[
var getinfo = function() {
var rpContext = input.getSubcontext("net.shibboleth.profile.context.RelyingPartyContext");
if (null === rpContext) {
return null;
}
var attrContext = rpContext.getSubcontext("net.shibboleth.idp.attribute.context.AttributeContext");
if (null === attrContext) {
return null;
}
var attributes = null
attributes = attrContext.getUnfilteredIdPAttributes();
if (null === attributes) {
return null;
}
attribute = attributes.get(custom);
if (null === attribute || attribute.getValues().isEmpty()) {
return null;
}
var iter = attribute.getValues().iterator();
var result = "";
while (iter.hasNext()) {
result = result + " " + iter.next().toString();
}
return result;
}
getinfo();
]]>
</value>
</constructor-arg>
</bean> Then a bean to emit a specific attribute can be easily created: <bean id="EmailExtraction" parent="AttributeValueExtraction" p:customObject="mail" />
The generic fields that are supported out of the box are as follows (note that not every field is always populated; that depends on the timing of errors and the specific transaction being audited): Field Description ST Timestamp for start of flow T Timestamp event is recorded e WebFlow Event URL URL URI URI DEST Destination URL of outgoing msg s IdP session ID HASHEDs Hashed IdP session ID AF Authentication flow ID SSO SSO flag a Client address UA User agent string P Profile ID RPC 5.2 Bean ID of RelyingPartyConfiguration (default or override) u Username HASHEDu Hashed username uu Impersonating username attr Attribute(s) ROP Requested authentication operator RPRIN Requested authentication principals
The more SAML-oriented fields that are supported out of the box are as follows (note that not every field is always populated; that depends on the timing of errors and the specific transaction being audited): Field Description SP Service provider name IDP Identity provider name p Protocol b Inbound binding bb Outbound binding RS RelayState n NameID value f NameID format SPQ NameID SPNameQualifier pf NameIDPolicy required format PSPQ NameIDPolicy required SPNameQualifier i Assertion ID d Assertion timestamp I Inbound message ID D Inbound message timestamp II InResponseTo III Outbound message ID DD Outbound message timestamp t AuthenticationInstant x SessionIndex ac AuthenticationContext S Status code SS Sub-status code SM Status message pasv IsPassive fauth ForceAuthn SCC Scoping ProxyCount from an AuthnRequest SCI Scoping IdP list from an AuthnRequest SCR Scoping Requester ID(s) from an AuthnRequest PRC ProxyRestriction ProxyCount PRA ProxyRestriction Audiences XX Signed inbound messages X Encrypted assertions XA Encryption algorithm
A few fields are defined for auditing specific information during authentication; many are specific to particular login flows. Note that the Duo fields are for the built-in Duo flow; the newer plugin has a larger set of fields documented on the DuoOIDC configuration page. Field Description tu Transformed username submitted for validation (Password flow only) AR Results of authentication attempt, either “Success” or classified error strings CV Specific CredentialValidator bean used (Password flow only) X509S Certificate subject (X.509 flows only) X509I Certificate issuer (X.509 flows only) DuoCID Duo client identifier/integration key (Duo flow only) DuoF Duo factor (Duo flow only) DuoDID Duo device ID (Duo flow only)
In the output stage, a bean named shibboleth.AuditFormattingMap is used to write any number of log records containing any fields you configure to the logging API, at which point the Logback configuration takes over and decides how to send that data to particular log files or other log sinks. The bean is a map between logging categories and formatting strings. The consent audit log relies on its own properties and beans to provide an audit format, so refer to that documentation for those specifics. Formatting strings contain fields denoted with a '%' character followed by a field label, and any whitespace or punctuation terminates a field label and is included in the record directly. Using '%%' will output a single percent character. An additional bean, shibboleth.AuditSuppressedProfiles, contains a list of profile identifiers that should not result in audit log records. The default list is used to exempt the "status" handler from being logged. A partial list of profile IDs can be found in the MetadataDrivenConfiguration topic. Finally, when errors are handled "locally" (see ErrorHandlingConfiguration), the IdP can be told whether to output an audit log record or not when the error event occurs. This is useful to suppress logging noise when common errors occur that would lead to a lot of useless auditing. The flag controlling this is inside the shibboleth.LocalEventMap bean in conf/errors.xml (the value of each map entry is the flag indicating whether to log).
Reference
General beans defined in audit.xml, errors.xml, and related system configuration are: Bean ID Type Function shibboleth.AuditFormattingMap Map<String,String> Map of logging categories to audit formatting strings for general audit logging shibboleth.AuditDateTimeFormat String DateTimeFormat string to apply to DateTime fields (but does not apply to the logback-generated field at the beginning of the log line) shibboleth.AuditDefaultTimeZone Boolean Whether to convert DateTime fields into the machine's default time zone (UTC is used otherwise) shibboleth.AuditFieldReplacementMap Map<String,String> Map of replacement strings to substitute in when populating audit fields (simple way to shrink long constants down to size) shibboleth.AuditSuppressedProfiles List<String> List of profiles to skip auditing shibboleth.LocalEventMap Map<String,Boolean> Map of local error events to flags indicating whether to output an audit log record shibboleth.SuppressedEvents List<String> List of events that should not be logged as "errors" by the process log (generally leaving them to the audit log) shibboleth.DefaultSuppressedEvents List<String> Default list of events used in place of shibboleth.SuppressedEvents bean In addition, there are a set of standard beans containing the map of field extractors for all of the various "interception points" of the processing of the IdP. A pair of beans are reserved for each point, a default set that comes out of the box, and a defined bean ID to override or add to those defaults. Each of these beans is of type Map<String,Function<ProfileRequestContext,Object>> shibboleth.FlowStartAuditExtractors / shibboleth.DefaultFlowStartAuditExtractors shibboleth.PostDecodeAuditExtractors / shibboleth.DefaultPostDecodeAuditExtractors shibboleth.PostLookupAuditExtractors / shibboleth.DefaultPostLookupAuditExtractors shibboleth.PostAssertionAuditExtractors / shibboleth.DefaultPostAssertionAuditExtractors shibboleth.PostResponseAuditExtractors / shibboleth.DefaultPostResponseAuditExtractors shibboleth.PostRequestAuditExtractors / shibboleth.DefaultPostRequestAuditExtractors shibboleth.PostInboundResponseAuditExtractors / shibboleth.DefaultPostInboundResponseAuditExtractors shibboleth.PostInboundAssertionAuditExtractors / shibboleth.DefaultPostInboundAssertionAuditExtractors shibboleth.LogoutRequestAuditExtractors / shibboleth.DefaultLogoutRequestAuditExtractors shibboleth.LogoutAuditExtractors / shibboleth.DefaultLogoutAuditExtractors shibboleth.ErrorViewAuditExtractors / shibboleth.DefaultErrorViewAuditExtractors shibboleth.consent.PreConsentAuditExtractors / shibboleth.consent.DefaultPreConsentAuditExtractors shibboleth.consent.ConsentAuditExtractors / shibboleth.consent.DefaultConsentAuditExtractors An additional set of beans for each login flow are also defined: shibboleth.authn.Password.AuditExtractors / shibboleth.authn.Password.DefaultAuditExtractors etc.
Properties are defined in services.properties to customize various aspects of audit logging: Property Type Default Function idp.service.logging.saml1sso String SSO Suffix added to audit logging category when various profiles/flows are audited, you can use this to route different kinds of audit records to different destinations based on general function idp.service.logging.saml1attrquery String AttributeQuery idp.service.logging.saml1artifact String ArtifactResolution idp.service.logging.saml2sso String SSO idp.service.logging.saml2attrquery String AttributeQuery idp.service.logging.saml2artifact String ArtifactResolution idp.service.logging.saml2slo String Logout idp.service.logging.logout String Logout idp.service.logging.cas String SSO idp.service.logging.status String Status idp.service.logging.resolvertest String ResolverTest idp.service.logging.serviceReload String Reload idp.audit.shortenBindings Boolean false Allows simpler "short names" of SAML bindings and other (configurable) constants to appear in the audit log, instead of full URIs idp.audit.hashAlgorithm String SHA-256 Hash algorithm to apply to various hashed fields idp.audit.salt String Salt to apply to hashed fields, must be set to use those fields Additional properties are housed in authn/authn.properties to enable and customize authentication auditing. The <Flow> token in the various property names below is a stand-in for the actual login flow suffix (Password, External, RemoteUser, etc.), as these are per-flow properties. Property Type Default Description idp.authn.audit.enabled Boolean false Enables the authentication audit logging feature idp.authn.<Flow>.audit.enabled Boolean %{idp.authn.audit.enabled} Enables audit logging for a specific login flow idp.authn.<Flow>.audit.category String Shibboleth-Audit.<Flow> Log category for audit records for a specific login flow idp.authn.Password.audit.format String %a|%T|%SP|%I|%s|%AF|%CV|%u|%tu|%AR|%UA Audit formatting string for this login flow idp.authn.Duo.audit.format String %a|%T|%SP|%I|%s|%AF|%u|%AR|%DuoCID|%DuoF|%DuoDID|%UA Audit formatting string for this login flow idp.authn.External.audit.format String %a|%T|%SP|%I|%s|%AF|%u|%AR|%UA Audit formatting string for this login flow idp.authn.Function.audit.format String %a|%T|%SP|%I|%s|%AF|%u|%AR|%UA Audit formatting string for this login flow idp.authn.IPAddress.audit.format String %a|%T|%SP|%I|%s|%AF|%CV|%u|%AR|%UA Audit formatting string for this login flow idp.authn.RemoteUser.audit.format String %a|%T|%SP|%I|%s|%AF|%CV|%u|%AR|%UA Audit formatting string for this login flow idp.authn.RemoteUserInternal.audit.format String %a|%T|%SP|%I|%s|%AF|%CV|%u|%AR|%UA Audit formatting string for this login flow idp.authn.SPNEGO.audit.format String %a|%T|%SP|%I|%s|%AF|%u|%AR|%UA Audit formatting string for this login flow idp.authn.X509.audit.format String %a|%T|%SP|%I|%s|%AF|%X509S|%X509I|%AR|%UA Audit formatting string for this login flow idp.authn.X509Internal.audit.format String %a|%T|%SP|%I|%s|%AF|%X509S|%X509I|%AR|%UA Audit formatting string for this login flow