Using SAML Proxying in the Shibboleth IdP to connect with Azure AD


A French version of this document can be found on the CANARIE website.


Overview 

This configuration recipe of the Shibboleth IdP leverages the SAML proxying features such that Azure AD can be used for sign-in while Shibboleth handles the features many R&E federation's trust models value. Using this approach uses just the IdP as a SAML proxy without the necessity of extra elements.

We will cover two main proxying approaches :

  • a full pass-through proxy that uses only the attributes from Azure AD as its attribute set
  • a hybrid using  Azure AD attributes and other attributes resolved from LDAP, including on-premises Active Directory.

The steps can be applied to any IdP installation and assume that a v4.0.1 IdP platform has been installed.  The Internet2 Trusted Access Platform IdP container was used during configuration testing and helped take care of some of the heavy lifting during installation (ie, the webserver, version of java, and externalizing configuration). 

Expected Audience

The configuration steps assume some familiarity with the Shibboleth IdP, its layout, and being familiar with software installation and sustainment. The steps are platform neutral and should work with the Windows or Unix version of the IdP.

Where to Install

This configuration can be applied to an IdP on-premises (on-prem) or in the cloud. The location of the IdP may be influenced by the location of your data. Unless all your data is kept in Azure AD you will likely be implementing a hybrid approach for attributes from both Azure AD and normal Shibboleth IdP attribute resolution.


Acknowledgements: Thanks to contributors  Former user (Deleted)the Shibboleth team, and others for guidance and input on the material.

IdP Proxying: Appearances and Perspectives

Proxying techniques are confusing at times and it is helpful to understand the approach and your perspective:

Azure ADs:

  • Shibboleth V4 IdP will look like a Relying Party to Azure AD
  • It will be a single non-Gallery App registered in the Enterprise Apps portal for Azure
    • means that all conditional access and policy rules are applied at an IdP-wide level, NOT per Shibboleth Relying Party.
    • also means that applying rules for MFA as controlled by Azure AD are for the entire IdP and NOT discrete Shibboleth Relying Parties.

Relying Party (Service Provider) from the multi-lateral trust/R&E federation community:

  • It will look like a fully functional multi-lateral capable IdP 
  • It will not require any changes in behaviour to the SP
  • Attribute asserted over the wire are what the SP expects. i.e. attributes are NOT seen as 'claims' 

Users: 

  • A user will sign into Azure AD
  • The Shibboleth IdP as proxy will redirect as necessary
  • The user will have two sessions:
    • one with Azure AD
    • one with the Shibboleth IdP

Federation Operator:

  • The entity registered in the R&E federation is the Shibboleth IdP, built for and by the R&E community and should meet all technical requirements as does any other Shibboleth IdP
  • Alignment to any federation baseline expectations is not just software but also how it's operated and managed and is in the hands of the IdP operator like any other IdP. 

IdP Proxying: What a Proxy Flow Looks Like

When the user initiates sign-on the following usually happens:

    1. An Authentication request arrives at Shibboleth IdP
    2. The Shibboleth IdP observes it's configured for SAML sign on and redirects to an upstream IdP (Azure AD). Settings for this are controlled in

      1. idp.properties
      2. authn/saml-authn-config.xml 
    3. User Authenticates or has an existing session at upstream IdP (Azure AD)
    4. The upstream IdP (Azure AD) constructs a SAML Assertion with one or more attributes and is sent with the user to the Shibboleth IdP 
    5. The Shibboleth IdP receives the assertion and:
      1. verifies that the assertion arrives from an entity it trusts as configured in metadata-providers.xml

      2. filters the assertion according to rules in attribute-filter.xml
      3. Extracts the real user as configured in attribute-based subject c14n.
        1. Looks through list of AttributesToResolve in the resolver and resolves each one.
        2. Looks through list of resulting attributes in AttributeSourceIds and picks the first valid one to be the principal's name to be used later
      4. Resulting trusted real username is used as $resolutionContext.principal (eg. existing LDAP data connector, etc.) during "standard" attribute resolution process

Implementing the Solution

Prerequisites

Be sure to have these assets in place and appropriate access to them before you start:

  1. A working Shibboleth Identity Provider at V4 or above running somewhere.
  2. An active Azure AD tenant that you have administrative control in
  3. The ability in Azure AD to create an enterprise Non-Gallery SAML App
  4. A suitable attribute available to both IdPs to use as a "joining" attribute used in local lookups if necessary.

Terms and Settings 

This document refers to some key settings and terms described here. For your installation have these on hand to speed up configuration:

  • Your original IdP EntityID: https://idp.example.com/idp
  • Your Upstream IdP EntityID which is Azure AD Identifier: https://sts.windows.net/<sts_tenant_id>/
    • sts_tenant_id can be found most easily by creating a relying parting in Azure AD to work with:
      • sign into the Azure AD Portal -> choose Enterprise Applications -> Create New Application  > Non Gallery Application → Actually create one → select SAML Sign On  - > select SAML Sign  On settings
      • Section 4 contains  the Azure AD Identifier which includes the trailing slash
  • The Proxy Joining attribute common to both services:
    • SamAccountName is often a good choice and appears as 'name' in claims.
      • will exist as a 'scoped' username
      • the scope being that of the domain used on-premises and is usually internet friendly (not something.local or something.int)
  • Relying Party/Service Provider Microsoft will use "Relying Party" at times to mean "Service Provider"

Steps and Tasks

Step 1. Configuring Trust Between Azure AD and the Shibboleth IdP

There are 3 main trust-related configurations to adjust:

  • In the Shibboleth IdP:
    1. to have the SPSSODescriptor present in its metadata for when metadata for the IdP is fetched by URL
    2.  to have a metadata entry for the Azure AD IdP in its metadata providers' configuration
  • In Azure AD

3. the Azure AD trust to the Shibboleth IdP as a Relying Party

Trust Task: 1. Update your IdP's metadata

As your IdP will need act as an SP, you'll need extra blocks in your entity's metadata.

  • Update your existing idp-metadata.xml file to include a <SPSSODescriptor> block. 
  • Copy the signing and encryption certificates from the IdP part of the metadata and replace the base URI used below as an example (https://idp.example.com/idp) with the base of your IdP.

Trust Task: 2. Register your IdP with Azure AD

An Azure AD Enterprise Application needs to be created of type 'Non-Gallery Application' and configured for SAML.

Steps to create the app:

  • sign in to the Azure AD Portal > choose Enterprise Applications > Create New Application > Non Gallery Application > Actually create one > select SAML Sign On > select SAML Sign  On settings

Once created edit the Basic SAML Configuration Section where you will use two things:

  • the EntityID: https://idp.example.com/idp/shibboleth (same as for your IdP )
  • the Assertion Consumer Service (ACS) URL: https://idp.example.com/idp/profile/Authn/SAML2/POST/SSO

Other items like Logout URL can also be provided but the Shibboleth side will not handle logout in this case, so this is best omitted.


Keep this Relying Party editing environment open for the next step.

Trust Task: 3. Register the upstream IdP's metadata locally

Your Shibboleth IdP doesn't know of the Azure AD IdP so you need to register it locally. This is handled just like any other metadata and can be included in your metadata-providers.xml as a new <MetadataProvider> element. For example:

The file AzureAD-idp.xml must contain valid SAML 2. Metadata for your tenant's entity containing:

  • an IDPSSODescriptor block containing a hand entered shibmd:scope element
  • an X509Descriptor element with use=signing
  • 2 SingleSignOnService elements

  • 1 SingleLogOut  element (not really supported but will allow error responses to be returned to Azure)

This can be most easily found and downloaded by clicking on 'Federation Metadata XML Download' in Section 3 of the Azure AD Relying Party information, however it needs to be scrubbed a bit to produce a lean version of the metadata.

Hint: Those on mac or unix may find it convenient processing the file through xmllint --format downloadedfile.xml > downloadedfile-formated.xml
(xmllint
can be found in a package named libxml2 on RedHat or libxml2-utils on Debian/Ubuntu.)

Hand-adjust the downloaded metadata to add both the XML NameSpace and a value for shibmd:Scope in the IDPSSODescriptor element:

  1. Add to the EntityDescriptor in your downloaded metadata: xmlns:shibmd="urn:mace:shibboleth:metadata:1.0
  2. add the Extensions element inside the IDPSSODescriptor element to align with the scope validation per the rule in Proxy Task 2 below:

When done, add an entry to metadata-providers.xml.  If you fetch this entry elsewhere or by other means (e.g. curl or otherwise)  ensure you know and understand its pedigree and don't blindly trust it without some diligence of some sort.


Trust Task: 4. Configure Azure AD Attribute Release to the Shibboleth IdP

While still in the edit mode of the Relying Party, configure the necessary attributes needed for the Shibboleth IdP operation.

  • For a pass-through proxy approach
    • All attributes originate from Azure AD and must be released to this Relying Party for the Shibboleth IdP to marshall downstream
  • For a hybrid proxy approach
    • Technically the single attribute you want to use for user lookup is required.


Step 2. Configure the IdP for Proxing Behaviour

The following changes are needed to adjust the IdP to proxying behaviour:

  1. Changing the authentication flow to SAML authentication
  2. Configuring which Entity to delegate to for the flow
  3. Update attribute filter to allow incoming attributes to be ingested
  4. Set up attribute extraction through Subject Canonicalisation (c14n and resolver)

Proxy Task 1. Change the IdP authentication flow to SAML

There are three steps for this task:

  1. Edit authn/saml-authn-config.xml to set the EntityID of the Azure AD Entity:
    1. Uncomment the shibboleth.authn.SAML.discoveryFunction bean and edit the target:
    2. REPLACE sts_tenant_id with your identifier. NOTE THE TRAILING SLASH IS TO BE INCLUDED

2. Enable the flow by updating idp.authn.flows in idp.properties to set it to SAML:

idp.authn.flows=SAML

Proxy Task 2. Update Your Attribute Filter

The IdP will not ingest attributes from the Azure AD Upstream IdP unless they're allowed in by a filter.

  1. Add a new <AttributeFilterPolicy> to permit trusted attributes to be handled.
  2. REPLACE sts_tenant_id with your identifier. NOTE THE TRAILING SLASH TO BE INCLUDED

Note:  There is a policy rule ScopeMatchesShibMDScope in effect on the attribute 'name' (known internally as azureName).

This will enforce that the idp.scope set in conf/idp.properties agrees with the Azure AD field to avoid any oddities. If they disagree, this rule will fail-closed and not sign a user.

Be sure the IdP metadata for the your Azure AD IdP has the right scope for things to work smoothly

Proxy Task 3. Enable IdP to Recognize Azure AD Claims 

Azure AD issues SAML assertions, however they are presented in an Azure/WS-Fed-centric naming convention (as "claims").

Our technique is to add in a new attribute mapping file that we can then use to parse the claims into internal attributes and in turn use them in the attribute resolver.

  1. Add a new file called in the IdP as attributes/azureClaims.xml if it doesn't already exist. This file contains each of your attributes from Azure AD that you want to use in the resolver.

The sample below is not exhaustive but has enough to get started on the claims file.

  • For azName we apply a scoped rule for safety considerations. The scope value MUST match the <shibmd:Scope> extension in the upstream IdP metadata.
  • The "saml2.nameFormat" rule property is urn:oasis:names:tc:SAML:2.0:attrname-format:unspecified as the claims don't carry the proper format on them.



2. Update attributes/default-rules.xml to include this new file as a reference by adding this line to it after the last import:

3. Update attribute-resolver.xml to reveal our new attributes from Azure AD by adding a new DataConnector:

Now that we have the the IdP able to interpret claims from Azure AD, we need to turn our attention to using them for user canonicalization and for populating our attributes.

Proxy Task 4. Subject Canonicalisation

This workflow takes the incoming assertion and extracts some data from it to work out the canonical (authoritative/normalized) username of the user logging in from Azure.

For AzureAD we are going to use the attribute claim: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name which we just mapped internally in the IdP to azName

1. Add a new <AttributeDefinition> to attribute-resolver.xml as shown below.

This will transform the azName attribute from the Subject (the filtered data from the incoming assertion) into canonicalNameToUseForJoin attribute for use elsewhere:

2. Now configure which attributes to resolve during c14n:

Define the shibboleth.c14n.attribute.AttributesToResolve and shibboleth.c14n.attribute.AttributeSourceIds beans in c14n/attribute-sourced-subject-c14n-config.xml to the attribute ID of the above <AttributeDefinition>:

In other proxy situations this may have more than one attribute but for Azure AD we are focused on a single one.

3. Review and ensure the bean in c14n/subject-c14n.xml has been uncommented:

At this point you should have a valid, locally understood username to pass into the "normal" resolver that already works, which is now available from places like:

$resolutionContext.principal

Proxy Task 5. Configuring Attribute pass-through and/or hybrid resolving

Attributes originating from Azure AD are referenced in attribute-resolver.xml like this:

Be sure to comment out the old definition of the names in the attribute-resolver.xml or you may get unexpected results.

Proxy Task 6. Handling REFEDS AuthnContext Requests (optional)

Azure does not currently have a documented way to influence the behavior of the AuthnContext in their SAML assertions.  However, Shibboleth provides the means to translate proxy requests and responses via authn/authn-comparison.xml. The example below handles REFEDS MFA requests:

Update the support matrix for the SAML authentication flow to understand the REFEDS MFA profile

(for v4.0.1) Update the authn/SAML bean in authn/general-authn.xml so it understands the REFEDS MFA profile by adding a supportedPrinciples property:

Changes required in v4.1 may be different and you should look at the authn.properties file. More details here: SAMLAuthnConfiguration

Testing

Restart your IdP for your changes to take effect.  Because this is a SAML proxy configuration it doesn't make sense to use  aacli since it won't have the required information available to it.

Some sites have a developer account which has a tenant that can test entirely isolated from the production tenant. Alternatively you can create a different Azure AD non-gallery SAML app per IdP – one for production, one for pre-production etc. There's no hard and fast rule on the best way however the developer route is free and provides a great model for isolation between credentials as it's a different cookie domain entirely.

Related content / Recommended Reading