The Shibboleth V1 software has reached its End of Life and is no longer supported. This documentation is available for historical purposes only.

MetaSearch

Owned by Scott Cantor
Last updated: Dec 11, 2008Version comment
6 min read

Shibboleth is often used and discussed in library access management scenarios, and one of the more complex use cases is MetaSearch, initiating searches across a number of repositories, databases, or other information services from an intermediary. This isn't all that complex until security is included and the following goals are to be accomplished:

  • Expressing identity and/or attributes of the initiating user to the back-end services without requiring those services to trust the intermediary for the accuracy of this information.
  • Expressing identity and/or attributes of the initiating user to the back-end services without exposing the information to the intermediary.
  • Avoiding the creation of a correlation handle such that back-end services can trivially collude to recognize that a set of searches performed by the intermediary are associated with the same user (and thus aggregate whatever information they received).

To accomplish this, we have the following possible sequence:

  1. Browser authenticates to MetaSearch intermediary service
  2. User initiates search.
  3. Intermediary requests new security tokens from user's IdP
  4. Intermediary invokes back-end services, attaching security token.
  5. (Optional) Back-end service issues its own security token to substitute for the original.

Browser Authentication to Intermediary

In step 1, initial authentication to the intermediary service takes place. This step can be achieved with any web browser authentication profile resulting in a SAML 2.0 assertion presented to the SP, such as the SAML 2.0 Browser SSO or EnhancedClient profiles.

The additional requirement is for the user to delegate permission to the SP to make additional token requests on his/her behalf. This should be a fairly innocuous policy to enable, as it doesn't in and of itself provide the SP with the ability to obtain any particular access. To reflect this capability, the !IdP is responsible for embedding additional SubjectConfirmation and Audience elements authorizing the SP to present the assertion back to the IdP in step 3.

Example

The example shows an "exchangeable" token issued to https://metasearch.serviceprovider.com/entity by https://idp.example.edu/entity , valid for an hour.

<saml:Assertion xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" Version="2.0"
		  ID="_682C46C8-198A-436C-9E0F-DBBC155DE413" IssueInstant="2006-01-27T02:28:36Z">
	 <saml:Issuer>https://idp.example.edu/entity</saml:Issuer>
	 <ds:Signature>...</ds:Signature>
	 <saml:Subject>

		  <!-- the identifier is scoped between the !IdP and the SP -->
		  <saml:NameID Format="urn:oasis:names:tc:SAML:2.0:nameid-format:transient">
			 E8042FB4-4D5B-48C3-8E14-8EDD852790DD
		  </saml:NameID>

		  <!-- the bearer authorization is for web SSO by the browser to the SP -->
		  <saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
				<saml:SubjectConfirmationData Address="192.168.1.1"
					 Recipient="https://metasearch.serviceprovider.com/Shibboleth.sso/SAML2/ACS"
					 NotBefore="2006-01-27T02:28:36Z" NotOnOrAfter="2006-01-27T02:33:36Z"/>
		  </saml:SubjectConfirmation>

		  <!-- the HoK authorization is for re-presentation to the !IdP by the SP -->
		  <saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:holder-of-key">
				<saml:NameID Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">
				  https://metasearch.serviceprovider.com/entity
				</saml:NameID>
				<saml:SubjectConfirmationData xsi:type="saml:KeyInfoConfirmationDataType">
					 <ds:KeyInfo>...<ds:KeyInfo>
				</saml:SubjectConfirmationData>
		  </saml:SubjectConfirmation>

	 </saml:Subject>

	 <!-- the conditions apply to all uses, and the assertion is scoped to both the !IdP and SP -->
	 <saml:Conditions NotBefore="2006-01-27T02:28:36Z" NotOnOrAfter="2006-01-27T03:28:36Z">
		  <saml:AudienceRestriction>
				<saml:Audience>https://metasearch.serviceprovider.com/entity</saml:Audience>
				<saml:Audience>https://idp.example.edu/entity</saml:Audience>
		  </saml:AudienceRestriction>
	 </saml:Conditions>

	 <saml:AuthnStatement AuthnInstant="2006-01-27T02:10:00Z"
				SessionIndex="_682C46C8-198A-436C-9E0F-DBBC155DE413">
		  <saml:SubjectLocality Address="192.168.1.1"/>
		  <saml:AuthnContext>
				<saml:AuthnContextClassRef>
				  urn:oasis:names:tc:SAML:2.0:ac:classes:Kerberos
				<saml:AuthnContextClassRef>
				<saml:AuthnContextDeclRef>
				  https://idp.example.edu/SAML2/AC/Kerberos.xml
				<saml:AuthnContextDeclRef>
		  </saml:AuthnContext>
	 </saml:AuthnStatement>

	 <saml:AttributeStatement>
	 ...
	 </saml:AttributeStatement>
</saml:Assertion>

User initiates search.

Obviously this step is out of scope, but it's mentioned to highlight the fact that it's a requirement for step 3 that the intermediary service identify the back-end services for which it needs to acquire credentials; that step can be made more efficient if a single request for credentials can be made.

Intermediary Requests New Security Tokens

To request the credentials it needs, the intermediary SP exchanges the original assertion issued to it in step 1 for one or more new assertions using the SAML Token Service profile defined in the LibertyWSFAuthentication spec. This takes the form of a SOAP request to the !IdP's SSO Service endpoint containing a SAML AuthnRequest message. The original assertion is attached to the signed request using the LibertyWSFSOAPBinding.

The AuthnRequest identifies the back-end services for which credentials are required. It can optionally specify a variety of additional policy information, but in most cases can be left fairly simple. The !IdP should usually have enough pre-established policy or can rely on a number of mechanisms (out of scope) to determine the security policies.

Example

The example shows the token issued in step 1 used to secure a request for new tokens issued by https://idp.example.edu/entity for securing calls to https://dspace.osu.edu/entity and
https://resources.publisher.com/entity

In the example, a single request results in two separate assertions. If correlation is permissible, than issuing a single assertion is simpler.

First, the SOAP request:

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">

	 <S:Header xmlns:wsa="http://www.w3.org/2005/08/addressing"
		  xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
		  xmlns:sbf="urn:liberty:sb"
		  xmlns:sb="urn:liberty:sb:2005-11">

		  <sbf:Framework version="2.0"/>
		  <sb:Sender wsu:Id="sender" providerID="https://metasearch.serviceprovider.com/entity"/>

		  <wsa:MessageID wsu:Id="mid">uuid:efefefef-aaaa-ffff-cccc-eeeeffffbbbb</wsa:MessageID>
		  <wsa:To wsu:Id="to">https://idp.example.edu/entity</wsa:To>
		  <wsa:Action wsu:Id="action">urn:liberty:ssos:2005-11:AuthnRequest</wsa:Action>
		  <wsa:ReplyTo wsu:Id="replyto">
				<wsa:Address>http://www.w3.org/2005/08/addressing/role/anonymous</wsa:Address>
		  </wsa:ReplyTo>

		  <wsse:Security S:mustUnderstand="1"
				xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">

				<wsu:Timestamp wsu:Id="ts">
					 <wsu:Created>2006-01-27T02:48:00Z</wsu:Created>
				</wsu:Timestamp>

				<!-- this is the assertion from step 1 -->
				<saml:Assertion xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" Version="2.0"
					 ID="_682C46C8-198A-436C-9E0F-DBBC155DE413" IssueInstant="2006-01-27T02:28:36Z">
					 ....
				</saml:Assertion>

				<!-- the signature is created by the sender to sign the attached token into the message -->
				<ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
				  <ds:SignedInfo>
					 <!-- include a ds:Reference for each header added according to SOAP binding -->
					 <ds:Reference URI="#mid">...</ds:Reference>
					 <ds:Reference URI="#to">...</ds:Reference>
					 <ds:Reference URI="#action">...</ds:Reference>
					 <ds:Reference URI="#replyto">...</ds:Reference>
					 <ds:Reference URI="#ts">...</ds:Reference>
					 <ds:Reference URI="#sender">...</ds:Reference>

					 <!-- include the SAML Assertion in the signature to avoid token substitution attacks -->
					 <ds:Reference URI="#_682C46C8-198A-436C-9E0F-DBBC155DE413">...</ds:Reference>

					 <!-- include the message body -->
					 <ds:Reference URI="#MsgBody">...</ds:Reference>
				  </ds:SignedInfo>
				  <ds:SignatureValue>
					 HJJWbvqW9E84vJVQkjjLLA6nNvBX7mY00TZhwBdFNDElgscSXZ5Ekw==
				  </ds:SignatureValue>

				  <ds:KeyInfo>
					 <wsse:SecurityTokenReference
						xmlns:wsse11="http://docs.oasis-open.org/wss/oasis-wsswssecurity-secext-1.1.xsd"
						wsse11:TokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-tokenprofile-1.1#SAMLV2.0">
						  <wsse:KeyIdentifier
							 ValueType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLID">
								_682C46C8-198A-436C-9E0F-DBBC155DE413
						  </wsse:KeyIdentifier>
					 </wsse:SecurityTokenReference>
				  </ds:KeyInfo>
				</ds:Signature>

		  </wsse:Security>
	 </S:Header>

	 <S:Body wsu:Id="MsgBody">
		  <samlp:AuthnRequest xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"
					 xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" Version="2.0"
					 ID="_E47945FB-C3F3-4f2C-991C-A2E2B2C59F3C" IssueInstant="2006-01-27T02:48:00Z"
					 ProtocolBinding="http://www.w3.org/2005/08/addressing/role/anonymous">

				<!--
				saml:Issuer omitted and ProtocolBinding set above in accordance with Liberty SSOS spec;
				lack of saml:Subject or samlp:NameIDPolicy signal the IdP to default the assertion content
				-->

				<!-- the conditions identify the back-end services we're asking to access -->
				<saml:Conditions>
					 <saml:AudienceRestriction>
						  <saml:Audience>https://dspace.osu.edu/entity</saml:Audience>
						  <saml:Audience>https://resources.publisher.com/entity</saml:Audience>
					 </saml:AudienceRestriction>
				</saml:Conditions>

		  </samlp:AuthnRequest>
	 </S:Body>
</S:Envelope>

Then, the SOAP response:

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">

	 <S:Header xmlns:wsa="http://www.w3.org/2005/08/addressing"
		  xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
		  xmlns:sbf="urn:liberty:sb"
		  xmlns:sb="urn:liberty:sb:2005-11">

		  <sbf:Framework version="2.0"/>
		  <sb:Sender providerID="https://idp.example.edu/entity"/>

		  <wsa:MessageID>uuid:071BCD36-FE77-470D-9AA9-9B5628D08727</wsa:MessageID>
		  <wsa:RelatesTo>uuid:efefefef-aaaa-ffff-cccc-eeeeffffbbbb</wsa:RelatesTo>
		  <wsa:To>https://metasearch.serviceprovider.com/entity</wsa:To>
		  <wsa:Action>urn:liberty:ssos:2005-11:Response</wsa:Action>

		  <wsse:Security>
				<wsu:Timestamp>
					 <wsu:Created>2006-01-27T02:49:00Z</wsu:Created>
				</wsu:Timestamp>
		  </wsse:Security>
	 </S:Header>

	 <S:Body>
		  <samlp:Response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"
					 xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
					 xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"
					 xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
					 Version="2.0" ID="_01CA983A-A889-45AD-BB5C-08DEA1041EE0"
					 IssueInstant="2006-01-27T02:49:00Z">

				<!-- saml:Issuer omitted in accordance with Liberty SSOS spec -->
				
				<samlp:Status>
					 <samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success"/>
				</samlp:Status>

				<saml:Assertion Version="2.0" ID="_B6EE95E8-54A1-48EB-9A24-68868419FCCB"
						  IssueInstant="2006-01-27T02:49:00Z">
					 <saml:Issuer>https://idp.example.edu/entity</saml:Issuer>
					 <ds:Signature>...</ds:Signature>
					 <saml:Subject>

						  <!-- the identifier is scoped between the !IdP and the back-end SP -->
						  <saml:EncryptedID>
								<xenc:EncryptedData>...</xenc:EncryptedData>
						  </saml:EncryptedID>

						  <!-- the HoK authorization is for presentation by the SP to the back-end SP -->
						  <saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:holder-of-key">
								<saml:NameID Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">
								  https://metasearch.serviceprovider.com/entity
								</saml:NameID>
								<saml:SubjectConfirmationData xsi:type="saml:KeyInfoConfirmationDataType">
									 <ds:KeyInfo>...<ds:KeyInfo>
								</saml:SubjectConfirmationData>
						  </saml:SubjectConfirmation>

					 </saml:Subject>

					 <saml:Conditions NotBefore="2006-01-27T02:49:00Z" NotOnOrAfter="2006-01-27T03:49:00Z">
						  <saml:AudienceRestriction>
								<saml:Audience>https://dspace.osu.edu/entity</saml:Audience>
						  </saml:AudienceRestriction>
					 </saml:Conditions>

					 <saml:AuthnStatement>
						  <!-- roughly same information as original assertion -->
						  <!-- note that a high granularity AuthnInstant is a correlation problem -->
					 </saml:AuthnStatement>

					 <saml:AttributeStatement>
						  <!-- encrypyed attributes tailored to back-end SP -->
					 </saml:AttributeStatement>
				</saml:Assertion>

				<saml:Assertion Version="2.0" ID="_74B67F56-9212-4131-9B84-B088C2B0B5AF"
						  IssueInstant="2006-01-27T02:49:00Z">
					 <saml:Issuer>https://idp.example.edu/entity</saml:Issuer>
					 <ds:Signature>...</ds:Signature>
					 <saml:Subject>

						  <!-- the identifier is scoped between the !IdP and the back-end SP -->
						  <saml:EncryptedID>
								<xenc:EncryptedData>...</xenc:EncryptedData>
						  </saml:EncryptedID>

						  <!-- the HoK authorization is for presentation by the SP to the back-end SP -->
						  <saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:holder-of-key">
								<saml:NameID Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">
								  https://metasearch.serviceprovider.com/entity
								</saml:NameID>
								<saml:SubjectConfirmationData xsi:type="saml:KeyInfoConfirmationDataType">
									 <ds:KeyInfo>...<ds:KeyInfo>
								</saml:SubjectConfirmationData>
						  </saml:SubjectConfirmation>

					 </saml:Subject>

					 <saml:Conditions NotBefore="2006-01-27T02:49:00Z" NotOnOrAfter="2006-01-27T03:49:00Z">
						  <saml:AudienceRestriction>
								<saml:Audience>https://resources.publisher.com/entity</saml:Audience>
						  </saml:AudienceRestriction>
					 </saml:Conditions>

					 <saml:AuthnStatement>
						  <!-- roughly same information as original assertion -->
						  <!-- note that a high granularity AuthnInstant is a correlation problem -->
					 </saml:AuthnStatement>

					 <saml:AttributeStatement>
						  <!-- encrypted attributes tailored to back-end SP -->
					 </saml:AttributeStatement>
				</saml:Assertion>

		  </samlp:Response>
	 </S:Body>
</S:Envelope>

Intermediary Invokes Back-End Services

Technically, we haven't had to define the application protocol between the intermediary and back-end until this step. The preceding steps could apply to any application protocol that permits the intermediary to attach the resulting assertions to the application message. It is the job of additional SAML profiles to define how this is done.

For SOAP services, the LibertyWSFSOAPBinding can be applied directly, in the same fashion as in step 3.

Example

The example shows an application request message issued by https://metasearch.serviceprovider.com/entity to
https://resources.publisher.com/entity 

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">

	 <S:Header xmlns:wsa="http://www.w3.org/2005/08/addressing"
		  xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
		  xmlns:sbf="urn:liberty:sb"
		  xmlns:sb="urn:liberty:sb:2005-11">

		  <sbf:Framework version="2.0"/>
		  <sb:Sender wsu:Id="sender" providerID="https://metasearch.serviceprovider.com/entity"/>

		  <wsa:MessageID wsu:Id="mid">uuid:01493C23-A6F2-47A4-98BF-D9C54189ADC2</wsa:MessageID>
		  <wsa:To wsu:Id="to">https://resources.publisher.com/entity</wsa:To>
		  <wsa:Action wsu:Id="action">...</wsa:Action>
		  <wsa:ReplyTo wsu:Id="replyto">
				<wsa:Address>http://www.w3.org/2005/08/addressing/role/anonymous</wsa:Address>
		  </wsa:ReplyTo>

		  <wsse:Security S:mustUnderstand="1"
				xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">

				<wsu:Timestamp wsu:Id="ts">
					 <wsu:Created>2006-01-27T03:00:00Z</wsu:Created>
				</wsu:Timestamp>

				<!-- this is an assertion from step 3 -->
				<saml:Assertion xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" Version="2.0"
					 ID="_74B67F56-9212-4131-9B84-B088C2B0B5AF" IssueInstant="2006-01-27T02:49:00Z">
					 ....
				</saml:Assertion>

				<!-- the signature is created by the sender to sign the attached token into the message -->
				<ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
				  <ds:SignedInfo>
					 <!-- include a ds:Reference for each header added according to SOAP binding -->
					 <ds:Reference URI="#mid">...</ds:Reference>
					 <ds:Reference URI="#to">...</ds:Reference>
					 <ds:Reference URI="#action">...</ds:Reference>
					 <ds:Reference URI="#replyto">...</ds:Reference>
					 <ds:Reference URI="#ts">...</ds:Reference>
					 <ds:Reference URI="#sender">...</ds:Reference>

					 <!-- include the SAML Assertion in the signature to avoid token substitution attacks -->
					 <ds:Reference URI="#_74B67F56-9212-4131-9B84-B088C2B0B5AF">...</ds:Reference>

					 <!-- include the message body -->
					 <ds:Reference URI="#MsgBody">...</ds:Reference>
				  </ds:SignedInfo>
				  <ds:SignatureValue>
					 HJJWbvqW9E84vJVQkjjLLA6nNvBX7mY00TZhwBdFNDElgscSXZ5Ekw==
				  </ds:SignatureValue>

				  <ds:KeyInfo>
					 <wsse:SecurityTokenReference
						xmlns:wsse11="http://docs.oasis-open.org/wss/oasis-wsswssecurity-secext-1.1.xsd"
						wsse11:TokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-tokenprofile-1.1#SAMLV2.0">
						  <wsse:KeyIdentifier
							 ValueType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLID">
								_74B67F56-9212-4131-9B84-B088C2B0B5AF
						  </wsse:KeyIdentifier>
					 </wsse:SecurityTokenReference>
				  </ds:KeyInfo>
				</ds:Signature>

		  </wsse:Security>
	 </S:Header>

	 <S:Body wsu:Id="MsgBody">
	 ...application specific content...
	 </S:Body>
</S:Envelope>

Back-End Substitutes Security Token

As an optional step, the back-end service might choose to offer a lighter-weight token to the intermediary if the application requires multiple SOAP exchanges. This effectively establishes a security context between the two servers based on the original assertion, just like web servers usually drop cookies in response to an assertion presented by the browser.

The difference is that here, the reason is solely efficiency. Browsers can't continually replay an assertion to a web server, but SOAP clients can certainly reuse assertions within their validity period.

The LibertyWSFSOAPBinding defines a profile of WS-Addressing that performs a so-called "endpoint update" operation that tells the client to substitute a new token for the original one it used on future calls to the service on behalf of the same user. This is typically session-oriented, and the client typically understands that once it's completed a task for that user, it can dispose of the token.

Example

The example shows an application response message issued by https://resources.publisher.com/entity to https://metasearch.serviceprovider.com/entity that contains the token replacement directive.

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">

	 <S:Header xmlns:wsa="http://www.w3.org/2005/08/addressing"
		  xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
		  xmlns:sbf="urn:liberty:sb"
		  xmlns:sb="urn:liberty:sb:2005-11">

		  <sbf:Framework version="2.0"/>
		  <sb:Sender providerID="https://resources.publisher.com/entity"/>

		  <wsa:MessageID>uuid:F84A4383-4B84-4471-B6C7-84F9BC2B7703</wsa:MessageID>
		  <wsa:RelatesTo>uuid:01493C23-A6F2-47A4-98BF-D9C54189ADC2</wsa:RelatesTo>
		  <wsa:To>https://metasearch.serviceprovider.com/entity</wsa:To>
		  <wsa:Action>...</wsa:Action>

		  <wsse:Security>
				<wsu:Timestamp>
					 <wsu:Created>2006-01-27T03:01:00Z</wsu:Created>
				</wsu:Timestamp>
		  </wsse:Security>

		  <!-- this updates the security token associated with this "session" -->
		  <sb:EndpointUpdate S:mustUnderstand="1" updateType="urn:liberty:sb:2004-04:Partial">
				<wsa:Address>urn:liberty:sb:2005-11:EndpointUpdate:NoChange</wsa:Address>
				<wsa:Metadata>
					 <disco:SecurityContext xmlns:disco="urn:liberty:disco:2005-11">
						  <disco:SecurityMechID>urn:liberty:security:2005-02:TLS:Bearer</disco:SecurityMechID>
						  <sec:Token xmlns:sec="urn:liberty:security:2005-11"
									 usage="urn:liberty:security:tokenusage:2005-11:SecurityToken">
								<wsse:SecurityTokenReference>
								  <wsse:Embedded>
									 <wsse:BinarySecurityToken wsu:Id="..." ValueType="app:ServiceSessionContext">
										  ZjgzOWZlNzgyZTk1ZWU3OWEyMTRlODVmNGZkYzE4MmQ2ZDNhMzc3Nwo=
									 </wsse:BinarySecurityToken>
								  </wsse:Embedded>
								</wsse:SecurityTokenReference>
						  </sec:Token>
					 </disco:SecurityContext>
				</wsa:Metadata>
		  </sb:EnpointUpdate>
	 </S:Header>

	 <S:Body>
	 ...application specific content...
	 </S:Body>
</S:Envelope>