The Shibboleth IdP V3 software has reached its End of Life and is no longer supported. This documentation is available for historical purposes only. See the IDP4 wiki space for current documentation on the supported version.

HTTPConnector

This feature is available with V3.4 of the software.

The HTTP data connector generates multiple attributes resulting from calling a web service. The connector itself is primarily just a framework for constructing a request to issue using the Apache HttpClient library and processing a response. There is no attempt to standardize a web service interface, but the design is geared towards making simply, templated GET requests and processing XML or JSON result sets using a script. More advanced use cases can be supported by means of pluggable Java interfaces.

Schema Name and Location

This xsi:type is defined by the urn:mace:shibboleth:2.0:resolver schema, located at http://shibboleth.net/schema/idp/shibboleth-attribute-resolver.xsd

General Configuration

The connector marries three essential pieces of configuration to be supplied by the deployer:

  • an HttpClient bean and various security and networking parameters necessary to invoke the web service safely
  • request creation
  • response processing

Two options for request creation exist, one for simple GET requests and one for more complex POST requests with a body. Both rely on a <URLTemplate> element to produce the URL, which can be populated with dependency data much like an LDAP search filter or database query. The POST option also supports a <BodyTemplate> element that can similarly generate a request body based on dependency data, typically JSON or XML. With POST, you also can control caching of results (if a results cache is used) by generating a cache entry key using a <CacheKeyTemplate> element.

The only supplied implementation for response processing is script-based, using the <ResponseMapping> element to supply a script to produce attribute data from the response body. For efficiency, the implementation assumes that the response processing script can consume the results in real time, and leaves any buffering of the data to the script, should that be necessary. Support is built-in for limiting the size of the data, allowed content types, and HTTP status codes accepted, to address the most common sanity checks.

At minimum, an httpClientRef attribute must be supplied to provide the client runtime bean to use. This will usually if not always be accompanied by an httpClientSecurityParametersRef attribute to supply security settings, although a few shortcut settings are available for certificate authentication use cases. A complete summary and examples can be found on the HttpClientConfiguration page.

Spring beans may be defined either in additionally loaded Spring resources configured into the Attribute Resolver service's resource collection, or in a central location such as global.xml

Caching Behavior

The built-in caching support (if enabled) keys the cached data for a GET request by the fully populated template URL generated and requested, typically including information about the subject in the URL. For POST requests, you may provide a template for producing an appropriate cache key, or omit it to disable caching.

Customized implementations of the ExecutableSearchBuilder<HTTPSearch> interface MUST implement appropriate cache key semantics, particularly if they inherit from that base class.

Examples

Any examples here omit the Spring beans that define the HTTP client and security parameters to use. Examples of these can be found in the HttpClientConfiguration topic.

HTTP connector for a JSON-based web service
 <DataConnector id="myHTTP" xsi:type="HTTP"
		httpClientRef="WebServiceHttpClient"
		httpClientSecurityParametersRef="ExampleOrgWSSecurity">
	<URLTemplate>
		<![CDATA[
		https://webservice.example.org/api/subject/$pathEscaper.escape($resolutionContext.principal)/groups
		]]>
	</URLTemplate>

	<ResponseMapping>
		<Script>
		<![CDATA[
		var HashSet = Java.type("java.util.HashSet");
		var HttpClientSupport = Java.type("net.shibboleth.utilities.java.support.httpclient.HttpClientSupport");
		var IdPAttribute = Java.type("net.shibboleth.idp.attribute.IdPAttribute");
		var StringAttributeValue = Java.type("net.shibboleth.idp.attribute.StringAttributeValue");
  
		// Limits length to 64k
		var body = HttpClientSupport.toString(response.getEntity(), "UTF-8", 65536);
		var result = JSON.parse(body);
 
		var attr = new IdPAttribute("group");
		var values = new HashSet();
		if (result.groups != null) {
			for (var i=0; i<result.groups.length; i++) {
				values.add(new StringAttributeValue(result.groups[i].name));
			}
		}
		attr.setValues(values);
		connectorResults.add(attr);
		]]>
		</Script>
	</ResponseMapping>

	<ResultCache expireAfterWrite="PT5M"/>
</DataConnector>

Configuration Reference

Attributes

Any of the common attributes can be specified. In addition the following attributes may be specified (the only required attribute is httpClientRef).

NameTypeDefaultDescription
httpClientRefBean ID
Bean ID of the HttpClient instance to use
httpClientSecurityParametersRefBean ID
Bean ID of the HttpClientSecurityParameters instance to use, ignored if one of the security shortcut settings are used
serverCertificateResource path
Path of resource containing a server certificate whose public key must match the server's. If set, httpClientSecurityParametersRef is ignored.
certificateAuthorityResource path
Path of resource containing a certificate authority used to validate the server's certificate. If set, httpClientSecurityParametersRef is ignored.
clientPrivateKeyResource path
Path of resource containing a private key used to authenticate the client to the server via TLS. If set, httpClientSecurityParametersRef is ignored.
clientCertificateResource path
Path of resource containing a certificate used to authenticate the client to the server via TLS. If set, httpClientSecurityParametersRef is ignored.
maxLengthLong0Limits size of response body to accept, or 0 for no limit. When used, only  non-chunked responses that include a content length will be accepted.
acceptStatusesCollection<Integer>200Acceptable HTTP status codes
acceptTypesCollection<String>
Acceptable MIME content types
headerMapRefMap<String,String>
Bean ID of a map of custom header names and values to set in the HTTP request

mappingStrategyRef

Bean ID
Bean ID of a HTTPResponseMappingStrategy to process the result set in a pluggable way
validatorRef
Bean ID
Bean ID of a Validator to control what constitutes an initialization failure (the default does no validation)
executableSearchBuilderRefBean ID
Bean ID of an ExecutableSearchBuilder<HTTPSearch> to produce the request to execute

templateEngine

Bean ID
Bean ID of a org.apache.velocity.app.VelocityEngine to use for processing the URL template, generally unnecessary

Child Elements

Any of the common child elements can be specified. In addition, the following may be specified.

NameCardinalityDescription

<URLTemplate>

0 or 1Template of a URL to execute via HTTP GET or POST
<BodyTemplate>0 or 1Template for a request body to submit via HTTP POST, requires use of <URLTemplate>
<CacheKeyTemplate>0 or 1Template to produce a cache key to associate with the result of an HTTP POST, requires use of <BodyTemplate>
<ResponseMapping>0 or 1Inline or external script to execute to process the response body

<ResultCache>


0 or 1

Defines how results should be cached.

<ResultCacheBean>

Bean ID (in the element content) defining how results should be cached as an externally defined com.google.common.cache.Cache<String,Map<String,IdPAttribute>> 

Externally (Spring) Defined Content

If the springResource or springResourceRef attributes are specified, then the configuration of the data connector bean is delegated to the supplied resources. The system will create a factory for an HTTPDataConnector object, and look for beans in the Spring resource(s) supplied that match the types of properties supported by that type and its parent classes. Note that since these are not public, but implementation classes, they are subject to change, which creates some risk during non-patch upgrades, so you must take additional precautions to use this feature.

In practice, the HTTP Data Connector may be supplied with beans of the following types:

In addition native bean IDs can be injected as follows:

  1. The HttpClient instance and its security settings are injected via the httpClientRef and httpClientSecurityParametersRef attributes.
  2. The builder for the request can be specified as an externally defined bean via the executableSearchBuilderRef attribute (as a replacement for the <URLTemplate> element and related elements). This allows for complete generality of the request-building process.
  3. The processing of the response can be specified with an externally defined bean via the mappingStrategyRef attribute (as a replacement for the <ResponseMapping> element).
  4. The caching of results can be specified as an externally defined bean via the <ResultCacheBean> element (as a replacement for the <ResultCache> element).
  5. Rarely, a non-default Velocity engine can be injected via the templateEngine attribute.