The Shibboleth V2 IdP and SP software have reached End of Life and are no longer supported. This documentation is available for historical purposes only. See the IDP v4 and SP v3 wiki spaces for current documentation on the supported versions.

IdPProdLoadTest

Owned by Former user (Deleted)
Last updated: Apr 23, 2013 by ndk@internet2.eduVersion comment
4 min read

IdP Load Testing

Load testing any new production level system is an important part of ensuring that the service is ready for prime time use. The Shibboleth team makes available a set of tests that can be used with The Grinder. While you can get the test scripts and run them in your own installation of Grinder it is much easier to check out a pre-built kit and then run the packaged tests.

The testing framework provided here is provided as a courtesy by the IdP maintainers. By using it you are taking responsibility for reading, and understanding, The Grinder and Shibboleth documentation and diagnosing issues that you may have. Extensive support will not be provided on the mailing list. If you do not know things like your service provider's assertion consumer service or session initiation URL this tool is not for you. 

Follow this link for some typical throughput numbers for some environments.  Every environment will be different and this should be used to estimate, not as a substitute for your own load tests.

Load Testing Framework Basics

Requirements

Getting and Installing the Framework

  1. SVN check out the testing kit from https://svn.shibboleth.net/extensions/java-loadtest/trunk
  2. Move the grinder-framework directory to wherever you wish to install it. This location will be referred to as GF_HOME for the rest of the document.
  3. Copy the test packages, the saml1sso_idp, saml2sso_idp, and sp_idp directories, to GF_HOME/tests

Installation is now complete.

Running the Framework

The load testing framework is composed of two parts, an agent and a console. The agent performs the actual work while the console, a GUI application, collects the results from agents and displays some metrics based on those results

To run the console execute the GF_HOME/bin/console.sh command. This will start the console which will listen on port 6372 for agent results. Be sure that this port is accessible to the agents you start.

To run the console execute the GF_HOME/bin/agent.sh <test_name> command. The name of the test is dependent on which test you will run and is given in the documentation for those tests. You can install agents on more than one machine and have them report back to a single client. This is useful if one agent machine is not capable of generating enough load.

Test Setup and Configuration

Each test package contains a test.properties which hold configuration information for the given test. The later half of these options are Grinder specific properties. In most cases you'll need to change the following properties.

Property Name

Value

grinder.consoleHost

The IP address of the machine on which the console is running

grinder.processes

Number of grinder processes to run. Should be equal to the number of CPU cores on the agent machine.

grinder.threads

Number of unique clients, per process, that will connect to the IdP

grinder.runs

Number of times each client will connect to the IdP, 0 (zero) means to keep going until the console indicates the test should be stopped.

SAML 1, IdP Only

This test creates a Shibboleth SSO authentication request (the requests used to initiate an SSO flow in Shibboleth 1), authenticates to the IdP using either HTTP BASIC or FORM based authentication, and checks that a SAML response was returned.

Test Name: saml1sso_idp

Configuration Properties

Property Name

Value

shib.idp.scheme

HTTP scheme for the SSO endpoint on the IdP - allowed values: http, https

shib.idp.host

host name of the IdP

shib.idp.port

port on which the IdP listens

shib.idp.path

Servlet context path. This is normally /idp

shib.idp.auth

type of authentication to be performed - allowed values: BASIC, FORM

shib.idp.realm

name of the HTTP BASIC authentication realm, unused in the case of FORM authentication

shib.user.name

user name with which to authenticate to the IdP

shib.user.pass

password with which to authenticate to the IdP

shib.sp.id

entity ID of the service provider from which the made up request is said to originate

shib.sp.acsURL

assertion consumer service URL of the service provider from which the made up request is said to originate

SAML 2, IdP Only

This test creates a SAML2 SSO authentication request, authenticates to the IdP using either HTTP BASIC or FORM based authentication, and checks that a SAML response was returned.

Test Name: saml2sso_idp

Configuration Properties

Property Name

Value

shib.idp.scheme

HTTP scheme for the SSO endpoint on the IdP - allowed values: http, https

shib.idp.host

host name of the IdP

shib.idp.port

port on which the IdP listens

shib.idp.path

Servlet context path. This is normally /idp

shib.idp.auth

type of authentication to be performed - allowed values: BASIC, FORM

shib.idp.realm

name of the HTTP BASIC authentication realm, unused in the case of FORM authentication

shib.user.name

user name with which to authenticate to the IdP

shib.user.pass

password with which to authenticate to the IdP

shib.sp.id

entity ID of the service provider from which the made up request is said to originate

shib.sp.authnreq

authentication request template - this property should not be modified

SP to IdP Request

This test accesses an SP, carries the created request to the IdP, authenticates to the IdP using either HTTP BASIC or FORM based authentication, and checks that a SAML response was returned. Note, the SP endpoint used must NOT require a discovery process.

Test Name: sp_idp

Configuration Properties

Property Name

Value

shib.idp.scheme

HTTP scheme for the SSO endpoint on the IdP - allowed values: http, https

shib.idp.host

host name of the IdP

shib.idp.port

port on which the IdP listens

shib.idp.path

Servlet context path. This is normally /idp

shib.idp.auth

type of authentication to be performed - allowed values: BASIC, FORM

shib.idp.realm

name of the HTTP BASIC authentication realm, unused in the case of FORM authentication

shib.user.name

user name with which to authenticate to the IdP

shib.user.pass

password with which to authenticate to the IdP

shib.sp.url

URL the will trigger the SP-initiated SSO process. This may be either an SP protected resource or an SP session initiator.