IdP Load Testing

Load testing any new production level system is an important part of ensuring that the service is ready for prime time use. The Shibboleth team makes available a set of tests that can be used with The Grinder. While you can get the test scripts and run them in your own installation of Grinder it is much easier to check out a pre-built kit and then run the packaged tests.

Follow this link for some typical throughput numbers for some environments.  Every environment will be different and this should be used to estimate, not as a substitute for your own load tests.

Load Testing Framework Basics

Requirements

• Unix OS based machine(s)
• Subversion client
• Java 1.5

Getting and Installing the Framework

1. SVN check out the testing kit from https://svn.shibboleth.net/extensions/java-loadtest/trunk
2. Move the grinder-framework directory to wherever you wish to install it. This location will be referred to as GF_HOME for the rest of the document.
3. Copy the test packages, the saml1sso_idp, saml2sso_idp, and sp_idp directories, to GF_HOME/tests

Installation is now complete.

Running the Framework

The load testing framework is composed of two parts, an agent and a console. The agent performs the actual work while the console, a GUI application, collects the results from agents and displays some metrics based on those results

To run the console execute the GF_HOME/bin/console.sh command. This will start the console which will listen on port 6372 for agent results. Be sure that this port is accessible to the agents you start.

To run the console execute the GF_HOME/bin/agent.sh <test_name> command. The name of the test is dependent on which test you will run and is given in the documentation for those tests. You can install agents on more than one machine and have them report back to a single client. This is useful if one agent machine is not capable of generating enough load.

Test Setup and Configuration

Each test package contains a test.properties which hold configuration information for the given test. The later half of these options are Grinder specific properties. In most cases you'll need to change the following properties.

SAML 1, IdP Only

This test creates a Shibboleth SSO authentication request (the requests used to initiate an SSO flow in Shibboleth 1), authenticates to the IdP using either HTTP BASIC or FORM based authentication, and checks that a SAML response was returned.

Test Name: saml1sso_idp

Configuration Properties

SAML 2, IdP Only

This test creates a SAML2 SSO authentication request, authenticates to the IdP using either HTTP BASIC or FORM based authentication, and checks that a SAML response was returned.

Test Name: saml2sso_idp

Configuration Properties

SP to IdP Request

This test accesses an SP, carries the created request to the IdP, authenticates to the IdP using either HTTP BASIC or FORM based authentication, and checks that a SAML response was returned. Note, the SP endpoint used must NOT require a discovery process.

Test Name: sp_idp

Configuration Properties