The Shibboleth IdP V3 software has reached its End of Life and is no longer supported. This documentation is available for historical purposes only. See the IDP4 wiki space for current documentation on the supported version.

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Current »

Capacity planning includes analysis of system components, environment, and anticipated load in order to build a system that performs as expected. Load testing is a vital tool to ensure that capacity plans have achieved their intended goals.

Planning an IdP Deployment

TBD

Load Testing

You must load test your system to have any confidence whatsoever that your plans meet reality. We provide some guiding principles below to provide a context for the tools and techniques used for load testing.

Know what you're testing.

In order to know what you're testing, you must determine what you want to test. While most test efforts naturally begin with the goal in mind, the challenge is to devise a test plan and a testing environment that actually accomplishes the goal. Most of the effort of testing should go into this step; after all, the data is only as good as the test. An example may help illustrate the point. It makes little sense for a system load test to be conducted by clients over the public Internet unless the effect of network latency is intended to be included in the scope of the test. In most cases deployers simply want to know the theoretical maximum throughput of a system; varying network latency in Internet-connected nodes will almost certainly complicate measurement and analysis toward that goal.

Keep it simple.

Create the simplest test plan and testing environment that is capable of measuring the desired aspects of a system. Simplicity favors measurement and data analysis and is arguably a goal in itself.

One good result means nothing.

All tests include systematic and random variation. In order to obtain meaningful results, it is necessary to perform multiple test runs for each variable value. Statistical analysis is a natural consequence of multiple test runs and is itself a sign of sound test methodology.

Load Testing Tools and Techniques

Apache JMeter is a free and capable testing tool that can be used to load test the IdP. A simple parameterized test plan, shibboleth-unsolicited-sso-test.jmx, is attached to this page. The test flexes the authentication and SAML messaging subsystems to get a reasonable sense of system throughput under common load patterns.

JMeter Unsolicited SSO Test Usage
TEST="/path/to/shibboleth-unsolicited-sso-test.jmx"
LOG="/path/to/jmeter.log"
PROPS="-Dcredentials=/path/to/users-and-passwords.csv"
PROPS="$PROPS -Didphost=https://idp.example.org"
PROPS="$PROPS -Didpport=443"
PROPS="$PROPS -Didpcontext=/idp"
PROPS="$PROPS -Dproviderid=https://sp-entity-id-you-trust.example.org/shibboleth"
export JVM_ARGS="-Xms1024m -Xmx1024m $PROPS"
export JAVA_HOME="/path/to/jdk"
export PATH="$JAVA_HOME/bin:$PATH"
/path/to/jmeter/bin/jmeter -n -j $LOG -t $TEST

The credentials file should be a comma-delimited list of username/password pairs, one per line. The test by default allocates 100 threads to perform authentication at the unsolicited SSO endpoint using each credential pair in turn. The test runs for 5 minutes (by default) and writes results to a file named shibboleth-unsolicited-sso-test-result-YYYYmmdd-HHMMSS.csv in the user's home directory. The summarize-results.py file, also attached to this page, may be used to process the test results and report a summary for each test run.

 

  • No labels