Configuring IdP Clustering

The recommended method for clustering the IdP is to use Terracotta to replicate the IdP's in-memory state between nodes. Terracotta is composed of two parts. Terracotta servers that are responsible for lock management and moving data between clients and Terracotta clients which run in the IdP software and publish/receive changed data to/from the server.

The Shibboleth team recommends running a Terracotta server on each IdP node. In this model one server will run in an active mode while the others run in passive mode. If the active server fails the passive servers elect a new active node and the clients reconnect to it.

Step 1: Adjusting your Firewall

Terracotta uses three ports. Two ports, client-to-server port (default: 9510) and a server-to-server port (default: 9530), are used for inter-node communications. Neither of these ports should be reachable from outside the cluster. A third port (default: 9520) is available and used for JMX. This port should be closed off totally if you are not using JMX or should be expressly limited to particular, trusted hosts. Access to this port allows access to IdP run time information which may include passwords and user information.

Step 2: Edit the Terracotta Configuration file

1. Download tc-config.xml
2. Edit the version numbers for the terracotta integration modules (tim-vector, tim-tomcat-6.0) if the modules are updated.
3. Edit the configuration file $IDP_HOME/conf/tc-config.xml. At the start of the file you will see an example server definition. You will need to add a server definition for each Terracotta server. We recommend just copying and pasting the example Server definition and changing the following attributes:
   • name - a unique name for this server (e.g. idpNode1)
   • host - the hostname or IP address of the server (this information is used by clients and servers when they contact a server).
   • bind - (optional attribute) gives the IP address/interface to which the server should bind (defaults to 0.0.0.0)

All other information in the example should be fine. Once you've made the changes distribute this file to each IdP node in your cluster.

<server name="idpNode1" host="node1.idp.example.org" bind="192.168.1.15">
    <dso>
        <persistence>
            <mode>permanent-store</mode>
        </persistence>
    </dso>

    <logs>$IDP_HOME$/cluster/server/logs</logs>
    <data>$IDP_HOME$/cluster/server/data</data>
    <statistics>$IDP_HOME$/cluster/server/stats</statistics>
</server>

Step 3: Install Terracotta

1. Download Terracotta
2. Install Terracotta
   • Unix users - expand the downloaded archive (/opt is the suggested location)
   • Windows users - run the downloaded .exe file
3. Set the environment variable TC_INSTALL_DIR to $TC_HOME
4. Run the command: $TC_HOME/bin/tim-get.sh install tim-vector 2.6.3#* This pulls down and installs a Terracotta Integration Module (TIM) used by the IdP
5. Run the command: $TC_HOME/bin/tim-get.sh install tim-tomcat-6.0 2.1.3#* This pulls down and installs the TIM for Tomcat 6.0
6. Run the command: $TC_HOME/bin/make-boot-jar.sh -f $IDP_HOME/conf/tc-config.xml#* This command will output the location of the created JAR. Write this down as you will need it when start the IdP. This JAR is specific to your JVM make and version.

Step 4. Adjust IdP Configuration (Optional)

If you are supporting artifact resolution you will also need to make a small change to your IdP's internal.xml file. Locate the bean definition that starts with <bean id="shibboleth.ArtifactMap" and replace it with the following definition.

<bean id="shibboleth.ArtifactMap" class="org.opensaml.common.binding.artifact.BasicSAMLArtifactMap" depends-on="shibboleth.LogbackLogging">
    <constructor-arg>
        <bean class="org.opensaml.common.binding.artifact.BasicSAMLArtifactMapEntryFactory">
            <property name="serializeMessage" value="true" />
        </bean>
    </constructor-arg>
    <constructor-arg ref="shibboleth.StorageService" />
    <constructor-arg type="long" value="300000" />
</bean>

Starting Terracotta Servers

Terracotta servers are started with the start-tc-server script located in the bin directory of your Terracotta install. This command takes two options:

• -n - the name of the server as given in the Terracotta configuration file
• -f - the fully qualified filesystem path or URL to the terracotta configuration file

start-tc-server.sh -n idpNode1 -f http://www.example.org/idp-cluster/tc-config.xml

Starting the IdP & Terracotta Client

In order start the Terracotta client within the JVM used by the IdP you must pass in three JVM arguments. How exactly you do this depends on your Servlet container.

• -Dtc.install-root=$TC_HOME
• -Dtc.config=CONFIG_FILE_PATH_OR_URL
• -Xbootclasspath/p:<PATH_TO_CLIENT_JAR_CREATED_BY_TERRACOTTA>

JAVA_OPTS="-Dtc.install-root=/opt/terracotta \
           -Dtc.config=http://www.example.org/idp-cluster/tc-config.xml \
           -Xbootclasspath/p:/opt/terracotta/lib/dso-boot/dso-boot-hotspot_osx_160_12.jar" \
/opt/tomcat/bin/startup.sh

For jdk-1.6.0_18 and above, you must add to JAVA_OPTS the flag -XX:+MustCallLoadClassInternal to avoid startup issues with Tomcat due to a change in classloading made in the JDK.  Also be sure to review the streamlining instructions described below for additional Tomcat and Terracotta options which you will need to tune to your environment for a truly robust installation.

Productionalizing

Instructions for streamlining formerly were listed here.  Should you be considering a production deployment, you should read those notes which are now documented here: IdPClusterIssues,