Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The JDBCStorageService is a database-level compatible replacement for the JPAStorageService and provides a Storage Service StorageService on top of an RDBMS.  It communicates directly with the database rather than using Hibernate ORM which has issues with reliability and the lack of provenance of its software artifacts.

It is possible to swap between the JPA and JDBC storage service (if running an older version of our software), and indeed to have different versions running on different nodes.

...

Plugin ID

Module(s)

Latest Version

Bug Reporting

net.shibboleth.plugin.storage.jdbc

None

2.0.0; download

https://shibboleth.atlassian.net/browse/JJDBC

For a detailed guide on how to install plugins, see here. In summary, use the plugin command that ships with the IdP to install the plugin from either a local file pre-downloaded, from a URL, or by pluginId 4.2

Installation

C:>\opt\shibboleth-idp\bin\plugin.bat -I net.shibboleth.plugin.storage.jdbc

or

$ /opt/shibboleth-idp/bin/plugin.sh -i http://shibboleth.net/downloads/identity-provider/plugins/pluginName/version/URL

Switching from JPAStorageService

If you are currently running with the JPAStorageService you can reconfigure to use the JDBCStorageService relatively easily

  • Locate the configuration (search for the class name org.opensaml.storage.impl.JPAStorageService

  • Remove the EntityManagerFactory bean, taking note of the dataSource property.

  • Remove also the VendorAdapter bean if present.

  • Change the StorageService bean

    • Replace class="org.opensaml.storage.impl.JPAStorageService" with parent="shibboleth.JDBCStorageService"

    • Remove the constructor parameter and instead add a pointer to the dataSource you noted above p:dataSource-ref="...."

At this stage you should be able to test the configuration.

Once it works you can change bean names appropriately and add any extra configuration as detailed below.

Database Preparation

If you are (preferred) by plugin ID.

Database Preparation

If not upgrading from a JPAStorageService configuration, then you need to:

  • Create the database table for the plugin to use.

  • Download the appropriate JDBC driver.

  • (Optionally, but recommended) Download Select a Connection Pooling implementation. DBCP2, included with the IdP software, is suggested as it requires no additional software.

(If you are moving migrating from the JPAStorageService you do not need to make any changes to your database and you can use the same configuration for the DataSource as you did for the JPAStorageService. See more below.)

Creating the Database

Example Schemas are shown below.

Note

Whatever you do, you MUST ensure the context and id columns are case-sensitively handled and compared. That is a requirement of the API that will be using the database. This is frequently NOT the default behavior of databases such as MySQL.

The specific examples that follow should NOT be assumed to be functional, as they likely are the product of different sources, varying amounts of testing (including none), and may not be current. Drivers get updated frequently and JDBC and database bugs appear and disappear with regularity. When in doubt, always grab new ones when problems appear.

...

Note

The value column must be arbitrarily wide to allow the JDBC storage service to back IdP Sessions

Switching from the JPAStorageService

If you are currently running with the JPAStorageService you can reconfigure to use the JDBCStorageService relatively easily

  • Locate the configuration (search for the class name org.opensaml.storage.impl.JPAStorageServicein your configuration.

  • Remove the EntityManagerFactory bean, taking note of the dataSource property.

  • Remove also the VendorAdapter bean if present.

  • Change the StorageService bean

    • Replace class="org.opensaml.storage.impl.JPAStorageService" with parent="shibboleth.JDBCStorageService"

    • Remove the constructor parameter and instead add a pointer to the dataSource you noted above p:dataSource-ref="...."

At this stage you should be able to test the configuration.

Once it works you can change bean names appropriately and add any extra configuration as detailed below.

Configuring and Using JDBC DataSources

Using JDBC requires defining a Spring bean representing the DataSource instance the IdP will create and inject into, among other places, this plugin’s StorageService definition. Note that a JDBC driver class is not the same as a DataSource. Generally a database driver will provide one or more DataSource implementation classes to choose from and this is the class you will need to create via a Spring bean to inject into other objects.

A data source will typically look like this (with most of the settings depending on the particular driver:

Code Block
<bean id="my.DataSource" class="..."
  p:url="jdbc:database:connectionprops" />

JDBC Driver

You need to locate, download and verify the JDBC driver for your database and place it in edit-webapp/WEB-INF/lib.After populating edit-webapp/WEB-INF/lib you should execute bin/build.sh or bin/build.bat as appropriate for your environment.

Connection Pooling

We recommend For higher loads, we generally assume the use of a DataSource that provides connection pooling. The Commons DBCP 2 library is included with the IdP and recommended can be used for this purpose without downloading additional libraries.

Note

Tomcat’s JDBC Pooling driver is known has been observed to cause connection leaks and is not supported.

...

Having located, downloaded and verified the connection pooling jar you should place it in As with the driver above, if you choose to use a third party pooling library not supplied with the software, you will need to add its jar(s) to edit-webapp/WEB-INF/lib.After populating edit-webapp/WEB-INF/lib you should execute bin/build.sh or bin/build.bat as appropriate for your environment

Configuration

You and rebuild the warfile.

Storage Service Configuration

Once a DataSource bean is in place, you need to add the definition of a bean derived inheriting from shibboleth.JDBCStorageService into an appropriate configuration file (usually global.xml). The options you can provide to the bear bean are detailed below.

Expand
titleBehavioral Options

The behavior of the Storage Service is controlled by the following options

Option Java Bean Property Name

Type

Default

Description

dataSource

Bean ID

Required

The Bean ID of the DataSource to use

cleanupInterval

Duration

“PT10M” (or the value of the property idp.storage.cleanupInterval if it is set)

The time between one cleanup and another. A value of 0 indicates that no cleanup will be performed.

retryableErrors

Comma-delimited list

A comma separate list of SQL errors which will cause a failed transaction to be retried (a maximum of transactionRetry times)

transactionIsolation

8 (Connection.TRANSACTION_SERIALIZABLE)

The level of transactional isolation required as described for the Connection Interface
Starting in V2.0.0, if 0 (TRANSACTION_NONE) is specified then the transactional isolation is not set at the Connection level

transactionRetries

Integer

3

Number of retries if insertion fails due to database transaction bugs

verify

Boolean

true

Whether to verify the database connection on startup

localLocking

Boolean

false

Whether to do thread level locking to arbitrate access (for this IdP) to the the database. This can be useful in high contention situations when multiple transaction retries are happening.

contextSize

Integer

255

The size of the ‘context’ column in you database. Only change this if you are using a non-standard DDI

keySize

Integer

255

The size of the ‘key’ column in you database. Only change this if you are using a non-standard DDI

valueSize

Integer

Integer.MAX_SIZE (231)

The mazimum size of the ‘value’ column in you database. Only change this if you are using a non-standard DDI

...

In the example below use of Commons DBCP is demonstrated (class="org.apache.commons.dbcp.BasicDataSource", p:url="..." in the DataSource bean). When using other Connection Pool implementations change the class and properties appropriately , (e.g.:

...

Tomcat DBCP2: class="org.apache.tomcat.dbcp.dbcp2.BasicDataSource", p:url="..."

...

,

...

HikariCP: class="com.zaxxer.hikari.HikariDataSource"

...

p:jdbcUrl="...")

Code Block
    <bean id="my.dataSource" class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close" lazy-init="true"
       p:driverClassName="......"
       p:url="jdbc:hsqldb:mem:StorageService"
       p:username="shibboleth"
       p:retryableErrors="4001, 4002"
       p:password="%{JDBCPassword}" />

    <bean id="JDBCStorageService" parent="shibboleth.JDBCStorageService"
          p:dataSource-ref="dataSource"
          p:transactionIsolation="4"
          p:retryableErrors="40001"
     />

...