The SP V4 design reduces the complexity of, but does not elimiinate, the need for web server or application framework agents in order to support a target environment. This document outlines the initial requirements for these agents and a proposed design breakdown of the software components needed.
Targets
- Determine C++ language standard to target while supporting the necessary platforms.
The initial target set of agents are those already supported: Apache (2.4 only), IIS 7+, and FastCGI. Of these, Apache and FastCGI need to be in C/C++, while it is in principle possible that IIS could be implemented via .NET, and thus in other languages. Given that the majority of the code needed for IIS already exists (or is portable and necessary to support the others anyway), it’s probably we would include IIS in the initial C/C++ set.
I am not inclined to roll back the code we do have and write a lot of new code in C. Yes, it’s more portable, easier and faster to build, etc. It’s also far more error prone and lacking in basic library facilities that modern C++ has available. That said, the nature of that C++ code needs to change radically, simplifying and modernizing it, dropping the use of templates wherever possible, and sticking to the standard library only. The nominal target would be C++11, but we need to research whether a newer version would be practical. Elimination of various Boost-based tricks is essential, especially non-standard lambda and parameter binding templates.
Dependencies
The only allowable dependencies must be usable on both Windows and non-Windows platforms (or there must be native alternatives to use). The set should be as minimal as possible. At this stage, only the following has been identified as critical needs:
Regular expressions
TLS-capable HTTP 1.1 client that allows basic control of server trust anchors (NOT the advanced metadata-driven trust evaluation support we use today).
C++11 has standard liibrary support for regular expressions.
The most obvious choices to use for HTTP are the native Windows client (for Windows obviously) and libcurl + OpenSSL (for everythiing else). In principle, libcurl with a different TLS implementation would work given that we stick to the higher level TLS options exposed by libcurl. Notably, this will pull libcurl and a TLS library into the in-process agent footprint, something we avoided in the current SP to avoid symbol conflicts between Apache and the agent. As long as the same version of OpenSSL is used between libcurl and Apache’s mod_ssl, that shouldn’t cause problems, and most Linux distributions would typically do that.
Library Structure
The current implementation is spread across 3 layers of libraries, with two of them forked into two variants using conditional macros to control the portions included in each fork.
One of the three is OpenSAML, and that will be almost if not entirely gone. Portions of the lowest layer, xmltooling, will survive in modified form, primarily the HTTP request/response abstraction that lived there, as that ultimately gets bridged up into the web server modules and handlers. There are various utility abstractions that may survive, though some of those should be obviated by newer C++ STL features.
Changes Desired or Needed
The highest layer is probably going to become a self-contained package of code (with the necessary portions of xmltooling copied in) that will be statically compiled into the various server modules. This will simplify the build and avoid the need for multiple binaries to be packaged to deliver a module.
Packaging everything into single binary modules will make it impractical to support plugin extensions via shared library. I think that’s a fair trade off given that most of the likely extension points are better handled via Java, but we’ll want to be able to allow for new implementations of some interfaces to be added to the build conditionally and we will continue to want to support a type-based plugin factory API for instantiating variants of components at runtime based on configuration.
Another major component that impacts much of the current code is a DOM-based abstraction for accessing configuration settings by mapping XML attributes and elements into string-based properties that the code can access (often by cascading across multiple layers of properties). Since XML is out of the question, this will need to be simplified into a simpler property-driven API but may still need to be “scoped” by component, which starts to look more like a Windows “ini” file than a flat property set. I have implemented that sort of thing before for Unix, but don’t have any of that code to hand anymore.
Summary of Components and Interfaces
This is a breakdown of the major pieces of the implementation. These proceed in a very inexact “lowest” to “highest” order, with later pieces typically depending on some of the earlier ones.
Non-XML-based configuration support
Logging abstraction
syslog
Windows event log
Apache error log
DDF (remoted data representation and serialization)
HTTP transport between agent and Java hub
Curl-based implementation
Windoes-based implementation
HTTP abstraction for web server integration
Session cache
Session abstraction
File-backed implementation
Chunked cookie implementation
Handler framework
Session initiator handler
Token consumer handler
Logout handlers
Other handlers
Web server bridges to interface to requests and responses
Portable authorization via RequestMap
Native modules
Apache 2.4 module
IIS module
FastCGI authorizer and requester
Component Details
Configuration Support
The current configuration is predominantly XML-based, supplemented/bridged to native Apache commands on that platform (none of the other current agents support a usable configuration mechanism).
The RequestMap will continue to be supported as XML, while other needs may be possible to address with a simpler format (JSON is not simpler and is much less flexble, but an INI or property file is definitely simpler). Boost has a somewhat odd-looking Property Tree module that appears to be header-only (so easy to use without adding runtime dependencies), and it actually supports reading and writing to INI, JSON, and XML(!) with some limitations in each case.
That may be a viable way to parse the RequestMap without delegating to Java, and in fact the representation is rather similar to what I built in Java using my DDF library. (The whole abstraction is somewhat similar to DDF in fact.)
For now, it suffices to say that we need to identify what the settings needed are and their general structure before settling on an approach, but this Boost option looks like a good fit simply due to flexibility.
Logging
Currently we use a fork of an old C++ logging library. The elimination of dependencies and the fact that having out of band logging streams from within a web server has never worked well demands we drop this in favor of “native” options. Logging is of course non-portable, and some web servers do their own logging.
We will need our own logging API with implementations for:
Windows Event logging (which necessitates an accompanying event DLL unfortunately)
syslog (on all but Windows)
Apache’s error logging (for Apache only of course)
I could envision wanting some tuning of logging levels (likely not at the level of individual categories), and possibly wanting support to write to multiple logging sinks, but we will NOT provide our own full implementation with customizable appenders, file rotation, etc.
Critically, the bulk of the important material will be on the Java side, and all of the audit logging will be there.
DDF
The DDF abstraction is discussed elsewhere. This is the core remoting layer to issue requests to the Java hub (as it has been now to connect to shibd). It is mostly implemented at this point in the form needed, with a few outstanding questions. The current serialization support is limited to a record-oriented format that should be suitable for C++-based agents, and would be a usable representation of data stored in files (e.g., the session cache). JSON serialization will be needed but likely only for future agents implemented in other languages and in Java, so is out of scope for this work.
HTTP Transport to Java Hub
The current SP relies on a basic socket protocol to read and write serialized DDF objects between agents and shibd. That was originally the plan for the new version but it became obvious that using HTTP alone had a lot of advantages, so the plan shifted to using a simple HTTP 1.1 transport wrapper to pass in and out bodies of serialized DDF objects. There’s no benefit to using HTTP 2 or QUIC here, though it probably wouldn’t be precluded on the hub end of things.
Using HTTP implies support for TLS, which was originally going to be handled via stunnel, but should be much better for deployers if supported natively. This will not be the “TLS as implemented nowhere else” that the SP today uses for SAML exchanges, but rather the primitive approach supported by most HTTP clients in which a set of static trust anchors are applied using simple APIs. Client TLS authentication could be an option but isn’t expected to be a primary tool due to the complexity of key management.
Portability demands that we build an interface to this function abstractly, and then implement it on each platform using the most obvious plumbling. On Windows that would likely be whatever the “best” native client is now (used to be WinHTTP, don’t know if it’s still the best choice). Everywhere else the best option is probably libcurl, which we are very familiar with.
The current SP includes an abstraction geared to SOAP, with a “transport” class implemented on top of libcurl. A simpler version of it should serve as a good basis for this work.