MessagingAPIRefactoring

Owned by Scott Cantor
Mar 04, 2021
6 min read

General goals/features (vs. OpenSAML v2 API)

General goals:

  • Support non-XML messaging
    • Existing API assumes all messages are XMLObjects. Fails for OpenID, and other non-XML cases.
  • Extensible state storage
    • Address the complex nested hierarchy of contexts currently used to store state in an extensible fashion. It is confusing and unwieldly due to multiple dimensions of extensibility.
  • Composable, reusable code components
    • We currently have this in a specialized way in the form of SecurityPolicyRules. Generalize this notion for all composable, reusable processing code. A major use case would be modules for SOAP WS-* processing requirements.
  • Transport abstraction
    • Currently there is an abstraction called a Transport, with In- and OutTransport variants and some subclasses. This seems both overly complex and confusing for users of OpenSAML, and also leads to problems with a client-side API (vs. server-side, which was largely the implicit, original focus).

Also:

  • Don't try and re-invent the wheel. Messaging is a ubiquitous programming problem. Borrow ideas from other messaging frameworks and libraries where appropriate.
    • Of major influence here were: Apache Axis2; Apache CXF; JAX-WS; Apache Struts 2/WebWork and the Servlet API itself.

 

Outline of Refactored API

Goals above addressed by:

  • Support non-XML messaging
    • MessageContext message slot is now generically parameterized.
  • Extensible state storage
    • Introduce general notion of a "context" which can hold class-indexed child subcontexts, which are themselves just contexts.
  • Composable, reusable code components
    • Implement some variant of the ubiquitous interceptor/handler pattern. This was introduced recently in OpenWS as a part of the delegation work. Evolve this idea further.
  • Transport abstraction
    • Use subinterfaces of the core message decoder and encoder interfaces to provide support for specific transports. Made possible by shift to per-request objects for these components.

General ideas:

  • Model message processing as a "logical pipeline":
    • Inbound MessageContext decoded -> invoke inbound HandlerChain -> "business logic" (aka profile handling) -> invoke outbound HandlerChain -> outbound MessageContext encoded.
  • Transport technology is not a part of this logical model, depends solely on what kind of specific decoder/encoder impls are used.
  • MessageContext represents a single message (inbound, outbound, etc), whose type is generically parameterized, as well as subcontexts associated with that message.
  • This generic parameterization of MessageContext is carried through to things that operate on a MessageContext, to ensure components that are composed are type compatible.
  • State info and logical views can be associated with a MessageContext via its subcontexts.
  • Subcontexts are a way to provide extensible state via a mechanism that provides type safety and avoids casting. This is seen as a better alternative to the Map<String, Object> pattern that is often seen.
  • MessageHandlers are the component that encapsulates reusable code for processing a MessageContext.
  • MessageHandlers can be composed into a MessageHandlerChain, a list of MessageHandlers invoked in order.
  • An invoked MessageHandlerChain is conceptually composed of handlers from different layers - binding/protocol-specific + logical payload-oriented (+ other?).
  • MessageContexts may be bound together via a parent operation context (e.g., InOutOperationContext), which optionally allows components operating on a MessageContext to walk from an outbound context to an inbound or vice-versa.
  • As a specialization of context, InOutOperationContext may also be used to store state not specifically associated with a particular message, e.g. cross-cutting concerns such as: attribute resolution, computed data and so on.

Details and Issues Remaining

General

  • What is the inbound/outbound "unit of messaging"?
    • Practically speaking, this is the generically-parameterized type of the MessageContext (and so also MessageHandler, MessageHandlerChain, MessageDecoder and MessageEncoder, etc).
    • Options:
      1. protocol dependent ("message"-oriented) - e.g. comprehensive front-channel SAML binding data (i.e. object which stores SAML protocol message + RelayState + other binding data); SOAP Envelope; OpenID param set, etc
      2. protocol independent ("payload"-oriented) - e.g. SAML protocol message (only); SOAP Body contents
    • (N.B. See MESSAGE vs. PAYLOAD providers and dispatchers in JAX-WS)
    • We don't need to bake either assumption into the API, but we do need to decide for any specific impls we will write (e.g. OpenSAML decoders/encoders, security handlers, etc).
    • For the OpenSAML implementations of decoders, encoders, and handlers, we have tentatively decided to use a payload-oriented approach.
    • For the payload-oriented model, binding specific data would be stored as subcontext data on the MessageContext
    • Note: with payload-oriented, SOAP cases would not be handled as in the past, where the "inbound message" was a SOAP envelope, rather than for example Envelope's Body payload (e.g., SAML protocol message).
    • TBD: An alternative to payload-oriented could be message-oriented, with logical MessageContext adapters that exposes the logical payload for those things that need it (e.g. profile handlers, message handlers that are payload oriented). Could also just rely on subcontext views for this purpose.
  • Singleton vs. per-request objects
    • Several types of components are now defined to be per-request stateful objects in the processing model: MessageDecoder, MessageEncoder
    • This aligns with, but doesn't require, the Spring notions of singleton vs. prototype beans, as produced by a Spring BeanFactory.
      • TBD: May also investigate utility and viability of request and session scoped beans, as provided by the Spring WebApplicationContext specialization.
    • Reasons for this include
      1. Ability for these components to store their own state, if they wish
      2. Move inputs to properties that are set on the object after instantiation, in order to avoid thorny API issues with dealing with extensibility. These properties can be defined on subclasses or sub-interfaces, avoiding the need to mix this dimension in with the core interface.
        • E.g. MessageDecoder and -Encoders have argument-less decode()/encode() methods. Impls that are specialized to operate on an HttpServletRequest/-Response will be impls of particular sub-interfaces that are HttpServlet oriented. The request and response are set/injected as properties of the per-request runtime object.

BaseContext

  • Represents the core context notion.
  • Has id and creation datetime properties, latter as convenience to have a single time reference where time is used in multiple places in processing and message generation.
  • May contain subcontexts, indexed by Class.
  • Various methods for retrieving, adding, removing and iterating over subcontexts.
  • Convenience autocreation of subcontexts via reflection, for subcontext impls that have a no-arg constructor. E.g. can just call getSubcontext(Foo.class) rather than create and add. The majority of context impls will probably conform to this requirement.
  • Has a getParent() to provide access to the owning BaseContext (to allow walking to the owning context, and/or to other subcontexts).
  • Intended uses:
    1. a bean which holds state
    2. a bean exposing a logical view onto some context data, avoiding the need copy data from, for example, a message to a context.
    3. a node of a context hierarchy which serves to logically group related sets of data represented as child subcontexts.

MessageContext

  • A simple specialization of BaseContext which has a dedicated property slot for holding a message being processed.
  • Represent a single message.
  • Generically parameterized with message type.
  • Subcontexts associated with a MessageContext should typically represent data that is semantically scoped to that particular message (e.g. extracted from it or intended to be applied to it), or a view upon data contained within that message. 

Operation contexts

  • Serves to bind multiple MessageContexts together
  • Can also store extensible state, for implementations that wish to use this as as core processing element.
  • There is no special interface, we currently just have one impl implementing the in-out message exchange pattern, called InOutOperationContext, which is just a straight subclass of BaseContext.

MessageDecoder and MessageEncoder

  • Generically parameterized with message type.
  • Are now strictly decoding/encoding to transport. No longer runs security policies, handlers, etc. This implements better separation of concerns.
  • Transport-specifc inputs (e.g., HttpServletRequest/-Response) are now handled by sub-interfaces and impls that address the specialization required.
  • Are now considered per-request, stateful components.
  • Primary method is now argument-less, e.g. decode() and encode(), with inputs and outputs handled as properties on the runtime object.

MessageHandler and MessageHandlerChain

  • Generically parameterized with message type.
  • May be singleton or per-request, is not defined. Latter case may be useful where the orchestrator of the processing flow (e.g., profile handler) needs to encapsulate run-time state directly in a handler.
  • TBD: MessageHandlerChains will likely typically be resolved by a handler chain resolver, which composes handlers and/or handler chains associated with a specific binding vs. logical profile/payload handler, vs. possibly other dimensions (e.g., relying party-specific handler or chain). (TBD: Or we could just leverage standard Spring config conventions for this purpose)

ProfileHandler

NOTE: in the end this abstraction is NOT used in the Shib IdP, which is based on Spring WebFlow, but was part of the original conception of the OpenSAML messaging model.  It could be used by non-Shibboleth code based on OpenSAML.  It is left here for historical context as to the thinking behind the design.

  • Generically parameterized with message type(s).
  • Are now considered per-request, stateful components.
  • Primary method is now argument-less, execute(), with inputs handled as properties on the runtime object.
  • TBD: Transport inputs/outputs (e.g., HttpServletRequest/-Response) could either:
    1. be set on the ProfileHandler by the profile request dispatcher
    2. injected directed onto message type- compatible instances of decoders and encoders, which in turn are injected onto the ProfileHandler.

Error/Exception/Fault handling

  • Errors can occur and exceptions thrown in multiple places in processing flow:
    1. decoding
    2. inbound handler chain invocation
    3. main logic
    4. outbound chandler chain invocation
    5. encoding
  • TBD: specifics not defined yet. For server side case, current working idea is to have one or more output pipelines (each consisting of a handler chain and encoder), that are selected by the thrown exception type, in a manner similar to the ordered selection of a catch clause, i.e. most specific to least specific exception types. This would likely be implemented by the component that orchestrates the entire messaging flow, e.g. ProfileHandler (if it implements all this internally) or a "processing pipeline" class (see example in examples package).
    • N.B. Axis2 fault in- and out-flows, CXF fault in- and out- InterceptorChains. Think JAX-WS has something too.