...
Relative to an IdP, there are fewer scenarios in which it typically becomes necessary to worry about these settings. Usually you can accept the defaults and things will work most of the time. The settings are provided for handling more unusual situations.
Version 2.6+
TBD
Version 2.5 and Earlier
Prior to Version 2.6, the signing and encryption settings can be used in the <ApplicationDefaults>, <ApplicationOverride>, and <RelyingParty> elements and allow for per-application, per-IdP behavior.
...
Failure while signing or encrypting is usually handled differently. When signing fails, it's common for the message to be sent unsigned. When encryption fails, the message isn't sent. The most common source of failure in either case is the inability to locate a key, but in the case of encryption, that key beongs to the peer, and is generally obtained from SAML metadata.
Transport Authentication
An additional setting (requireTransportAuth) exists to control whether to allow for a back-channel peer to authenticate itself to the SP using a message signature, or whether the TLS connection must be authenticated. It defaults to "true", requiring the server's TLS certificate to be trusted (generally through metadata). Prior to V2.6, this setting has to be explicitly turned off to allow for signature-based communication in both directions, which is the main reason why the back channel has traditionally only worked when a separate port and certificate are deployed by the peer.