Self-Signed Certificate Detection and Handling

While self-signed certificates can be used in controlled environments such as internal testing, because they are not verified by a trusted certificate authority (CA) they must not be used in production environments. If the use of a self-signed certificate is intentional, such as for testing purposes, the MDC Gateway and Traffic Routing Agent (TRA) components include ways to override default handling.

In the context of Transport Layer Security (TLS), certificates are critical to establishing secure connections. Certificates ensure that one party can trust the identity of another party. The TLS hierarchy of trust is represented by a certificate chain with an end-entity certificate at one end and a root certificate issued by a root CA at the other end. The root CA is the ultimate trusted authority that signs intermediate CAs. Intermediate CAs act as the bridge between the root CA and the end-entity certificate, distributing trust while maintaining security.

TRA and MDC Gateway use the length of the chain to determine if a certificate is self-signed. A correctly signed certificate has at least one intermediate CA, signed by a root CA, ensuring a chain length of at least two. If the chain includes no additional certificates, or only one, either an intermediate or root CA is missing (or both are missing) and the MATRIXX components flag the end-entity certificate as self-signed. IF TRA or MDC Gateway detects a self-signed certificate, they block further request processing. This policy can be applied to both TLS and HTTP/2 connections.

MDC Gateway

For MDC Gateway, in mtx_config.xml, the configuration parameters tls_listener_reject_self_signed_certificate and http2_listener_reject_self_signed_certificate can be set to allow processing for TLS or HTTP/2 connections, respectively.

In the event of a self-signed certificate, MATRIXX Data Container (MDC) connections are reset with a TCP RST flag immediately after receiving an initial request. With an HTTP/2 connection, an initial request prompts an HTTP 500 Internal Server Error, and the connection is reset with the GOAWAY frame that signals to the client that the connection is closing gracefully. No further requests are processed on the connection.

For more information about applying configuration with sed.extra files, see the discussion about configuring MATRIXX Engine in MATRIXX Configuration.

TRA

For TRA, the security-reject-self-signed-certificate parameter controls detection and handling. The parameter can be set globally in tra_config.xml or per virtual server (VS) as a vsopt attribute.

For more information about how to set security-reject-self-signed-certificate globally, see the discussion about configuring tra_config.xml parameters in MATRIXX Configuration. For more information about how to set security-reject-self-signed-certificate as a vsopt attribute, see the discussion about TRA virtual server protocol elements in MATRIXX Configuration.