Throttling

SBA Gateway can be throttled to protect the platform against overload conditions.

Figure 1 shows an example deployment with ingress and container throttling:

Multiple consumer instances (A), outside the Kubernetes cluster, send traffic towards the CHF service. Each consumer instance creates multiple HTTP/2 connections (B). Individual requests are multiplexed over these connections using HTTP/2 streams.

HTTP/2 connections access the CHF service via an external ingress (C). An ingress implementation, like nginx, for example, can load-balance requests to SBA instances and provide global throttling configuration, such as a maximum total request rate for the CHF service. See the discussion about 5G SBA Reference Architecture for more information on possible HTTP/2 load balancers. (Rate limiting configuration details are load balancer specific.)

Requests are forwarded to instances of SBA Gateway, where container-level throttling is also provided (D) in case individual instances become overloaded.

Finally, requests are forwarded to the Engine for processing via the TRA (E). Container-level throttling monitors outstanding requests to the TRA to prevent a backlog.

Table 1 shows configuration properties and default values as specified in gateway.yaml.

Table 1. Throttling Configuration Properties

Property	Description	Default Value
maxConcurrentRequests	The maximum concurrent active requests. (0 is unlimited). If the number of active requests goes over this value, requests are suspended and queued. If the queue is full requests are rejected with a 503 Service Unavailable error. The default setting of 0 allows concurrent active requests to be unlimited.	0
maxQueuedRequests	The maximum number of requests allowed to be queued. This value is only used if the value of maxConcurrentRequests is greater than 0.	1
maxRatePerConsumer	The maximum number of requests per second allowed from each consumer, by source IP address. Requests exceeding this rate are responded to with a 429 Too Many Requests error. The default setting of 0 allows the number of requests per second to be unlimited.	0
maxRate	The maximum number of requests per second for all consumers of the SBA Gateway instance. The default setting of 0 allows the number of requests per second to be unlimited.	0

Container overload throttling with maxConcurrentRequests and maxQueuedRequests is triggered if the SBA Gateway instance has a backlog of requests caused by a downstream problem causing overload of the TRA or MATRIXX Engine. Use container overload throttling in addition to ingress throttling.

The SBA Gateway may be deployed in an infrastructure that does not support rate limiting at ingress. For these scenarios, SBA Gateway implements a simple rate limiting algorithm using maxRate and maxRatePerConsumer.