Monitoring SBA Gateway with Prometheus

SBA Gateway network functions include a Prometheus client that exposes various metrics, such as memory usage or the number of incoming HTTP requests. The Prometheus client is enabled by default, but it can be disabled if required.

For instructions on using Prometheus in a cloud native MATRIXX installation, see the discussion about monitoring with Prometheus and Grafana.

The Prometheus client exposes metrics at the /metrics URL:

$ curl http://localhost:9098/metrics
# HELP jvm_memory_pool_allocated_bytes_total Total bytes allocated in a given JVM memory pool. Only updated after GC, not continuously.
# TYPE jvm_memory_pool_allocated_bytes_total counter
# HELP sba_http_request_total The total HTTP requests to the SBA
# TYPE sba_http_request_total counter
# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds.
# TYPE jvm_gc_collection_seconds summary
jvm_gc_collection_seconds_count{gc="PS Scavenge",} 4.0
jvm_gc_collection_seconds_sum{gc="PS Scavenge",} 0.123
jvm_gc_collection_seconds_count{gc="PS MarkSweep",} 2.0
jvm_gc_collection_seconds_sum{gc="PS MarkSweep",} 0.071

SBA Gateway exposes standard JVM metrics. SBA Gateway (JVM and other metrics) shows the additional metrics that are available, in addition to these standard metrics.

In addition, SBA Gateway provides metrics from the following underlying Java libraries:

Vertx
Prometheus Java Client
Kafka Client
For information about these libraries, refer to the Vertx, Prometheus, and Kafka documentation.

Note: Timer is not a Prometheus type. A Timer is presented as 2 count-type metrics suffixed with _seconds_count (count of events) and _seconds_sum (total time).

Table 1. SBA Gateway (JVM and other metrics)
Component	Metric Name	Type	Labels	Description
ALL	mtx_sba_jms_message_received	Counter	queueName: Name of ActiveMQ Queue application: `sba-5gc-networkfunctions-chf`	The number of JMS messages received.
ALL	mtx_sba_jms_message_failures	Counter	queueName: Name of ActiveMQ Queue application: `sba-5gc-networkfunctions-chf`	The number of JMS message failures.
	mtx_sba_chf_notify_request_latency	Timer		The time between SBA Gateway receiving a notify message from the activeMQ queue and sending to the network, and sending the response back to the engine.
	mtx_sba_backstop_message_processed	Counter	backstopHandler: Backstop handler name requestType: Request message type	The number of messages processed by the backstop.
	mtx_sba_http_request_throttled	Counter	methodName: HTTP request method statusCode: Response code	The total HTTP requests throttled by SBA Gateway.
	mtx_sba_engine_client_message_counter	Counter		The number of messages sent to MATRIXX Engine.
	mtx_sba_engine_client_message_failure_counter	Counter		The number of message failures sent to the engine.
ALL	mtx_sba_engine_client_request_latency	Timer	application: `sba-5gc-networkfunctions-chf`	The request latency for the engine client.
ALL	mtx_sba_engine_client_connection_counter	Gauge	application: `sba-5gc-networkfunctions-chf`	The number of active connections to the engine.
ALL	nrf_registered	Gauge	instanceId: CHF instance ID. urls: List of URLs registered.	The number of NRF instances with which SBA Gateway registers successfully.
ALL	mtx_sba_http_request_throttled_total	Counter	methodName: `HTTP Method` statusCode: `HTTP Status` application: `sba-5gc-networkfunctions-chf`	The total HTTP requests to SBA Gateway that were throttled.
ALL	log4j2_events_total	Counter	level: debug, warn, trace, error, fatal or info application: `sba-5gc-networkfunctions-chf`	The count of log messages by level.
ALL	mtx_sba_engine_client_message_counter_total	Counter	application: `sba-5gc-networkfunctions-chf`	The number of messages sent to the engine.
ALL	mtx_sba_engine_client_message_failure_counter_total	Counter	application: `sba-5gc-networkfunctions-chf`	The number of failed messages sent to the engine.

Note: Metrics are also available using SNMP. See the discussion about the SNMP Prometheus adapter for details.

Recommended Metrics for SBA Gateway

The following metrics are recommended for use in Grafana dashboards:

Table 2. SBA Gateway Metrics
Subsystem	Source	Metric
HTTP Client (outgoing requests to NRF or Network)	Vertx Library	vertx_http_client_requests_total vertx_http_client_errors vertx_http_client_response_time_seconds
HTTP Server (incoming requests from Network)	Vertx Library	vertx_http_server_active_connections vertx_http_server_requests_total vertx_http_server_response_time_seconds
Engine Client	SBA Gateway	mtx_sba_engine_client_connection_counter mtx_sba_engine_client_request_latency_seconds mtx_sba_engine_client_message_counter mtx_sba_engine_client_message_failure_counter
ActiveMQ	SBA Gateway	mtx_sba_jms_message_received mtx_sba_jms_message_failures
Kafka Producer	Kafka Client	kafka_producer_record_send_total kafka_producer_record_error_total kafka_producer_record_retry_total kafka_producer_request_latency_avg kafka_producer_buffer_available_bytes
	Vertx Library	vertx_pool_queue_pending
	SBA Gateway	com_matrixx_cdr_producer_failed_total com_matrixx_cdr_producer_pending
NRF	SBA Gateway	nrf_registered