Home
MATRIXX Monitoring and Logging
Monitoring Components
The following information describes monitoring different components for MATRIXX.
Monitoring MATRIXX Route Cache
Monitor the Route Cache by reading watermark log messages for each Route Cache table and Route Cache SNMP statistics. The SNMP statistics are defined in the matrixx_rc_mib.txt MIB file and returned by the print_snmp_stats.py script. They include statistics on subscriber and session data, statistics on insert, fetch, erase, and iterator operations, general statistics on subscriber records, and garbage collection.
Route Cache Watermark Log Messages
Route Cache watermark log messages inform system administrators about the usage level of each Route Cache table, such as the table that contains cached subscriber ID data and the table that contains cached session ID data. The number of free or available route records in each Route Cache table is tracked. If watermark messages are logged frequently, or communicate a critical state, it may be an indication that the size of the table must be increased to accommodate more route records, such as more cached subscriber identity data or more cached session ID data. Route Cache watermark log messages are logged in the mtx_debug.log file.

Welcome
MATRIXX Release Notes
MATRIXX Architecture
MATRIXX Installation and Upgrade
MATRIXX Configuration
MATRIXX Security
MATRIXX Integration
MATRIXX Diameter Integration
MATRIXX Call Control Framework Integration
MATRIXX TM Forum Integration
MATRIXX 5G Integration
MATRIXX 5G Event Streaming
MATRIXX Event Streaming
MATRIXX Administration
MATRIXX Web App Administration
MATRIXX Monitoring and Logging
- About MATRIXX Monitoring and Logging
  MATRIXX Monitoring and Logging describes monitoring and logging for MATRIXX components in a MATRIXX environment.
- Metrics and Alerts
  The following information describes monitoring metrics and alerts information for MATRIXX. MATRIXX Support recommends using the Prometheus and Grafana third-party products to monitor a MATRIXX environment, along with aggregated logging.
- Logging
  The following describes logging information for MATRIXX.
- Tracing
  The following information describes tracing information for MATRIXX.
- Monitoring Components
  The following information describes monitoring different components for MATRIXX.
  - Monitoring MATRIXX Business API Gateway
    Business API Gateway SNMP statistics are defined in the matrixx_web_mib.txt file and you display them using the snmp-net-utils snmpwalk command.
  - Monitoring MATRIXX Call Control Framework
    The topics in this section have information about monitoring Call Control Framework (CCF) components, including Network Enabler and CAMEL Gateway.
  - Monitoring Diameter Events
    Diameter Event Logger listens to the Diameter Gateway through a TCP connection and logs diameter application messages to a JSON file.
  - Monitoring Event Streaming
    Event streaming errors are logged in the Event Streaming Framework mtx_event_streamer.log file. Event Streaming Framework uses Log4j 2.x to log error, debugging, and informational messages. You can also monitor event streaming performance using the performance_analysis_event_server.py script. For information about the script, see the discussion about performance_analysis_event_server.py.
  - Monitoring MATRIXX Route Cache
    Monitor the Route Cache by reading watermark log messages for each Route Cache table and Route Cache SNMP statistics. The SNMP statistics are defined in the matrixx_rc_mib.txt MIB file and returned by the print_snmp_stats.py script. They include statistics on subscriber and session data, statistics on insert, fetch, erase, and iterator operations, general statistics on subscriber records, and garbage collection.
    - Route Cache Watermark Log Messages
      Route Cache watermark log messages inform system administrators about the usage level of each Route Cache table, such as the table that contains cached subscriber ID data and the table that contains cached session ID data. The number of free or available route records in each Route Cache table is tracked. If watermark messages are logged frequently, or communicate a critical state, it may be an indication that the size of the table must be increased to accommodate more route records, such as more cached subscriber identity data or more cached session ID data. Route Cache watermark log messages are logged in the mtx_debug.log file.
    - Display Route Cache Statistics
      Route Cache statistics are collected for subscriber and session data, including statistics on insert, fetch, erase, and iterator operations, and general statistics on subscriber records, and garbage collection.
    - Reported Route Cache Statistics
      The following Route Cache (RC) statistics are reported by the print_snmp_stats.py script.
  - Monitoring Payment Service Logging
    Payment service logs messages to the payment-service.log.
  - Monitoring MATRIXX SNMP Statistics
    By default, the SNMP Agent running on a MATRIXX Engine monitors the status and health of all the servers and clusters it has. It collects statistics to aid in identifying potential processing issues.
  - Monitoring MATRIXX Traffic Routing Agent
    Traffic Routing Agent SNMP statistics are defined in the matrixx_tra_mib.txt MIB file and are returned by the print_snmp_stats.py script when run on any of the Traffic Routing Agent servers. They include categories such as TCP traffic, UDP traffic, Diameter Gateway traffic, and MDC Gateway traffic.
  - Monitoring MATRIXX Web Apps
    Use the MATRIXX system logs and SNMP statistics and notifications to monitor the MATRIXX web-facing applications, such as RS Gateway, Gateway Proxy, and My MATRIXX.
  - Monitoring MATRIXX SBA Gateway
    The MATRIXX SBA Gateway SNMP statistics are defined in the matrixx_web_mib.txt file, using the snmp-net-utils snmpwalk command to display them.
  - Monitoring 5G Event Streaming
    5G event streaming can be monitored with standard Kafka and custom metrics.
  - Monitoring Kafka CDR Consumer
    Use aggregated logging and the Prometheus and Grafana third-party products to monitor Kafka CDR Consumer.
- Appendixes
MATRIXX Policy
MATRIXX Kafka CDR Consumer
MATRIXX Pricing and Rating
MATRIXX Pay Now
MATRIXX Subscriber Management
MATRIXX Subscriber Management API
MATRIXX Business API SDK
My MATRIXX Help
MATRIXX Backoffice Customer Tool Help
MATRIXX Third-Party Licenses
Glossary

Route Cache Watermark Log Messages

Route Cache watermark log messages inform system administrators about the usage level of each Route Cache table, such as the table that contains cached subscriber ID data and the table that contains cached session ID data. The number of free or available route records in each Route Cache table is tracked. If watermark messages are logged frequently, or communicate a critical state, it may be an indication that the size of the table must be increased to accommodate more route records, such as more cached subscriber identity data or more cached session ID data. Route Cache watermark log messages are logged in the mtx_debug.log file.

The Route Cache watermark log messages shown below are logged for any Route Cache table, where db_table_name is the table name, such as subscriber_id.db or session_id.db for the subscriber ID and session ID Route Cache tables, respectively.

Note: SNMP traps are generated whenever usage events occur that cross or clear a Route Cache table threshold. See the discussion about system notifications in MATRIXX Monitoring and Logging for information about Route Cache SNMP traps.

Table 1. Example Route Cache Watermark Messages
Route Cache Table Watermark Message	Description
LM_INFO MtxRouteCache::watermarkCrossed: `db_table_name` usage threshold crossed 75%	The table is at more than 75% capacity given the maximum number of records allowed.
LM_WARN MtxRouteCache::watermarkCrossed: `db_table_name` usage threshold crossed 95%	The table is at more than 95% capacity given the maximum number of records allowed.
LM_CRITI MtxRouteCache::watermarkCrossed: `db_table_name` capacity full	The maximum number of records allowed in the table has been reached.
LM_WARN MtxRouteCache::watermarkCleared: `db_table_name` usage dropped below 95% but running low on resources	The number of records in the table has dropped below 95% capacity, but resources are low.
LM_WARN MtxRouteCache::watermarkCleared: `db_table_name` usage dropped below 75% but running low on resources	The number of records in the table has dropped below 75% capacity, but resources are low.
LM_INFO MtxRouteCache::watermarkCleared: `db_table_name` usage dropped below 60%	The number of records in the table has dropped below 60% capacity.