Call Control Framework Monitoring Messages

Table 1. Call Control Framework Monitoring Messages
Severity	Message	Description	Consequences	Actions
LM_CRITI	LM_CRITI .* [camel_gateway_1:1:1:1(.)] CAM. UnitDataReceiverTask::rxUnitData: UnitData:unitdata_receiver_task: Temporarily prohibiting subsystems because the queue to the application is full.	Something on the processing pod (usually the Charging Server or Transaction Server) is having performance issues and cannot keep up with the level of traffic offered. CAMEL Gateway has temporarily shut down access from the NEs to reduce the load.	All TCAP messages received by CCF are discarded until the overload condition abates.	Perform an analysis to determine why the system is not performing as expected. Common causes include overly complicated pricing logic and slow or defective disks. Consider taking measures to reduce the load or add more hardware.
LM_ERROR	LM_ERROR .* [charging_server_1:1:1:1(.)] ActivityUtils::internalCheckMaxActiveSessions: Exceeded the maximum number of active sessions (.) on device with OID: `OID`	The maximum number of active sessions for the listed device is exceeded. The number of sessions per device is configurable and includes all sessions (TCAP sessions, charging sessions, and other sessions). This could indicate hanging TCAP sessions if the ABORT or END messages from the network or test tool are not being received. This error can also be returned when one subscriber is making calls too fast (for example, when running the sigtranTest tool or a robot caller).	The subscriber cannot make any calls until the number of sessions is decreased, which results from various session cleanup tasks in the system.	Use print_subscriber.py to see which sessions the subscriber still has. If possible, collect TCP dumps between NEs and STPs for the time when this was occurring and analyze them to try to find the offending message sequences.
LM_ERROR	LM_ERROR .* [camel_gateway_1:1:2:1(.*)] MtxTcap::TcapSenderTask::handleInput: No active transaction for sent TCAP END, ABORT, CONTINUE or pre-arranged end.	This error occurs when trying to send a message when the dialog has ended or when the dialog is ended by both ends simultaneously.	Probably no consequences as, either way, the call is over. If this occurs frequently, then it might indicate an issue that should be investigated.	If this issue occurs frequently, collect TCP dumps between NEs and STPs for the time when this was occurring and analyze them to try to find the offending message sequences.
LM_ERROR	LM_ERROR >* [camel_gateway_1:1:1:1(.)] . MtxTcap::FromSccpTask::decodeUnitDataMessageToTcap: Error decoding SCCP message:	CAMEL Gateway believes that the received TCAP message does not contain valid BER encoded ASN.1 according to the protocol specifications. This could be due to receiving an incorrect message or an incorrect definition in asn1_dictionary.xml.	The call is not processed.	Contact MATRIXX Support with the full error. The next line in the log gives the full hex dump of the message, which can be decoded by hand to see if it really is incorrect.
LM_ERROR	LM_ERROR .* [camel_gateway_1:1:1:1(.*)] Logging:camel_logging_task::Task::putNext: Caught MtxUtil::Exception thrown after Enqueue call.	Something on the processing pod (usually the Charging Server or Transaction Server) is having performance issues and cannot keep up with the level of traffic offered.	Some TCAP messages are discarded.	Perform an analysis to determine why the system is not performing as expected. Common causes include overly complicated pricing logic and slow or defective disks. Consider taking measures to reduce the load or add more hardware.
LM_ERROR	LM_ERROR .* [network_enabler_1:1:1:1(.*)] UnitDataForwarderTask::rxUnitData: Failed to forward UnitData message to AI., SSN., TT., NAI., NP., GT. cause code 1.	The NE does not have an active route to forward a TCAP message to the given SCCP address. This could be a message incoming to MATRIXX or a message sent from MATRIXX.	TCAP transactions are not completed due to not being able to forward these messages.	Check if the given SCCP address is valid and whether a route is configured for that address. If a route is configured, check the SNMP statistics to see whether the link for that route is up. If the link is down, check other error messages relating to that link. A link may be down due to misconfiguration, IP network issues, or SS7 network issues. If this is an internal link (NE to CAMEL Gateway link), then it may also indicate that CAMEL Gateway is down.
LM_ERROR	LM_ERROR.* [network_enabler_1:1:2:1(5102.63367)] SigtranSocket::preconfigureSocket:.*`link_name`: bind to local address `IP_address`:`port` failed: Cannot assign requested address.	An NE has been configured to bind to an address that is not local to the NE. Almost certainly due to misconfiguration.	No messages can be sent on or received from the link in question.	Correct the configuration.
LM_ERROR	LM_ERROR .* [network_enabler_1:1:1:1(.*)] SigtranListener::accepted: `TCP\|SCTP` `IP_address`:`port`: Unwanted connection from `IP_address`:`port`.	The NE has received a connection from an IP address and port that it is not configured to receive connections from.	This can be caused by misconfiguration, which would result in lost messages. In a Kubernetes installation, it can also be caused by load balancers pinging the NE to make sure it is still alive. This is not a issue as long as it does not occur so frequently that the real connection cannot be made.	Correct the misconfiguration. If the issue is caused by a load balancer, decrease the load balancer ping frequency.
LM_ERROR	LM_ERROR .* [network_enabler_1:1:1:1(.*)] SigtranSocket::handle_input: `link_name`: Reading socket: Connection reset by peer.	The IP connection to the entity at the end of the link (STP or CAMEL Gateway) has been dropped because the process at the other end has deliberately dropped the connection or has stopped.	Messages on this link cannot be sent or received until this link comes back up.	Not an issue if this happened due to a known outage that is unlikely to happen again and other messages indicate that this link is now up. However, if the link is still down or this error happens multiple times, then investigate CAMEL Gateway or STP for issues.
LM_ERROR	LM_ERROR .* [network_enabler_1:1:1:1.)] . \| M3uaSccpLink::unexpectedMessage: `link_name`: Received unknown SCCP message code .*	The named link has received an unknown type of SCCP message from the network. It has been discarded. This can be the result of load balancers or programs, such as curl, pinging the NE.	Probably not an issue assuming that these are not real SCCP messages that are getting discarded and there are not too many of them.	Stop pinging the NE using a program.
LM_ERROR	LM_ERROR 26517\|26700 2024-07-15 03:21:16.962234 [network_enabler_1:1:1:1(5256.a1c8ac8)] CAM-ffff6795d1fa6b98-NA- 63c40392-NA,92,6598541076 \| M3uaSccpLink::sccpParseError: NE101_STP1_MSS22: Failed to parse SCCP message: SCCP message is too short.	The named link has received an invalid SCCP message from the network. It has been discarded. This can be the result of load balancers or programs, such as curl, pinging the NE.	Probably not an issue assuming that these are not real SCCP messages that are getting discarded and there are not too many of them.	Stop pinging the NE using a program.
LM_ERROR	LM_ERROR .* [camel_gateway_1:1:1:1(.*)] MtxTcap::ToSccpTask::sendUnitDataMessageToNetwork: Error encoding TCAP message to Unitdata message:	MATRIXX failed to send a message because it cannot encode it using the ASN.1 basic encoding rules. This is caused by either a bug in the MATRIXX software or by a bug in the custom asn1_dictionary.xml file. A dump of the offending message is given on the next line.	This is serious if large numbers of these errors occur because it can lead to incomplete TCAP transactions.	Contact MATRIXX support with the following files attached to the ticket: Full log file. asn1_dictionary.xml file. mtx_config.xml file. create_config.info file.
LM_ERROR	LM_ERROR .* [camel_gateway_2:1:1:1(.)] \| M3uaLink::reportException: `link_name`: Sending Error: Protocol Error message <.>	A message was sent to CAMEL Gateway that is not a valid SCCP message. This could be from a load balancer or someone using ping, curl, or some similar program. This could also be due to a misconfiguration that allows non-SCCP traffic to be sent to CAMEL Gateway.	This may not be serious as long as it really is a load balancer or person pinging CAMEL Gateway.	Investigate where these messages are coming from.
LM_ERROR	LM_ERROR.* [camel_gateway_1:1:2:1(.)] . MtxTcap::TcapInformation::isOverloaded: Overload detected by TCAP library due to rate exceeded.	The number of TCAP BEGINs per second received by CAMEL Gateway has exceeded the value specified by the maximum_number_of_begins_per_second specified in mtx_config.xml. New TCAP BEGIN messages are discarded for the remainder of the second to protect the system from overloading.	While the overload lasts, some BEGIN messages (that is, messages starting new calls) are rejected. Messages for existing calls (that is CONTINUE, END, and ABORT messages) are processed normally.	If an overload does not normally occur, then no action is necessary. However, if maximum_number_of_begins_per_second is set too low to cope with normal traffic, then consider increasing it, but be aware that new hardware may also be needed to cope with the extra traffic. Note: The maximum_number_of_begins_per_second value is calculated based on the answer to several questions in create_config.info. For more information, see the discussion about CAMEL Gateway general configuration in MATRIXX Call Control Framework Integration.
LM_ERROR	LM_ERROR.* [camel_gateway_1:1:2:1(.*)] MtxTcap::TcapInformation::isOverloaded: Overload detected by TCAP library due to output queue too full.	This error only occurs with a CAMEL Gateway process running on the processing pod. The percentage of the queue from CAMEL Gateway to the Charging Server has exceeded the configured maximum_input_queue_percentage_full parameter defined in mtx_config.xml. CAMEL Gateway will reject TCAP BEGIN messages until the queue length decreases in order to protect the system from overload.	While the overload lasts, some BEGIN messages (that is, messages starting new calls) are rejected. Messages for existing calls (that is CONTINUE, END, and ABORT messages) are processed normally.	Perform an analysis to determine why the system is not performing as expected. Common causes include overly complicated pricing logic and slow or defective disks. Consider taking measures to reduce the load or add more hardware.
LM_ERROR	LM_ERROR.* [network_enabler_1:1:1:1(.*)] SigtranLink::handleTimeout:`link_name`: Heartbeat FAILURE, closing link immediately.	For the named link, an M3UA heartbeat has not been received in time, so the link has been closed. The NE continues trying to bring the link up, which could be a result of a mismatch of M3UA configuration parameters at either end of the link. This is more likely due to IP networking issues or a failure of the entity at the other end of the link.	No messages are sent or received from the link in question.	Investigate any IP network issues. Also, check that the M3UA settings on each end of the link match.
LM_ERROR	LM_ERROR.* [network_enabler_1:1:1:1(.*)] UnitDataLink::rxScmg:`link_name`: Ignoring SST for unknown PC<point code> from AI `address_indicator`, PC `point_code`, SSN 1.	The NE is receiving Subsystem test (SST) messages for a point code other than the configured point code of this NE.	Traffic from the point code sending the SST cannot be processed.	Check whether the NE is configured with the correct point code. Check the configuration of the entity that is sending the SST messages.
LM_WARN	LM_WARN .* [network_enabler_1:1:1:1(.*)] SigtranSocket::connected:`link_name`: Connect to `IP_address`:`port` failed: Connection refused.	The NE tried to establish a link but failed. This could indicate a misconfiguration or issue with the entity at the other end of the link.	Messages on this link cannot be sent or received until this link comes back up.	Check the configuration of the link. Check the health of the entity at the other end of the link.
LM_WARN	LM_WARN.* [camel_gateway_31:1:1:1(.*)] \| SigtranLink::accept: `link_name`: already connected; rejecting second connection.	This error can also be logged by the NE if any of the NE's links are configured to listen and not to connect. The connection is rejected because the link is already connected. This can be a result of load balancers pinging the listening address and port.	If it is a load balancer pinging CAMEL Gateway or the NE, then this does not matter.	Check the SNMP/Prometheus stats to see if the link in question is UP and handling messages.
LM_WARN	LM_WARN .* [network_enabler_1:1:2:1(.)] . \| DomainLookupTask::handleInput: Cache entry not found - waiting for domain resolution.	This happens when multiple sub-domains are used. An entry for a subscriber has not been found in the route cache on an NE machine. Normally, the transaction servers on the processing pods push subscriber information out to all the route caches when a subscriber is provisioned. One possibility for this error is that the subscriber is not in the database, indicating a mismatch between the subscribers provisioned on the HLRs and the subscribers provisioned in the MATRIXX database. Another possibility is that there was an interruption of communication between the transaction server and the route cache, preventing the record from being pushed to the route cache. The route cache now polls all the transaction servers to try to find the subscriber record.	This is not necessarily serious. The call still succeeds if the route cache finds the subscriber. If not, the call fails. If this happens a lot, then something is wrong with the system.	If this happens a lot, investigate which subscribers and domains are involved. Try to work out why the route cache is not pushing records for these subscribers.
LM_WARN	LM_WARN .* [network_enabler_1:1:1:1(.*)] SigtranLink::handleTimeout: `link_name`: Connect to `IP_address`:`port` timed out.	The NE has failed to establish a connection to the remote end of this link. This could be due to misconfiguration, IP network issues, or a fault with the entity at the other end of the link.	No messages are sent or received on this link.	Check that the correct address and port are configured. Check that the entity with that address and port is up and healthy. Make sure the IP address and port are reachable from the NE.
LM_WARN	LM_WARN .* [network_enabler_1:1:2:1(.*)] SigtranLink::rxAspupAck: `link_name`: Unexpected ASPUP ACK received.	This is probably the result of the NE and the entity the NE is talking to on this link both being configured to be an ASP.	No messages are sent or received on this link.	Make sure that the entity at one end of the link is configured to be an ASP and the other end is configured to be an SGP. Normally, the NE is configured to be an ASP and the STP is configured to be an SGP.
LM_WARN	LM_WARN .* [network_enabler_1:1:1:1(.*)] SigtranLink::rxAspacAck: `link_name`: Received unwanted ASPAC ACK for routing context(s) null.	This is probably the result of the NE and the entity the NE is talking to on this link both being configured to be an ASP.	No messages are sent or received on this link.	Make sure that the entity at one end of the link is configured to be an ASP and the other end is configured to be an SGP. Normally, the NE is configured to be an ASP and the STP is configured to be an SGP.
LM_INFO	LM_INFO .* [network_enabler_1:1:1:1.*)] SigtranLink::rxAspupAck: `link_name`: ASP is up.	This is the final stage in the NE taking a link up. The NE is acting as an ASP so this is probably a link to an STP.	If no errors or warnings are subsequently reported for this link, then the link is up and ready to take traffic. This message can be regarded as clearing any previous errors or warnings for this link.	None.
LM_INFO	LM_INFO .* [network_enabler_1:1:1:1(.)] UnitDataLink::rxScmg: `internal_link_name`: Received SSA for PC . SSN .*	CAMEL Gateway has just informed the NE that it is ready to accept traffic for the stated point code and SSN. The NE informs the STPs that it is ready to take traffic if this has not already been done.	Normal message seen at start-up suggesting that all is well.	None.
LM_INFO	LM_INFO .* [network_enabler_1:1:1:1(.)] UnitDataLink::rxScmg:`internal_link_name`: Received SSP for PC . SSN .*	CAMEL Gateway has just informed the NE that it is NOT ready to accept traffic for the stated point code and SSN. The NE informs the STPs that it is ready to take traffic if no other CAMEL Gateways are available to take traffic. However, if other CAMEL Gateways are available, then traffic is sent to these and it is not a problem.	This may indicate a problem. However, it may be that CAMEL Gateway has been shut down for planned maintenance. It is also normal for CAMEL Gateways serving the standby engine to send SSP messages because traffic should not be sent to the standby engine.	No action necessary as long as the SSP has been received as part of planned maintenance or from a CAMEL Gateway serving the standby engine.

LM_CRITI

LM_CRITI .* [camel_gateway_1:1:1:1(.*)] CAM.* UnitDataReceiverTask::rxUnitData: UnitData:unitdata_receiver_task: Temporarily prohibiting subsystems because the queue to the application is full.

Something on the processing pod (usually the Charging Server or Transaction Server) is having performance issues and cannot keep up with the level of traffic offered. CAMEL Gateway has temporarily shut down access from the NEs to reduce the load.

All TCAP messages received by CCF are discarded until the overload condition abates.

Perform an analysis to determine why the system is not performing as expected. Common causes include overly complicated pricing logic and slow or defective disks. Consider taking measures to reduce the load or add more hardware.

LM_ERROR

LM_ERROR .* [charging_server_1:1:1:1(.*)] ActivityUtils::internalCheckMaxActiveSessions: Exceeded the maximum number of active sessions (.*) on device with OID: OID

The maximum number of active sessions for the listed device is exceeded. The number of sessions per device is configurable and includes all sessions (TCAP sessions, charging sessions, and other sessions). This could indicate hanging TCAP sessions if the ABORT or END messages from the network or test tool are not being received. This error can also be returned when one subscriber is making calls too fast (for example, when running the sigtranTest tool or a robot caller).

The subscriber cannot make any calls until the number of sessions is decreased, which results from various session cleanup tasks in the system.

Use print_subscriber.py to see which sessions the subscriber still has. If possible, collect TCP dumps between NEs and STPs for the time when this was occurring and analyze them to try to find the offending message sequences.

LM_ERROR

LM_ERROR .* [camel_gateway_1:1:2:1(.*)] MtxTcap::TcapSenderTask::handleInput: No active transaction for sent TCAP END, ABORT, CONTINUE or pre-arranged end.

This error occurs when trying to send a message when the dialog has ended or when the dialog is ended by both ends simultaneously.

Probably no consequences as, either way, the call is over. If this occurs frequently, then it might indicate an issue that should be investigated.

If this issue occurs frequently, collect TCP dumps between NEs and STPs for the time when this was occurring and analyze them to try to find the offending message sequences.

LM_ERROR

LM_ERROR >* [camel_gateway_1:1:1:1(.*)] .* MtxTcap::FromSccpTask::decodeUnitDataMessageToTcap: Error decoding SCCP message:

CAMEL Gateway believes that the received TCAP message does not contain valid BER encoded ASN.1 according to the protocol specifications. This could be due to receiving an incorrect message or an incorrect definition in asn1_dictionary.xml.

The call is not processed.

Contact MATRIXX Support with the full error. The next line in the log gives the full hex dump of the message, which can be decoded by hand to see if it really is incorrect.

LM_ERROR

LM_ERROR .* [camel_gateway_1:1:1:1(.*)] Logging:camel_logging_task::Task::putNext: Caught MtxUtil::Exception thrown after Enqueue call.

Something on the processing pod (usually the Charging Server or Transaction Server) is having performance issues and cannot keep up with the level of traffic offered.

Some TCAP messages are discarded.

Perform an analysis to determine why the system is not performing as expected. Common causes include overly complicated pricing logic and slow or defective disks. Consider taking measures to reduce the load or add more hardware.

LM_ERROR

LM_ERROR .* [network_enabler_1:1:1:1(.*)] UnitDataForwarderTask::rxUnitData: Failed to forward UnitData message to AI., SSN., TT., NAI., NP., GT. cause code 1.

The NE does not have an active route to forward a TCAP message to the given SCCP address. This could be a message incoming to MATRIXX or a message sent from MATRIXX.

TCAP transactions are not completed due to not being able to forward these messages.

Check if the given SCCP address is valid and whether a route is configured for that address. If a route is configured, check the SNMP statistics to see whether the link for that route is up. If the link is down, check other error messages relating to that link. A link may be down due to misconfiguration, IP network issues, or SS7 network issues. If this is an internal link (NE to CAMEL Gateway link), then it may also indicate that CAMEL Gateway is down.

LM_ERROR

LM_ERROR.* [network_enabler_1:1:2:1(5102.63367)] SigtranSocket::preconfigureSocket:.*link_name: bind to local address IP_address:port failed: Cannot assign requested address.

An NE has been configured to bind to an address that is not local to the NE. Almost certainly due to misconfiguration.

No messages can be sent on or received from the link in question.

Correct the configuration.

LM_ERROR

LM_ERROR .* [network_enabler_1:1:1:1(.*)] SigtranListener::accepted: TCP|SCTP IP_address:port: Unwanted connection from IP_address:port.

The NE has received a connection from an IP address and port that it is not configured to receive connections from.

This can be caused by misconfiguration, which would result in lost messages. In a Kubernetes installation, it can also be caused by load balancers pinging the NE to make sure it is still alive. This is not a issue as long as it does not occur so frequently that the real connection cannot be made.

Correct the misconfiguration. If the issue is caused by a load balancer, decrease the load balancer ping frequency.

LM_ERROR

LM_ERROR .* [network_enabler_1:1:1:1(.*)] SigtranSocket::handle_input: link_name: Reading socket: Connection reset by peer.

The IP connection to the entity at the end of the link (STP or CAMEL Gateway) has been dropped because the process at the other end has deliberately dropped the connection or has stopped.

Messages on this link cannot be sent or received until this link comes back up.

Not an issue if this happened due to a known outage that is unlikely to happen again and other messages indicate that this link is now up. However, if the link is still down or this error happens multiple times, then investigate CAMEL Gateway or STP for issues.

LM_ERROR

LM_ERROR .* [network_enabler_1:1:1:1.*)] .* | M3uaSccpLink::unexpectedMessage: link_name: Received unknown SCCP message code .*

The named link has received an unknown type of SCCP message from the network. It has been discarded. This can be the result of load balancers or programs, such as curl, pinging the NE.

Probably not an issue assuming that these are not real SCCP messages that are getting discarded and there are not too many of them.

Stop pinging the NE using a program.

LM_ERROR

LM_ERROR 26517|26700 2024-07-15 03:21:16.962234 [network_enabler_1:1:1:1(5256.a1c8ac8)] CAM-ffff6795d1fa6b98-NA- 63c40392-NA,92,6598541076 | M3uaSccpLink::sccpParseError: NE101_STP1_MSS22: Failed to parse SCCP message: SCCP message is too short.

The named link has received an invalid SCCP message from the network. It has been discarded. This can be the result of load balancers or programs, such as curl, pinging the NE.

Probably not an issue assuming that these are not real SCCP messages that are getting discarded and there are not too many of them.

Stop pinging the NE using a program.

LM_ERROR

LM_ERROR .* [camel_gateway_1:1:1:1(.*)] MtxTcap::ToSccpTask::sendUnitDataMessageToNetwork: Error encoding TCAP message to Unitdata message:

MATRIXX failed to send a message because it cannot encode it using the ASN.1 basic encoding rules. This is caused by either a bug in the MATRIXX software or by a bug in the custom asn1_dictionary.xml file. A dump of the offending message is given on the next line.

This is serious if large numbers of these errors occur because it can lead to incomplete TCAP transactions.

Contact MATRIXX support with the following files attached to the ticket:

Full log file.
asn1_dictionary.xml file.
mtx_config.xml file.
create_config.info file.

LM_ERROR

LM_ERROR .* [camel_gateway_2:1:1:1(.*)] | M3uaLink::reportException: link_name: Sending Error: Protocol Error message <.*>

A message was sent to CAMEL Gateway that is not a valid SCCP message. This could be from a load balancer or someone using ping, curl, or some similar program. This could also be due to a misconfiguration that allows non-SCCP traffic to be sent to CAMEL Gateway.

This may not be serious as long as it really is a load balancer or person pinging CAMEL Gateway.

Investigate where these messages are coming from.

LM_ERROR

LM_ERROR.* [camel_gateway_1:1:2:1(.*)] .* MtxTcap::TcapInformation::isOverloaded: Overload detected by TCAP library due to rate exceeded.

The number of TCAP BEGINs per second received by CAMEL Gateway has exceeded the value specified by the maximum_number_of_begins_per_second specified in mtx_config.xml. New TCAP BEGIN messages are discarded for the remainder of the second to protect the system from overloading.

While the overload lasts, some BEGIN messages (that is, messages starting new calls) are rejected. Messages for existing calls (that is CONTINUE, END, and ABORT messages) are processed normally.

If an overload does not normally occur, then no action is necessary. However, if maximum_number_of_begins_per_second is set too low to cope with normal traffic, then consider increasing it, but be aware that new hardware may also be needed to cope with the extra traffic.

Note: The maximum_number_of_begins_per_second value is calculated based on the answer to several questions in create_config.info. For more information, see the discussion about CAMEL Gateway general configuration in MATRIXX Call Control Framework Integration.

LM_ERROR

LM_ERROR.* [camel_gateway_1:1:2:1(.*)] MtxTcap::TcapInformation::isOverloaded: Overload detected by TCAP library due to output queue too full.

This error only occurs with a CAMEL Gateway process running on the processing pod. The percentage of the queue from CAMEL Gateway to the Charging Server has exceeded the configured maximum_input_queue_percentage_full parameter defined in mtx_config.xml. CAMEL Gateway will reject TCAP BEGIN messages until the queue length decreases in order to protect the system from overload.

While the overload lasts, some BEGIN messages (that is, messages starting new calls) are rejected. Messages for existing calls (that is CONTINUE, END, and ABORT messages) are processed normally.

Perform an analysis to determine why the system is not performing as expected. Common causes include overly complicated pricing logic and slow or defective disks. Consider taking measures to reduce the load or add more hardware.

LM_ERROR

LM_ERROR.* [network_enabler_1:1:1:1(.*)] SigtranLink::handleTimeout:link_name: Heartbeat FAILURE, closing link immediately.

For the named link, an M3UA heartbeat has not been received in time, so the link has been closed. The NE continues trying to bring the link up, which could be a result of a mismatch of M3UA configuration parameters at either end of the link. This is more likely due to IP networking issues or a failure of the entity at the other end of the link.

No messages are sent or received from the link in question.

Investigate any IP network issues. Also, check that the M3UA settings on each end of the link match.

LM_ERROR

LM_ERROR.* [network_enabler_1:1:1:1(.*)] UnitDataLink::rxScmg:link_name: Ignoring SST for unknown PC<point code> from AI address_indicator, PC point_code, SSN 1.

The NE is receiving Subsystem test (SST) messages for a point code other than the configured point code of this NE.

Traffic from the point code sending the SST cannot be processed.

Check whether the NE is configured with the correct point code. Check the configuration of the entity that is sending the SST messages.

LM_WARN

LM_WARN .* [network_enabler_1:1:1:1(.*)] SigtranSocket::connected:link_name: Connect to IP_address:port failed: Connection refused.

The NE tried to establish a link but failed. This could indicate a misconfiguration or issue with the entity at the other end of the link.

Messages on this link cannot be sent or received until this link comes back up.

Check the configuration of the link. Check the health of the entity at the other end of the link.

LM_WARN

LM_WARN.* [camel_gateway_31:1:1:1(.*)] | SigtranLink::accept: link_name: already connected; rejecting second connection.

This error can also be logged by the NE if any of the NE's links are configured to listen and not to connect. The connection is rejected because the link is already connected. This can be a result of load balancers pinging the listening address and port.

If it is a load balancer pinging CAMEL Gateway or the NE, then this does not matter.

Check the SNMP/Prometheus stats to see if the link in question is UP and handling messages.

LM_WARN

LM_WARN .* [network_enabler_1:1:2:1(.*)] .* | DomainLookupTask::handleInput: Cache entry not found - waiting for domain resolution.

This happens when multiple sub-domains are used. An entry for a subscriber has not been found in the route cache on an NE machine. Normally, the transaction servers on the processing pods push subscriber information out to all the route caches when a subscriber is provisioned. One possibility for this error is that the subscriber is not in the database, indicating a mismatch between the subscribers provisioned on the HLRs and the subscribers provisioned in the MATRIXX database. Another possibility is that there was an interruption of communication between the transaction server and the route cache, preventing the record from being pushed to the route cache. The route cache now polls all the transaction servers to try to find the subscriber record.

This is not necessarily serious. The call still succeeds if the route cache finds the subscriber. If not, the call fails. If this happens a lot, then something is wrong with the system.

If this happens a lot, investigate which subscribers and domains are involved. Try to work out why the route cache is not pushing records for these subscribers.

LM_WARN

LM_WARN .* [network_enabler_1:1:1:1(.*)] SigtranLink::handleTimeout: link_name: Connect to IP_address:port timed out.

The NE has failed to establish a connection to the remote end of this link. This could be due to misconfiguration, IP network issues, or a fault with the entity at the other end of the link.

No messages are sent or received on this link.

Check that the correct address and port are configured. Check that the entity with that address and port is up and healthy. Make sure the IP address and port are reachable from the NE.

LM_WARN

LM_WARN .* [network_enabler_1:1:2:1(.*)] SigtranLink::rxAspupAck: link_name: Unexpected ASPUP ACK received.

This is probably the result of the NE and the entity the NE is talking to on this link both being configured to be an ASP.

No messages are sent or received on this link.

Make sure that the entity at one end of the link is configured to be an ASP and the other end is configured to be an SGP. Normally, the NE is configured to be an ASP and the STP is configured to be an SGP.

LM_WARN

LM_WARN .* [network_enabler_1:1:1:1(.*)] SigtranLink::rxAspacAck: link_name: Received unwanted ASPAC ACK for routing context(s) null.

This is probably the result of the NE and the entity the NE is talking to on this link both being configured to be an ASP.

No messages are sent or received on this link.

Make sure that the entity at one end of the link is configured to be an ASP and the other end is configured to be an SGP. Normally, the NE is configured to be an ASP and the STP is configured to be an SGP.

LM_INFO

LM_INFO .* [network_enabler_1:1:1:1.*)] SigtranLink::rxAspupAck: link_name: ASP is up.

This is the final stage in the NE taking a link up. The NE is acting as an ASP so this is probably a link to an STP.

If no errors or warnings are subsequently reported for this link, then the link is up and ready to take traffic. This message can be regarded as clearing any previous errors or warnings for this link.

None.

LM_INFO

LM_INFO .* [network_enabler_1:1:1:1(.*)] UnitDataLink::rxScmg: internal_link_name: Received SSA for PC .* SSN .*

CAMEL Gateway has just informed the NE that it is ready to accept traffic for the stated point code and SSN. The NE informs the STPs that it is ready to take traffic if this has not already been done.

Normal message seen at start-up suggesting that all is well.

None.

LM_INFO

LM_INFO .* [network_enabler_1:1:1:1(.*)] UnitDataLink::rxScmg:internal_link_name: Received SSP for PC .* SSN .*

CAMEL Gateway has just informed the NE that it is NOT ready to accept traffic for the stated point code and SSN. The NE informs the STPs that it is ready to take traffic if no other CAMEL Gateways are available to take traffic. However, if other CAMEL Gateways are available, then traffic is sent to these and it is not a problem.

This may indicate a problem. However, it may be that CAMEL Gateway has been shut down for planned maintenance. It is also normal for CAMEL Gateways serving the standby engine to send SSP messages because traffic should not be sent to the standby engine.

No action necessary as long as the SSP has been received as part of planned maintenance or from a CAMEL Gateway serving the standby engine.