iptables Configuration

For the Network Enablers (NEs) to connect successfully, Kubernetes must be in full control of the TCP connections.

If NE connections to the engine are timing out, it could be because Kubernetes is not in full control of the TCP connections. This can occur, for example, when Docker is already running on the node or when a CNI plug-in has been configured incorrectly. Messages similar to the following indicate that the NE is timing out:

LM_WARN  136|142 2020-06-19 12:13:00.643040 [network_enabler_1:1:1:1(5200.75793)] | SigtranLink::handleTimeout: blade_1_1_1: Connect to 10.102.103.213:29051 timed out

If this is the case, enter the following command to check the iptables forwarding rules on Kubernetes nodes:

$ sudo iptables -L FORWARD

For example:

$ sudo iptables -L FORWARD
Chain FORWARD (policy DROP)
target     prot opt source               destination         
KUBE-FORWARD  all  --  anywhere             anywhere             /* kubernetes forwarding rules */
KUBE-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes service portals */
DOCKER-USER  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
cali-FORWARD  all  --  anywhere             anywhere

It is important that the KUBE-FORWARD and KUBE-SERVICES Kubernetes rules are at the top of the rules list. If the Docker rules are before them, it could be because Docker was started before Kubernetes.

You can correct this situation by running the following commands:

$ sudo systemctl stop docker.service
$ sudo systemctl disable docker.service
$ sudo iptables -F
$ sudo systemctl restart kubelet
$ sudo systemctl restart docker

Note: Kubernetes usually starts Docker as part of its start-up and makes sure it starts at the right point.

If you are using Calico and the Calico rules appear before Kubernetes, then set the Calico configuration to append its iptables rules as follows, and then restart the Calico pods:

kubectl --namespace kube-system edit ds calico-node
 #      Locate the spec: > template: > spec: > containers: section inside the YAML file and change ChainInsertMode by adding the following code block
- name: FELIX_CHAININSERTMOD
  value: Append

Note: You must ensure that Kubernetes rules have priority over any services or plug-ins that are present.