Multiple Clusters with Dynamic Provisioning

Deploying across multiple clusters on platforms such as Amazon EKS is more difficult because you do not know the external addresses ahead of time. This example uses Helm to create the load balancer services (with the required annotations), allow the IP addresses to be dynamically provisioned, and then updates the configuration after IP addresses have been assigned.

This example assumes you have two Kubernetes clusters, cluster 1 and cluster 2, and a kubeconfig file configured so that you can access both clusters from one location. You can specify which cluster the kubectl and helm commands should target by using the --context and --kube-context options. The kubeconfig file is configured so that you can access cluster 1 and cluster 2 using context1 and context2.

The following is the distribution of Topology Operator components across clusters and namespaces in this example:

  • The masters and agents are in cluster 1 namespace matrixx-operators.
  • The agents in cluster 2 are in namespace matrixx-operators.
  • MATRIXX Engine s1e1 components are in cluster 1 namespace matrixx-engine-s1.
  • Engine s1e2 components are in the cluster 2 namespace matrixx-engine-s1.
Note: On Amazon EKS:
  • Engine processing and publishing pods require network load balancers.
  • Topology agent pods can use network load balancers or application load balancers.
The following are examples of network load balancer service annotations:
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
The following is an example of an application load balancer service annotation:
kubernetes.io/ingress.class: alb