How to resolve connection timeout issue in Kubernetes Ingress

When building infrastructure at scale and you have to handle billions of data, there are a lot of places where we need to optimize it, for better performance and lower server costs.

We had similar problems after deploying our multi-regional infrastructure. In DigitalOcean we use load-balancers service in order to forward requests to clusters. In user-based services, we don’t have to worry about connection timeout because in most cases 30 seconds timeout isn’t a problem.

But in ad tech, maintaining a connection from a browser for that long stresses out load balancers. Because even no-brainer network load balancers also have limitations. For example in DigitalOcean load balancers can handle 10,000 connections concurrently per node, which is a lot! But in AdTech you have to finish serving ads within milliseconds time. So holding on to a connection for 30,000 milliseconds on default where you don’t really need more than 60 milliseconds is just nasty.

We engineers when try to solve using already defined values, or options that might be provided by nginx ingress. But I think going back to basics can solve this. Just we need to add, how we used to write configuration as snippets in nginx config.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: cluster-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    service.beta.kubernetes.io/do-loadbalancer-name: "load-balancer"
    service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
    nginx.ingress.kubernetes.io/server-snippet: "keepalive_timeout 1s; client_body_timeout 1s;"

Add config to nginx.ingress.kubernetes.io/server-snippet just like I did and you can control your session timeouts.

This is a result that I’ve produced in tests, instant connection drop. Hopefully, this helps!