NodePort and LoadBalancer — the external reachability path

NodePort and LoadBalancer Services expose the cluster to external traffic. The diagnostic is the node port, the cloud load balancer, and the cloud provider’s integration. The remediation is to fix the external reachability. The discipline is the same canonical flow extended with the external layer.

The NodePort Service

A NodePort Service exposes the Service on a static port (NodePort, default range 30000-32767) on every node’s IP. The node port is the cluster’s external interface.

flowchart LR
    A[External] --> B[NodePort: <node-ip>:<node-port>]
    B --> C[ClusterIP]
    C --> D[EndpointSlice]
    D --> E[Pod IP]

The NodePort flows: external traffic → node port → ClusterIP → EndpointSlice → Pod IP.

The LoadBalancer Service

A LoadBalancer Service provisions a cloud-provider load balancer (e.g., AWS NLB, GCP TCP LB) that fronts the NodePort on the cluster’s nodes. The cloud LB is the external interface.

flowchart LR
    A[External] --> B[Cloud LB: <external-ip>]
    B --> C[NodePort: <node-ip>:<node-port>]
    C --> D[ClusterIP]
    D --> E[EndpointSlice]
    E --> F[Pod IP]

The LoadBalancer flows: external traffic → cloud LB → node port → ClusterIP → EndpointSlice → Pod IP.

The diagnostic

The canonical diagnostic for NodePort and LoadBalancer:

# 1. Get the Service's spec
kubectl get service billing -n prod -o yaml

# 2. Check the Endpoints
kubectl get endpointslices -n prod -l kubernetes.io/service-name=billing

# Substitute your own values before running:
NODE_IP=192.0.2.11
NODE_PORT=30080
EXTERNAL_IP=203.0.113.40
EXTERNAL_PORT=80

# 3. Test the NodePort from inside the cluster
kubectl run -it --rm --restart=Never --image=curlimages/curl test -- \
  curl -v http://"$NODE_IP":"$NODE_PORT"

# 4. Test the LoadBalancer from outside the cluster
curl -v http://"$EXTERNAL_IP":"$EXTERNAL_PORT"

# 5. Check the cloud provider's resources
kubectl describe service billing -n prod | grep -A5 "LoadBalancer Ingress"

The diagnostic is the external reachability.

NodePort failures

The most common NodePort failures:

• NodePort not exposed. The node’s firewall is blocking the node port. The remediation is to open the firewall.
• Port range exhausted. The NodePort range is exhausted (30000-32767 has 2768 ports).
• Node not reachable. The node’s IP is not reachable from the external network (cloud security group, on-prem firewall).

# Substitute your own values before running:
SOURCE_POD=debug-6c9f4d5b7d-hq2vn
NODE=node-01.example.com
NODE_IP=192.0.2.11
NODE_PORT=30080

# Test the NodePort from inside the cluster
kubectl exec -it "$SOURCE_POD" -- curl -v http://"$NODE_IP":"$NODE_PORT"

# Test from the node itself
ssh "$NODE" "curl -v http://localhost:$NODE_PORT"

# Test from outside
curl -v http://"$NODE_IP":"$NODE_PORT"

LoadBalancer failures

The most common LoadBalancer failures:

• Cloud LB not provisioned. The cloud provider’s controller is failing to provision the LB. The remediation is to check the controller’s logs.
• LB health check failing. The cloud LB’s health check is failing, so the LB is not routing traffic to any node. The remediation is to fix the health check.
• Backend security group/NetworkPolicy. The cloud LB’s backend security group is blocking traffic to the node port. The remediation is to open the security group.

# Check the cloud LB's status
kubectl describe service billing -n prod | grep -A10 "LoadBalancer Ingress"

# Check the cloud LB's health check
# (cloud-specific command, e.g., aws elbv2 describe-target-health)

The remediation

The remediation depends on the cause:

• NodePort not exposed. Open the firewall on the node.
• Cloud LB not provisioned. Restart the cloud provider’s controller.
• LB health check failing. Fix the health check configuration.
• Backend security group blocking. Open the security group.

The remediation is the external reachability.

Production discipline

A NodePort or LoadBalancer failure is the cluster’s hypothesis. The discipline is to walk the canonical flow extended with the external layer, identify the failing layer, apply the remediation. The flow is the diagnostic; the external reachability is the recovery.

• Test the NodePort from three vantage points. From a Pod, from the node itself over localhost, and from outside the cluster — the first hop that fails names the failing layer.
• Test the LoadBalancer from outside the cluster. kubectl describe service shows whether the cloud controller ever populated LoadBalancer Ingress.
• Check the cloud LB’s health check. A failing health check drops every node from the backend, so the LB routes to nothing even when the node port itself is open.

Quiz