Node group configuration — the scaling targets

Node groups are the scaling targets of the Cluster Autoscaler. The Cluster Autoscaler scales node groups; it does not manage individual nodes. This lesson walks the node group configuration on AWS, GCP, and Azure, and the auto-discovery tags.

The cloud-provider concepts

Each cloud provider has a different node group concept:

The Cluster Autoscaler integrates with each.

flowchart LR
    A[Cluster Autoscaler] --> B[AWS ASG]
    A --> C[GCP MIG]
    A --> D[Azure VMSS]
    B --> E[EC2 instances]
    C --> F[Compute Engine instances]
    D --> G[VM instances]

The AWS ASG

The ASG is the AWS node group:

resource "aws_autoscaling_group" "workers" {
  name                = "k8s-workers"
  vpc_zone_identifier = ["subnet-abc", "subnet-def"]
  min_size            = 3
  max_size            = 10
  desired_capacity    = 5

  launch_template {
    id      = aws_launch_template.workers.id
    version = "$Latest"
  }

  tag {
    key                 = "k8s.io/cluster-autoscaler/enabled"
    value               = "true"
    propagate_at_launch = true
  }
  tag {
    key                 = "k8s.io/cluster-autoscaler/cluster-name"
    value               = "my-cluster"
    propagate_at_launch = true
  }
}

The k8s.io/cluster-autoscaler/* tags are the auto-discovery mechanism. The Cluster Autoscaler finds ASGs with these tags.

The GCP MIG

The MIG is the GCP node group:

resource "google_compute_instance_group_manager" "workers" {
  name               = "k8s-workers"
  base_instance_name = "k8s-worker"
  zone               = "us-central1-a"
  target_size        = 5

  version {
    instance_template = google_compute_instance_template.workers.self_link
  }

  named_port {
    name = "http"
    port = 80
  }
}

The GCP node group is managed via the GCP console or gcloud.

The Azure VMSS

The VMSS is the Azure node group:

resource "azurerm_linux_virtual_machine_scale_set" "workers" {
  name                = "k8s-workers"
  resource_group_name = "k8s"
  location            = "eastus"
  sku                 = "Standard_D2s_v5"
  instances           = 5
  admin_username      = "azureuser"

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }
}

The Azure node group is managed via the Azure portal or az.

The instance types

The instance type determines the capacity per node:

# AWS
instance_type = "t3.large"  # 2 vCPU, 8 GB

# GCP
machine_type = "n1-standard-2"

# Azure
sku = "Standard_D2s_v5"

The instance type is per-node-group. A cluster typically has multiple node groups with different instance types:

flowchart LR
    A[Cluster Autoscaler] --> B[general-purpose workers]
    A --> C[memory-intensive workers]
    A --> D[GPU workers]
    B --> E[t3.large]
    C --> F[r5.2xlarge]
    D --> G[p3.2xlarge]

The Cluster Autoscaler scales the appropriate node group based on the pod’s resource requirements.

The min/max capacity

The cluster’s total capacity is bounded by the min/max of each node group:

workers-general: min=3, max=10, desired=5
workers-memory: min=0, max=5, desired=0
workers-gpu: min=0, max=2, desired=0

Total min: 3
Total max: 17

The Cluster Autoscaler respects the bounds. The cluster cannot scale below min or above max.

The auto-discovery

The auto-discovery mechanism is the link between the Cluster Autoscaler and the node groups:

# AWS ASG auto-discovery
- --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/cluster-name=my-cluster

# GCP MIG auto-discovery
- --node-group-auto-discovery=mig:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/cluster-name=my-cluster

# Azure VMSS auto-discovery
- --node-group-auto-discovery=azurerm:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/cluster-name=my-cluster

The Cluster Autoscaler scans the cloud provider for resources with the matching tags.

The expander

The expander strategy determines which node group to scale when multiple groups are candidates:

--expander=random
--expander=least-waste
--expander=most-pods
--expander=priority

• random: randomly select a node group.
• least-waste: select the node group that wastes the least resources.
• most-pods: select the node group that fits the most pods.
• priority: select the node group with the highest priority (configured via ConfigMap).

The default is random.

The cross-course references

• The Terraform course covers IaC-managed node groups.
• The AWS course covers ASG configuration.
• The Observability course covers autoscaler metrics.

Quiz

Production discipline

• Tag the node groups correctly. The auto-discovery tags are required.
• Set the min/max appropriately. The bounds must match the workload’s pattern.
• Use the right expander. least-waste is the production default.
• Test the scale-up and scale-down. Catch the issues before production.
• Monitor the autoscaler metrics. The node group size, the unschedulable pods.
• Document the node group config. The instance types, the bounds, the tags.

The node group configuration is the Cluster Autoscaler’s contract. Operating it well is configuring the bounds, the tags, and the expander.