Cluster Autoscaler integration with cloud providers — the contract

The Cluster Autoscaler integrates with cloud providers via cloud-specific IAM permissions and API contracts. A misconfigured IAM is the most common cause of cluster autoscaler failures. This lesson walks the IAM permissions, the API contracts, and the failure modes.

The AWS IAM

The Cluster Autoscaler requires IAM permissions to manage ASGs:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeAutoScalingInstances",
        "autoscaling:DescribeLaunchConfigurations",
        "autoscaling:DescribeTags",
        "autoscaling:SetDesiredCapacity",
        "autoscaling:TerminateInstanceInAutoScalingGroup",
        "ec2:DescribeLaunchTemplateVersions"
      ],
      "Resource": "*"
    }
  ]
}

The IAM policy is attached to the Cluster Autoscaler’s ServiceAccount.

flowchart LR
    A[Cluster Autoscaler] --> B[IAM ServiceAccount]
    B --> C[AWS STS]
    C --> D[ASG API]

The IAM ServiceAccount

The IAM ServiceAccount is the EKS service account:

resource "aws_iam_role" "cluster-autoscaler" {
  name = "cluster-autoscaler"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Federated = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:oidc-provider/${var.oidc_provider}"
        }
        Action = "sts:AssumeRoleWithWebIdentity"
        Condition = {
          StringEquals = {
            "${var.oidc_provider}:sub" = "system:serviceaccount:kube-system:cluster-autoscaler"
          }
        }
      }
    ]
  })
}

The role is assumed by the ServiceAccount via WebIdentity.

The Cluster Autoscaler deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
    spec:
      serviceAccountName: cluster-autoscaler
      containers:
      - name: cluster-autoscaler
        image: registry.k8s.io/autoscaling/cluster-autoscaler:v1.34.x

The serviceAccountName is the IAM-bound ServiceAccount.

The GCP IAM

The GCP Cluster Autoscaler requires IAM permissions:

resource "google_service_account" "cluster-autoscaler" {
  account_id   = "cluster-autoscaler"
  display_name = "Cluster Autoscaler"
}

resource "google_project_iam_member" "cluster-autoscaler" {
  project = var.project_id
  role    = "roles/compute.instanceGroupManager"
  member  = "serviceAccount:${google_service_account.cluster-autoscaler.email}"
}

The GCP IAM role is compute.instanceGroupManager.

The Azure IAM

The Azure Cluster Autoscaler requires a managed identity:

resource "azurerm_user_assigned_identity" "cluster-autoscaler" {
  name                = "cluster-autoscaler"
  resource_group_name = "k8s"
  location            = "eastus"
}

resource "azurerm_role_assignment" "cluster-autoscaler" {
  scope                = data.azurerm_virtual_machine_scale_set.workers.id
  role_definition_name = "Virtual Machine Contributor"
  principal_id         = azurerm_user_assigned_identity.cluster-autoscaler.principal_id
}

The Azure role is Virtual Machine Contributor.

The cloud-specific tags

The auto-discovery tags are cloud-specific:

The tags are the contract between the autoscaler and the cloud provider.

The failure modes

The common failure modes:

Missing IAM permissions

cluster-autoscaler: failed to scale up node group: AccessDenied

The autoscaler cannot call the ASG API. Verify the IAM policy.

Missing tags

cluster-autoscaler: no node groups discovered

The autoscaler cannot find any ASGs with the tags. Verify the ASG tags.

Wrong cluster name

cluster-autoscaler: discovered ASG my-cluster-workers but cluster-name mismatch

The ASG’s tag does not match the autoscaler’s cluster name. Verify the cluster name.

Quota exceeded

cluster-autoscaler: scaling up node group: quota exceeded

The cloud account has hit a quota. Verify the cloud quotas.

flowchart LR
    A[Cluster Autoscaler] --> B{Scale up attempt}
    B -->|IAM denied| C[Check IAM policy]
    B -->|Tags missing| D[Check ASG tags]
    B -->|Cluster name mismatch| E[Check cluster name]
    B -->|Quota exceeded| F[Check cloud quotas]

The debug

# Substitute your own value before running:
AUTOSCALER_ROLE_ARN=arn:aws:iam::123456789012:role/cluster-autoscaler

# Verbose logs
kubectl logs -n kube-system deploy/cluster-autoscaler -v=4

# IAM diagnosis
aws sts get-caller-identity --role-arn "$AUTOSCALER_ROLE_ARN"

# ASG tags
aws autoscaling describe-tags --filters "Name=key,Values=k8s.io/cluster-autoscaler/enabled"

The debug is per-cloud.

Cross-course references

• The AWS course covers IAM configuration.
• The Terraform course covers IaC-managed IAM.
• The Observability course covers autoscaler metrics.

Quiz

Production discipline

• Verify the IAM permissions. The policy must grant the autoscaler the necessary permissions.
• Use the right tags. The auto-discovery tags are required.
• Monitor the cloud quotas. Request quota increases in advance.
• Test the integration. Run staging to verify the autoscaler can scale up.
• Document the IAM policy. The role, the permissions, the trust relationship.
• Audit the integration. Periodic check of the IAM and the tags.

The cloud integration is the Cluster Autoscaler’s contract. Operating it well is verifying the IAM, the tags, and the quotas.