Rancher — fleet management with downstream clusters

Rancher is a Kubernetes fleet management platform that centralises cluster operations: import, RBAC, projects, GitOps, and monitoring across multiple clusters. This lesson walks the Rancher architecture, the import workflow, the project model, Fleet GitOps, and the operational discipline.

The Rancher architecture

flowchart LR
    A[Rancher server] --> B[Downstream cluster 1]
    A --> C[Downstream cluster 2]
    A --> D[Downstream cluster 3]
    A --> E[Fleet controller]
    E --> F[Git repository]
    F -->|reconciles| B
    F -->|reconciles| C
    F -->|reconciles| D
    A --> G["Project / Namespace model"]
    G --> H[Unified RBAC]

The components:

• Rancher server — a central Kubernetes cluster running the Rancher management components.
• Downstream clusters — Kubernetes clusters managed by Rancher (EKS, AKS, GKE, K3s, RKE2, custom).
• Cluster API agent — runs in each downstream cluster; proxies API calls to the Rancher server.
• Fleet controller — the GitOps component; reconciles workloads across clusters from Git.
• Project / Namespace model — the tenancy model that groups namespaces across clusters.

The cluster import workflow

flowchart TD
    A[Existing Kubernetes cluster] --> B{Rancher agent install}
    B --> C[cluster-agent Deployment]
    B --> D[fleet-agent Deployment]
    C --> E[Cluster registered in Rancher]
    D --> F[GitOps enabled]
    E --> G["Manage via Rancher UI/API"]

The import:

1. Create the cluster in Rancher (UI or API).
2. Rancher generates a registration manifest with cluster-admin token.
3. Apply the manifest to the downstream cluster.
4. The cluster-agent and fleet-agent Deployments are installed in the downstream cluster.
5. The cluster registers with the Rancher server.

# On the downstream cluster
kubectl apply -f registration-manifest.yaml

After registration, the downstream cluster appears in the Rancher UI and can be managed centrally.

The project model

flowchart LR
    A[Rancher project] --> B[Cluster 1 namespace]
    A --> C[Cluster 2 namespace]
    A --> D[Cluster 3 namespace]
    E["RBAC: project member"] --> A

A Rancher project groups namespaces across clusters. A user with project-level RBAC can manage resources in all the project’s namespaces, regardless of which cluster they live on. This is the multi-cluster tenancy model.

The model:

• Cluster — the Kubernetes cluster.
• Project — a logical grouping of namespaces, possibly across clusters.
• Namespace — the Kubernetes namespace within a cluster.

Rancher RBAC is layered on top of Kubernetes RBAC: project members have roles; roles translate to Kubernetes Roles/ClusterRoles in the namespaces.

Fleet GitOps

flowchart LR
    A[Git repository] --> B[Fleet controller]
    B --> C["Cluster 1: apply manifests"]
    B --> D["Cluster 2: apply manifests"]
    B --> E["Cluster 3: apply manifests"]
    F[Cluster selector] --> B
    G[Bundle] --> B

Fleet is Rancher’s GitOps engine. A GitRepo resource points to a Git repository; Fleet watches the repository and applies manifests to downstream clusters based on labels and selectors.

A typical Fleet deployment:

apiVersion: fleet.cattle.io/v1alpha1
kind: GitRepo
metadata:
  name: prod-app
  namespace: fleet-default
spec:
  repo: https://github.com/example/prod-app
  branch: main
  paths:
    - manifests/
  targets:
    - clusterSelector:
        matchLabels:
          env: prod

The selector env: prod matches downstream clusters with that label. Fleet applies the manifests to those clusters only.

The operational failure modes

Rancher fails in production for predictable reasons:

• Rancher server unavailable. Without the Rancher server, downstream clusters cannot be managed from the UI/API. The clusters themselves continue to run; Rancher is the management plane, not the data plane.
• Cluster-agent disconnect. A downstream cluster’s cluster-agent loses connection to the Rancher server. The cluster appears offline in Rancher but is still functional.
• RBAC misconfiguration. Project-level RBAC translated to Kubernetes RBAC is misconfigured; users cannot access expected namespaces.
• Fleet GitRepo out of sync. A GitRepo with a bad branch or path causes Fleet to fail to apply manifests. The downstream clusters drift.
• Rancher version skew. A Rancher server upgrade that is not followed by cluster-agent upgrades produces API incompatibilities.

Quiz

Production discipline

Rancher in production rests on five non-negotiable elements:

• HA Rancher server. A 3-node Rancher server is the minimum for production. A single-node server is a single point of failure.
• Back up the Rancher server’s cluster state. The Rancher server’s own cluster state (clusters, projects, RBAC, GitRepos) must be backed up. Losing it means re-importing every downstream cluster.
• Monitor cluster-agent connectivity. A disconnected cluster-agent appears offline in Rancher; alert on disconnections.
• Test Rancher version upgrades. A Rancher upgrade requires cluster-agent upgrades; mismatched versions produce API errors.
• Document the Fleet GitOps topology. The GitRepo selectors, the bundle structure, and the cluster labels must be in the runbook.

Rancher is the management plane. Its availability determines whether downstream clusters can be managed centrally. The discipline is to HA the management plane and to treat it as production infrastructure.