Multi-tenancy patterns — namespaces as logical clusters

Multi-tenancy in Kubernetes comes in two flavours: shared cluster (many tenants in one cluster, isolated by namespace) and cluster-per-tenant (one cluster per tenant). This lesson covers the trade-offs, the four pillars of namespace-as- tenant, and the production patterns for tenant onboarding.

The two multi-tenancy models

flowchart LR
    subgraph SharedCluster[Shared cluster]
        NS1[Namespace: tenant-a]
        NS2[Namespace: tenant-b]
        NS3[Namespace: tenant-c]
    end
    subgraph CP1[Cluster 1]
        CNA[Namespace: tenant-a]
    end
    subgraph CP2[Cluster 2]
        CNB[Namespace: tenant-b]
    end
    subgraph CP3[Cluster 3]
        CNC[Namespace: tenant-c]
    end

Two patterns:

• Shared cluster (namespace-as-tenant): many tenants in one cluster, isolated by namespace. Cheaper; more efficient resource use; harder to enforce isolation.
• Cluster-per-tenant: each tenant gets its own cluster. Stronger isolation; more expensive; harder to operate at scale.

The choice depends on:

• Trust level: untrusted tenants need cluster-per-tenant. Semi-trusted tenants can share a cluster with strong namespace isolation.
• Compliance: PCI-DSS, HIPAA may require cluster separation.
• Cost: cluster-per-tenant costs more (separate control plane, separate node pools).

For most production estates, namespace-as-tenant with strong isolation is the right model.

The four pillars of namespace-as-tenant

flowchart LR
    Namespace[Namespace: tenant-a] --> RBAC[RBAC: per-tenant Roles]
    Namespace --> Quota[ResourceQuota: tenant total]
    Namespace --> PSS[PSS: restricted profile]
    Namespace --> NP[NetworkPolicy: default deny]
    RBAC --> Isolation[Tenant isolation]
    Quota --> Isolation
    PSS --> Isolation
    NP --> Isolation

Four controls combine to make a namespace a real tenant:

1. RBAC: each tenant has its own Role + RoleBinding; developers see only their namespace.
2. ResourceQuota: each tenant has its own quota; cannot consume the cluster.
3. PSS restricted: Pods in the tenant’s namespace must comply with restricted.
4. NetworkPolicy default-deny: Pods in the tenant’s namespace cannot reach other tenants.

Without all four, isolation is incomplete.

Tenant onboarding checklist

For each new tenant namespace:

# 1. Create the namespace
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-a-prod
  labels:
    kubernetes.io/metadata.name: tenant-a-prod
    # 4. PSS restricted
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest

---
# 2. LimitRange (defaults)
apiVersion: v1
kind: LimitRange
metadata:
  name: default
  namespace: tenant-a-prod
spec:
  limits:
  - type: Container
    default: {cpu: 500m, memory: 512Mi}
    defaultRequest: {cpu: 100m, memory: 128Mi}

---
# 3. ResourceQuota
apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-a-quota
  namespace: tenant-a-prod
spec:
  hard:
    requests.cpu: "32"
    requests.memory: 64Gi
    limits.cpu: "64"
    limits.memory: 128Gi
    pods: "200"
    persistentvolumeclaims: "20"

---
# 4. NetworkPolicy default-deny
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: tenant-a-prod
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]

---
# 5. NetworkPolicy allow internal + DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-internal
  namespace: tenant-a-prod
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]
  ingress:
  - from:
    - podSelector: {}
  egress:
  - to:
    - podSelector: {}

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: tenant-a-prod
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
    ports:
    - protocol: UDP
      port: 53

---
# 6. RBAC for the tenant
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: tenant-a-developer
  namespace: tenant-a-prod
rules:
- apiGroups: [""]
  resources: ["pods", "services", "configmaps"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tenant-a-developers
  namespace: tenant-a-prod
subjects:
- kind: Group
  name: tenant-a-developers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tenant-a-developer
  apiGroup: rbac.authorization.k8s.io

This is a complete tenant onboarding: namespace, defaults, quota, PSS, NetworkPolicy, RBAC. Production discipline: have a templated onboarding runbook; every new tenant gets all four pillars.

Tenant patterns

Per-environment:

tenant-a-dev
tenant-a-staging
tenant-a-prod

Each tenant has multiple environments; the environments are separate namespaces.

Per-team per-environment:

team-a-prod
team-b-prod
team-a-staging
team-b-staging

Multiple teams, multiple environments; isolation between teams.

Per-application:

tenant-a-app-web
tenant-a-app-db
tenant-a-app-cache

Some teams prefer per-application namespaces for finer isolation. The trade-off: more namespaces to manage.

Cluster-per-tenant patterns

For high-trust boundaries (PCI-DSS, regulated industries), cluster-per-tenant is the right choice:

flowchart LR
    Tenant1[Tenant 1] --> Cluster1[Cluster 1: dedicated]
    Tenant2[Tenant 2] --> Cluster2[Cluster 2: dedicated]
    Tenant3[Tenant 3] --> Cluster3[Cluster 3: dedicated]

Each tenant:

• Has its own API server, control plane, etcd.
• Has its own node pool (or shared cloud account).
• Cannot reach other tenants’ namespaces because they don’t exist on this cluster.

The trade-offs:

• Pros: complete isolation; simpler security model; easier compliance.
• Cons: more expensive; harder to operate (multiple control planes); less efficient resource use.

For most production estates, this is overkill. Use it for regulated workloads or untrusted tenants.

Production discipline

• Use namespace-as-tenant for trusted tenants. Cheaper, more efficient; logical isolation is sufficient.
• Use cluster-per-tenant for regulated workloads. PCI, HIPAA, etc. require physical isolation.

Tenant onboarding:

1. Create namespace with PSS labels.
2. Apply LimitRange with defaults.
3. Apply ResourceQuota.
4. Apply default-deny NetworkPolicy + allow internal + allow DNS.
5. Apply RBAC Role + RoleBinding for the tenant’s group.
6. Document the tenant’s quotas and access.
7. Test: deploy a Pod in the namespace; verify it cannot reach other namespaces.

Tenant offboarding:

1. Capture state (kubectl get all -o yaml).
2. Delete workloads (kubectl delete all).
3. Delete PVCs (with data backup if needed).
4. Delete the namespace (kubectl delete ns).
5. Verify RBAC bindings are removed.
6. Update documentation.

Auditing tenant isolation

# Find namespaces without all four pillars
for ns in $(kubectl get ns -o jsonpath='{.items[*].metadata.name}'); do
  has_rbac=$(kubectl get rolebindings,clusterrolebindings -n $ns -o jsonpath='{.items}' | jq 'length')
  has_quota=$(kubectl get resourcequota -n $ns -o jsonpath='{.items}' | jq 'length')
  has_pss=$(kubectl get ns $ns -o jsonpath='{.metadata.labels.pod-security\.kubernetes\.io/enforce}')
  has_np=$(kubectl get networkpolicy -n $ns -o jsonpath='{.items}' | jq 'length')
  echo "$ns: rbac=$has_rbac quota=$has_quota pss=$has_pss np=$has_np"
done

Production discipline: every production namespace has RBAC, Quota, PSS, and NetworkPolicy. Auditing weekly surfaces gaps.

Cross-course references

• The Linux course part XXV-Linux-Firewall covers iptables/nftables; NetworkPolicy is the cluster-level equivalent for namespaces.
• The Proxmox course part XXVII-Proxmox-MultiCluster covers multi-cluster architectures; the cluster-per-tenant pattern is similar.
• The Ansible course part XXXV-Ansible-Scripting covers tenant onboarding; the cluster patterns are similar.

Quiz