Self-hosted runners — control, operational cost, and the security cost

A self-hosted runner is a host the team owns. The forge does not create the VM, does not patch the kernel, does not decide what tools are installed. The team does. In return, the team accepts the operational work: registering the runner, keeping it online, patching it, watching it, and rebuilding the environment when something drifts.

The reason teams accept the work is the control: the runner can sit in a private VPC, can have GPUs, can have tools the curated GitHub image does not ship, and can talk to internal services the public internet cannot reach.

Registration and lifecycle

The official self-hosted runner application is distributed as a tarball that contains the runner binary, the configuration script (config.sh), the run script (run.sh), and a service helper (svc.sh).

# Create a directory and unpack the runner
mkdir -p /opt/actions-runner && cd /opt/actions-runner
RUNNER_VERSION=2.336.0  # course teaching target; pin the exact reviewed release
tar xzf "./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz"

# Configure the runner against a repository (or org)
./config.sh --url "$REPO_URL" --token "$REGISTRATION_TOKEN"

# Run interactively for testing
./run.sh

The two scripts separate the two jobs the runner does:

• config.sh registers the runner with the forge using a short-lived registration token. It writes a .runner file and (optionally) a .credentials file containing the long-lived runner token used for re-authentication.
• run.sh starts the runner process. It polls the forge for jobs, downloads them, and executes them.

flowchart LR
    A["config.sh\n(once, per host)"] --> B[".runner file"]
    B --> C["run.sh\n(long-running)"]
    C --> D["Poll forge for jobs"]
    D --> E["Execute job steps"]
    E --> F["Report status"]

For production use, run.sh is wrapped by svc.sh, which installs and starts a systemd service so the runner survives reboots:

# Install as a systemd service running as the current user
sudo ./svc.sh install
# Start it
sudo ./svc.sh start
# (status, stop, uninstall are also available)
sudo ./svc.sh status

What self-hosted gives you

flowchart TB
    subgraph VPC["Private VPC"]
        R["Self-hosted runner"]
        R -->|"internal network"| ART["Internal artifact\nregistry"]
        R -->|"PrivateLink"| AWS["AWS APIs"]
        R -->|"VPN"| ONP["On-prem\ndatacenter"]
    end

• Private network access. The runner can reach services inside a private VPC, an on-prem datacenter, or a hybrid network that the public internet cannot.
• Custom tooling. Whatever the team needs — a specific compiler version, a proprietary SDK, a licensed binary — is installable on the host.
• Specialised hardware. GPUs, FPGAs, ARM cores, large memory, local NVMe, anything the host has.
• Persistent state across jobs. A cache directory, a prebuilt toolchain, a warmed-up Docker layer.

The operational cost

Every benefit is also an obligation:

• Patching. The runner host needs OS patches, kernel patches, and Docker/engine patches on the same cadence as every other production host.
• Monitoring. The runner process must be monitored; a runner that has silently stopped is a workflow that has silently queued.
• Scaling. Capacity is the team’s responsibility. If a burst of workflows queues, the team has to add runners.
• Environment drift. Without a rebuild policy, a runner’s tools and state drift over weeks; reproducibility across runners degrades.

The security cost: persistent state

This is the trade-off that catches teams out. A self-hosted runner is, by default, persistent: the same host runs job after job after job. A GITHUB_TOKEN from job 1 can leak into the filesystem and be read by job 2. A malicious step in job 3 can install a backdoor that job 4 inherits.

flowchart LR
    J1["Job 1\n(writes secret to /tmp)"] --> H["Persistent host"]
    J2["Job 2\n(reads /tmp, finds the secret)"] --> H
    J3["Job 3\n(runs arbitrary code)"] --> H
    J4["Job 4\n(backdoor from job 3 still there)"] --> H

The forge’s recommendation is explicit: self-hosted runners should be ephemeral, single-use, or isolated per workflow. A long-lived self-hosted runner that runs untrusted code is not safe for production secrets.

Mitigations

The persistent-state risk is real but mitigable:

1. Run untrusted code on a separate pool. Fork PRs land on runners tagged untrusted; the trusted pool runs only internal-PR jobs.
2. Rebuild the runner image frequently. Treat runners like immutable infrastructure: rebuild the AMI / VM image weekly, redeploy.
3. Use ephemeral runners (Part XL-03). The runner exists for one job and is destroyed.
4. Restrict the runner’s blast radius. Network egress, IAM role, filesystem permissions, and runtime users should all be tight.

Production discipline

1. Document the runner as production infrastructure. It gets the same monitoring, patching, and alerting as a database.
2. Tag runners by purpose. prod, staging, untrusted, gpu, arm — clear labels prevent the wrong job landing on the wrong host.
3. Treat secrets as if the runner will be compromised. Use OIDC short-lived credentials, scope by job, and audit what is mounted.
4. Rebuild the runner image on a schedule. Drift is the silent failure mode of long-lived self-hosted runners.

Windows runner operations are not Linux operations

Windows self-hosted runners use the same GitHub control plane but have a different host boundary. Register them through the supported config.cmd interface and, for an approved persistent exception, install the service via config.cmd --runasservice (with --windowslogonaccount and --windowslogonpassword, or the interactive prompts) — the svc.sh helper is Linux/macOS only and there is no svc.cmd on Windows. Manage the service with Get-Service/Start-Service/Stop-Service on actions.runner.*, and remove the runner with config.cmd remove --token. Run the service as a dedicated least-privilege account or gMSA; constrain the runner and work roots with NTFS ACLs; test the corporate proxy and private root CA from the service identity; and select shell: pwsh explicitly in workflows.

Windows-specific evidence includes Service Control Manager state, Application event logs, WinHTTP and process proxy settings, the machine certificate store, PowerShell execution policy, antivirus/EDR actions, long-path policy, .gitattributes line-ending behavior, case-only paths, and directory reparse points. Do not translate Bash commands mechanically or assume an administrator’s interactive TLS test represents the service account. Lab 31 turns these differences into a PowerShell operations and phase-two evidence exercise.

Cross-course references

• Linux for Production Sysadmins - Part XXXIII (NetHard) covers the egress controls that a self-hosted runner requires.
• Ansible for Production Sysadmins - Part XXXVII (RepoArch) covers the runner-pools pattern.
• Terraform for Production Sysadmins - Part XLII (TerraformCloud) discusses alternative runners such as Terraform Cloud’s execution environment.

Quiz