Cloud image — running VyOS on AWS, GCP, Azure, and OpenStack

A VyOS box in a public cloud is a routing VM that boots from a pre-built image rather than from an ISO. The operator does not have console access in the traditional sense — there is no KVM, no serial cable. The first-boot configuration must travel with the box via the cloud provider’s metadata service, and the only way back from a mistake is to terminate the instance and start a new one. This lesson walks the four major clouds, the cloud-init payload the operator injects, and the failure modes the routing engineer must recognise before the cloud router peers with production.

How a cloud image boots

Every cloud image of VyOS follows the same lifecycle:

flowchart TB
  A[Operator: launch instance<br/>+ cloud-init user data] --> B[Cloud provider metadata]
  B --> C[VM boots VyOS 1.5 cloud image]
  C --> D[cloud-init reads user data]
  D --> E{Top-level key VyOS<br/>actually enables?}
  E -->|"vyos_config_commands<br/>or write_files"| F[Load the set lines into<br/>the candidate configuration]
  E -->|"anything else"| G[Image defaults only<br/>login with the launch key]
  F --> H[commit; save<br/>run automatically]
  G --> I[Operator: ssh vyos@EIP]
  H --> I

AWS EC2 deployment

VyOS is published on the AWS marketplace; which regions carry the AMI depends on the listing you subscribe to, so find the image ID in the region you are building in rather than copying one. The operator launches the AMI, selects a key pair, and provides the first-boot configuration as user data.

# User data for AWS (cloud-config form), VyOS 1.5
#cloud-config
vyos_config_commands:
  - set system host-name 'vyos-cloud-01'
  - set system name-server '1.1.1.1'
  - set system name-server '9.9.9.9'
  - set interfaces ethernet eth0 address 'dhcp'
  - set interfaces ethernet eth0 description 'OUTSIDE'
  - set interfaces ethernet eth1 address '10.0.0.2/24'
  - set interfaces ethernet eth1 description 'INSIDE'
  - set protocols bgp system-as '64512'
  - set protocols bgp neighbor 169.254.0.1 remote-as '64512'
  - set protocols bgp neighbor 169.254.0.1 address-family ipv4-unicast
  - set service ssh port '22'

Three things about that blob are VyOS-specific and worth reading twice.

The lines are CLI commands, not YAML structure. Each entry is exactly what you would type in configuration mode. If a line would be rejected at the CLI it is rejected here too, except that now nobody is watching the error scroll past.

The BGP tree is the 1.4+ shape. The local ASN is a leaf — set protocols bgp system-as '64512' — and peers hang off set protocols bgp neighbor <address>, not off the ASN. The pre-1.4 form set protocols bgp 64512 neighbor ... is rejected on 1.5. Creating the address-family ipv4-unicast node under the neighbour is what activates the peer for that family; there is no separate activate leaf to add.

SSH keys do not come from the user data. ssh_authorized_keys is a stock cloud-init key and VyOS does not enable it. The key pair you selected at launch reaches the vyos user through the provider’s instance metadata, which the VyOS datasource does read. An additional key is added the VyOS way, as another vyos_config_commands line:

  - set system login user vyos authentication public-keys operator@laptop key 'AAAAC3Nza...'
  - set system login user vyos authentication public-keys operator@laptop type 'ssh-ed25519'

GCP deployment

GCP passes cloud-init user data as an instance metadata key literally named user-data:

IMAGE_PROJECT=$(gcloud compute images list --filter=vyos \
  --format='value(project)' | head -1)

gcloud compute instances create vyos-router-01 \
  --image-family=vyos \
  --image-project="$IMAGE_PROJECT" \
  --machine-type=n2-standard-2 \
  --network-interface=subnet=outside \
  --network-interface=subnet=inside,private-network-ip=10.0.0.2 \
  --metadata-from-file=user-data=vyos-cloud-init.yaml \
  --zone=us-central1-a

Two details are load-bearing and neither is VyOS’s doing. The metadata key must be user-data — anything else is metadata the datasource never looks at. And the image family and project are whatever the marketplace listing you subscribed to publishes; read them with gcloud compute images list rather than copying a string out of a lesson, because they change between listings.

Azure deployment

Azure passes user data through --custom-data:

URN=$(az vm image list --publisher vyos --all \
  --query '[0].urn' --output tsv)

az vm create \
  --resource-group vyos-rg \
  --name vyos-router-01 \
  --image "$URN" \
  --size Standard_B2ms \
  --admin-username vyos \
  --ssh-key-values @vyos.pub \
  --custom-data vyos-cloud-init.yaml \
  --vnet-name vyos-vnet \
  --subnet outside

An Azure image URN is four colon-separated fields — publisher:offer:sku:version — so it is worth querying rather than guessing. A marketplace image also needs its terms accepted once per subscription (az vm image terms accept --urn "$URN") before the first az vm create will succeed.

OpenStack deployment

OpenStack is the one case where the operator usually uploads the image themselves, from the qcow2 VyOS publishes, and then launches it with --user-data:

openstack server create \
  --image vyos-1.5-cloud-amd64 \
  --flavor m1.small \
  --key-name operator \
  --user-data vyos-cloud-init.yaml \
  --network outside \
  --network inside \
  vyos-router-01

The --image argument is the name you gave the image at openstack image create time, so it is the one string in this lesson you genuinely control. Note also that the NIC order on the command line is the order the guest sees: the first --network becomes eth0.

How the result is validated

The first-boot configuration either committed successfully or it did not. The operator confirms via:

vyos@vyos-cloud-01:~$ show version

Version:          VyOS 1.5.0-circinus
Release train:    circinus
Release flavor:   generic

vyos@vyos-cloud-01:~$ show configuration

interfaces {
  ethernet eth0 {
      address dhcp
      description OUTSIDE
  }
  ethernet eth1 {
      address 10.0.0.2/24
      description INSIDE
  }
}
protocols {
  bgp {
      neighbor 169.254.0.1 {
          address-family {
              ipv4-unicast {
              }
          }
          remote-as 64512
      }
      system-as 64512
  }
}
system {
  host-name vyos-cloud-01
  name-server 1.1.1.1
  name-server 9.9.9.9
}

Read the protocols bgp stanza carefully, because it is the clearest confirmation that the box is on the 1.4+ tree: system-as is a leaf beside neighbor, and the neighbour is not nested inside an ASN node. A blob written against the pre-1.4 tree does not produce a different-looking stanza here — it produces no stanza at all, because the command was rejected.

If the configuration shows only the image defaults, either the handler never saw your payload or it refused it. The operator terminates the instance, fixes the payload, and relaunches.

How the cloud metadata service works

sequenceDiagram
  participant VyOS as VyOS guest
  participant Meta as Metadata service 169.254.169.254
  participant Cloud as Cloud control plane

  VyOS->>Meta: GET /latest/user-data
  Meta->>Cloud: Forward request
  Cloud-->>Meta: Return user data blob
  Meta-->>VyOS: User data
  VyOS->>VyOS: cloud-init parses
  VyOS->>VyOS: VyOS handler converts to config.boot
  VyOS->>VyOS: commit and save

Each cloud exposes its metadata service on link-local 169.254.169.254. From the guest shell (configure is not needed — this is ordinary Linux), the operator can fetch the same payload the datasource saw. On AWS with IMDSv2 required, that is two calls: get a token, then use it.

IMDS=http://169.254.169.254
TOKEN=$(curl -sX PUT "$IMDS/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" "$IMDS/latest/user-data"

What comes back is the literal blob you pasted at launch. If it is empty, the instance was launched without user data and no amount of debugging on the guest will change that. If it comes back but the configuration did not apply, the payload reached the box and the handler rejected it — which is a different problem, and the log that names it is /var/log/cloud-init.log:

sudo grep vyos /var/log/cloud-init.log

How it fails

The production failure modes the engineer must recognise:

• Malformed YAML in the user data. cloud-init logs the parse error and the box boots on the image defaults. The instance is reachable and looks healthy; it simply is not the router you described. sudo grep vyos /var/log/cloud-init.log names the line.
• A blob written for general-purpose cloud-init. runcmd, bootcmd, packages and users are not enabled on VyOS. The YAML is valid, cloud-init is happy, and nothing is configured. This failure produces no error anywhere — it is the one that costs an afternoon.
• A single rejected command taking the whole commit with it. vyos_config_commands is loaded as one candidate configuration and committed once. A typo in line 9 means lines 1-8 do not survive either. Treat the payload as atomic, and lint it by pasting it into a scratch VM before it goes near a launch template.
• The wrong NIC is eth0. VyOS presents Ethernet ports as eth0, eth1, … in the configuration tree regardless of what the kernel calls them, so a user-data line naming ens5 is simply an unknown interface. What actually bites in the cloud is ordering: whether the ENI you think of as OUTSIDE became eth0 or eth1 depends on the attachment order at launch, and a two-NIC router that gets it backwards puts its default route on the inside subnet. Check with show interfaces ethernet on the first launch, and pin the mapping with set interfaces ethernet eth0 hw-id <mac> when the template will be reused.
• Key pair mismatch. The instance is up, the routing is right, and the key the operator holds is not the key the box accepts. There is no console to fix it from. Recovery: relaunch.
• Subnet routing not configured. A cloud router launched into a subnet without route tables pointing back to the on-prem network sees the box but cannot reach the on-prem peer.
• Egress-only NAT not configured. A cloud box with no public IP cannot originate connections to package mirrors or time servers; add system image over HTTPS fails silently.

Rollback

A cloud image rollout is rolling-forward only. There is no rollback in the cloud-provider sense. The recovery path is:

1. Terminate the broken instance.
2. Launch a new instance with the fixed user data.
3. Detach the elastic IP / public IP from the broken instance and re-attach to the new one (if not yet released).
4. Verify BGP / IPSec / WireGuard adjacency on the new instance.

The state that matters — peering, routes, certificates — should be re-established from the new instance. The on-prem side sees a new IP but the same AS number and the same configuration logic.

Production discipline

Cross-course references

The Ansible course’s XL-Ansible-Bootstrap covers how to drive the cloud launch from an Ansible playbook. The Observability course’s XV-Observability-CloudExport covers shipping telemetry from cloud routers to a central Prometheus. The Linux course’s XXII-Linux-NetTroubleshoot covers how to debug cloud networking from the host perspective.

Quiz