Prometheus exporter on OPNsense — getting the firewall into a time-series database · OPNsense

OPNsense’s built-in graphs are useful for one operator at one moment. The moment the operator wants to graph a metric across thirty days, correlate firewall-health with network-health, or alert on a threshold that crosses a boundary — the built-in graphs are not enough. The answer is a time-series database. The most common one in production today is Prometheus, and OPNsense has a plugin that produces metrics in the Prometheus format: os-opnsense-exporter.

This lesson covers what the exporter exposes, how the Prometheus scrape is configured, what the label-cardinality traps are, and how the exporter’s metrics relate to the underlying PF, sysctl, and FreeBSD sources.

What the exporter exposes

The os-opnsense-exporter plugin runs a small HTTP server on a configurable port (default 9100) on the OPNsense firewall. The endpoint is /metrics and the response is a Prometheus-formatted text document. The exporter pulls data from the OPNsense configuration API, from PF counters, from sysctl values, and from vmstat-style system calls, and exposes them as Prometheus metrics.

The metric categories the exporter covers:

PF state table — current entries, searches, inserts, removals, memory, limits.
PF rule statistics — per-rule evaluations, matches, bytes, packets.
Interface counters — bytes in/out, packets in/out, errors, drops per interface.
System — CPU utilisation, memory used/free, load averages, uptime.
ARP table — entries per interface.
DHCP leases — active leases per interface, lease utilisation.
VPN — IPsec tunnel state, WireGuard peer state, OpenVPN session count.
Unbound — query rate, cache hits/misses, uptime.
Suricata — alert count, packet count, dropped count.
CARP — state per VHID, demotion.

The exporter is a Python or PHP process that re-queries these sources every scrape. The cost is real — every scrape triggers several system commands and config reads — but at default 15-second intervals on a modern OPNsense box, the overhead is negligible.

Installing and configuring the plugin

The plugin is installed from the OPNsense web UI: System → Firmware → Plugins → os-opnsense-exporter. Once installed, it appears under Services → Prometheus Exporter. The configuration is minimal:

Listen address — defaults to 0.0.0.0; in production bind to the management address.
Listen port — defaults to 9100.
Enable — checkbox.

After enabling, the metric endpoint is live:

Read-only / Safeexporter /metrics— The /metrics endpoint returns Prometheus text format. Each metric has HELP and TYPE comments, followed by sample lines. The first five metrics are PF state-table counters. The last two are interface counters with an interface label — note the cardinality is bounded by the number of interfaces (typically <20). A label whose value is unbounded (a per-flow label, a per-IP label) would produce a cardinality explosion.

$ curl -s http://192.0.2.1:9100/metrics | head -30

# HELP opnsense_pf_states_total Current number of entries in the PF state table
# TYPE opnsense_pf_states_total gauge
opnsense_pf_states_total 3127
# HELP opnsense_pf_searches_total Total PF table searches
# TYPE opnsense_pf_searches_total counter
opnsense_pf_searches_total 18452394
# HELP opnsense_pf_inserts_total Total PF state insertions
# TYPE opnsense_pf_inserts_total counter
opnsense_pf_inserts_total 189324
# HELP opnsense_pf_removals_total Total PF state removals
# TYPE opnsense_pf_removals_total counter
opnsense_pf_removals_total 186197
# HELP opnsense_pf_state_limit_drops_total Drops due to state table full
# TYPE opnsense_pf_state_limit_drops_total counter
opnsense_pf_state_limit_drops_total 0
# HELP opnsense_interface_bytes_in_total Bytes received per interface
# TYPE opnsense_interface_bytes_in_total counter
opnsense_interface_bytes_in_total{interface="igb0"} 1.84e+09
opnsense_interface_bytes_in_total{interface="igb1"} 3.21e+08

Illustrative output

Configuring the Prometheus scrape

On the Prometheus server, a scrape job is added to prometheus.yml:

scrape_configs:
  - job_name: 'opnsense'
    scrape_interval: 30s
    static_configs:
      - targets: ['192.0.2.1:9100']
        labels:
          device: 'opnsense-fw-01'
          site: 'dc1'

The scrape_interval of 30 seconds is a reasonable balance between resolution and load. The exporter’s metrics do not change between 1-second and 30-second intervals for most use cases; 15 to 60 seconds is typical.

After restarting Prometheus, the target appears in the targets UI. The operator should confirm:

The state is UP.
The scrape duration is under 1 second.
No scrape_error samples are present.

Label cardinality — the trap

Prometheus stores each time series as a unique combination of metric name and labels. A metric with no labels has one time series per scrape; a metric with three labels with five values each has 125 time series. A metric with a label whose value is unbounded — the source IP of every state, the destination port of every flow — can produce millions of time series. The Prometheus server can technically handle millions, but the operational cost becomes untenable.

The os-opnsense-exporter is intentionally low-cardinality. The label values it uses are bounded:

interface — bounded by the number of interfaces (typically <20).
vhid — bounded by the number of CARP groups (typically <10).
rule_label — bounded by the number of named rules (typically <100).

The exporter does not expose per-flow, per-IP, or per-port metrics. If the operator needs those, they should be aggregated (top-N destinations, top-N source IPs) before being exposed, not raw per-flow counters.

Why per-flow metrics break Prometheus

The temptation is to expose a metric like opnsense_flow_bytes_total{src_ip="..."} to track bandwidth per source IP. The trap is that src_ip is unbounded — any new host on the network produces a new time series, and the series never go away (the IP may be re-used tomorrow). A network with 10,000 active IPs produces 10,000 time series for that single metric; multiply by per-flow, per-port, per-protocol and the series count explodes.

Prometheus’s response to a cardinality explosion is to slow down — each scrape reads more series, the storage grows, queries become expensive. The right approach is to aggregate at the source: top-N by sampling, or bucketed histograms, or pre-computed rollups stored as low-cardinality metrics. The exporter already follows this discipline; the operator who adds custom per-flow metrics should learn the discipline before exporting.

The relationship to PF, sysctl, and vmstat

The exporter’s metrics are a thin layer over FreeBSD and PF sources:

PF state table → pfctl -s info
Interface counters → netstat -I <iface> -b
CPU utilisation → vmstat 1 or sysctl kern.cp_time
Memory → sysctl hw.physmem, vmstat
ARP entries → arp -an | wc -l

The operator who understands the sources can verify the exporter’s output manually. If the exporter reports 3127 state entries, pfctl -s info should report the same number. If it does not, the exporter has a bug or is querying at a different moment.

Read-only / Safeverifying exporter output— Cross-checking the exporter's metric against pfctl. The two numbers match (3127). If they diverge — for example the exporter reports 3000 while pfctl reports 3200 — the exporter's scrape happened at a different moment or there is a bug in the exporter's parsing. The operator should never trust a single metric source without a cross-check.

$ pfctl -s info | grep 'current entries'; echo '---'; curl -s http://192.0.2.1:9100/metrics | grep opnsense_pf_states_total

current entries                        3127
opnsense_pf_states_total 3127

Illustrative output

Summary

The os-opnsense-exporter plugin exposes OPNsense metrics in Prometheus text format on port 9100 by default.
The exporter covers PF state, interface counters, system, ARP, DHCP, VPN, Unbound, Suricata, and CARP — bounded, low-cardinality metrics.
The Prometheus scrape job is configured with a static target; a 30-second scrape interval is typical.
Label cardinality is the trap: per-flow, per-IP, or per-port metrics explode the time-series count and slow the Prometheus server.
The exporter’s metrics should be cross-checked against pfctl, netstat, and vmstat for trust.