API safety and rate limiting — staged writes, idempotency, and the operational guardrails

An API call that modifies configuration is a small mutation against a stateful system. The mutation may fail validation and change nothing; it may save and never be applied; it may be applied and fail to compile; it may race with another writer and be silently overwritten. The discipline that keeps automation safe is not “the call will succeed” — it is “every step has a defined outcome and a defined recovery path”.

This lesson covers the four modes an API call can run in (read-only, staged write, apply, direct action), the configuration revision that serves as the rollback point, the retry and timeout discipline, and the way to tell a change that never saved from one that saved but was never applied.

The four call modes

Every API call has a mode, even if the operator has not named it:

1. Read-only. GET on a search, get or status command. Returns data; never modifies state. Safe to retry, safe to parallelise, no rollback path needed.
2. Staged write. A set or add command. It changes the configuration and records a new revision, but nothing in the data plane moves. Reversible by reverting the revision, and — importantly — invisible to traffic until applied.
3. Apply. The apply or reconfigure command that pushes the staged configuration into the running service. This is the call with operational consequences, and the one to place deliberately rather than after every write.
4. Direct action. A command that does something immediately and has no staged half — killing states, restarting a service, running a diagnostic. These have no revision to revert; the recovery is whatever the operator does next.

A disciplined automation script distinguishes the modes by design. Reads are free. Staged writes are batched and cheap to undo. The apply is the moment the change becomes real, so it happens once, deliberately, after every staged write in the set has succeeded — and the revision to roll back to is recorded before the first of them.

$ curl -sk -u "$KEY:$SECRET" https://localhost/api/core/backup/backups/this | jq '.items[0]'

{
"time": "1786000354",
"time_iso": "2026-08-14T09:12:34+00:00",
"description": "/api/firewall/filter/addRule made changes",
"username": "automation-firewall-rules",
"filesize": 184213,
"id": "config-1786000354.xml"
}

Transactional success vs partial failure

The framework splits a configuration change into two calls, and understanding the split is what makes automation safe.

The save call — addRule, setRule, addItem, and their equivalents — does three things:

1. Validate the request body against the model schema.
2. If valid, write the object into /conf/config.xml and record a new configuration revision.
3. Return {"result": "saved"} with the object’s UUID.

Nothing in the data plane has changed at this point. The apply call — filter/apply, alias/reconfigure, and their per-service equivalents — is what regenerates the configuration files from the templates and reloads the service. If the generated ruleset does not compile, the apply reports the failure and the service keeps running the configuration it already had.

The third row is the partial-failure mode, and it is the one worth designing against. It looks identical to the first row from the outside — an operator reading the GUI sees the new rule in both cases — so the only way to tell them apart is to check what the firewall is running rather than what it is configured to run.

The saving grace is that this split is also the recovery mechanism. Because a failed apply leaves the running configuration untouched, a change-set that fails at apply has not broken anything yet; reverting the configuration revision and re-applying returns both halves to the known-good state.

$ curl -sk -u "$KEY:$SECRET" -X POST https://localhost/api/firewall/filter/apply | jq .

{
"status": "OK
"
}

Timeout, retry and backoff

Three rules for any workflow that loops over API calls:

1. Set a timeout on every call. curl --max-time 30 (or the equivalent in every HTTP library). The call should never block the script for more than a reasonable interval; if the firewall is hung, the script should fail fast rather than wait for the operating system to time out the socket.
2. Retry reads, not writes. A read that fails is safe to retry — the state it asks about will still be there. A write that fails may have applied silently before the error response; retrying risks creating the same change twice. Idempotent operations (an add of an object that does not yet exist) can be retried with a UUID check; non-idempotent operations (a set of an existing object) cannot.
3. Back off on 5xx, never on 4xx. A 5xx or a refused connection says the firewall could not serve the request — the web server has a finite pool of PHP workers, and an apply occupies one for as long as the reload takes. Exponential backoff (1s, 2s, 4s) is the polite response. A 4xx is a client error: the request was understood and rejected, so retrying it unchanged produces the same rejection.

Detect, pause, rollback: the three-state discipline

For change-sets that touch more than one object, the discipline is:

1. Detect. Before the change-set, capture the current state — searchRule returns the table, getRule/{uuid} returns a specific object — and record the current configuration revision id from backups/this.
2. Pause. Between each staged write, check the response. A saved result with a UUID means proceed; a 4xx means stop. Because nothing has been applied yet, stopping here costs nothing but a revert.
3. Rollback. If any write fails, or the final apply reports anything other than success, revert to the recorded revision and re-apply. Do not continue writing; a half-staged change-set that gets applied is worse than one that was never started.

The ordering matters: staging everything and applying once means a failure part-way through the change-set never reaches traffic. A script that applies after every individual write gives up that property and exposes each intermediate state to the network.

Reading state during a write storm

A monitor that reads while a writer is busy sees whatever the framework has committed at the moment of the read. The reader may see the state before, after, or partway through a multi-step change-set. The reader’s responsibility is to be tolerant of inconsistent snapshots — the GUI shows the same transient state, so this is not specific to API readers.

Three defensive reads:

• After each staged write, re-read the affected object by UUID and confirm the values are what you sent. This catches fields that were silently dropped because the model does not define them.
• When a verification fails, log what was expected and what was found before retrying. “The rule is missing” and “the rule is present with an empty description” are different failures with different causes, and a retry loop that only checks presence will not tell them apart.
• Distinguish reads of the configuration from reads of the running state. searchRule answers “what is configured”; pfctl -sr answers “what is running”. During a change-set the two legitimately disagree, and a monitor that treats the disagreement as an alert will page on every routine change.

Summary

• Every API call has a mode: read-only, staged write, apply, or direct action. Choose deliberately.
• OPNsense writes a configuration revision on every change automatically. Record the current revision id from GET /api/core/backup/backups/this before a change-set; roll back with POST /api/core/backup/revertBackup/{id}.
• A saved response means the configuration changed and the data plane did not. The separate apply is what makes the change real, and its result is the gate on the change-set.
• Stage every write in the set, then apply once. A failure part-way through then never reaches traffic.
• Retry reads, not non-idempotent writes. Set timeouts. The framework serialises individual writes but not whole change-sets, so keep one change-set in flight at a time.