Questions, and things that go wrong#

Why is my p95 so much worse than what users see?#

Three usual causes, in the order to check them.

The generator was the bottleneck. surge watches its own CPU and its goroutine scheduling delay, and marks the run untrustworthy above 85%. If the report says so, the latency you measured is partly this machine's queueing. Spread the load with -shard or -agents.

There was no warmup. Connection setup, JIT and a cold cache are all inside the percentiles unless you exclude them. warmup: 30s keeps them on the chart and out of the statistics.

Connections are not being reused. -timings names the dominant phase. If it is connect or tls, check keep-alive, a proxy in the path, or a pool too small — and read reuse%: below 90% the setup cost is inflating every latency above it.

The run passed but the system was clearly broken#

A run with no thresholds cannot fail; surge says so in the verdict rather than printing a green word. And a request that answers 200 with the wrong body is not an error by the error-rate definition — it is a failed check, which is why asserting the field the next step needs matters more than asserting status: 200.

Every later step gets 401#

An extract stopped matching, so every step after it sent the literal ${token}. surge verify reports it before the run:

✗ token did not match (json: data.token) — every later step using
  ${token} will send it literally

This is the failure that costs two hours and reads as the target being broken. Run verify before anything long or large.

Why does the report say the mix is not what I asked for?#

Because it usually is not. Weight decides how many users pick a role, not how fast they iterate. A role that thinks for a second completes fewer iterations than one that does not, whatever its weight. Every average in the report is weighted by the actual column, so if the mix matters, change the think time or the weights.

Closed or open model?#

Closed (-vus N) is N users in a loop, each waiting for the last response: when the system slows, your load slows with it. Right for "200 people are using it".

Open (-rate R) is R arrivals a second whatever happens. Right for "we get 500 requests a second" — and the only one that will show you a cliff. A closed-model run against a struggling system quietly applies less load and reports a healthy-looking latency.

Arrivals were dropped — what does that mean?#

The open model ran out of free virtual users, so the rate on the chart was never applied and the target received less load than the scenario asked for. Raise max_vus until it reaches zero. If it will not go down, the generator is the bottleneck.

Can I add percentiles to the request table?#

-percentiles 50,95,99,99.9. The HTML report also keeps the full ladder behind a toggle on the requests table.

How do I test a WebSocket, a queue, gRPC, a database?#

WebSocket is in the YAML (ws:, on:, send:, receive:). The rest is a Go scenario: c.Time("name", fn) measures anything you can call and records it into the same stats, thresholds and report as an HTTP request. Recipes has one of each.

My scenario says unknown step field#

surge rejects unknown fields with the line number rather than ignoring them, so this is a typo or a field from a newer version. surge validate after every edit, and the scenario reference lists every field there is.

Can two machines' percentiles be added together?#

Not by averaging them, which is what most tooling does and is wrong. surge merges the raw histogram buckets, so a fleet's p99 is the p99. Use -snapshot on each machine and surge merge, or run a coordinator with -agents.

Is the HTML report safe to send to someone?#

It is one self-contained file and needs no network. It does contain response bodies from failures, which is what makes it useful and also the most likely place for something private. output.no_failure_bodies: true (or -no-failure-bodies) leaves them out.

Can surge replace my APM?#

No. It generates load and measures what the client saw. -monitor scrapes the target's own metrics and ranks which server-side series moved with the latency, which tells you where to look first — that is association, not cause, and the report says so rather than claiming otherwise.

What surge does not do#

It does not prepare your environment, scale your service, hold your feature flags or watch your dashboards, and it cannot tell whether the thing it measured was configured the way production is. Those are steps in a plan with a name against them — which is what /surge:perf-plan writes.