Scenario reference#

Every field of a surge YAML scenario. Unknown fields are rejected with the line number, so a typo fails surge validate instead of silently doing nothing.

name: checkout            # report title (default: the filename)
type: load                # smoke|load|stress|spike|soak|capacity
base_url: https://api.example.com
http:      { … }          # client settings
headers:   { … }          # sent on every request
vars:      { … }          # scenario-wide values for ${…}
feeders:   { … }          # CSV data sources
load:      { … }          # how much traffic, for how long
thresholds: [ … ]         # pass/fail rules
setup:     [ … ]          # commands run before the load
teardown:  [ … ]          # commands run after it, always
on_start:  [ … ]          # once per virtual user
flow:      [ … ]          # what one user does
wait:      2s..5s         # pause between iterations
users:     [ … ]          # several roles instead of one flow
output:    { … }          # report files

on_start, flow and wait at the top level are the one-role shorthand — they describe a single population. With a users: list each population carries its own, and the top-level ones are refused: applying them to every role would invent a login nobody wrote.

http#

http:
  timeout: 10s
  insecure_skip_verify: true

headers#

Sent on every request; a step's own headers win on conflict. Values accept ${…}.

headers:
  User-Agent: surge
  X-Env: "${env.ENVIRONMENT}"

vars#

Scenario-wide values, usable anywhere ${…} works. Override from the CLI with -var key=value.

vars:
  api: /api/v2
  tenant: acme

users — several roles in one test#

Real traffic is not one journey repeated. It is mostly people browsing, some of them buying, and the occasional admin doing something expensive. A test made of one of those measures a mix that does not exist.

Give either a top-level flow: — the one-role shorthand — or a users: list, never both.

users:
  - name: browser
    weight: 7                 # ~78% of the virtual users
    wait: 2s..8s              # this role's own pace, between iterations
    flow:
      - name: browse
        get: /api/items

  - name: buyer
    weight: 2
    wait: 1s..3s
    on_start:                 # once per virtual user, not per iteration
      - name: login
        post: /auth/login
        json: { username: "${users.username}", password: "${users.password}" }
        extract:
          token: { json: data.token }
    flow:
      - name: checkout
        post: /api/checkout
        headers: { Authorization: "Bearer ${token}" }

  - name: admin
    count: 1                  # exactly one, however the test is scaled
    wait: 30s
    flow:
      - name: report
        get: /admin/report

on_start is where a login belongs. Put it in flow and you measure a login on every pass, which no real user does. Put it in on_start and whatever it extracts — a token, a tenant id — stays available to every iteration after.

count is for the role that must not scale: one admin running an expensive report stays one admin whether the test is 50 virtual users or 5,000.

Two variables are always available: ${__VU} is the virtual user's number, and ${__USER} is the name of its role.

surge validate prints the division, because a mistyped weight still runs:

    users      3 populations
      browser — weight 7 (~78% of the rest) · 200ms..600ms
      buyer — weight 2 (~22% of the rest) · 500ms..1s
        on_start (once per virtual user)
          1. POST /auth/login → token
      admin — exactly 1 · 2s

feeders#

CSV files that feed data into iterations. The header row names the columns, and each column is available as ${feedername.column}.

feeders:
  users:    { path: users.csv, mode: per_vu }     # one account each
  products: { path: data/products.csv, mode: random }

per_vu: one account per virtual user#

The other three modes draw a fresh row at the start of every iteration, so virtual user 1 logs in as alice, and on its next pass is bob — while still holding alice's token and alice's cookies. For anything stateless that is fine. For a login it is not a user at all.

per_vu binds one row to each virtual user for the length of the run:

mode: per_vu            mode: circular
VU 1  alice ×9          VU 1  alice ×3  bob ×2  carol ×4  dave ×1
VU 2  bob   ×11         VU 2  bob   ×3  dave ×7
VU 3  carol ×10         VU 3  bob   ×4  carol ×5  dave ×1
VU 4  dave  ×10         VU 4  alice ×7  bob  ×1  carol ×1  dave ×1

This matters beyond tidiness. Per-account rate limits, per-account data volumes, row locks and cache keys all behave differently when a hundred virtual users share three accounts, and the result is a measurement of contention on three rows rather than of the system.

Two rules the mode enforces rather than leaving to you:

• Fewer rows than virtual users is allowed but never silent. A hundred users over three rows prints feeder "users" has 3 rows for 100 virtual users, so 97 of them share an identity with another before any load.
• The open model refuses it. There, each arrival is its own user and the virtual users are pooled workers; binding an identity to whichever worker picked an arrival up would be fiction. Use circular or once.

Splitting the accounts across machines#

Two machines each running mode: per_vu would both number their virtual users from 1 and both hand out the first row, so the same account would be logged in twice at once. surge divides the rows as well as the load: machine i of n takes rows i, i+n, i+2n, and no two machines can hand out the same one.

This happens by itself in a coordinated run (-agents). For machines with no coordinator, say which is which — decided from the shard number alone, with nothing to agree on:

# on three machines, or three CI jobs
surge run shop.yaml -shard 1/3 -snapshot part1.json
surge run shop.yaml -shard 2/3 -snapshot part2.json
surge run shop.yaml -shard 3/3 -snapshot part3.json

surge merge part*.json -threshold "p95 < 500ms"

Each machine applies a third of the load and takes a third of the rows. No CSV needs splitting by hand, and a machine left with no rows at all says so rather than running empty.

Only per_vu and once are divided. circular and random mean "reuse freely", so every machine keeps the whole file — dividing those would quietly shrink the data each machine draws from.

Why not a shared database of accounts? It works, and it is what most teams reach for. The cost is that every iteration then makes a network round trip to claim an account, and that wait happens inside the measurement: the generator queues on its own coordination and reports the delay as the target's latency. Striping needs no round trip and no shared state, because the division is decided by arithmetic.

In a Go scenario the same thing is c.VU(), which is stable for the life of the virtual user, as an index into your own data.

username,password,tier
alice,pw-alice,gold
bob,pw-bob,silver

→ ${users.username}, ${users.password}, ${users.tier}.

load#

How much traffic, for how long. Give stages, or the shorthand duration + vus/rate, or iterations + vus.

# constant: 50 users for 2 minutes
load: { vus: 50, duration: 2m }

# ramp: 0 → 100 users, hold, down
load:
  stages:
    - { duration: 30s, target: 100 }
    - { duration: 2m,  target: 100 }
    - { duration: 30s, target: 0 }
  warmup: 30s

# open model: find the knee
load:
  model: open
  max_vus: 500
  stages:
    - { duration: 30s, target: 200 }
    - { duration: 60s, target: 1000 }

# smoke test: 100 iterations, then stop
load: { iterations: 100, vus: 5 }

Closed vs open. Closed keeps N users busy: if the target slows down, your offered load drops with it and the problem hides. Open keeps the arrival rate fixed, which is how real traffic behaves. Use open to find a breaking point, closed to model a known population.

thresholds#

Pass/fail rules. All must pass, or the run exits 1. Expressions that do not parse fail before any traffic is sent.

<metric>[(<request or action name>)] <op> <value>

Operators: < <= > >=. Durations accept ms/s/m; a bare number means milliseconds. Without (name) the rule applies to all requests together.

thresholds:
  - p95 < 500ms
  - p99(api/checkout) < 2s  # just the step named "api/checkout"
  - p95(checkout) < 1s      # the whole `group: checkout` action
  - error_rate < 1%
  - check_rate > 99%
  - rps > 200

A scope names a request or a group: action, which is what makes group: worth writing: "checkout under one second" becomes a rule the run can fail on rather than a number in a table. A name that is both a request and an action is refused rather than guessed — rename one. A scope that never ran fails the threshold with nothing named X was measured, so a typo costs the run rather than passing quietly.

The same expressions scope inside abort_on: and a capacity-search SLO, and surge compare reports per request and per action.

flow#

What one virtual user does, start to finish. One pass = one iteration. A step is a request, a pause, or a request followed by a pause.

flow:
  - name: login
    post: /auth/login
    json: { username: "${users.username}", password: "${users.password}" }
    extract:
      token: { json: data.token }
    think: 1s..2s

  - name: item detail
    get: /api/items/${item_id}      # the name keeps stats on one row
    headers: { Authorization: "Bearer ${token}" }

  - think: 2s..5s                   # a pause on its own

extract#

Capture a value for later steps. Exactly one source per variable.

A failed extraction counts as an extract failure and ends the iteration (unless continue_on_error: true), because the next step would send an empty value and lie about the result.

check#

Assertions counted separately from errors — a slow 200 can fail a check without being an "error". They feed check_rate.

check:
  status: [200, 201]
  json_exists: [order_id]
  json: { status: confirmed }
  max_duration: 800ms

Declaring status also redefines failure for that step: with check: { status: 404 }, a 404 is a success and a 200 is an error. That is how you load-test error paths.

group — one user action across several calls#

A checkout that is three API calls is one thing the user did, and "checkout took 480ms" is the sentence an SLA is written in. Three rows of per-request latency is not that sentence, and adding three p95s together is wrong twice over: percentiles do not add, and the user also waited for whatever happened between the calls.

Consecutive steps that share a group name are timed end to end:

flow:
  - { name: api/cart,    post: /api/cart,    group: checkout }
  - { name: api/pay,     post: /api/pay,     group: checkout }
  - { name: api/confirm, post: /api/confirm, group: checkout }

user actions  (end to end, think time excluded)

action                              ran   failed      avg      p50      p95
checkout                            826      112    480.0    410.0    910.0

• The requests still appear individually. The action is a separate measurement in a separate table, and it is never added to the totals — every request inside it is already counted there, and a three-call checkout reported as four requests would make the throughput wrong.
• An action fails when any step in it failed, whatever the others did.
• Think time declared between steps of the same group is subtracted. A think is the scenario pausing on purpose, not the system being slow, and leaving it in would make an action's latency a property of the YAML file.
• Only consecutive steps group. A step without group: between two that have it closes the first action and starts a second.
• Actions merge across a fleet like requests do.

ws — WebSocket#

A game, a feed or a chat opens one connection per user and keeps it for the session. So the dial goes in on_start, where it is paid once, and the messages go in the flow:

on_start:
  - name: ws/connect
    ws: /game                    # relative to base_url; http→ws, https→wss
    as: game                     # the handle later steps use
    headers: { Authorization: "Bearer ${token}" }
    subprotocol: [json]          # optional
    read_limit: 1048576          # one message, default 1MB

flow:
  - name: ws/join
    on: game
    send: '{"op":"join","room":"${room}"}'
    receive: true                # send + receive = one round trip
    check:
      body_contains: '"ok":true'
    extract:
      player: { json: player.id }

  - name: ws/tick
    on: game
    receive: true                # on its own: the wait, which is the push
    repeat: 5                    # read five messages, measuring each
    check:
      body_contains: '"type":"tick"'

  - name: ws/leave
    on: game
    close: true                  # only to test what the server does

• send with receive is one measurement: the round trip — the WebSocket equivalent of a request's latency. It assumes the next message is the reply, which holds for request/response protocols and not for a broadcast; use separate steps there.
• receive alone measures the wait, which is what a push is. A timeout is recorded as a failure, because a stream that goes quiet is a failure and not a pause.
• repeat: N reads N messages and measures each, so the gap between them is in the report.
• Checks run against the message, so body_contains, json, json_exists and max_duration all work. status: does not: a message has no status code, and the scenario is refused if it asks for one.
• extract: pulls values out of a message exactly as it does a response body, so a player id from the join is available to every later step.
• Connections close when the virtual user stops. close: true is for testing what the server does when a client goes away mid-session.
• The handshake is measured as its own request, named by the step, so a connection that is slow to establish shows up rather than hiding inside the first message.

Putting the dial in the flow measures a handshake on every iteration. That is a different test, and rarely the one meant.

There is no ping. A pong is a control frame, processed only while a read is in flight, so pinging an idle connection waits for a reply nobody is listening for. Measure something the server actually answers.

setup and teardown — the commands around the load#

The second run is the one that matters. A load test is a measurement only if the thing it measures starts in the same state every time, and "remember to seed the accounts and clear the carts" written in a wiki is not a state anyone can reproduce. The commands that do it live in the scenario, and they end up in the report, so a run read six months later says what was done to the environment and not only what the environment did.

setup:
  - ./scripts/seed-accounts.sh 150        # a string: a shell runs it
  - ["psql", "-f", "seed.sql"]            # a list: argv, no shell at all
teardown:
  - ./scripts/clean.sh
hook_timeout: 10m                         # per command, default 5m

• The string form goes through sh -c, so pipes, redirects, globs and && work — and so does every other thing a shell does with a stray character. The list form is executed directly: no word splitting, no expansion, an argument containing a space stays one argument. Prefer the list unless you need the shell.
• Commands run with the scenario file's directory as the working directory, the same rule feeder paths follow. Never cd first.
• Setup stops at the first failure and no load is sent. There is no point measuring a system that was not prepared.
• Teardown runs whatever happened — a failed run, a breached threshold, a Ctrl-C — because the runs that leave a mess are the runs that went wrong. It also keeps going after one of its own steps fails: the step that puts the feature flag back should still run when the step that drops the table did not.
• They run once, on the machine that starts the test, not on every worker. Seeding the same accounts from four agents is four races.
• A feeder whose file does not exist yet is fine when there is a setup: surge validate lets it through and surge preflight reports it as not there yet, setup writes it rather than missing.

A scenario that can run commands is as powerful as a shell script. Read one before you run it, as you would a script.

output#

output:
  json:     reports/summary.json    # machine-readable summary
  html:     reports/report.html     # self-contained page, no network
  csv:      reports/timeline.csv    # per-second sec,reqs,errs,avg,p50,p95,max,vus
  failures: reports/failures.csv    # one row per distinct failure, with its example
  no_failure_bodies: false          # true leaves response bodies out of the report

Directories are created as needed. CLI flags -json, -html, -csv and -failures override.

failures is the error log in a form you can grep and diff: request, role, method, error class, status, count, share, first and last second, which checks failed, the message, the URL, and the first 512 bytes the server replied.

no_failure_bodies turns the last of those off. The reply is the most useful thing in a failure and the most likely to hold a customer's data, so it is kept by default and withheld on request — not the other way round, and not silently.

Templating: ${…}#

Usable in URLs, headers, bodies, JSON values, form fields, check values and think. Resolution order: extracted variables and feeder columns → vars → env. → functions. An unknown expression is left as-is, so a typo is visible in the request instead of silently empty.

json:
  order_id: "${uuid()}"
  qty: "${randint(1,5)}"
  user: "${users.username}"
  at: "${now_iso()}"
  key: "${env.API_KEY}"

CLI overrides#

Any of these override the file, so one scenario covers every environment:

surge run checkout.yaml \
  -base-url https://staging.example.com \
  -var tenant=acme \
  -vus 100 -duration 5m -warmup 30s \
  -threshold "p99 < 1s" \
  -json out.json -html out.html

surge validate checkout.yaml parses everything, checks that feeder files exist and that regexes and thresholds compile, and prints the plan — without sending a single request.