surge — design & analysis#

Status: original design record · 2026-11-06 · owner: luann (agentic-fx)

This is the analysis the project started from. Several things listed below as deferred have since shipped; where that is so, it is marked. The record is kept because the reasoning still explains why the engine is shaped the way it is.

1. Problem#

A load test is run to answer three questions — how much can this take, is it slower than last time, and why — and the usual tooling answers none of them directly. Four structural limits, none of them accidental:

A compiled language removes the first and the fourth outright. The second and third are design decisions, not performance ones, and are the reason this is a new engine rather than a faster client for an old one.

2. Goals (v1.0)#

1. One static binary, no runtime deps; cross-compile for the generator fleet.
2. Two ways to write scenarios
   • Library mode: User types with weighted Tasks, OnStart/OnStop, Wait(min,max) — a main.go that compiles to the test itself.
   • CLI mode: YAML scenarios, zero code — flows, extract/check, CSV feeders, ${...} templating.
3. Open and closed workload models, both with ramping stages, warmup exclusion and graceful stop.
4. CI-first: threshold expressions (p95 < 500ms, error_rate < 1%, p99(login) < 800ms) → exit code; JSON summary for machines; self-contained HTML report for humans; per-second CSV.
5. Honest metrics: HDR-style log-linear histogram (bounded memory, ~3% error), per-request-name stats, status distribution, top error classes, per-second timeline.
6. Throughput: ≥ 50k req/s on a developer laptop against a local target (small responses), from one process using every core — so a single machine stays a single process.

Non-goals in v1.0 (explicit, with a path)#

• Distributed master/worker — deferred, because one box saturates a 10 Gbps NIC for typical API payloads and the usual reason to distribute is to escape a single core. Design sketch in §7 so it would be additive. Shipped in 0.3.0; the sketch held.
• Native WebSocket / gRPC clients — deferred. The engine is already protocol-agnostic (§5): the scheduler drives a VU interface and the collector accepts samples from any source, so library users can measure any protocol with ctx.Time(name, func() error). Both shipped, plus databases, SAP OData, mainframe and real browsers.
• Web UI — reports go to files; a live page seemed to duplicate them. Shipped in 0.3.0: watching a run as it happens turned out to be worth more than the reports it duplicates, because a blown budget is visible at 00:20 instead of at the end.
• JS/embedded scripting — Go is our scripting language; YAML covers the no-code case.

3. Why Go (vs Rust)#

The bottleneck in practice is the target system, not the generator. Go's ceiling is far above what we can usefully apply; Rust buys nothing we need and costs maintainability. Decision: Go.

4. Architecture#

cmd/surge        CLI: run | quick | validate | version
surge (root)     public library: Scenario, User, Task, Ctx
internal/
  config         YAML schema, defaults, validation (strict keys, line numbers)
  engine         scheduler (closed/open/iterations), HTTP executor,
                 flow VU (YAML interpreter), feeders, ${...} interpolation
  metrics        log-linear histogram, sample collector, summary
  threshold      "p95 < 500ms" parser + evaluator
  report         console live + summary, JSON, CSV, HTML

Data flow: VU goroutines → buffered chan (16k) → single collector goroutine → histograms/timeline → summary → thresholds → reporters. The hot path takes no locks; aggregation is serialized in one consumer.

5. Engine model#

The scheduler knows nothing about HTTP. It drives:

type VU interface {
	Start(ctx context.Context)     // once per spawned VU (library: User.OnStart)
	Iteration(ctx context.Context) // one scenario pass   (flow once / one weighted Task)
	Stop()                         // on deschedule       (library: User.OnStop)
}

type Spawner func(id int64) VU

• Closed model: controller ticks at 100ms, linearly interpolates the target VU count from stages, spawns goroutines or closes per-VU stop channels.
• Open model: 20ms ticks accumulate fractional arrivals from the stage rate; each arrival runs one iteration on a VU taken from a reuse pool (capped by max_vus). If the pool is exhausted the arrival is counted as dropped — visible proof the target (or generator) can't keep up, instead of silently lowering load.
• Iterations mode: fixed iteration count across N VUs (smoke/functional).
• Warmup samples appear on the timeline but are excluded from final stats.
• Grace stop: in-flight iterations get grace_stop (default 5s) to finish, then the run context is cancelled.

HTTP executor: one shared http.Transport (HTTP/2 on by default, tuned idle pool), per-request timing, bounded body read (max_body_bytes, default 10MB), error classification (timeout / refused / reset / dns / HTTP 4xx-5xx) so the report shows why requests failed, not just counts.

6. Metrics#

• Histogram: exact below 32µs, then 32 sub-buckets per power of two — ~3% relative error, 15KB fixed memory, O(1) record. Percentiles clamped to observed min/max.
• Request error rule: transport error or status ≥ 400, unless the step declares check.status — then the expectation is the truth (e.g. a flow that wants a 404).
• Checks are separate from errors: a slow-but-200 response can fail max_duration without being an "error".
• Timeline: per-second reqs/errs/avg/max/VUs for charts and CSV.

7. Distributed — the original sketch (since built, as described)#

• surge agent --listen 0.0.0.0:7777 --token … on each generator; surge run --agents a,b,c becomes the coordinator: ships the scenario (YAML or the compiled binary is already identical on agents), splits stage targets, streams Summary deltas every 2s over gRPC, merges histograms (bucket-wise add — the histogram was chosen to be mergeable), evaluates thresholds centrally.
• Nothing in v1 blocks this: collector, histogram and summary are already merge-friendly; the scheduler takes scaled stage targets.

8. Docs plan#

• README.md — positioning, quickstart (both modes), feature matrix.
• docs/scenario-reference.md — every YAML field.
• docs/library.md — Go API guide.
• docs/design.md — this file.
• examples/ — runnable: quick-start YAML, full checkout flow with feeder + extract + thresholds, library mode main.go, demo target server.

9. Decision log#