Benchmarks

All benchmarks run on Apple M-series, PostgreSQL 18, Python 3.14t (free-threaded), Zig ReleaseFast. Methodology: wrk -t4 -c20 -d8s, 3 runs, median selected, jitter reported.

REST API Throughput (Bookstore API)

Endpoint	rps	p50	p99	Jitter
GET /health (no DB)	19,890	0.98ms	2.79ms	1.1%
GET /api/v1/books/stats (aggregate)	11,420	1.73ms	12.21ms	7.6%
GET /api/v1/reviews/ (cursor pagination)	7,500	2.51ms	6.58ms	0.7%
GET /api/v1/books/1 (detail + select_related)	6,349	2.85ms	14.19ms	1.1%
GET /api/v1/books/ (list + serializer)	5,337	3.37ms	20.92ms	7.7%
GET /api/v1/books/?search=python (FTS)	4,340	4.14ms	15.99ms	0.4%

Template Rendering (HyperNews)

Endpoint	rps	p50	p99
GET /login (template-only)	40,459	0.38ms	2.38ms
GET /forums (directory)	40,328	0.35ms	2.67ms
GET / (cached homepage)	40,000+	0.36ms	3.35ms
GET /user/alice (multi-query)	40,608	0.35ms	5.38ms

Database (pg.zig vs psycopg3)

Operation	pg.zig	psycopg3	Speedup
SELECT by PK	21K ops/s	10K ops/s	2.06x
SELECT range	—	—	4.18x
UPDATE	—	—	1.52x
COPY bulk import	536K rows/s	12K rows/s	42.8x

Guard System Overhead

Guard Type	Overhead
Single guard (Require.role)	0.21 us
3-guard chain	0.40 us
GuardSpec creation	0.85 us

JSON (SIMD Zig vs Python stdlib)

Operation	Native	stdlib	Speedup
json_loads (tiny object)	94ns	576ns	6.1x
json_loads (integer)	48ns	467ns	9.8x
json_loads (float)	80ns	518ns	6.5x
json_loads (boolean)	49ns	441ns	9.0x
json_dumps (dict)	196ns	—	—

String Operations (SIMD Zig vs Python stdlib)

Operation	Native	stdlib	Speedup
html_escape (with chars)	111ns	376ns	3.4x
url_encode (long path)	113ns	1390ns	12.3x
url_decode (percent)	88ns	1505ns	17.1x
parse_query_string (10p)	1574ns	5596ns	3.6x

Validation (Native Zig)

Operation	Throughput
Model creation (init_model_full)	1.6M/sec
Per-field validation	6.7M fields/sec
Batch int validation (SIMD)	51.5M ints/sec
Batch model validation	13.1M models/sec
SIMD email validation	63ns/email

Template Compilation

Operation	Native	Jinja2	Speedup
Compile	7.1us	1.66ms	234x
Render (cached)	36us	61us	1.7x

WhereNode Compile (Zig vs Python)

Scenario	Python	Zig	Speedup
Simple leaf	442ns	169ns	2.6x
3-filter AND	1737ns	464ns	3.7x
Complex nested (4 children)	3364ns	868ns	3.9x

Native Metric Primitives

Operation	Latency	Target
counter_inc	78ns	50ns
gauge_set	76ns	50ns
histogram_observe	81ns	100ns
counter_vec_inc	132ns	250ns
histogram_vec_observe	124ns	300ns

WebSocket: native Zig server vs. websockets (PyPI reference)

Headline: native is 1.8–2.3× faster than websockets on throughput and lower-latency at every payload size. This is only visible when the benchmark is driven by a multi-process load generator (benchmarks/websocket/loadgen.py). A single asyncio client process does the same per-message work as a single-threaded server, so it caps at one core's worth of load and cannot saturate a multi-core server — earlier single-client numbers were measuring the client's ceiling and undercounting native. Native scales with cores (one OS thread per connection under free-threaded Python 3.14t); the single-loop reference is pinned to one core and its throughput actually degrades as more client load is applied. Every connection also does an out-of-band warmup burst before the timed window opens; latency is measured single-connection, unpipelined, with its own warmup.

Payload	native msgs/sec	reference msgs/sec	speedup	native p50	reference p50
32 B	145k	81k	1.8×	46us	77us
4096 B	140k	72k	1.9×	50us	80us
65536 B	75k	32k	2.3×	82us	105us

Native also wins at concurrency=1 (19k vs 13k msgs/sec) and its aggregate throughput keeps climbing with offered load (up to ~145k) while the reference plateaus around ~85k and then declines.

Startup latency (process spawn to /health responding, median of 5 trials, 5ms readiness-poll granularity so quantization doesn't hide the number): native ~85ms vs. reference ~40ms. Confirmed via direct measurement that HYPER_THREAD_POOL_SIZE is not the driver here — startup time is flat (~86ms) whether the pool spawns 4, 8, or 24 threads, so the gap is import time + interpreter/native-extension load + Zig server bind, not thread provisioning. The import-time fixes below (#8) reduced this component; the remainder is inherent to loading a fuller framework than a single-purpose library.

Connection model — the default (shared) vs. the thread opt-out. WEBSOCKET_CONCURRENCY=shared (the default) multiplexes connections over a small event-loop pool; WEBSOCKET_CONCURRENCY=thread dedicates one OS thread per connection (max live connections = THREAD_POOL_SIZE). Both driven to 96 concurrent connections:

Model	Connections held	Throughput	Peak RSS	Peak threads
shared (default)	96 / 96	165k msg/s	83 MB	32
thread (opt-out)	24 / 96	145k msg/s	82 MB	26

The default shared model holds all connections (the thread opt-out caps at its thread-pool size), at higher throughput and essentially the same memory — and memory stays ~flat as connections grow. This is why it's the default. It requires cooperative handlers (no thread parked per connection); see server.md and realtime.md.

Interop: both servers pass all 9 RFC 6455 correctness checks (text/binary/ Unicode/empty-message echo, ping/pong, clean close, concurrent send ordering, multi-connection isolation) — see benchmarks/websocket/interop.py.

Architecture, not a bug: the native server dedicates one OS thread (from a fixed pool, default 24, HYPER_THREAD_POOL_SIZE) to each live connection so it can genuinely use multiple CPU cores under Python 3.14's free-threaded build; connections beyond the pool size queue rather than fail. The websockets reference runs a single-process asyncio event loop with no such ceiling, but no multi-core parallelism either. Spending more memory/threads for more concurrent-connection capacity is a deliberate, good trade as long as it's a tunable knob — which it is (HYPER_THREAD_POOL_SIZE, HYPER_THREAD_STACK_SIZE).

Perf audit — findings, in order of impact (full detail, including a python -X importtime breakdown and a cProfile trace that caught a kqueue-syscall regression mid-fix, is in the generated report):

1. Benchmark methodology (biggest finding) — the throughput comparison was client-limited; a single asyncio client can't saturate a multi-core server. Fixed with a multi-process load generator, which revealed native is 1.8–2.3× faster, not slower. The premise that native was losing was itself a measurement artifact.
2. RFC 6455 handshake bug (wrong magic GUID) — made the native server unable to complete a handshake with any spec-compliant client at all.
3. Executor-per-connection leak — a brand-new ThreadPoolExecutor per connection instead of one shared bounded pool.
4. Per-message thread-hop on receive — loop.run_in_executor cost ~27-29us per round trip in isolation (vs. ~0.01us direct). Fixed with a non-blocking _ws_try_recv (MSG_DONTWAIT against a per-connection buffer) plus _ws_get_fd, so Python awaits loop.add_reader(fd, ...) instead of a thread pool. Old executor path kept as an automatic fallback.
5. Reader re-registered every message — fix 3's first version paid two kqueue syscalls per message; cProfile showed select.kqueue.control() at 61% of all time. Fixed by registering the reader once per connection.
6. No TCP_NODELAY — Nagle's algorithm was active on every WS socket; asyncio enables TCP_NODELAY by default. Fixed via setsockopt.
7. Two syscalls per frame write (header, then payload) — fixed with a single writev() call (NetStream.writeAllVectored2).
8. O(n) memmove per frame extracted from the receive buffer — fixed with a read cursor that only compacts lazily, so a batch of N buffered frames costs zero memmoves instead of N.
9. 113MB import-time bloat — from hyperdjango import HyperApp alone cost 113MB / ~153ms (vs. 15.7MB / ~20ms for websockets), traced via python -X importtime to three places eagerly importing Django's forms/ORM/template compat layer for features (SessionAuth/.oauth2(), HyperForm/HyperSerializer, file-based route discovery) a bare WebSocket app never touches. Deferred all three to first-use (matching the framework's own lazy __getattr__ convention) — cut baseline import memory to 51.5MB, zero functional change.
10. Thread-pool stack size was hardcoded at Zig's 16MB default — now tunable via HYPER_THREAD_STACK_SIZE, default unchanged (a 1MB override crashed the server outright in testing, confirming the default is load-bearing, not just caution).

Net effect on this quick-matrix run (concurrency=8, before → after all fixes): p50 latency now beats the reference at every payload size tested; throughput rose from ~65-70% of reference to ~90-97%; peak RSS under sustained load dropped from ~155MB to ~87MB. The full websocket test suite (152/152), 217/217 core HTTP/server tests, and the full repo suite (11521/11684, remaining failures pre-existing and environment-specific) passed after every change. The remaining throughput gap at higher concurrency reflects the thread-pool-vs-single-event-loop architectural trade-off above, not an unaddressed inefficiency. See the full report for details.

Reproduce

uv run hyper-build --release
uv run python scripts/bench_bookstore_wrk.py
uv run python scripts/bench_hypernews_wrk.py
uv run python scripts/bench_guard_overhead.py

# WebSocket: native vs. websockets (PyPI), interop + throughput + latency +
# memory + a live-measured perf audit. --full for the complete matrix,
# --profile to also attempt py-spy flamegraphs (usually needs sudo).
uv run --group benchmark-comparison python -m benchmarks.websocket.run
uv run --group benchmark-comparison python -m benchmarks.websocket.run --full

Results saved to logs/bench_*.json and logs/bench_*.txt, and to benchmarks/websocket/out/{results.json,report.md,report.html} for the WebSocket suite.

Clean Ubuntu machine — full setup from scratch

Bringing up a bare Ubuntu 24.04 box (everything runs on localhost) to build, run the suite, and benchmark. Commands assume sudo.

1. System packages

sudo apt-get update
sudo apt-get install -y build-essential curl git wget xz-utils ca-certificates wrk

# PostgreSQL 18 + pgvector from the PGDG apt repo (Ubuntu's default repos are older)
sudo install -d /usr/share/postgresql-common/pgdg
sudo curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc \
  https://www.postgresql.org/media/keys/ACCC4CF8.asc
echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] \
https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" \
  | sudo tee /etc/apt/sources.list.d/pgdg.list
sudo apt-get update
sudo apt-get install -y postgresql-18 postgresql-18-pgvector

2. Zig 0.16+ and uv

# Zig 0.16 — grab the linux-x86_64 0.16.x tarball from https://ziglang.org/download/
# (apt's zig is too old). Example:
cd /opt && sudo wget https://ziglang.org/download/0.16.0/zig-x86_64-linux-0.16.0.tar.xz
sudo tar xf zig-x86_64-linux-0.16.0.tar.xz
echo 'export PATH=/opt/zig-x86_64-linux-0.16.0:$PATH' | sudo tee /etc/profile.d/zig.sh
source /etc/profile.d/zig.sh && zig version   # expect 0.16.x

# uv (manages Python 3.14t for you — no system Python needed)
curl -LsSf https://astral.sh/uv/install.sh | sh
source "$HOME/.local/bin/env"

3. PostgreSQL: role, database, extension

sudo systemctl enable --now postgresql
# Create a role matching your login user + the test database it connects to
sudo -u postgres createuser -s "$USER"
sudo -u postgres createdb -O "$USER" hyperdjango_test
psql -d hyperdjango_test -c 'CREATE EXTENSION IF NOT EXISTS vector;'   # pgvector

The DATABASE_URL connects over TCP to localhost with no password, but Ubuntu's default pg_hba.conf requires one for TCP. On a dedicated benchmark box, trust local connections (localhost only — do not do this on a shared or internet-facing host):

HBA=/etc/postgresql/18/main/pg_hba.conf
sudo sed -i -E 's|^(host\s+all\s+all\s+127\.0\.0\.1/32\s+).*|\1trust|' "$HBA"
sudo sed -i -E 's|^(host\s+all\s+all\s+::1/128\s+).*|\1trust|' "$HBA"
sudo systemctl reload postgresql
psql "postgresql://$USER@localhost:5432/hyperdjango_test" -c 'SELECT 1;'   # verify passwordless

4. Build and verify

git clone https://github.com/anthropics/hyperdjango.git && cd hyperdjango
uv sync --group dev --group benchmark-comparison     # deps incl. FastAPI/Flask baselines
uv run hyper-build --install --release               # optimized native ext
export DATABASE_URL="postgresql://$USER@localhost:5432/hyperdjango_test"
uv run hyper-test rest serializers                   # quick smoke; drop args for the full suite

5. Kernel + PostgreSQL tuning (throughput + stability)

Everything is localhost, so the usual ceiling is ephemeral-port and accept-queue exhaustion, not bandwidth. Persist these and apply for the session:

# /etc/sysctl.d/99-hyperdjango-bench.conf
sudo tee /etc/sysctl.d/99-hyperdjango-bench.conf >/dev/null <<'SYSCTL'
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
fs.file-max = 2097152
SYSCTL
sudo sysctl --system

# File-descriptor limits (the load generator + server open many sockets)
echo "* soft nofile 1048576" | sudo tee -a /etc/security/limits.conf
echo "* hard nofile 1048576" | sudo tee -a /etc/security/limits.conf
ulimit -n 1048576   # for the current shell

PostgreSQL — edit /etc/postgresql/18/main/postgresql.conf (sizes assume a large box; scale to your RAM), then sudo systemctl restart postgresql:

max_connections = 10000            # project-recommended headroom for the parallel suite
shared_buffers = 8GB               # ~25% of RAM
effective_cache_size = 24GB        # ~50-75% of RAM
work_mem = 32MB
maintenance_work_mem = 1GB
max_worker_processes = 64          # ~= core count
max_parallel_workers = 64
max_parallel_workers_per_gather = 4
synchronous_commit = off           # benchmark throughput (relaxes durability — bench DB only)

max_connections = 10000 needs enough SysV shared memory / semaphores; on most modern kernels the defaults are fine, but if PostgreSQL refuses to start, raise kernel.shmmax / kernel.sem accordingly.

HTTP scaling on large / high-core machines

The numbers above come from an 18-core dev box (~137K rps plaintext at W=8). On a large server (64/128/200+ cores), run the worker-scaling sweep to find the peak as worker threads scale toward the core count — the threaded model keeps climbing on more cores before the native-dispatch floor, and typically beats the reactor at peak (the reactor wins on idle-connection scaling / low worker counts).

# One-time OS/loadgen prep (so the kernel and load generator aren't the ceiling)
ulimit -n 1048576
sudo sysctl -w net.core.somaxconn=65535 net.ipv4.tcp_max_syn_backlog=65535
# install the lightweight load generator: apt-get install -y wrk

export DATABASE_URL="postgresql://$USER@localhost:5432/hyperdjango_test"
uv run hyper-build --release

# Worker-scaling sweep — threaded vs reactor, push W toward the core count.
# Edit --worker-counts to bracket your machine (e.g. up to 128/200/256).
uv run python -m benchmarks.http.run \
  --mode workers \
  --frameworks hyperdjango-threaded,hyperdjango-reactor \
  --worker-counts 8,16,32,64,96,128,160,200,256 \
  --sweep-concurrency 512 \
  --duration 10 --warmup 2 \
  --client wrk

# Concurrency sweep at a high fixed W, with FastAPI/Flask baselines for context.
uv run python -m benchmarks.http.run \
  --mode concurrency \
  --frameworks hyperdjango-threaded,hyperdjango-reactor,fastapi,flask \
  --workers 128 --duration 10 --warmup 2 --client wrk

Results land in benchmarks/http/out/: report.html (interactive Plotly dashboard), report_workers.md, and results_workers.json. Read the rps-vs-worker-count plateau to find where throughput saturates. Add --quick for a fast smoke test to validate the setup before the full sweep.