# Benchmarks
A cache raises two questions: **how much does it help, and what does it cost?** No single number answers both, so this page reports several. Everything below comes from one run of the benchmark harness in [`bench/`](https://github.com/unjs/ocache/tree/main/bench): Node v24.19.0, seed 1, load profile `steady`, 7 scenarios, 26 cached configurations, 31,718 measured requests.
None of the numbers on this page are typed by hand. They are all filled in from the run's JSON when the page is generated, so you can re-run the benchmark on your own hardware and regenerate the page to see your own results.
The whole run in one figure: every workload with ocache against the same workload without it, ranked by how much p99 latency improved, over the medians across all of them. Each row here shows one storage backend — the chart names it — and one offered rate; the rest of this page takes that apart.
## How the benchmark works
Each scenario models a realistic workload rather than a library feature: a page render, a JSON list, a per-user dashboard, an image render, an upstream API with a rate cap. Every scenario runs **twice — once with ocache in the path and once without** — so the difference between the two rows is exactly what caching changed. The cached side is then repeated across several storage backends. Backend latency is simulated from real-world p50/p99 figures rather than set to zero. This run covers `memory`, `redis-local`, `redis-az`, `redis-az-bytes`, `sql`, `kv-edge`, `object-store`; each scenario uses the backends that make sense for it, and its chart lists which ones.
A backend name ending in `-bytes` is not a different backend. It is the same simulated wire
as its twin — `redis-az-bytes` and `redis-az` share one latency model exactly — running
through [`createBlobStorage`](/docs/storage#keeping-bodies-as-bytes-on-a-byte-only-driver), which
stores each entry as one byte frame instead of a JSON document. The pair is there so the
codec can be read off the difference between two rows rather than argued about.
A few choices keep the results honest:
- **Requests do not wait for each other.** They arrive on a fixed random schedule (a Poisson process), and latency is measured from when a request _should_ have started, not when the server got around to it. A benchmark that waits for each response before sending the next one hides queueing delay — the thing users actually feel under load.
- **Waiting and computing are modelled separately.** When the origin waits on I/O (a database, an upstream call), caching that away buys **latency**. When the origin computes on the main thread, caching it away buys **capacity** — the server can handle more traffic. Most scenarios include both.
- **Caches are warmed up first.** The hit ratio you see reflects the shape of the traffic, not how short the measurement was — which is what a long-running server actually experiences.
- **Both sides see identical traffic.** Same keys, same arrival times, same simulated latencies. The only difference is whether ocache is in the path.
> [!NOTE]
> Handlers are called directly in process — no network socket, no HTTP parser, no framework — so the absolute request rates are higher than a real deployment reaches. What matters is the comparison between the no-cache and cached rows, not the absolute numbers. The [harness README](https://github.com/unjs/ocache/blob/main/bench/README.md) explains the full method and its known limits.
## How much caching helps
| scenario | offered rps | no-cache p99 (ms) | cached p99 (ms) | p99 vs no cache | origin-call reduction |
| ------------------------ | ----------: | ----------------: | --------------: | --------------: | --------------------: |
| `ssr-product-page` | 60 | 120 | 42.27–59.58 | 2.02x–2.84x | 93.5% |
| `api-list` | 300 | 84.42 | 42.66–57.44 | 1.47x–1.98x | 93.0% |
| `personalized-dashboard` | 220 | 150 | 99.25–100 | 1.50x–1.51x | 81.7% |
| `og-image` | 4 | 1229 | 184–185 | 6.65x–6.68x | 98.0% |
| `upstream-proxy` | 25 | 865 | 0.33–102 | 8.46x–2589.70x | 92.7%–93.4% |
| `markdown-render` | 30 | 596 | 25.16–113 | 5.26x–23.69x | 92.7% |
| `fanout-aggregate` | 90 | 786 | 450–461 | 1.70x–1.75x | 76.7%–77.3% |
Look at the whole range, not just the best row. Across every cached configuration in this run, p99 latency improves between 1.47x and 2589.70x, with a median of 2.81x. What sets the extremes is the origin, not the cache: when the origin is the bottleneck — a render that blocks the main thread, or an upstream that only allows so many requests at once — a cache hit removes an entire queue, not just one round trip, and the improvement becomes enormous. That is what happens to `upstream-proxy` at 2589.70x.
The scenario that gains the least is `personalized-dashboard`, at 1.51x even on its fastest backend — and yet it still reduces origin invocations by 81.7%. That is the point: **faster responses and less origin work are two separate wins**, and a workload can get one without the other. Hard cases like this one — per-user keys, short-lived sessions, many distinct keys — are included on purpose. A benchmark made only of easy wins would not tell you anything.
**Origin-call reduction** compares foreground and background origin invocations per admitted measured request with the matching no-cache row. It maps to origin work, but is deliberately not described as a request-level cache-hit share: deduplicated misses can share one invocation, while a stale response can launch a background refresh. Here it ranges from 76.7% to 98.0%, driven by traffic distribution and key cardinality:
## Latency, scenario by scenario
One chart per scenario: p50 through p99 for every configuration on a log axis, with the main-thread CPU spent per request next to each row. The bold **no cache** row is the workload without ocache; every row below it is the same workload running through one storage backend.
## More capacity, not just lower latency
A cache hit skips whatever work the origin would have done on the main thread — and that work is what limits how much traffic one server instance can handle. In the heaviest scenario here, `og-image`, main-thread CPU drops from 182.089 ms per request to 8.058 ms. When the origin mostly waits on I/O instead, the same hit buys latency but very little capacity — the CPU column in the charts above shows which case you are in.
## What a cache hit costs
Serving from cache is not free: ocache has to hash a key, check the entry is still valid, decode it, and build a `Response`. Measured against in-memory storage with nothing else running, a hit adds 7.8 to 29.8 µs for a cached handler and 4.1 to 4.5 µs for a cached function, across payloads from 4 KiB to 64 KiB.
| payload | handler direct (µs) | handler cached (µs) | handler adds (µs) | function adds (µs) |
| ------: | ------------------: | ------------------: | ----------------: | -----------------: |
| 4 KiB | 35.8 | 55.2 | **+19.5** | **+4.3** |
| 6 KiB | 37.9 | 58.9 | **+21.0** | **+4.5** |
| 8 KiB | 51.0 | 58.8 | **+7.8** | **+4.3** |
| 15 KiB | 52.4 | 81.0 | **+28.6** | **+4.1** |
| 20 KiB | 56.6 | 81.7 | **+25.1** | **+4.5** |
| 40 KiB | 92.3 | 114.2 | **+21.9** | **+4.2** |
| 64 KiB | 119.7 | 149.5 | **+29.8** | **+4.2** |
Notice the cost does not grow with payload size — nothing on the hit path copies the body, so the variation between rows is per-call work and measurement noise, not a size effect. The handler path costs several times the function path, and that gap is the price of being a real HTTP cache instead of a simple memo table: deriving a key from the request, validating the entry, merging headers, and building a `Response` on every call.
## When caching is not worth it
A hit is never free, and neither is the storage read that finds it: caching trades a trip to your origin for a trip to your storage backend. Adding the largest measured hit-path overhead (0.03 ms) to each backend's median read time gives the break-even point — the minimum origin cost a handler needs before caching pays off at all.
| backend | read p50 (ms) | read p99 (ms) | a handler pays off above (ms) | models |
| ---------------- | ------------: | ------------: | ----------------------------: | ------------------------------------------------------- |
| `memory` | 0 | 0 | 0.03 | in-process Map (createMemoryStorage) |
| `redis-local` | 0.12 | 0.5 | 0.15 | unix socket or sidecar valkey |
| `redis-az` | 0.6 | 3 | 0.63 | same-region TCP |
| `redis-az-bytes` | 0.6 | 3 | 0.63 | same-region TCP, one blob per entry (createBlobStorage) |
| `sql` | 2 | 15 | 2.03 | Postgres or D1 key-value table |
| `kv-edge` | 6 | 40 | 6.03 | Cloudflare KV / Deno KV, eventually consistent |
| `object-store` | 30 | 120 | 30.03 | S3 / R2 GetObject |
If your handler is faster than its row, it is faster with no cache at all. Two things raise the bar further: a **low hit ratio**, because every miss pays for a storage read that found nothing, and a **remote backend**, which also has to decode the response body on your thread — that part _does_ grow with payload size, and this table does not include it.
> [!TIP]
> The storage profiles are estimates fitted to a published p50/p99 pair, not measurements of your backend. They live in one table in [`bench/harness/storage.ts`](https://github.com/unjs/ocache/blob/main/bench/harness/storage.ts) — measure your own backend and edit it before quoting any of these numbers.
## Reproducing this page
```sh
pnpm bench --json=bench/results/steady.json --md=bench/results/steady.md
pnpm bench:docs bench/results/steady.json
```
The first command runs every scenario and writes the full report, including the tables this page summarizes. The second renders the charts, inlines them, and regenerates this page from the same JSON — the charts are part of the page, not files next to it. (`pnpm bench:chart` writes them out as standalone SVGs if you want them separately.) The prose lives in `bench/docs.md`; the numbers never do.