Benchmarks

A cache raises two questions: how much does it help, and what does it cost? No single number answers both, so this page reports several. Everything below comes from one run of the benchmark harness in bench/: Node v24.19.0, seed 1, load profile steady, 7 scenarios, 26 cached configurations, 31,718 measured requests.

None of the numbers on this page are typed by hand. They are all filled in from the run's JSON when the page is generated, so you can re-run the benchmark on your own hardware and regenerate the page to see your own results.

The whole run in one figure: every workload with ocache against the same workload without it, ranked by how much p99 latency improved, over the medians across all of them. Each row here shows one storage backend — the chart names it — and one offered rate; the rest of this page takes that apart.

#How the benchmark works

Each scenario models a realistic workload rather than a library feature: a page render, a JSON list, a per-user dashboard, an image render, an upstream API with a rate cap. Every scenario runs twice — once with ocache in the path and once without — so the difference between the two rows is exactly what caching changed. The cached side is then repeated across several storage backends. Backend latency is simulated from real-world p50/p99 figures rather than set to zero. This run covers memory, redis-local, redis-az, redis-az-bytes, sql, kv-edge, object-store; each scenario uses the backends that make sense for it, and its chart lists which ones.

A backend name ending in -bytes is not a different backend. It is the same simulated wire as its twin — redis-az-bytes and redis-az share one latency model exactly — running through createBlobStorage, which stores each entry as one byte frame instead of a JSON document. The pair is there so the codec can be read off the difference between two rows rather than argued about.

A few choices keep the results honest:

• Requests do not wait for each other. They arrive on a fixed random schedule (a Poisson process), and latency is measured from when a request should have started, not when the server got around to it. A benchmark that waits for each response before sending the next one hides queueing delay — the thing users actually feel under load.
• Waiting and computing are modelled separately. When the origin waits on I/O (a database, an upstream call), caching that away buys latency. When the origin computes on the main thread, caching it away buys capacity — the server can handle more traffic. Most scenarios include both.
• Caches are warmed up first. The hit ratio you see reflects the shape of the traffic, not how short the measurement was — which is what a long-running server actually experiences.
• Both sides see identical traffic. Same keys, same arrival times, same simulated latencies. The only difference is whether ocache is in the path.

#How much caching helps

Look at the whole range, not just the best row. Across every cached configuration in this run, p99 latency improves between 1.47x and 2589.70x, with a median of 2.81x. What sets the extremes is the origin, not the cache: when the origin is the bottleneck — a render that blocks the main thread, or an upstream that only allows so many requests at once — a cache hit removes an entire queue, not just one round trip, and the improvement becomes enormous. That is what happens to upstream-proxy at 2589.70x.

The scenario that gains the least is personalized-dashboard, at 1.51x even on its fastest backend — and yet it still reduces origin invocations by 81.7%. That is the point: faster responses and less origin work are two separate wins, and a workload can get one without the other. Hard cases like this one — per-user keys, short-lived sessions, many distinct keys — are included on purpose. A benchmark made only of easy wins would not tell you anything.

Origin-call reduction compares foreground and background origin invocations per admitted measured request with the matching no-cache row. It maps to origin work, but is deliberately not described as a request-level cache-hit share: deduplicated misses can share one invocation, while a stale response can launch a background refresh. Here it ranges from 76.7% to 98.0%, driven by traffic distribution and key cardinality:

#Latency, scenario by scenario

One chart per scenario: p50 through p99 for every configuration on a log axis, with the main-thread CPU spent per request next to each row. The bold no cache row is the workload without ocache; every row below it is the same workload running through one storage backend.

#More capacity, not just lower latency

A cache hit skips whatever work the origin would have done on the main thread — and that work is what limits how much traffic one server instance can handle. In the heaviest scenario here, og-image, main-thread CPU drops from 182.089 ms per request to 8.058 ms. When the origin mostly waits on I/O instead, the same hit buys latency but very little capacity — the CPU column in the charts above shows which case you are in.

#What a cache hit costs

Serving from cache is not free: ocache has to hash a key, check the entry is still valid, decode it, and build a Response. Measured against in-memory storage with nothing else running, a hit adds 7.8 to 29.8 µs for a cached handler and 4.1 to 4.5 µs for a cached function, across payloads from 4 KiB to 64 KiB.

Notice the cost does not grow with payload size — nothing on the hit path copies the body, so the variation between rows is per-call work and measurement noise, not a size effect. The handler path costs several times the function path, and that gap is the price of being a real HTTP cache instead of a simple memo table: deriving a key from the request, validating the entry, merging headers, and building a Response on every call.

#When caching is not worth it

A hit is never free, and neither is the storage read that finds it: caching trades a trip to your origin for a trip to your storage backend. Adding the largest measured hit-path overhead (0.03 ms) to each backend's median read time gives the break-even point — the minimum origin cost a handler needs before caching pays off at all.

If your handler is faster than its row, it is faster with no cache at all. Two things raise the bar further: a low hit ratio, because every miss pays for a storage read that found nothing, and a remote backend, which also has to decode the response body on your thread — that part does grow with payload size, and this table does not include it.

#Reproducing this page

pnpm bench --json=bench/results/steady.json --md=bench/results/steady.md
pnpm bench:docs bench/results/steady.json

The first command runs every scenario and writes the full report, including the tables this page summarizes. The second renders the charts, inlines them, and regenerates this page from the same JSON — the charts are part of the page, not files next to it. (pnpm bench:chart writes them out as standalone SVGs if you want them separately.) The prose lives in bench/docs.md; the numbers never do.