Cloudflare's Cache Transcoding: Zstd Inside Pingora to Save Petabytes

Cloudflare is staring down rising memory costs — RAM and hard disk prices have exploded over the past year. To stretch its deployed storage further, the company prototyped a system called Cache Transcoding, built during an internship in its 1.1.1.1 Intern Program. The idea: compress eligible assets with Zstandard (zstd) before writing them to cache, keep them compressed on disk and during Tiered Cache transfers, and decode only when serving to the client.

In initial testing, this encoding shrank eligible assets to about one-third of their original on-disk size on average. The trade is a small increase in CPU usage — paid once per cache fill — for significant storage and bandwidth savings that accrue every time the asset is served.

Why Zstandard?

Zstd is a lossless compression algorithm open-sourced by Facebook in 2016. It balances compression ratio with speed. Cloudflare's earlier browser compression testing found zstd compressed data 42% faster than Brotli while producing nearly the same file size, and produced files 11.3% smaller than gzip at comparable speed. The prototype uses zstd level 3, which gives most of the compression benefit without turning cache fills into a CPU bottleneck.

Traditionally, Cloudflare stores an asset using the content encoding supplied by its origin. If an origin sends uncompressed bytes, those bytes are stored and transferred as-is. Cache Transcoding adds compression inside the cache itself.

Not everything is worth compressing

Images, video, and fonts are usually already compressed. In Cloudflare's traffic sample, this media slice represented 21.4% of requests but 63.3% of bytes — compressing it again would waste CPU. Compressible text (HTML, JSON, CSS, JavaScript) represented 67.3% of requests and 22.3% of bytes. Within that text slice, about 71% arrived uncompressed with Content-Encoding unset, and it compresses well. In the controlled test corpus, eligible assets compressed by roughly 2.8 times.

Measured costs: encoding at zstd level 3 costs 4.31 ns per byte (about 232 MB/s), paid once per fill; decoding costs 1.56 ns per byte (about 641 MB/s), paid on every serve. Encoding is more expensive per byte, but assets are served far more often than they are filled.

Paying the compression cost once

Compression is never free. The key question is whether byte savings justify the CPU cost. At zstd level 3, the model kept extra CPU cost to a few percent under tested traffic and reuse assumptions. Limiting transcoding to popular content didn't help — decoding happens on every serve, so restricting the feature to hot assets reduced storage savings without cutting CPU proportionally. The simpler policy performed better: transcode all eligible compressible text at or above 4 KiB, capturing nearly all measured storage benefit within the CPU budget.

How Cache Transcoding works

On a cache miss, the Pingora-based proxy encodes the body with zstd before writing to disk, records that the stored representation is compressed, and preserves the original content length. On a cache hit, the stored zstd object is read and decoded. With Tiered Cache, the compressed representation moves between tiers in compressed form; decoding happens only on the client-facing hop. A storage encoding marker prevents an object from being encoded more than once.

Eligibility checks

The prototype only transcodes a 200 OK response when Content-Encoding is unset, the Content-Type is compressible text, and the response has a known Content-Length of at least 4 KiB. Slice subrequests, responses using active upstream compression, range requests, precompressed responses, unknown length bodies, and binary content remain unchanged. The 4 KiB threshold removed many tiny requests while excluding only about 1% of otherwise eligible bytes.

Testing over one million requests

Cloudflare exercised the prototype against a controlled test zone, correlating requests across logs, Prometheus metrics, and Jaeger traces. One performance campaign sent more than a million requests across 10 cache servers, half with Tiered Cache disabled and half enabled. Two assets (approximately 195 KiB and 272 KiB) compressed by roughly 2.8 times. This was a deliberately compressible corpus, so the measured ratio isn't a fleet-wide constant.

Compress once, benefit many times

The experiment shows significant efficiencies are still available in Cloudflare's caching service. The trade is favorable under tested conditions, preserving content and staying within CPU budget. Next steps include evaluating higher zstd levels, broader content types and sizes, and exploring range requests, pre-compressed origins, and passing compressed objects directly to downstream components that support them.