A Next.js 16 prefetch loop on Cloudflare Workers, and what the bill taught me

On October 9 the Worker behind myfoodanalysis.com served 4.57 million requests to about a thousand users, because every browser tab left open on the site was requesting the same route-tree prefetch about 130 times a second. OpenNext’s cache interception was answering Next.js 16’s segment prefetches with the whole page. Turning interception off stopped it the same day; the real fix is @opennextjs/aws 4.1.6 or later, and the site now runs Next 16.3.8 with @opennextjs/cloudflare 1.20.10 and interception back on.

Why interception was on

MyFoodAnalysis runs Next.js 16 on Cloudflare Workers through OpenNext and has a page for each of 455,801 foods. About 90 percent of its traffic is crawlers. Its first bill on October 8 was small: KV writes $5.00, KV storage $3.50 and Workers CPU $1.86. But OpenNext’s KV page cache held 206 GB in 2.29 million keys. At $0.50 per GB-month, storage alone was heading for about $100 a month. OpenNext puts the build id in every key and sets no expiry, and I had deployed 32 times in two weeks.

Worse, Next treated nearly every cached page as stale. It looks up a page’s lifetime in an in-memory map that only a render in the same isolate fills, and the comment in its calculateRevalidate says what happens otherwise: “if an entry isn’t present in routes we fallback to a default of revalidating after 1 second”. Most Worker isolates had never rendered the page they served, so each view triggered a background re-render, an API call and a cache write.

On October 8 I moved the cache to R2 with a seven-day lifecycle rule, put OpenNext’s regional cache (the Cache API in each Cloudflare location) in front of it, and set enableCacheInterception: true. Interception lets OpenNext answer a cached page itself, without starting Next. A fresh food page came back MISS once, then x-opennext-cache: HIT with s-maxage counting down from 86400. The smoke tests passed, and I deployed.

The loop

Next’s <Link> prefetches links as they come into view, and in Next 16 the router asks for each link’s route tree with a Next-Router-Segment-Prefetch: /_tree header. OpenNext’s interceptor had a branch for segment requests, but it was gated on !prefetchInlining, and Next has turned prefetchInlining on by default since 16.2. That branch never ran, so every segment request got the full page payload. The @opennextjs/aws 4.1.6 changelog describes the result: “The router never recorded the prefetch as satisfied and re-requested it indefinitely.”

One tab on the Food Finder made 3,293 requests in 25 seconds. They were all _tree prefetches for /finder and /app, all answered HIT. None of my checks could see it. curl doesn’t prefetch, and neither do smoke tests or ten fetches of a page. It takes a browser that stays on the page.

How to tell if you have it

Compare requests with people. Put the Worker’s requests per day in Workers analytics next to the users your analytics count. Millions of requests from a thousand users is a loop, not a bot. Check the paths too. Mine were almost all nav pages (/finder, /app, /, /categories, /compare, /terms), not the food pages crawlers go for.

Leave a browser open and count. I now open a page in headless Chromium and count its requests for twenty seconds. Simplified:

# uv run --with playwright python3 request-loop.py https://example.com/finder 20
import sys, collections
from playwright.sync_api import sync_playwright

url, secs = sys.argv[1], int(sys.argv[2])
reqs = collections.Counter()
with sync_playwright() as p:
    page = p.chromium.launch().new_page()
    page.on("request", lambda r: reqs.update([(
        r.url.split("?")[0], r.headers.get("next-router-segment-prefetch", "-"))]))
    page.goto(url, wait_until="load")
    page.wait_for_timeout(secs * 1000)
    print(sum(reqs.values()), "requests")
    for (path, seg), n in reqs.most_common(10):
        print(f"{n:6d}  seg={seg}  {path}")

A healthy page makes a handful, about one prefetch per nav link, and the same page now makes 33 in 20 seconds. Hundreds a second with seg=/_tree means you have the loop.

The fix

On October 10 I first turned interception off with enableCacheInterception: false, the line that had started it. Production dropped to 33 requests in 20 seconds on that page, from 2,600, but the one-second staleness came back.

The real fix had shipped before I ever turned interception on. @opennextjs/aws 4.1.6 came out on September 28, and @opennextjs/cloudflare 1.20.7 picked it up the next day. My lockfile still held 1.20.6, with aws 4.1.4. I moved to the current release, which requires Next 16.3.8 or later, and turned interception back on:

"@opennextjs/cloudflare": "^1.20.10",
"next": "^16.3.8",
"wrangler": "^4.148.0"

On the preview I checked both failures before deploying. A fresh food page returned MISS once, then HIT with s-maxage at 86400, 86398 and 86395. The browser script counted 29 requests. While the loop ran, the Worker served 120,000 to 230,000 requests an hour. By this morning it was serving 8,400 to 8,800.

What the bill taught me

Next I looked at R2. In less than three days the bucket had grown to 46 GB and 261,000 objects. A listing showed that 98 percent of it was rendered pages at about 177 KB each (HTML plus the React payload, uncompressed). Crawlers fetching 100,000 to 150,000 different food pages a day had written them, and only a quarter to a third were ever read again. And 42 of the 46 GB belonged to two builds the October 10 deploys had already orphaned.

An R2 write costs $4.50 per million. A food page takes about 100 ms of CPU to render, and at $0.02 per million CPU milliseconds that’s about $2 per million renders. Each write earned about 0.4 later hits. Even after the roughly $1 a month of CPU it saved, the store was losing about $7 a month. On a long-tail site that crawlers read once or twice a day, a cache write costs more than the render it saves.

So tonight’s deploy drops the store. A rendered page now lives only in the Cache API of the location that rendered it, for its revalidate time of a day. The Cache API doesn’t replicate between locations. Behind it sits a read-only store that serves only the pages prerendered at build time, from the Worker’s static assets:

import { defineCloudflareConfig } from "@opennextjs/cloudflare";
import { withRegionalCache } from "@opennextjs/cloudflare/overrides/incremental-cache/regional-cache";
import staticAssets from "@opennextjs/cloudflare/overrides/incremental-cache/static-assets-incremental-cache";
import memoryQueue from "@opennextjs/cloudflare/overrides/queue/memory-queue";
import type { IncrementalCache } from "@opennextjs/aws/types/overrides.js";

// Prerendered pages, read-only. The static-assets cache's name tells the
// OpenNext build to copy the prerendered entries into the assets.
const prerenderedOnly: IncrementalCache = {
  name: staticAssets.name,
  get: (key, cacheType) => staticAssets.get(key, cacheType),
  set: async () => {},
  delete: async () => {},
};

export default defineCloudflareConfig({
  incrementalCache: withRegionalCache(prerenderedOnly, {
    mode: "long-lived",
    shouldLazilyUpdateOnCacheHit: false,
  }),
  queue: memoryQueue,
  enableCacheInterception: true,
});

The R2 binding comes out of wrangler.jsonc, and the old objects expire under the lifecycle rule. The trade-off is that each location now renders a page once a day, where before one render served the whole world. That means a little more CPU and API traffic, and no writes. After the deploy, a prerendered page returned HIT on its first fetch, fresh food pages returned MISS then HIT, and the browser script still counted 33.

While the cache was in KV, storage alone was heading for about $100 a month. With R2, the read I took early this morning put the run rate at about $26 a month: the $5 Workers base, about $9 of CPU, about $10 of R2 writes and about $2 of storage. Tonight’s deploy removes the last two and adds some CPU. I’ll read next week what that costs.

How I ran into it

On October 9, after the first fix, the analytics showed 1.68 million requests over eight hours from four addresses. They all had an ordinary Chrome user agent and nearly all went to the nav pages. I took it for a crawler flood, but it was visitors’ browsers with a tab left open. The next day I opened /finder in headless Chromium and counted. Now, before and after any change to the cache or routing layer, I leave a page open in a browser and count its requests. And I read the Worker’s requests per hour the morning after a deploy, not a week later.