Cloudflare Workers for SEO: Rewriting the Metadata Layer at the Edge

The short answer: a Cloudflare Worker is a small JavaScript program that Cloudflare runs on every matching request before it reaches your server. Because it sits between the crawler and your origin, it can read the response in flight and rewrite most of what search engines actually look at: the <title>, the meta description, canonical and hreflang links, JSON-LD structured data, and the HTTP headers. This article works through that metadata layer in detail, with before/after HTML and code you can adapt, then covers the two things that usually go wrong: caching and per-request cost.

What a Worker actually intercepts

A Worker attached to a route (for example example.com/*) receives every request that matches. Your handler calls fetch(request) to forward it to the origin, and gets a Response back. Between those two moments you can do anything: rewrite the URL, read the User-Agent header, call an API, or transform the HTML.

The transform is where SEO work happens. Cloudflare’s HTMLRewriter is a streaming HTML parser with a jQuery-like API. You register handlers for elements by selector, and it walks the document rewriting only what you target, without ever touching the bytes on your origin server. The full API is documented in the Cloudflare HTMLRewriter reference.

Cloudflare’s own five-minute explainer, Cloudflare Workers Explained, contrasts exactly this: a traditional Node.js and Express origin where every user-facing tweak means a redeploy, versus a Worker that intercepts the request at the edge and changes the response in transit. Worth watching before you write anything.

The one mental model to keep: the Worker sees your origin response, and what it returns is what the visitor and Google both receive. Whatever your visitor’s browser downloads, Googlebot downloads the same bytes. That fact drives both the opportunities below and the risks at the end.

Everything you can rewrite in the metadata layer

Title and meta description

This is the highest-leverage rewrite on the page. The title is the first line of your result in search, the description is the snippet under it. If your CMS concatenates “Untitled” into every blog title, or a migration left 400 pages with the same description, a Worker fixes all of them in one deploy:

export default {
  async fetch(request) {
    const url = new URL(request.url);
    const res = await fetch(request);
    const type = res.headers.get('content-type') || '';

    if (res.status !== 200 || !type.includes('text/html')) return res;

    const path = url.pathname;

    return new HTMLRewriter()
      .on('title', {
        element: (el) => el.setInnerContent(titles[path] ?? defaultTitle),
      })
      .on('meta[name="description"]', {
        element: (el) => el.setAttribute('content', descriptions[path] ?? defaultDescription),
      })
      .transform(res);
  },
};

Before, what the origin sends to everyone:

<head>
  <title>Untitled</title>
  <meta name="description" content="Welcome to our website.">
</head>

After the Worker has rewritten it:

<head>
  <title>Ergonomic Chairs for Developers | AcmeCorp</title>
  <meta name="description"
        content="Twelve ergonomic chairs tested over 200 hours of desk work, with weight limits, back support, and our pick for tall developers.">
</head>

The origin file on disk never changes. The same request that renders for a visitor is rewritten in memory on Cloudflare’s network before anyone consumes it, and the CMS keeps serving the template it knows.

Two details matter. setInnerContent on title replaces the text between the tags; if the tag is missing entirely, the handler never runs, so add one to a head handler (el.prepend(newTitleTag, { html: true })) to create it. And the rewrite only works on HTML responses: the content-type and status guards in the snippet exist so a JSON API or a 404 page is passed through untouched.

Canonical and hreflang

When a template hardcodes a canonical to a staging domain, or a ?sort= parameter creates ten copies of one page, the canonical is the tag that tells Google which URL is authoritative. hreflang does the same job across language versions. Both are plain attributes, so they are simple for the rewriter:

.on('link[rel="canonical"]', {
  element: (el) => el.setAttribute('href', `https://${url.host}${path}`),
})
.on('link[rel="alternate"][hreflang="fr"]', {
  element: (el) => el.setAttribute('href', `https://${url.host}/fr${path}`),
})

Same caveat as the title: an existing tag gets corrected, a missing one needs head.prepend(...). For multilingual sites this is the difference between one strong page and several weak ones: Google’s own documentation treats hreflang as the signal for which language version to show where.

JSON-LD blocks before `</head>`

Structured data earns your listings the rich markup (breadcrumbs, reviews, FAQ accordions) and, increasingly, tells answer engines what a page is about. You can inject a whole application/ld+json script where no plugin ever would: right before the closing </head> tag, using the head element’s append, which inserts content just before the end tag:

.on('head', {
  element: (el) => el.append(`
    <script type="application/ld+json">
    {"@context":"https://schema.org","@type":"Article",
     "headline":"${titles[path]}",
     "datePublished":"${publishedDates[path]}"}
    </script>`, { html: true }),
})

The second argument, { html: true }, tells HTMLRewriter to treat your string as markup rather than escaped text. One genuine caveat: escape any literal </script> sequence that appears inside your JSON before injecting, or a crafted string could close your tag early and let HTML after it land outside the script. (This is a well-known injection pattern; the HTMLRewriter reference documents the contentOptions argument.)

Response headers

Headers go on the response the same way:

const out = new HTMLRewriter().on(...).transform(res);
out.headers.set('x-robots-tag', 'max-snippet:50, max-image-preview:large');
out.headers.set('cache-control', 'public, max-age=3600');
return out;

x-robots-tag gives you per-page control over how Google truncates your snippet, which complements the description work above. The second header is a different story, and it is where the Worker meets Cloudflare’s cache.

Googlebot vs visitors: where the line is

The Worker can also read the request before fetching anything, and branch on it:

const ua = request.headers.get('user-agent') || '';
if (/googlebot/i.test(ua)) {
  // send the crawler a canonical form, or skip a variant that is mid-test
}

This is the most misused capability in edge SEO, so the boundary matters. Google’s spam policies define cloaking as “presenting different content to users and search engines with the intent to manipulate search rankings,” and the same page names the exact technique, “inserting text or keywords into a page only when the user agent… is a search engine,” as an example of it. If what you serve Googlebot is substantially different from what a human sees, a manual action is the downside, not a hypothetical.

Where the check is defensible: making sure Googlebot gets the same metadata everyone gets, when some other layer (a cookie wall, a bot-fighting script, an interstitial) would otherwise hide it; and giving a search engine the canonical of the page you want indexed. What the existing articles on this site cover, conditional title injection and A/B testing titles at the edge, works only if the “B” variant you show Googlebot is also a real page state a human could plausibly get. Keep the difference to the metadata layer, keep every variant truthful about the page’s content, and treat anything beyond that as a cloaking question, not an SEO trick.

What the Cloudflare cache changes

Here is the part most writeups get wrong, because the default is surprising. Per Cloudflare’s default cache behavior, the CDN does not cache HTML or JSON by default (it caches static file extensions, and robots.txt). So a Worker that calls fetch(request) and rewrites HTML does not magically get caching: every page view and every crawl is a round trip to your origin, plus the CPU cost of the rewrite.

That changes once you opt in. Serving a cached copy also stops the origin and the rewriter from running on every hit, which is what makes edge SEO cheap at scale. The pattern Cloudflare documents for the Cache API is:

const cacheKey = new Request(request.url, request);
const hit = await caches.default.match(cacheKey);
if (hit) return hit;

// ... fetch origin and build the rewritten response `out` ...

out.headers.set('cache-control', 'public, max-age=3600');
ctx.waitUntil(caches.default.put(cacheKey, out.clone()));
return out;

The stored page’s Cache-Control header controls the TTL: the API respects Cache-Control, Expires, ETag and Last-Modified on the response you put. The ctx.waitUntil writes the cache entry after the visitor’s copy has started streaming, so caching never delays the response. (If you instead want Cloudflare’s own edge cache to handle it, fetch(request, { cf: { cacheEverything: true, cacheTtl: 3600 } }) is the one-line alternative.)

The edge itself is a network of machines like these, and the whole point of caching is that most visitors never reach your origin at all.

Server racks in a data center — Photo by cookiecutter, Pexels License, via Pexels

Surrogate keys, which is Cloudflare’s name for them, are Cache Tags. If you have come from Fastly, you know the pattern: you tag a response and later purge or inspect everything carrying that tag. Cloudflare implements it with the Cache-Tag response header, for example Cache-Tag: blog,product-images. The edge strips the header before it reaches the visitor, and you can purge every URL with a tag at once from the dashboard or API, which forces cf-cache-status: MISS on the next request. That is the escape hatch when a rewrite changes and you do not want to wait out the TTL: bump the tag name, purge the tag, done.

Four pitfalls worth designing around

Pitfall	What it looks like	How to avoid it
Latency per request	Every uncached page costs an origin fetch plus parsing and rewriting. Buffering the whole body with `res.text()` before rewriting adds a full round trip. On the free Workers plan the CPU budget is 10 ms per request (Cloudflare’s docs note the average Worker uses about 2.2 ms).	Stream with `.transform(res)` when you can; only buffer when you must; add caching once the rewrite is correct.
Breaking the origin response	Rewriting a JSON response as HTML, crashing on a page with no `</head>`, or double-injecting after a retry.	Guard on `content-type` and status; wrap the transform in try/catch and return the original `res` on error so the site stays up even if the Worker is broken.
Caching too aggressively	A page that rewrites by user-agent gets cached under a plain URL key, so the first Googlebot visit caches Googlebot’s title and everyone else gets served it for an hour.	Cache by user-agent class for those pages, or don’t cache them; use `Cache-Tag` so a bad value can be purged instantly.
Cloaking	Serving materially different content to crawlers.	Keep the difference to metadata that honestly describes the visible page; read Google’s spam policies before shipping anything crawler-specific.

Two more dependable gotchas, both from the same cache docs: a response carrying Set-Cookie is never cached, so login walls and A/B frameworks that set cookies sit outside the win; and free, Pro and Business plans cap cacheable object size at 512 MB, irrelevant for HTML but worth knowing when you tag assets.

Worker or origin plugin?

If your team owns the templates, a plugin that writes metadata in the CMS is usually the right default: it edits the same file the visitor gets, it is legible to the next developer, and it cannot add per-request latency. The Worker wins when a plugin is impossible or wrong:

Situation	Origin plugin	Cloudflare Worker
CMS you can edit, metadata identical for everyone	Yes, prefer it	Unnecessary
Hosted or headless CMS you cannot touch (SaaS, static host, another agency’s build)	Not available	The only option
Metadata must vary per request (user-agent, language, country, A/B variant)	Awkward, template-level	Natural fit, one code path
Multiple sites, one metadata system	One plugin per codebase	One Worker, shared logic
You want the origin to stay the single source of truth and the search-facing view to be a layer on top	No	The whole point

That last row is the architecture that products like SEOEdgeAI are built on: a Worker in front of the site, an edge proxy that rewrites titles, descriptions and JSON-LD from a strategy it derives from Search Console data, and the origin untouched. When your metadata changes weekly, a proxy that can rewrite without a deploy is not a hack, it is the mechanism.

For the copy-paste patterns behind each of these rewrites, our seven practical Cloudflare Worker examples builds each one in isolation. This article is about the layer they all live in, the tags between the user and Google, and how to change them without ever changing the page underneath.