2 September 2026 · 6 min read · Jason Youngjin Koh

ChatGPT cannot see your website

Our own site served zero characters of body text to any crawler that did not run JavaScript. We are a marketing agency. Here is how we found out, and how you can check yours in thirty seconds.

AI crawlers do not run JavaScript. If your website builds its content in the browser, GPTBot, ClaudeBot and PerplexityBot receive an empty shell: a title, a few meta tags, and a script tag. Every word a human reads on your homepage never reaches them. Our own site had exactly this problem until September 2026.

How do you check your own site in thirty seconds?

Copy a full sentence from your homepage. Open a terminal and run one command. If it returns 0, that sentence exists only in JavaScript and no AI crawler has ever read it.

You can do the same thing without a terminal. In Chrome, view source rather than inspect element. View source shows what the server actually sent. Inspect element shows the page after JavaScript has run, which is the version an AI crawler never sees. If view source is nearly empty, so is your site as far as ChatGPT is concerned.

Our own site served nothing

We are an AI growth and performance marketing agency and our website was invisible to AI. Before September 2026, scalemaster.com.au returned a 1,482 byte shell to any crawler that did not execute JavaScript. Not a thin page. Not a badly written page. Zero characters of body text.

0
Characters of body text scalemaster.com.au served to non JavaScript crawlers before September 2026

Nobody noticed because everything looked fine. The site rendered beautifully in a browser, Google indexed it, and Search Console was quiet. Google renders JavaScript for indexing, so the one system we were watching was the one system that could see us. Every other reader got a blank page, and none of them send you a report about it.

We fixed it by serving real content in the first response and putting a static entity declaration in the head. Here is the before and after, measured the same way you can measure yours.

Do AI crawlers really never execute JavaScript?

As far as the public evidence goes, yes. Vercel and MERJ analysed AI crawler traffic at scale and found no evidence of JavaScript execution by the major AI crawlers. GPTBot downloaded JavaScript files in roughly 11.5% of requests and ClaudeBot in roughly 23.8%, and neither ran them. They fetch the raw HTML response and read the text in it.

Googlebot is the exception. It renders JavaScript, which is why a site can rank respectably in Google and remain completely unreadable to ChatGPT, Claude and Perplexity at the same time. If you have been reassured by your Google rankings, that reassurance does not transfer.

Which kinds of sites have this problem?

Anything that assembles its content in the browser. That is a large share of modern websites, including plenty of expensive ones.

• React, Vue or Svelte applications built as single page apps with no server rendering.
• Next.js or Nuxt projects configured in pure client mode, which is common when a template was adopted quickly.
• Site builder exports that ship an application shell and hydrate the content afterwards.
• Any page where the main copy arrives from an API call after the initial load, including product listings, pricing tables and location pages.
• Content that only exists inside a component rendered on interaction, such as a tab or a load more button.

The tell is always the same. View source is short, and the words on the screen are not in it.

How do you fix it, in order of leverage?

Four fixes, and the first one is worth more than the other three combined. Do them in order and stop when the curl check returns a number above zero and the content is genuinely there.

1. Serve the content in the first HTTP response. Server side rendering, static generation, or a prerender step at build time. Whichever your stack supports most cheaply.
2. Put JSON-LD in the static head of the document, written into the HTML rather than injected by a script.
3. Write a real robots.txt that names GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot and allows them explicitly.
4. Publish a real sitemap.xml with genuine lastmod dates and only URLs that return 200.

Serve the content in the first response

This is the whole article in one line. Everything else is decoration if the words are not in the response body. You do not need to rebuild the site. On most stacks a prerender step, or switching a handful of routes to server rendering, is a day of work. If a full rebuild is genuinely the only option, prerendering just the homepage and your top five pages captures most of the value.

Put your structured data in the static head

JSON-LD injected by JavaScript has the same problem as body text injected by JavaScript. It is not there when it is read. Write it into the HTML template so it ships with the first response. One organisation block with a stable identifier does more work than a dozen schema types added by a plugin.

Write a robots.txt that names the AI crawlers

A default robots.txt with a blanket allow usually works, but many agency and platform defaults quietly block unfamiliar user agents, and some security products block them at the edge before robots.txt is ever consulted. Naming the crawlers explicitly removes the ambiguity, and it makes the decision visible to whoever inherits the site after you.

Check the edge as well as the file. If your CDN or firewall has a bot protection rule turned on, test it with a request that identifies itself as GPTBot and confirm you get a 200 rather than a 403.

What changes after you fix it?

You become eligible. Being readable is a precondition for being cited, not a guarantee of it. What we can say honestly is that before the fix the probability was zero, because there was nothing to read, and that a crawler cannot recommend a business whose description it has never seen.

Be careful with anyone who promises a specific lift in AI citations from a rendering fix. The honest framing is that this removes a hard blocker. Whether you then get cited depends on the same things it always depended on: whether your page answers the question, and whether anything outside your site corroborates you.

The companion problem is entity resolution: what happens when an engine can read your site perfectly and still describes a different company with your name. Read next

What to hand your developer

Send them this list. It is specific enough to be actioned without a meeting.

• Confirm the homepage and top five pages return their main copy in the raw HTML response, verified with curl and not with a browser.
• Move the JSON-LD into the static document head.
• Add explicit allow rules for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot in robots.txt.
• Confirm the CDN or WAF returns 200, not 403, to a request identifying as GPTBot.
• Regenerate sitemap.xml with real lastmod dates and verify every URL returns 200.

Then run the curl check again yourself. It takes thirty seconds, it needs no login, and it is the only piece of this you should verify personally.