Can AI Crawlers Read JavaScript? Rendering for Answer Engines

Google has rendered JavaScript for years, so many teams stopped worrying about it. A page could load an empty shell, fetch its content with JavaScript, and still rank. AI answer engines changed that assumption. Most of their crawlers read the HTML the server sends and nothing more.

This article explains what the research shows, how to check whether a client's site is affected, and the practical fixes for common frameworks and site builders.

What does the research show?

The clearest evidence comes from a study by Vercel and the analytics firm MERJ, published in December 2024. It analysed crawler traffic across Vercel's network, which hosts a large share of modern JavaScript sites, and checked which crawlers executed JavaScript.

The findings were stark:

The study also measured volume. GPTBot alone made hundreds of millions of requests across Vercel's network in a month, with Claude's crawler close behind. These crawlers are active. They just do not see what the browser sees.

The study is from late 2024, and crawler behaviour can change. We have not seen a newer study showing that the major AI crawlers now render JavaScript, so the safe assumption for client work is that they do not.

Why this matters for AI visibility

If an AI crawler fetches a page and the HTML contains only a loading spinner and a script tag, the engine has nothing to read, quote or cite. The page may rank in Google, because Googlebot renders it, while being invisible to ChatGPT search, Claude and Perplexity.

The common symptoms are:

• A client ranks well on Google but is never cited by Perplexity or ChatGPT for topics it covers in depth.
• AI answers describe the client using only information from third-party sites.
• Prices, product details or reviews that load through JavaScript never appear in AI answers.

Which sites are affected?

Sites are at risk when important content is assembled in the browser. Common cases:

• Single-page applications built with React, Vue or Angular without server-side rendering.
• Headless sites where the front end fetches content from an API after the page loads.
• Content in tabs, accordions or carousels that load their content only when clicked.
• Product details, prices and stock pulled in by client-side scripts.
• Reviews and ratings injected by third-party widgets.
• Some site builders and themes that render sections client-side.

Traditional server-rendered sites, such as most WordPress themes, are usually fine, though plugins and page builders can still load key content with JavaScript.

How to check what an AI crawler sees

You do not need special tools. Three quick checks cover most cases.

1. View the raw HTML

Fetch the page without running scripts. From a terminal:

curl -s -A "Mozilla/5.0 (compatible; GPTBot/1.0)" https://example.com/page | less

Search the output for the page's main heading, key sentences, prices and internal links. If they are missing, AI crawlers that do not render will miss them too.

2. Disable JavaScript in the browser

Open the browser's developer tools, disable JavaScript and reload. What remains is roughly what a non-rendering crawler sees. It is a quick way to show clients the problem.

3. Compare rendered and raw content at scale

For larger sites, crawl once with JavaScript rendering on and once with it off, then compare word counts, headings and links per URL. Pages with large differences are the ones to fix first.

How to fix it

The goal is simple: the server should send HTML that already contains the content. How you get there depends on the stack.

Server-side rendering

The server builds the full HTML for each request. Frameworks such as Next.js, Nuxt, SvelteKit and Remix support this directly. Make sure the content-bearing components render on the server, not only after data is fetched in the browser.

Static site generation

Pages are built to HTML ahead of time. This is ideal for content that changes occasionally, such as blog posts, guides, documentation and most marketing pages. Next.js, Astro, Hugo and Eleventy all support it.

Incremental or on-demand regeneration

For large catalogues, pages are generated statically and refreshed periodically or when content changes. This combines the reach of static HTML with fresh data.

Pre-rendering services

If rebuilding the front end is not an option, a pre-rendering service can serve rendered HTML to crawlers. Make sure the content served to crawlers matches what users see; serving different content would be cloaking.

Fix the components, not just the pages

Even on server-rendered sites, check individual components:

• Tabs and accordions should include their content in the HTML and use CSS to hide it, rather than fetching it on click.
• Reviews from third-party widgets should also be rendered server-side, or summarised in the page's own HTML.
• Prices and availability should be in the initial HTML.
• Internal links should be real anchor elements with href attributes, not click handlers.

Notes for common platforms

Next.js. Server components and static generation send full HTML by default. Problems usually come from content fetched inside client components with hooks after the page loads. Move content fetching to the server and keep client components for interaction only.

React single-page apps (Create React App, Vite). These render in the browser by default, so AI crawlers see an empty shell. The long-term fix is moving to a framework with server rendering; a pre-rendering service is a stopgap.

WordPress. Core and most themes render on the server. Check page builders, review plugins, pricing tables and "load more" features, which sometimes fetch content with JavaScript.

Shopify. Theme templates render on the server, but apps that add reviews, size guides, bundles or dynamic pricing often inject content client-side. Check product pages with JavaScript disabled.

Webflow, Squarespace and Wix. Core page content generally renders on the server. Embedded widgets, dynamic collections loaded by scripts and some interactive components may not. Test the pages that matter.

Headless CMS front ends. Whether content is visible depends entirely on how the front end is built. Insist on server rendering or static generation for any page that should be cited.

What about Google?

Googlebot renders JavaScript, but rendering can be delayed and it costs crawl resources. Google's own documentation recommends server-side or static rendering where possible. Fixing rendering for AI crawlers usually helps Google too, by making content available on the first crawl.

A priority list for client sites

1. Identify the pages that matter most for AI visibility. Typically service pages, product pages, comparisons and guides that match the client's prompt set.
2. Run the raw HTML check on each. Note missing content.
3. Fix templates, not individual pages. One template fix often covers hundreds of URLs.
4. Re-check after deployment. Confirm content is present in raw HTML.
5. Watch citations over the following weeks. Live-retrieval engines such as Perplexity can pick up the change quickly. See AI share of voice.

Rendering is one item in a broader audit. For the full list, see our technical SEO audit checklist.

How this fits with crawler access

Rendering and access are two separate gates. A crawler must be allowed to fetch the page, which is controlled by robots.txt and firewall rules (see AI crawler user agents), and it must be able to read what it fetches. A site can pass one gate and fail the other. Check both.

How Rankwyn helps

Rankwyn's audits fetch each priority page the way a non-rendering AI crawler does and compare it with the rendered page, flagging templates where headings, prices or key content are missing from the raw HTML. Results roll up across every client site in one view, alongside citation tracking in the AEO Citation Radar.

Key takeaways

• The major AI crawlers from OpenAI, Anthropic, Perplexity and Meta do not execute JavaScript; Google's and Apple's do.
• Content assembled in the browser is invisible to most AI answer engines, even if it ranks on Google.
• Check raw HTML with curl or with JavaScript disabled, and compare rendered and raw crawls at scale.
• Fix with server-side rendering, static generation or carefully matched pre-rendering, and fix components such as tabs, reviews and prices.
• Rendering and crawler access are separate gates; both must pass.

The bottom line

If the server does not send the content, most AI engines never see it. Check what each priority page looks like without JavaScript, fix the templates that fail, and confirm the result in raw HTML. It is one of the most common and most fixable reasons a strong site goes unmentioned in AI answers. For the wider strategy, see our AEO and GEO guide.