How to Implement Dynamic Rendering with Puppeteer and Node.js

Oct 8, 2026October 8, 2026·Updated Oct 9, 2026October 9, 2026·by Aki from Encited

Implement dynamic rendering with Puppeteer and Node.js: setup, wait strategies, request blocking, caching and scaling, and when hosted prerendering fits better.

Open in ChatGPT

Table of contents

Puppeteer dynamic rendering means running the renderer yourself. Headless Chrome loads each page, waits for your JavaScript to finish, and saves the HTML, and a routing rule sends crawlers to that HTML. The first version takes an afternoon. Then the upkeep starts: knowing when a page has finished loading, returning real status codes, keeping Chrome alive, refreshing snapshots, and keeping up with new crawlers.

Encited runs all of that for you. Self-hosting can work for a small site that rarely changes and has an engineer to own it. For a large or fast-changing site, a managed service costs less than the engineering time.

What a dynamic rendering setup needs

Every setup has the same five parts, whether you build them or pay for them:

1. A routing rule at your CDN or server that checks the user agent and sends crawlers to rendered pages.
2. A renderer. Headless Chrome, driven by Puppeteer or Playwright, loads the page and saves the final HTML.
3. A cache of rendered pages, so crawlers aren't waiting on a fresh render for every request.
4. A refresh process that re-renders pages when they change and on a schedule.
5. Monitoring that tells you when snapshots break.

For the concept, read what dynamic rendering is.

How Puppeteer renders JavaScript

A crawler that doesn't run JavaScript makes one HTTP request and reads the HTML that comes back. For a React, Vue or Angular single-page app, that's an empty container and a list of scripts. GPTBot, ClaudeBot, PerplexityBot and link preview bots all work this way.

Dynamic rendering with headless browsers works through a library like Puppeteer, which controls Google Chrome or Chromium from Node.js. It opens the URL in a real browser tab, which downloads and runs your JavaScript, calls your APIs and builds the page. page.content() then returns the HTML of the page as it stands, with the text, links, titles and structured data your framework added. That HTML is what a dynamic rendering setup serves to crawlers.

Your options for self-hosting

If you still run Rendertron, Rendertron alternatives compares the replacements.

Set up a Puppeteer renderer

1. Install Puppeteer. Current Puppeteer releases need Node.js 18 or later. npm install puppeteer (or yarn add puppeteer, pnpm add puppeteer) also downloads a matching build of Chrome for Testing, a few hundred MB on disk. To use a Chrome or Chromium you already have, install puppeteer-core instead and point it at the browser with executablePath, or set PUPPETEER_SKIP_DOWNLOAD (older guides say PUPPETEER_SKIP_CHROMIUM_DOWNLOAD).

2. Prepare the container. Headless Chrome needs the usual system libraries for fonts and graphics. Docker limits /dev/shm to 64 MB by default, which Chrome outgrows. Launch Chrome with --disable-dev-shm-usage, or give the container more shared memory with --shm-size. Many guides also add --no-sandbox and --disable-setuid-sandbox to get Chrome running in Docker. Those turn off Chrome's sandbox, so use them only when the container can't support it, since the renderer loads pages you don't fully control. Never add --disable-web-security to a renderer: it lets pages read data across origins.

3. Launch one browser and reuse it. Launching Chrome for every render wastes time and memory. Launch once, open a new tab for each render, and close the tab when it's done.

4. Cap concurrency. Each open tab holds a full page in memory. Put renders behind a queue that allows a fixed number at a time, and raise the number only after you've measured (see sizing below). The puppeteer-cluster package gives you a pool of pages or browsers with retries and a concurrency limit if you'd rather not build the queue.

5. Restart Chrome on a schedule. Long-running Chrome processes grow over time. Relaunch the browser after a set number of renders, once the renders still running have finished, and whenever it disconnects.

A production-ready render function

import puppeteer from "puppeteer";

const ALLOWED_HOSTS = new Set(["www.example.com"]);
const BLOCKED_TYPES = new Set(["image", "media", "font"]);
const MAX_RENDERS_PER_BROWSER = 500;

let browserPromise;
let rendersSinceLaunch = 0;
let activeRenders = 0;

function getBrowser() {
  if (!browserPromise) {
    const launching = puppeteer.launch({ args: ["--disable-dev-shm-usage"] });
    const forget = () => {
      if (browserPromise === launching) browserPromise = undefined;
    };
    launching.then((b) => b.on("disconnected", forget), forget);
    browserPromise = launching;
  }
  return browserPromise;
}

function retireBrowserIfDue() {
  if (activeRenders > 0 || rendersSinceLaunch < MAX_RENDERS_PER_BROWSER) return;
  const old = browserPromise;
  browserPromise = undefined;
  rendersSinceLaunch = 0;
  old?.then((b) => b.close()).catch(() => {});
}

export async function render(url) {
  if (!ALLOWED_HOSTS.has(new URL(url).hostname)) {
    throw new Error(`Host not allowed: ${url}`);
  }

  activeRenders++;
  rendersSinceLaunch++;
  let page;

  try {
    page = await (await getBrowser()).newPage();

    await page.setRequestInterception(true);
    page.on("request", (req) => {
      if (BLOCKED_TYPES.has(req.resourceType())) req.abort();
      else req.continue();
    });

    const response = await page.goto(url, {
      waitUntil: "networkidle2",
      timeout: 20000,
    });

    if (await page.evaluate(() => "prerenderReady" in window)) {
      await page.waitForFunction(() => window.prerenderReady === true, {
        timeout: 10000,
      });
    }

    const metaStatus = await page
      .$eval('meta[name="prerender-status-code"]', (el) => Number(el.content))
      .catch(() => null);
    const status =
      metaStatus >= 200 && metaStatus < 600 ? metaStatus : (response?.status() ?? 200);

    return { status, html: await page.content() };
  } finally {
    await page?.close().catch(() => {});
    activeRenders--;
    retireBrowserIfDue();
  }
}

What each part does:

• ALLOWED_HOSTS stops the renderer from loading any URL it's handed. A renderer that fetches arbitrary URLs can be pointed at internal services, which Semgrep showed happening to real dynamic rendering engines. Block internal IP ranges at the network level too.
• Request interception skips images, video and fonts. Crawlers read the HTML, so those downloads only slow the render down. Keep stylesheets if your app reads layout or hides content with CSS.
• networkidle2 waits until no more than two connections have been open for half a second. It tolerates analytics beacons and websockets that never close, which would stall networkidle0.
• The ready flag lets the app say when it's done. Pages that set window.prerenderReady = false at startup and true after their data loads get captured at the right moment. Pages without the flag skip the wait.
• The status meta tag turns your app's not-found screen into a real 404. Set <meta name="prerender-status-code" content="404"> on that route, and the renderer returns 404 instead of the shell's 200. A missing or invalid value falls back to the page's own status.
• The browser pool shares one Chrome across renders, launches it only once even when several renders start together, and relaunches it after MAX_RENDERS_PER_BROWSER renders once no render is still using it.

How to wait for dynamic content to load

Pick the first option that fits your app:

1. A ready flag. The most reliable. Your app sets it once data has loaded, and the renderer waits for it with page.waitForFunction.
2. A selector. If the app always renders a known element last, wait for it with page.waitForSelector("[data-loaded]"). This breaks quietly when the markup changes.
3. Network idle. networkidle2 works for most pages without app changes. Chained API calls that start after a pause can still be missed.
4. A fixed delay. A last resort. A short delay cuts off slow pages, and a long one slows every render.

Always keep a hard timeout. Without one, a page that never settles holds a tab until Chrome runs out of memory.

Route crawlers to the renderer

In an Express app, the routing rule looks like this:

const CRAWLERS =
  /googlebot|bingbot|applebot|gptbot|oai-searchbot|chatgpt-user|claudebot|claude-user|perplexitybot|perplexity-user|linkedinbot|twitterbot|slackbot|facebookexternalhit/i;

app.use(async (req, res, next) => {
  const isPage = req.method === "GET" && !/\.\w+$/.test(req.path);
  if (!isPage || !CRAWLERS.test(req.get("user-agent") ?? "")) return next();

  try {
    const url = `https://www.example.com${req.originalUrl}`;
    const snapshot = (await cache.get(url)) ?? (await renderAndCache(url));
    res.status(snapshot.status).set("Vary", "User-Agent").send(snapshot.html);
  } catch {
    next(); // Render failed: serve the app as usual
  }
});

It sends only page requests from known crawlers to snapshots, passes assets and API calls through, and sets Vary: User-Agent so a CDN doesn't serve the snapshot to people. If a render fails, the crawler gets your app instead of an error. Serve from the cache whenever you can. Crawlers that wait on a live render can time out.

The bot routing guide has the full crawler list and what to leave out. The Rendertron alternatives post shows the same rule as a Cloudflare Worker and an nginx proxy.

Sizing the server

Memory per tab depends on your pages, so measure instead of guessing:

1. Pick your heaviest pages. Long product listings, pages with many widgets, pages with large data loads.
2. Render them at the concurrency you plan to run, in the same container image you'll deploy.
3. Watch peak memory and CPU of the Chrome processes while they render, and note how long each render takes.
4. Size with headroom, and set the concurrency cap so peak memory stays below the container limit.

Published setup guides put the floor at 512 MB of memory and about 300 MB of disk for Chrome. That's enough to start one browser. Heavy pages at real concurrency need far more. Disk use beyond that is a temporary profile per browser.

Cache the output

Store each snapshot with its status code and the time it was rendered, keyed by URL: Redis, a key-value store or object storage all work. Serve crawlers from the cache, refresh entries on a schedule that matches how often each section changes, and re-render a page as soon as you publish.

Plan for the load on your own site too. Re-rendering thousands of pages sends your origin a burst of traffic. Slow the queue down when your server's response time or error rate goes up, or real visitors feel it.

Where self-hosted renderers break

The code above handles the first problems. These stay with you:

Chrome itself. Tabs hang and processes crash. The restart policy covers slow growth, and you still need health checks and alerts.

Freshness. Snapshots go stale as content changes. Product pages might need refreshing hourly and a blog once a week. You also need a way to re-render a page right after you publish.

Cached failures. A render during a deploy or an outage can save the error page. Check each page before saving it, or crawlers get that error until the next refresh.

Redirects. Redirects your app performs in the browser come back as a 200. Turn them into 301s with redirect rules in front of the app.

The crawler list. New AI crawlers keep appearing. Any user agent missing from your rule gets the empty app. Someone has to watch logs and update the list.

Chrome updates. Chrome ships a new stable version about every four weeks. An outdated Chrome stops parsing new JavaScript syntax and renders pages blank while still returning 200. Update Puppeteer, which pins its Chrome version, at the same pace.

Large teams do run this in-house. Mercari built its own dynamic rendering service, and Trendyol wrote about rendering 2 billion pages with a headless browser. Both show how much work it takes at scale.

Test what crawlers get

1. Request a page as a crawler. curl -A "Googlebot" https://www.example.com/pricing and curl -A "GPTBot" ... should return the full page text.
2. Request it as a browser. curl -A "Mozilla/5.0" ... should return your normal app, without the snapshot.
3. Request a missing URL as a crawler. It should return 404.
4. Check Google's view. URL Inspection in Search Console shows the HTML Googlebot fetched.
5. Compare crawlers. Our crawler simulator shows what Googlebot, Bingbot and AI crawlers each receive.

Self-hosted vs Encited

When to self-host

• The site is small and changes rarely, so snapshots can be built on deploy.
• An engineer has time to own the renderer long term.
• Compliance requires everything to run inside your own infrastructure.

When to use Encited

If you're looking for a better solution than Puppeteer for dynamic rendering, a managed service takes over the browsers, the cache, the refresh schedule and the crawler list.

• You have thousands of pages, or content that changes daily.
• Nobody on the team wants to be on call for headless Chrome.
• You want AI crawlers served properly without building it.

Encited's fleet of thousands of browsers renders your pages ahead of time, on refresh schedules you set per URL group. Every page gets a health check before it renders, and broken pages are never cached. The renderer waits until the page settles, or for your ready signal, and slows down when your server does. Search crawlers get lean HTML, AI crawlers get HTML or Markdown, and both are served from 300+ locations.

Setup is one DNS change or a middleware snippet for your host, with no changes to your app. On Express, that's one middleware file that calls the Encited render API with your API key, with no package to install (Express guide, Next.js guide). Pages can be re-rendered on demand through the cache invalidation API. Plans start at $19 a month with about 10,000 renders included, and extra renders cost from $1.25 per 1,000.

FAQ

How do you implement dynamic rendering with Puppeteer in Node.js? Install Puppeteer on Node.js 18 or later, launch one shared browser, and write a render function that opens a tab, waits for the page to finish loading, and returns page.content() with the right status code. Cache the results, and add middleware that sends crawler requests to the cache. Encited does the same without the scripts to maintain.

Does dynamic rendering with Puppeteer improve SEO for single-page apps? It gets your content to crawlers. Googlebot indexes it without waiting for its render queue, and AI crawlers and link previews, which don't run JavaScript, see it at all. Rankings still depend on the content itself.

How do I wait for dynamic content to load in Puppeteer? Use a ready flag your app sets after its data loads, and wait for it with page.waitForFunction. Without one, waitUntil: "networkidle2" works for most pages. Keep a hard timeout either way.

How do I return a 404 from a Puppeteer renderer? Have your not-found route set <meta name="prerender-status-code" content="404">, read it in the renderer after the page loads, and return that status with the snapshot.

What launch flags does Puppeteer need in Docker? Usually --disable-dev-shm-usage, or a larger --shm-size on the container, because Docker's default 64 MB of shared memory is too small for Chrome. Avoid --no-sandbox unless the container can't run Chrome's sandbox.

How does Puppeteer handle JavaScript compared to a static HTML fetcher? A static fetcher gets only the HTML the server sends. Puppeteer runs the page in Chrome, so it sees everything JavaScript adds: content from APIs, titles and meta tags set at runtime, and internal links.

Is Puppeteer or Playwright better for dynamic rendering? Both drive headless Chrome and both work. Playwright has more built-in waiting helpers. Every hard part in this post applies to both.

When should you choose self-hosted Puppeteer over a managed prerendering service? Self-host when the site is small, changes rarely, and an engineer can own the renderer, or when compliance keeps everything in-house. Choose a managed service when you have thousands of pages, content that changes daily, or nobody to run headless Chrome. Encited also shows which crawlers fetched each page and what they got.

How much memory does a headless Chrome renderer need? It depends on your pages and how many render at once. Measure your heaviest pages at your planned concurrency before sizing servers.

Is Rendertron still maintained? No. See Rendertron alternatives if you're still running it.

Prerender vs Puppeteer: which should I use for dynamic rendering? Puppeteer is a library: you write the renderer, cache, routing and refresh yourself. Prerender.io's open-source server wraps headless Chrome into a rendering service, and you still host it and build the caching and refresh. A managed service like Encited runs all of it, including the crawler list and AI crawler formats.

What are the alternatives to dynamic rendering? Server-side rendering, static site generation and hydration, which Google recommends for new builds, and build-time prerendering when every URL is known ahead of time. Dynamic rendering stays the option for apps you can't rebuild and for content that loads after the first response. The trade-offs are in dynamic rendering vs server-side rendering.

Can I prerender at build time instead? Yes, if your URLs are known at build time and content only changes on deploys. Dynamic rendering suits sites where content changes between deploys.