Encited

How to Integrate Pre-rendering with Next.js

How Next.js pre-rendering works#

Next.js can call Encited from middleware.js or the Next.js 16 proxy.js convention for client-rendered routes that need crawler-ready HTML.

Eligible crawler HTML requests call Encited before route rendering. Browser traffic, assets excluded by the matcher, and render fallbacks continue through Next.js.

Prerequisites#

  • A server or edge deployment; a static-only export cannot execute request middleware.
  • middleware.js for Next.js 15 and earlier or proxy.js for Next.js 16 and later.
  • LOVABLEHTML_API_KEY configured in the deployment environment.

Set up pre-rendering with Next.js#

1. Add the integration file#

Use middleware.js for Next.js 15 and earlier or proxy.js for Next.js 16+. The default export in the snippet works with either filename.

middleware

// middleware.js (Next.js 15 and earlier)
// For Next.js 16+, rename this file to proxy.js and export proxy instead.
import { NextResponse } from 'next/server';

const CRAWLER = /(?:(OAI-SearchBot|ChatGPT-User(?:\/\d+\.\d+)?|GPTBot|ChatGPT|OpenAI-))|(?:(anthropic-ai|ClaudeBot|claude-web|Claude-Web|Claude-User|Claude-SearchBot))|(?:\b(GrokBot|xAI-Grok|xAI-Bot|xAI-SearchBot)\b)|(?:(PerplexityBot|Perplexity-User))|(?:(Google-InspectionTool|GoogleOther(?:-Image|-Video)?|Googlebot|Google-CloudVertexBot|Storebot-Google|Google-NotebookLM|GoogleAgent-URLContext|GoogleAIOverviewRenderer|Google-Lens|Google-Jetski-Antigravity))|(?:(Microsoft-Copilot|CopilotBot|Copilot-Chat|Microsoft-CopilotPlugin))|(?:(BingBot|bingbot|BingPreview))|(?:facebookexternalhit.*Twitterbot|Twitterbot.*facebookexternalhit)|(?:(Meta-ExternalAgent|meta-externalagent|meta-externalfetcher|MetaBot|FacebookBot|facebookexternalhit))|(?:LinkedInBot)|(?:Amazonbot)|(?:(Applebot-Extended|Applebot|AppleNewsBot))|(?:Bytespider)|(?:(DuckAssistBot|DuckDuckBot))|(?:(cohere-training-data-crawler|cohere-ai|CohereAI))|(?:MistralAI-User)|(?:AI2Bot)|(?:CCBot)|(?:Diffbot)|(?:omgili)|(?:TimpiBot)|(?:YouBot)|(?:ExaSearchBot)|(?:(Bravebot|Brave-Search))|(?:Yandex)|(?:Twitterbot)|(?:Discordbot)|(?:(Slackbot|Slack-ImgProxy))|(?:TelegramBot)|(?:(Pinterest|Pinterestbot))|(?:Snapchat)|(?:Redditbot)|(?:Tumblr)|(?:(Mastodon|http\.rb))|(?:Viber)|(?:^Line\/)|(?:WhatsApp)|(?:(SkypeUriPreview|Skype))|(?:(Teams|MicrosoftPreview))|(?:Zoom)|(?:Notion)|(?:(Quora|QuoraBot|Quora-Bot|Poe-Bot))|(?:Medium)|(?:(Pocket|PocketParser|PocketImageCache))|(?:(Flipboard|FlipboardProxy))|(?:(Embedly|embed\.ly))|(?:YelpBot)|(?:(Baiduspider|Baidu))|(?:(Sogou|sogou))|(?:(Yeti|NaverBot|Naver))|(?:SeznamBot)|(?:(Qwantify|Qwant))|(?:Ecosia)|(?:MojeekBot)|(?:Mail\.RU_Bot)|(?:(Yahoo! Slurp|Y!J-))|(?:(SemrushBot|SEMrush))|(?:(AhrefsBot|AhrefsSiteAudit))|(?:(DotBot|Rogerbot|moz\.com))|(?:MJ12bot)|(?:Screaming Frog)|(?:SISTRIX)|(?:SEOkicks)|(?:serpstatbot)|(?:RSiteAuditor)|(?:(?:Encited|LvHTML)-SEOAuditBot)|(?:Barkrowler)|(?:HubSpot Crawler)|(?:(AwarioSmartBot|AwarioBot))|(?:DataForSeoBot)|(?:SERanking)|(?:Seobility)/i;
const ENCITED_FORWARD_HEADERS = [
  'accept-language', 'sec-fetch-mode', 'sec-fetch-site', 'sec-fetch-dest',
  'sec-fetch-user', 'upgrade-insecure-requests', 'referer', 'user-agent',
];
const ENCITED_RESPONSE_HEADERS = [
  'x-lovablehtml-render-cache', 'x-lovablehtml-snapshot-key', 'cache-control',
  'etag', 'content-digest', 'signature-input', 'signature',
];

export default async function encitedProxy(request) {
  const accept = request.headers.get('accept') || '';
  const userAgent = request.headers.get('user-agent') || '';
  const isHtml = !accept || accept === '*/*' || accept.includes('text/html');
  if (request.method !== 'GET' || !isHtml || !CRAWLER.test(userAgent)) {
    return NextResponse.next();
  }

  try {
    const headers = {
      // Set the LOVABLEHTML_API_KEY environment variable in your deployment.
      'x-lovablehtml-api-key': process.env.LOVABLEHTML_API_KEY,
      accept: 'text/html',
    };
    for (const name of ENCITED_FORWARD_HEADERS) {
      const value = request.headers.get(name);
      if (value) headers[name] = value;
    }

    const rendered = await fetch(
      'https://encited.com/api/prerender/render?url=' + encodeURIComponent(request.url),
      { headers, redirect: 'manual' },
    );

    const location = rendered.headers.get('location');
    if (rendered.status === 301 && location) {
      return NextResponse.redirect(location, 301);
    }
    if (rendered.status === 304) return NextResponse.next();
    if (rendered.status === 200 && (rendered.headers.get('content-type') || '').includes('text/html')) {
      const responseHeaders = { 'content-type': 'text/html; charset=utf-8' };
      for (const name of ENCITED_RESPONSE_HEADERS) {
        const value = rendered.headers.get(name);
        if (value) responseHeaders[name] = value;
      }
      return new NextResponse(rendered.body, {
        status: 200,
        headers: responseHeaders,
      });
    }
  } catch {
    // Fail open.
  }

  return NextResponse.next();
}

export const config = {
  matcher: ['/((?!api|_next/static|_next/image|favicon.ico).*)'],
};

2. Deploy Next.js#

Commit the integration file and deploy through your normal Next.js hosting workflow.

npm run build

Test the integration#

1. Send a crawler request

curl -sS -D - -o /dev/null \
  -A "Googlebot" \
  -H "Accept: text/html" \
  "https://your-domain.com/"

Look for a successful HTML response and the x-lovablehtml-render-cache header.

2. Confirm browser passthrough

curl -sS -D - -o /dev/null \
  -A "Mozilla/5.0" \
  -H "Accept: text/html" \
  -H "Accept-Language: en-US" \
  -H "Sec-Fetch-Mode: navigate" \
  -H "Sec-Fetch-Dest: document" \
  -H "Upgrade-Insecure-Requests: 1" \
  "https://your-domain.com/"

This request should be served by your existing website without an Encited render-cache header.

Common errors#

Next.js does not execute the integration file

Use middleware.js on Next.js 15 and earlier or proxy.js on Next.js 16+, keep the default export, and deploy a server-capable output.

The matcher skips pages that need pre-rendering

Review the exported matcher and ensure it excludes assets while still matching every public client-rendered page.

Request and security behavior#

  • Only eligible GET requests for public HTML pages are considered for pre-rendering.
  • Along with the public page URL, only the API key and crawler-classification headers are sent to Encited.
  • Cookie and Authorization headers are never forwarded to Encited.
  • If Encited is unavailable or does not return rendered HTML, the request falls through to your website.

If your site sits behind a firewall, bot protection, or rate limiting, those rules block us before this middleware helps. Set a render token and add one firewall rule so renders and audit crawls get through.

Common questions#

When does Next.js need a pre-rendering integration?#

Use it for routes that are primarily client-rendered. Routes already using SSR or static generation normally deliver complete HTML without it.

Should Next.js pre-rendering use middleware.js or proxy.js?#

Use middleware.js through Next.js 15 and proxy.js for Next.js 16+. The provided default export works with either filename.

Will Next.js API routes and static assets reach Encited?#

No. The request checks and matcher keep non-GET requests, non-HTML traffic, and static assets on the normal Next.js path.