How Koa pre-rendering works
Koa.js can place Encited at the front of its middleware stack so crawlers receive rendered HTML before routers and SPA fallbacks run.
The Koa middleware handles eligible crawler HTML requests and calls next() for browser traffic, assets, and any render response that should fall back.
Prerequisites
- Node.js 18 or later for the native fetch API.
- Access to register middleware before static files, authentication, and routers.
- LOVABLEHTML_API_KEY configured in the production Node.js process.
Set up pre-rendering with Koa
Create the Koa middleware
Save the snippet as encited-prerender.js. It uses the native fetch API available in Node.js 18 and later.
Register it first
Call app.use(encitedPrerender) before static files, authentication, and router middleware.
Deploy or restart Koa
Set LOVABLEHTML_API_KEY and restart the Node.js process.
npm start// encited-prerender.js - app.use before routesconst CRAWLER = /(?:(OAI-SearchBot|ChatGPT-User(?:\/\d+\.\d+)?|GPTBot|ChatGPT|OpenAI-))|(?:(anthropic-ai|ClaudeBot|claude-web|Claude-Web|Claude-User|Claude-SearchBot))|(?:(GrokBot|xAI-Grok))|(?:(PerplexityBot|Perplexity-User))|(?:(Google-InspectionTool|GoogleOther(?:-Image|-Video)?|Googlebot|Google-CloudVertexBot|Storebot-Google))|(?:(Microsoft-Copilot|CopilotBot|Copilot-Chat|Microsoft-CopilotPlugin))|(?:(BingBot|bingbot|BingPreview))|(?:facebookexternalhit.*Twitterbot|Twitterbot.*facebookexternalhit)|(?:(Meta-ExternalAgent|meta-externalagent|meta-externalfetcher|MetaBot|FacebookBot|facebookexternalhit))|(?:LinkedInBot)|(?:Amazonbot)|(?:(Applebot-Extended|Applebot))|(?:Bytespider)|(?:(DuckAssistBot|DuckDuckBot))|(?:(cohere-training-data-crawler|cohere-ai|CohereAI))|(?:MistralAI-User)|(?:AI2Bot)|(?:CCBot)|(?:Diffbot)|(?:omgili)|(?:TimpiBot)|(?:YouBot)|(?:(Bravebot|Brave-Search))|(?:Yandex)|(?:Twitterbot)|(?:Discordbot)|(?:(Slackbot|Slack-ImgProxy))|(?:TelegramBot)|(?:(Pinterest|Pinterestbot))|(?:Snapchat)|(?:Redditbot)|(?:Tumblr)|(?:(Mastodon|http\.rb))|(?:Viber)|(?:^Line\/)|(?:WhatsApp)|(?:(SkypeUriPreview|Skype))|(?:(Teams|MicrosoftPreview))|(?:Zoom)|(?:Notion)|(?:(Quora|QuoraBot|Quora-Bot|Poe-Bot))|(?:Medium)|(?:(Pocket|PocketParser|PocketImageCache))|(?:(Flipboard|FlipboardProxy))|(?:(Embedly|embed\.ly))|(?:(Baiduspider|Baidu))|(?:(Sogou|sogou))|(?:(Yeti|NaverBot|Naver))|(?:SeznamBot)|(?:(Qwantify|Qwant))|(?:Ecosia)|(?:MojeekBot)|(?:Mail\.RU_Bot)|(?:Yahoo! Slurp)|(?:(SemrushBot|SEMrush))|(?:(AhrefsBot|AhrefsSiteAudit))|(?:(DotBot|Rogerbot|moz\.com))|(?:MJ12bot)|(?:Screaming Frog)|(?:SISTRIX)|(?:SEOkicks)|(?:serpstatbot)|(?:RSiteAuditor)|(?:LvHTML-SEOAuditBot)|(?:Barkrowler)|(?:HubSpot Crawler)|(?:(AwarioSmartBot|AwarioBot))/i;const ENCITED_FORWARD_HEADERS = ['accept-language', 'sec-fetch-mode', 'sec-fetch-site', 'sec-fetch-dest','sec-fetch-user', 'upgrade-insecure-requests', 'referer', 'user-agent',];const ENCITED_RESPONSE_HEADERS = ['x-lovablehtml-render-cache', 'x-lovablehtml-snapshot-key', 'cache-control','etag', 'content-digest', 'signature-input', 'signature',];export async function encitedPrerender(ctx, next) {const accept = ctx.get('accept') || '';const userAgent = ctx.get('user-agent') || '';const isHtml = !accept || accept === '*/*' || accept.includes('text/html');if (ctx.method !== 'GET' || !isHtml || !CRAWLER.test(userAgent)) return next();try {const headers = {'x-lovablehtml-api-key': process.env.LOVABLEHTML_API_KEY,accept: 'text/html',};for (const name of ENCITED_FORWARD_HEADERS) {const value = ctx.get(name);if (value) headers[name] = value;}const rendered = await fetch('https://encited.com/api/prerender/render?url=' + encodeURIComponent(ctx.href),{ headers, redirect: 'manual' },);const location = rendered.headers.get('location');if (rendered.status === 301 && location) {ctx.status = 301;ctx.redirect(location);return;}if (rendered.status === 304) return next();if (rendered.status === 200 && (rendered.headers.get('content-type') || '').includes('text/html')) {ctx.status = 200;ctx.type = 'html';for (const name of ENCITED_RESPONSE_HEADERS) {const value = rendered.headers.get(name);if (value) ctx.set(name, value);}ctx.body = Buffer.from(await rendered.arrayBuffer());return;}} catch {// Fail open.}return next();}
Test the integration
1. Send a crawler request
curl -sS -D - -o /dev/null \
-A "Googlebot" \
-H "Accept: text/html" \
"https://your-domain.com/"Look for a successful HTML response and the x-lovablehtml-render-cache header.
2. Confirm browser passthrough
curl -sS -D - -o /dev/null \
-A "Mozilla/5.0" \
-H "Accept: text/html" \
-H "Accept-Language: en-US" \
-H "Sec-Fetch-Mode: navigate" \
-H "Sec-Fetch-Dest: document" \
-H "Upgrade-Insecure-Requests: 1" \
"https://your-domain.com/"This request should be served by your existing website without an Encited render-cache header.
Common errors
The Koa router responds before Encited
Register encitedPrerender with app.use before static middleware, authentication, and every router.
The Koa process cannot authenticate the render request
Set LOVABLEHTML_API_KEY in the production process environment and restart every Koa instance.
Request and security behavior
- Only eligible GET requests for public HTML pages are considered for pre-rendering.
- Along with the public page URL, only the API key and crawler-classification headers are sent to Encited.
- Cookie and Authorization headers are never forwarded to Encited.
- If Encited is unavailable or does not return rendered HTML, the request falls through to your website.
Common questions
How does pre-rendering work with Koa.js middleware?
The first Koa middleware detects eligible crawlers, returns Encited rendered HTML, and awaits next() for normal application traffic.
Where should Koa.js pre-rendering middleware be registered?
Register it before static files, sessions, authentication, routers, and the SPA fallback.
How do I test crawler pre-rendering in Koa.js?
Send a Googlebot request with Accept: text/html and confirm the response contains x-lovablehtml-render-cache.
