How to Integrate Pre-rendering with Fastly VCL

How Fastly pre-rendering works

Fastly can classify crawler requests in custom VCL and send only eligible public HTML requests to Encited before the request reaches your origin.

Crawler GET requests for HTML are routed to Encited. Browser traffic, assets, non-GET requests, and failed render calls restart against your existing Fastly origin.

Prerequisites

A Fastly service using custom VCL with permission to clone and activate service versions.
The generated Fastly backend name that currently serves your website.
An Encited API key that can be stored in the private VCL configuration.

Set up pre-rendering with Fastly

Clone the active Fastly service version

Create an editable version so the current production configuration remains available for rollback.

Create the custom VCL

Start from Fastly's custom VCL boilerplate, paste the code below, and replace F_your_origin_backend with the generated backend name that currently serves your website.

Add the Encited VCL and API key

The API key is inserted into the VCL because Fastly VCL cannot read process environment variables. The configuration strips cookies and authorization data, routes matching crawler GET requests to Encited, and restarts against your origin if Encited does not return rendered HTML.

Validate and activate the service version

Validate the VCL, activate the new version, and allow a minute for it to propagate across Fastly.

fastly service version validate --service-id YOUR_SERVICE_ID --version latest

encited-prerender.vcl

CopyDownload

# Encited middleware for Fastly
# Replace F_your_origin_backend with the backend name from Fastly's generated VCL.

backend Encited_Host {
  .between_bytes_timeout = 10s;
  .connect_timeout = 1s;
  .dynamic = true;
  .first_byte_timeout = 25s;
  .host = "encited.com";
  .max_connections = 200;
  .port = "443";
  .ssl = true;
  .ssl_cert_hostname = "encited.com";
  .ssl_check_cert = always;
  .ssl_sni_hostname = "encited.com";
}

sub vcl_recv {
  # A failed or non-rendered Encited response restarts against your origin.
  if (req.restarts > 0 && req.http.X-Encited-Fallback == "1") {
    set req.backend = F_your_origin_backend;
    set req.url = req.http.X-Encited-Original-URL;
    set req.http.host = req.http.X-Encited-Original-Host;
    unset req.http.X-Encited-Request;
    unset req.http.X-Encited-Fallback;
    unset req.http.X-Encited-Original-URL;
    unset req.http.X-Encited-Original-Host;
    return(lookup);
  }

  if (
    req.method == "GET" &&
    (req.http.Accept == "" || req.http.Accept ~ "(?i)text/html|\*/\*") &&
    req.http.User-Agent ~ "(?i)(?:(OAI-SearchBot|ChatGPT-User(?:\/\d+\.\d+)?|GPTBot|ChatGPT|OpenAI-))|(?:(anthropic-ai|ClaudeBot|claude-web|Claude-Web|Claude-User|Claude-SearchBot))|(?:(GrokBot|xAI-Grok))|(?:(PerplexityBot|Perplexity-User))|(?:(Google-InspectionTool|GoogleOther(?:-Image|-Video)?|Googlebot|Google-CloudVertexBot|Storebot-Google))|(?:(Microsoft-Copilot|CopilotBot|Copilot-Chat|Microsoft-CopilotPlugin))|(?:(BingBot|bingbot|BingPreview))|(?:facebookexternalhit.*Twitterbot|Twitterbot.*facebookexternalhit)|(?:(Meta-ExternalAgent|meta-externalagent|meta-externalfetcher|MetaBot|FacebookBot|facebookexternalhit))|(?:LinkedInBot)|(?:Amazonbot)|(?:(Applebot-Extended|Applebot))|(?:Bytespider)|(?:(DuckAssistBot|DuckDuckBot))|(?:(cohere-training-data-crawler|cohere-ai|CohereAI))|(?:MistralAI-User)|(?:AI2Bot)|(?:CCBot)|(?:Diffbot)|(?:omgili)|(?:TimpiBot)|(?:YouBot)|(?:(Bravebot|Brave-Search))|(?:Yandex)|(?:Twitterbot)|(?:Discordbot)|(?:(Slackbot|Slack-ImgProxy))|(?:TelegramBot)|(?:(Pinterest|Pinterestbot))|(?:Snapchat)|(?:Redditbot)|(?:Tumblr)|(?:(Mastodon|http\.rb))|(?:Viber)|(?:^Line\/)|(?:WhatsApp)|(?:(SkypeUriPreview|Skype))|(?:(Teams|MicrosoftPreview))|(?:Zoom)|(?:Notion)|(?:(Quora|QuoraBot|Quora-Bot|Poe-Bot))|(?:Medium)|(?:(Pocket|PocketParser|PocketImageCache))|(?:(Flipboard|FlipboardProxy))|(?:(Embedly|embed\.ly))|(?:(Baiduspider|Baidu))|(?:(Sogou|sogou))|(?:(Yeti|NaverBot|Naver))|(?:SeznamBot)|(?:(Qwantify|Qwant))|(?:Ecosia)|(?:MojeekBot)|(?:Mail\.RU_Bot)|(?:Yahoo! Slurp)|(?:(SemrushBot|SEMrush))|(?:(AhrefsBot|AhrefsSiteAudit))|(?:(DotBot|Rogerbot|moz\.com))|(?:MJ12bot)|(?:Screaming Frog)|(?:SISTRIX)|(?:SEOkicks)|(?:serpstatbot)|(?:RSiteAuditor)|(?:LvHTML-SEOAuditBot)|(?:Barkrowler)|(?:HubSpot Crawler)|(?:(AwarioSmartBot|AwarioBot))" &&
    req.url.ext !~ "(?i)js|css|xml|txt|png|jpg|jpeg|gif|pdf|ico|woff|woff2|ttf|svg|webmanifest"
  ) {
    set req.http.X-Encited-Request = "1";
    set req.http.X-Encited-Original-URL = req.url;
    set req.http.X-Encited-Original-Host = req.http.host;
    set req.url = "/api/prerender/render?url=" + urlencode("https://" + req.http.host + req.url);
    set req.backend = Encited_Host;
    set req.http.host = "encited.com";
    set req.http.x-lovablehtml-api-key = <your-api-key>;
    set req.http.Accept = "text/html";
    unset req.http.Cookie;
    unset req.http.Authorization;
    return(pass);
  }

  # Preserve Fastly-generated configuration and snippets.
  #FASTLY recv
}

sub vcl_hash {
  #FASTLY hash
}

sub vcl_hit {
  #FASTLY hit
}

sub vcl_miss {
  #FASTLY miss
}

sub vcl_pass {
  #FASTLY pass
}

sub vcl_fetch {
  if (req.http.X-Encited-Request == "1") {
    if (
      (beresp.status != 200 && beresp.status != 301) ||
      (beresp.status == 200 && beresp.http.Content-Type !~ "(?i)text/html")
    ) {
      # Pass the fallback signal to vcl_deliver. Restarting there guarantees
      # request state survives Fastly clustering.
      set beresp.http.X-Encited-Fallback = "1";
    }
    set beresp.cacheable = false;
    return(deliver);
  }

  #FASTLY fetch
}

sub vcl_error {
  #FASTLY error
}

sub vcl_deliver {
  if (
    req.http.X-Encited-Request == "1" &&
    resp.http.X-Encited-Fallback == "1" &&
    req.restarts == 0
  ) {
    set req.http.X-Encited-Fallback = "1";
    unset resp.http.X-Encited-Fallback;
    restart;
  }

  unset resp.http.X-Encited-Fallback;
  unset resp.http.X-Encited-Request;
  unset resp.http.X-Encited-Original-URL;
  unset resp.http.X-Encited-Original-Host;

  #FASTLY deliver
}

sub vcl_log {
  #FASTLY log
}

Test the integration

1. Send a crawler request

curl -sS -D - -o /dev/null \
                                  -A "Googlebot" \
                                  -H "Accept: text/html" \
                                  "https://your-domain.com/"

Look for a successful HTML response and the x-lovablehtml-render-cache header.

2. Confirm browser passthrough

curl -sS -D - -o /dev/null \
                                  -A "Mozilla/5.0" \
                                  -H "Accept: text/html" \
                                  -H "Accept-Language: en-US" \
                                  -H "Sec-Fetch-Mode: navigate" \
                                  -H "Sec-Fetch-Dest: document" \
                                  -H "Upgrade-Insecure-Requests: 1" \
                                  "https://your-domain.com/"

This request should be served by your existing website without an Encited render-cache header.

Common errors

Fastly reports an unknown origin backend

Replace F_your_origin_backend with the generated backend name from your active VCL before validating the service version.

The new VCL validates but crawler requests still reach the origin

Confirm the edited service version is active and that the API key placeholder was replaced before activation.

Request and security behavior

Only eligible GET requests for public HTML pages are considered for pre-rendering.
Along with the public page URL, only the API key and crawler-classification headers are sent to Encited.
Cookie and Authorization headers are never forwarded to Encited.
If Encited is unavailable or does not return rendered HTML, the request falls through to your website.

Common questions

How does pre-rendering work with Fastly VCL?

Custom VCL classifies crawler HTML requests at the edge, calls Encited for rendered HTML, and keeps all other requests on the existing backend.

Does the Fastly pre-rendering integration fail open?

Yes. An unavailable Encited endpoint, a non-render response, or an ineligible request restarts against your original Fastly backend.

How do I test crawler pre-rendering on Fastly?

Send a GET request with a Googlebot user agent and Accept: text/html, then check for the x-lovablehtml-render-cache response header.

No-code DNS setup All setup guides