The rule, for your team's review

Rule version 0.1, draft of 21 October 2026, prepared by AnswerShare. Nothing here is deployed. Your team reviews, edits, merges and deploys it.

What we propose, and why

We propose a Next.js rewrite rule in next.config.js, in the beforeFiles phase, that sends page requests from named AI crawlers to AnswerShare as an external rewrite. We are not proposing middleware, and the rule does not go in vercel.json.

• Why not vercel.json. Our 19 October email said the rule would go in vercel.json. That was a mistake; the diagram we sent with it shows the correct place, next.config.js. Vercel applies vercel.json rewrites only after it checks for a matching file ("precedence is given to the filesystem prior to rewrites being applied", Vercel's vercel.json documentation). Your pages are prerendered: on 21 October your home page and /es-US both answered with x-nextjs-prerender: 1. A vercel.json rule would therefore be skipped for most of your pages. Next.js beforeFiles rewrites are checked before files and pages, so the rule works on prerendered pages.
• Why not middleware. A rewrite rule is configuration. Next.js reads rewrites() once, at build time, and no code of ours runs on any request. Middleware would run on every page request, including your visitors' requests. It would also need a credential in your Vercel environment variables, and it would have to be kept current with Next.js releases. Your 21 October reply said the move to the rewrite rule addressed your biggest concerns, so we kept it.
• The trade-off, stated plainly. A rewrite rule has no built-in fallback. If AnswerShare cannot answer, Vercel returns an error to that AI crawler for that request. People and search engines are never affected. The section "If AnswerShare errors or times out" gives each case and what we do about it.

How it decides

1. User agent only. The rule reads the request's User-Agent header. It never looks at IP addresses or networks.
2. Named AI crawlers in. A request is routed only if its user agent contains one of the AI crawler names in the list below. The list comes from AnswerShare's canonical crawler classification, the AI_BOTS set in src/bot-classification.ts of our shared library (aryah-shared), at commit b43382d of 2 October 2026. Changes reach you only as pull requests.
3. Search crawlers out, always. The rule refuses any user agent that contains a search crawler's name, even one that also contains an AI name. That list is our SEARCH_BOTS set from the same file: Googlebot in all its variants, Google-InspectionTool (Search Console's inspection tool), Bingbot, Applebot, DuckDuckBot, YandexBot, Baiduspider and PetalBot. Everyone else, including every person, gets your site untouched.
4. Pages only. The rule skips Next.js internals (/_next/), /api/, /_vercel/, /.well-known/ and any path ending in a file extension: images, scripts, styles, icons, robots.txt, sitemap.xml and manifest.webmanifest. None of the 497 page addresses in your sitemap ends in a file extension, so every page is covered and every asset is excluded.
5. Not in-app requests or form submissions. The rule also skips any request carrying the RSC header (Next.js in-app navigation data) or the Next-Action header (Next.js form submissions), and /api/ is excluded by path. We found no checkout, cart or account pages on www; the rental portal sign-up page is an ordinary page with a form.

   Question for HOLT: Does www have any other account, checkout or transaction paths we should exclude from the rule?

6. English and Spanish alike. Next.js runs your existing language middleware first and this rule after it, so the rule covers English pages at / and Spanish pages under /es-US. We test both on the preview.

Matching is case-sensitive, because Next.js header patterns are. Each name matches as written in the list and in lower case, which covers the published user agent of every crawler on the list. A crawler that spells its name with some other capitalisation gets your normal page, which is the safe direction.

What reaches AnswerShare

Only requests that match the rule. Vercel forwards the crawler's request as it arrived: the page path and query, the crawler's user agent and its other request headers, plus Vercel's standard forwarding headers (x-forwarded-for, x-real-ip, x-forwarded-host, x-forwarded-proto), which carry the crawler's IP address. Requests from people and search crawlers never match, so nothing from them reaches us. This is the same description we gave in our answer to your question 13.

Every response we send back is marked Cache-Control: private, no-store. Vercel honours upstream cache headers on external rewrites, so it does not store the AI version, and an AI copy never sits in a cache where a visitor could get it.

If AnswerShare errors or times out

The rule points at an AnswerShare edge service that builds each response. Here is what an AI crawler gets in each case. In every case, people and search crawlers get your normal page, because their requests never reach us.

The last two cases are the cost of having no code on your visitors' requests. A crawler that gets an error retries later. The stop conditions in the plan call for a revert if failures pass 1% of AI-crawler requests in an hour or last 15 minutes.

The fall-back to your normal page in the second and third rows is built and running on our side. On 22 October 2026 we sent GPTBot requests for your home page and for /es-US to our edge service. Each got your own page back with status 200, marked Cache-Control: private, no-store and x-answershare-variant: passthrough, in under 2 seconds. We test it again on your preview (tests T14 and T15).

Blast radius of each failure

For each failure below: what your human layer gets (your visitors, plus Googlebot and the other search crawlers), what our layer gets (the AI crawlers on the list, and the records we keep), how the failure is detected, and how it is contained. Three things hold in every case.

• People and search crawlers stay outside it. The rule matches on user agent only, and it lists every search crawler under missing, so their requests never reach us. A failure on our side cannot change what they get or how fast they get it. The exceptions are the link previews and the two optional files, noted where they apply.
• Nothing is left behind in a cache. Every response from our edge service, its errors included, is marked Cache-Control: private, no-store, so Vercel keeps no copy. Once a failure ends, the next request gets the right page.
• Today we record only the AI version. We record a crawler visit when we serve the AI version of a page. A fall-back or an error leaves no record on our side. Your Vercel logs still show the request. Before the preview we will record every fall-back too, with its reason, so the crawler visits on measuring results include them.

The page has no AI version yet

Our content service fails or is slow

Your site is slow or down when our edge service fetches your page

Our edge service is down or unreachable

Our edge service accepts the connection but never answers

The alternative, if you prefer it. If HOLT would rather AI crawlers always fall back to your page, even when our edge service is down, the way to do that is a small Next.js middleware with a timeout: if we do not answer within 2.5 seconds, the request falls through to your normal page. That is the middleware design in our 8 October overview, with the credential kept in your environment variables as you asked. Its costs are the ones listed above: code on every page request, a credential in your environment variables, and upkeep across Next.js releases. We recommend the rule. We will provide the middleware for review if you choose it.

The snippet

For next.config.js. If your config is next.config.mjs or next.config.ts, the same object goes in your existing export. If you already have rewrites, keep them: a rewrites() that returns an array maps to afterFiles below.

// AnswerShare AI-crawler rule for www.texasfirstrentals.com (rule v0.1, test run)
// Crawler names: AnswerShare aryah-shared src/bot-classification.ts,
// AI_BOTS and SEARCH_BOTS at commit b43382d (2 October 2026).
// Matching is by user agent only. No IP or network lookups.
// To remove: delete this block and the answershareRewrites entry in rewrites().

const ANSWERSHARE_ORIGIN = 'https://customers.texasfirstrentals.production.answershare.com';

// AI crawlers and AI agents that get the AI version of a page.
const AI_CRAWLERS = [
  // OpenAI
  'GPTBot', 'OAI-SearchBot', 'ChatGPT-User', 'ChatGPT',
  // Anthropic
  'ClaudeBot', 'Claude-SearchBot', 'Claude-User', 'claude-web', 'anthropic-ai',
  // Perplexity
  'PerplexityBot', 'Perplexity-User',
  // Google AI agents: not search crawlers. Remove any you prefer.
  'Gemini-Deep-Research', 'Gemini-Research', 'Gemini-User', 'GeminiBot', 'Gemini-AI',
  'GoogleAgent-Mariner', 'GoogleAgent-URLContext', 'Google-CloudVertexBot',
  'GoogleOther', 'Google-Extended',
  // Apple: AI only. Plain Applebot is a search crawler and is excluded below.
  'Applebot-Extended',
  // Meta
  'Meta-ExternalAgent', 'FacebookExternalHit',
  // Others
  'Bytespider', 'Amazonbot', 'YouBot', 'Diffbot', 'CCBot', 'Cohere-AI',
  'DeepSeekBot', 'MistralAI-User', 'Qwen-Bot', 'GrokBot', 'xAI-Grok', 'Grok-DeepSearch',
  // AnswerShare's own measurement crawler
  'AnswerShareBot',
];

// Search crawlers always get your normal site, even if a user agent
// also contains a name from the list above.
const SEARCH_CRAWLERS = [
  'Googlebot', 'Googlebot-Image', 'Google-InspectionTool', 'Bingbot', 'Applebot',
  'DuckDuckBot', 'YandexBot', 'Baiduspider', 'PetalBot',
];

// Next.js anchors header patterns (^...$) and matches them case-sensitively.
// Each name matches as written above and in lower case.
// Names contain only letters, digits and hyphens, so no escaping is needed.
const uaPattern = (names) =>
  `.*(?:${[...new Set(names.flatMap((n) => [n, n.toLowerCase()]))].join('|')}).*`;

const answershareRewrites = [
  // Rule 1: page requests from a named AI crawler get the AI version of that page.
  {
    // Every page path except /_next/, /_vercel/, /api/, /.well-known/
    // and any path ending in a file extension (assets, robots.txt, sitemap.xml).
    source: '/:path((?!(?:_next|_vercel|api|\\.well-known)(?:/|$)|.*\\.[A-Za-z0-9]+$).*)',
    has: [{ type: 'header', key: 'user-agent', value: uaPattern(AI_CRAWLERS) }],
    missing: [
      { type: 'header', key: 'user-agent', value: uaPattern(SEARCH_CRAWLERS) },
      { type: 'header', key: 'rsc' },                       // in-app navigation data
      { type: 'header', key: 'next-action' },               // form submissions
      { type: 'header', key: 'x-answershare-passthrough' }, // our fall-back fetch; stops loops
    ],
    destination: `${ANSWERSHARE_ORIGIN}/:path`,
  },
  // Rule 2: two public files, identical for every visitor and crawler.
  // Both return 404 on your site today.
  { source: '/llms.txt', destination: `${ANSWERSHARE_ORIGIN}/llms.txt` },
  { source: '/llms-full.txt', destination: `${ANSWERSHARE_ORIGIN}/llms-full.txt` },
];

module.exports = {
  // ...your existing settings, unchanged...
  async rewrites() {
    return {
      beforeFiles: answershareRewrites,
      afterFiles: [], // your existing rewrites, if any
      fallback: [],
    };
  },
};

Requires Next.js 13.3 or later (for missing).

Question for HOLT: Which Next.js version does the site run?

Rule 2 is optional for the test run. It adds two files that do not exist today and serves them the same way to everyone, so it changes nothing that people or search engines currently get. If you would rather leave it out, delete those two lines. Vercel does not allow rewrites of /.well-known/, so any file we supply for that folder would be a static file you commit. We are not asking for one in the test run.

What your team does

Nothing goes live from these steps; they only prepare your preview.

1. Put the rule in next.config.js.
   Click flow: your code repository for www.texasfirstrentals.com (GitHub, GitLab or Bitbucket, whichever you use) > the project root folder > next.config.js (or next.config.mjs / next.config.ts) > the rewrites() function > the "beforeFiles" list.
   Required state: the AnswerShare block from the attached rule file sits inside "beforeFiles". Keep your existing rewrites where they are (they stay in "afterFiles").
   You should see: a rewrites() function that returns beforeFiles, afterFiles and fallback, with the AnswerShare block first.
2. Make sure vercel.json has no AnswerShare rule.
   Click flow: the same repository > the project root folder > vercel.json > the "rewrites" array.
   Required state: no entry whose destination contains answershare.com.
   You should see: either no vercel.json rewrites at all, or only your own.
3. Put the change in its own deployment, so Instant Rollback removes only this.
   Click flow: your repository > create a branch > commit the next.config.js change > open a pull request.
   Required state: the pull request contains only the next.config.js change.
   You should see: Vercel adds a Preview deployment to the pull request.
4. Open the Preview deployment so we can test it.
   Click flow: https://vercel.com/dashboard > your Texas First Rentals project > Deployments > the newest "Preview" row > Visit.
   Required state: the deployment status is Ready.
   You should see: your normal site in the browser. Nothing changes for a person visiting.
5. Tell us the Preview address and how we get through Vercel Deployment Protection.
   Click flow: Vercel dashboard > your project > Settings > Deployment Protection.
   Required state: either "Protection Bypass for Automation" is enabled and you read the value to us on a call (never in email), or you allow our test server address for the preview.
   You should see: the setting shown as Enabled, with a bypass value you can reveal.

Revert in one step

One step: in Vercel, use Instant Rollback to return production to the deployment before the rule. It takes effect without a rebuild. Ship the rule in a deployment of its own so that a rollback removes nothing else.

Question for HOLT: Is Instant Rollback available on your Vercel plan, and will your team use it for this rollback?

Permanent: delete the AnswerShare block and the answershareRewrites entry, and deploy. Nothing else in your project depends on it, and there is nothing to clean up afterwards: no DNS record, no environment variable, no secret.

We rehearse both on the preview before go-live (test T18).

The crawler list, entry by entry

"User-triggered" means the crawler fetches a page because a person just asked an assistant something. "Crawler" means a background index or training crawl. We use this split for the user-intent share on measuring results. Entries marked "Review" are ones we suggest your team looks at.