Markdown content negotiation for AI agents

When an AI agent fetches your homepage to summarize it, the agent renders your HTML, scrapes the text, and trims the irrelevant parts. That pipeline is lossy and slow. Every nav, every footer, every cookie banner is overhead the agent has to discard.

There is a cleaner pattern: when an agent sends Accept: text/markdown, return a clean markdown representation of the same page. Browsers still get HTML. Agents get something they can parse in a single shot.

This is content negotiation in the boring HTTP sense. The mechanism is decades old. The novelty is using it as an interoperability layer for agents.

What changes

Before:

GET / HTTP/2
Host: example.com
Accept: text/html

→ 200 OK
Content-Type: text/html; charset=utf-8

<!doctype html>
<html>... 80 KB of nav, hero, footer, scripts ...</html>

After (when an agent asks for markdown):

GET / HTTP/2
Host: example.com
Accept: text/markdown

→ 200 OK
Content-Type: text/markdown; charset=utf-8

# Example

> One-line tagline.

## What this site does

Explains the value prop in three sentences. No menu, no footer, no decoration.

The markdown payload is typically 5 to 20 percent the size of the HTML, parses without a DOM, and contains zero layout noise.

Implementation in Next.js 16

The request lifecycle in Next.js 16 lives in proxy.ts at the project root. (The older middleware convention has been renamed to proxy. Same idea, different filename.)

// proxy.ts
import { NextResponse } from "next/server";
import type { NextRequest } from "next/server";

function acceptsMarkdown(request: NextRequest): boolean {
  const accept = request.headers.get("accept") ?? "";
  return accept.toLowerCase().includes("text/markdown");
}

export function proxy(request: NextRequest) {
  const { pathname } = request.nextUrl;
  if (pathname !== "/") return NextResponse.next();

  if (acceptsMarkdown(request)) {
    const url = request.nextUrl.clone();
    url.pathname = "/markdown-home";
    return NextResponse.rewrite(url);
  }

  return NextResponse.next();
}

export const config = { matcher: ["/"] };

The markdown route handler returns the canonical markdown body:

// app/markdown-home/route.ts
import { NextResponse } from "next/server";

const BODY = `# Example

> One-line tagline.

## What this site does

...
`;

export async function GET() {
  return new NextResponse(BODY, {
    status: 200,
    headers: {
      "content-type": "text/markdown; charset=utf-8",
      "cache-control": "public, max-age=300",
    },
  });
}

That is it. Two files, no dependencies.

Verifying

curl -H "Accept: text/markdown" https://yourdomain/

You should see your markdown body and a 200 response. A normal browser request to the same URL should still return HTML.

What to put in the markdown body

The markdown response is your "explain me to an agent in one fetch" surface. Keep it tight:

One H1 with the site name.
One H2 paragraph or blockquote with the value prop.
An H2 list of the most important entry points (with absolute URLs).
Skip marketing flourishes. Skip nav. Skip footer.

A good rule: aim for 300 to 800 words and zero links to internal anchors. Agents do not navigate anchors, they re-fetch pages.

When not to do this

If your homepage is already a doc page (a README-style landing site), markdown negotiation adds little. If your homepage is a marketing page with carousels and animations, the markdown variant is a dramatic improvement.

Build the snippet

The Markdown Negotiation Snippet tool generates both files for you, parameterized by your source path and target route. Drop the output straight into your repo.