Blog
Practical writing on AI agent readiness, technical SEO, and the machine-readable surface of the modern web. No fluff, no republished press releases.
ChatGPT visibility checklist for websites
How to make a public website easier for ChatGPT-style tools and AI agents to discover, fetch, parse, and cite without relying on unsupported tricks.
Query fan-out SEO: how to write for AI Mode
Google's AI search can expand one prompt into many related searches. Here is how to structure content for query fan-out without creating thin duplicate pages.
AI Overviews SEO checklist for 2026
Google says AI Overviews and AI Mode still depend on core Search systems. Here is the practical checklist for crawlability, content quality, snippets, and source eligibility.
proxy.ts in Next.js 16: middleware, renamed and explained
Why middleware became proxy, what changed in the request lifecycle, and the patterns that ship cleanly: link headers, content negotiation, auth, and AI bot filters.
Agent-friendly 403, 404, and rate limit responses
When you have to refuse a bot, do it well. The error pages and headers that help AI agents recover instead of retrying blindly into your CDN bill.
Check robots.txt for Googlebot and AI Crawlers Without Mixing Policies
Learn how to check robots.txt separately for Googlebot, GPTBot, ClaudeBot, and PerplexityBot so search access and AI access remain intentional.
Google Search Console for AI readiness
GSC is still the best free signal source for crawl, indexability, and structured-data health. Six reports to read every week as part of your agent-readiness loop.
robots.txt Checker: How to Audit a Live File in 10 Minutes
A practical robots.txt checker workflow for fetch status, rule testing, sitemap discovery, and Google Search Console verification.
robots.txt Tester: Check a URL Path Before Google Crawls It
Use a robots.txt tester to confirm whether Googlebot or an AI crawler can fetch a specific path before you publish new directives.
robots.txt Validator Checklist: 9 Errors to Find Before Deploy
Validate robots.txt safely by checking location, syntax, crawler groups, blocked paths, sitemaps, and live Google verification.
Test robots.txt Before Deploy: A Regression Checklist for SEO Changes
A pre-deployment test plan for robots.txt changes: protect important crawl paths, compare crawler policies, and verify the live rollout.
FAQ rich results are being retired: what to do next
Google stopped showing FAQ rich results on May 7, 2026. Here is the migration plan for what to keep, what to delete, and what to ship instead.
OpenAPI for AI agents: specs that get used
Most OpenAPI files are technically valid but unreadable by agents. Eight rules that turn an OpenAPI document into something an LLM can actually call.
Monitor AI bots in your server logs
Which crawlers visit, how often, what they fetch, and what to do with the data. A practical recipe for parsing access logs into a useful AI traffic dashboard.
Verify the bot is real: User-Agent is not auth
User-Agent strings are trivially spoofable. The 2026 stack for authenticating AI crawlers: reverse DNS, IP allow lists, and Web Bot Auth signed requests.
Every AI bot user agent worth knowing in 2026
The complete reference: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, and more. Owner, purpose, and recommended directive for each.
Content-Signal: declaring AI usage policy
The robots.txt extension that lets you allow search but block training. The exact syntax, valid combinations, and what happens when vendors honor it.
MCP vs A2A: which agent protocol does what
Two protocols, two scopes. Model Context Protocol gives one agent access to tools; A2A lets agents talk to each other. Here is how to pick and ship.
Agentic SEO: how AI search reranks the web
Why classic SEO still matters but is no longer enough. The shift from blue links to AI-mediated discovery, and what to optimize next.
Schema.org JSON-LD that AI agents actually use
A practical reference for the four schema types that produce real value in 2026: Organization, Article, BreadcrumbList, and FAQPage.
sitemap.xml in Next.js: a production checklist
Generate a sitemap that search engines and AI agents trust. Stable lastmod, correct canonicals, sitemap indexes, and verification that won't lie to you.
Block GPTBot, ClaudeBot, PerplexityBot in 2026
A step-by-step robots.txt and Content-Signal recipe for opting out of AI training crawlers without hurting search visibility.
/.well-known/ for agents: the 2026 reference
A field guide to the well-known URIs that matter for AI agents and modern web platforms: api-catalog, agent-card, openid-configuration, security.txt, and more.
Open Graph and Twitter Cards that render
Most social previews break for the same five reasons. The exact tags, dimensions, and validators to ship cards that survive Facebook, X, LinkedIn, and Slack.
llms.txt vs llms-full.txt: when to use which
The two emerging conventions look similar but solve different problems. Here is how to pick, what to put in each, and how to deploy them on Next.js.
RFC 9727 explained: a single endpoint for APIs
How /.well-known/api-catalog and the Linkset format let AI agents and tools discover every API on your site without scraping HTML.
Markdown content negotiation for AI agents
Why returning text/markdown when an agent sends Accept: text/markdown saves tokens, improves accuracy, and is two files of code on Next.js.
The AI agent readiness checklist for 2026
Six signals that determine whether autonomous agents can find, parse, and quote your content correctly. Run this before every release.