Blog
Practical writing on AI agent readiness, technical SEO, and the machine-readable surface of the modern web. No fluff, no republished press releases.
Monitor AI bots in your server logs
Which crawlers visit, how often, what they fetch, and what to do with the data. A practical recipe for parsing access logs into a useful AI traffic dashboard.
Verify the bot is real: User-Agent is not auth
User-Agent strings are trivially spoofable. The 2026 stack for authenticating AI crawlers: reverse DNS, IP allow lists, and Web Bot Auth signed requests.
Every AI bot user agent worth knowing in 2026
The complete reference: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, and more. Owner, purpose, and recommended directive for each.
Content-Signal: declaring AI usage policy
The robots.txt extension that lets you allow search but block training. The exact syntax, valid combinations, and what happens when vendors honor it.
MCP vs A2A: which agent protocol does what
Two protocols, two scopes. Model Context Protocol gives one agent access to tools; A2A lets agents talk to each other. Here is how to pick and ship.
Agentic SEO: how AI search reranks the web
Why classic SEO still matters but is no longer enough. The shift from blue links to AI-mediated discovery, and what to optimize next.
Schema.org JSON-LD that AI agents actually use
A practical reference for the four schema types that produce real value in 2026: Organization, Article, BreadcrumbList, and FAQPage.
sitemap.xml in Next.js: a production checklist
Generate a sitemap that search engines and AI agents trust. Stable lastmod, correct canonicals, sitemap indexes, and verification that won't lie to you.
Block GPTBot, ClaudeBot, PerplexityBot in 2026
A step-by-step robots.txt and Content-Signal recipe for opting out of AI training crawlers without hurting search visibility.
/.well-known/ for agents: the 2026 reference
A field guide to the well-known URIs that matter for AI agents and modern web platforms: api-catalog, agent-card, openid-configuration, security.txt, and more.
Open Graph and Twitter Cards that render
Most social previews break for the same five reasons. The exact tags, dimensions, and validators to ship cards that survive Facebook, X, LinkedIn, and Slack.
llms.txt vs llms-full.txt: when to use which
The two emerging conventions look similar but solve different problems. Here is how to pick, what to put in each, and how to deploy them on Next.js.
RFC 9727 explained: a single endpoint for APIs
How /.well-known/api-catalog and the Linkset format let AI agents and tools discover every API on your site without scraping HTML.
Markdown content negotiation for AI agents
Why returning text/markdown when an agent sends Accept: text/markdown saves tokens, improves accuracy, and is two files of code on Next.js.
The AI agent readiness checklist for 2026
Six signals that determine whether autonomous agents can find, parse, and quote your content correctly. Run this before every release.