Do I need llms.txt or special markup to appear in Google AI Overviews?

#What Google says

Google's page on AI features and your website states the following:

• A page must be indexed and eligible to be shown in Google Search with a snippet to be a supporting link.
• There are no additional technical requirements, and no other special optimizations are necessary.
• No special schema.org structured data is needed.
• You do not need to create new machine-readable files, AI-specific text files or new markup.
• Meeting these requirements does not guarantee crawling, indexing or serving.

#What to do instead

Google lists ordinary search fundamentals. In order of effort:

1. Allow crawling in robots.txt and in any CDN or hosting layer in front of the site. A CDN that answers a crawler with an error blocks it even when robots.txt allows it.
2. Verify the site in Search Console and check that the page is indexed.
3. Keep the important content as text in the HTML, supported by images and video where they help.
4. Link internally so pages are easy to find.
5. Make any structured data match the visible text on the page.
6. Provide a good page experience.

#What llms.txt is

llms.txt is a proposal by Jeremy Howard, first published on 3 September 2024. A site places a markdown file at /llms.txt that gives AI agents a short guide to the site with links to more detail. The file has an H1 with the site name, which is the only required element, then an optional blockquote summary, optional detail paragraphs, H2 sections of link lists, and an optional "Optional" section for secondary links. The proposal also suggests serving a clean markdown copy of a page at the same address with .md appended.

It is cheap to produce, and this site generates its own at build time from the same data as the pages. Google says it is not needed, and I know of no published commitment from an assistant vendor that it changes citations, so it is a convenience and not a strategy.

#Other assistants: crawler controls

OpenAI documents three agents as independent robots.txt settings:

OpenAI says its search systems can take about 24 hours to adjust after a robots.txt update. Because the settings are independent, you can allow OAI-SearchBot to appear in search while disallowing GPTBot to opt out of training.

#What the research says about GEO

The paper that named generative engine optimization (Aggarwal et al., accepted to KDD 2024) reports that some content methods raised visibility in generative engine responses by up to 40 percent. That is a ceiling for the best method on the authors' own benchmark, not a forecast for any single site, so use it as a reason to try cheap, honest improvements: clear answer-first writing, dated sources, and verifiable numbers.

I describe the changes made on this site in SEO, AEO and GEO for a Developer Portfolio.