What is Content Signals?

What is Content Signals?

How it works

Content signals extend robots.txt with separate declarations for AI training, AI input at answer time, and traditional search indexing, so a publisher can permit one and refuse another. They are declarations of preference rather than technical enforcement: a compliant crawler honours them, and a non-compliant one is unaffected. Actual blocking requires server-side or edge rules.

When to use it

Set them when your position on AI training differs from your position on being cited in AI answers, which is the common case for companies that want visibility without contributing training data.

Worked example

Signals split into three groups: structural (headings, schema, tables), factual (numbers with sources and dates), and behavioural (inbound links, freshness). Structural and factual signals are the ones you control directly. A page with a real HTML table, a dated source, and three inbound internal links clears the mechanical bar; the rest is site-level judgement.

Common mistakes

• Blocking AI crawlers wholesale, then wondering why the brand never appears in AI answers.
• Treating a robots directive as enforcement rather than a request.
• Copying another site's file without checking which user agents it names.

FAQ