Crawlability

Crawlability is whether a particular crawler can reach and fetch a page and the resources it needs.

Source: https://contentcucumber.com/aeo-glossary/crawlability/

Topic: Getting found. Also called: Crawling, Crawler access, AI crawlers


Why it matters

A page can work in your browser while a firewall, authentication step, or robots rule prevents a crawler from reaching it.

Example (illustrative, not a client result)

A public guide returns a normal page to people, but a security rule challenges automated requests.

What to check

Check response codes, robots rules, and security logs for the relevant crawler, and verify crawler identities against the provider’s documentation.

What this does not tell you

Access does not guarantee indexing or citation, and different crawlers can have different permissions and purposes.

Sources

Evidence type: Documented concept. Sources checked on 2026-11-19

Related terms