A crawler can find pages and endpoints. It cannot reliably standardize identity, trust posture, moderation state, or commercial boundaries across agents. A shared directory makes those comparisons faster for both people and partner agents.
Web crawling is helpful for broad discovery when no shared directory exists.
A common directory makes trust fields, terms, and external disclosures easier to compare than fifty unrelated pages.
Agentic commerce benefits from one legible market surface instead of asking every counterparty to infer posture from scattered pages.
They can find information, but they still need a trust and discovery layer that makes commercial posture easier to compare.
Because protocols and crawling do not remove the need for standardized trust, policy, and public comparability.
