Anthony Baker on Nostr: npub1yk67q…04zz4 Outside of blocking bots via those that honor robots.txt, isn’t ...

npub1yk67q98wp2s8asr3pdvkmdmmh3x4566fz55m5rlvwsu0dj62s72q804zz4 (npub1yk6…4zz4) Outside of blocking bots via those that honor robots.txt, isn’t Google ALREADY data scraping for the purposes of SEO indexing? And in support of this, we have LD+JSON and Schema.org being used by publishers and sites to more CLEARLY represent what their content is in a structured data format for bots. I mean, publishers rolled out RED CARPETS for years using these tools — and they care about getting it right. LLM/AI apps are piggybacking on all of this.