Scott Williams 🐧 on Nostr: Googlebot is unhinged. From Google docs: # Example 3: Block all crawlers except ...
Googlebot is unhinged.
From Google docs:
# Example 3: Block all crawlers except AdsBot (AdsBot crawlers must be named explicitly)
User-agent: *
Disallow: /
Published at
2025-02-10 20:21:31Event JSON
{
"id": "21976d357cc0afde41838245d1bfecce65798afb01c364dcaf64c7f26fe2589e",
"pubkey": "beee28c6450eb5d21bf94279d1f3f41e524c87abf86393835b45e6a1e98390e7",
"created_at": 1739218891,
"kind": 1,
"tags": [
[
"proxy",
"https://mastodon.online/users/vwbusguy/statuses/113981449279408878",
"activitypub"
]
],
"content": "Googlebot is unhinged. \n\nFrom Google docs:\n\n # Example 3: Block all crawlers except AdsBot (AdsBot crawlers must be named explicitly)\n User-agent: *\n Disallow: /",
"sig": "ef8282358991e2fa53bb6aba2958ccfd5cd826eb14dd0db3d612e171a26c8ab80531db6d4aba2007c21344589f5b2401e26e38df1d29d9f567367e161046152a"
}