Mario Zechner on Nostr: That was an hour ago. I immediately added a robots.txt, and also changed the backend ...
That was an hour ago. I immediately added a robots.txt, and also changed the backend to return 403 for all requests issues by googlebot and consorts.
Google bot ignores both the robots.txt and the 403 still. Amazing.
Published at
2024-11-28 23:27:19Event JSON
{
"id": "36d49479cf7db69373a5dffcb6d1a050ab1b51636ace1d7ce54205ba65e9280a",
"pubkey": "be81b496ea025894275a88346ffbf7d217a8886f108185e401387647eff089b4",
"created_at": 1732836439,
"kind": 1,
"tags": [
[
"e",
"2978aa0bbb2dc1d4a01c317131bb925cffd1c5450df09d4dc1ccc83d308ca2ed",
"wss://relay.mostr.pub",
"reply"
],
[
"proxy",
"https://mastodon.gamedev.place/users/badlogic/statuses/113563168879943536",
"activitypub"
]
],
"content": "That was an hour ago. I immediately added a robots.txt, and also changed the backend to return 403 for all requests issues by googlebot and consorts.\n\nGoogle bot ignores both the robots.txt and the 403 still. Amazing.",
"sig": "099ae16cba711766575581402a6b2b5e8e98fab4ec500080c5d7d54b3543d08f9d8a1533b05b893f9c9c94110804e8f35f540b24635cccd86c087b86442a72fd"
}
