ChipTuner on Nostr: If they weren't so clear about their crawling. I also just made repo urls public on ...

If they weren't so clear about their crawling. I also just made repo urls public on my website Friday, so likely why crawlers picked it up. Its just been a steady stream of outgoing data XD

User Agent says "Amazbot" and "ClaudeBot/1.0; +claudebot@anthropic.com"

It's just this recursive path spam. I turned logging on briefly. Mind you there are no "icon" directories in these repos.

Example:
/cgit/vnuge/vnlib-core.git/tree/lib/Utils/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/icons/ubuntu-logo.png HTTP/1.1" 200 3121 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)"