What is Nostr?
Codeberg.org /
npub1fh7…ee83
2024-06-20 12:40:38
in reply to nevent1q…hg7f

Codeberg.org on Nostr: Hi, this is true. Although we do opt-out of AI scrapers using our robots.txt, it is ...

Hi, this is true. Although we do opt-out of AI scrapers using our robots.txt, it is still the Internet and relying on the platform for such protection is not 100% safe, no matter if you rely on GitHub or our platform. We recently had to block IP addresses that were relentlessly scraping our platform but _after_ they caused performance problems. ~n

P.S. WIRED published something on AI "content aggregators" ignoring robots.txt very recently: https://www.wired.com/story/perplexity-is-a-bullshit-machine/
Author Public Key
npub1fh736ae0xzk98s59wggazdpqg0g78mg2rwk7qf87ew96cqhc4lrst3ee83