Karl Voit :emacs: :orgmode: on Nostr: Exclusive-Multiple #AI companies bypassing web standard to scrape publisher sites, ...
Exclusive-Multiple #AI companies bypassing web standard to scrape publisher sites, licensing firm says
https://finance.yahoo.com/news/exclusive-multiple-ai-companies-bypassing-143742513.html?guccounter=1
"A Wired investigation published this week found Perplexity likely bypassing efforts to block its web crawler via the Robots Exclusion Protocol, or "robots.txt," a widely accepted standard meant to determine which parts of a site are allowed to be crawled."
https://finance.yahoo.com/news/exclusive-multiple-ai-companies-bypassing-143742513.html?guccounter=1
"A Wired investigation published this week found Perplexity likely bypassing efforts to block its web crawler via the Robots Exclusion Protocol, or "robots.txt," a widely accepted standard meant to determine which parts of a site are allowed to be crawled."