Catherine Berry on Nostr: npub1u27r9…a3qfq So an extension of robots.txt, essentially. "Only read this if ...
npub1u27r9h3j9pvrplaffsmpn698e8xhmuqhdgcxldv67ammql9pumnqha3qfq (npub1u27…3qfq)
So an extension of robots.txt, essentially. "Only read this if your planned use of the data conforms with <license>." And crawlers could be made aware of what licenses they should not index.
Of course, this relies on compliance under threat of a lawsuit if a crawler operator is caught misusing your data. My sense is that this would scare off only small companies. Large companies have enough good lawyers to reliably dodge lawsuits, while for state actors, lawsuits aren't a threat at all.
So an extension of robots.txt, essentially. "Only read this if your planned use of the data conforms with <license>." And crawlers could be made aware of what licenses they should not index.
Of course, this relies on compliance under threat of a lawsuit if a crawler operator is caught misusing your data. My sense is that this would scare off only small companies. Large companies have enough good lawyers to reliably dodge lawsuits, while for state actors, lawsuits aren't a threat at all.