Lauren Weinstein on Nostr: The nightmarish dangers of generative AI text, code, and other related systems, is ...
The nightmarish dangers of generative AI text, code, and other related systems, is not when they get things correct. The horror is when they get things "partly" correct -- and mix in errors that are not fully reviewed, understood, or corrected -- usually in convincing ways. Human nature is to be seduced by this sort of situation, and to be lulled into a false confidence that only cursory (if any) reviews are necessary.
I see this situation with #Google AI Overviews all the time. Errors sprinkled all through them, often in ways only an expert on the topic would recognize. Incredibly dangerous.
It's a trap.
L
Published at
2024-09-28 16:59:52Event JSON
{
"id": "65af2d7e5701efaf5ab03c3dc16ea0105282b0a65f0c2456aaf56eb927b9ccf5",
"pubkey": "a447aea32251d6c533dc9cbbf15d3e41529f8a6719c99cee944bfd9aa0928ef0",
"created_at": 1727542792,
"kind": 1,
"tags": [
[
"t",
"google"
],
[
"proxy",
"https://mastodon.laurenweinstein.org/users/lauren/statuses/113216244426304661",
"activitypub"
]
],
"content": "The nightmarish dangers of generative AI text, code, and other related systems, is not when they get things correct. The horror is when they get things \"partly\" correct -- and mix in errors that are not fully reviewed, understood, or corrected -- usually in convincing ways. Human nature is to be seduced by this sort of situation, and to be lulled into a false confidence that only cursory (if any) reviews are necessary. \n\nI see this situation with #Google AI Overviews all the time. Errors sprinkled all through them, often in ways only an expert on the topic would recognize. Incredibly dangerous.\n\nIt's a trap.\n\nL",
"sig": "cd1fcffbba6c203237175b828190c8632011b0cbebe68296c55d5bf9822912cf7def7260ae8cf554f75f4b0c4c1dad3aa40cd58ed35ca0280f425af2bab9be86"
}
