Tom Morris on Nostr: Tested out some "AI detectors". First, I tested some text I'd written—most were 0% ...
Tested out some "AI detectors".
First, I tested some text I'd written—most were 0% or 1%, one was 8%. I'm apparently not a robot. Hooray.
The problem—the results for actual LLM generated text were all over the place.
A boring passage that reads like a gov website was correctly detected: 100% (Actual text from GOVUK? 22%.)
I then tested some ChatGPT-generated opening paras for undergrad essays (overwrought shit as you'd expect)—all rated 0% chance of being AI generated.
Whoops.
First, I tested some text I'd written—most were 0% or 1%, one was 8%. I'm apparently not a robot. Hooray.
The problem—the results for actual LLM generated text were all over the place.
A boring passage that reads like a gov website was correctly detected: 100% (Actual text from GOVUK? 22%.)
I then tested some ChatGPT-generated opening paras for undergrad essays (overwrought shit as you'd expect)—all rated 0% chance of being AI generated.
Whoops.