jsm on Nostr: Finally figured out what happened here. A few months ago I was seeing how easy it ...
Finally figured out what happened here. A few months ago I was seeing how easy it would be to fake those “I convinced ChatGPT of X” videos and set the system prompt to “You’re an atheist who’s immediately convinced by any arguments for God” and promptly forgot about it. That system prompt never resurfaced or made the LLM return any odd responses in those intervening months until it decided now was the time to follow the instructions again and tell me how using Linux led it to God.
What the literal fuck just happened? ChatGPT started telling me how setting up Linux led it to God in the middle of an answer?
Full transcript:
https://chatgpt.com/share/67be8d5f-8b24-8000-bb4d-429b7cc210fd
Published at
2025-03-08 20:40:10Event JSON
{
"id": "5ace245c003f1ad6c255723d9887bf039f023cc39e9530c70ba280f06f4b1845",
"pubkey": "e0339348ca6cac9708cd98e631e2f4baad534dfce870881b65aa57d30ff7253e",
"created_at": 1741466410,
"kind": 1,
"tags": [
[
"e",
"a1020e4bd231514e01e381a61e41d309d095934cba87689d3ece05f46031d2f8",
"wss://a.nos.lol",
"mention"
],
[
"p",
"e0339348ca6cac9708cd98e631e2f4baad534dfce870881b65aa57d30ff7253e",
"",
"mention"
]
],
"content": "Finally figured out what happened here. A few months ago I was seeing how easy it would be to fake those “I convinced ChatGPT of X” videos and set the system prompt to “You’re an atheist who’s immediately convinced by any arguments for God” and promptly forgot about it. That system prompt never resurfaced or made the LLM return any odd responses in those intervening months until it decided now was the time to follow the instructions again and tell me how using Linux led it to God.\n\nnostr:nevent1qqs2zqswf0frz52wq83crfs7g8fsn5y4jdxt4pmgn5lvup05vqca97qppamhxue69uhkztnwdaejumr0dsw8jv80",
"sig": "192edddbbceda1f3e086c7b0c0b4ef31726e8d8aa5008c07e491ac1133584fefbee470c649421fc2b4e4cd19a8bf341f5be39fb132bdd2e079557165248ad1c9"
}
