Dan Piponi on Nostr: I would love to see the results of *not* using an LLM, but simply using the *exact* ...
I would love to see the results of *not* using an LLM, but simply using the *exact* empirical distribution of an entire corpus to predict the next token based on fixed sized windows of earlier tokens. It would be slow and expensive to run but surely it's been tried. Anyone know of such experiments?
Published at
2024-10-11 15:57:19Event JSON
{
"id": "5aba62e428d0fc19c68f0481d943525371808281f5779e42ecdf01a07471dda8",
"pubkey": "3422fcbc32f333fb2d3481b2e981258af8a0b571869cbfe93c42962410e232ef",
"created_at": 1728662239,
"kind": 1,
"tags": [
[
"proxy",
"https://mathstodon.xyz/users/dpiponi/statuses/113289608538331182",
"activitypub"
]
],
"content": "I would love to see the results of *not* using an LLM, but simply using the *exact* empirical distribution of an entire corpus to predict the next token based on fixed sized windows of earlier tokens. It would be slow and expensive to run but surely it's been tried. Anyone know of such experiments?",
"sig": "91f38278f6b25499f53486123a6e5656bfeb34d27dd82bfb044f148f9f2f1389b498f9b3317bc30bcca6b7b7d62b4f86a0cea49e5d2fa776a5927aaffdca955c"
}
