Thomas Renkert🦞 on Nostr: Hi Roberto von Archimboldi, I think Christopher Pollin already gave a more than ...
Hi Roberto von Archimboldi (npub1zjt…6a2t), I think Christopher Pollin (npub1nvs…uz84) already gave a more than extensive list to your question.
In my view, to put it simply: overall, humanity seems to have communicated much more truths than falsehoods. Which, to me, is not surprising. Falsehoods are not useful to the majority of people. If you go on reddit or stackexchange with a specific question, chances are you'll get an answer that is in some way helpful.
That being said, not long ago models predicted more truly false chains of tokens. The datasets of the current state of the art are heavily curated, i.e. false info, biases, propaganda, and so on have actively been purged to improve performance.
But what really will bring hallucinations down further in the near future are tool use, RAGs, and perhaps other architectures like MAMBA.
In my view, to put it simply: overall, humanity seems to have communicated much more truths than falsehoods. Which, to me, is not surprising. Falsehoods are not useful to the majority of people. If you go on reddit or stackexchange with a specific question, chances are you'll get an answer that is in some way helpful.
That being said, not long ago models predicted more truly false chains of tokens. The datasets of the current state of the art are heavily curated, i.e. false info, biases, propaganda, and so on have actively been purged to improve performance.
But what really will bring hallucinations down further in the near future are tool use, RAGs, and perhaps other architectures like MAMBA.