Daniel Wigton on Nostr: Ollama running llama3:70b I only get 1token per second with a 4090 but all the models ...
Ollama running llama3:70b I only get 1token per second with a 4090 but all the models smaller than that hallucinate frequently.
Published at
2024-07-10 09:25:54Event JSON
{
"id": "0f4139b04a6e5f3405c843ee4db7bdd42da32a8646a1cf6cc3c703398f30d770",
"pubkey": "75656740209960c74fe373e6943f8a21ab896889d8691276a60f86aadbc8f92a",
"created_at": 1720603554,
"kind": 1,
"tags": [
[
"e",
"acb566ee54238d8b5f5fe90d59dc4df50fa966307e82933d6379c7e315306a06",
"",
"root"
],
[
"p",
"a4cb51f4618cfcd16b2d3171c466179bed8e197c43b8598823b04de266cef110"
]
],
"content": "Ollama running llama3:70b I only get 1token per second with a 4090 but all the models smaller than that hallucinate frequently. ",
"sig": "bffdc33ef31c67aab4d0047a8a82a58a1d3cff457790d199e083605fe09e367807cef44ba036712be4fff031dbfcf794ded07b2b524edbd12537ec6d105e493f"
}
