My Actual Brain on Nostr: I’m surprised at the performance an #llm can run at on my 8700-cpu. It’s a bit ...
I’m surprised at the performance an #llm can run at on my 8700-cpu.
It’s a bit slow and I don’t think it’s worth getting a gpu just to make it run at a faster speed, but maybe in time I’ll reconsider it.
If I were going to get a #gpu, what would you recommend?
#gpt #ollama #llama2
Published at
2024-03-14 10:24:42Event JSON
{
"id": "88877db6c850632fcfe245088997f0bde59de5d78deb3115c10b9b615d5e3f1d",
"pubkey": "aadf9e00528eba62656fbffaeacd17040a604968e398e0ba31a62ddf09d4e932",
"created_at": 1710411882,
"kind": 1,
"tags": [
[
"t",
"llm"
],
[
"t",
"gpu"
],
[
"t",
"gpt"
],
[
"t",
"ollama"
],
[
"t",
"llama2"
],
[
"proxy",
"https://fosstodon.org/users/my_actual_brain/statuses/112093553129248186",
"activitypub"
]
],
"content": "I’m surprised at the performance an #llm can run at on my 8700-cpu.\n\nIt’s a bit slow and I don’t think it’s worth getting a gpu just to make it run at a faster speed, but maybe in time I’ll reconsider it.\n\nIf I were going to get a #gpu, what would you recommend?\n\n#gpt #ollama #llama2",
"sig": "cc11aeaae5d50195e23fe23a9d685b92c265963a36ad28e3f10aab1ab4522fee7255e692bd633e45ae1db6f90312974a97d57e75ba7fcd226ee66896155de9e5"
}
