Sunflower Björnskalle 🌻 on Nostr: So I loaded Qwen2.5 7B and 14B this morning. 7B seems faster than LLaMa 3.1 8B (is ...
So I loaded Qwen2.5 7B and 14B this morning. 7B seems faster than LLaMa 3.1 8B (is that a thing?) and uses the same amount of RAM. Doesn't spin up my fans as soon for some reason. I don't like the answers about JS as much but I haven't tested it a lot. 14B barely runs on my computer, putting other things into swap. The answers are better but not that much better. I'm not really sure how I could test the difference.
This is all on CPU by the way.
#ollama #llm #llama #qwen #ai
Published at
2024-11-10 08:11:16Event JSON
{
"id": "cf7de5f661d622f760484ff44d4c63ca2a49c3588e2d4bb2da663567b99ed07f",
"pubkey": "ebdd255d38b43576b4a85e0f64879ef80d7bcc6dd2713236091ed8e27a5505cd",
"created_at": 1731226276,
"kind": 1,
"tags": [
[
"t",
"ollama"
],
[
"t",
"llm"
],
[
"t",
"llama"
],
[
"t",
"qwen"
],
[
"t",
"ai"
],
[
"proxy",
"https://mastodon.online/users/apodoxus/statuses/113457645230463203",
"activitypub"
]
],
"content": "So I loaded Qwen2.5 7B and 14B this morning. 7B seems faster than LLaMa 3.1 8B (is that a thing?) and uses the same amount of RAM. Doesn't spin up my fans as soon for some reason. I don't like the answers about JS as much but I haven't tested it a lot. 14B barely runs on my computer, putting other things into swap. The answers are better but not that much better. I'm not really sure how I could test the difference.\n\nThis is all on CPU by the way.\n\n#ollama #llm #llama #qwen #ai",
"sig": "647bf47a9ded821b1e8a0e892ba4d4fdf59bafbe33c2f01bece0d89121fab4c4ee5e66fc1ff4f00eec74a06b3959b8ba0608d3e0580263a54b78a476dda91748"
}
