Simon Willison on Nostr: So far I've run Qwen2.5-Coder-32B successfully in two different ways: once via Ollama ...
So far I've run Qwen2.5-Coder-32B successfully in two different ways: once via Ollama (and the llm-ollama plugin) and once using Apple's MLX framework and mlx-llm - details on how I ran both of those are in my article.
Published at
2024-11-12 23:39:59Event JSON
{
"id": "592284477c42259d7ae6ddb555718bf9c6fc7494ab101adb23f212842d7ec359",
"pubkey": "8b0be93ed69c30e9a68159fd384fd8308ce4bbf16c39e840e0803dcb6c08720e",
"created_at": 1731454799,
"kind": 1,
"tags": [
[
"e",
"ca1c7a50bfe85049786f5bfdc1756ef883be16acd9977b0ade9595b716dfdfec",
"wss://relay.mostr.pub",
"reply"
],
[
"proxy",
"https://fedi.simonwillison.net/users/simon/statuses/113472621761886472",
"activitypub"
]
],
"content": "So far I've run Qwen2.5-Coder-32B successfully in two different ways: once via Ollama (and the llm-ollama plugin) and once using Apple's MLX framework and mlx-llm - details on how I ran both of those are in my article.",
"sig": "7d580509d5cf4175862a8f8bf62457a1d2d12ed8aa46fce4af52585d2cb34d187492a6b632e509d6425223572f74a76afb9b52c81cee13b8d1736e0234b68086"
}
