Simon Willison on Nostr: Here's a one-liner that should work for you if you run uv on a Mac with 64GB of RAM ...
Here's a one-liner that should work for you if you run uv on a Mac with 64GB of RAM (it will download ~32GB of model the first time you run it)
uv run --with mlx-lm \
mlx_lm.generate \
--model mlx-community/Qwen2.5-Coder-32B-Instruct-8bit \
--max-tokens 4000 \
--prompt 'write me a python function that renders a mandelbrot fractal as wide as the current terminal'
Published at
2024-11-12 23:41:15Event JSON
{
"id": "5fb19965396c656534fcf552e90f4e85a432f4eaadd91c73f7b0aa68a3dff6ed",
"pubkey": "8b0be93ed69c30e9a68159fd384fd8308ce4bbf16c39e840e0803dcb6c08720e",
"created_at": 1731454875,
"kind": 1,
"tags": [
[
"e",
"592284477c42259d7ae6ddb555718bf9c6fc7494ab101adb23f212842d7ec359",
"wss://relay.mostr.pub",
"reply"
],
[
"proxy",
"https://fedi.simonwillison.net/users/simon/statuses/113472626749271413",
"activitypub"
]
],
"content": "Here's a one-liner that should work for you if you run uv on a Mac with 64GB of RAM (it will download ~32GB of model the first time you run it)\n\nuv run --with mlx-lm \\\n mlx_lm.generate \\\n --model mlx-community/Qwen2.5-Coder-32B-Instruct-8bit \\\n --max-tokens 4000 \\\n --prompt 'write me a python function that renders a mandelbrot fractal as wide as the current terminal'",
"sig": "3d6223efdec3a27c37532e8105f1cad78764450dc012195eeaac9df45ae1086f12048b0b53d349334a9e3917e05541f852e9d67667370af15cdd0d826b3da5c3"
}
