John Dee on Nostr: 4090. I haven't had much time to compare any models yet, and I don't know how to read ...
4090. I haven't had much time to compare any models yet, and I don't know how to read those comparison charts. I think larger models can be quantized to fit into less VRAM but performance suffers as you get down to 4 and 2-bit.
Published at
2024-07-24 21:48:22Event JSON
{
"id": "c2dfbbbc7d8937b03a6d036fccfe11048d88837f05f4c938d06ada981ef9d8fa",
"pubkey": "fe32298e29aab4ec2911c0dbdda485c073f869c5444ee92f7ae247ed20516265",
"created_at": 1721857702,
"kind": 1,
"tags": [
[
"e",
"00003c9dda7204845a2ef6a0a5a08d7572caf85dc738e34a05648f113a342f49",
"",
"root"
],
[
"e",
"29478e99a31e19458338a5086420b3851b1ab615ff63660483c41ad664c2fa96"
],
[
"e",
"0fc9223b51a87f18bf8bf8bd651964192c5602720850c5545124fa58a28fdd9e",
"",
"reply"
],
[
"p",
"fe32298e29aab4ec2911c0dbdda485c073f869c5444ee92f7ae247ed20516265"
],
[
"p",
"b2d670de53b27691c0c3400225b65c35a26d06093bcc41f48ffc71e0907f9d4a"
]
],
"content": "4090. I haven't had much time to compare any models yet, and I don't know how to read those comparison charts. I think larger models can be quantized to fit into less VRAM but performance suffers as you get down to 4 and 2-bit.",
"sig": "09e9225bc5e1a8a656733992363c0e59e1c2aae79d22a54669c93c412112912ff1e45acc09e83ad22b30f1f96ce6039d41484be92d42fea6963cd08ffe79135d"
}
