armstrys on Nostr: FYI after quite a bit of testing I settled on qwen2.5 coder 1.5b for autocomplete and ...
FYI after quite a bit of testing I settled on qwen2.5 coder 1.5b for autocomplete and llama 3.2 1b for chat. These models are tiny, but bigger models had too poor of speed performance on an M1 laptop for daily use. I’m sure the results will pale in comparison to larger models, but it is certainly better than nothing for free!
Published at
2024-10-05 23:47:03Event JSON
{
"id": "83ae8ec101f39644e2b4117ebc3093a347358de22300dd335e36e21349460c84",
"pubkey": "c80b5248fbe8f392bc3ba45091fb4e6e2b5872387601bf90f53992366b30d720",
"created_at": 1728172023,
"kind": 1,
"tags": [
[
"e",
"06a394668799cc6713680e98115eb9163bb5f141e9a4434121b1bd00bac877e1",
"",
"root"
],
[
"e",
"3b09aefb95c0e575445b76f4b0e959896db0fa3e202392eecb6ca61f0fe6d04b",
"",
"reply"
],
[
"p",
"7bdef7bdebb8721f77927d0e77c66059360fa62371fdf15f3add93923a613229"
],
[
"p",
"c80b5248fbe8f392bc3ba45091fb4e6e2b5872387601bf90f53992366b30d720"
],
[
"p",
"1bda7e1f7396bda2d1ef99033da8fd2dc362810790df9be62f591038bb97c4d9"
],
[
"client",
"Nostur",
"31990:9be0be0fc079548233231614e4e1efc9f28b0db398011efeecf05fe570e5dd33:1685868693432"
]
],
"content": "FYI after quite a bit of testing I settled on qwen2.5 coder 1.5b for autocomplete and llama 3.2 1b for chat. These models are tiny, but bigger models had too poor of speed performance on an M1 laptop for daily use. I’m sure the results will pale in comparison to larger models, but it is certainly better than nothing for free!",
"sig": "787c76ff823897720f4d88fab8657b219eaf3c2cfc5fa57c6b5f38b4e943b60af5919e1b66ac46860d3d65a16a3271a127e01791ccd2e4fcea175438a444c268"
}
