penguin42 on Nostr: I've got a local #AI running for the first time; llama.cpp running ...
I've got a local #AI running for the first time; llama.cpp running 'SmolLM2-1.7B-Instruct' - which is about 3.5GB download and uses about 4GB of host RAM; it's pretty impressive for such a small beasty. It's managing pretty reasonable German translation on Toots (although seems to be confused on tenses). It tends to make stuff up if you ask it 'what is a ...'.
🧵
Published at
2025-02-08 18:07:26Event JSON
{
"id": "5a5afbe998bfa641d32a23a27f549512ed99f30f077b5fb6e9175a0788164f2f",
"pubkey": "eee5ad6518e05d777010543e26a8f2469cf321f303314125d44a34e08e66f230",
"created_at": 1739038046,
"kind": 1,
"tags": [
[
"t",
"ai"
],
[
"proxy",
"https://mastodon.org.uk/users/penguin42/statuses/113969597387653522",
"activitypub"
]
],
"content": "I've got a local #AI running for the first time; llama.cpp running 'SmolLM2-1.7B-Instruct' - which is about 3.5GB download and uses about 4GB of host RAM; it's pretty impressive for such a small beasty. It's managing pretty reasonable German translation on Toots (although seems to be confused on tenses). It tends to make stuff up if you ask it 'what is a ...'.\n\n🧵",
"sig": "c0696a61f5086ccfa6f1cef0c400ac0e3d415f3bdd278b705272663f47eb874c76be8b8d2e88a2c2549f19ca14ee2d6475d16bbb352f837a4275765b537a3986"
}
