What is Nostr?
roshii
npub1vxl…2vvs
2024-07-22 07:58:26
in reply to nevent1q…kjjs

roshii on Nostr: 7B parameters models work fine with 16 GB RAM, getting ~3 word/s on Ryzen 5 5600X. ...

7B parameters models work fine with 16 GB RAM, getting ~3 word/s on Ryzen 5 5600X. Running on GPU helps a lot even with a low end one: getting ~12 word/s on RX6400 with 4 GB GDDR6.
I was not able to run models with more parameters, limited by RAM.
Author Public Key
npub1vxlqpu554sq6yz3306ayp9sdz087xqpckt9usu20f43gqynuwldsjm2vvs