What is Nostr?
jb55 / Will
npub1xts…kk5s
2024-06-11 18:20:45

jb55 on Nostr: Apple’s on-device ai model is 3B params, 2 to 4bit quantization: “On this ...

Apple’s on-device ai model is 3B params, 2 to 4bit quantization:

“On this benchmark, our on-device model, with ~3B parameters, outperforms larger models including Phi-3-mini, Mistral-7B, and Gemma-7B. Our server model compares favorably to DBRX-Instruct, Mixtral-8x22B, and GPT-3.5-Turbo while being highly efficient.”

Interesting! This was the size of model I was considering for damus mobile. Looks like I can just use apple intelligence apis instead 🤔 . These small local models are pretty good at summarization, which I’m guessing why they showcased that a lot in notifications, mail, imessage, etc.

https://machinelearning.apple.com/research/introducing-apple-foundation-models
Author Public Key
npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s