Andrej Karpathy / @karpathy (RSS Feed) on Nostr: There's a chance that LoRA finetunes work so well that it dramatically alters the ...

There's a chance that LoRA finetunes work so well that it dramatically alters the finetuning vs. retrieval + few-shot prompting power dynamic in favor of the former for many applications.

PEFT (Parameter Efficient Finetuning, LoRA included) are emerging techniques that make it…

nitter.moomoo.me/danielgross/status/1648728342940758016#m (https://nitter.moomoo.me/danielgross/status/1648728342940758016#m)

https://nitter.moomoo.me/karpathy/status/1649127655122550784#m