Andrej Karpathy / @karpathy (RSS Feed) on Nostr: Random note on k-Nearest Neighbor lookups on embeddings: in my experience much better ...

Random note on k-Nearest Neighbor lookups on embeddings: in my experience much better results can be obtained by training SVMs instead. Not too widely known.

Short example:

github.com/karpathy/randomfu… (https://github.com/karpathy/randomfun/blob/master/knn_vs_svm.ipynb)

Works because SVM ranking considers the unique aspects of your query w.r.t. data.

https://nitter.moomoo.me/pic/card_img%2F1647023936436322305%2FXH-pt3Tf%3Fformat%3Djpg%26name%3D800x419

https://nitter.moomoo.me/karpathy/status/1647025230546886658#m