Andrej Karpathy / @karpathy (RSS Feed) on Nostr: The analogy between GPTs of today to the CPUs of early days of computing are ...

The analogy between GPTs of today to the CPUs of early days of computing are interesting. GPT is a funny kind of programmable text computer. Have to think through it more 🤔 but e.g.:

\## Memory

GPT-4 RAM is ~log2(50K vocab size)\*(32K context length)/(8 bits/byte) ~= 64kB,… nitter.moomoo.me/i/web/status/164… (https://nitter.moomoo.me/i/web/status/1644183721405464576)

https://nitter.moomoo.me/karpathy/status/1644183721405464576#m