Emergent Behaviours on Nostr: Disagree on what? On the fact that a SOTA model can be created without $100 million+ ...
Disagree on what? On the fact that a SOTA model can be created without $100 million+ of compute and 4T+ token training set? No way the Chinese innovated it down to $5 million. Psyop. They distilled at the very least.
I think the only win is that we all know the Reinforcement Learning is the key to results like this. OpenAI wasn’t going to tell us.
I’m all for open source models and AI research for sure! I am ashamed at OpenAI, honestly. But I’m not about to sing China’s praises for this.
Published at
2025-01-30 22:31:47Event JSON
{
"id": "ebf641f826c5146caa52511a57fb045ba2ec21cdcd2492a0e8fa0478fd6c3b30",
"pubkey": "c24bb3ce6631aa73237fc086a91d0a201559ab64694ec0fb53cc3e895e1bee84",
"created_at": 1738276307,
"kind": 1,
"tags": [
[
"e",
"16225e729714a831daa41a1e382140ad18a47bd947bca5a726e0a2613c535db9",
"",
"root",
"607e732b12a724d5e112ccd4ba1fd198c4b56df3c871339e200901a5a26d5638"
],
[
"e",
"0ba117ecffc20f3ff6421db3e7a58f1ff78d14970ec320ffdfaaa899554e1c60",
"wss://relay.primal.net",
"reply"
],
[
"p",
"607e732b12a724d5e112ccd4ba1fd198c4b56df3c871339e200901a5a26d5638"
]
],
"content": "Disagree on what? On the fact that a SOTA model can be created without $100 million+ of compute and 4T+ token training set? No way the Chinese innovated it down to $5 million. Psyop. They distilled at the very least.\n\nI think the only win is that we all know the Reinforcement Learning is the key to results like this. OpenAI wasn’t going to tell us.\n\nI’m all for open source models and AI research for sure! I am ashamed at OpenAI, honestly. But I’m not about to sing China’s praises for this.",
"sig": "4ae2d05a8ba7d52eb186eb88035ccc4fe9ae18d625cb023b793aaaf2c10aa8908c266285fdb6baf1813c2456d155d68f457dc4fa332e7fa1d340690cbf1ffe5d"
}
