jsm on Nostr: The regular Transformer may have just been unseated as the de facto architecture for ...
The regular Transformer may have just been unseated as the de facto architecture for LLMs. I’m no expert but Diff Transformers seem like a massive improvement over regular Transformers.
https://arxiv.org/abs/2410.05258Published at
2024-10-09 14:05:17Event JSON
{
"id": "e42bf56c5ada5b0212450b2cbfddd3999e1be532eb3a0e20ee594f2eb3d150ee",
"pubkey": "e0339348ca6cac9708cd98e631e2f4baad534dfce870881b65aa57d30ff7253e",
"created_at": 1728482717,
"kind": 1,
"tags": [],
"content": "The regular Transformer may have just been unseated as the de facto architecture for LLMs. I’m no expert but Diff Transformers seem like a massive improvement over regular Transformers.\n\nhttps://arxiv.org/abs/2410.05258",
"sig": "e0d9f92e1b4c323991dea9d6849c2e5a7a787a99352c0d648d82694647a1e21da639e068e273d3ec89a0f3934bf31d8081fcd9acb2a1e077e27e674198934de6"
}
