Nazo on Nostr: npub1su4t3…6a0dw It, uh, hasn't been getting better for a while really. The current ...
npub1su4t3c2h2fl2ws6hfcnn5r7jdn87qvhvaafe5uv3hcwygxz3rjssf6a0dw (npub1su4…a0dw) It, uh, hasn't been getting better for a while really.
The current tech has likely already reached its peak for "smarts." At this point the only possible improvement is maybe efficiency of downsampling making it possible to get slightly larger models to run ok on lower hardware. They'll still have the fundamental problems.
And yes, the current methods of training are actually making them get worse and worse. (Synthetic data is bad but no one wants to hire humans for real data.)
The current tech has likely already reached its peak for "smarts." At this point the only possible improvement is maybe efficiency of downsampling making it possible to get slightly larger models to run ok on lower hardware. They'll still have the fundamental problems.
And yes, the current methods of training are actually making them get worse and worse. (Synthetic data is bad but no one wants to hire humans for real data.)