Joe Fabisevich :verified: on Nostr: nprofile1q…gsda9 Yes, I totally agree with that quote! And if you try a model like ...

nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqye6kenedjm4h3rhher5r98jlqfa7ayt59tpkmrqgy3002rzlcc0s0gsda9 (nprofile…sda9) Yes, I totally agree with that quote! And if you try a model like Microsoft’s Phi-4, you see an example of how to make a high quality model trained on lots of synthetic data, because of the care that went into how and what to train on initially. It goes to show that you can achieve a local model that provide GPT-4o mini quality results with the right choices around training and data, which is why I think model collapse is already an outdated concept.

https://techcommunity.microsoft.com/blog/aiplatformblog/introducing-phi-4-microsoft%E2%80%99s-newest-small-language-model-specializing-in-comple/4357090