Tyler the Enginigger on Nostr: But it's also how you use these models, which ones you keep in memory at what time. ...

But it's also how you use these models, which ones you keep in memory at what time.

Running the same model, but with a regular python script to do stuff on the backend instead of just loading all the models at once and sticking them into memory (dumb fuck) you can at least try to get there.

I still ran out of memory at a later step because I probably need 16gb vram+ instead of 12. - but I don't _really_ know.