JTaggart on Nostr: https://huggingface.co/relaxml/Llama-2-70b-chat-QTIP-2Bit New quant method allegedly ...

https://huggingface.co/relaxml/Llama-2-70b-chat-QTIP-2Bit

New quant method allegedly no drop in quality, let's you fit 70b llamas on a 3090

If I procrastinate upgrading my hardware long enough I just won't need to