Raul007 on Nostr: Awesome. I was wondering whether it was worth it to grab some extra system RAM to try ...
Awesome. I was wondering whether it was worth it to grab some extra system RAM to try a 70B model on CPU only, but it runs just as slow as my CPU w/ GPU offloading.
🤷 I'll just have to wait for used 3090s to come down in price.
Published at
2023-08-25 22:07:08Event JSON
{
"id": "dd4dca789b7e93c24e576b3e02f1a4ccd9b75115405cac86c60c82e0825eff99",
"pubkey": "fca3f1847bdda5b29e631edfb8ce0991af688041051dfb6d3bf236880afd1678",
"created_at": 1693001228,
"kind": 1,
"tags": [
[
"e",
"de54a56178aee40e870abe709b7318b8965d0df1a17a0c668420d642225b2faf",
"",
"reply"
],
[
"p",
"d3af49435bb13c695232fd8ca8c9e73db8c65630553baef83f9544e81cef9be7"
]
],
"content": "Awesome. I was wondering whether it was worth it to grab some extra system RAM to try a 70B model on CPU only, but it runs just as slow as my CPU w/ GPU offloading. \n\n🤷 I'll just have to wait for used 3090s to come down in price.",
"sig": "357124c3bf6d775af2a733f55dcd209203ac77440bc750cb55fec632eeb8436d8e0f24145334ef31407e787db49a30e579bdf86d94997b8176e9bbb4b6452106"
}
