HAPPYPOTAMUS on Nostr: They were lying, about the cost specifically You can run local models at a usable ...
They were lying, about the cost specifically
You can run local models at a usable speed with a decent CPU and a lot of RAM
I'm sure they also use a lot of token caching, surely they're not recomputing "implement fizzbuzz" 10000x a day
Published at
2024-12-24 21:05:07Event JSON
{
"id": "370ca012188eb229fed8c94fe9db49daae6ace2108a68f1fafb554ff6197086c",
"pubkey": "6a12c2a4ac2cc6898b083ec3d026a07d2dbbe429a894bc3e46c08db21b125fc8",
"created_at": 1735074307,
"kind": 1,
"tags": [
[
"p",
"3bf0c63fcb93463407af97a5e5ee64fa883d107ef9e558472c4eb9aaaefa459d",
"wss://relay.mostr.pub"
],
[
"e",
"0000d5368f2ab0ecd43a672d48309bdb05824bf00b8db2098eaee3ad807210f7",
"wss://relay.mostr.pub",
"reply"
],
[
"proxy",
"https://nicecrew.digital/objects/b00240b9-583a-4e8e-be4f-0eb60fb1ef8b",
"activitypub"
]
],
"content": "They were lying, about the cost specifically\n\nYou can run local models at a usable speed with a decent CPU and a lot of RAM\n\nI'm sure they also use a lot of token caching, surely they're not recomputing \"implement fizzbuzz\" 10000x a day",
"sig": "edbd3b867c2082b41b8211e1277ffabf1169748a5e429cbd5a27cf290df6638d0df877983af24c64d65b1746a305238be610be0fdce1b870e51d761535dbe613"
}
