BlokchainB on Nostr: “When it comes to hosting your own model, the main cost, as mentioned, would be ...
“When it comes to hosting your own model, the main cost, as mentioned, would be hardware. Consider, for example, hosting an open-source Falcon 180B on AWS. The default instance recommended by AWS is ml.p4de.24xlarge, with a listed price of almost 33 USD per hour (on-demand). This means that such a deployment would cost at least 23,000 USD per month, assuming it doesn't scale up or down and that no discounts are applied.”
COT DAMN!!!!! $23k a month for LL
https://www.tensorops.ai/post/understanding-the-cost-of-large-language-models-llmsShout to Ansel Linder for providing this link in his telegram chat!
Published at
2024-06-24 15:59:00Event JSON
{
"id": "689a2c06ac1e889aa5eefb3259412270ea30108c40aa060732309b010eb893d5",
"pubkey": "38b07a31f3b23dbeb9f59deb7bec5b993173fb4022206980f3809d0b68abf959",
"created_at": 1719244740,
"kind": 1,
"tags": [
[
"r",
"https://www.tensorops.ai/post/understanding-the-cost-of-large-language-models-llms"
]
],
"content": "“When it comes to hosting your own model, the main cost, as mentioned, would be hardware. Consider, for example, hosting an open-source Falcon 180B on AWS. The default instance recommended by AWS is ml.p4de.24xlarge, with a listed price of almost 33 USD per hour (on-demand). This means that such a deployment would cost at least 23,000 USD per month, assuming it doesn't scale up or down and that no discounts are applied.”\n\nCOT DAMN!!!!! $23k a month for LL\n\nhttps://www.tensorops.ai/post/understanding-the-cost-of-large-language-models-llms\n\nShout to Ansel Linder for providing this link in his telegram chat!",
"sig": "d50c8ea6187fdc17751759910b649c3669e8dfe5ac9faf525fa874c01d17089d84328e12d1975dabc7b529e70e9eb8087c9f35d4a474ee894e61a0e7a55341e5"
}
