Eluc on Nostr: You would need a data center full of these Nvidia premium GPU to run a 107B model. 7B ...
You would need a data center full of these Nvidia premium GPU to run a 107B model.
7B or 8x7B models can run on a consumer PC.
Published at
2024-07-24 09:23:23Event JSON
{
"id": "047436697f34da1bb6a85a742d8a5dcc2d1f8f5f2949128c2860c1b4f6edfd77",
"pubkey": "cf7ad05f8e99de8eadbbfbd5ca1c0f9b75499bce07074966b277688ca5e1d726",
"created_at": 1721813003,
"kind": 1,
"tags": [
[
"e",
"fc1509642a661c3583e415e8b0081dc09943c4e45ed7976e045db96ebdf018ad",
"",
"root"
],
[
"e",
"83b6e7dfc5e2d6870541a3e96c959872cbd8eeb9c0a01a7f53f85a2a60025c9b"
],
[
"e",
"c7586e58e606caa5d14bee8c63d9c291221ea4aef585c0a5ab2bc16844fbc1f5",
"",
"reply"
],
[
"p",
"cf7ad05f8e99de8eadbbfbd5ca1c0f9b75499bce07074966b277688ca5e1d726"
],
[
"p",
"1bc70a0148b3f316da33fe3c89f23e3e71ac4ff998027ec712b905cd24f6a411"
]
],
"content": "You would need a data center full of these Nvidia premium GPU to run a 107B model. \n\n7B or 8x7B models can run on a consumer PC.",
"sig": "feb89c7c54ac91b6d899ebb0ba7bea3b53cd3c741febbba2deea458e89bd8e41551ab51b0cf46220667537e09b12f9d04123b26b31a6217d77711bd053df2e76"
}
