calle 👁️⚡👁️ on Nostr: The python shit is usually not the bottleneck, it probably uses native libs for the ...
The python shit is usually not the bottleneck, it probably uses native libs for the LLM stuff. Are you using the TPU?
Published at
2025-03-06 21:54:24Event JSON
{
"id": "ac23473187c0373f7c2851cd08333afadfff4d276091e6f549ab6585b401fafe",
"pubkey": "50d94fc2d8580c682b071a542f8b1e31a200b0508bab95a33bef0855df281d63",
"created_at": 1741298064,
"kind": 1,
"tags": [
[
"e",
"4e4bdbd92f20c4f4838612060da69b7c941e2bf28be9b62b1abf902b5bb5df08",
"",
"root"
],
[
"e",
"b67862167cbec5893bf429308652358be76b57bd3b72074538c4a29c958474f2"
],
[
"e",
"4710831eecc1fe430fd7e45f2005f56ec4ac63631eb4ccac0ed6fed4f51b2752",
"",
"reply"
],
[
"p",
"460c25e682fda7832b52d1f22d3d22b3176d972f60dcdc3212ed8c92ef85065c"
],
[
"p",
"50d94fc2d8580c682b071a542f8b1e31a200b0508bab95a33bef0855df281d63"
]
],
"content": "The python shit is usually not the bottleneck, it probably uses native libs for the LLM stuff. Are you using the TPU?",
"sig": "bd6d0a6b57dad681f2dea7271cfa92de5e6f7a8b89ff8f7c2ec0ec48ea26962ddbeafe776d651deb79791137e2ec04d851dac52546dc6777c01b79a2ba034d90"
}
