Mathaetaes on Nostr: this is way beyond my understanding, but there’s a step in there where they ...
this is way beyond my understanding, but there’s a step in there where they quantize an fp32 to an int8. I don’t know how much of the fp32 range a model typically uses, but int8 is a hell of a lot less precise. I’d be curious to see a side by side comparison of outputs of a model trained on the same data, running on this vs on a GPU or whatever they typically run on.
Published at
2024-04-17 08:38:24Event JSON
{
"id": "19d03d5ab0798d7d37d986e93998e72c0d8fc2d70e46c990b937d2a4dea3f6a6",
"pubkey": "330a1b5579daabec624a4de6c64a810b6a98b60b473a5e9ad12bb56961b71704",
"created_at": 1713343104,
"kind": 1,
"tags": [
[
"e",
"ae125e6e3de446f36ae0335d2dbc651dece1be614192d09208603a6ebf33e5d3",
"",
"root"
],
[
"p",
"ccca24c1e7c8dc9068b0c0f6ed38670d995d42c6f9ed5fdcc725baa0e39da1a6"
],
[
"proxy",
"https://infosec.exchange/users/mathaetaes/statuses/112285653665477315",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://infosec.exchange/users/mathaetaes/statuses/112285653665477315",
"pink.momostr"
]
],
"content": "this is way beyond my understanding, but there’s a step in there where they quantize an fp32 to an int8. I don’t know how much of the fp32 range a model typically uses, but int8 is a hell of a lot less precise. I’d be curious to see a side by side comparison of outputs of a model trained on the same data, running on this vs on a GPU or whatever they typically run on.",
"sig": "98baeee1cf05d98bd424538371200928ea3214920888681cf91cdcec7358dcf8da631bb313221e9bdd5848fb6a7792433bdde642ab02b8aac943a05ebc236e40"
}
