aljaz on Nostr: you are right, 14b works decently, I should switch to vllm from ollama and that ...
you are right, 14b works decently, I should switch to vllm from ollama and that should speed it up even more
aljaz
npub1alj…g9jp
2025-01-30 15:06:48
in reply to nevent1q…5n7l
Author Public Key
npub1aljazgxlpnpfp7n5sunlk3dvfp72456x6nezjw4sd850q879rxqsthg9jpSeen on
wss://nos.lolwss://nostr.momwss://nostr.winewss://offchain.pubwss://relay.damus.iowss://relay.nostr.bandwss://relay.primal.netPublished at
2025-01-30 15:06:48Event JSON
{
"id": "7e7a3ecfe82c52817e67005d807dc77e87dba810c763a757afe785f74a0c7181",
"pubkey": "efe5d120df0cc290fa748727fb45ac487caad346d4f2293ab069e8f01fc51981",
"created_at": 1738249608,
"kind": 1,
"tags": [
[
"e",
"7c826f0d8fa190876df885e7433c450196bddcae86c3f157b05239397eab474a",
"wss://nostrelay.yeghro.site",
"root",
"f821179bd5d691c61b4356336388adadc3cc1304f49e771a87c38de7ed4fd022"
],
[
"e",
"3a86e335ede5ca58a48714e85c0ad0c7cc74a2ea2966beae32d987b1783dcdb9",
"wss://nostrelay.yeghro.site",
"reply"
],
[
"p",
"f821179bd5d691c61b4356336388adadc3cc1304f49e771a87c38de7ed4fd022",
""
],
[
"p",
"efe5d120df0cc290fa748727fb45ac487caad346d4f2293ab069e8f01fc51981"
],
[
"r",
"wss://nos.lol/"
],
[
"r",
"wss://nostr.bitcoiner.social/"
],
[
"r",
"wss://nostr.mom/"
],
[
"r",
"wss://nostr.oxtr.dev/"
],
[
"r",
"wss://nostr.wine/"
],
[
"r",
"wss://offchain.pub/"
],
[
"r",
"wss://premium.primal.net/"
],
[
"r",
"wss://relay.damus.io/"
],
[
"r",
"wss://relay.mostr.pub/"
],
[
"r",
"wss://relay.nostr.band/"
],
[
"r",
"wss://relay.nostr.net/"
],
[
"r",
"wss://relay.primal.net/"
],
[
"r",
"wss://relay.snort.social/"
],
[
"r",
"wss://wot.nostr.net/"
]
],
"content": "you are right, 14b works decently, I should switch to vllm from ollama and that should speed it up even more",
"sig": "084395865ee0d59d043edd2f2947423700e9679af28f245ee4a67a558a2d3aa11ee4fcc9f78ab4044d395b41dc3181411761baaef5288a6e523523217afe883d"
}