Five on Nostr: ## Why I am sidelining DVMs for now When I started to grow tired of random posts on ...
## Why I am sidelining DVMs for now
When I started to grow tired of random posts on nostr I had a plan to create a DVM to tackle the problem of noise. I wanted a feed that caters to people like me in it for the professional discussions and meaningful FOSS work.
I used the LLM approach because simple hashtag parsing can only do so much. First I thought I would fine-tune a BERT-like model that should be easier to self-host. It turned out to be a harder goal to achieve than I expected.
I wanted to use GPT4-o to help me generate data for the fine-tuning. Sadly that was a bummer. It screwed me over so many times with garbage, I got exhausted.
Then I downloaded a bunch of high-rated stackexchange posts to be clustered and used for the fine-tuning.
This already took me more time than I wanted and realized that for a PoC I might as well use openai api. So did that and started experimenting with different prompts for GPT4-o mini.
In the meantime I got acquainted with the python based nostr-dvm framework and setup the basics I need for a DVM service.
After some grind I got everything working but still was not very pleased with the classification results.
Now, I know I could have put in even more time to really nail that prompt but I kinda lost my faith and appetite. I successfully use AI to learn and generate rudimentary code and chat about ideas. But what I needed is to generate a feed without manual intervention. AI people tend to recommend techniques to improve results that are borderline witchcraft. And every single time GPT finds a way to hallucinate plenty of stuff that I did not expect, no matter how hard I seem to try. This is my experience from months of daily interaction with GPT4-o too, which is much better than the mini version.
So no, I won't get trapped in the ai hype again. It is what it is: without human oversight these things are still worthless. I'm not aiming for a 90% usable thing, and I don't have a straightforward way to get to a 100%. No one has because these stochastic models are not AIs at all. They are mimicking parrots, nothing more. And this direction is a dead-end if you ask me.
All in all to customize a high-quality feed I tend to agree now that something like #nostrscript seems to be the better way to go from jb55 (nprofile…evy3) .
It has the human element but is enhanced with the right tech to be much more than just picking hashtags to follow.
We might see a bunch of better use-cases for DVM-s but this is my overall sentiment right now.
When I started to grow tired of random posts on nostr I had a plan to create a DVM to tackle the problem of noise. I wanted a feed that caters to people like me in it for the professional discussions and meaningful FOSS work.
I used the LLM approach because simple hashtag parsing can only do so much. First I thought I would fine-tune a BERT-like model that should be easier to self-host. It turned out to be a harder goal to achieve than I expected.
I wanted to use GPT4-o to help me generate data for the fine-tuning. Sadly that was a bummer. It screwed me over so many times with garbage, I got exhausted.
Then I downloaded a bunch of high-rated stackexchange posts to be clustered and used for the fine-tuning.
This already took me more time than I wanted and realized that for a PoC I might as well use openai api. So did that and started experimenting with different prompts for GPT4-o mini.
In the meantime I got acquainted with the python based nostr-dvm framework and setup the basics I need for a DVM service.
After some grind I got everything working but still was not very pleased with the classification results.
Now, I know I could have put in even more time to really nail that prompt but I kinda lost my faith and appetite. I successfully use AI to learn and generate rudimentary code and chat about ideas. But what I needed is to generate a feed without manual intervention. AI people tend to recommend techniques to improve results that are borderline witchcraft. And every single time GPT finds a way to hallucinate plenty of stuff that I did not expect, no matter how hard I seem to try. This is my experience from months of daily interaction with GPT4-o too, which is much better than the mini version.
So no, I won't get trapped in the ai hype again. It is what it is: without human oversight these things are still worthless. I'm not aiming for a 90% usable thing, and I don't have a straightforward way to get to a 100%. No one has because these stochastic models are not AIs at all. They are mimicking parrots, nothing more. And this direction is a dead-end if you ask me.
All in all to customize a high-quality feed I tend to agree now that something like #nostrscript seems to be the better way to go from jb55 (nprofile…evy3) .
It has the human element but is enhanced with the right tech to be much more than just picking hashtags to follow.
We might see a bunch of better use-cases for DVM-s but this is my overall sentiment right now.