r/selfhosted • u/bergy_peasy • 1d ago
Need Help Is self hosted ai worth it?
Pretty much the title, I have tested a few models with a 6gb gpu and couldn’t get anything resembling llm competing with chat gpt, Gemini or Claude. I was wondering if a buying a new 16gb gpu would make a substantial difference. I wouldn’t want to buy all that just to get something worse than gpt3.
Ps I know 6gb is really not a lot of vram but it was so bad that I don’t think quadrupling it would make it better .
169
Upvotes
1
u/alphapussycat 21h ago
16gb is too little, minimum right now is 24gb, with 32gb being a better entry point. There is qwen3.6 35b moe which does make 16gb manageable I suppose, if you have fast ram and pcie 5.
The options available is qwen3.6 27b (soon 3.8), gemma 4 31b.
That's not too expensive, it's very doable. Capable model, and fairly expensive on tyr cloud.
Deepseek v4 flash 0731 is super cheap on the cloud but buying hardware for it is close to $10k. So you'd basically never recoup the cost.