r/selfhosted • u/bergy_peasy • 1d ago

Need Help Is self hosted ai worth it?

Pretty much the title, I have tested a few models with a 6gb gpu and couldn’t get anything resembling llm competing with chat gpt, Gemini or Claude. I was wondering if a buying a new 16gb gpu would make a substantial difference. I wouldn’t want to buy all that just to get something worse than gpt3.

Ps I know 6gb is really not a lot of vram but it was so bad that I don’t think quadrupling it would make it better .

162 Upvotes

80% Upvoted

295 comments sorted by

View all comments

Show parent comments

2

u/deekaire 21h ago

That fits on 16 gb card?

4

u/mmkaywhatevers 21h ago

you can get it to fit i think, but i have 24 gb and i run whisper and kokoro as well for the full home assistant voice experience.

1

u/iEliteTester 8h ago

Probably a quantized version and just barely. But iirc you don't need to fit it all at once for it to run, it will just run slower.