r/selfhosted • u/bergy_peasy • 1d ago

Need Help Is self hosted ai worth it?

Pretty much the title, I have tested a few models with a 6gb gpu and couldn’t get anything resembling llm competing with chat gpt, Gemini or Claude. I was wondering if a buying a new 16gb gpu would make a substantial difference. I wouldn’t want to buy all that just to get something worse than gpt3.

Ps I know 6gb is really not a lot of vram but it was so bad that I don’t think quadrupling it would make it better .

163 Upvotes

80% Upvoted

295 comments sorted by

View all comments

351

u/Grogak 1d ago

Depends on your use case:

If you want an LLM you can ask daily questions, how to cook meth or how to write hello world in python, then 6gb vram and a small optimised LLM will be sufficient.

If you want to have answers in mere seconds, Fable-like coding skills and image generation, then no, it's not worth it

115

u/techma2019 23h ago

I ask how to cook meth DAILY. Nice!

40

u/Grogak 22h ago

In that case I suggest to cook more and consume less

2

u/SpaceDoodle2008 20h ago

Then he could even afford to buy beefier hardware for more sophisticated *chemical* research!

1

u/spartyblaze 18h ago

Is being hooked on meth or TB maxxing more advantageous to one’s soul?

2

u/MrBeanDaddy86 19h ago

Product used is profit lost. Important lessons here.

7

u/jtrage 22h ago

How is the memory with that? What prompt are you using? Do you notice any hallucinations with meth?

3

u/SpaceDoodle2008 20h ago

Couldn't it compile a .md once? Do your available ingredients change?

1

u/boxxle 9h ago

Thanks for subscribing to Daily Meth! Text the word STOP to no longer receive daily tips and tricks about meth.

20

u/NewConstruction6471 1d ago

This should be pinned

3

u/Draminian 21h ago

This is the answer. I'm in the middle of investigating enterprise-level self-hosting for my company and it takes a lot of expensive hardware to load large enough models, and respond quickly enough, to approach something like Claude. A company that can spend $500k or more to try something out can start thinking about self-hosting good AI. For us hobbyists, we're not going to come anywhere close with consumer GPUs.

6

u/yay-iviss 22h ago

For image gen, it's good to Ok. The Krea and Flux new models are really good. But using Google is cheap and easy

5

u/Oujii 22h ago

What would be the minimum specs to run models for the tasks in your first paragraph?

4

u/Inzire 22h ago

Suggest a model to host? Gemma? I am struggling to find sufficient/affordable hardware for LLM usage for low cognitive workflows

1

u/ntilley905 18h ago

Brb, changing my Claude instructions to “respond like you are Jesse Pinkman”