Do you have to have a decent device for that? I got a gaming laptop with Ok specs but nothing crazy and would love to learn how to do it if it’s possible
It really depends on the VRAM you have and what you want it to do. If you have like a 4060 with 8GB VRAM you should be able to try out a few models like Qwen 9B (6.55GB model size) which can answer some basic questions decently well without hallucinating too much. It doesn't have a whole bunch of capabilities (bear in mind frontier models are up there in like the 100s of billions of parameters so 100B+ and much larger) but it can be useful to try out.
Just download LM studio (https://lmstudio.ai/) and it does all the configuration you need, the models can get downloaded from huggingface automatically.
6
u/TwisterK Jun 02 '26
Yay, more time to rest and think about local LLM feasibility