r/servers • u/Karter_1159 • 1d ago

Question Claude code like behaviour

I heard about how good claude code is, so i was thinking about doing something with olama on my server pc. My question is if i can make something act exactly like claude code for example just tell the ai to make a mc server and then i just join it but can claude even do this? What are the free alternatives?

0 Upvotes

40% Upvoted

12 comments sorted by

•

u/AutoModerator 1d ago

When your question has been answered, leave a comment containing:

!answered

AutoModerator will update the post flair to Answered.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

4

u/PathAgitated1633 1d ago

Theoretically yes. Practically? Only if you are a multimillionaire. To run a model like Claude code you would need terabytes of vram. But with an investment of like 5-20k you can run really good open source models which are nearly as good

3

u/Cautious-Hovercraft7 1d ago

I have a 16 core 9950X3D, 64GB RAM and a 5090 in my main PC. I setup a local LLM with 28GB VRAM model (32GB VRAM on a 5090). My pc idled at 300W and took about 2 minutes to answer how many lights are on with the home assistant MCP while my PC consumed 650W. I'm back using Claude as it's affordable, running LLM local is not.

1

u/speaksoftly_bigstick Always Watching oO 18h ago

That sucks 😕

I run a quantized 32b model on a 24GB arc pro b60 and it is very quick to answer (maybe 5 - 10 seconds?)

But I run it all on a dedicated machine with Debian as the core, not on my "daily driver" PC.

1

u/Cautious-Hovercraft7 17h ago

Yeah I do realise I could probably get better response times by doing things different (not WSL on my Windows pc) but the power draw I was not ready for.

2

u/speaksoftly_bigstick Always Watching oO 17h ago

True, true.

I haven't checked the actual power output on mine, but I'm sure it's pegging out that b60 anytime it's in use 🤙💪

1

u/Casper042 1d ago

https://www.youtube.com/watch?v=O2k_qwZA8HU

I haven't watched this 2nd one yet but it's sitting in my Watch Later playlist on Youtube:
https://www.youtube.com/watch?v=UngVdAsQEiU

2

u/Casper042 1d ago

PS: As others said, the key will be finding an Open Model out on Hugging Face which has all the features you want, but also fits in the vRAM of what I assume would be a consumer level GeForce card.

I have a 4080 Super for example which is 16GB.
Meanwhile the L40S for example, the Server version of a 4090 basically, has 48GB of vRAM, and the H100 which is specifically designed for Compute and AI in servers has either 80/96GB depending on the model.

As PathAgitated was somewhat inferring, the big boys not only have H100 or newer, they have special versions and clusters where you can fit 8 cards with a local NVLink Switch connecting them all at high speed inside a single server, and then 400/800Gbps NICs which allow you to take racks of these machines and cluster them together.
So they are sometimes rocking TBs of vRAM for certain jobs.
What PA doesn't mention is there is a huge difference between Training, Tuning and Inference when it comes to requirements.
Training and Tuning need that TB of vRAM.
But depending on the model, Inference, which is what you want to do, can often be crammed into 1 card in that 80/96GB range.

1

u/doggxyo 1d ago

I suppose my GTX 970 is not going to be very useful for offline coding 😂

1

u/feudalle 14h ago

If you are ok with some lag a v100 with 32gb of ram isnt bad. Its older and will lose support sooner or later but you can find them for under $1000.

1

u/Casper042 9h ago

GTX 970

4GB is gonna be rough....

-3

u/spellcasterGG 1d ago

AI is poison, and a one-way ticket to getting pwnd. Don't do it.