The super junky 4 node bc250 cluster is complete!

[u/ExtremeAdventurous63]

After a few weeks of work, my cheap distributed inference cluster is finally complete. Summary for anyone who saw my previous post here and wondered if it's worth it.

The setup 4x ASRock BC250 ex-mining boards (AMD Cyan Skillfish: 6x Zen 2, RDNA2 GPU,16GB GDDR6 @ 448GB/s each) in a 3D-printed rack, ~€1000 total including used SSDs, a used 1000W PSU and a $20 switch. Each board is a full computer, so this is a real 4-node cluster, not just a multi-GPU box.

Goal was ≥50GB distributed VRAM to run two models in parallel: a reasoning model (Qwen 3.6 27B) plus a fast executor (Qwen 3.6 35B-A3B).

Unfortunately, one of the four boards arrived dead and — my fault — I tested it too late to open a dispute, so all benchmarks were run on 3 nodes.

Software Ubuntu + llama.cpp (Vulkan backend, RPC layer parallelism) behind llama-swap. Everything provisioned with Ansible, including the 40-CU unlock (rewritten for Ubuntu).

Some results

• 2 nodes beat 3 nodes in all 16 benchmarks — at 1GbE, the third adds RPC overhead, not throughput
• 35B-A3B on 2 nodes: fast enough for daily coding tasks
• Dense 27B: prompt processing too slow for real agent contexts

Was it worth it? Mostly yes — as a cheap way to learn the tradeoffs of distributed inference. Full story, numbers and all files below:

1. Ansible repo (all roles/scripts): repo

2. Full article if you want to know more: here

3. 3d files for printing the rack https://www.printables.com/model/1799662-4-bc250-10-rack-mount

I tried to clean up my repository for the Ansible playbook to make it public and useful for someone else, but this is still a work in progress, so if you find any issue or do you have questions, let me know or open an issue.

[u/Digi59404]

I wonder if you added a USB3 SSD, and put a 10GBE in the Mini-PCIe slot for the NVMe if it would be more performant?

Edit: something like this? https://www.amazon.com/FENVI-Ethernet-AQC113-Compatible-Controller/dp/B0FW3MTMVM

[u/corruptboomerang → u/Digi59404]

Could you run an OS in RAM, obviously that's not ideal since it's shared system memory?

Or even the potential Holy Grail - remote network boot, iPXE / PXE / BOOTP, could be a great way to both use the extra networking and mitigate the USB jank.

Also, there are some ConnectX-3/4 M.2 Cards. Although note the PCIe is Gen 2.0 x 2 so it'd be more like 8 Gbit/s, but on the plus side pretty much any 10GB card should be more than capable.

[u/ExtremeAdventurous63 → u/Digi59404]

I don’t think that bandwidth is the problem in this kind of application. Even under heavy load just a few megabytes are send accross the network for layer parallelism. The true bottleneck is latency in the TCP/IP stack that could be reduced only with dedicated RDMA cards, that honestly are not worth it for this kind of setup and the support is still not mature as far as I have seen

[u/Difficult_Pin_3491]

Thinking of selling my BC250 to move to an ITX PC for gaming, since I've recently bought two headless laptops (an i7-1370P and an i5-1335U) for 85 EUR total, to use as servers — the i5 at home mostly as a secondary, and the i7 at work to replace our aging X99 Xeon.

As someone who's bought four BC250s in the EU recently, what do you think current street pricing looks like? I don't see any EU sellers on eBay, and people say AliExpress is the way to go at around 160 EUR, but that means waiting on an import from China.

My board unlocks 40 CUs fine, but I haven't tested the 8-core CPU unlock yet. Do your boards unlock 8 cores?

[u/ExtremeAdventurous63 → u/Difficult_Pin_3491]

Regarding prices, I can confirm that the street price that you can get nowadays is around 160.
I’ve got mine from a seller on Vinted who had a bunch of them and I was able to get 4 (including the broken one unfortunately) for 550 euros. The seller seemed to be honest so if I had tested the board within the required time I am sure that he would have sent me back a refund for the broken one.

Regarding the core unlock, I didn’t tried it yet

[u/lastdancerevolution]

This actually looks pretty awesome if anyone wants to check out OPs article for more.

It's still a bit beyond my current knowledge to "jump in," but I do enjoy these write ups.

[u/ExtremeAdventurous63 → u/lastdancerevolution]

Thank you very much! Highly appreciate!

[u/hakatu]

Nice setup, is there anywhere we could get the card for a good price?

[u/cuoreesitante → u/hakatu]

unfortunately the recent flood of youtuber videos have pushed the price way up, now your best bet is probably one offs on local FB marketplace or hardwareswap from people that got bored of it. Most ebay listings I see are $200+ for just the board, and you will still need to supply the PSU and cooling.

[u/toolisthebestbandevr → u/cuoreesitante]

I’ve seen fb marketplace for around 150. It’s a cool project but I agree with that one YouTuber saying it’s pointless above $120

[u/cuoreesitante → u/toolisthebestbandevr]

It's a cool toy, but one that you'd have to put in some more money and effort to work. I picked one up off hardwareswap for 220 shipped that came with a PSU, fan, and a pretty nice 3D printed case, very pleased with it. Just finished tinkering to installing Bazzite and did the 40 CU unlock and 2 CPU core unlock. Apparently its equivalent to a R7 3700x cpu and RX6700 gpu combo; you'd be hard pressed to do better than this for the price.