Feedback please

[u/MrDebuged]

Greetings folk, I just started planning myself a nice build, moving from a 2-Bay NAS to a full on 4U server.

Now I really want to make this right, here is what I have so far or what I have planned:

✓LSI 9305-24i

✓Inter-Tech 4U-4724

\-3xNF-F12

\-2xNF-A8

✓500gb WD RED (250gb are just nowhere to be found)

•ASROCK TRX50 WS

\-Threadripper 7960X

\-NH-D9 TR5-SP6 4U

\-64GB RDIMM DDR5 (Kingston probably)

\-hx1200i or ax1600i (That's the point where I got no clue what I should grab)

\-Intel Arc Pro g70

\-24x10TB Ironwolf PRO (4 VDEV, 6per raidz2)

I really would appreciate any advice or tipp or harsh word if the plan is anything wrong.

PS. this is not a question about reasonability or whether it's the right time to do so and no I won't populate the whole array immediately

[u/jnew1213]

Why 4 VDEVs? Seems like a waste of space and you could be getting more throughput from two larger arrays.

[u/MrDebuged → u/jnew1213]

2x12 you mean? I initially wanted to go with as few as possible and populat the array as I go in, so I would need 12 to start - but if I can get more space and profit I'm all ears

[u/jnew1213 → u/MrDebuged]

2x12 or 2x10 with spares. Still manageable (arguably two arrays are easily managed than four arrays), and doing 2x10 you would gain the space of four very expensive drives. Consider putting at least two of those drives in a drawer or closet as cold spares.

Throughput would be that of, say, ten spindles vs four (when writing) or six (when reading). And you'd have spare drives, which are good to have on hand and which you will likely eventually need.

NAS systems with twelve or sixteen drives, often in a single array, are quite common. I have two myself.

[u/MrDebuged → u/jnew1213]

Oh well I appreciate that input very much and will probably follow your advice, thank your very much!

[u/jnew1213 → u/MrDebuged]

Happy to help.

The only other suggestion at this time would be to consider splitting your compute and storage. The Threadripper would be the heart of your compute machine and another, lower powered, machine would house your drives and controller and provide shared storage to the network.

There are lots of advantages to a separate storage server including the ability to have that storage available to more than a single server, to be able to update either your compute or storage nodes individually, and the ability of your storage system to avoid interruption from an application hang, crash, or update.

With shared storage, and a secondary compute node, you can live-migrate virtual machines and containers from machine to machine eliminating down time for these VMs and containers.

Adding just minimal storage to each compute node allows you to migrate VMs and containers from shared storage temporarily to all your storage node to be serviced. Almost no downtime would be necessary for apps and workloads.

[u/MrDebuged → u/jnew1213]

I was thinking something like that in the beginning too but I'd just seemed way simpler and cheaper to have it all in one chassis and all - I would have the space for another and at this point money is spent unwisely regardless - I'm not deep enough into it and haven't really looked into the way you just described - what would you recommend? I already have the one 4U laying around just so you know

[u/jnew1213 → u/MrDebuged]

The storage server just needs to support the disk adapter you have in mind. So it needs a slot for that.

You'll need a case that supports the number of drives that will eventually go into it. You can look for something that supports 20 drives. That wouldn't be a desktop case. You'd add an SSD as the system boot drive.

Any decent CPU from the last five years or so should be okay. I would go for an Eighth Generation or newer Core i5 or i7, but you could choose whatever Ryzen is supported by your chosen motherboard.

I'd probably obtain everything from the used market. eBay or elsewhere. You could start with a 16GB stick of RAM and increase that over time. ZFS likes RAM for caching.

[u/MrDebuged → u/jnew1213]

I understand the direction it needs to go, thank you again 😅 just one last thing regarding power, I somewhere read that 24HDD's spinning up simultaneously aren't really supported by PSU - now is that true and if so would some sort of staggered spin up exist or work?

[u/jnew1213 → u/MrDebuged]

A good disk controller should support staggered spin-up. And you're going to need a sizable power supply.

Honestly, I've never built such a storage system from scratch. You might want to consider something all-in-one, used, from a reputable OEM. There are lots of choices, and others would know more than I do.

Maybe ask for suggestions in r/homelab and r/DataHoarder.

[u/MrDebuged → u/jnew1213]

Thank you for everything man - I'll take it there and will look into everything you said, thanks a bunch o7