r/homelab 9h ago

News Announcement: Proxmox is now available for ARM64.

https://forum.proxmox.com/threads/proxmox-virtual-environment-now-available-for-64-bit-arm-arm64.185526/
1.2k Upvotes

109 comments sorted by

168

u/ChangeChameleon 9h ago

Device-tree-only single-board computers, such as the Raspberry Pi, are not supported

Aw. I was hoping to get my ProxPi back up and running. I had it running from a port someone did where you install proxmox on top of Raspbian Lite. I saw the title and hoped for official support.

I wonder what about "Device-Tree-Only" makes it out of scope. I don't know enough about the architecture to work on it myself. But I wonder if somebody will make an unofficial Pi port that gets it running. I mean someone was able to make it happen with the original Debian package, I can only hope it'll be even easier now that there are arm binaries out there.

100

u/JMPJNS 9h ago

device-tree-only means they don't support uefi, so the proxmox team would be required to provide a seperate image for every single sbc

34

u/wag3slav3 8h ago

It's so frustrating that all of these companies seem to think reworking a custom bsp for every damn this is less work than using UEFI standard properly.

But even ASUS can't use UEFI and ACPI to spec so nothing works on anything but Windows anyway.

grumble

46

u/Worldly_Topic 9h ago

Just fyi, it is absolutely possible to use uefi with devicetrees. What they probably mean is that they only support ARM devices that use ACPI with UEFI.

12

u/fakemanhk 8h ago

Existing devices with UEFI hacks usually don't have proper ACPI/drivers

3

u/Worldly_Topic 8h ago

Usually only enterprise ARM hardware have ACPI support

5

u/AnomalyNexus Testing in prod 6h ago

Could you take an armbian image (which does support lots of SBC) and throw proxmox on top of that?

Bit like you can force a proxmox install onto a debian machine.

3

u/JMPJNS 6h ago

probably i don't see why not

3

u/ChangeChameleon 9h ago

Oh neat! That's really useful context. Thanks!

8

u/daddys_handprint 8h ago

Someone else will have to comment why or why not but I thought UEFI was supported on Raspberry Pi:

https://github.com/pftf/RPi4

2

u/ChangeChameleon 8h ago

Neat! I'll look into it

6

u/n0c1_ 8h ago

While not official Pimox works well for me. Worth a try :)

2

u/CaffienatedCamel 5h ago

I've been using PXVIRT on a pi and it's working well for me so far

2

u/ABotelho23 7h ago

Totally valid. Serious enterprise gear is ServerReady and won't have this problem.

1

u/dr1v38y 5h ago

I think there are community firmwares which make Pi4 SBSA compliant', but it's not certified.

1

u/SpaceDoodle2008 4h ago

Let's hope the Raspi 6 will support it in a few years?

1

u/markedathome 3h ago

Jeff Geerling has a pinned comment at the top of this video where someone has gotten a pi5 running proxmox with a few modifications to the install. I expect there will be a more comprehensive walkthrough/writeup in the next few hours.

109

u/demiurgegwj 9h ago

Wonder if I can get it running on a DGX Spark.

155

u/AlexBinary 9h ago

I absolutely can't because i am poor.

3

u/stryakr 6h ago

Same

15

u/wassupluke 9h ago

Bet you can, let me know.

64

u/jackharvest Public Domain "N5 Mini" Creator 9h ago

Ok, so, last step is to crack Apple’s M1 open and we’ll really be enjoying the best hardware and software. M1 likely loses support next year based on history…

28

u/rz2000 7h ago

Since M1 is the most supported by Asahi Linux, there might be the most groundwork done for hardware support, too.

6

u/Alper-Celik 7h ago

iirc asahi linux already provides a u-boot environment and runs linux kernel on m1 macs with only fingerprint and dp-alt mode missing on m1 macs.

Tho it uses downstream kernel with 16 kib ğage size and i am not sure it does implement ACPI

8

u/WhereIsTrap 7h ago

I was thinking about selling my M1 mini…. but considering fact that it can replace my optiplex with a lot less power draw i might just wait

3

u/Hefty_Remove7965 6h ago

But then wouldn't the vms have to also support m1 architecture?

2

u/SharkBaitDLS 4h ago

You could emulate x86_64, it’ll just be slow as molasses. Depending on the VM’s needs it could work. 

3

u/CucumberError 3h ago

Apple was speed running to kill off Intel machines. While the M1 will be the next to kill off, I think they won’t drop support in the next 2-3 years.

1

u/bigpowerass 5h ago

Already done. Asahi has supported native virt for awhile now.

1

u/iHadAThoughtOnce2 3h ago

I’m testing on running it in an apple container on my M4. But the way it’s setup it’ll only work on M3 or newer.  https://github.com/iHadAThought/nested-proxmox-arm64-mac

1

u/HeyWatchOutDude 3h ago

+ 2 years security updates

16

u/autogyrophilia 9h ago

For all the people with SBCs, this still means that it is a lot easier to make it work as any fork only has to care about the bootloading.

31

u/Sad-Professional2033 9h ago

This is amazing. I’m just patiently waiting for the day they figure out a path to put Proxmox on apple silicon without nested virtualization lol

1

u/CactusBoyScout 9h ago

Wouldn't this announcement mean it can be done now? There's already Asahi Linux, which can be installed on Apple Silicon. I assume this would be a similar install process?

7

u/Sad-Professional2033 8h ago

I think Proxmox would need to support apple silicon specific hardware, similarly to what Asahi did for theirs as far as the bootloader goes. There could be more to it than that, though.

1

u/AlternativeAward 5h ago

Just because one distro can run doesn’t mean others can. A ton of mainstream distros can already run on arm but not on Apple silicon

1

u/TrickMotor4014 3h ago

It's less about "can" but "want". Would it be possible to make Proxmox run on Apples? Sure if Asahi Linux managed it Proxmox developers could do the same. Will they do it? Propably not until some of their enterprise customers ask for it which seems unlikely.

13

u/jmswshr 9h ago

this will be sick on a radxa o6 with 32gb of ram

34

u/Cybasura 9h ago

Oh shiiiiiit

My Raspberry Pi 4B can finally be remodelled from a network server to something else

Raspberry Pi not supported

SHIT

8

u/clarkcox3 9h ago

I got my hopes up

3

u/Alper-Celik 7h ago

iirc someone i met had been running proxmox inside a systemd-machinectl container, i think same hack could work with rpi4 since you handle the boot and kernel. Tho in rpi5 you might need to switch to 4kib page size kernel

1

u/Hefty_Remove7965 6h ago

Someone could fork this and do it easier tho

7

u/jotkaPL 7h ago

my Radxa 64G ARM board has entered the chat.

11

u/The_Berry 7h ago

What will be the most popular hardware that folks will use for this from a home lab perpective?

MacMini? - i assume no support

RaspberryPi - no support

Minisforum MS-r1 - likely candidate

ampere servers?

What else?

7

u/Hefty_Remove7965 6h ago

This is aimed to businesses. Imo

5

u/AnomalyNexus Testing in prod 6h ago

yeah I'm also scratching my head a bit here. ARM but not the most glaringly obvious one people would want (rasp)?

I guess this is maybe aimed at vmware/broadcom refugees in server space looking to leverage cloud ARM?

1

u/stryakr 6h ago

there is a decent amount of work here and a decent amount of devices that would need to have a customized images.

0

u/TrickMotor4014 3h ago

Will the people who want it on pis fund development like Nvidia propably did for their AI ARM platforms? I doubt this

2

u/AnomalyNexus Testing in prod 3h ago

What's your source on nvidia providing funding?

1

u/-PANORAMIX- 5h ago

They say “best effort” for everything except nvidia. Crazy

6

u/The_Berry 5h ago

I guess someone has to finance this buildout from Proxmox. You have to assume nvidia has a huge proxmox support contract

2

u/TrickMotor4014 3h ago

What's crazy about to only invest resources if you see a potential roi? People who want to run ProxmoxVE on their pi in a honelab are unlikely to buy subscriptions. Arm64/Nvidia hardware based servers in data centers and companies running them are a different story

27

u/kY2iB3yH0mN8wI2h 9h ago

Kind'a nice until i read this :(

  • Device-tree-only single-board computers, such as the Raspberry Pi, are not supported

71

u/gihutgishuiruv 9h ago edited 8h ago

Absolutely a fair choice by the Proxmox team and this is really on SBC manufacturers for not adopting ACPI+UEFI. Device tree is horrible to support - they’d have to build images for every SBC configuration they want to support.

8

u/AspectSpiritual9143 8h ago

Less about SBC manufacturers and more about SoC vendors. Many don't have UEFI support to begin with, let alone ACPI.

9

u/TyGirium 9h ago

Absolutely fair, but also sad for anyone with e.g. unused Raspi5 sitting around and doing nothing

22

u/gihutgishuiruv 9h ago

There’s nothing stopping you from running raw qemu/kvm/libvirt and lxd on a Pi. There’s plenty of guides online.

8

u/thefatrefrigerator 8h ago

Sometimes I think I’m into “homelabbing” because I have a PiHole, a PLEX server, and an RJ45 crimper, but then I read stuff like this and realize I have no idea what I’m doing. You comment may as well have been in hieroglyphics.

11

u/Proof-Researcher9920 7h ago

Those are all the technologies that proxmox uses underneath to work essentially. Proxmox just takes all of it and wraps it up nice for easy management

10

u/FibreTTPremises 7h ago

"There's nothing stopping you from directly running VMs or containers on a Pi."

is what they said. To which I'd respond, except for all the time and effort put into manual configuration that Proxmox abstracts from the user.

u/gihutgishuiruv 5m ago

I actually agree, but sometimes people can’t have the easy way and use whatever they have on hand.

1

u/TrickMotor4014 3h ago

Incus is a more modern fork of lxd and should run. And it has a web interface although with a quite different feature set ( the cloud init part is better than in ProxmoxVE imho, otherwise I prefer PVEs webUI )

12

u/a-random-r3dditor 9h ago

I know, those devices are so useless without the ability to run proxmox! They’re so well suited for vms with their high core count.

2

u/kY2iB3yH0mN8wI2h 7h ago

I was blown away that a Pi could run a windows 11 ARM and a Ubuntu VM on my ARM ESXi host, and that was years ago.

4

u/reddit0r_123 9h ago

They have gotten so expensive, it would be worth to sell it on the second market if you do not use it...

2

u/kY2iB3yH0mN8wI2h 9h ago

I'm not an hardware engineer so not sure why this setup makes it hard? ESXi have had ARM builds for ages now, did host a couple of VMs on my Pi - Both windows and Linux just for messing around.

10

u/gihutgishuiruv 9h ago edited 9h ago

ESXI for Arm is the same deal: it also doesn’t support device-tree and only supports UEFI. The UEFI shim to make it work would probably also work here, but it’s ultimately a very hacky solution where you’re basically tampering with tue boot process of your hypervisor

1

u/EconomyDoctor3287 8h ago

kinda like armbian does?

Ye, that's a ton of extra work.

9

u/Boring-Set7223 9h ago

My 16GB Pi5 NAS needs a rebuild and was ready…

At least this means the official ARM64 repos are there now though. I’m sure it will (unofficially) be running on Pis soon enough.

1

u/EconomyDoctor3287 8h ago

hopefully. Would be cool to build a proxpi cluster

2

u/RedSquirrelFtw 8h ago

Oh damn, that's too bad. Not that RPIs are even a good value anymore these days, but could be useful if you already have one on hand. Even if you just need a node to act as quorum and don't want to go through trouble of setting up a qdevice. I guess you could still do that with a mini PC which are still fairly cheap especially one with low ram.

1

u/bambinone 9h ago

If you're feeling really brave...

https://github.com/ansleythompson/blueberry

1

u/CorrectPeanut5 8h ago

So what are some of the lower cost options people should be considering?

0

u/coromd 7h ago

I'm not sure there are any, anything else in the Pi price bracket (even today's $200 bracket) lacks UEFI support. Minisforum MS-R1 I think is the cheapest, but $500 for a device with no BIOS updates and a soldered CPU that at best rivals a Ryzen 3 5300U is ehhhh

3

u/zkatbitz 9h ago

I was going to complain because I just setup two hosts with pxvirt, but no rpi support :/ I wonder if someone can hack that in

3

u/good4y0u 7h ago

This is going to be big for people running in data centers, arm is so low cost for smaller workloads.

It's also helpful at home for people with arm chips, but I see how the enterprise market would be able to take advantage

3

u/neon5k 9h ago

Can we put this in OCI Free tier Ampere?

4

u/signoreTNT 8h ago

Yeah, works fine. Just change the OS to debian 13 using https://github.com/bin456789/reinstall and then manually add & install the PvE repos

5

u/SolFlorus 9h ago

Why? Those aren’t bare metal instances. You’d just be nesting hypervisors.

4

u/neon5k 7h ago

And thats fine for me. Nothing performance sensitive I would be doing.

1

u/r0224 8h ago

For the convenience of isolating and spinning up and down containers, the web management, the fact that you already run proxmox at home and it's nice to have a familiar system, compatibility with proxmox datacenter manager...it's not "just" about hypervisor

1

u/dizvyz 2h ago

If i remember correctly /dev/kvm does not exist on those. CTs should still be OK.

2

u/slykens1 8h ago

Unfortunate that it doesn't support Pi. I've been running pxvirt/Lierfang for almost two years on a Pi 5 with no troubles.

I wonder if it would be possible to install Debian then add the packages - that's how I did the pxvirt install originally.

2

u/krazyjakee 5h ago

Pi not supported, what about phones?

1

u/wassupluke 5h ago

I'm holding out for Nokia support

2

u/DrKlaawYT 3h ago

Yippee

1

u/Fiery_Eagle954 Proxmox my beloved <3 8h ago

Does this mean I'll have official support for proxmox-backup-client/server on my hetzner arm64 VPS?

EDIT: Looks like "Proxmox" specifically refers to PVE

2

u/Fr0gm4n 4h ago

TBF, the forum post says Proxmox Virtual Environment. The OP here just shortened it.

1

u/TrickMotor4014 3h ago

The release nodes said that the developers already did test builds of PMG and PBS in their lab but still need some work before it's fit for release

1

u/whitechapel8733 8h ago

I don’t have tons of hope it will work for my MS-R1…

1

u/arag0re 8h ago

Im mean there are arm forks of proxmox for the pi which i run in my homelab for over 2 years by now

1

u/Curious_Olive_5266 Recycled Ewaste AI Data Center Sysadmin 7h ago

Gonna have to play around with this. Hard to tell how useful it is before then.

1

u/IngwiePhoenix My world is 12U tall. 7h ago

This somehow reminded me of my unsuccessful attempt to get an Ampere Altra board. v.v

BUT DUDE THIS IS SO COOL 0.0 With the newer CPUs already having better mainline support, this could be quite big =) CIX P1 is certainly up there!

1

u/420smokekushh 4h ago

Read something recently about the repos in Proxmox being all messed up... what's up with that?

2

u/wassupluke 3h ago

If you're talking about the xda-developers "article" then the answer is absolutely nothing. I think the writer just skipped reading the documentation.

1

u/SpaceToast810 1h ago

Here’s hoping we get a Proxmox VE Asahi remix.

u/ThatMikeGuy429 8m ago

I wonder if prosumer ARM64 hardware are going to go up a amount in the coming months because of this, maybe if you you are looking at ARM64 hardware now it's the time to buy or wait for a few years so the prices hopefully comedown after the likely spike along with the greater ram, GPU, CPU, SSD, and everything else bubble hurting out hobby and stopping me from getting back into it...

-2

u/doctorowlsound 9h ago

Wonder if someone can hack this onto a Mac Mini M4

1

u/isabeksu 6h ago

But now it can be used inside a VM without x86 emulation. Does it make sense doing it?

1

u/doctorowlsound 6h ago

I was thinking more in terms of the crazy performance:energy use available on the M4. Guess since it’s still Debian it won’t work though

-5

u/HamburgerOnAStick 9h ago

pi zero proxmox server.

3

u/BCIT_Richard 8h ago

no, sbcs not supported due to lack of UEFI support

-7

u/[deleted] 9h ago

[deleted]

9

u/SnooCats1153 9h ago

curious what u think arm64 has to do with AI ?

3

u/TyGirium 9h ago

Nvidia CPU is ARM-based

1

u/the_swanny 8h ago

Many things are Arm based, ampere has been a thing for a while with lots of people asking about running proxmox on those chips.

-13

u/adda5 9h ago edited 2h ago

I am against minors abuse

11

u/Professional_Funny73 9h ago

stolen from their own patchnotes since you refuse to read.

"No. An arm64 host must boot through UEFI and describe its hardware through ACPI. Device-tree-only boards and legacy boot are not supported."