When you use a remote blockstore like they are as your storage, you also generally are always booting via PXE, and so far have a provider controlled kernel.
In OnMetal this not the case -- you get local storage, you have your local kernel that you can re-install or do whatever you want to -- and if we have a control plane outage, you can reboot -- there is no dependency once your server is created on a provider hosted PXE server.
OnMetal isn't cheapo-ARM servers, but it is possible to build a "bare metal" cloud, which is truly bare metal, like you would normally get, and no remote block stores for your root FS.
OpenStack Ironic[1] can do this today, given you had right hardware. Making that hardware "mutli-tenant" is not a trivial project, mostly because the firmware stacks are still very closed, but it can be done. Top of Rack switches are also another area of complexity/pain, but also feasible to overcome with a little engineering.
[1]: OpenStack has its complexities, but the OnMetal project has upstreamed everything feasible we can.
True; I think it is reasonable project to take on if you are willing to staff developers... if you are trying to build a "bare metal" public cloud, having developers on staff is going to be a prerequisite for quite some time going forward.
