> When I joined Oxide last year, one of the first things I was introduced to was our racklettes; that is, smaller than average racks of four sleds and two switches each. These racklettes can be initialized and run just like a real rack, precisely because they are a real rack. Just not in any form factor it would be economical to sell.
Probably shouldn't have mentioned this publicly ;) Every time an Oxide post comes up, there's people in the comments expressing fervent wishes for just this sort of thing. Now that they know it exists, I imagine the the voices asking to make it available for developers and homelabs will grow even louder.
They mentioned this on the podcast, the problem is that since they're development tools the networking stack is still scaled for a full rack with pretty beefy switches.
Either they need to support a whole second network stack with firmware etc which they don't want to do now, or a racklette would cost way too much and almost nobody would pay the price.
I court see a lot of small businesses wanting one. Or as an addon to their existing Colo setup.
Could also be great for DR sites and smaller remote offices. Power is a big factor in deployments and I can see a smaller little turnkey setup that uses less be very popular
Ha! Even a racklette might be a smidge too big for the homelab. But you know, given these Voxel 'sleds' are ~6GB RAM each, we could probably get a solid mini-PC farm going now....
FTA: At home, I run Voxel labs on a Minisforum MS-A2 mini-PC with a 32-thread AMD Ryzen 9 9955HX and 96GB RAM.
I have 128 gb dd4 system sitting at home - Emulation (mostly of clients nonsense) has been massively insightful over the last few years. And as a dev, being able to do all my work "not on my laptop" has been a major blessing.
When prices normalize build a couple bulky machines and a have a stack of small and cheap (I have a bunch of 8th gen i5's I got used). You can do a lot (and give yourself room to grow/upgrade) on a pretty tight budget.
Just out of curiosity, did you do this for the kick of it or did you find the stack to actually be nice to use? Not that it takes much to convince me to nuke my homelab for the 10th time, but given the nature of Oxide, I'd assume they are running pretty reliable software (very willing to be wrong here).
I don't want to speak for swherdman, but I don't know that you'd want to nuke anything just to use this. If you want to learn about the Oxide stack, go for it and it'll be a great teacher. But if you're just looking for a better way to run your homelab, I don't know that this would be it.
Of course, I say this as someone running a Nucleo STM32H753ZI board with Hubris as an SP for my home NAS....
Hey that's really interesting, thanks for the share! It looks like this is using the standalone Omicron build, but on a Proxmox VM, right? Great stuff for running the CP in easy mode!
Basically yea, there are lots of projects that are "open source" you cant actually build because some components are missing, its great to play around with if you want to try integrating with the stack but i think its much easier to get access to the platform now than when i build this.
I also got it working nested in WSL but never wiring up the process
Fun to see the emulation heading up the stack, I wish my employer could afford some real oxide racks!
About 4 months ago, I booted helios + omicron on bare metal on a server in my homelab (following the YouTube here: https://www.youtube.com/watch?v=qwXtS0-r1uQ). Performance wasn't great on my old hardware (32cores, dual Xeon 2.6ghz, 256gb DDR4), and it used like 30% CPU while idle, but it was fun poking around and playing with the software while I could.
Yeah that's all I could tell at the time. Things were still snappy enough at my low usage, so I just assumed it was my older hardware being a bear. But it was definitely bumping up my power draw unfortunately.
Like 3/4 of my career was building stuff like this along with the software that ran in it for basically this exact use case. We even had it integrated into our CI. Spent upwards of a decade building things like this.
One thing we realized was that we needed to be able to simulate upgrades like this.
Lead to a lot of complexity but it was worth it.
This testing was some of the most valuable and because we had it automated it really let us build better things faster and find issues and regressions and verify new functionality :)
> When I joined Oxide last year, one of the first things I was introduced to was our racklettes; that is, smaller than average racks of four sleds and two switches each. These racklettes can be initialized and run just like a real rack, precisely because they are a real rack. Just not in any form factor it would be economical to sell.
Probably shouldn't have mentioned this publicly ;) Every time an Oxide post comes up, there's people in the comments expressing fervent wishes for just this sort of thing. Now that they know it exists, I imagine the the voices asking to make it available for developers and homelabs will grow even louder.
They mentioned this on the podcast, the problem is that since they're development tools the networking stack is still scaled for a full rack with pretty beefy switches.
Either they need to support a whole second network stack with firmware etc which they don't want to do now, or a racklette would cost way too much and almost nobody would pay the price.
I court see a lot of small businesses wanting one. Or as an addon to their existing Colo setup.
Could also be great for DR sites and smaller remote offices. Power is a big factor in deployments and I can see a smaller little turnkey setup that uses less be very popular
Ha! Even a racklette might be a smidge too big for the homelab. But you know, given these Voxel 'sleds' are ~6GB RAM each, we could probably get a solid mini-PC farm going now....
FTA: At home, I run Voxel labs on a Minisforum MS-A2 mini-PC with a 32-thread AMD Ryzen 9 9955HX and 96GB RAM.
I have 128 gb dd4 system sitting at home - Emulation (mostly of clients nonsense) has been massively insightful over the last few years. And as a dev, being able to do all my work "not on my laptop" has been a major blessing.
When prices normalize build a couple bulky machines and a have a stack of small and cheap (I have a bunch of 8th gen i5's I got used). You can do a lot (and give yourself room to grow/upgrade) on a pretty tight budget.
[dead]
If you just want to try emulating the virtualization stack i build this about 6-9 months ago https://github.com/swherdman/whoah-cli
Steps to build if you prefer it manually: https://swherdman.com/blog/We-Have-Oxide-at-Home-P1/ https://swherdman.com/blog/We-Have-Oxide-at-Home-P2/
Just out of curiosity, did you do this for the kick of it or did you find the stack to actually be nice to use? Not that it takes much to convince me to nuke my homelab for the 10th time, but given the nature of Oxide, I'd assume they are running pretty reliable software (very willing to be wrong here).
I don't want to speak for swherdman, but I don't know that you'd want to nuke anything just to use this. If you want to learn about the Oxide stack, go for it and it'll be a great teacher. But if you're just looking for a better way to run your homelab, I don't know that this would be it.
Of course, I say this as someone running a Nucleo STM32H753ZI board with Hubris as an SP for my home NAS....
100% don't use this. Its much more of a because i could do it rather than you should.
Hey that's really interesting, thanks for the share! It looks like this is using the standalone Omicron build, but on a Proxmox VM, right? Great stuff for running the CP in easy mode!
Basically yea, there are lots of projects that are "open source" you cant actually build because some components are missing, its great to play around with if you want to try integrating with the stack but i think its much easier to get access to the platform now than when i build this.
I also got it working nested in WSL but never wiring up the process
Fun to see the emulation heading up the stack, I wish my employer could afford some real oxide racks!
About 4 months ago, I booted helios + omicron on bare metal on a server in my homelab (following the YouTube here: https://www.youtube.com/watch?v=qwXtS0-r1uQ). Performance wasn't great on my old hardware (32cores, dual Xeon 2.6ghz, 256gb DDR4), and it used like 30% CPU while idle, but it was fun poking around and playing with the software while I could.
30% is surprising, did you dig into it at all? Sounds like a bunch of active polling or something?
Yeah that's all I could tell at the time. Things were still snappy enough at my low usage, so I just assumed it was my older hardware being a bear. But it was definitely bumping up my power draw unfortunately.
Like 3/4 of my career was building stuff like this along with the software that ran in it for basically this exact use case. We even had it integrated into our CI. Spent upwards of a decade building things like this.
One thing we realized was that we needed to be able to simulate upgrades like this.
Lead to a lot of complexity but it was worth it.
This testing was some of the most valuable and because we had it automated it really let us build better things faster and find issues and regressions and verify new functionality :)
Oxide is doing such fun stuff, really wish I had the skill set needed for their job postings.
Oxide racks would be very cool with those 245TB ssds and VAST data style software.
[flagged]