This is a project to compile the necessary information to create a personal micro datacenter. This is largely for educational purposes, but can also be used for a personal lab or hosting personal services (website, files, etc)
This is largely opinion, but for the purposes of this project, we'll define it as the following.
A micro datacenter is a fully functioning, ultra low power, and distributed datacenter. It contains all the functionality of a real datacenter, but small enough that you can run it off a 20A 110V outlet.
Also, it's bare metal...
But why?
Most folks, myself included, have always run some form of lab at home. This comes with a host of challenges and annoyances.
- Enormous power requirements for anything more than 1/2 a rack of servers
- Expensive, power hungry servers
- Old servers that are out of support, clunky, old, etc
- Overloading those servers with VMs
- Many folks run VMWare ESX, but that can be pricey
- Upgrades? What's that?
- Who actually has the funds to regularly ugprade their lab?
In this draft design, I'll lay out the rack design and hardware necessary to power dozens of servers.
- Mobile 1/2 cabinet (or 1/4 cabinet if you want to do a smaller version)
- 2x 6U DIN Mount with cable management
- Raspberry Pi 4
- Raspberry Pi PoE HAT
- POE Switch (Ex. NETGEAR 24 Port POE Switch)
- Raspberry Pi DIN Rail Mount
- USB to SATA Cable
- 2.5inch SSD
Do you need to do 1/2 a cabinet? No. You could do this smaller. But for this example, I'll do 1/2 a cabinet with room for things like battery backup / UPS
- 1/2 Cabinet: $374.00
- 6U DIN Rack/Cable Mgmt: $199.00
Total: $574.00
The basic host configuration is as follows
- PoE Powered Raspberry Pi
- Additional USB SSD disk for your workloads
- Vertically mounted on DIN rails
- No case required - better airflow and cooling
Cost
- SSD: 54$
- Raspberry Pi 4 (8G): 75$
- DIN Mounts for both: 15$
- Network Port (Switch / Num Ports): $ 11.45 per port (1)
- POE Hat: 20$
Total: $175.45
Could you get used gear for this cheap? Yes. Would it be more powerful? Yes. But...
- It's bulky as hell (rackmount or old desktop)
- Power hungry
- Is not expandible
- Doesn't grow past the built in number of SATA ports
With the above BOM (Bill of Materials) we'll be able to get roughly 18 servers in to 6U. This assumes 2" width per server. If you don't do SSDs, you can go denser.
Basically you'd need a 24port switch for every 6U.
So in 7U, you'd have the switch, and 18 servers, with cable management.
In that 22U cabinet, you could fit 54 servers.... now we're talking.
FRRouting
So you want to learn IPv6? Well, you could get your own ASN and IP space for about $1000.
- ASN: 500$
- /36 IPv6: 500$
/36 is the smallest ARIN will allocate at the time of writing
Seems a little steep for most home SREs. But imagine all the money you'll save in not buying rackmount servers or powering them.
With the routing setup and OpenSwitch, you can advertise your IP space through a provider like Equinix Metal (ask me how I know). You'll be able to have your own publicly routable IPv6 space. You can also use AWS BYOIP and TransitGateway Connect
If you're more adventurous, have a bottomless wallet, or already have IPv4 space, you can do that too.
You can use either wireguard or IPSec up to an Equinix Metal host or to AWS (or any other provider that can handle BGP)
FRRouting with /32s
nftables, bpfilter, iptables, etc
MinIO
Rack Mounted DIY Battery Packs