Datacenter at Home

Embarking on a homelab journey often starts with a spark of curiosity and a dash of ambition. At first, my idea was to host my own NAS so that I could unmarry from the cloud; however, this quickly evolved into a multi-server rack with enterprise-level configuration and security. This transformation wasn't just about growth in scale; it was about creating a robust infrastructure capable of supporting my day-to-day digital needs with resilience and efficiency. Let's delve deeper into the intricacies of my homelab setup, a testament to the power of hyper-converged infrastructure and the meticulous engineering that sustains it.

At the core of the infrastructure are 3 physical servers, each running Proxmox Virtual Environment, which is a versatile linux-based hypervisor that underpins the entire infrastructure. Proxmox's flexibility and efficiency make it the perfect candidate for running a variety of virtual machines and LXC containers (even though I run my containers in a k3s cluster instead). All services and workflows run on virtualized machines hosted on the 3 PVE machines. Three of these VMs (one in each server) act as the master nodes for my k3s deployment; all other VMs are either running appliances or dedicated services. Two of the servers are actually used Lenovo Ultra Small Form Factor PCs, and the other one is built from scratch. The latter has a 6-drive HDD bay, and is where my TrueNAS Scale VM lives. Since TrueNAS operates with the ZFS file system, it needs direct access to the disks in order to read the SATA metadata; Proxmox facilitates this with physical disk passthrough. It also supports direct PCI passthrough if I decide to get a dedicated GPU for encoding/AI workloads. Even though it is not recommended to virtualize a NAS, it's hard to justify a beefy Xeon CPU just to run OpenZFS workloads.

Another server in the stack is dedicated to networking, running an OPNsense appliance that oversees firewalling and routing. This setup ensures that my network is not only secure from external threats but also smartly managed to facilitate seamless communication between different services and devices. The backbone of this interconnected ecosystem is a trio of servers, each hosting k3s master/worker Debian nodes. These nodes are provisioned declaratively with Ansible, leveraging a GitOps workflow through Flux. This methodological approach ensures consistency, reproducibility, and scalability, allowing the infrastructure to evolve without compromising reliability.



Connectivity within this homelab is nothing short of revolutionary, with each server equipped with 10Gbps SFP+ NICs. The inclusion of a Juniper EX3300 L3 switch, featuring 4 SFP+ 10Gbps slots, elevates the network's data transfer capabilities, ensuring that high-speed connectivity is not just a luxury but a standard. This setup facilitates incredibly fast LAN speeds, making large-file data transfers and backup restorations a breeze.

Storage solutions within this homelab are meticulously engineered, with TrueNAS serving as the cornerstone of persistent storage. This FreeBSD-based NAS system leverages ZFS to create a networked file system that is both highly available and fault-tolerant. The configuration includes 2 x 6 TB HDDs in a mirrored pool, supplemented by a 500 GB NVMe SSD as an L2 cache and 64 GB of RAM for L1 caching. This layered caching strategy is crucial for optimizing data access speeds, allowing for the full utilization of the 10Gbps network capacity during intensive data transfer operations.

A notable feature of this homelab is its physical footprint. Two of the servers are ultra-small form factor PCs, a design choice that posed an interesting challenge when integrating the sizeable 10Gbps NICs. This constraint didn't hinder performance but rather added a layer of complexity and satisfaction to the assembly process.



An essential aspect of managing this homelab is the use of the main server's BMC webUI, accessed through the IPMI interface over Ethernet. This setup bypasses the need for traditional video output to a monitor, allowing for remote management and troubleshooting of the server, further emphasizing the system's versatility and user-centric design.



The logical topology of this homelab, detailed in the accompanying diagram, reveals not just the complexity and efficiency of the setup but also its connectivity with external services like Cloudflare and AWS. This integration highlights the homelab's role not just as a standalone system but as a node within a larger network of services, benefiting from the robustness and scalability of cloud solutions while maintaining the personalization and control of a private infrastructure.



This homelab is more than just a collection of hardware and software; it's a dynamic ecosystem that balances performance, security, and scalability. It represents the culmination of a journey from curiosity to critical infrastructure, demonstrating the power of modern virtualization, networking, and storage solutions in creating a resilient, efficient, and deeply personal digital environment.

A home datacenter is what most Americans deserve, but don't need.