My lab started with a need to simplify my work load, but had big ideas attached to it. A single server purchase started it off, then it quickly snowballed as I learned more about what was possible on a budget. The lab grew bigger as the ideas became clearer. My home lab hasn’t reached the bottom of the hill just yet, but I ready to start telling the story of how it came to be and what lies ahead.
My lab started with a need to simplify my work load, but had big ideas attached to it. A single server purchase started it off, then it quickly snowballed as I learned more about what was possible on a budget. The lab grew bigger as the ideas became clearer. My home lab hasn’t reached the bottom of the hill just yet, but I ready to start telling the story of how it came to be and what lies ahead.
The story starts about a year ago when I decided to leave my role as a systems administrator for a career in education. I took a job teach Information Technology courses at a local university that had a heavy focus on hands on learning. Beside preparing lectures, I also had to prepare virtual lab environments for students to learn from real world scenarios.
I quickly found it difficult and time consuming to use the environment provided for me. The lack of access made requests for simple changes took too much time, making it difficult to get the configurations just right. I started developing lab modules on my desktop, but even with 32GB of RAM, I was unable to smoothly run 10+ systems that some of the more complex environments used.
One evening I pulled the trigger on my first home lab server to ease this pain. I found a system on Amazon that seemed to fit my needs. A Dell PowerEdge R710 with dual Xeon 5670’s, 144GB of memory, and 12TB of raw storage. A few days later it arrived at my door and by that evening it was running VMware ESXi; I was off to the races building lab modules.
That one server satisfied my current need, but I found myself quickly building out a test bed for new ideas. Two new issues came up and I needed more. (Or maybe I just wanted more.) Even though I could now create and test the environments easily, I wanted to add better automation to the deployment of the lab modules. My idea was to replicate the student learning environment and find ways to do this. So clearly, I was going to need more servers.
Then I got lucky and found an amazing deal on some Fujitsu RX300 S7’s at a local computer recycling non-profit. Inside each I found dual Xeon e5-2620 processors and 64GB of RAM. I also was able to score a deal on a Cisco SG300 28 port switch and a couple of APC UPS’s. I turned two of the RX300’s into a vSphere cluster and picked up a Dell Precision T3500 tower to act as a make shift FreeNAS iSCSI device for shared storage.
I began to play around with Ansible at this point (which is still a work in progress), but I quickly found that a 1GB SAN wasn’t sufficient and the RX300’s are very picky with RAM. I decided to put together a plan to create a more robust environment, with some scalability in mind. I also laid the ground work for my phase 3 project.
I searched the “interwebs” and found a very respectable deal on a pair of Dell PowerEdge R620’s from an online dealer. These came with dual Xeon e5-2630s, 64GB of RAM and 10GbE NICs. I added some RAM I got from the non-profit to outfit each with 192GB.
I added a 10GbE NIC and some Intel 480GB SSDs to my R710 and turned it into a NAS using XigmaNAS (formally NAS4Free). In time I can add an external HBA card and storage chassis if I need to expand. I attached all of this to a Mikrotik 10GbE -16port switch (Cloud Router Switch 317-1G-16S+RM) which does a fine job for the price and is passively cooled so it doesn’t add any more noise.
I repurposed the Dell T3500 for a Pfsense box on the edge of my network. This allowed me setup OpenVPN for remote access, segment my network with VLANs, and utilize some of the other many features in the future. I also picked up an HP z210 desktop, plopped in some SSDs I had lying around for a mirrored boot drive running Windows Server 2019. I repurposed the 12TBs of storage from my R710 and strapped them in (some with zip ties) for backup storage. Currently Veeam’s community edition is backing up critical VMs for me.
I know you are wondering what I did with those Fujitsu servers. Don’t worry, they won’t be idle for long. I’m prepping them for phase 3. I recently purchased a couple of 10K 600GB spinners and 10GB Mellanox Connect x2 cards for them. I’m also trying to find more RAM that will make them happy.
Phase 3 is a little project that I’ve had in the back of my head for a while. I don’t know what to call it, but I’m thinking somewhere along the lines of “Sandbox in a box”, “Homelab to go”, or “Cyber Range on the Fly”. Regardless of what it ends up being called, the idea is to automate and package a set of open source tools that can allow anyone with minimal technical knowledge to roll out (onto old bare metal) and have a completely functional lab environment for training or teaching. The original idea was for high school teachers that wanted a cheap and easy way to bring IT into the classroom with hopes of being able to tie into the Ohio Cyber Range (more details on this in a future story.)
The story of this lab is not over, it hasn’t even really begun. Just the other day I came across a good deal on a Dell R320 and am still trying to decide a way to integrate that into the system. I may be a teacher, but I am still learning everyday myself and this lab has opened my mind to so many things already. I’d love to hear your thoughts, questions, and ideas for my lab.
– Ryan Moore, @RyanMoore88
This post is part of an ongoing series of user-submitted content dedicated to exploring the fun and challenges of building, maintaining and sometimes building again a home lab. This series is in partnership with our friends at /r/homelab. If you're interested in sharing your set up, please send an email to [email protected]
