Learn by Doing – Build a Home Lab

6 March 20216 March 2021 DeanLeave a comment

Getting your start in IT administration is exciting but can also be a daunting experience. I landed my first IT job working a helpdesk and, in just over a year, moved into a System Administrator role. While I was generally able to get things done, I was well aware that I didn’t understand how a lot of the infrastructure worked. Domain Controllers, firewalls, hypervisors, Active Directory, Group Policy… the list goes on. It was all new to me.

As someone who learns best by building things for myself, it’s tough trying to learn new things in an already established environment. It’s rare that a core piece of infrastructure needs rebuilt and is probably not a task that would be handed to the new guy. I was interested and motivated enough to self-learn, so it was time to look for another way.

Sooner or later I came across the concept of a “home lab” (thanks /r/homelab) and that was the game changer! For a relatively low cost I could build out my own “Enterprise environment” at home and create a network from nothing. I took full advantage of this and found it to be brilliant learning experience that I still use to this day.

The Server

Back in 2015 I bought a Lenovo ThinkServer TS140. It cost me £330 (new) and is about the size of your typical desktop tower PC. It lives behind my sofa. Out of the box it came with an Intel Xeon CPU E3-1226 v3 @ 3.30GHz, 4GB 1600MHz DDR3 UDIMM RAM, a 1TB HDD and a single NIC. I didn’t think the 4GB of RAM would cut it, so I added an additional 16GB, bringing it up to a total of 20GB RAM.

This reasonably priced server is still running my home lab environment in 2021! The only other upgrade it has seen since then was the addition of an SSD, but even that was salvaged from an old laptop.

These days some people might ask why I bother running a server at home instead of hosting everything with a cloud provider. If it weren’t for the fact that I already have the server, I might consider cloud hosting. It would be a good chance to play around with Azure but, ultimately, I don’t want the recurring monthly cost. My 6 year old ThinkServer is still doing just fine. If I ever feel the need for an upgrade, I’d be tempted to look in the direction of an Intel NUC, but they’re significantly more expensive.

Installing a Hypervisor – VMware ESXi

Anyway, now that I have the physical server, I want it to be running multiple virtual servers. To do this, I install a Type 1 hypervisor. At work I use VMware’s ESXi, so that’s what I use in my home lab.

You can download, install and use VMware’s ESXi hypervisor for free. The main limitation being that you cannot run vCenter for free. vCenter is what allows you to manage multi-host clusters and do the fancy stuff like migrating running VMs between hosts. In my single server setup none of that is needed, so it’s not a deal breaker.

Installing ESXi is a breeze. You just download the installer from VMware, create a bootable USB, pop it into your server and power it on. Once it runs through the installation, all that’s really left to do is set the root password and the network settings. From there, go to your desktop machine, open your browser and go to the IP address of the server. You should be greeted with a web interface for managing the host. Done!

VMware Networking

Something I like to do with my home lab environment is keep it separated from my home LAN. This allows me to more easily mimic an enterprise environment in the lab without having unwanted side effects on my home devices. E.g. I might want to run my own DHCP and DNS servers in the lab but don’t want my normal devices using those services. I do this by using virtual switches in VMware and creating a separate network that only exists in the virtual environment. However, it is useful to be able to access the lab from my main desktop PC, so I use a firewall (also a virtual machine) to act as a gateway between the two networks. I then set a route on my PC to send traffic for that network through the gateway. More on that later.

In VMware I need to create two vSwitches. vSwitch0 is linked to the physical NIC on the Lenovo server. vSwitch1 is not linked to a physical NIC. I then create a couple of Port Groups in VMware: External, which is associated with vSwitch0 and Internal, which is associated with vSwitch1.

Whenever I add a new VM within VMware, I link it’s virtual NIC to the “Internal” port group (vSwitch1). This means that all my VMs can speak to each other, but don’t have direct access to my home LAN or the internet. Again, I’ll address this in the next section, but the end goal is to get the network looking something like the diagram below. If you click on the image an enlarged version will open in a new tab.

Adding a Firewall – Sophos XG

As I mentioned earlier in the post, the firewall is used to act as a gateway between my home LAN and the lab environment. It runs as a VM with two virtual NICs. One NIC is linked to the “External” port group and the other is connected to the “Internal” port group. This gives the firewall VM access to both networks. The other virtual servers can use the firewall as their default gateway and speak to the outside world, assuming firewall rules exist to allow this.

Normally I use pfSense for my firewall, but this time I wanted to try something different. At work we use Sophos UTM but now often hear that Sophos XG is the product that Sophos would prefer you to use. As it turns out, Sophos allow you to use either firewall free of charge for home use. UTM Home Edition comes with the limitation of only allowing management of 30 IP addresses. XG Home Edition, on the other hand, imposes a hardware limit of 4 CPU cores and 6GB RAM. Other than that, it’s the same product as the paid for enterprise version. In my lab I’m unlikely to exceed either limit so the home editions are great to play with.

I actually had a bit of a rough time installing Sophos XG, but it turned out to be completely my fault. By default, Sophos XG only allows access to the admin interface from the LAN port (my internal virtual network). I wanted to do the initial setup from my desktop PC, which is on the WAN (my home LAN) side. After some fiddling in the XG console, I found a command that allowed access from the WAN port:

system appliance_access enable

This did the trick and I could now access the admin interface from the WAN port. Sophos XG definitely feels far more user friendly than the convoluted screens of Sophos UTM but I’ve seen others comment that it lacks the full feature set of UTM. Regardless, it’s perfectly fine for my use case. It took no time to get my firewall rules set up and begin testing, but testing is where the problems (of my own making!) started.

Fixing a Silly Mistake

I spent more time than I’m willing to admit trying to figure out why my VMs could not access the internet. They could resolve external DNS names, but not access the web, ping hosts, etc. I checked all the network adapter settings, I checked the VM networking settings, and checked all my firewall rules. I did all of these things multiple times. Everything was right, so why couldn’t I get internet access?

If you were paying attention to the screenshot of the “system appliance_access enable” command that I ran earlier, you’ll know the problem. The output of that command states: “All internet traffic will be dropped”. The solution was to revert that change with “system appliance_access disable”. Now, everything works as expected. Thankfully, Sophos XG has another method of allowing access to the admin interface from the WAN port without dropping all internet traffic.

Under normal circumstances you would almost never want your firewall’s management interface to be accessible from the WAN port. In my case the WAN is actually my home network (which is why I’m happy to enable access) but in an enterprise environment it will likely be “the internet”. You certainly don’t want random internet users/bots trying to find ways to exploit the management interface and gaining access to your firewall config, so don’t make it easy for them!

All Done

And with that, my initial home lab network is essentially complete. All that’s left to do is start installing some VMs and using them to learn new things.

Back in 2015 when I first set up my home lab I used it to run a Windows Domain. Microsoft provide evaluation releases of Windows Server, which you can use for 180 days free of charge. I created a couple of Domain Controllers and configured them to act as DNS + DHCP servers. From there I made sure to domain join any other Windows servers I added and even domain joined a laptop I had that was running Windows 7 Pro.

The goal was essentially to try and create a miniature version of the environment I use at work. Building a Windows Domain from the ground up is a great learning experience and far easier than you might think. You don’t have to stop there either. Setup a WSUS server. Play around with Group Policy. Configure web filtering on your firewall. Add your imaginary friends as Active Directory users!

Having the ability to play with all these technologies at home is a fantastic learning tool. Not only do you learn how all these things work, but you also learn how to troubleshoot enterprise level issues without the pressure of an entire company waiting for the fix.

If you’re trying to set up a home lab for the first time, chances are that at some point something wont work. You’ll run into issues with your network config or the firewall rules or DNS/DHCP settings. If something isn’t working, don’t give up! Spend the time to troubleshoot it. The whole point of a home lab is to learn new things and troubleshooting these types of issues is definitely a useful skill to have.

IT Automation with PowerShell

5 September 202012 March 2022 DeanLeave a comment

There are still a lot of IT administrators out there who have never taken the time to learn PowerShell (or any scripting language). PowerShell is a great tool that can save you a lot of time by automating those repetitive and/or tedious tasks that you’re always being asked to do.

It can take some time to learn the fundamentals of scripting when you are just getting started, but it’s well worth the time investment. Eventually, someone will give you what sounds like a very repetitive/tedious task (example to come) and your first response will no longer be an internal sigh. Instead, it will be excitement at the chance to write some PowerShell! Okay, maybe that’s a slight exaggeration.

A Real Life Example

This is a “real life” example. I had a requirement to create a folder structure within a network share. Each user in a specific Active Directory group required their own folder. The name of the folder was to match that user’s AD account username. The environment I work in is reasonably small, so that results in around 100 folders. However, the number of folders doesn’t really matter here, it could just as easily have been 1,000 or 10,000.

What would your initial reaction be if asked to perform that same task? Would you launch Excel, compile a list of the users who require a folder and then create 100 folders by hand in File Explorer, marking them off as you go? There are definitely IT admins out there who would take that approach.

It’s arguably a bad approach, but the logic behind the steps involved it isn’t so bad. You could use the steps in that process as the building blocks for a PowerShell script.

Break the problem down

Stop for a minute and think about the smaller steps involved in completing the bigger task. For most people, I imagine the thought process would be something like this:

Get a list of all users in the required AD group
Get the usernames of all those users
For each user in the list:
1. Check to see if they already have a folder
2. If one doesn’t exist, create a new folder
3. If there is already a folder, move on to the next user

Now that the problem is broken down into those smaller steps, you can almost translate it line for line into PowerShell. It is already starting to look more like a script!

Writing the PowerShell script

Even if you have no experience of PowerShell at all, hopefully you can look over the code below and it will make some kind of sense. At the very least it should somewhat resemble the steps listed above.

Import-Module ActiveDirectory

$groupName  = "AD Group Name"
$rootFolder = "\\SERVER\FileShare\"

$users = Get-ADGroupMember -Identity $groupName | `
Where-Object { $_.objectClass -eq 'user' } | `
Select-Object -Property samAccountName | `
Sort-Object

foreach($user in $users)
{
    $path = $rootFolder + $user.samAccountName

    if (-not (Test-Path $path))
    {
        New-Item -Path $path -ItemType Directory
    }
}

If you’ve never used PowerShell and have little or no experience of scripting, there will admittedly be parts of that code that aren’t very intuitive. You’ll likely be wondering what all the symbols mean. However, I bet if you spent some time working through a few PowerShell tutorials, most of it would make sense in no time.

How it works

Lets have a look through the script bit by bit and I’ll explain what it does.

Import-Module Active Directory

Because the script is going to be interacting with Active Directory, the first line tells the computer that we want to import all the Active Directory commands. These aren’t included as standard with PowerShell, which is why we need to import them. These actually need to be installed on the computer/server that will be running the script.

$groupName  = "AD Group Name"
$rootFolder = "\\SERVER\FileShare\"

These two lines are assigning values to variables. Think back to algebra class. If X = 4 then X + 2 = 6. X is just a placeholder for a value and that’s what $groupName and $rootFolder are. They’re just convenient names for values that we’ll be using later.

When I run this script for real I replace those dummy values with something useful. $groupName would equal the name of the AD group that contained the users. $rootFolder would contain the path to the file share that will contain all the sub-folders.

Getting the list of users

$users = Get-ADGroupMember -Identity $groupName | `
Where-Object { $_.objectClass -eq 'user' } | `
Select-Object { $_.samAccountName } | `
Sort-Object

This line, which spans multiple lines for readability, is probably the most intimidating looking bit of the script. Basically, I’m creating a list of all the usernames, sorted alphabetically, in the $users variable.

On the first line, after the “=”, I’m getting a list of all members of the AD group named $groupName. Now remember, $groupName is a placeholder that translates to the name of the AD group we’re interested in.

In the second line, “Where-Object” is acting as a filter. Because AD groups can contain more than just users as members (e.g. other groups, computers, etc.) we want to filter out everything that isn’t a user.

The third line, “Select-Object” is selecting only the attributes of the users I’m interested in. A user has many attributes, but in this case I only need the samAccountName (their username) so that’s all I pull through.

The last line “Sort-Object” is then just sorting the list of usernames alphabetically. This step isn’t really necessary. I just like processing things in order.

foreach ($user in $users)
{
    ....
}

Remember, the list of usernames lives in the $users variable. What I want to do is go through those users, one at a time, and set up their folder.

The foreach loop works it’s way through the list, user by user. The current user being processed is held in the $user variable. Notice it’s singular, rather than plural. That’s just how I like to name my variables. $users is the whole list and $user is the individual currently being processed.

For every user, it’s going to run the code between the { and } brackets.

Generating the file path

$path = $rootFolder + $user.samAccountName

I’m using a few variables in this line. I create a path variable and use it to hold the full file path of the current user’s folder, regardless of whether or not it exists. To create the path, I take the $rootFolder value, which was defined as “\\SERVER\FileShare\” and add the user’s samAccountName to the end of it. So it might end up looking like “\\SERVER\FileShare\Dean”.

Creating the folder (if needed)

if (-not (Test-Path $path))
{
    New-Item -Path $path -ItemType Directory
}

This is the part where I check if the file path already exists and, if not, create the folder. Test-Path is doing the work of checking if the folder already exists and New-Item is then creating a directory at that path if one does not already exist.

And that’s it.

Have you seen the light?

If you are one of the people who would have manually created the folders, hopefully this has given you a new perspective. Of course you’re not going to be able to sit down and start writing scripts just by having read this post, but maybe you’ll now have motivation to hunt down some PowerShell tutorials and start learning.

In my example I wrote a script that saved me from having to manually create 100 folders. It maybe took me 15-30 minutes to write and test that script. How long would it have taken to manually create the 100 folders? What if there were actually 1,000 folders to create? Well, the script can handle that too, without requiring any changes!

What if more users were added to that AD group next week and each required a new folder? Well, you could just run the script again and it would create the new folders.

But wait… having to run the script would be a manual task. Granted, it’s still much less time consuming than creating folders manually, but it’s still a little inconvenient.

Perhaps you could create a scheduled task to run the script every morning. That way any new users added to the AD group would automatically have a folder created. Plus, if anyone unintentionally deleted their folder it would be re-created (albeit without the contents).

Where to go from here?

Look for online learning content
Get familiar with the basics of PowerShell
Bookmark the PowerShell documentation
Put together a list of your simple, repetitive admin tasks that could be automated. Make these your first projects to tackle
Start scripting!

Scripting is a skill and, like anything worth learning, it takes practice. You’re not going to become a master overnight. You’re not going to memorise every PowerShell cmdlet. What’s important is that you start changing the way you approach problems.

Look for manual tasks that you perform regularly. Break them down into steps. Translate those steps into PowerShell until you’ve built out the whole process. It might take you a while to get the script working, but once it is working that’s one less recurring manual task on your to do list!

Hopefully you see how much this can benefit you and those that are dependent on your work. Automate the mind-numbingly dull tasks out of your working life and spend more time focusing on the big picture projects that benefit everyone.

Any IT admin can create 1,000 folders. The smart admin will have a script do all the work for them!

Dean Thomson

Blog of an IT administrator

Category: Personal Development