linux_benchmarking


Designing and creating a homelab network running Proxmox

Now that I have an automated and simple way to create virtual machine templates as well as virtual machines, it is time to start thinking about what I want to create.

Designing the virtual networks

For now, the only available network on Proxmox is the Linux bridge vmbr0 which is directly connected to my local Lan.

In addition to that, I would like to have three other networks:

  • A development network: This will host many different things including a kubernetes cluster, a jenkins server and whatever technology I happen to be playing around or want to experiment with as well as virtual machines for whatever other personal projects I happen to be working on. It will also serve as a lower environment to test in any experimental features.

  • A production network: This will host services which I happen to find useful or handy in my daily life/operations. This environment will be handled with a little bit more care than the previous one.

Examples:

  • Kubernetes
  • Jenkins
  • Hashicorp Vault
  • NextCloud
  • Gitlab
  • Grafana
  • Prometheus
  • Portainer
  • OpenVPN
  • Some Telegram bots

  • NFS server

  • A security network: This will serve as a security lab where I can experiment with different versions of vulnerable software, hone my pentesting or reverse engineering skills.

  • A Security sandbox network: This one will be an airtight sealed network where viruses, worms, and different kinds of malware could be studied freely.

One of the main criterias for this network is that it will require strict isolation and monitoring to make sure nothing escapes and nothing I do within it has consequences or affects rest of my network.

I hope this paints a vague picture of how the environment will look like, it is of course not set in stone as many things will change given how many unknown restrictions and issues lie ahead.

Now back to the practical side of things, I mentioned wanting to create three additional networks within our proxmox environment. Creating the kind of network I would like to create is relatively easy, all that is needed is modifying the /etc/network/interfaces file to include the definition of our networks. We are obviously not going to go ahead and modify these files manually on the system for reasons which should be obvious, so how are we going to do it? the answer is: Ansible.

Why ansible? Well, modifying a file on a remote system and maintaining it in that state seems like the kind of task Ansible was made to handle..

Let’s start by setting up Ansible properly.

How to start an Ansible project

According to Ansible’s best practices section, this is how the a directory layout of a project should look like:

production                # inventory file for production servers
staging                   # inventory file for staging environment

group_vars/
   group1.yml             # here we assign variables to particular groups
   group2.yml
host_vars/
   hostname1.yml          # here we assign variables to particular systems
   hostname2.yml

library/                  # if any custom modules, put them here (optional)
module_utils/             # if any custom module_utils to support modules, put them here (optional)
filter_plugins/           # if any custom filter plugins, put them here (optional)

site.yml                  # master playbook
webservers.yml            # playbook for webserver tier
dbservers.yml             # playbook for dbserver tier

roles/
    common/               # this hierarchy represents a "role"
        tasks/            #
            main.yml      #  <-- tasks file can include smaller files if warranted
        handlers/         #
            main.yml      #  <-- handlers file
        templates/        #  <-- files for use with the template resource
            ntp.conf.j2   #  <------- templates end in .j2
        files/            #
            bar.txt       #  <-- files for use with the copy resource
            foo.sh        #  <-- script files for use with the script resource
        vars/             #
            main.yml      #  <-- variables associated with this role
        defaults/         #
            main.yml      #  <-- default lower priority variables for this role
        meta/             #
            main.yml      #  <-- role dependencies
        library/          # roles can also include custom modules
        module_utils/     # roles can also include custom module_utils
        lookup_plugins/   # or other types of plugins, like lookup in this case

    webtier/              # same kind of structure as "common" was above, done for the webtier role
    monitoring/           # ""
    fooapp/               # ""

Since we are starting off with a mildly small project, we just want to automate the task of modifying a file (/etc/network/interfaces) on a remote linux machine, and then restart the networking service so that our changes take effect.

Our project layout should look look something like this:

hosts                     # inventory file for our servers

group_vars/
   ghima.yml              # This is the variable group for our Proxmox, the name ghima means "Cloud" in Algerian

site.yml                  # master playbook
ghima.yml            # playbook for webserver tier

roles/
    ghima-networking/               
        tasks/            #
            main.yml      #  <-- Here we will store the part which orchestrates the changes we wish to make
        handlers/         #  <-- No handlers needed for now, but let's include it either way
        templates/        #  <-- Same goes for templates
        vars/             #  <-- Same here

Now that we have a skeleton for our project, let’s look at the individual files we need:

  • hosts: 
    [ghima]
192.168.4.2

    This file contains the inventory of machines we’re using Ansible to manage, for now we only have a proxmox server.

  • group_vars/ghima.yml
---

dev_network_ip: 192.168.5
dev_network_if: vmbr1

prod_network_ip: 192.168.6
prod_network_if: vmbr2

sec_network_ip: 192.168.7
sec_network_if: vmbr3

sandbox_network_ip: 192.168.8
sandbox_network_if: vmbr4

This file contains the set of variables to expect to need while setting up the networking part of the proxmox server.

  • roles/ghima-networking/tasks/main.yml:
---

- name: create development/production and security networks
  blockinfile:
    path: /etc/network/interfaces
    block: |
      auto 
      iface  inet static
        address .1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '.0/24' -o  -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '.0/24' -o  -j MASQUERADE
      
      auto 
      iface  inet static
        address .1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '.0/24' -o  -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '.0/24' -o  -j MASQUERADE
      
      auto 
      iface  inet static
        address .1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '.0/24' -o  -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '.0/24' -o  -j MASQUERADE

      auto 
      iface  inet static
        address .1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '.0/24' -o  -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '.0/24' -o  -j MASQUERADE
      

- name: Restart service networking
  ansible.builtin.service:
    name: networking
    state: restarted

In this file, we define two actions to execute:

  • Making sure the /etc/network/interfaces file contains the configuration block defining the addition 4 virtual networks.
  • Restarting the networking service to make sure the changes added above are applied.

Now that we have a ready to use Ansible playbook, let’s deploy it: ansible-playbook site.yml -i hosts -k

For now we are using password based authentication to deploy our changes, this is of course not ideal but I will amend that in the future.

In between all of these networks we will be needing a firewall to control the traffic flow, for that purpose, and at least initially, the Proxmox firewall should suffice. We will delve into that deeper in the future.

We might eventually want to include some DHCP servers in each network, but for now assigning IP addresses manually through cloud-init should be conveninent enough.