In this blog post, I describe how we can use nftables to set up a secure environment where IOT devices can access the home assistant server, but only the home assistant can access the Internet.

Initial Setup

It does not make much sense to talk about the filtering rules without first describing my home network setup. I use as my router the CWWK P5, a mini PC with 4 ethernet ports. The first port connects to the modem, and the remaining three connect to my gaming PC, my home server, and an Omada EAP-225, a dedicated wireless access point.

I use VLAN tagging to split my home network into multiple segments.

The wireless access point has VLAN tagging functionality, and so does the Linux router, which uses systemd-networkd to configure a VLAN-aware bridge to join together the 4 ethernet ports.

The home assistant, a piece of software that manages IOT devices, runs as a VM on the home server. The IOT devices all communicate to the home assistant through the wireless access point. The router would then transparently forward the L2-layer traffic between the wireless access point and the home assistant VM. All packets in this case must be tagged with the VLAN id 40.

To make things more concrete, I’ll spell out the relevant nic names on the router. The interface enp1s0 is connected to the Internet. The interface enp4s0 is connected to the wireless AP, and the interface enp3s0 is connected to the computer hosting the home assistant VM. The bridge br enslaves enp1s0, enp3s0, and enp4s0. To route traffic from VLAN id 40 to the internet, we also create the interface vlan.40 to listen on L2 packets tagged with vlan id 40 from the bridge port associated with br (which we also refer to as br as is conventional on Linux). By default, the router would happily forward the traffic to the outside world from vlan.40 to enp1s0.

The threat model we consider here presumes that the IOT devices might attempt to contact the outside world and upload private data to the attacker. By default, the router is set up so that no ports of the IoT devices or the home assistant has been exposed to the public internet. However, the current setup doesn’t prevent the IOT devices from accessing the Internet, which is a risk. Our goal here is to allow home assistant access to the Internet since it is a piece of software that we trust and it needs internet access for updates and add-ons, while simultaneously blocking the Internet access for the IOT devices.

Selectively blocking Internet access with mark and vlan matching

Nftables supports filtering traffic based on VLAN tags. The clause vlan id 40 ... matches packets that are tagged with the VLAN tag 40. However, we don’t want to do this indiscriminately as the packets from the home assistant are also tagged with the same vlan id.

Another complication is that vlan tagging is a layer-2 concept, which sits one layer below layer-3 (i.e. the layer where you can talk about udp and tcp packets). By default, according to the nftables documentation, vlan filtering cannot happen in the inet family tables as the information about the vlan id is forgotten once the packet has reached that stage. To visualize the process, you can refer to the flow diagram on the nftables wiki page. Once that packets travels to the green part of the flow chart, the vlan information is, by default, not available.

Instead, we use a bridge filter to gain access to the VLAN access information. If you don’t have a bridge filter already, you would need to create an empty bridge filter table and add in the VLAN matching rule.

table bridge iotfilter {
  chain myinput {
    type filter hook input priority filter
    policy accept

    vlan id 40 iifname enp4s0 ...
  }
}

Through the above code snippet, which you can add to your nftables.conf, we create a table named iotfilter of the bridge family, and create a chain named myinput added to the built-in input hook.

The semantics/meaning of the input hook is quite different from that of an inet table. A packet will enter the myinput chain not only when its ip destination address is the router, but also when it uses the router as a gateway (see the wiki for a thorough explanation) to reach the Internet.

The match statement vlan id 40 iifname enp4s0 allows us to match such packets with vlan tag 40 and from the wireless AP, which we know can only be from IoT devices.

What exactly should we do to such packets? Simply replacing the ... with drop is incorrect, as we need the router to handle dhcp requests. Instead, our goal is to drop specifically the packets whose destination is not the router itself, but somewhere on the internet which can only be reached by using the router as a gateway.

The first thing I thought about using is the fib expression, which allows you to declaratively find out if a packet is addressed to the machine by querying the routing table and the ip addresses configured on the interfaces. Sadly, the fib functionality doesn’t seem available in the bridge family and nftables would complain with a non-descriptive message if you try to use it.

Instead, we can use the set the firewall mark on the packets and then drop those packets if they ever reach the forward stage.

Here’s the full setup, with the inet family table containing lots of ... as you may already have rules there.

table bridge iotfilter {
  chain myinput {
    type filter hook input priority filter
    policy accept
    # mark packets from IoT devices with 0x40
    # or any other number of your choice
    vlan id 40 iifname enp4s0 mark set 0x40
  }
}

table inet filter {
  ...
  chain forward {
    type filter hook forward priority filter; policy drop
    # if a packet has the mark 0x40 we set earlier,
    # we drop it because it must be an IoT packet
    mark 0x40 drop

    # packets with vlan tag 40 but from the home assistant server
    # goes through just fine as packets from enp3s0 are not marked
    ...
  }
}

With this setup, the home assistant can access the internet just fine as packets from enp3s0 are left unmarked and continues to be processed by the rest of the forwarding logic, whereas the attempts by the IoT devices are dropped unless they are addressed to the router itself, in which case they would have been processed by the input chain in the inet filter table.

Some extra rules

If you successfully set your firewall rules as I described, you can rest assured that your cat cam video footage won’t be uploaded and broadcasted on the Internet. Note that the rules above are robust against IP spoofing, as we filter based on the device where the packets are received. Although of course attacks to the application-layer can still allow an attacker to take over the home assistant server.

There are some rules you can add to the forward chain of the bridge filter to harden your setup and guard against attacks like spoofing. Here’s one rule you can add to the bridge filter to prevent an IoT device from responding to DHCP requests. In other words, the only valid DHCP packets should only be seen by the router. Again, you might want to read this page about nftables bridge to understand when exactly a packet goes through the forward chain.

table bridge iotfilter {
  chain forward {
    type filter hook forward priority filter
    policy accept
    # should not allow dhcp packets on vlan 40 (iot) that don't originate from the router
    vlan id 40 udp dport {67,68} drop
  }
  ...
}

Testing the setup

To validate the success of the setup, make sure you can still access the Internet from the home assistant VM.

To make sure that the IoT devices can’t access the Internet, connect your laptop or phone to the IoT wifi network and try accessing a website or ping 1.1.1.1 and make sure that both fail. You’d also want to check that your policy isn’t filtering out the dhcp requests. Despite not having internet access, your phone or laptop should be assigned an IP address and you should be able to ping the router and home assistant.

A better architecture?

I used to run home assistant on an nspawn system container which runs Arch Linux. As I had more control over the system on which home assistant was installed, I used the system container itself as a router for the IoT devices. The container had two interfaces, one that connects to the IoT devices and one that connects to the Internet. The kind of access control I described in this blog post can be achieved by simply disabling ip forwarding to the interface with internet connection.

However, the Arch Linux package for home assistant got deprecated lately and the recommended solution on the Arch Linux Wiki is to use a docker container, which really shouldn’t be run inside a system container.

I decided to migrate my installation to the home assistant VM, a VM that manages and updates itself. There might be ways to do advanced network configuration with the home assistant VM, but I didn’t have the patience to navigate the GUI for a functionality that may not even exist.

Using the VM as a router has the downside that it is more difficult to provision public IPv6 addresses. In this discussion thread on the lxc forum, it is recommended that we use the router that interfaces with the ISP to assign IPv6 subnets instead of using a managed network to do nested prefix delegation, which doesn’t seem supported. In short, relying on a single global router requires a bit more firewall configuration to achieve the same level of security, but it also gives us an easy way to make the clients ipv6 capable.