Paul's Internet Landfill/ 2013/ Isolating Subnets in pfSense

Isolating Subnets in pfSense

The pfSense firewall distribution is one of my favourite pieces of software. It is powerful and flexible, has wide adoption, and is under active development.

For the most part, the GUI for firewall rules is intuitive to use. But it has a huge problem: it makes isolating subnets unintuitive. Consider the following set of networks:

GUEST uses the OPT1 interface, and WORKSHOP uses the OPT2 interface.

I want to accomplish the following:

For years, I assumed that this was easy to set up. For years, I have been doing it wrong, and unless you are careful you might do so as well.

Terminology

Let's define the following aliases:

Let's define the following isolation exceptions:

Let's assume that "subnet" is a synonym for the pfSense concept of "interface" (even though that is not strictly true)

Failed Attempt 1: Defensive Subnets

My first approach was as follows:

Here is what the rules for LAN would be (in pseudocode, not real pf notation):

Here are the rules for GUEST:

Here are the rules for WORKSHOP:

I am not actually sure whether the

are actually necessary (in this attempt or any others) but I included them to be safe.

Here are the problems with this approach:

To understand this behaviour, you need to understand what pfSense does behind the scenes in translating rules from the nice GUI into actual pf firewall rules that the underlying FreeBSD system can use. Here is my understanding of how rules are generated:

The quick keyword has an effect here (especially for floating rules) but as far as I can tell it does not help us much.

You need not take this assertion on faith: just SSH into your pfSense box and run the pfctl -s rules or pfctl -vv -s rules commands.

What this means for our example is that all rules on the LAN tab are processed before any rule in the GUEST (aka OPT1) or WORKSHOP (aka OPT2) tabs. In particular, the following rule:

is executed before

or

rules. That first rule allows traffic to go from the LAN Subnet to any other subnet, including the GUEST and WORKSHOP ones, so the later rules do not matter. Oops!

Similarly, the rule:

in the GUEST tab is processed before

on the WORKSHOP tab, which means that the WORKSHOP subnet will be blocked from nethack_hosts regardless of the second rule.

In retrospect (and once I saw what was going on with pf rule generation) this behaviour made perfect sense. But it is not the behaviour I expected when I first used the GUI interface -- for some reason I thought that pfSense would magically know that I wanted to block traffic between subnets but still allow generalized internet traffic.

It is possible that you could massage this particular example into working with a "defence first" approach, but I kept getting stuck, so I started investigating floating rules.

Semi-Failed Attempt 2: Floating Rules

pfSense version 2.0 introduced the idea of "floating rules" -- rules that can apply to multiple interfaces, and which would be processed before any of the interface-specific tabs. I thought I could use this to poke holes in the isolated subnets (which would solve the problem of WORKSHOP getting access to nethack_hosts above).

The real problem with this approach is preventing LAN from accessing the other subnets willy-nilly. I came up with something that kind of worked, but it effectively required all rules to be in the "Floating" tab:

The ruleset would look something like this:

(I simplified things a little by using all_subnets as shorthand for individual subnets in a few places.)

This looks workable for this example, but it has a real disadvantage: effectively, all rules have to be in the "Floating" tab, which means the other tabs do not get used. This is not bad for a simple example, but it gets quite messy (and therefore quite error-prone) when you have seven interfaces with many unique exceptions per interface. Making matters worse is that the tabular display for floating rules does not indicate the set of interfaces for which the rule applies -- if I have a rule that should ONLY apply to the GUEST interface, I cannot see this until I click into the rule.

One wildcard with this approach is the "quick" keyword. I never figured out how to use it properly in this context (although I have used it in traffic shaping, to put traffic into queues).

Working (?) Attempt 3: Polite Subnets

Here is the approach that ended up working for me. No doubt it is the approach that most people come up with from the beginning, but I is dumb:

This takes the opposite approach from Attempt 1: instead of subnets defending themselves from unwanted traffic from other interfaces, they depend on other interfaces prohibiting unwanted traffic from escaping to them. From a security standpoint this seems insane -- very few security scenarios assume that other actors (in this case, other subnets) are benevolent. But in our case all sysadmins control the behaviour of all subnets, and so should be able to coordinate the actions of those subnets together. If you somehow had a situation where different sysadmins had access to only the firewall rules of their own subnet (and could not be trusted to behave well towards other subnets) then this approach would fail.

With this approach, here is what the rules look like for LAN:

Here are the rules for GUEST:

Here are the rules for WORKSHOP:

It looks almost the same as Attempt 1, except that rules like:

are switched to

This approach is weird to my brain, but (as far as I can tell) it works. Unlike Attempt 2 it allows you to use the firewall GUI tabs in a helpful way. It also scales well to many many subnets, just by use of the all_subnets alias.

It could very well be that my approach is STILL all wrong. Maybe it is catastrophically wrong! I feel stupid even posting this on the Internet, given that I am far from a pfSense expert (and I expect pfSense experts are scoffing at me now). But I believe this gets me a lot closer to the behaviour that I want than my earlier approaches, so I thought I would share.