Jails in FreeBSD provide a simple yet flexible way to set up a proper server layout. In the most setups the actual server only acts as the host system for the jails while the applications themselves run within those independent containers. Traditionally every jail has it’s own IP for the user to be able to address the individual services. But if you’re still using IPv4 this might get you in trouble as the most hosters don’t offer more than one single public IP address per server.

Create the internal network

In this case NAT (“Network Address Translation”) is a good way to expose services in different jails using the same IP address.

First, let’s create an internal network (“NAT network”) at 192.168.0.0/24. You could generally use any private IPv4 address space as specified in RFC 1918. Here’s an overview: https://en.wikipedia.org/wiki/Private_network. Using pf, FreeBSD’s firewall, we will map requests on different ports of the same public IP address to our individual jails as well as provide network access to the jails themselves.

First let’s check which network devices are available. In my case there’s em0 which provides connectivity to the internet and lo0, the local loopback device.

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
  [...]
  inet 172.31.1.100 netmask 0xffffff00 broadcast 172.31.1.255
  nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

For our internal network, we create a cloned loopback device called lo1. Therefore we need to customize the /etc/rc.conf file, adding the following two lines:

cloned_interfaces="lo1"
ipv4_addrs_lo1="192.168.0.1-9/29"

This defines a /29 network, offering IP addresses for a maximum of 6 jails:

ipcalc 192.168.0.1/29
Address:   192.168.0.1          11000000.10101000.00000000.00000 001
Netmask:   255.255.255.248 = 29 11111111.11111111.11111111.11111 000
Wildcard:  0.0.0.7              00000000.00000000.00000000.00000 111
=>
Network:   192.168.0.0/29       11000000.10101000.00000000.00000 000
HostMin:   192.168.0.1          11000000.10101000.00000000.00000 001
HostMax:   192.168.0.6          11000000.10101000.00000000.00000 110
Broadcast: 192.168.0.7          11000000.10101000.00000000.00000 111
Hosts/Net: 6                     Class C, Private Internet

Then we need to restart the network. Please be aware of currently active SSH sessions as they might be dropped during restart. It’s a good moment to ensure you have KVM access to that server ;-)

service netif restart

After reconnecting, our newly created loopback device is active:

lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet 192.168.0.1 netmask 0xfffffff8
  inet 192.168.0.2 netmask 0xffffffff
  inet 192.168.0.3 netmask 0xffffffff
  inet 192.168.0.4 netmask 0xffffffff
  inet 192.168.0.5 netmask 0xffffffff
  inet 192.168.0.6 netmask 0xffffffff
  inet 192.168.0.7 netmask 0xffffffff
  inet 192.168.0.8 netmask 0xffffffff
  inet 192.168.0.9 netmask 0xffffffff
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Setting up pf

pf part of the FreeBSD base system, so we only have to configure and enable it. By this moment you should already have a clue of which services you want to expose. If this is not the case, just fix that file later on. In my example configuration, I have a jail running a webserver and another jail running a mailserver:

# Public IP address
IP_PUB="1.2.3.4"

# Packet normalization
scrub in all

# Allow outbound connections from within the jails
nat on em0 from lo1:network to any -> (em0)

# webserver jail at 192.168.0.2
rdr on em0 proto tcp from any to $IP_PUB port 443 -> 192.168.0.2
# just an example in case you want to redirect to another port within your jail
rdr on em0 proto tcp from any to $IP_PUB port 80 -> 192.168.0.2 port 8080

# mailserver jail at 192.168.0.3
rdr on em0 proto tcp from any to $IP_PUB port 25 -> 192.168.0.3
rdr on em0 proto tcp from any to $IP_PUB port 587 -> 192.168.0.3
rdr on em0 proto tcp from any to $IP_PUB port 143 -> 192.168.0.3
rdr on em0 proto tcp from any to $IP_PUB port 993 -> 192.168.0.3

Now just enable pf like this (which is the equivalent of adding pf_enable=YES to /etc/rc.conf):

sysrc pf_enable="YES"

and start it:

service pf start

Install ezjail

Ezjail is a collection of scripts by erdgeist that allow you to easily manage your jails.

pkg install ezjail

As an alternative, you could install ezjail from the ports tree. Now we need to set up the basejail which contains the shared base system for our jails. In fact, every jail that you create get’s will use that basejail to symlink directories related to the base system like /bin and /sbin. This can be accomplished by running

ezjail-admin install

In the next step, we’ll copy the /etc/resolv.conf file from our host to the newjail, which is the template for newly created jails (the parts that are not provided by basejail), to ensure that domain resolution will work properly within our jails later on:

cp /etc/resolv.conf /usr/jails/newjail/etc/

Last but not least, we enable ezjail and start it:

sysrc ezjail_enable="YES"
service ezjail start

Create a jail

Creating a jail is as easy as it could probably be:

ezjail-admin create webserver 192.168.0.2
ezjail-admin start webserver

Now you can access your jail using:

ezjail-admin console webserver

Each jail contains a vanilla FreeBSD installation.

Deploy services

Now you can spin up as many jails as you want to set up your services like web, mail or file shares. You should take care not to enable sshd within your jails, because that would cause problems with the service’s IP bindings. But this is not a problem, just SSH to the host and enter your jail using ezjail-admin console.