Building a FreeBSD OpenVPN Firewall

Appliance
PF, IPFW Dummynet, OpenVPN w/ Active Directory Authentication, Nagios and
NTOP

By: Ian Evans

Version 5
 Document History

Version Title Date Author

1 Draft – Preliminary 01/01/08 Ian Evans
2 Core – Production 02/01/08 Ian Evans
3 Mods, Illustrations and 05/01/09 Ian Evans
Additional Material
4 Additional Illustrations 11/01/09 Ian Evans
5 Major revisions, added 03/16/10 Ian Evans
FreeBSD 8 support

Document Contributors and Authors

Title Name
Author Ian Evans
 Disclaimer: Use this guide at your own risk! I accept no responsibility for
damages that may occur.

This guide comes after many painful hours getting everything setup just right. The most
challenging part of this process will most likely be the problems encountered with missing or
broken packages in the ports system. I found that using a combination of ports and pkg_add
seemed to do the trick. Be patient, because you will find the end product to be one of the most
stable, secure and reliable firewalls you will ever use.

Recommended hardware

Server systems. I always had good luck with the Compaq/HP DL380 or DL360 systems. FreeBSD
supports all of the components and installs very easily on these systems.
Processor. Try to use a dual-core or higher. You can buy one for under $100.00 now, so there is no
reason to go cheap on the processor. As services are added to the appliance, you will be glad you
invested in a decent CPU.
Memory. Try to use registered memory if possible. This type of memory has better recovery
mechanisms built in and is designed for 24x7 environments. A good 2-4GB of DDR2 or DDR3 of
registered dual channel memory will do the trick.
Motherboard and Chipset. SuperMicro makes some great server boards for those that want to
built your own. Cheap desktop boards will work, but may not live up to the requirements of a 24x7
operation. FreeBSD works well with pretty much any Intel, AMD or Nvidia chipset. I personally had
very good luck with the Nforce series chipsets. Pay very close attention to those integrated NIC's
on the motherboards... some of them are not supported, especially if they are new.
Network Card. You will need three Gigabit network cards (the WAN port can be 100Mbps). Intel
Pro1000's (e1000 driver) are good network cards and are widely supported. I have also used
Syskonnect and Nvidia Nforce network cards with great success. Most systems have two
integrated Gigabit NIC's, so one add-on card may be all that is required.
RAID. There are many options out there (software vs hardware, etc). I always recommend true
hardware RAID. The card is a dedicated resource and does not require any software on the O/S to
build or maintain the RAID set. Get a decent card that has at least 256MB of cache, a battery
backup (BBU) and runs on at least a PCI-X 133 or PCIe 8x bus. Most decent cards you can buy now
will run on a PCIe bus, which will give a significant performance boost over legacy PCI buses.
Hard Disks. SATA is getting much better, but I still recommend 10-15K SAS for most 24x7
environments. You get a substantial performance boost and these drives have a higher MTBF
rating/Warranty. Try to get at least 4 drives so you can create a RAID1+0 array. This will give you
the best blend of performance and reliability without costing too much. If you must go with SATA
drives, make sure they are enterprise grade (examples: WD RE3, WD Raptor, Seagate ES.2).

Preconfiguration Steps

BIOS. FreeBSD can be a little picky about ACPI at times so make sure your BIOS is up to the latest
revision. If you have updated and you experience problems during the beginning of the
installation, try to start the installation without ACPI enabled (there will be an option for this at the
initial boot menu).
RAID Controller. When creating your hardware RAID set, it may be a good idea to add a hot-
swap-spare. You also want to make sure you have enabled write-back cache and the battery
backup unit (BBU) prior to completing the RAID configuration.

Download and install FreeBSD 8.0

- Download the FreeBSD 8.0 x86_64 edition ISO from FreeBSD.org and burn the ISO image to a CD
or DVD.
- Select the standard installation and install the entire ports tree. This makes finding a package
easier at a later time.
- Select the default partitioning options. This can be accomplished by selecting "A" for auto and
then "Q" to save. FreeBSD fixed the limitation with the /var partition a couple releases ago, so auto
configuration is going to be best.
- You only need to set your static management IP for the internal network interface at this time.
Leave the other two cards alone for the moment... you will configure them later.

When prompted, reboot your machine:

REBOOT

Update the your ports using the portsnap tool. This tool is now installed by default in
newer BSD versions. Alternatively, you can use dpkg to install the ports you need as
well.

portsnap fetch && portsnap extract

You will now start to create and customize your own kernel file. Let's start by copying
the new kernel file for editing:

cd /usr/src/sys/amd64/conf/ && cp GENERIC ENHANCED

Edit your newly created ENHANCED KERNEL and add the options exactly as shown
below. You may also want to remove some of the unused devices in the new kernel file
as well. Just add a “#” in front of any device you wish to exclude from the new build
and it will omit the driver. Be very careful which devices you exclude! Your system may
fail to boot if you are not careful!

Important note!:The options below need to be in the same area as the rest of the
options in your kernel file.

edit /usr/src/sys/amd64/conf/ENHANCED

# Add PF Firewall Support

device pf
# Add PF Logging Support
device pflog
# Add PF firewall synchronization capability for CARP. We will use this later for redundancy.
device pfsync
# Add in the options for IPFW / Dummynet (This will be used to control downloads, download
queues,etc)
options IPFIREWALL
options IPFIREWALL_VERBOSE
options IPFIREWALL_VERBOSE_LIMIT
options IPFIREWALL_DEFAULT_TO_ACCEPT
options DUMMYNET
# Tune kernel heartbeat. This will help overall performance, but will come at cost of memory &
CPU.
options HZ=1000
# Add in ALTQ options for PF. I only use CBQ, but it is nice to have the other options if needed.
options ALTQ
options ALTQ_CBQ
options ALTQ_RED
options ALTQ_RIO
options ALTQ_HFSC
options ALTQ_CDNR
options ALTQ_PRIQ
# Add SMP Support to ALTQ
options ALTQ_NOPCC

ctrl+[ to save.

Now that all of your options have been added, let's build and install the updated
kernel. Depending on your processor configuration, this may take a considerable
amount of time.

cd /usr/src/ && make buildkernel KERNCONF=ENHANCED && make installkernel

KERNCONF=ENHANCED

FreeBSD does a great job out of the box tuning network and kernel performance, but
we need to further optimize the system for high volumes of traffic. Sysctl allows you to
fine tune kernel parameters instantly. Consider sysctl similar to the registry editor, but
much more useful. Changes take right after the command is applied.
Add the following parameters into /etc/sysctl.conf to everything sticks at boot:

edit /etc/sysctl.conf

net.inet.tcp.rfc1323=1
kern.ipc.maxsockbuf=16777216
net.inet.tcp.sendspace=1048576
net.inet.ip.intr_queue_maxlen=1000
net.inet.ip.dummynet.hash_size=256
kern.ipc.maxsockbuf=900000000
net.inet.tcp.sendspace=300000000
net.inet.tcp.recvspace=300000000
net.inet.udp.recvspace=300000000
net.inet.tcp.recvspace=1048576

ctrl+[ to save.

Install OpenVPN from ports. You can also use dpkg to do this as well.

cd /usr/ports/security/openvpn && make install clean

Edit and save new configuration to openvpn.conf file.

edit /usr/local/etc/openvpn/openvpn.conf

# Specify device
dev tun

# Server and client IP and Pool

server x.x.x.x 255.255.255.0
ifconfig-pool-persist /usr/local/etc/openvpn/ipp.txt
proto udp

# Tell OpenVPN to auth to your Windows AD server

plugin /usr/local/lib/openvpn-auth-pam.so openvpn

# Certificates for VPN Authentication

ca /usr/local/etc/openvpn/keys/ca.crt
cert /usr/local/etc/openvpn/keys/server.crt
key /usr/local/etc/openvpn/keys/server.key
dh /usr/local/etc/openvpn/keys/dh1024.pem
# Have OpenVPN check for revoked certs
crl-verify /home/<yourdir>/keys/crl.pem

# Routes to push to the client

push "route x.x.x.x x.x.x.x.x"
push "route x.x.x.x x.x.x.x"
# Routes for individual hosts may be required if someone chose to use a common network that
exists on the client and server side.
#push "route x.x.x.x 255.255.255.255"

# Uncomment if you want DNS pushed to clients. Use with caution!

#push "dhcp-option DNS x.x.x.x"

# Use compression on the VPN link

comp-lzo

# Duplicate CN's allowed

duplicate-cn

# Make the link more resistant to connection failures

keepalive 10 86400
persist-tun
persist-key

# Run OpenVPN as a daemon and drop privileges to user/group nobody user nobody
group nobody
daemon

reneg-sec 0

ctrl+[ to save.

Create directory for CA and adjust permissions:

cp -r /usr/local/share/doc/openvpn/easy-rsa /home/<youraccount>/
cd /home/<youraccount>/easy-rsa/
chmod -R 775 /home/<youraccount/easy-rsa/

Now build the CA and keys. Answer all of the questions carefully! Please note that you
will need to switch shells to ensure these commands work. After you are complete, you
may switch back C Shell by typing in "csh".

1) sh
2) . vars
3) ./clean-all
4) ./build-ca
5) ./build-key-server server
6) ./build-key user1
7) ./build-dh
8) cp -r keys /usr/local/etc/openvpn/

Edit syslog to log OpenVPN requests:

edit /etc/syslog.conf:
!openvpn
*.* /var/log/openvpn.log

ctrl+[ to save.

Edit the OpenVPN startup to allow OpenVPN service to start when the system boots:

edit /usr/local/etc/rc.d/openvpn
Change enable from "NO" to "YES":

Now we are going to ensure all of the necessary services and options start
automatically. FreeBSD labels network cards according to manufacturer, so you will
need to substitute your interface name in place of the default names I have added (e.g.
bge0 may be sk0 on your system).

edit /etc/rc.conf

# Set your interface IP's (use whatever IP's you want)

# Internal network interface
ifconfig_bge0="inet x.x.x.x netmask x.x.x.x"
# DMZ network interface
ifconfig_bge2="inet x.x.x.x netmask x.x.x.x"
# WAN Interface
ifconfig_bge1="inet <your internet IP here> netmask 255.255.255.0"
# Enable BIND forwarder at boot
named_enable="YES"
# Enable PF at boot
pf_enable="YES"
# Enable pf logging
pflogd_enable="YES"
# Ensure your BSD box is set to gateway mode
gateway_enable="YES"
# SSHv2 access
sshd_enable="YES"
# Enable additional security logging at boot
accounting_enable="YES"
# Fix bug in PF with Active/Passive Proxy requests
ftpproxy_enable="YES"
ftpproxy_flags="-r"
# Enable OpenVPN on boot
openvpn_enable="YES"
# Enable ipfw/dummynet
firewall_enable="YES"
firewall_script="/usr/local/etc/ipfw.rules"

ctrl+[ to save.

Install and configure BIND to handle internal DNS requests:

cd /usr/ports/dns/bind9 && make install clean

Rename existing named.conf and create a new file with content below:

mv /var/named/etc/namedb/named.conf /var/named/etc/namedb/named.old
edit /var/named/etc/namedb/named.conf
options {
directory "/etc/namedb";
pid-file "/var/run/named/pid";
dump-file "/var/dump/named_dump.db";
statistics-file "/var/stats/named.stats";
forwarders { x.x.x.x; };
forward only;
};

zone "." in {
type hint;
file "db.cache";
};

zone "0.0.127.in-addr.arpa" in {
type master;
file "db.127.0.0";
};

ctrl+[ to save.

Install ftp-proxy for PF. This is a workaround for the PF FTP bug:

cd /usr/ports/ftp/ftp-proxy && make install clean

Configure DUMMYNET for additonal traffic shaping. As I mentioned earlier, this will be
used in conjunction with PF ALTQ to ensure bidirectional QoS. You can use my example
below as a starting point:

edit /usr/local/etc/ipfw.rules

IPF="ipfw -q add"
ipfw -q -f flush
ipfw pipe 1 config mask src-ip 0x00000000 bw 10Mbit/s
ipfw add 10 pipe 1 all from any to any xmit bge1
#Add any deny rules in here:
ipfw add 1200 allow all from any to any
# Block spammer net (add whatever net's you want in here)
ipfw add 1300 deny ip from x.x.x.x/xx to any
ipfw add 65535 deny all from any to any

ctrl+[ to save.

Load the rules:

sh /usr/local/etc/ipfw.rules

Verify DUMMYNET is functioning properly:

ipfw pipe show

You should now see queuing for your internal hosts. Everything will now be placed into
bit buckets.
Now let's configure the main PF component. You will need to adjust the interface
names to reflect what you have installed in your server (e.g. sk0, eth0, etc):

edit /etc/pf.conf

### Define Interfaces ####

# External Interface
ext_if = "bge2"
# DMZ Interface
#dmz_if = "bge1"
# Internal Interface - All internal traffic and nets
int_if = "bge0"
# VPN Interface - OpenVPN tunnel for VPN clients
vpn_if="tun0"

### Macros. These save significant time if you are creating more than a couple rules. ###
# Private nets
privnets = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }"

# Log all external traffic

set loginterface $ext_if

### Optimization options. These options are useful for fine tuning firewall performance. As seen
below, these options are essential to protect the firewall from DDoS, etc ###

# Protect firewall from buffer overflow and DDOS attacks

set limit { frags 30000, states 25000 }
# Default firewall optimization level - Set to conservative to ensure proper state handling
set optimization conservative

# Normalize and reassemble traffic on interfaces - Enable this on all interfaces to ensure all
packets are normalized. This will help protect against certain types of frag attacks.
scrub in all

### ALTQ - Traffic Shaping ###

# Traffic shaping on external interface (Create as many queues as you need here)
altq on $ext_if priq bandwidth 2Mb queue {web, misc}
queue web priority 7
queue misc priority 1 priq(default)

### NAT ###

# Enable NAT
nat on $ext_if from $int_if:network to any -> ($ext_if) round-robin sticky-address

# Fix PF FTP NAT Issue

nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"
rdr on $int_if proto tcp from any to any port 21 -> 127.0.0.1 port 8021

# Redirection rules for internal hosts

# RDR SSL request to DMZ Web Host

rdr on $ext_if proto tcp from any to any port 443 -> x.x.x.x port 443

### Tables ###

# Add explicitly denied hosts into the following tables

table <spammers> persist file "/etc/spammers"
table <spamhaus> persist file "/tmp/drop.lasso"

### RULES ###

# Block all connections by default

block log all

# Allow all traffic to loopback interface. Blocking this is useless.

pass quick on lo0 all

# Allow all traffic on internal interface. This can be restricted if needed.

pass quick on $int_if all

# Allow VPN traffic to all internal interfaces. This can be restricted if needed.
pass quick on $vpn_if all

# Anchor for ftp-proxy

anchor "ftp-proxy/*"

# Prevent hijacking on external and internal interfaces

antispoof for $int_if
antispoof for $ext_if

# Block all connections to external interfaces from bogus nets

block drop in on $ext_if from $privnets to any
block drop in on $ext_if from any to $privnets

# Pass HTTPS
pass in on $ext_if inet proto tcp from any to any port 443 modulate state queue web
# Passive mode shim for FTP issues with PF
pass in on $ext_if inet proto tcp from any port 20 to $ext_if port 55000 >< 57000 user proxy
modulate state queue web
# Webmin access ##DO NOT ENABLE! DEBUGGING ONLY!##
#pass in on $ext_if inet proto tcp from any to any port 10000 flags S/SA keep state queue web
# NTOP stats for BSD firewall ##DO NOT ENABLE! DEBUGGING ONLY!##
#pass in on $ext_if inet proto tcp from any to any port 3000 flags S/SA keep state queue web
# Allow OpenVPN traffic from any net - OpenVPN client must have 2 factor auth to complete the
connection!
pass in on $ext_if proto udp from any to port 1194 modulate state queue misc

#### Basic rules ####

# Block all ICMP traffic regardless of source

block in on $ext_if inet proto icmp all
# Drop connections from spammers instantly
block in on $ext_if from { <spammers>, <spamhaus> } to any
# Pass out all traffic on external interface
pass out on $ext_if keep state queue (web, misc)

ctrl+[ to save.

Now let's see how PF is working:

pftl -s all | more

Create spammers and drop lists for PF. Download the latest drop.lasso from Spamhaus
and paste contents into /tmp/lasso once you have created it as shown below:

touch /etc/spammers
touch /tmp/drop.lasso

Install NTOP and PFTOP for monitoring purposes. These two utilities are great to
monitor firewall status and ensure everything is secure:

cd /usr/ports/sysutils/ntop && make install clean

cd /usr/ports/sysutils/pftop && make install clean

Open NTOP and you should now see traffic statistics:

Get OpenVPN to authenticate to your Active Directory or LDAP server. Add the
following:

edit /etc/pam.d/openvpn

auth required pam_radius.so no_warn try_first_pass

ctrl+[ to save.
Install and configure IAS on your Active Directory Server:

1) Install IAS (Internet Authentication Service) on your Active Directory Server through add remove
programs.
2) Open IAS mmc console.
3) Right click on Internet Authentication Service (Local).
4) Right click RADIUS Clients > New Radius Client > Enter your Friendly Name > Enter your Client
Address (IP or DNS) > Next.
5) Client-Vendor "Radius Standard" > Enter your shared secret > Finish.
6) Right click on Remote Access Policy > New > Remote Access Policy > Next > Setup a custom
policy > Policy name: openvpn > Next > Policy conditions > Add > Select "NAS Identifier" > Add >
Type a word or wildcard "OpenVPN" > Grant Remote Access Permission > Next > Edit Profile >
Ensure only "Unencrypted Authentication" is selected > Apply.
7) Create user accounts in Active Directory. Be sure to enable these users in Remote/Dial-In
access!

Restart the FreeBSD server and verify services and connectivity:

REBOOT

Verify OpenVPN is up and running:

ps -aux | grep openvpn

Client configuration. The user will need their keys in proper directory for the
connection to work. Use something like Filezilla to securely transfer the necessary files
to your client OpenVPN keys directory. You will need the .crt, .key and .conf file in this
location for things to work properly. Now, let's create the client configuration file
(openvpn.conf):

touch /etc/openvpn/openvpn.conf

client
# Tunnel mode opposed to tap
dev tun
# Use UDP instead of TCP/IP
proto udp
# Port 1194 is the default. If you used a different port on the server, change it here
remote xxx.xxxxxx.com 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
cert ian.crt
key ian.key
# Require password auth along side of the certs
auth-user-pass
reneg-sec 0
ns-cert-type server
comp-lzo
verb 3
route-method exe
# This will fix some issues with Windows clients and adding static routes
route-delay 2
# Adjust keepalives to stop annoying timeouts
keepalive 10 86400
ping-timer-rem
Pending some minor adjustments, you should now have a fully functional FreeBSD
firewall appliance.