DO NOT REPRINT

FORTINET

Basic IPsec VPN

In this lesson, we will show you how to set up site-to-site IPsec VPN.
VPNs are heavily used in todays IT infrastructure to join private corporate networks across the Internet.
IPsec is an RFC standard. Whether you have FortiGate devices only or mix in another vendors devices,
the principles are essentially the same.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

After completing this lesson, you should have these practical skills that you can use to set up a simple
IPsec tunnel for a site-to-site VPN.
During this, we will explain how to choose between configuring a policy-based or route-based VPN. You
will also learn how to verify the status of each tunnel.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

A Virtual Private Network (VPN) allows people in remote places separated by the Internet to securely
access resources on your local network. For example, if workers are traveling or working from home,
you can use a VPN to give LAN access to them. You can also use a VPN to interconnect multiple
campuses.
There are multiple types of VPN: PPTP, L2TP, SSL VPN, and IPsec are popular choices.
PPTP is fast, but security is weak, and easily defeated.
IPsec requires a gateway or installation of client software. So it is more complicated to set up for
mobile users than SSL VPN, where they can simply utilize their web browser instead.
SSL VPN is designed for tunnels between a single client and a LAN, not between entire offices.
Because of this, many networks now use a combination of SSL VPN for mobile user access
and Ipsec or L2TP for tunnels between offices.
Often, tunnel is used as a synonym for VPN, although not all VPNs technically are tunnels, as we will
see in a minute.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

When should you use IPsec? What is it?

It is a vendor-neutral standard set of protocols used to join two physically distinct LANs, as if they were a
single logical LAN, despite being separated by the Internet.
In theory, RFC 2409 and 4305 do support null encryption that is, you can make VPNs which not
encrypt traffic. The RFCs also support null data integrity. But does that provide any advantages over
plain traffic? No. No one can trust traffic that may have had an attack injected by an attacker. Rarely do
people want data sent by an unknown person. Most people also want private network data, such as
credit card transactions and medical records, to remain private.
So in reality, regardless of vendor, IPsec VPNs almost always have settings for 3 important benefits:
Authentication, to verify the identity of at least the initiator (and sometimes also the
responder);
Data integrity, or HMAC, to prove that encapsulated data has not been tampered with as it
traverses a potentially hostile network;
Confidentiality, or encryption, to ensure that only the intended recipient can read the message.
And, of course VPNs have virtual routing and network settings to use when joined to the remote LAN.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

When we say the IPsec protocol, what layers & protocols are we talking about?
IPsec injects itself above the third layer: IP. Whats encapsulated? It depends on the mode. IPsec
can operate in two modes: transport mode, or tunnel mode.
Transport mode directly encapsulates what would usually be the fourth layer (TCP transport, for
example) and above.
Once the IPsec encapsulation is removed, there is no additional routing layer left. Thats why its also
called direct peer-to-peer or client-to-client. So this mode is not technically a tunnel, even though
many people use the word VPN and tunnel interchangeably. (Tunneling technically means
encapsulating an IP packet inside another IP packet.) Transport mode does not traverse NAT well
especially carrier-grade symmetric NAT and depending on the case, may require NAT Traversal,
ALG or hole punching, or may not work. This is because port numbers are inside the encrypted ESP
payload.
Tunnel mode is a true tunnel. Encapsulation first adds a second IP layer, then the original transport
layer (TCP, UDP, etc.). The second IP layer contains a private network that is routable on the remote
network. Once the IPsec packet reaches the remote LAN, and is unwrapped, the packet can
continue on its journey.
To fit an IPsec packet into the frame, when FortiGate applies ESP, one payload may be split in order to
fit into two packets. So you dont need to adjust frame MTU. But this does mean that you might need
more bandwidth for VPN traffic.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

Lets look at the 2 methods of encapsulation: Which should you choose? Why might some extra
bandwidth be needed? Why is NAT traversal necessary?
Blue underlined parts of each packet are additional bits that are required by ESP. It varies by transport
vs. tunnel mode.
Relative to a non-IPsec packet, notice that the green Layer 4 transport area of the frame is now shorter.
Remember, the 1500 byte default frame MTU has not changed. Payload length is variable, and filled with
padding. So this doesnt always matter. But if the additional ESP bits cause the packet payload to not fit,
then FortiGate must split the payload into multiple frames. IKE is in separate packets, too, and also
requires additional bits to be transmitted.
You are trading some bandwidth for:
Security and,
Routability (in the case of tunnel mode)
Notice that after you remove the VPN-related headers, a transport mode packet cant be transmitted any
further it has no second IP header inside. So its not routable.
Thats OK if the packet is decrypted at an endpoint such as the FortiGate itself (think of encrypted Syslog
tunnels, and some special cases such as multicast, GRE-IPSec and L2TP-IPSec for Windows/Android
clients), but not usually if there are more router hops until the packet reaches its destination. For those
purposes, youll need tunnel mode instead.
Notice, too, that TCP or UDP port numbers are inside the ESP payload. They will be encrypted. So NAT
cant rewrite them for port forwarding or port overloading.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

Because encapsulation styles and other settings vary, and any mismatches cause VPNs to fail, starting
with FortiOS 5.2, there are VPN templates.
You can use these to simplify VPN setup reducing the guesswork about what settings are compatible
between devices.
But sometimes you may need to create a tunnel manually, or pass it though a NAT device. So lets show
you how.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

If youre passing your VPN through NAT devices such as firewalls, it helps to know which protocols to
allow.
Really, IPsec means three separate protocols.
IKE, which is used to authenticate peers, exchange keys, and negotiate the encryption and
checksums that will be used; essentially, it is the control channel,
AH, which is the authentication header the checksums that verify the integrity of the data
ESP, which is the encapsulated security payload the encrypted payload, essentially, the data
channel
So if you need to pass IPsec traffic through another firewall, remember: allowing just 1 protocol or port
number is not enough.
Note that although the IPsec RFC mentions AH, it does not offer encryption, an important benefit. So it is
not used by FortiGate. As a result, you dont need to allow IP protocol 51.
To make a VPN, configure matching settings on both ends whether the VPN is between 2 FortiGates,
or between a FortiGate and FortiClient, or between a 3rd party device and a FortiGate. If the settings
dont match, tunnel setup will fail.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

Lets talk about how FortiGate starts an IPsec tunnel.

If youre creating a custom VPN tunnel, it will help you to understand which settings to use, and how
tunnels work.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

On FortiGate, there are two ways a packet can initiate an IPsec VPN: by matching a route, or by
matching a policy. (In our old documentation, route-based used to be called interface-based, and
policy-based used to be called tunnel-based.)
How do you know when to use policy-based or routed-based?
Generally, try to use route-based. It offers more flexibility and control. We can implement very complex
routing scenarios, such as where tunneled traffic is required to be routed with policy-based routing, or if
you require GRE-over-IPsec.
In comparison, policy-based VPNs must be used when the FortiGate is in transparent mode, or if the
other peer requires L2TP-over-IPsec.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

In addition to different limitations, how to configure them is different.

In a route-based VPN, FortiGate automatically adds a virtual interface with that name. Two firewall
policies with the action ACCEPT are usually required: one for sessions originating on the local
network, and another for sessions from the remote network. You also need to route the VPN traffic to
the virtual network interface. (Usually, youll use a static route.)
In a policy-based VPN, only one firewall policy with the action IPSEC is required. The policy is
bidirectional. By default, the GUI hides policy-based VPNs. To show policy-based VPN settings, use
the CLI setting set gui-policy-based-ipsec enable.
Both sides of your VPN dont need to be configured in the same route-based or policy-based
mode. You can configure one peer as routed-based, and the other as policy-based. But the Phase 1 and
2 settings must match.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

If you have a simple case like the site-to-site scenario in this lesson use the VPN wizard.
But if you need to tailor your VPN settings, you can still make a custom VPN.
When making a route-based VPN, one additional step is usually required: you must also create a route
to direct VPN traffic to the new virtual interface for IPsec. (If you use the wizard, though, this is done
automatically.)

DO NOT REPRINT
FORTINET

Basic IPsec VPN

When the VPN wizard is completed, FortiGate automatically creates many of the required objects:
Addresses and address groups
Static routes
Policies
Phase 1 and Phase 2 settings
To immediately check the status of your tunnel, click Show Tunnel List. This can be your first test of
whether your VPN is working.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

How does FortiGate bring up a VPN?

Lets begin by talking about Internet Key Exchange also called IKE Phase I.
This is when each endpoint of the tunnel the initiator and the responder connect and begin to set up
the VPN.
When they first connect, the channel is not secure yet. An attacker in the middle could intercept
unencrypted keys. And both ends have no strong guarantee of each others identity, either. So how can
they exchange sensitive private keys?
They cant. First, both ends have to create a temporary secure channel. Theyll use this to protect strong
authentication, and negotiate the real keys for the real tunnel later. Lets show how this works.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

(slide uses animation)

This is Phase 1, where peers say hello and create an IKE SA that defines a temporary secure channel.
(click)
What is an SA?
A security association is simply the algorithms and parameters used to encrypt and authenticate data
between 2 points. Settings must agree. Otherwise the Phase 1 will fail. (Each side wouldnt be able to
decrypt or authenticate traffic from the other.) As you can see, which settings are used can be inflexible
what we call aggressive mode or somewhat flexible what we call main mode. Details are in the
advanced IPsec lesson.
(click)
In Phase 1, FortiGate IKE SAs are a secure channel that are used for:
The Diffie-Hellman keys that will be used by Phase 2, and
To build the final ESP tunnels.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

At the end of Phase I, FortiGate uses the Diffie-Hellman method. It uses the public key (that both
ends know) plus a mathematical factor called a nonce in order to generate a common private key.
This is crucial. With Diffie-Hellman, even if an attacker can listen in to the messages containing
the public keys, they cannot determine the secret key. This is why it works even with a weakly
authenticated IKE channel, where a user name and password and FortiToken have not been
exchanged, for example.
The new private key is used to calculate additional keys: for symmetric encryption and authentication.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

If your VPN must pass through a NAT device, as we mentioned, ESP encryption would normally prevent
the NAT device from being able to read and remap the port numbers inside.
To solve this, Phase I was extended. It added NAT traversal, also called NAT-T. When NAT-T is
enabled in both ends, peers can detect any NAT device along the path. If NAT is found, then:
Both Phase 2 and remaining Phase 1 packets change to UDP port 4500
FortiGate and client encapsulate ESP within UDP port 4500
So if you have two FortiGates that are behind, for example, an ISP modem that has NAT, you will
probably need to enable this setting.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

Once details such as dead peer detection, NAT, and symmetric keys have been determined, your
FortiGate is ready to establish the real SA that is, IPsec SA which defines the ESP channel that will
be used to encapsulate and transmit data through the VPN.
It does this via IKE Phase II.
There can be 1 tunnel for Phase I, but 2 or more tunnels for Phase II. Lets see how.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

Once Phase 1 has established a somewhat secure channel and private keys, Phase 2 begins.
Phase 2 negotiates security parameters for the IPsec SA not to be confused with the IKE SA. It is this
IPsec SA not IKE that ESP will use to transmit data between LANs.
IKE Phase 2 does not end once ESP begins. Phase 2 periodically renegotiates cryptography. This
maintains security. Also, if you enable Perfect Forward Secrecy, each time the Phase 2 session
key expires, FortiGate will use Diffie-Hellman to recalculate a new common secret key. So even if
the same encryption algorithms are selected each time, the ESP tunnel will be changing to use a
different private key, making it much harder for an attacker to crack the tunnel.
Each Phase 1 can have multiple Phase 2. When would this happen?
For example, you may want to use different encryption keys for each subnet whose traffic is crossing the
tunnel. How does FortiGate select which Phase 2 to use? The Quick Mode setting.
Additionally, most traffic is two-way traffic. So this means there are usually two tunnels, and two ESP
SAs: one for each direction.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

During Phase 2, we must configure a pair of settings called Quick Mode Selectors. They identify and
direct traffic to the appropriate Phase 2 if there are multiple.
In other words, it allows granular SAs.
Selectors behave similarly to a firewall policy. VPN traffic must match selectors in one of the Phase 2
SAs. If it does not, the traffic is dropped.
When configuring selectors, specify the source and destination IP subnet that will match each Phase
2. You can also specify the protocol number, and source and destination ports for the allowed traffic.
In point-to-point VPNs, such as when connecting a branch office FortiGate to headquarters
FortiGate, both sides configuration must mirror each other.
Quick mode selectors for dial-up VPNs are different, and details are in the advanced IPsec lesson.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

Once all settings are configured, each time that a host on your local LAN sends a packet where the
destination is on the remote LAN, FortiGate should automatically bring up the VPN tunnel. It should
remain available for some time, as long as the tunnel is being used.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

If you need detailed control of your VPN, such as for IKE version 2, you can still configure it manually.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

If you are configuring a custom VPN, you can start from the wizard. Click Custom VPN Tunnel (No
Template).
Configure the remote FortiGates WAN IP address, and indicate which network interface on this local
FortiGate is the gateway that leads to it. FortiGate will use this to connect to the other end.
If your peers use pre-shared keys for the initial (IKE) authentication, both peers must be configured with
the same pre-shared key. For Phase 1, choose which encryption and authentication to propose, and so
on. They should match, too. If peers cant agree on IKE security, even Phase 1 wont be established. So
if in doubt, make sure Phase 1 and Phase 2 settings on both FortiGates match.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

You already identified the other FortiGates WAN IP (the Remote Gateway), so now also indicate your
local FortiGates WAN IP. Remember: during IKE, each side must have some way to identify its peer so
that it can label the IKE SA.
Once Phase 1 completes, Phase 2 begins. This sets up the ESP tunnels that will be used for actual data
transfer. For each subnet on each end of the VPN, you can specify different levels of ESP security. For
example, connections to the Finance LAN might need larger key sizes and stronger authentication. To
do this, configure multiple Phase 2 entries. For simplicity, here, we show only one Phase 2: the Local
Address is our LAN, and the Remote Address is the remote LAN.
Remember that if traffic doesnt match an IPsec SA, the IPsec engine will drop the packet. Usually,
its more intuitive to filter traffic with firewall policies. So if you dont want to use SA filtering, you can just
set the quick mode selectors to be 0.0.0.0/0.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

If you used the wizard for everything, it would have created routes and policies suitable for a route-based
VPN. What if you, for example, have a FortiGate in transparent mode?
Remember, first, you must enable the GUI to show policy-based IPsec options. Configure your phases
as before, then create a policy. When policy-based VPN settings are visible, an additional Action
setting is available when you configure a policy. Choose IPsec. Then choose the policy-mode tunnel
settings.
If you enable Allow traffic to be initiated from the remote site, you only need to make one policy. It will
govern both directions.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

With a route-based VPN, firewall policies are different.

There are two policies usually, not one.
The interface doesnt match wan1; it matches the virtual interface, which in this example is named
HQ-to-Branch.
The VPN wizard is the easiest way to make these. If you did that, you can skip this step.
But if you want to manually set up a VPN, use these as examples.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

In route-based VPN, you need to route VPN traffic destined for the remote LAN to the IPsec interface. If
you used the wizard, this was created for you, automatically.
(In a policy-based VPN, traffic is routed to wan1 or another external interface instead. Since there is
usually a default route, which routes all non-local packets towards the Internet, thats why policy-based
VPNs can usually skip this step.)
To do this, usually youll add a static route.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

In the GUI, there is a tool to monitor the status of your IPsec VPNs. Through this tool, you can see how
much traffic has passed through each tunnel. You can also start and stop individual tunnels, and get
additional details.
If the tunnel is up, there will be a green arrow appearing next to its name. If it is down or not in use, then
a red arrow is displayed.
For example, here, simply by looking at the remote Gateway column, you can find a misconfiguration
problem: the IP should be an interface on the remote FortiGate, not a subnet IP. So it is impossible to
bring up.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

This example shows 3 different VPN tunnels: Client_VPN, Home_VPN, and Office_VPN.
The phase 1 Office_VPN appears twice because it has two separate phase 2 associated with the same
phase 1. The other VPNs have one Phase 2 per Phase 1.
For each phase 2, we can see the phase 1 name, key life remaining time, status and the quick mode
selectors.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

If your tunnel is not starting, it helps to know the expected behavior. This varies by type.
This outlines the steps. Depending on whether you are creating a route (interface-based) or policy-based
VPN, FortiGate will use a different mechanism.
One common mistake is to configure a policy-based VPN, but to set the action to ACCEPT and this
causes FortiGate to egress clear text packets, not encrypted ones.
Another common mistake is to route eggressing packets to the wrong port. Remember, route-based
VPNs must egress through the virtual interface, not the WAN.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

Like with any feature, IPsec uses some system resources. Requirements vary by the number of VPNs.
Strong cryptography involving large key sizes can increase resource usage noticeably. Many models of
FortiGate have specialized FortiASIC chips to increase IPsec cryptographic performance, so especially if
you have many tunnels simultaneously, check that your configuration offloads cryptography to these
chips where possible. In some cases, you may be able to offload incoming traffic to one ASIC, and
outgoing traffic to another ASIC.
Details are in the hardware acceleration lesson.

DO NOT REPRINT
FORTINET

Basic IPsec VPN

To review, these are the topics weve talked about. We presented an overview of the IPsec technology,
which includes Internet Key Exchange, phase 1, phase 2, Diffie-Hellman and Quick Mode Selectors. We
also showed the difference between policy-based and route-based VPNs, and how to use the VPN
monitor.