Keywords: Linux, IPsec, VPN, security, FreeSWAN, testing

# Testing FreeS/WAN

This document discusses testing FreeS/WAN.

Not all types of testing are described here. Other parts of the documentation describe some tests:

installation document
  testing for a successful install
configuration document
  basic tests for a working configuration
web links document
  General information on tests for interoperability between various IPsec implementations. This includes links to several test sites.
interoperation document.
  More specific information on FreeS/WAN interoperation with other implementations.
performance document
  performance measurements

The test setups and procedures described here can also be used in other testing, but this document focuses on testing the IPsec functionality of FreeS/WAN.

## Testing with User Mode Linux

User Mode Linux allows you to run Linux as a user process on another Linux machine.

As of 1.92, the distribution has a new directory named testing. It contains a collection of test scripts and sample configurations. Using these, you can bring up several copies of Linux in user mode and have them build tunnels to each other. This lets you do some testing of a FreeS/WAN configuration on a single machine.

You need a moderately well-endowed machine for this to work well. Each UML wants about 16 megs of memory by default, which is plenty for FreeS/WAN usage. Typical regression testing only occasionally uses as many as 4 UMLs. If one is doing nothing else with the machine (in particular, not running X on it), then 128 megs and a 500MHz CPU are fine.

Documentation on these scripts is here. There is also documentation on automated testing here.

## Configuration for a testbed network

A common test setup is to put a machine with dual Ethernet cards in between two gateways under test. You need at least five machines; two gateways, two clients and a testing machine in the middle.

The central machine both routes packets and provides a place to run diagnostic software for checking IPsec packets. See next section for discussion of using tcpdump(8) for this.

This makes things more complicated than if you just connected the two gateway machines directly to each other, but it also makes your test setup much more like the environment you actually use IPsec in. Those environments nearly always involve routing, and quite a few apparent IPsec failures turn out to be problems with routing or with firewalls dropping packets. This approach lets you deal with those problems on your test setup.

What you end up with looks like:

### Testbed network

          subnet a.b.c.0/24
                 |
          eth1 = a.b.c.1
             gate1
          eth0 = 192.168.p.1
                 |
                 |
          eth0 = 192.168.p.2
             route/monitor box
          eth1 = 192.168.q.2
                 |
                 |
          eth0 = 192.168.q.1
             gate2
          eth1 = x.y.z.1
                  |
           subnet x.y.z.0/24

    Where p and q are any convenient values that do not interfere with other
    routes you may have. The ipsec.conf(5) file then has, among other things:

    conn abc-xyz
          left=192.168.p.1
          leftnexthop=192.168.p.2
          right=192.168.q.1
          rightnexthop=192.168.q.2

Once that works, you can remove the "route/monitor box", and connect the two gateways to the Internet. The only parameters in ipsec.conf(5) that need to change are the four shown above. You replace them with values appropriate for your Internet connection, and change the eth0 IP addresses and the default routes on both gateways.

Note that nothing on either subnet needs to change. This lets you test most of your IPsec setup before connecting to the insecure Internet.

### Using packet sniffers in testing

A number of tools are available for looking at packets. We will discuss using tcpdump(8), a common Linux tool included in most distributions. Alternatives offerring more-or-less the same functionality include:

Ethereal
  Several people on our mailing list report a preference for this over tcpdump.
windump
  a Windows version of tcpdump(8), possibly handy if you have Windows boxes in your network
Sniffit
  A linux sniffer that we don't know much about. If you use it, please comment on our mailing list.

See also this index of packet sniffers.

tcpdump(8) may misbehave if run on the gateways themselves. It is designed to look into a normal IP stack and may become confused if you ask it to display data from a stack which has IPsec in play.

At one point, the problem was quite severe. Recent versions of tcpdump, however, understand IPsec well enough to be usable on a gateway. You can get the latest version from tcpdump.org.

Even with a recent tcpdump, some care is required. Here is part of a post from Henry on the topic:

    > a) data from sunset to sunrise or the other way is not being
    > encrypted (I am using tcpdump (ver. 3.4) -x/ping -p to check
    > packages)

    What *interface* is tcpdump being applied to?  Use the -i option to
    control this.  It matters!  If tcpdump is looking at the ipsecN
    interfaces, e.g. ipsec0, then it is seeing the packets before they are
    encrypted or after they are decrypted, so of course they don't look
    encrypted.  You want to have tcpdump looking at the actual hardware
    interfaces, e.g. eth0.

    Actually, the only way to be *sure* what you are sending on the wire is to
    have a separate machine eavesdropping on the traffic.  Nothing you can do
    on the machines actually running IPsec is 100% guaranteed reliable in this
    area (although tcpdump is a lot better now than it used to be).

The most certain way to examine IPsec packets is to look at them on the wire. For security, you need to be certain, so we recommend doing that. To do so, you need a separate sniffer machine located between the two gateways. This machine can be routing IPsec packets, but it must not be an IPsec gateway. Network configuration for such testing is discussed above.

Here's another mailing list message with advice on using tcpdump(8):

    Subject: RE: [Users] Encrypted???
       Date: Thu, 29 Nov 2001
       From: "Joe Patterson" <jpatterson@asgardgroup.com>

    tcpdump -nl -i $EXT-IF proto 50

    -nl tells it not to buffer output or resolve names (if you don't do that it
    may confuse you by not outputing anything for a while), -i $EXT-IF (replace
    with your external interface) tells it what interface to listen on, and
    proto 50 is ESP.  Use "proto 51" if for some odd reason you're using AH, and
    "udp port 500" if you want to see the isakmp key exchange/tunnel setup
    packets.

    You can also run `tcpdump -nl -i ipsec0` to see what traffic is on that
    virtual interface.  Anything you see there *should* be either encrypted or
    dropped (unless you've turned on some strange options in your ipsec.conf
    file)

    Another very handy thing is ethereal (http://www.ethereal.com/) which runs
    on just about anything, has a nice gui interface (or a nice text-based
    interface), and does a great job of protocol  breakdown.  For ESP and AH
    it'll basically just tell you that there's a packet of that protocol, and
    what the spi is, but for isakmp it'll actually show you a lot of the tunnel
    setup information (until it gets to the point in the protocol where isakmp
    is encrypted....)

## Verifying encryption

The question of how to verify that messages are actually encrypted has been extensively discussed on the mailing list. See this thread.

If you just want to verify that packets are encrypted, look at them with a packet sniffer (see previous section) located between the gateways. The packets should, except for some of the header information, be utterly unintelligible. The output of good encryption looks exactly like random noise.

A packet sniffer can only tell you that the data you looked at was encrypted. If you have stronger requirements -- for example if your security policy requires verification that plaintext is not leaked during startup or under various anomolous conditions -- then you will need to devise much more thorough tests. If you do that, please post any results or methodological details which your security policy allows you to make public.

You can put recognizable data into ping packets with something like:

            ping -p feedfacedeadbeef 11.0.1.1

"feedfacedeadbeef" is a legal hexadecimal pattern that is easy to pick out of hex dumps.

For other protocols, you may need to check if you have encrypted data or ASCII text. Encrypted data has approximately equal frequencies for all 256 possible characters. ASCII text has most characters in the printable range 0x20-0x7f, a few control characters less than 0x20, and none at all in the range 0x80-0xff. 0x20, space, is a good character to look for. In normal English text space occurs about once in seven characters, versus about once in 256 for random or encrypted data.

One thing to watch for: the output of good compression, like that of good encryption, looks just like random noise. You cannot tell just by looking at a data stream whether it has been compressed, encrypted, or both. You need a little care not to mistake compressed data for encrypted data in your testing.

Note also that weak encryption also produces random-looking output. You cannot tell whether the encryption is strong by looking at the output. To be sure of that, you would need to have both the algorithms and the implementation examined by experts.

For IPsec, you can get partial assurance from interoperability tests. See our interop document. When twenty products all claim to implement 3DES, and they all talk to each other, you can be fairly sure they have it right. Of course, you might wonder whether all the implementers are consipring to trick you or, more plausibly, whether some implementations might have "back doors" so they can get also it wrong when required.. If you're seriously worried about things like that, you need to get the code you use audited (good luck if it is not Open Source), or perhaps to talk to a psychiatrist about treatments for paranoia.

## Mailing list pointers

Additional information on testing can be found in these mailing list messages:

- a user's detailed setup diary for his testbed network
- a FreeS/WAN team member's notes from testing at an IPsec interop "bakeoff"