# How can I connect to an IPSec VPN hosted by OS X Server?

## c00l.wave

At our company we use OS X Server 10.4 (OS X enhanced with pre-configured opensource packages for server-use). Since PPTP is considered insecure (and our Linksys router is unable to forward GRE protocol) we can only use L2TP/IPSec as VPN. That works just fine with Mac clients but since all Mac clients use (a modified?) racoon there must be a way to connect Linux machines to that server.

I compiled IPSec-support, XFRM_TUNNEL (+ MODE_TUNNEL and MODE_TRANSPORT) and crypto modules into my gentoo 2.6.19-r5 kernel, copied the racoon config from my iBook and tried to bring it up using racoonctl vpn-connect external - no success. Racoon (ipsec-tools 0.6.3) keeps telling me: (IPs are removed; internal is my machine, external the VPN server)

```

2007-02-22 11:46:29: INFO: accept a request to establish IKE-SA: external

2007-02-22 11:46:29: INFO: initiate new phase 1 negotiation: internal[500]<=>external[500]

2007-02-22 11:46:29: INFO: begin Identity Protection mode.

2007-02-22 11:46:30: INFO: received Vendor ID: KAME/racoon

2007-02-22 11:46:30: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02

2007-02-22 11:46:30: INFO: received Vendor ID: KAME/racoon

2007-02-22 11:46:30: ERROR: ignore the packet, received unexpecting payload type 130.

```

Then the last 2 lines repeat until timeout: (hex removed)

```

2007-02-22 11:47:03: ERROR: phase1 negotiation failed due to time up. hex:hex

```

Killing racoon on my OS X client and running it verbose in foreground, I can see racoon should print something like "NAT detected, switching to port 4500", "ISKMP-SA established" and "respond new phase 2 negotiation" at that point. The only result I can find searching on Google is from July 2004 and suggests to try "FEATURES += nat_traversal" or switch to a newer version. As that post is 2 1/2 years old I assume the ebuild makes use of that feature. In fact, racoon prints "INFO: internal[500] used for NAT-T" on startup and during emerge all Makefiles of ipsec-tools contain "NATT_OBJS = nattraversal.o" which is used by racoon: "racoon_LDADD = $(CRYPTOBJS) $(HYBRID_OBJS) $(NATT_OBJS) $(FRAG_OBJS) $(LEXLIB) \" and somewhere else "$(CRYPTOBJS) $(HYBRID_OBJS) $(NATT_OBJS) $(FRAG_OBJS) \".

I'm a novice in IPSec but I found I may have to call setkey before. However, that makes no difference. I tried to call setkey -c with:

```

spdadd external internal any -P in ipsec esp/transport//require;

spdadd internal external any -P out ipsec esp/transport//require;

```

That should be the same as the Mac client shows up on setkey -DP.

Does anybody know why that payload type fails on my client? Did I miss to compile something into my kernel?

----------

## dev-urandom

I am a n00b when it comes to an IPsec VPN connection, but I use kvpnc and vpnc for connecting to a cisco vpn server. I could suggest them to you - the GUI is friendly, it supports ipsec and maybe a different client might fix the issue  :Wink: 

----------

## c00l.wave

I've already tried kvpnc but got the same problem (obvious since it only calls racoon). Config shouldn't be wrong since it works fine with racoon on OS X, so I assume something is missing on my system or Apple modified racoon to be incompatible with other systems (Although I cannot imagine Apple doing something like that, why should they?).

----------

## c00l.wave

I finally managed to get through ISAKMP-SA. All I needed to do was to append nat_traversal on; to my config and define an empty listen section. Now I hang somewhere between phase I and II:

```
2007-02-24 03:02:45: INFO: @(#)ipsec-tools 0.6.3 (http://ipsec-tools.sourceforge.net)

2007-02-24 03:02:45: INFO: @(#)This product linked OpenSSL 0.9.8d 28 Sep 2006 (http://www.openssl.org/)

2007-02-24 03:02:45: NOTIFY: NAT-T is enabled, autoconfiguring ports

2007-02-24 03:02:45: INFO: 127.0.0.1[500] used as isakmp port (fd=8)

2007-02-24 03:02:45: INFO: 127.0.0.1[500] used for NAT-T

2007-02-24 03:02:45: INFO: 127.0.0.1[4500] used as isakmp port (fd=9)

2007-02-24 03:02:45: INFO: 127.0.0.1[4500] used for NAT-T

2007-02-24 03:02:45: INFO: internal[500] used as isakmp port (fd=10)

2007-02-24 03:02:45: INFO: internal[500] used for NAT-T

2007-02-24 03:02:45: INFO: internal[4500] used as isakmp port (fd=12)

2007-02-24 03:02:45: INFO: internal[4500] used for NAT-T

2007-02-24 03:02:47: INFO: accept a request to establish IKE-SA: external

2007-02-24 03:02:47: INFO: initiate new phase 1 negotiation: internal[500]<=>external[500]

2007-02-24 03:02:47: INFO: begin Identity Protection mode.

2007-02-24 03:02:47: INFO: received Vendor ID: KAME/racoon

2007-02-24 03:02:47: INFO: received Vendor ID: RFC 3947

2007-02-24 03:02:47: INFO: Selected NAT-T version: RFC 3947

2007-02-24 03:02:47: INFO: Hashing external[500] with algo #2

2007-02-24 03:02:47: INFO: Hashing internal[500] with algo #2

2007-02-24 03:02:47: INFO: Adding remote and local NAT-D payloads.

2007-02-24 03:02:48: INFO: received Vendor ID: KAME/racoon

2007-02-24 03:02:48: INFO: Hashing internal[500] with algo #2

2007-02-24 03:02:48: INFO: NAT-D payload #0 doesn't match

2007-02-24 03:02:48: INFO: Hashing external[500] with algo #2

2007-02-24 03:02:48: INFO: NAT-D payload #1 doesn't match

2007-02-24 03:02:48: INFO: NAT detected: ME PEER

2007-02-24 03:02:48: INFO: KA list add: internal[4500]->external[4500]

2007-02-24 03:02:48: INFO: ISAKMP-SA established internal[4500]-external[4500] spi:hex:hex

2007-02-24 03:02:49: INFO: respond new phase 2 negotiation: internal[4500]<=>external[4500]

2007-02-24 03:02:49: ERROR: failed to get sainfo.

2007-02-24 03:02:49: ERROR: failed to get sainfo.

2007-02-24 03:02:49: ERROR: failed to pre-process packet.

2007-02-24 03:02:57: INFO: caught signal 2

2007-02-24 03:02:58: INFO: KA remove: internal[4500]->external[4500]

2007-02-24 03:02:58: INFO: racoon shutdown

```

These error messages appear more or less randomly. Google returns some hints to try tunnels instead of transports, remove /32 from sainfo, replacing (type? protocol?) numbers by "any" or adding new ones. I tried to experiment a bit but couldn't find a way to get any further.

BTW: The login to a OS X VPN is linked with system logins on server-side. Where am I supposed to enter that information (username and password) to authenticate?

----------

## jlg

did you ever get that working?  

just setup a xserve today  and I have to do this as well.

----------

