Ubuntu, Sonicwall, and OpenSwan Part 2

In the process of getting an Ubuntu 10.04 machine to connect to a Sonicwall appliance at my office (part 0), I followed documentation and ran into an error of “Informational Exchange message must be encrypted (part 1).”

I did a lot of searching on the error, but mostly came up with other people having the error and not finding a solution. I started tinkering with settings to see if I could get other error messages that might be more helpful.

First, I edited /etc/ipsec.conf to turn off aggressive mode:
aggrmode=no

This gave me a different error:
Possible authentication failure: no acceptable response to our first encrypted message

Ah ha! That led me to re-check my shared secret in /etc/ipsec.secrets where I found a typo. Then I re-checked /etc/ipsec.conf and concluded that I misunderstood the documentation and clearly the rightsubnet should be my OFFICE subnet, not my home subnet.

I refreshed my ipsec connection settings sudo ipsec auto --replace sonicwall and tried again.

A new error, it just hung here:
002 "sonicwall" #1: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
004 "sonicwall" #1: STATE_MAIN_I4: ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY cipher=oakley_3des_cbc_192 prf=oakley_sha group=modp1024}

I searched around for this and happened across this post on ubuntuforums that indicated that the user had to set the leftid (local Ubuntu machine id) to @GroupVPN [and it's case-sensitive]. Odd, I thought, but I tried it, changing it in /etc/ipsec.conf and /etc/ipsec.secrets.

Sure enough, another new error!

003 "sonicwall" #1: next payload type of ISAKMP Hash Payload has an unknown value: 112
[note: this unknown value changed regularly, it was NOT always 112.]

At this point I got excited because I knew I was at least talking to the Sonicwall appliance and we were just having trouble understanding one another. I started focusing on key exchange/IKE and I happened to come across a few notes about IKE DH Group 5 requiring an edit to /etc/ipsec.conf to change the setting for ike to ike=3DES-SHA1-modp1536.

Me to self, “Hm. We’re DH2. That’s 1024 bits. Since Group 5 is 1536 bits and this says it should be set to 3DES-SHA1-modp1536, let me try setting ike=3DES-SHA1-modp1024.”

I updated and it seemed to connect but never asked me for my authentication information, it just hung here:
003 "sonicwall" #1: malformed payload in packet
002 "sonicwall" #1: sending notification PAYLOAD_MALFORMED to XXX.XXX.XXX.XXX:XXXX

I hrmed a bit more — so close! — and remembered that I still had aggressive mode off. I popped into /etc/ipsec.conf and turned it back on. aggrmode=yes

I updated the connection and tried to open it again:
xantha@moloko:~$ sudo ipsec auto --replace sonicwall
xantha@moloko:~$ sudo ipsec whack --name sonicwall --initiate

Success! It prompted me for a username and password, I supplied it, we were authenticated! (some parts redacted here)
041 "sonicwall" #1: sonicwall prompt for Username:
Name enter: USERID
040 "sonicwall" #1: sonicwall prompt for Password:
Enter secret:
002 "sonicwall" #1: XAUTH: Answering XAUTH challenge with user='USERID'
002 "sonicwall" #1: transition from state STATE_XAUTH_I0 to state STATE_XAUTH_I1
004 "sonicwall" #1: STATE_XAUTH_I1: XAUTH client - awaiting CFG_set
003 "sonicwall" #1: ignoring informational payload, type IPSEC_INITIAL_CONTACT msgid=00000000
003 "sonicwall" #1: received and ignored informational message
002 "sonicwall" #1: XAUTH: Successfully Authenticated

Woo hoo!

[aside: some of the errors may have been due to things being repaired in the order I repaired them... hard to say. I just know what errors I got and what steps I took to resolve them. YMMV of course. ]

What do you think?