Transparent proxy of SSL traffic using Pound to HAProxy backend patch and how-to

OK so I've previously blogged about how to get TPROXY and HAProxy working nicely together. But what if you want to terminate SSL traffic on the load balancer in order to use HaProxy to insert cookies in the standard HTTP stream to the backend servers?

Well thats easy if you are using HAProxy for your SSL Termination, and STunnel now has TPROXY support built in but for Pound you're going to need some patches:

Many thanks to Krisztián Ivancsó  for working on the TPROXY patch for Pound for us, now we can finally do this!

First of all lets explain what we are trying to achieve. We have clients coming in from the external subnet 10.0.0.x with both HTTP and HTTPS requests to our virtual server (10.0.0.142), The HTTPS traffic is terminated by pound and sent to an HAProxy backend (10.0.0.142:81) which in turn inserts session cookies and passes the traffic to the backend servers (192.168.2.x).

The HTTP traffic hits a separate HAProxy instance on (10.0.0.142:80) where cookies are inserted and traffic passed to the backend servers (192.168.2.x). Why a second instance? Unfortunately it is not currently possible to have TPROXY running for Pound and HAProxy using the same IP and port combination (which makes sense if you think about it).

Just a reminder - "why are we doing all of this?", because we want to clients source IP address to be presented to the backend server even though the traffic is coming through a proxy!

So lets assume that you have already set up HAProxy in TPROXY mode for full transparency.

First of all we need to grab a recent copy of Pound and the TPROXY patch, configure, make & install etc.

wget http://www.loadbalancer.org/download/PoundSSL-Tproxy/Pound-2.4.5.tgz
tar -xvf Pound-2.4.5.tgz
cd Pound-2.4.5
wget http://www.loadbalancer.org/download/PoundSSL-Tproxy/poundtp-2.4.5.diff
patch -p1 < poundtp-2.4.5.diff
./configure
make TPROXY=1
make install

Make sure the firewall rules are set correctly for standard TPROXY (HAProxy binds to these automatically), as in the previous blog.

# Standard rules for TPROXY setup
#!/bin/bash
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 111
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 111 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100

Add the rules to make sure that local Pound -> HAProxy traffic is transparent:

# Rules to match PoundSSL > Haproxy backend
iptables -t mangle -A OUTPUT -s 10.0.0.142 -p tcp --sport 81 -j DIVERT
iptables -t mangle -A OUTPUT -d 10.0.0.142 -p tcp --dport 81 -j DIVERT

Then configure HAProxy making sure you have two instances with the same real servers, one for HTP traffic and one for the transparent terminated HTTPS traffic:

# HAProxy configuration file generated by load balancer appliance
global
#uid 99
#gid 99
daemon
stats socket /var/run/haproxy.stat mode 600
maxconn 40000
ulimit-n 81000
pidfile /var/run/haproxy.pid
defaults
mode    http
contimeout    4000
clitimeout    42000
srvtimeout    43000
balance    roundrobin
listen    VIP_Name 10.0.0.142:80
mode    http
option    forwardfor
source 0.0.0.0 usesrc clientip
cookie    SERVERID insert nocache indirect
server RIP_Name 192.168.2.98:80 weight 1 cookie RIP_Name check  inter 2000 rise 2 fall 3
server    backup 127.0.0.1:80 backup  source 0.0.0.0
option redispatch
option abortonclose
maxconn 40000
listen    SSL_Backend 10.0.0.142:81
mode    http
option    forwardfor
source 0.0.0.0 usesrc clientip
cookie    SERVERID insert nocache indirect
server RIP_Name 192.168.2.98:80 weight 1 cookie RIP_Name check  inter 2000 rise 2 fall 3
server    backup 127.0.0.1:81 backup  source 0.0.0.0
option redispatch
option abortonclose
maxconn 40000

Make sure that when you do the Pound configuration that Pound is running as root:

# Pound2 configuration file generated by load balancer appliance
#User    "nobody"
#Group    "nobody"
LogLevel    0
Client     30
TimeOut     60
ListenHTTPS
Address 10.0.0.142
Port 443
Cert "/usr/local/etc/server1.pem"
Service
BackEnd
Address 10.0.0.142
Port 81
TProxy 1
End
End
End

Obviously make sure that:

• The client is in the 10.0.0.x network.
• The load balancer has IPs in both networks.
• The backend server uses the load balancer as the default gateway.
• You have your fingers crossed :-).

I put this together fairly quickly, so please let me know if I have missed anything. With any luck it should make it into the v6.7 appliance fairly soon...available as a manual update for now until v6.7 is ready.

Since writing this Ivan has added three new features (I've updated the download link to have this latest version).

Three new features:

• You can use patched Pound without root privileges
• It adds a new global TProxy option which disables/enables TProxy globally. (If this global option is set, then pound will preserve needed privileges to work as transparent proxy as a simple user. If it's not set pound will work as unpatched version and will not preserve additional rights.)
• Let tproxy to set a random port for source IP when Pound connects to backend. It means original source port is not preserved in communication with backend.
  Example config:

User    "ivan"
Group   "ivan"

LogFacility -
LogLevel 5

TProxy 1

ListenHTTP
Address 192.168.254.22
Port    81
End
Service
BackEnd
Address 192.168.254.22
Port    80
TProxy 1
End
End

User    "ivan"
Group   "ivan"

LogFacility -
LogLevel 5

TProxy 1

ListenHTTP
Address 192.168.254.22
Port    81
End
Service
BackEnd
Address 192.168.254.22
Port    80
TProxy 1
End
End