Difference between revisions of "Connection and VPN Bonding"

From Hack Sphere Labs Wiki
Jump to: navigation, search
(→‎NAT Forwarding as Internet Gateway)
(→‎OpenVPN)
Line 96: Line 96:
 
#!/bin/bash
 
#!/bin/bash
  
−
modprobe bonding mode=3 miimon=100
+
modprobe bonding mode=0 miimon=100
 
modprobe tun
 
modprobe tun
  
Line 121: Line 121:
 
</nowiki>
 
</nowiki>
 
}}
 
}}
 +
 +
Take note of the 'modprobe bonding mode=0 miimon=100' line.  The bonding mode is important depending on what you would like to do:
 +
 +
Possible values are:
 +
 +
    balance-rr or 0
 +
        Round-robin policy: Transmit packets in sequential order from the first available slave through the last. This mode provides load balancing and fault tolerance.
 +
 +
    active-backup or 1
 +
        Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch.
 +
 +
    In bonding version 2.6.2 or later, when a failover occurs in active-backup mode, bonding will issue one or more gratuitous ARPs on the newly active slave. One gratutious ARP is issued for the bonding master interface and each VLAN interfaces configured above it, provided that the interface has at least one IP address configured.
 +
    Gratuitous ARPs issued for VLAN interfaces are tagged with the appropriate VLAN id. This mode provides fault tolerance. The primary option, documented below, affects the behavior of this mode.
 +
 +
    balance-xor or 2
 +
        XOR policy: Transmit based on the selected transmit hash policy. The default policy is a simple
 +
 +
    Alternate transmit policies may be selected via the xmit_hash_policy option.
 +
    This mode provides load balancing and fault tolerance.
 +
 +
    broadcast or 3
 +
        Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.
 +
 +
    802.3ad or 4
 +
        IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification.
 +
 +
    Slave selection for outgoing traffic is done according to the transmit hash policy, which may be changed from the default simple XOR policy via the xmit_hash_policy option, documented below. Note that not all transmit policies may be 802.3ad compliant, particularly in regards to the packet mis-ordering requirements of section 43.2.4 of the 802.3ad standard. Differing peer implementations will have varying tolerances for noncompliance.
 +
 +
        Prerequisites:
 +
            Ethtool support in the base drivers for retrieving the speed and duplex of each slave.
 +
            A switch that supports IEEE 802.3ad Dynamic link aggregation.
 +
 +
    Most switches will require some type of configuration to enable 802.3ad mode.
 +
 +
    balance-tlb or 5
 +
        Adaptive transmit load balancing: channel bonding that does not require any special switch support. The outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave.
 +
 +
        Prerequisite:
 +
            Ethtool support in the base drivers for retrieving the speed of each slave.
 +
 +
    balance-alb or 6
 +
        Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation.
 +
 +
    The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server.
 +
    Receive traffic from connections created by the server is also balanced. When the local system sends an ARP Request the bonding driver copies and saves the peer's IP information from the ARP packet.
 +
    When the ARP Reply arrives from the peer, its hardware address is retrieved and the bonding driver initiates an ARP reply to this peer assigning it to one of the slaves in the bond.
 +
    A problematic outcome of using ARP negotiation for balancing is that each time that an ARP request is broadcast it uses the hardware address of the bond. Hence, peers learn the hardware address of the bond and the balancing of receive traffic collapses to the current slave. This is handled by sending updates (ARP Replies) to all the peers with their individually assigned hardware address such that the traffic is redistributed. Receive traffic is also redistributed when a new slave is added to the bond and when an inactive slave is re-activated. The receive load is distributed sequentially (round robin) among the group of highest speed slaves in the bond.
 +
    When a link is reconnected or a new slave joins the bond the receive traffic is redistributed among all active slaves in the bond by initiating ARP Replies with the selected mac address to each of the clients. The updelay parameter (detailed below) must be set to a value equal or greater than the switch's forwarding delay so that the ARP Replies sent to the peers will not be blocked by the switch.
 +
 +
        Prerequisites:
 +
            Ethtool support in the base drivers for retrieving the speed of each slave.
 +
            Base driver support for setting the hardware address of a device while it is open. This is required so that there will always be one slave in the team using the bond hardware address (the curr_active_slave) while having a unique hardware address for each slave in the bond. If the curr_active_slave fails its hardware address is swapped with the new curr_active_slave that was chosen.
  
 
==NAT Forwarding as Internet Gateway==
 
==NAT Forwarding as Internet Gateway==

Revision as of 09:47, 27 February 2012

Objective

Perferablly bond multiple 3G modems together to create a stable faster connection. I am trying to aggregate 3 unstable connections into one.

Ideas

  • Bond 2-3 OpenVPN tun interfaces.
    • LAGG
    • Kernel Bonding
      • LACP (Stable connections, same BW)
  • Linux Advanced Routing & Traffic Control - http://lartc.org/

Notes

Custom Linux

I tested with debian.

Server Configuration

I used a Debian VPS because I wanted to route all my traffic out to the internet through the bond.

OpenVPN

su -
aptitude update
aptitude upgrade
aptitude install openvpn

tap configuration is a bit different then tun configuration. Since it works via layer two you do not need to worry about layer 3 stuff like IPs in the config file.

Setup a CA, Certs, ta.key: http://wiki.hackspherelabs.com/index.php?title=OpenVPN#Setup but here are some commands for reference: 

mkdir /etc/openvpn/easy-rsa
cp /usr/share/doc/openvpn/examples/easy-rsa/2.0/* /etc/openvpn/easy-rsa
cd /etc/openvpn/easy-rsa/
source ./vars
./clean-all
./build-ca
./build-key-server servername
./build-dh
cd keys
openvpn --genkey --secret ta.key
cd ..
./build-key-pkcs12 clientx

You need some openvpn config files in /etc/openvpn/ and here is an example of a tap server openvpn config file: 

 Template error: are you trying to use the = sign? Visit Help:Template#Escape template-breaking characters for workarounds.
 Template error: are you trying to use the = sign? Visit Help:Template#Escape template-breaking characters for workarounds.

You need a vpn server for each modem that you want to bond. You will need to configure a different port and ip for each one while also a different tap interface.

You need to stop and disable openvpn from starting because the bonding.sh script will call openvpn. 

/etc/init.d/openvpn stop
update-rc.d openvpn disable

You also need the utilities that this script calls 

aptitude install uml-utilities ifenslave

 Template error: are you trying to use the = sign? Visit Help:Template#Escape template-breaking characters for workarounds.
 Template error: are you trying to use the = sign? Visit Help:Template#Escape template-breaking characters for workarounds.

Take note of the 'modprobe bonding mode=0 miimon=100' line. The bonding mode is important depending on what you would like to do:

Possible values are: 

   balance-rr or 0 
       Round-robin policy: Transmit packets in sequential order from the first available slave through the last. This mode provides load balancing and fault tolerance.

   active-backup or 1 
       Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch.

   In bonding version 2.6.2 or later, when a failover occurs in active-backup mode, bonding will issue one or more gratuitous ARPs on the newly active slave. One gratutious ARP is issued for the bonding master interface and each VLAN interfaces configured above it, provided that the interface has at least one IP address configured.
   Gratuitous ARPs issued for VLAN interfaces are tagged with the appropriate VLAN id. This mode provides fault tolerance. The primary option, documented below, affects the behavior of this mode.

   balance-xor or 2 
       XOR policy: Transmit based on the selected transmit hash policy. The default policy is a simple

   Alternate transmit policies may be selected via the xmit_hash_policy option.
   This mode provides load balancing and fault tolerance.

   broadcast or 3
       Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.

   802.3ad or 4
       IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification.

   Slave selection for outgoing traffic is done according to the transmit hash policy, which may be changed from the default simple XOR policy via the xmit_hash_policy option, documented below. Note that not all transmit policies may be 802.3ad compliant, particularly in regards to the packet mis-ordering requirements of section 43.2.4 of the 802.3ad standard. Differing peer implementations will have varying tolerances for noncompliance.

       Prerequisites:
           Ethtool support in the base drivers for retrieving the speed and duplex of each slave.
           A switch that supports IEEE 802.3ad Dynamic link aggregation.

   Most switches will require some type of configuration to enable 802.3ad mode.

   balance-tlb or 5
       Adaptive transmit load balancing: channel bonding that does not require any special switch support. The outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave.

       Prerequisite:
           Ethtool support in the base drivers for retrieving the speed of each slave.

   balance-alb or 6 
       Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation.

   The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server.
   Receive traffic from connections created by the server is also balanced. When the local system sends an ARP Request the bonding driver copies and saves the peer's IP information from the ARP packet.
   When the ARP Reply arrives from the peer, its hardware address is retrieved and the bonding driver initiates an ARP reply to this peer assigning it to one of the slaves in the bond.
   A problematic outcome of using ARP negotiation for balancing is that each time that an ARP request is broadcast it uses the hardware address of the bond. Hence, peers learn the hardware address of the bond and the balancing of receive traffic collapses to the current slave. This is handled by sending updates (ARP Replies) to all the peers with their individually assigned hardware address such that the traffic is redistributed. Receive traffic is also redistributed when a new slave is added to the bond and when an inactive slave is re-activated. The receive load is distributed sequentially (round robin) among the group of highest speed slaves in the bond.
   When a link is reconnected or a new slave joins the bond the receive traffic is redistributed among all active slaves in the bond by initiating ARP Replies with the selected mac address to each of the clients. The updelay parameter (detailed below) must be set to a value equal or greater than the switch's forwarding delay so that the ARP Replies sent to the peers will not be blocked by the switch.

       Prerequisites:
           Ethtool support in the base drivers for retrieving the speed of each slave.
           Base driver support for setting the hardware address of a device while it is open. This is required so that there will always be one slave in the team using the bond hardware address (the curr_active_slave) while having a unique hardware address for each slave in the bond. If the curr_active_slave fails its hardware address is swapped with the new curr_active_slave that was chosen.

NAT Forwarding as Internet Gateway

The entire reason I wanted to do this was to forward internet traffic through multiple modems. So on my debian box: 

nano /etc/sysctl.conf

Uncomment: #net.ipv4.ip_forward=1 

echo 1 > /proc/sys/net/ipv4/ip_forward

You then can forward incoming traffic with: 

iptables -t nat -A POSTROUTING -o bond0 -j MASQUERADE

nano iptables.nat.sh

An put the line in it.

Client

USB Drive/Modem CD Rom Eject

The first step is to get the OS to eject the CD drive: http://ubuntuforums.org/showthread.php?t=1002262

After you plug the device in edit /etc/udev/70-persistent-cd.rules find your device (Novatel_Mass_Storage) and add: 

, RUN+="/usr/bin/eject %k"

You will have to do this for each of these type of modems.


I used wvdial to test the modem. It looks like pppd accepts .chat scripts too. Here is my wvdial script: 

[Dialer Defaults]
Init1 = ATZ
#Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init2 = ATQ V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init3 - ATQ V1 E1 S0=0 &C1 &D2 +FCLASS=0
#? - Init5 = AT+CGDCONT=1,"IP",""
Carrier Check = yes
Dial Command = ATX1DT
Modem Type = Analog Modem
Baud = 460800
New PPPD = yes
Modem = /dev/ttyUSB0
ISDN = 0
Phone = #777
Password = JustAnyOldPW
Username = 5555555555@vzw3g.com

Replace 5555555555 with your devices phone number.

Notes

pfSense

With the USB760 modem you need to eject the drive before it works. 

cdcontrol -f /dev/cd0

This needs to be automated. This thread suggests a devd rule: http://forum.pfsense.org/index.php/topic,43285.0.html

bsd router/firewall/more

Notes

ZeroShell

Zeroshell is a Linux distribution for servers and embedded devices aimed at providing the main network services a LAN requires. It is available in the form of Live CD or Compact Flash image and you can configure and administer it using your web browser.

  • Says it supports VPN Bonding
  • 3G support (+It is linux)

Notes

udev/cdcontrol Creation

Retrieved from "https://wiki.hackspherelabs.com/index.php?title=Connection_and_VPN_Bonding&oldid=610"