I just went through hell in a handbasket trying to get 802.3ad Link Aggregation set up on a Centos 5.2 Xen box. Setting up link aggregation itself isn't that bad - http://wiki.centos.org/TipsAndTricks/BondingInterfaces for a simple guide (after your managed switch is configd) - but what ever I did, I was unable to get both interfaces simultaneously active.
About the only useful debugging info I got was that the MAC was in use. I was puzzled because as far as I know, link agg takes over the primary MAC and sets that up for both NICs. Furthermore, the same exact hardware was working great on Fedora 10.
bonding: bond0: Warning: the permanent HWaddr of eth0
- [MAC ADDR]- is still in use by bond0. Set the HWaddr of eth0
to a different address to avoid conflicts.
bonding: bond0: releasing active interface eth0
bonding: bond0: making interface eth1 the new active one.
bonding: bond0: Removing slave eth1
bonding: bond0: releasing active interface eth1
ADDRCONF(NETDEV_UP): bond0: link is not ready
bonding: bond0: Adding slave eth0.
Luckily, I stumbled across this bug report. If you scroll down to the last comment, this appears to be a Xen specific issue. By default Xen tries to set its bridge up on eth0, and I assume this prevents the kernel bonding driver from taking over the NIC. By opening up /etc/xen/xend-config.sxp and adding:
(network-script 'network-bridge netdev=bond0')
Xen will bridge to the bond0 interface, and everything will work as expected.
Another trick I had to do was add a start delay to the networking scripts. This is useful if your hardware is crap (cough Broadcom), you need a dhcp lease and it fails, or you are running STP, link aggr., etc. On Fedora, RHEL, and derivitives this is accomplished by adding the NETWORKDELAY directive to /etc/sysconfig/network:
NETWORKING=yes
NETWORKDELAY=31
If you need more granularity, you can set delays to specific adapters in the /etc/sysconfig/ifcfg-{x} files with the LINKDELAY directive.
Just a couple of hard lesssons from the trenches, hopefully this will save someone else some time.
Comments
comments powered by Disqus