Bonding with Turris Omnia


#1

Hi there,

as older posts didn’t go into the correct direction (can be seen here and here), I’d like to start that topic again.

Theory:
a) SoC is connected to 3 GbE-lanes:

  1. eth0 (lan0-lan3) connected to port 5
  2. eth1 (wan) - SoC-connection not visible to switch configuration
  3. eth2 (lan4) connected to port 6

As eth1 should be used for up to 1GbE WAN-connection one could use eth0 and eth2 to bond/aggregate traffic.
I do not expect the benefit to be utmost high, but hey, it should be better than just separating traffic via VLANs.
Reason to do so for me is to connect a smart switch to get higher connection speeds.

b) Existing package kmod-bonding:
Kernel supports bonding (existing /lib/modules/4.4.119-[…]/bonding.ko)
Searching google for ““kmod-bonding” openwrt” there is only a couple of entries:

  1. https://gist.github.com/braian87b/d069487f4e0eb260f73c9938a97d9bc0
  2. https://forum.openwrt.org/viewtopic.php?id=69209
  3. https://forum.lede-project.org/t/bonding-802-3ad/6236 (summary)

So it seems to me no one ever had an openwrt device capable of using that :thinking::star_struck:

First try was to just use rc.local to echo necessary commands:

ifconfig bond0 down
echo 802.3ad > /sys/devices/virtual/net/bond0/bonding/mode
echo fast > /sys/devices/virtual/net/bond0/bonding/lacp_rate
echo layer3+4 > /sys/devices/virtual/net/bond0/bonding/xmit_hash_policy
ifconfig bond0 up

echo +eth0 > /sys/devices/virtual/net/bond0/bonding/slaves
echo +eth2 > /sys/devices/virtual/net/bond0/bonding/slaves

reboot -> syslog:

2018-04-17T02:43:17+02:00 info kernel[]: [   32.219033] bond0: Setting LACP rate to fast (1)
2018-04-17T02:43:17+02:00 info kernel[]: [   32.219325] bond0: Setting xmit hash policy to layer3+4 (1)
2018-04-17T02:43:17+02:00 info kernel[]: [   32.229780] 8021q: adding VLAN 0 to HW filter on device bond0
2018-04-17T02:43:17+02:00 info kernel[]: [   32.231708] bond0: Adding slave eth0
2018-04-17T02:43:17+02:00 err kernel[]: [   32.231717] bond0: Error: Device is in use and cannot be enslaved
2018-04-17T02:43:17+02:00 err kernel[]: [   32.237849] bond0: option slaves: unable to set because the bond device is up
2018-04-17T02:53:28+02:00 info kernel[]: [   30.138315] bond0: Adding slave eth2
2018-04-17T02:53:28+02:00 err kernel[]: [   30.138326] bond0: eth2 is up - this may be due to an out of date ifenslave
2018-04-17T02:53:28+02:00 info kernel[]: [   30.162580] IPv6: ADDRCONF(NETDEV_UP): bond0: link is not ready
2018-04-17T02:53:28+02:00 info kernel[]: [   30.162592] 8021q: adding VLAN 0 to HW filter on device bond0

So it seems ifenslave might be necessary in order to get this working, but infenslave is not available in openwrt/turris repo - do I need to build it myself?


NFS NAS and network speed, performance tuning
Requesting new packages / new versions
#2

Hi,
I made issue for this, so it could be done after 3.10.1 https://gitlab.labs.nic.cz/turris/openwrt/issues/184


#3

Just blind shot but have you tried to bring device eth0 and eth2 down before enslaving them?

To my understanding doing same with ip utils is something like:

ip l add bond0 type bond
ip l set master bond0 dev eth0
ip l set master bond0 dev eth2

Skipping bond configuration of course.

I am still convinced that ifenslave is not required. Rather I would say that it’s just an old error message from kernel. I suspect that it might have been possible to enslave link that is up in old versions but that at some point changed. So clever coder added this message but that was probably just long time ago when ifenslave was relevant. At least that is my hunch.


#4

Yes I tried, but after that device is not reachable. Maybe I missed another thing to configure, but the following didn’t work:

/etc/config/network

[...]
config interface 'lan'
	option force_link '1'
	option ifname 'eth0'
	option type 'bridge'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option delegate '0'
[...]
config interface 'zwei'
	option ifname 'eth2.2 zwei0 zwei1'
	option type 'bridge'
	option proto 'static'
	option ipaddr '192.168.2.1'
	option netmask '255.255.255.0'

config interface 'drei'
	option ifname 'eth2.5 drei0 drei1'
	option type 'bridge'
	option proto 'static'
	option ipaddr '192.168.0.1'
	option netmask '255.255.255.0'
[...]
config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '3t 4t 5'
	option vid '1'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '3t 4t 6t'
	option vid '2'

config switch_vlan
	option device 'switch0'
	option vlan '5'
	option ports '3t 4t 6t'
	option vid '5'

/etc/rc.local

ifconfig bond0 down
echo 802.3ad > /sys/devices/virtual/net/bond0/bonding/mode
echo fast > /sys/devices/virtual/net/bond0/bonding/lacp_rate
echo layer3+4 > /sys/devices/virtual/net/bond0/bonding/xmit_hash_policy
ifconfig eth0 down
ifconfig eth2 down
ifconfig bond0 up
ifconfig eth0 up
ifconfig eth2 up

Unfortunately I cannot see logfiles to check what went wrong as I cannot access the device anymore.


#5

Do you have any news on that?


#6

I am interested in this as well.


#7

there should soon be nbase-t to usb3 adapter on the market…


#8

Interesting idea, but I am rather investing in fully managed switches in gbit than 10gbit unmanaged fan equiped switches. Makes no sense to me when I have no 10gbe NIC or anything. But years will pass and 10gbit will happen. Cetin is slowly making 500/50gbe bonded lanes for homes a reality. when we get over 1gbe we will need at least n-base-T. But back to topic. Since I do have switch with LAG and LACP I would be glad to use 2 ports as one. Even though this is just to accomodate for connection to WAN and HDD at the same time.


#9

i agree that it would be nice to have bonding working.
on the other hand;
there are numerous fanless, managed nbase-t switches available for small budget.
first consumer mainboard with 10gig onboard was already a few years ago …


#10

I think you are totaly missing the point.
If you have omnia you can connect switches only with gigabit of speed why would you want nbase-T network?


#11

one can connect the omnia with 2,5 or 5 gig (vs 2 with bonding) to a commodity fanless, managed, nbase-t switch using said adapter.

given the non-progress of this issue, i think it is relevant.


#12

If I didn’t already buy my complete hardware and or had more nbase-t ports available, this adapter would be an option for me.
But as I don’t want to spend another 400€ on another switch, simple bonding would be much more (cost) effective :wink:


#13

I am finaly going with LAG (2x) between switches and STP with one active line to router and one blocked.
Anyway There is no good reason to have NAS on router. It is terible decision in more complex network built on enterprise grade network devices.
So with no bonding I can do quite well still.
Another question rising from this is how to do some QOS on router turris omnia in order to not alow any bottleneck for internet devices. With my down and up link it is no concern but I do want to set it well. :slight_smile:


#14

Of course you can, but switch chip is capable of switching only 1GBIT.


#15

exactly, but the usb3.0 ports do 5gbit/s

qos: install luci-app-sqm and make one instance of piece_of_cake on your wan interface with down/up values about 10% below your measured maximum transferspeed.


#16

There’s only use for 2 GBit/s accumulated ethernet network speed (putting aside TO NAS solution which is bad idea as you correctly pointed out) on TO:

  • 1 GBit/s for WAN
  • 1 GBit/s for WLAN (gross rates of 5 GHz wave 2 with multiple users accumulating traffic should be somewhat below 1 GBit/s, adding the traffic for 2,4 GHz users you would be around 1 GBit/s)

So there’s actually no need 5 GBit/s in normal setups.
Therefore bonding would be completely satisfying our needs.

The only setup going beyond 2 GBit/s I could imagine would be installing two 5 GHz wave2-cards and maxing them out with MU-MIMO. But in those cases you would normally better get some more access points…

Btw: maxing out the USB3-ports is bad idea on TO if you have 2,4 GHz WLAN-card installed…


#17

@paja: do you have any news on that topic for us?


#18

Of course you can, but switch chip is capable of switching only 1GBIT.

It may be possible to use a mPCIe network card (eg. https://www.amazon.de/PCI-Express-Gigabit-Ethernet-Controller-controlle/dp/B01MQWPOY8)
But I’m not sure what max speed the mPCIe slots are supported by the cpu.


#19

It looks like if the first end-user item is on its way: QNAP QNA-UC5G1T using a https://www.aquantia.com/products/controllers/aqtion-aqc111u-aqc112u/ . But it has yet to prove vlan-support (etc; no docs available) and seems to be in need of a dedicated driver (which I do not now how we should get it into TO).
But anyhow I deem it really unnecessary to have link speeds beyond 2GBE (see above).


#20

yea, was a bit optimistic on the timeframe :wink:

bonding was/is the right feature…