Mwan3 Multiwan LTE setup

Software SW tweaks
#1

I managed to setup muliwan on turris omnia and I report here the procedure for comments and future reference.
I use mwan3 & luci-app-mwan3 packages and I follow this openwrite wiki page
Step 0: Test wan connection
Step 0.1: Install and test the Lte connection
Step 1: login via ssh and add two lines for the packages above in the /etc/updater/user.lua.

Summary

vi /etc/updater/user.lua

My /etc/updater/user.lua looks like:

[details=Summary] cat /etc/updater/user.lua
–[[
A place for user definitions.
Repository “name” “URI” { ca = “file:///etc/ssl/ca.pem”, pubkey = “file:///etc/repo.pubkey” }
Install “pkgname” “other”
]]
Install "mwan3"
Install “luci-app-mwan3”[/details]
Step 1.1: Update the router by issuing the updater.sh command

Summary

# updater.sh

Step 2: Configure the wan zone of the firewall with option option conntrack '1'

Summary 
# vi /etc/config/firewall
config zone
	option name 'wan'
	option conntrack '1'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	option network 'wan6 Lte wan'

Step 3: Use Luci to configure (enable) the interfaces and configure members, policies and rules or edit the /etc/config/mwan3 file. My configuration looks like:

[details=Summary]# cat /etc/config/mwan3

config interface 'wan'
	list track_ip '8.8.4.4'
	list track_ip '8.8.8.8'
	list track_ip '208.67.222.222'
	list track_ip '208.67.220.220'
	option count '1'
	option timeout '2'
	option interval '5'
	option down '3'
	option up '8'
	option reliability '1'
	option enabled '1'

config policy 'wan_only'
	option last_resort 'unreachable'
	list use_member 'wan_m1_w1'

config policy 'wan2_only'
	option last_resort 'unreachable'
	list use_member 'wan2_m1_w1'

config policy 'balanced'
	option last_resort 'unreachable'
	list use_member 'wan_m1_w2'
	list use_member 'wan2_m1_w2'

config policy 'wan_wan2'
	list use_member 'wan_m1_w1'
	list use_member 'wan2_m2_w2'
	option last_resort 'unreachable'

config policy 'wan2_wan'
	option last_resort 'unreachable'
	list use_member 'wan2_m1_w1'
	list use_member 'wan_m2_w2'

config interface 'Lte'
	option enabled '1'
	option reliability '1'
	option count '1'
	option timeout '2'
	option interval '5'
	option down '5'
	list track_ip '8.8.4.4'
	list track_ip '8.8.8.8'
	option up '10'

config member 'wan_m1_w1'
	option interface 'wan'
	option metric '1'
	option weight '1'

config member 'wan2_m2_w2'
	option interface 'Lte'
	option metric '2'
	option weight '2'

config member 'wan2_m1_w1'
	option interface 'Lte'
	option metric '1'
	option weight '1'

config member 'wan_m1_w2'
	option interface 'wan'
	option metric '1'
	option weight '2'

config member 'wan2_m1_w2'
	option metric '1'
	option weight '2'
	option interface 'Lte'

config member 'wan_m2_w2'
	option interface 'wan'
	option metric '2'
	option weight '2'

config rule 'default'
	option proto 'all'
	option sticky '0'
	option use_policy 'wan_wan2'
	option dest_ip '0.0.0.0/0'

[/details]

Just remember that for balanced rules, the higher the Weight, more traffic will pass through this interface.

If you just want the Lte to be failover use my config rule.

Please comment on my setup

2 Likes
#2

It seems that in my setup DNSSEC causes problems with the DNS resolution.
Is there any hint on how to debug this problem?

#3

Update: Using cz.nic dns servers DNSSEC seems to work fine

#4

I re-installed mwan3 today. Previously I had a network storm that finally rendered my router unusable. I’ve built up the new config over several days - saved interim steps. With today’s reinstall of mwan3, I am seeing a return of the messages that I associate with the earlier storm:

2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.432662] net_ratelimit: 1478 callbacks suppressed
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.437644] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.449637] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.460908] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.472847] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.483847] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.495097] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.506397] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.517538] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.528675] br-lan: received packet on eth0 with own address as source address
2017-01-13T19:20:44-05:00 warning kernel[]: [ 7619.539753] br-lan: received packet on eth0 with own address as source address

@lampra could you please check your system log to see if you have similar messages in any quantity?
Thanks,
Robert

#5

I just found this post on the old CZ forum: https://www.turris.cz/forum/topic_show.pl?tid=1508

Buried in the discussion is a reference to bridging in the CPU conflicting with a bridged lan. :bulb:

When configuring mwan3, I had followed this openWRT wiki page instruction, which includes the statement “…as with all VLANs, should also include the built-in CPU port as a tagged member…”: https://wiki.openwrt.org/doc/howto/mwan3. Following that advice, I had added Tagged to the CPU port for the new vlan (vlan 3) AND for the existing wan vlan (vlan 2). After all, that constituted having CPU in all vlans, and Untagged was impossible - created error messages.

Recalling a comment made previously in discussions about these errors, I decided to back out this configuration one step at a time to see if I had created a routing loop by following the openWRT instructions. I replaced Tagged with Off in the CPU port for vlan 2. Save & Apply. The error message immediately stopped showing up in my system log.

The Switch vlan configuration that ultimately worked for me is:
VLAN ID Port 0 Port 1 Port 2 Port 3 Port 4 CPU Port 6
1 untag untag untag untag off untag off
2 off off off off off off untag
3 off off off off untag tag off

Although I’m still working out why this works, the following openWRT wiki page is helpful (…now that I know what to look for!!!): https://forum.openwrt.org/viewtopic.php?id=55452

#6

although this solution is working. I use it as LTE failover. But as mwan3 is doing checks all the time to see if LTE is up and running I get credit by the LTE provider for this traffic. As it’s a predpaid plan which should only be used when WAN is down mwan3 seems to be no option as it’s slowly but constantly eating my remaining prepaid credit).

Is there a trick that mwan3 will bring up LTE only when WAN is down and then starts checking for a fallback or does anyone know of a good article on how to achieve this if mwan3 is not the solution?

#7

update: Although I’m affected by this issue since 3.5 it seems that the issue is now interleaving. I tried rebooting now for 6 times and found out that card came up fine on 2nd, 4h and 6th reboot and it failed on 1st, 3rd and 5th reboot.

Is this pure coincidence or is this due to the update to 3.5 (because with 3.4 I sometimes had to reboot 6-7 times until I get a single success).

#8

Maybe you could employ watchcat to monitor network connection “pingability” to8.8.8.8 otherwise the router is rebooted or exchange network configuration scripts and restart network that will redirect your traffic automatically to LTE.

opkg install luci-app-watchcat
/etc/uci-defaults/50-watchcat
uci set system.@watchcat[0].period=1h
ter
/etc/init.d/watchcat enable

#9

It will be wonderful if someone can update this to connect to UPC WiFree network and post instructions how to archieve connection to UPC WiFree in case of primary connection failure.

#10

You can tell mwan3 to do no ping checks at all and always assume that the connection is online. Then trafic should only flow when your primary connection is deemed down by mwan3.

#12

Thanks a lot will try that

#13

@marcerlser Yes you are right, the solution above is consuming some data as it pings the tracking ip;s (8.8.8.8 and 8.8.4.4). To my understanding this is a few MBits per month. If this is a problem for you, I am not sure if mwan3 is a solution for you. This is because i expect that there will be some traffic passing from the LTE interface if the LET interface is enabled (even if you stop the pings) so I expect that you will need an other solution (a script maybe) that will be enabling the LTE interface if wan connection is lost.

Maybe you could test with option enabled ‘0’ in /etc/config/mwan3 under config interface ‘Lte’ and also disable the interface. Then use a script that will be enabling the interface and also changing option enabled ‘1’ in /etc/config/mwan3 under config interface ‘Lte’.

Now about the card, I assume that you are referring to the LTE modem. I do have problems also but I have not figured out the problem yet. I assume that it may be a problem of my provider. It takes long time for the modem to establish a stable connection. Could you please check your logs also?
Take a look below.

#14

Why not try the easy solution first? If no traffic is routed through the interface that should be enough to register as 0 traffic at the provider. Only if that doesnt work one can look for other solutions. But there is no sense in choosing the most difficult route without trying the easy solution first…

#15

When the interface is enabled, there is always data passing through the interface.
The modem sends the commands after reboot so I assume zero traffic is not possible.

#16

actually the version with no ping checks (assume always active) works fine even if lte link is established. It seems that I’m not billed for simply beeing connected.

Thanks for the solution

#17