OpenVPN-client routing problem

Cabal · July 2, 2019, 7:58am

I have a problem with OpenVPN in Turris 4.0
OpenVPN configuration is moved from 3.11.5
Firewall settings was recreated in Luci.
I’m able to ping all local openvpn interfaces (in vpn zone) from machine in lan zone.
I’m able to ping all remote openvpn interfaces and remote hosts from router.
I’m not able to ping any remote openvpn interface or remote host from machine in lan zone (Destination Port Unreachable)
I assume that problem is in routing / masquerading, but cannot track it.
I’ve tried moving interfaces to WAN zone, but with no effect.
All required routes on Omnia are present.

network

config interface ‘loopback’
option ifname ‘lo’
option proto ‘static’
option ipaddr ‘127.0.0.1’
option netmask ‘255.0.0.0’

config globals ‘globals’
option ula_prefix ‘fd1b:6a2c:65f1::/48’

config interface ‘lan’
option type ‘bridge’
option proto ‘static’
option netmask ‘255.255.255.0’
option ip6assign ‘60’
option bridge_empty ‘1’
list ifname ‘lan0’
list ifname ‘lan1’
list ifname ‘lan2’
list ifname ‘lan3’
list ifname ‘lan4’
option _turris_mode ‘managed’
option ipaddr ‘192.168.9.1’

config interface ‘wan’
option ifname ‘eth2’
option proto ‘dhcp’
option ipv6 ‘0’

config interface ‘wan6’
option ifname ‘@wan’
option proto ‘none’

config interface ‘guest_turris’
option type ‘bridge’
option proto ‘static’
option ipaddr ‘10.111.222.1’
option netmask ‘255.255.255.0’
option bridge_empty ‘1’
option enabled ‘0’

config interface ‘client1’
option proto ‘none’
list ifname ‘tun_client1’

config interface ‘client2’
option proto ‘none’
list ifname ‘tun_client2’
[other client interfaces here]

firewall

config defaults
option syn_flood ‘1’
option input ‘ACCEPT’
option output ‘ACCEPT’
option forward ‘REJECT’
option drop_invalid ‘1’

config zone
option name ‘lan’
option input ‘ACCEPT’
option output ‘ACCEPT’
option forward ‘ACCEPT’
option network ‘lan’

config zone
option name ‘wan’
option input ‘REJECT’
option output ‘ACCEPT’
option forward ‘REJECT’
option masq ‘1’
option mtu_fix ‘1’
option network ‘wan wan6’

config rule
option name ‘Allow-DHCP-Renew’
option src ‘wan’
option proto ‘udp’
option dest_port ‘68’
option target ‘ACCEPT’
option family ‘ipv4’

config rule
option name ‘Allow-Ping’
option src ‘wan’
option proto ‘icmp’
option icmp_type ‘echo-request’
option family ‘ipv4’
option target ‘ACCEPT’

config rule
option name ‘Allow-IGMP’
option src ‘wan’
option proto ‘igmp’
option family ‘ipv4’
option target ‘ACCEPT’

config rule
option name ‘Allow-DHCPv6’
option src ‘wan’
option proto ‘udp’
option src_ip ‘fc00::/6’
option dest_ip ‘fc00::/6’
option dest_port ‘546’
option family ‘ipv6’
option target ‘ACCEPT’

config rule
option name ‘Allow-MLD’
option src ‘wan’
option proto ‘icmp’
option src_ip ‘fe80::/10’
list icmp_type ‘130/0’
list icmp_type ‘131/0’
list icmp_type ‘132/0’
list icmp_type ‘143/0’
option family ‘ipv6’
option target ‘ACCEPT’

config rule
option name ‘Allow-ICMPv6-Input’
option src ‘wan’
option proto ‘icmp’
list icmp_type ‘echo-request’
list icmp_type ‘echo-reply’
list icmp_type ‘destination-unreachable’
list icmp_type ‘packet-too-big’
list icmp_type ‘time-exceeded’
list icmp_type ‘bad-header’
list icmp_type ‘unknown-header-type’
list icmp_type ‘router-solicitation’
list icmp_type ‘neighbour-solicitation’
list icmp_type ‘router-advertisement’
list icmp_type ‘neighbour-advertisement’
option limit ‘1000/sec’
option family ‘ipv6’
option target ‘ACCEPT’

config rule
option name ‘Allow-ICMPv6-Forward’
option src ‘wan’
option dest ‘*’
option proto ‘icmp’
list icmp_type ‘echo-request’
list icmp_type ‘echo-reply’
list icmp_type ‘destination-unreachable’
list icmp_type ‘packet-too-big’
list icmp_type ‘time-exceeded’
list icmp_type ‘bad-header’
list icmp_type ‘unknown-header-type’
option limit ‘1000/sec’
option family ‘ipv6’
option target ‘ACCEPT’

config rule
option name ‘Allow-IPSec-ESP’
option src ‘wan’
option dest ‘lan’
option proto ‘esp’
option target ‘ACCEPT’

config rule
option name ‘Allow-ISAKMP’
option src ‘wan’
option dest ‘lan’
option dest_port ‘500’
option proto ‘udp’
option target ‘ACCEPT’

config include
option path ‘/etc/firewall.user’

config zone ‘guest_turris’
option name ‘guest_turris’
option input ‘REJECT’
option forward ‘REJECT’
option output ‘ACCEPT’
option enabled ‘0’
option network ‘guest_turris’

config forwarding ‘guest_turris_forward_wan’
option name ‘guest to wan forward’
option src ‘guest_turris’
option dest ‘wan’
option enabled ‘0’

config rule ‘guest_turris_dns_rule’
option name ‘guest dns rule’
option src ‘guest_turris’
option proto ‘tcpudp’
option dest_port ‘53’
option target ‘ACCEPT’
option enabled ‘0’

config rule ‘guest_turris_dhcp_rule’
option name ‘guest dhcp rule’
option src ‘guest_turris’
option proto ‘udp’
option src_port ‘67-68’
option dest_port ‘67-68’
option target ‘ACCEPT’
option enabled ‘0’

config rule ‘wan_ssh_turris_rule’
option name ‘wan_ssh_turris_rule’
option enabled ‘0’
option target ‘ACCEPT’
option dest_port ‘22’
option proto ‘tcp’
option src ‘wan’

config rule ‘wan_http_turris_rule’
option name ‘wan_http_turris_rule’
option enabled ‘0’
option target ‘ACCEPT’
option dest_port ‘80’
option proto ‘tcp’
option src ‘wan’

config rule ‘wan_https_turris_rule’
option name ‘wan_https_turris_rule’
option enabled ‘0’
option target ‘ACCEPT’
option dest_port ‘443’
option proto ‘tcp’
option src ‘wan’

config rule ‘turris_wan_6in4_rule’
option enabled ‘0’

[redirects]

config zone
option forward ‘REJECT’
option output ‘ACCEPT’
option name ‘VPN’
option input ‘REJECT’
option masq ‘1’
option mtu_fix ‘1’
option network ‘client1 client2 etc…’

config include ‘bcp38’
option type ‘script’
option path ‘/usr/lib/bcp38/run.sh’
option family ‘IPv4’
option reload ‘1’

config include ‘miniupnpd’
option type ‘script’
option path ‘/usr/share/miniupnpd/firewall.include’
option family ‘any’
option reload ‘1’

config forwarding
option dest ‘VPN’
option src ‘lan’

config forwarding
option dest ‘wan’
option src ‘lan’

protree · July 2, 2019, 8:10am

forwarding is rejected by default for Interface VPN (GOOD)

Forwarding from VPN to LAN is allowed, so VPN clients can ping LAN clients (GOOD if desired)

What is missing is forwarding from LAN to VPN, so at the moment no LAN clients can ping VPN clients. Add:

config forwarding
option dest ‘lan’
option src ‘VPN’

EDIT:

To get a better overview you can use LuCi → Firewall → Zones:

It shows you which zones can forward to each other on the right, default Ruleset for each zone in the middle (default if no other rule for this interface matches) and Masquerading (Source NAT) on the right.

In your (not working) case LAN → VPN should be missing.

You can then easily edit Inter-Zone-Forwarding for each zone if you click on Edit on the right

Cabal · July 2, 2019, 8:26am

Lan => VPN forwarding should be enabled.
Look:

I’ve also tried move all TUN interfaces from VPN to WAN, but it didn’t help.

protree · July 2, 2019, 8:33am

Ok… But now VPN → LAN is missing? And I can’t find “rule forwarding lan → VPN” in your config you posted above? I assume you changed configuration in the meanwhile?

Nevertheless it seems that there is another problem. I just saw that the forwarding rule was missing and thought I could help you easily because I thought maybe you just oversaw it.

I have no time to help you further today, sorry. You could check output of “ip routes” to see if all routes are really there… Maybe someone else can help…

I wouldn’t recommand that

Good luck

Cabal · July 2, 2019, 8:51am

Rule Lan -> VPN is at the end of /etc/config/firewall (second from end).
VPN -> lan is missing because I treat VPN as kind of wan, I don’t want lan to be accesslible from VPN.
I know that mixing wan and vpn interfaces is not a good idea, but just wanted to test if there is something wrong with VPN zone.
IP routes are ok, all are on place (and hosts on the other side are accessible from router, access from lan is missing).
Thanks for your efforts.

protree · July 2, 2019, 10:15am

dumb me I mixed up dest and src…

Cabal · July 2, 2019, 7:21pm

Ok, the problem is in zone_vpn* firewall rules.
They are created, but not referenced nowhere.

iptables-save | grep vpn

root@Turris:~# iptables-save | grep vpn
:zone_vpn_postrouting - [0:0]
:zone_vpn_prerouting - [0:0]
-A zone_vpn_postrouting -m comment --comment “!fw3” -j MASQUERADE
:zone_vpn_dest_ACCEPT - [0:0]
:zone_vpn_dest_REJECT - [0:0]
:zone_vpn_forward - [0:0]
:zone_vpn_input - [0:0]
:zone_vpn_output - [0:0]
:zone_vpn_src_REJECT - [0:0]
-A zone_lan_forward -m comment --comment “!fw3: Zone lan to vpn forwarding policy” -j zone_vpn_dest_ACCEPT
-A zone_vpn_forward -m conntrack --ctstate DNAT -m comment --comment “!fw3: Accept port forwards” -j ACCEPT
-A zone_vpn_forward -m comment --comment “!fw3” -j zone_vpn_dest_REJECT
-A zone_vpn_input -m conntrack --ctstate DNAT -m comment --comment “!fw3: Accept port redirections” -j ACCEPT
-A zone_vpn_input -m comment --comment “!fw3” -j zone_vpn_src_REJECT
-A zone_vpn_output -m comment --comment “!fw3” -j zone_vpn_dest_ACCEPT

Lan and wan zones are referenced in main tables (INPUT, OUTPUT, FORWARD, etc…)
Vpn zone is not.
Have no idea why.

peci1 · July 3, 2019, 6:09am

Two ideas:

Ping needs forwarding in both directions (or NAT to the VPN interface). You can run wireshark on one end and ping at the other to see if the packets are coming there but not back.

You also need to set up routes correctly on the clients, not only on the router. Usually, when you connect to VPN, it only sets route vpn -> 192.168.x.y, but you need vpn -> 192.168.x.*. Try it manually first (after connecting to the VPN), and if it works, you can try setting the route in a script that runs after connecting to the VPN.

Cabal · July 3, 2019, 6:32am

As far as I know, routes on client machine are useless: client has no knowlege about vpn interfaces, and has default route (for all packets, that does not match definied routes), that points to router.
That worked fine in 3.11.5. Stopped in 4.0 beta 3-4.

Forwarding is in one direction, but with NAT. Should work (and worked in 3.11.5).

Cabal · July 18, 2019, 9:37am

Ok.
I’ve spent a little bit time to check what iptables-rules are missing.
Then I’ve created shell script for custom iptables rules (configured in Luci), and it works (only after applying after vpn start).

ifconfig | grep tun | while read if x; do
	iptables -t nat -A PREROUTING -i ${if} -m comment --comment "!fw3" -j zone_vpn_prerouting
	iptables -t nat -A POSTROUTING -o ${if}  -m comment --comment "!fw3" -j zone_vpn_postrouting
	iptables -A INPUT -i ${if} -m comment --comment "!fw3" -j zone_vpn_input
	iptables -I FORWARD 6 -i ${if} -m comment --comment "!fw3" -j zone_vpn_forward
	iptables -A OUTPUT -o ${if} -m comment --comment "!fw3" -j zone_vpn_output
done

iptables -A zone_vpn_src_REJECT  -m comment --comment "!fw3" -j reject
iptables -A zone_vpn_dest_REJECT -m comment --comment "!fw3" -j reject
iptables -A zone_vpn_dest_ACCEPT -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
iptables -A zone_vpn_dest_ACCEPT -m comment --comment "!fw3" -j ACCEPT
iptables -A zone_vpn_input -m comment --comment "!fw3" -j zone_vpn_src_REJECT
iptables -A zone_vpn_output -m comment --comment "!fw3: Custom wan output rule chain" -j output_vpn_rule
iptables -A zone_vpn_output -m comment --comment "!fw3" -j zone_vpn_dest_ACCEPT

Cabal · August 10, 2019, 4:49pm

Workaround does not work as well as expected.
It is known bug in OpenWRT:
https://bugs.openwrt.org/index.php?do=details&task_id=1278&string=firewall

Modified script looks like below (it is now verified if works after firewall restart)

firewall.user

#!/bin/sh

/sbin/ifconfig | /bin/grep tun | while read if x; do
/usr/sbin/iptables -t nat -A PREROUTING -i ${if} -m comment --comment “openvpn” -j zone_vpn_prerouting
/usr/sbin/iptables -t nat -A POSTROUTING -o ${if} -m comment --comment “openvpn” -j zone_vpn_postrouting
/usr/sbin/iptables -A INPUT -i ${if} -m comment --comment “openvpn” -j zone_vpn_input
/usr/sbin/iptables -I FORWARD 6 -i ${if} -m comment --comment “openvpn” -j zone_vpn_forward
/usr/sbin/iptables -A OUTPUT -o ${if} -m comment --comment “openvpn” -j zone_vpn_output
done

/usr/sbin/iptables -A zone_vpn_src_REJECT -m comment --comment “openvpn” -j reject
/usr/sbin/iptables -A zone_vpn_dest_REJECT -m comment --comment “openvpn” -j reject
/usr/sbin/iptables -A zone_vpn_dest_ACCEPT -m conntrack --ctstate INVALID -m comment --comment “openvpn: Prevent NAT leakage” -j DROP
/usr/sbin/iptables -A zone_vpn_dest_ACCEPT -m comment --comment “openvpn” -j ACCEPT
/usr/sbin/iptables -A zone_vpn_input -m comment --comment “openvpn” -j zone_vpn_src_REJECT
/usr/sbin/iptables -A zone_vpn_output -m comment --comment “openvpn: Custom wan output rule chain” -j output_vpn_rule
/usr/sbin/iptables -A zone_vpn_output -m comment --comment “openvpn” -j zone_vpn_dest_ACCEPT

Cabal · August 11, 2019, 6:19pm

Script above also does not work as expected (it is good on firewall restart, but after some time firewall is refreshed and added rules are on the beginning).
I need to learn more about how firewall works.
Or fix fw3…

Cabal · September 8, 2019, 6:59pm

Another try.
I still don’t know root cause, but I’m a little bit closer.
fw3 prints error message because interface names are not registered in ubus (propably by netifd).

So a little update to /etc/rc.local should help.

/etc/rc/local

#!/bin/sh

NETS=/sbin/uci get firewall.@zone[2].network
for NET in ${NETS}; do
IFACE=/sbin/uci get "network.${NET}.ifname"
ubus call network.interface.${NET} remove_device ‘{“name”: "’${IFACE}’"}’
ubus call network.interface.${NET} add_device ‘{“name”: "’${IFACE}’"}’
done

/etc/init.d/firewall restart

exit 0

Changes to /etc/firewall.user are removec.
Results will be known in few days after reloads.
For now it behaves as desired at boot.