Connection problems between LXC container and physical machines on LAN (due to MTU)

wowbaggerHU · February 9, 2022, 9:46am

Hi Everyone,

I have a problem where I have an MTU of 9000 configured for the local “LAN” network via LUCI.
I also am pushing the 9000 MTU value over DHCP to client machines, and they seem to use that value too.
I also have a Ubuntu-based LXC container which seems to also be connected/bridged to the “LAN” network, and that also picks up the 9000 MTU value.
However when connecting to the LXC container over SSH, the ssh connection stalls at the beginning, seemingly for no reason. Also, if I use a custom custom/short MAC list when connecting, and I resume a screen session with a large screen size, the screen update and the whole connection stops.
When looking at this in tcpdump, I saw that the container is sending a packet with length 2852, which doesn’t get through to my physical machine. It is only visible on the Turris Omnia router.
This indicates an MTU problem, however I don’t know which interface should receive an updated MTU for this to work?

vcunat · February 9, 2022, 1:39pm

Side note: I personally think that in almost all scenarios (in SOHO at least) there’s too little to gain from increasing MTU. I’m not including some special datacenter use cases.

hagrid · February 9, 2022, 3:53pm

Well, can you paste the output of ip -d link while the container is running?

wowbaggerHU · February 9, 2022, 4:11pm

Sure… as far as I can see MTU is 1500 everywhere:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 532
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 1 addrgenmode eui64 numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 100 
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 532
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 4 addrgenmode eui64 numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 100 
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 532
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 addrgenmode eui64 numtxqueues 8 numrxqueues 8 gso_max_size 65536 gso_max_segs 100 
5: lan0@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 1 
    bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8001 port_no 0x1 designated_port 32769 designated_cost 0 designated_bridge 7fff.D8:58:D7:00:3E:6D designated_root 7fff.D8:58:D7:00:3E:6D hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 portname p0 switchid 00000000 
6: lan1@eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 1 
    bridge_slave state disabled priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8002 port_no 0x2 designated_port 32770 designated_cost 0 designated_bridge 7fff.D8:58:D7:00:3E:6D designated_root 7fff.D8:58:D7:00:3E:6D hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 portname p1 switchid 00000000 
7: lan2@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 1 
    bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8003 port_no 0x3 designated_port 32771 designated_cost 0 designated_bridge 7fff.D8:58:D7:00:3E:6D designated_root 7fff.D8:58:D7:00:3E:6D hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 portname p2 switchid 00000000 
8: lan3@eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 1 
    bridge_slave state disabled priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8004 port_no 0x4 designated_port 32772 designated_cost 0 designated_bridge 7fff.D8:58:D7:00:3E:6D designated_root 7fff.D8:58:D7:00:3E:6D hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 portname p3 switchid 00000000 
9: lan4@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 1 
    bridge_slave state disabled priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8005 port_no 0x5 designated_port 32773 designated_cost 0 designated_bridge 7fff.D8:58:D7:00:3E:6D designated_root 7fff.D8:58:D7:00:3E:6D hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 portname p4 switchid 00000000 
10: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/tunnel6 :: brd :: promiscuity 0 
    ip6tnl ip6ip6 remote any local any hoplimit inherit encaplimit 0 tclass 0x00 flowlabel 0x00000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
11: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0 promiscuity 0 
    sit ip6ip remote any local any ttl 64 nopmtudisc 6rd-prefix 2002::/16 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
12: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 32
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 
    ifb addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
13: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 32
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 
    ifb addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
14: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0 promiscuity 0 
    gre remote any local any ttl inherit nopmtudisc numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
15: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 
    gretap remote any local any ttl inherit nopmtudisc addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
16: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 
    erspan remote any local any ttl inherit nopmtudisc addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
17: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 addrgenmode eui64 numtxqueues 4 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
18: wlan1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 addrgenmode eui64 numtxqueues 4 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
19: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF promiscuity 0 
    bridge forward_delay 200 hello_time 200 max_age 2000 ageing_time 30000 stp_state 0 priority 32767 vlan_filtering 0 vlan_protocol 802.1Q bridge_id 7fff.D8:58:D7:00:3E:6D designated_root 7fff.D8:58:D7:00:3E:6D root_port 0 root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer    0.00 tcn_timer    0.00 topology_change_timer    0.00 gc_timer  283.38 vlan_default_pvid 1 vlan_stats_enabled 0 group_fwd_mask 0 group_address AA:BB:CC:DD:EE:FF mcast_snooping 0 mcast_router 1 mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 4 mcast_hash_max 512 mcast_last_member_count 2 mcast_startup_query_count 2 mcast_last_member_interval 100 mcast_membership_interval 26000 mcast_querier_interval 25500 mcast_query_interval 12500 mcast_query_response_interval 1000 mcast_startup_query_interval 3125 mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables 0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
26: veth1VJDK1@if25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default qlen 1000
    link/ether AA:BB:CC:DD:EE:FF brd AA:BB:CC:DD:EE:FF link-netnsid 0 promiscuity 1 
    veth 
    bridge_slave state forwarding priority 32 cost 2 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8006 port_no 0x6 designated_port 32774 designated_cost 0 designated_bridge 7fff.D8:58:D7:00:3E:6D designated_root 7fff.D8:58:D7:00:3E:6D hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

hagrid · February 9, 2022, 4:26pm

So I guess you see the issue.

wowbaggerHU · February 9, 2022, 4:45pm

Okay, but I already set the MTU for the “LAN” network to 9000 in LUCI. What else am I supposed to do? AFAIK that should be the place to set such things.
The “LAN” network consists of br-lan, which in turn is lan0 through lan4 and the two WLAN adapters.

turrios · February 9, 2022, 7:59pm

Here is how I fixed mtu to 8000 in order to take advantage of my gigabit.
You set the real eth nic mtu to 9000 and you create a bridge with 8000 mtu where you bind the lan nics and the virtual (veth) nic.
Wifi cannot be part of that bridge. I run an LXC VM with mtu 8000 connected to that bridge.

part of /etc/config/network:

config interface 'lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option _orig_ifname 'eth0 eth2 wlan0 wlan1'
        option _orig_bridge 'true'
        option mtu '8000'
        option type 'bridge'
        option ifname 'lan0 vethvm1'
        option delegate '0'

config device
        option name 'eth1'
        option mtu '9000'

config device
        option name 'lan0'
        option mtu '8000'

system · October 24, 2022, 11:13am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.