After ugrade router is running 100% CPU, network do have incredible lags and not working almost at all…
I cant debug or something, as its not almost working…
Most process time takes by forris-controller
For others, I think it will be useful to say if this is Turris 1.x or some unusual configuration. (I noticed unbound at a quick glance.)
True, it.s 1.0 turris… After 1h ive kill most of process and now seems that there is huge events to process on kernel level…
But every action is for several minutes…its terrible. Untill upgrade it was working all fine.
If it helps somehow…
root@turris:~# cat /proc/interrupts
CPU0 CPU1
19: 2473 0 OpenPIC 19 Level fsl-lbc
24: 0 0 OpenPIC 24 Level [PCI] PME, aerdrv, PCIe PME
25: 0 0 OpenPIC 25 Level [PCI] PME, aerdrv, PCIe PME
26: 0 0 OpenPIC 26 Level [PCI] PME, aerdrv, PCIe PME
27: 5086 0 OpenPIC 4 Level ath9k
28: 0 23 OpenPIC 28 Level ehci_hcd:usb1
29: 91252482 0 OpenPIC 29 Level eth0_g0_tx
30: 0 2560859 OpenPIC 30 Level eth0_g0_rx
31: 203351 0 OpenPIC 31 Level eth2_g0_tx
32: 0 49696 OpenPIC 32 Level eth2_g0_rx
33: 0 0 OpenPIC 33 Level eth2_g0_er
34: 0 0 OpenPIC 34 Level eth0_g0_er
35: 0 2681581 OpenPIC 35 Level eth1_g0_tx
36: 105638071 0 OpenPIC 36 Level eth1_g0_rx
40: 0 0 OpenPIC 40 Level eth1_g0_er
41: 12626 0 fsl-msi-224 0 Edge ath10k_pci
42: 242 0 OpenPIC 42 Level serial
43: 1702 0 OpenPIC 43 Level i2c-mpc, i2c-mpc
59: 0 0 OpenPIC 59 Level fsl_espi
72: 0 11368 OpenPIC 72 Level mmc0
224: 12626 0 OpenPIC 224 Edge fsl-msi-cascade
225: 0 0 OpenPIC 225 Edge fsl-msi-cascade
226: 0 0 OpenPIC 226 Edge fsl-msi-cascade
227: 0 0 OpenPIC 227 Edge fsl-msi-cascade
228: 0 0 OpenPIC 228 Edge fsl-msi-cascade
229: 0 0 OpenPIC 229 Edge fsl-msi-cascade
230: 0 0 OpenPIC 230 Edge fsl-msi-cascade
231: 0 0 OpenPIC 231 Edge fsl-msi-cascade
507: 67399 1024743 OpenPIC 2043 Edge ipi call function
508: 875104 415973 OpenPIC 2044 Edge ipi reschedule
509: 0 0 OpenPIC 2045 Edge ipi tick-broadcast
LOC: 2446290 2395094 Local timer interrupts for timer event device
LOC: 8 1 Local timer interrupts for others
SPU: 1 0 Spurious interrupts
PMI: 0 0 Performance monitoring interrupts
MCE: 0 0 Machine check exceptions
root@turris:~#
and dmesq is flooded with:
[ 2723.655341] br-lan: received packet on eth0 with own address as source address
[ 2723.655350] br-lan: received packet on eth0 with own address as source address
[ 2724.906014] turris-00000000: IN=eth2 OUT= MAC=d8:58:d7:00:0a:d3:44:82:e5:76:9b:5a:08:00 SRC=170.106.115.151 DST=164.215.116.174 LEN=52 TOS=0x00 PREC=0x00 TTL=48 ID=10894 PROTO=TCP SPT=11918 DPT=9454 WINDOW=65535 RES=0x00 SYN URGP=0
[ 2728.656431] net_ratelimit: 67533 callbacks suppressed
[ 2728.656441] br-lan: received packet on eth0 with own address as source address
[ 2728.656452] br-lan: received packet on eth0 with own address as source address
[ 2728.656467] br-lan: received packet on eth0 with own address as source address
[ 2728.657397] br-lan: received packet on eth0 with own address as source address
[ 2728.657420] br-lan: received packet on eth0 with own address as source address
[ 2728.657429] br-lan: received packet on eth0 with own address as source address
[ 2728.657439] br-lan: received packet on eth0 with own address as source address
[ 2728.658221] br-lan: received packet on eth0 with own address as source address
[ 2728.658236] br-lan: received packet on eth0 with own address as source address
[ 2728.658253] br-lan: received packet on eth0 with own address as source address
[ 2733.657424] net_ratelimit: 78237 callbacks suppressed
[ 2733.657434] br-lan: received packet on eth0 with own address as source address
[ 2733.657446] br-lan: received packet on eth0 with own address as source address
[ 2733.657462] br-lan: received packet on eth0 with own address as source address
[ 2733.657471] br-lan: received packet on eth0 with own address as source address
[ 2733.657481] br-lan: received packet on eth0 with own address as source address
[ 2733.657490] br-lan: received packet on eth0 with own address as source address
[ 2733.657499] br-lan: received packet on eth0 with own address as source address
[ 2733.657508] br-lan: received packet on eth0 with own address as source address
[ 2733.657517] br-lan: received packet on eth0 with own address as source address
[ 2733.657526] br-lan: received packet on eth0 with own address as source address
[ 2738.658525] net_ratelimit: 90225 callbacks suppressed
[ 2738.658535] br-lan: received packet on eth0 with own address as source address
[ 2738.658546] br-lan: received packet on eth0 with own address as source address
[ 2738.658561] br-lan: received packet on eth0 with own address as source address
[ 2738.658570] br-lan: received packet on eth0 with own address as source address
[ 2738.658579] br-lan: received packet on eth0 with own address as source address
[ 2738.658588] br-lan: received packet on eth0 with own address as source address
[ 2738.658597] br-lan: received packet on eth0 with own address as source address
[ 2738.658606] br-lan: received packet on eth0 with own address as source address
[ 2738.659509] br-lan: received packet on eth0 with own address as source address
[ 2738.659533] br-lan: received packet on eth0 with own address as source address
[ 2738.858363] turris-00000000: IN=eth2 OUT= MAC=d8:58:d7:00:0a:d3:44:82:e5:76:9b:5a:08:00 SRC=185.94.111.1 DST=164.215.116.174 LEN=40 TOS=0x08 PREC=0x20 TTL=242 ID=54321 PROTO=TCP SPT=53604 DPT=179 WINDOW=65535 RES=0x00 SYN URGP=0
[ 2741.784202] turris-00000000: IN=eth2 OUT= MAC=d8:58:d7:00:0a:d3:44:82:e5:76:9b:5a:08:00 SRC=45.146.164.98 DST=164.215.116.174 LEN=40 TOS=0x00 PREC=0x20 TTL=245 ID=62594 PROTO=TCP SPT=43292 DPT=53121 WINDOW=1024 RES=0x00 SYN URGP=0
[ 2741.845011] turris-00000000: IN=eth2 OUT= MAC=d8:58:d7:00:0a:d3:44:82:e5:76:9b:5a:08:00 SRC=45.146.164.98 DST=164.215.116.174 LEN=40 TOS=0x00 PREC=0x00 TTL=245 ID=62593 PROTO=TCP SPT=43292 DPT=53121 WINDOW=1200 RES=0x00 RST URGP=0
[ 2743.660461] net_ratelimit: 89106 callbacks suppressed
[ 2743.660471] br-lan: received packet on eth0 with own address as source address
[ 2743.661308] br-lan: received packet on eth0 with own address as source address
[ 2743.661321] br-lan: received packet on eth0 with own address as source address
[ 2743.661336] br-lan: received packet on eth0 with own address as source address
[ 2743.661363] br-lan: received packet on eth0 with own address as source address
[ 2743.661375] br-lan: received packet on eth0 with own address as source address
[ 2743.661384] br-lan: received packet on eth0 with own address as source address
[ 2743.661394] br-lan: received packet on eth0 with own address as source address
[ 2743.661403] br-lan: received packet on eth0 with own address as source address
[ 2743.661424] br-lan: received packet on eth0 with own address as source address
It’s obvious some network setup probably, but have no idea what can be problem… probably some DNS masquarade or something… but strange is, that it started after upgrade…
Sorry, but we are not able to provide you any support based on screenshot w/o useful details to be able to reproduce it. What we only know that your CPU usage is 100%.
If you want to help with the issue, which you are having, I suggest reading these two articles:
Usual questions:
- Does the issue happen after reboot?
- Did you somehow edited configuration files related to mosquitto/fosquitto?
- Do you have any free space left?
- What I should do to be able to reproduce it?
I can think of more questions if you want, but as I said. You need to provide more details.
1 Like
Well, what I noticed is, that there were autoupdate, as there were TOS3 and not there is 5. All starts right after restart. Problem is, that it almost not working and I’m not able to connect to device and every command takes 5 min to even write.
Before upgrade I didn’t change anything and I’m not aware any change of mosquito / fosquitto.
Space isn’t problem
root@turris:~# df -h
Filesystem Size Used Available Use% Mounted on
/dev/mmcblk0p2 14.3G 818.2M 13.4G 6% /
tmpfs 512.0K 0 512.0K 0% /sys/fs/cgroup
tmpfs 1011.5M 16.7M 994.7M 2% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
That’s the problem - it used to work with TOS3 without problem. I’ve lost WiFi, so I thought it may be some stuck or something, so I did restart of router (removing of power and plug-in back).
But after reboot, Wifi start working, but no internet and when I ping the router, it reponse every 10th or more request… I hade to move to wired network to be able to connect to router, but it timeouts a lot. Now, after killing most of process, I can, at least somehow work with, but still, one core is completely utilised by softevents. Probably because of network?. Do you see some issues in config? As I don’t…
root@turris:~# cat /etc/config/network
config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config interface 'lan'
option ifname 'eth0 eth1'
option force_link '1'
option type 'bridge'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '64'
option ip6hint '0'
option delegate '0'
config interface 'wan'
option ifname 'eth2'
option proto 'dhcp'
option peerdns '0'
option dns '8.8.8.8 8.8.4.4'
option hostname 'turris'
option ipv6 '1'
config interface 'wan6'
option _orig_ifname '@wan'
option _orig_bridge 'false'
option proto 'dhcpv6'
option ifname 'eth2'
option reqaddress 'try'
option reqprefix 'auto'
config switch
option name 'switch0'
option reset '1'
config switch_vlan
option device 'switch0'
option vlan '1'
option ports '0 1 2 3 4'
option vid '1'
config switch_vlan
option device 'switch0'
option vlan '2'
option ports '5 6'
option vid '2'
config interface 'guest_turris'
option enabled '1'
option type 'bridge'
option proto 'static'
option ipaddr '10.111.222.1'
option netmask '255.255.255.0'
option bridge_empty '1'
list ifname 'guest_turris_0'
list ifname 'guest_turris_1'
config interface 'vpn_turris'
option enabled '1'
option ifname 'tun_turris'
option proto 'none'
option auto '1'
But for me, looks that there is even some more network interfaces as well…
root@turris:~# ifconfig -a
root@turris:~# ifconfig -a
br-guest_turris Link encap:Ethernet HWaddr 06:FA:AC:26:DE:55
inet addr:10.111.222.1 Bcast:10.111.222.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
br-lan Link encap:Ethernet HWaddr D8:58:D7:00:0A:D1
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::da58:d7ff:fe00:ad1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:264776243 errors:0 dropped:0 overruns:0 frame:0
TX packets:248460 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:38473151519 (35.8 GiB) TX bytes:70049336 (66.8 MiB)
eth0 Link encap:Ethernet HWaddr D8:58:D7:00:0A:D1
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:150818774 errors:0 dropped:6155 overruns:0 frame:0
TX packets:259468702 errors:0 dropped:0 overruns:45 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3048269303 (2.8 GiB) TX bytes:3137752359 (2.9 GiB)
Base address:0x4000
eth1 Link encap:Ethernet HWaddr D8:58:D7:00:0A:D2
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:260557126 errors:0 dropped:1208 overruns:0 frame:0
TX packets:151065875 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3230286511 (3.0 GiB) TX bytes:3117459277 (2.9 GiB)
Base address:0x6000
eth2 Link encap:Ethernet HWaddr D8:58:D7:00:0A:D3
inet6 addr: fe80::da58:d7ff:fe00:ad3/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:229341 errors:0 dropped:1 overruns:0 frame:0
TX packets:271166 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:188648167 (179.9 MiB) TX bytes:181314487 (172.9 MiB)
Base address:0x8000
gre0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-80-00-00-00-00-00-00-00-00-00
NOARP MTU:1476 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
gretap0 Link encap:Ethernet HWaddr 00:00:00:00:00:00
BROADCAST MULTICAST MTU:1462 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ifb0 Link encap:Ethernet HWaddr 7E:D8:61:D5:F3:B6
BROADCAST NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ifb1 Link encap:Ethernet HWaddr C2:E1:5A:F5:9D:9D
BROADCAST NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ip6tnl0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:1452 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:146245 errors:0 dropped:0 overruns:0 frame:0
TX packets:146245 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:14776290 (14.0 MiB) TX bytes:14776290 (14.0 MiB)
sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
teql0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
tun_turris Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.111.111.1 P-t-P:10.111.111.1 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 B) TX bytes:1303 (1.2 KiB)
wlan0 Link encap:Ethernet HWaddr 04:F0:21:31:D8:30
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
wlan1 Link encap:Ethernet HWaddr 60:02:B4:7D:89:1C
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)```Yep, looks that rename of /etc/conf/network fixed the issue. But it doesn’t work the LAN now. What is minimal conf, so I can see the LAN interfaces in configuration?
this is problematic part causing some loopback. Without it it works fine, but problem is, that then, logically, it doesn’t work communication on LAN. Any idea how it should be in TOS5?
Well, I’ve rollback with schnaps and even when I set manual upgrade, router did upgraded itself, however this time network confic was migrated too as well…
config interface 'lan'
option force_link '1'
option type 'bridge'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '64'
option ip6hint '0'
option delegate '0'
list ifname 'lan1'
list ifname 'lan5'
list ifname 'lan4'
list ifname 'lan3'
list ifname 'lan2'
and no more problems with CPU. So it was all around new config standard.
config interface 'lan'
option type 'bridge'
option proto 'static'
option ipaddr '10.97.1.1'
option netmask '255.255.255.0'
option ifname 'lan1 lan2 lan3 lan4 lan5'
1 Like
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.