Blue LED / continious reboots

General discussion
1

As described here and also here in this thread, I have problems with my Turris Omnia. The router reboots continuously and the LED are blinking blue. In this condition I’m losing connection to the router and not even being able to ping - nor when I’m setting a static IP an plug in the LAN directly to the router.

As I’m not sure it’s related to the OS release, I’ll open a separated thread.

Can someone help me debugging this issue? What does these blue LED mean?

-deleted-

Here are some logs at the time when the error occurs (around 18:13).

2017-09-17T18:13:53+02:00 warning kernel[]: [   14.714057] ath10k_pci 0000:02:00.0: Direct firmware load for ath10k/pre-cal-pci-0000:02:00.0.bin failed with error -2
2017-09-17T18:13:53+02:00 warning kernel[]: [   14.724792] ath10k_pci 0000:02:00.0: Falling back to user helper
2017-09-17T18:13:53+02:00 warning kernel[]: [   14.743658] ath10k_pci 0000:02:00.0: Direct firmware load for ath10k/cal-pci-0000:02:00.0.bin failed with error -2
2017-09-17T18:13:53+02:00 warning kernel[]: [   14.754046] ath10k_pci 0000:02:00.0: Falling back to user helper
2017-09-17T18:13:53+02:00 warning kernel[]: [   14.843812] ath10k_pci 0000:02:00.0: board id is not exist in otp, ignore it
2017-09-17T18:13:53+02:00 warning kernel[]: [   14.850931] ath10k_pci 0000:02:00.0: Direct firmware load for ath10k/QCA988X/hw2.0/board-2.bin failed with error -2
2017-09-17T18:13:53+02:00 warning kernel[]: [   14.861398] ath10k_pci 0000:02:00.0: Falling back to user helper
2017-09-17T18:14:01+02:00 warning dnsmasq[1953]: overflow: 5 log entries lost
2017-09-17T16:14:07+02:00 warning pppd[2271]: Connected to *** via interface eth1.11
2017-09-17T18:14:08+02:00 warning dnsmasq[2427]: script process exited with status 1
2017-09-17T18:14:09+02:00 warning dnsmasq[2427]: script process exited with status 1
2017-09-17T16:14:09+02:00 warning odhcpd[1742]: A default route is present but there is no public prefix on br-lan thus we don't announce a default route!
2017-09-17T16:14:44+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T16:14:44+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T16:14:48+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:14:48+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 1 times, suppressed by syslog-ng on turris
2017-09-17T16:14:48+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:14:48+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 1 times, suppressed by syslog-ng on turris
2017-09-17T16:15:02+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:15:02+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 1 times, suppressed by syslog-ng on turris
2017-09-17T16:15:21+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:15:21+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:15:21+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:15:21+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:15:32+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:15:32+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:15:44+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:15:44+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:15:50+02:00 warning odhcpd[1742]: DHCPV6 SOLICIT IA_NA from 0001000120bdc5066476badaa720 on br-lan: ok *** *** 
2017-09-17T16:15:51+02:00 warning odhcpd[1742]: DHCPV6 REQUEST IA_NA from 0001000120bdc5066476badaa720 on br-lan: ok *** *** 
2017-09-17T16:15:56+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:15:56+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:16:00+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:16:00+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:17:04+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:17:04+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:17:55+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:17:55+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:18:02+02:00 warning odhcpd[1742]: DHCPV6 REBIND IA_NA from 0001000120ef8de7b827ebb8b478 on br-lan: ok 
2017-09-17T16:18:11+02:00 warning odhcpd[1742]: DHCPV6 REBIND IA_NA from 0001000120ef8de7b827ebb8b478 on br-lan: ok 
2017-09-17T16:18:30+02:00 warning odhcpd[1742]: DHCPV6 REBIND IA_NA from 0001000120ef8de7b827ebb8b478 on br-lan: ok 
2017-09-17T16:19:07+02:00 warning odhcpd[1742]: DHCPV6 REBIND IA_NA from 0001000120ef8de7b827ebb8b478 on br-lan: ok 
2017-09-17T16:19:10+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:19:10+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:19:28+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:19:28+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:19:28+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:19:28+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T18:19:28+02:00 warning dnsmasq[2427]: script process exited with status 1
2017-09-17T16:19:44+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:19:44+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:19:45+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:19:45+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T18:19:45+02:00 warning dnsmasq[2427]: script process exited with status 1
2017-09-17T16:19:48+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:19:48+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:19:49+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:19:49+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T18:19:49+02:00 warning dnsmasq[2427]: script process exited with status 1
2017-09-17T16:20:03+02:00 warning watchdog[]: Restarted nethist
2017-09-17T16:20:18+02:00 warning odhcpd[1742]: DHCPV6 REBIND IA_NA from 0001000120ef8de7b827ebb8b478 on br-lan: ok 
2017-09-17T16:20:21+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:20:21+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T16:20:22+02:00 warning dhcp_host_domain_ng.py[]: Add_lease, hostname check failed
2017-09-17T18:20:22+02:00 warning dhcp_host_domain_ng.py[1898]: Last message 'Add_lease, hostname ' repeated 2 times, suppressed by syslog-ng on turris
2017-09-17T18:20:22+02:00 warning dnsmasq[2427]: script process exited with status 1

Thanks in advance.

2

This is really weird. It just constantly reboots it self. It boots up but some time during start it just reboots again.

I would suggest unfortunately to either try to debug it using serial cable (usb2ttl https://www.turris.cz/doc/en/troubleshooting/serial_link) if you have one or otherwise you probably have to do factory reset. Or write to support with request to send it to us. But this seems to be software error. I suspect some kernel error.

3

Thank you for your answer.

I’ve already reset the router several times. Rollback to latest snapshot, factory resetting and also re-flashing the Omnia with a USB-stick. The issue is still there - sometimes after minutes, sometimes it works for a couple hours.

Is there a way that it is relatad to my settings? I always reload my previously saved settings after the reset.

Nice, I didn’t know the router has a serial-port. Unfortunately I do not have this kind of serial-cable (null-modem is available). But this makes debugging a little easier.

BTW does this cable work as well: DeLock 83758?

4

I will make a test with a replacement/testing Turris Omnia from our company so I can see if it makes a difference (maybe hardware related issue).

5

It might be hardware related but it seems to me as being weird that it happened with 3.8 release. That would be interesting coincidence. Have you tries not to recover from backup? If it’s not something in that backup. Just configure Internet connection out of the box and testing it that way?

Yes that cable should work too but you have to change order of pins. You have to have RX and TX right next to GND.

6

The issue seems to be related to a faulty power supply.

I’ll try to debug over serial cable and maybe there are more information about it. I changed the power supply now it works without an hassle since then. When I change back the power supply, within minutes or hours the issue occurs - also on the second Turris Omnia, with different settings.

1 Like
7

This is the point where the Omnia reboots (screenlog with serial cable):

2017-09-20T09:11:43+02:00 err ntpdate[2226]: name server cannot be used: Try again (-3)
2017-09-20T09:11:43+02:00 emerg sfpswitch.py[1534]: Called /etc/init.d/kresd start
2017-09-20T09:11:44+02:00 crit mountd[2296]: Starting OpenWrt (auto)mountd V1
2017-09-20T09:11:44+02:00 crit mountd[2296]: trying to mount /tmp/run/mountd/ as the autofs root
2017-09-20T09:11:45+02:00 debug updater[2313]: src/lib/journal.c:123 (journal_open): Opening journal
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: DEBUG:src/lib/journal.c:123 (journal_open):Opening journal
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: line not found
2017-09-20T09:11:45+02:00 debug updater[2313]: src/lib/interpreter.c:104 (interpreter_error_result): 
Stack Traceback
===============
(2) Lua function '?' at line 64 of chunk '"logging"]'
	Local variables:
	 err = string: "[string \"transaction\"]:325: No journal to recover"
	 err2string = Lua function '?' (defined at line 45 of chunk "logging"])
	 msg = string: "\
[string \"transaction\"]:325: No journal to recover"
	 (*temporary) = table: 0x4c8090  {msg:
[string "transaction"]:325: No journal to recover}
(3)  C function 'function: 0x4c5e00'
(4) field C function 'recover'
(5) Lua global 'recover' at line 325 of chunk '"transaction"]'
	Local variables:
	 run_state = table: 0x28e8d0  {initialized:true, init:function: 0x28e900, lfile:userdata: 0x2dd568, release:function: 0x2bbd20 (more...)}
(6) Lua function '?' at line 386 of chunk '"transaction"]'
	Local variables:
	 (*temporary) = Lua function '?' (defined at line 352 of chunk "transaction"])
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: DEBUG:src/lib/interpreter.c:104 (interpreter_error_result):
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: Stack Traceback
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: ===============
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: (2) Lua function '?' at line 64 of chunk '"logging"]'
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	Local variables:
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	 err = string: "[string \"transaction\"]:325: No journal to recover"
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	 err2string = Lua function '?' (defined at line 45 of chunk "logging"])
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	 msg = string: "\
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: [string \"transaction\"]:325: No journal to recover"
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	 (*temporary) = table: 0x4c8090  {msg:
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: [string "transaction"]:325: No journal to recover}
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: (3)  C function 'function: 0x4c5e00'
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: (4) field C function 'recover'
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: (5) Lua global 'recover' at line 325 of chunk '"transaction"]'
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	Local variables:
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	 run_state = table: 0x28e8d0  {initialized:true, init:function: 0x28e900, lfile:userdata: 0x2dd568, release:function: 0x2bbd20 (more...)}
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: (6) Lua function '?' at line 386 of chunk '"transaction"]'
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	Local variables:
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 	 (*temporary) = Lua function '?' (defined at line 352 of chunk "transaction"])
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: 
2017-09-20T09:11:45+02:00 crit updater[2313]: src/opkg-trans/main.c:95 (main): 
[string "transaction"]:325: No journal to recover
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: DIE:src/opkg-trans/main.c:95 (main):
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: [string "transaction"]:325: No journal to recover
2017-09-20T09:11:45+02:00 emerg sfpswitch.py[1534]: Aborted
2017-09-20T09:11:45+02:00 info cron[2320]: (CRON) STARTUP (V5.0)
2017-09-20T09:11:45+02:00 info cron[2320]: (root) BAD FILE MODE (crontabs/root)
2017-09-20T09:11:45+02:00 info smartd[2328]: smartd 6.5 2016-05-07 r4318 [armv7l-linux-4.4.13-05df79f63527051ea0071350f86faf76-7] (localbuild)
2017-09-20T09:11:45+02:00 info smartd[2328]: Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
2017-09-20T09:11:45+02:00 info smartd[2328]: Opened configuration file /etc/smartd.conf
2017-09-20T09:11:45+02:00 info smartd[2328]: Configuration file /etc/smartd.conf parsed.
2017-09-20T09:11:45+02:00 info smartd[2328]: Device: /dev/hdb, open() failed: No such device
2017-09-20T09:11:45+02:00 info smartd[2328]: Monitoring 0 ATA/SATA, 0 SCSI/SAS and 0 NVMe devices
2017-09-20T09:11:45+02:00 info smartd[2332]: smartd has fork()ed into background mode. New PID=2332.
Router Turris successfully started.
2017-09-20T09:11:46+02:00 warning []: Unable to use //tmp//yin2yang.xsl (No such file or directory).
2017-09-20T09:11:46+02:00 warning []: YANG format data models will not be available via get-schema.
2017-09-20T09:11:46+02:00 warning []: Unable to get tty (Not a tty) to get the client's hostname (session 2336).
2017-09-20T09:11:47+02:00 emerg sfpswitch.py[1534]: setting up led Auto-configuration for PCI1
2017-09-20T09:11:47+02:00 emerg sfpswitch.py[1534]: setting up led Auto-configuration for PCI2
2017-09-20T09:11:47+02:00 emerg sfpswitch.py[1534]: setting up led Auto-configuration for PCI3
2017-09-20T09:11:47+02:00 emerg sfpswitch.py[1534]: setting up led Auto-configuration for WAN
2017-09-20T09:11:47+02:00 emerg turris[]: Router Turris successfully started.
2017-09-20T09:11:47+02:00 info procd[]: - init complete -

Maybe you can get some information for your needs.