Turris OS 5.0.0 is going to be in HBK

5 posts were split to a new topic: Turris 1.x and LXC on Turris OS 4.0+

Package ddns-scripts comes from OpenWrt packages feed and we are not maintainers of it. I suggest to open an issue in this repository: https://github.com/openwrt/packages/issues

Anyone running a software RAID with mdadm on Turris? On my HBK branch and a RAID1 mdadm segfaults:

root@turris:~# mdadm --assemble --scan
Segmentation fault

Would you please share output from opkg list-installed | grep mdadm?

Ok, installed HBK completely new by using: https://repo.turris.cz/hbk/medkit/omnia-medkit-latest.tar.gz

Issue 1: luci-proto-relay can not be installed:

root@turris:~#   opkg install luci-proto-relay
Installing luci-proto-relay (git-20.016.50399-e1df28d-1.0) to root...
Downloading https://repo.turris.cz/hbs/omnia/packages/luci/luci-proto-relay_git-20.016.50399-e1df28d-1_all.ipk
Collected errors:
 * check_data_file_clashes: Package luci-proto-relay wants to install file /usr/lib/lua/luci/model/network/proto_relay.lua
	But that file is already provided by package  * luci-compat
 * opkg_install_cmd: Cannot install package luci-proto-relay.

Issue 2: relayed produces an error during installation:

root@turris:~#   opkg install relayd
Installing relayd (2016-02-07-ad0b25ad-2.36) to root...
Downloading https://repo.turris.cz/hbs/omnia/packages/base/relayd_2016-02-07-ad0b25ad-2_arm_cortex-a9_vfpv3.ipk
Configuring relayd.
Command failed: Not found

Issue 3: 4 RAID devices of RAID controller Marvell 88SE9215 not detected
I have in sum 4 HDDs connected to the RAID controller, 1 SSD and 1 USB drive are connected to the USB3 interfaces, but I can only see the USB connected once:

root@turris:~# ls -lh /dev/sd*
brw-------    1 root     root        8,   0 Mar 25 00:25 /dev/sda   # this one is the USB drive to install HBK)
brw-------    1 root     root        8,   1 Mar 25 00:25 /dev/sda1
brw-------    1 root     root        8,  16 Mar 25 00:25 /dev/sdb   # this one is a USB drive that contains the /srv/lxc filesystem
brw-------    1 root     root        8,  17 Mar 25 00:25 /dev/sdb1

Missing: /dev/sdc, /dev/sdd, /dev/sde, /dev/sdf

root@turris:~# lspci -v
00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 6820 (rev 04) (prog-if 00 [Normal decode])
	Device tree node: /sys/firmware/devicetree/base/soc/pcie/pcie@1,0
	Flags: fast devsel
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00010000-00010fff [size=4K]
	Memory behind bridge: e0000000-e00fffff [size=1M]
	Prefetchable memory behind bridge: 00000000-000fffff [size=1M]
	Capabilities: [40] Express Root Port (Slot+), MSI 00
lspci: Unable to load libkmod resources: error -12

00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 6820 (rev 04) (prog-if 00 [Normal decode])
	Device tree node: /sys/firmware/devicetree/base/soc/pcie/pcie@2,0
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: e0200000-e04fffff [size=3M]
	Prefetchable memory behind bridge: 00000000-000fffff [size=1M]
	Capabilities: [40] Express Root Port (Slot+), MSI 00

00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 6820 (rev 04) (prog-if 00 [Normal decode])
	Device tree node: /sys/firmware/devicetree/base/soc/pcie/pcie@3,0
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: None
	Memory behind bridge: e0600000-e08fffff [size=3M]
	Prefetchable memory behind bridge: 00000000-000fffff [size=1M]
	Capabilities: [40] Express Root Port (Slot+), MSI 00

01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9215 (rev 11) (prog-if 01 [AHCI 1.0])
	Subsystem: Marvell Technology Group Ltd. Device 9215
	Flags: bus master, fast devsel, latency 0
	I/O ports at 10020 [size=8]
	I/O ports at 10030 [size=4]
	I/O ports at 10028 [size=8]
	I/O ports at 10034 [size=4]
	I/O ports at 10000 [size=32]
	Memory at e0010000 (32-bit, non-prefetchable) [size=2K]
	Expansion ROM at e0000000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
	Capabilities: [70] Express Legacy Endpoint, MSI 00
	Capabilities: [e0] SATA HBA v0.0
	Capabilities: [100] Advanced Error Reporting

02:00.0 Network controller: Qualcomm Atheros QCA986x/988x 802.11ac Wireless Network Adapter
	Flags: bus master, fast devsel, latency 0, IRQ 78
	Memory at e0200000 (64-bit, non-prefetchable) [size=2M]
	Expansion ROM at e0400000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 2
	Capabilities: [50] MSI: Enable+ Count=1/8 Maskable+ 64bit-
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [160] Device Serial Number 00-00-00-00-00-00-00-00
	Kernel driver in use: ath10k_pci

03:00.0 Network controller: Qualcomm Atheros QCA986x/988x 802.11ac Wireless Network Adapter
	Subsystem: Device 19b6:d03c
	Flags: bus master, fast devsel, latency 0, IRQ 80
	Memory at e0600000 (64-bit, non-prefetchable) [size=2M]
	Expansion ROM at e0800000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 2
	Capabilities: [50] MSI: Enable+ Count=1/8 Maskable+ 64bit-
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Virtual Channel
	Capabilities: [160] Device Serial Number 00-00-00-00-00-00-00-00
	Kernel driver in use: ath10k_pci

Controller is detected, but somehow not the devides.
See dmesg: https://pastebin.com/xZUgKZz6
Using the old Turris Omnia OS from: https://repo.turris.cz/omnia/medkit/omnia-medkit-latest.tar.gz works like a charme here. Detects the RAID controller, plus the devices.

Issue 4: Installing mdadm directly with opkg always wants to downgrade?!

root@turris:~# opkg install mdadm
Installing mdadm (4.0-4.23) to root...
Downloading https://repo.turris.cz/hbs/omnia/packages/base/mdadm_4.0-4_arm_cortex-a9_vfpv3.ipk
Not downgrading package kernel on root from 4.14.172-1-ea5a483151eea3aecd64000c249a47e5.0 to 4.14.162-1-0a66bb0316b4402bf65555c64ceed313.0.
Collected errors:
 * opkg_install_cmd: Cannot install package mdadm.

Using FORIS (Updater -> NAS) option works fine, but today the below happens.
Packages: https://pastebin.com/wxqQE3X2

Issue 5: Issuing today pkgupdate kills LUCI (Network view/Wireless view) and FORIS
LUCI:

Failed to execute view dispatcher target for entry '/admin/network/network'.
The called action terminated with an exception:
/usr/lib/lua/luci/template.lua:74: Failed to load template 'view'.
Error while parsing template '/usr/lib/lua/luci/view/view.htm':
No such file or directory
stack traceback:
	[C]: in function 'error'
	/usr/lib/lua/luci/template.lua:74: in function '__init__'
	/usr/lib/lua/luci/util.lua:65: in function 'Template'
	/usr/lib/lua/luci/template.lua:27: in function 'render'
	/usr/lib/lua/luci/dispatcher.lua:929: in function </usr/lib/lua/luci/dispatcher.lua:928>

and

Failed to execute view dispatcher target for entry '/admin/network/wireless'.
The called action terminated with an exception:
/usr/lib/lua/luci/template.lua:74: Failed to load template 'view'.
Error while parsing template '/usr/lib/lua/luci/view/view.htm':
No such file or directory
stack traceback:
	[C]: in function 'error'
	/usr/lib/lua/luci/template.lua:74: in function '__init__'
	/usr/lib/lua/luci/util.lua:65: in function 'Template'
	/usr/lib/lua/luci/template.lua:27: in function 'render'
	/usr/lib/lua/luci/dispatcher.lua:929: in function </usr/lib/lua/luci/dispatcher.lua:928>

Foris http://192.168.1.1/foris/config/

# 500 Internal Server Error

Issue 6: only HBL related: mdadm segfaults
I noticed that the segfault bug with mdadm (see my last post) only occurs on HBL, somehow I did not cleanly install HBK the last time.

You are still using HBS! After using a medkit from HBK, you need to use switch-branch hbk before doing changes in Updater tab. As medkit is default configured to point to HBS.

More details are here:

ufff! ok! Will try again :slight_smile:

Oh yes this is quite annoying feature of HBK medkit. I installed it yesterday made rollback to factory and even it show turris_version 5.0.0 updater started to downgrade to 4.0.3 as I wanted not that to happen I logon on console shortly after changing password with forris first time and did switch-branch hbk which rendereder whole thing unusable as opkg failed after I done this because incorrect certificates. So I had it to rollback and do again and go through only viable path

  1. factory rollback to HBK medkit of 5.0.0
  2. initial setup in forris then let the updater do downgrade to 4.0.3 and wait till this finish and router reboot
  3. logon on console and switch-branch to hbk and do pkgupdate to upgrade again to 5.0.0
  4. do schnapps backup of fresh HBK 5.0.0
  5. run opkg scripts to install kmods,libs,apps in this order
  6. copy my configs to /etc/config

Oh yes, I would also appreciate if @Pepe or the crew has the ability to fix this issue.
One downloads the medkit with HBK and expects that everything is using now the HBK version. Because if I see it correctly, it is just one UCI flag that needs to be set correctly:

root@turris:~# uci show | grep -i hbk
updater.turris.branch='hbk'

Update for @Pepe:
I installed the https://repo.turris.cz/hbk/medkit/omnia-medkit-latest.tar.gz. Using just this version does not show the HDD devices that are connected to the Marvell 88SE9215 SATA controller.
Still, I moved on with:

  switch-branch hbk

To this I executed the commands required related to the packages I am interested in:

  uci add_list updater.turris.pkglists='hardening'
  uci add_list updater.turris.pkglists='luci-controls'
  uci add_list updater.turris.pkglists='lxc'
  uci add_list updater.turris.pkglists='nas'
  uci commit updater
  pkgupdate --batch

One reboot afterwards:

  • Marvell 88SE9215 SATA devices correctly reported under /dev/sd[c,d,e,f]
  • Even Mikrotik R11e-5HacT wifi is now functioning, although it is quite a troublesome piece of hardware
  • mdadm --assemble --scan succeeded without segfault
  • Installing luci-proto-relay succeeded without issue
  • No LUCI/FORIS kill :slight_smile:

Sole little issue left:

root@turris:~#   opkg install relayd
Installing relayd (2016-02-07-ad0b25ad-2.59) to root...
Downloading https://repo.turris.cz/hbk/omnia/packages/base/relayd_2016-02-07-ad0b25ad-2_arm_cortex-a9_vfpv3.ipk
Configuring relayd.
Command failed: Not found

It seems as if there is something missing. Still, the installation is kind of successful so that relayd can be used afterwards.

When we are already in the discussion of ā€œwish listā€ (as above):
I do like the ability to specify NAS, LXC and HARDENING as overall packet bundles. But would it be possible to make it more fine-grained for experienced users? E.g. NAS comes with the asm1062-fix, but I have a Marvell SATA controller, expecially because the ASM1062 controller is - very politely - not good on ARM systems. (It destroyed in the past multiple mdadm RAIDs on different ARM hardware.) Hence, I do not need to have the package installed. And I do not want to get samba installed either.

This can not be fixed. There are builds for HBD, HBL, and HBK. Builds to HBT and later to HBS is a copy of HBK. It is necessary to have medkits prepared for a stable branch as a release branch. If we will set HBK, then you will download HBS and you will end up in HBK and thatā€™s not what most people want. We can improve the documentation about it.

This is most noticeable just now as there are different versions of OpenWrt.

Thatā€™s known since Turris OS 4.0 and expected. Some packages which were preinstalled in Turris OS 3.x needs to be installed separately in Turris OS 4.0+ versions, but they are still part of user lists, which you can choose in Foris. The way how to get it is working is to install NAS list and you will get it working as if you have installed SATA controller, it is very likely that you will use it for NAS. We can have a separate discussion about why someone should have installed packages for NAS if he does not have NAS and related things to it. This, on the other hand, free up some storage space and in LuCI provide simple ways for advanced users instead of providing many options and they will get lost.

Yes, we received feedback about it as well that package lists might be huge and full-featured, but someone would like to have just pieces of it. If I am not mistaken, it should be improved in the future and it will be included in Turris OS 5.1/5.2.

The installation of the package was successful. If it is not installed it will come with a different error. Most of those Command failed: Not found are harmless, but most packages come as it is from their feeds and you can report it to the maintainers.

1 Like

Since yesterday the pakon stopped working, last entry at 2020-03-30 10:02:52.
I am running 5.0.0 on Omnia

$ opkg list-installed | grep pakon
foris-controller-pakon-module - 0.2.1-3.7-3.14
foris-pakon-plugin - 2.6.1-3.7-2.14
foris-pakon-plugin-l10n-cs - 2.6.1-3.7-2.0
pakon - 1.2.1-1.57
pakon-lists - 6-1.43
suricata-pakon - 1-8.59
$ pgrep -f pakon-monitor.py
8279

$ ps aux | grep pakon
root      8279  0.7  0.5  13372 10596 ?        S    23:05   0:00 python3 /usr/libexec/pakon-light/pakon-monitor.py
root      8550  0.0  0.0   1192   548 pts/0    S+   23:06   0:00 grep pakon
root      9123  0.0  1.2  30744 26372 ?        S    12:40   0:23 python3 /usr/libexec/pakon-light/pakon-handler.py

Database and its backup seem to be both ok:

$ ls -l /var/lib/pakon.db
-rw-r--r--    1 root     root      16146432 Mar 31 23:05 /var/lib/pakon.db
$ ls -l /srv/pakon/pakon.db.xz
-rw-r--r--    1 root     root       2902160 Mar 31 22:57 /srv/pakon/pakon.db.xz
$ /usr/bin/sqlite3 /var/lib/pakon.db "pragma integrity_check"
ok

Just out of curiosity, is suricata required to be functional for pakon? If I try to start suricata manually it gives this error:

$ PID_FILE="/var/run/suricata/suricata.pid"
$ QUEUE_NUM=10
$ /usr/bin/suricata -c /etc/suricata-pakon/suricata.yaml -q $QUEUE_NUM --pidfile $PID_FILE
Error loading shared library libpcap.so.1: No such file or directory (needed by /usr/bin/suricata)
Error relocating /usr/bin/suricata: pcap_compile_nopcap: symbol not found
Error relocating /usr/bin/suricata: pcap_activate: symbol not found
Error relocating /usr/bin/suricata: pcap_setfilter: symbol not found
Error relocating /usr/bin/suricata: pcap_freealldevs: symbol not found
Error relocating /usr/bin/suricata: pcap_set_timeout: symbol not found
Error relocating /usr/bin/suricata: pcap_findalldevs: symbol not found
Error relocating /usr/bin/suricata: pcap_create: symbol not found
Error relocating /usr/bin/suricata: pcap_close: symbol not found
Error relocating /usr/bin/suricata: pcap_stats: symbol not found
Error relocating /usr/bin/suricata: pcap_open_dead: symbol not found
Error relocating /usr/bin/suricata: pcap_dump: symbol not found
Error relocating /usr/bin/suricata: pcap_set_snaplen: symbol not found
Error relocating /usr/bin/suricata: pcap_breakloop: symbol not found
Error relocating /usr/bin/suricata: pcap_geterr: symbol not found
Error relocating /usr/bin/suricata: pcap_set_promisc: symbol not found
Error relocating /usr/bin/suricata: pcap_open_offline: symbol not found
Error relocating /usr/bin/suricata: pcap_compile: symbol not found
Error relocating /usr/bin/suricata: pcap_freecode: symbol not found
Error relocating /usr/bin/suricata: pcap_dump_close: symbol not found
Error relocating /usr/bin/suricata: pcap_datalink: symbol not found
Error relocating /usr/bin/suricata: pcap_dump_open: symbol not found
Error relocating /usr/bin/suricata: pcap_dispatch: symbol not found
Error relocating /usr/bin/suricata: pcap_set_buffer_size: symbol not found

I have these suricata packages installed:

$ pkg list-installed | grep suricata
suricata-bin - 4.0.7-3.15
suricata-conntrack-flows - 1-0.0
suricata-pakon - 1-8.59

And following libpcap

$ opkg list-installed | grep libpcap
libpcap - 1.9.1-2.0

$ opkg files libpcap
Package libpcap (1.9.1-2.0) is installed on root and has the following files:
/usr/lib/libpcap.so.
/usr/lib/libpcap.so.0.8

There is a preinstalled package ip-full by default in every configuration.

I havenā€™t created any symlink nor removed the ip utility and I see, there is a correct symlink even I configure the router from scratch. I am not able to reproduce your issue neither on both routers, which I have nearby. See following output:

root@turris:~# ls -al /sbin/ip
lrwxrwxrwx    1 root     root            20 Mar 21 13:17 /sbin/ip -> /usr/libexec/ip-full

The link should be there:

I would like to know, if you were doing any changes related to ip and also when did you installed HBK on your router.

Pakon utilizes Suricata and if Suricata does not work, then Pakon is not able to show new data. I looked at it and yesterday, there were some changes in libpcap in OpenWrt 19.07. This could be why it does not work in your case.

Suricata is looking in your case for libpcap.so.1, but you have installed libpcap.so.0.8 as I do. I didnā€™t have installed Pakon, but I installed it now and it works. Can you try to force-reinstall Suricata or libpcap if it helped?

I reinstalled both (libpcap and suricata-bin) and that fixed the problem.
Thanks for help @Pepe.

1 Like

You posted a comment with output, but you havenā€™t posted any details about it. Can you please try to give us some leads so we will be able to reproduce it? Does that happened just once to you or are you able to reproduce it continuously?

When I am looking at the output, which you shared, I am thinking if you were doing any changes in the system configuration or downloading many files like thousands/millions at once.

If it happens once again, please send us diagnostics and we will look at it.

Dear @Pepe,

any insights why the sub-tabs for netbooted MOX do not work? I sent the diagnostics to techsuport via email after your request. Thanks for feedback.

Yes I did a mistake and wanted to reinstall all the opkg packages at once a I had been trying to reinstalll everrything as fast as possible so it probably caused this issue. Unfortunally recently I had been trying different branches HBK, HBL, HBS with that downgrade / upgrade procedure that is built in into medkit before I realized and learned how to proceed with upgrade and that after every step have to give TOS time to downgrade and then upgrade only after that I could start to reinstall packages safely. So as it happened some time ago I can not provide more information now.

1 Like

Hi, Iā€™m pretty sure I didnā€™t mess with the ip commands myself. I donā€™t locally change binaries apart from installing/reinstalling packages, since the setup will be fragile and break on any upgrade.

Everything was well on hbs and this happened on the upgrade to hbk. I did it on 2020-03-18, according to the snapshots on the router.

After the upgrade I starting testing my router to see if everything still worked. I noticed that mwan3 wasnā€™t. Then I saw the errors in the log, then I checked what ip version I had, and then it was a symlink to busybox. Since I didnā€™t put it there, it must have been the upgrade.

I do have ip-full installed, that is not the problem. As I mentioned before, the problem is that it only installs a binary to libexec and does not ship with any symlink:

root@turris:~# opkg files ip-full
Package ip-full (5.0.0-2.1.0) is installed on root and has the following files:
/usr/libexec/ip-full

If you check what package provided the /sbin/ip binary:

root@turris:~# opkg search /sbin/ip
busybox - 1.30.1-5.16

The symlink is being provided by the busybox package.

No amount of local modification in my end can change the output of the above commands.

I have updated turris-netboot-tools to version 0.6.1-1.0 today.
I assume after installation it ran netboot-manager regen -f, it went trough however with these warning messages. The tabs in Foris still do not work (wifi status is not shown, only exclamation mark next to device name). reForis shows instead orange question mark for the same device.

mox.its:8.18-23.11: Warning (unit_address_vs_reg): /images/kernel@0: node has a unit name, but no reg property
mox.its:17.20-19.15: Warning (unit_address_vs_reg): /images/kernel@0/hash@1: node has a unit name, but no reg property
mox.its:20.20-22.15: Warning (unit_address_vs_reg): /images/kernel@0/hash@2: node has a unit name, but no reg property
mox.its:24.19-37.11: Warning (unit_address_vs_reg): /images/ramdisk@0: node has a unit name, but no reg property
mox.its:31.20-33.15: Warning (unit_address_vs_reg): /images/ramdisk@0/hash@1: node has a unit name, but no reg property
mox.its:34.20-36.15: Warning (unit_address_vs_reg): /images/ramdisk@0/hash@2: node has a unit name, but no reg property
mox.its:38.15-50.11: Warning (unit_address_vs_reg): /images/fdt@0: node has a unit name, but no reg property
mox.its:44.20-46.15: Warning (unit_address_vs_reg): /images/fdt@0/hash@1: node has a unit name, but no reg property
mox.its:47.20-49.15: Warning (unit_address_vs_reg): /images/fdt@0/hash@2: node has a unit name, but no reg property
mox.its:55.16-60.11: Warning (unit_address_vs_reg): /configurations/conf@1: node has a unit name, but no reg property

Developers are looking into the issue, which you sent to us. Thereā€™s an issue, where you can track any changes related to it.

It is labeled with High priority. It means that it blocks the next release and if my assumptions are correct that this prevent us to release it to HBT.