[HBL] 5.15 kernel upgrade blocks all internet traffic from clients

@mgulick are rainbow leds also broken for you with this update?

EDIT: And what about updater tab in ReForis?

Yes, rainbow LEDs are also not working for me. Looks like there was a commit two days ago in HBL though that may fix it? patches/openwrt: backport kmod-leds-turris-omnia (d91b828a) · Commits · Turris / Turris OS / Turris Build · GitLab

The updater in ReForis does appear to work for me. I can click on the “Check and Install Updates” button and after a while it comes back and says “you’re up to date”. It looks like an automatic update did install last night (a base-files update, which didn’t fix the firewall issue).

HBL is unstable branch and similar behavior is fine.

Did the kernel really block incoming traffic?

1 Like

What kind of non standard hotplug items do you have in /etc/hotplug.d/iface ?

The problem might be if onw hotplug script with a number below the firewall’s blocks indefinetly for whatever reason the firewall script will never run.

I am having this situation since an early TOS5 version, which apprently was caused by ntpclient, but really because OpenWrt’s hotplug system seems bit brittle…

See:

1 Like

@viktor, no the problem isn’t the kernel. It was just tied to the large upgrade that most notably updated the kernel. The problem is actually that the firewall isn’t loading properly on boot. See my second reply.

@moeller0 nothing looks out of the ordinary to me:

root@turris:~# ls -la /etc/hotplug.d/iface/
drwxr-xr-x    1 root     root           144 Jul 22 00:51 .
drwxr-xr-x    1 root     root            80 May  6 13:10 ..
-rw-r--r--    1 root     root           155 Nov 23  2021 00-netstate
-rw-------    1 root     root           436 Apr 28 01:28 11-sqm
-rw-------    1 root     root           336 May  4 09:30 20-firewall
-rwxr-xr-x    1 root     root          1116 Jul 18 09:31 40-resolver-reload
-rw-r--r--    1 root     root           323 May  6 13:10 95-ddns
-rwxr-xr-x    1 root     root           145 Jun 16 06:51 99-foris-controller

I don’t see any indication in the system log that any of these are hanging, however I also don’t see any indication that any of them are loading. There are several messages that say the firewall is reloading:

Jul 23 16:17:32 turris firewall: Reloading firewall due to ifup of server (br-lan.20)
Jul 23 16:17:38 turris firewall: Reloading firewall due to ifup of media (br-lan.25)
Jul 23 16:17:40 turris firewall: Reloading firewall due to ifup of iiot (br-lan.30)
Jul 23 16:17:43 turris firewall: Reloading firewall due to ifup of iot (br-lan.35)
Jul 23 16:17:46 turris firewall: Reloading firewall due to ifup of ipcam (br-lan.40)
Jul 23 16:17:49 turris firewall: Reloading firewall due to ifup of guest (br-lan.45)
Jul 23 16:17:54 turris firewall: Reloading firewall due to ifup of wan (eth2)

I can manually start/stop/reload the firewall from the command line with /etc/init.d/firewall <start|stop|reload> and it appears to work as expected. Just not on boot.

Use /etc/rc.local instead as the workaround.

1 Like

I’m having what I think is the same problem, also on HBL.
Only difference, is that restarting the firewall, not matter how many times won’t work. Weirdly enough, to have the firewall rules come back, I have to load the firewall status page(which has all the rulesthere, but all statistics zeroed) in LuCi, and then restart the firewall, and that finally populates the tables…
Quite similar to this: https://github.com/openwrt/openwrt/issues/8568

@ArthurMLago this is actually true for me as well. Just restarting or reloading the firewall didn’t restore internet access. Running /etc/init.d/firewall/restart immediately after boot resulted in the following output:

root@turris:~# /etc/init.d/firewall restart
Warning: Unable to locate ipset utility, disabling ipset support
Warning: Section @rule[10] (server DHCP) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[11] (server DNS) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[12] (media DHCP) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[13] (media DNS) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[14] (iiot DHCP) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[15] (iiot DNS) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[16] (iot DHCP) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[17] (guest DHCP) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[18] (guest DNS) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[19] (ipcam DHCP) does not specify a protocol, assuming TCP+UDP
Warning: Section @rule[22] (iot NTP) does not specify a protocol, assuming TCP+UDP
Warning: Section @redirect[1] (Gitea SSH) does not specify a protocol, assuming TCP+UDP
 * Flushing conntrack table ...
 * Set tcp_ecn to off
 * Set tcp_syncookies to on
 * Set tcp_window_scaling to on
 * Running script '/etc/firewall.user'

After running iptables -L and then /etc/init.d/firewall reload, it restored all of the chains, but I still didn’t have firewall access. I’m not sure if a firewall restart would have resolved it at that point, but I did go to luck and open the firewall page at this point, and then ran /etc/init.d/firewall restart, which did finally restore internet access. Although the circumstances of this issue are different than the issue you linked to (i.e. we are not running fw3 inside lxc AFAIK), the syndome appears identical.

The following does seem to properly start the firewall from the command line after boot:

for t in filter nat mangle raw; do iptables -t $t -L; done
/etc/init.d/firewall restart
2 Likes

Not related to kernel 5.15, but yes, the fix is coming for that.

Thanks for reporting. This should be fixed in tomorrow’s build.

2 Likes

Could you please elaborate a bit what was/is the root cause of the observed mis-behavior? I have a similar phenotype on TOS 5.3 and this information might help me fix my issue as well.

Thanks in advance!

Incompatibilities between 5.15 kernel and old version of firewall3. Mostly likely unrelated to anything on TOS 5.3.X

2 Likes

Is the update already there?

HBL and other development branches are rolling updates, builds compiled daily. However, they are still untested. Experienced users might want to use those and yes, I checked that updated firewall3 is indeed updated, which solves the missing firewall rules after boot.

What’s left is to fix DSA and VLANs.

We know what we are running on. Those with Asiarf cards have no other option than run on HBL as kernel modules are not backported to stable branch.

3 posts were split to a new topic: Twinkies stories in updating Turris 1.x on HBL

They will not be backported or should we rather delay Turris OS 6.0 as much as we can and stay on the outdated OpenWrt version for a longer time? I don’t think so. If you are willing to help, you might want to send us a pull request.

Also, your posts are off-topic in this thread. You should create your own thread for such issues.

Nope, I see that libnss is compiled in the HBL branch in today’s build.

2 Likes

I risked and I did pkgupdate and it seems that firewall issue is fixed. I think the topic can be marked as solved but I am no OP.

The issue is also fixed for me after pkgupdate. I will mark the issue as solved.