Updater error in HBL (and obviously in HBD too)

Hi,

I am on HBL and at least, last 3 days, I see the same error in Foris/reForis:

Latest one:

    Error from 2020/04/23 07:28:16

    Updater failed:

    inconsistent: Requested package fix-pkglists-options that is not available.

I have tried to switch-branch to HBD and all ended like this:

root@turris:~# switch-branch hbd
+ uci set 'updater.turris=turris'
+ '[' n '=' y ]
+ uci set 'updater.turris.mode=branch'
+ uci set 'updater.turris.branch=hbd'
+ uci commit updater.turris
+ '[' n '=' y ]
+ sed -i 's#https://repo\.turris\.cz/.*/\([^/]*\)/packages/#https://repo\.turris\.cz/hbd/\1/packages/#' /etc/opkg/distfeeds.conf
+ '[' n '=' y ]
+ updater_args=
+ config_get_bool updater_approvals autorun approvals 0
+ local _tmp
+ config_get _tmp autorun approvals 0
+ eval export -n -- '_tmp=${CONFIG_autorun_approvals:-${4}}'
+ export -n -- '_tmp=0'
+ _tmp=0
+ export -n 'updater_approvals=0'
+ '[' 0 '=' 1 ]
+ updater_args=--batch
+ pkgupdate --reinstall-all --batch
INFO:Target Turris OS: 6.0-future
line not found
line not found
line not found
line not found
line not found
ERROR:
inconsistent: Requested package fix-pkglists-options that is not available.
+ echo 'Updater execution exited with error. Please see previous output to know what went wrong.'
Updater execution exited with error. Please see previous output to know what went wrong.
+ opkg update
Downloading https://repo.turris.cz/hbd/omnia/packages/core/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_core
Downloading https://repo.turris.cz/hbd/omnia/packages/core/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/base/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_base
Downloading https://repo.turris.cz/hbd/omnia/packages/base/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/cesnet/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_cesnet
Downloading https://repo.turris.cz/hbd/omnia/packages/cesnet/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_luci
Downloading https://repo.turris.cz/hbd/omnia/packages/luci/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/node/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_node
Downloading https://repo.turris.cz/hbd/omnia/packages/node/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/openwisp/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_openwisp
Downloading https://repo.turris.cz/hbd/omnia/packages/openwisp/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_packages
Downloading https://repo.turris.cz/hbd/omnia/packages/packages/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_routing
Downloading https://repo.turris.cz/hbd/omnia/packages/routing/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/sidn/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_sidn
Downloading https://repo.turris.cz/hbd/omnia/packages/sidn/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_telephony
Downloading https://repo.turris.cz/hbd/omnia/packages/telephony/Packages.sig
Signature check passed.
Downloading https://repo.turris.cz/hbd/omnia/packages/turrispackages/Packages.gz
Updated list of available packages in /var/opkg-lists/turrisos_turrispackages
Downloading https://repo.turris.cz/hbd/omnia/packages/turrispackages/Packages.sig
Signature check passed.
+ set +x

                \||/
                |  @___oo
      /\  /\   / (__,,,,|
     ) /^\) ^\/ _)
     )   /^\/   _)
     )   _ /  / _)
 /\  )/\/ ||  | )_)
<  >      |(,,) )__)
 ||      /    \)___)\
 | \____(      )___) )___
  \______(_______;;; __;;;
You are now in branch containing software build every night. It often contains bugs and very often requires manual intervention!
It is based on the latest OpenWrt development branch with the latest Turris OS changes.
Turris team provides no guarantees and no support for this branch. You can get some help on forum (https://forum.turris.cz/).
If you encounter some bugs than please debug cause and report it to developers trough gitlab (https://gitlab.labs.nic.cz/),
You shouldn't be in this branch unless you are advanced user and OpenWrt developer!
To return to stable branch run this command: switch-branch hbs

So I went back to HBL, where I would like to stay for now (AdBlock tested only on 5.1.0).

This problem is at my side or is there some maintenance/preparation by Turris team (preparation for TOS5 release into HBS?), which is causing this?

It was mentioned here as well What happened to the HBD|L build bots?.

The package is not available as there was no new builds.

As @pepe wrote:

The rules are applied to everyone.

Ok, so I am not alone with this and I have to wait.

Thanks.

So, some update has arrived (noticed after wifi connection lost). After reboot, only accessible interfaces are reForis and SSH. Foris and LuCI inaccessible.

In reForis, there is message about new update:

but after clicking on Install now, error occures:

Package management pages give following errors:

Update settings - An unknown API error occurred.
Updates - An error occurred while fetching data.
Packages - An unknown API error occurred.

Diagnostics report sent to Turris support.

Omnia (HBL)

Cannot fetch updater status. (reForis)

About tab

An error occurred while fetching data.

And after few minutes finaly updated.

Updater failed: 

runtime: [string "file:////etc/updater/conf.d/turris-pkglists..."]:17: bad argument #1 to 'pairs' (table expected, got nil)

Package management (after reboot)

An unknown API error occurred.

@viktor Same thing happening for me with pkgupdate failing. It seems to happen because it thinks I have no additional package lists enabled.

EDIT:
Done some more research, and it seems like the pkglists formatting has changed slightly. @Pepe I’m assuming this is what the fix-pkglists-options package was supposed to fix? But it looks like what it’s done is break the config file. For reference, this is what my /etc/config/pkglists looks like:

config pkglists 'pkglists'

config pkglist 'nas'
    option samba '1'
    option dlna '1'
    option transmission '1'
    option raid '1'
    option encrypt '1'

If I change it to:

config pkglists 'pkglists'
        list pkglist 'nas'

config 'nas'
        option samba '1'
        option dlna '1'
        option transmission '1'
        option raid '1'
        option encrypt '1'

It works again.

root@turris5:~# uci set pkglists.pkglists.pkglist=“pakon”
root@turris5:~# uci commit
root@turris5:~# pkgupdate
INFO:Target Turris OS: 5.1.0
WARN:Package wpad is in cyclic dependency. It might fail its post-install script.
WARN:Package hostapd is in cyclic dependency. It might fail its post-install script.
INFO:Queue install of pkglists/turrispackages/1.4.1-1.0
INFO:Queue install of luci-lib-jsonc/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-lib-ip/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-lib-nixio/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-base/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-compat/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-app-commands/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-commands-en/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-app-firewall/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-firewall-en/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-app-opkg/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-opkg-en/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-commands-cs/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-firewall-cs/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-opkg-cs/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-proto-ipv6/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-proto-ppp/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-theme-bootstrap/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-mod-network/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-mod-system/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-mod-status/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-mod-admin-full/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-base-en/luci/git-20.115.52331-39a8290-1.0
INFO:Queue install of luci-i18n-base-cs/luci/git-20.115.52331-39a8290-1.0
INFO:Queue removal of reforis-snapshots-plugin
INFO:Queue removal of fix-pkglists-options
INFO:Queue removal of reforis-diagnostics-plugin
INFO:Queue removal of foris-controller-schnapps-module
INFO:Queue removal of reforis-snapshots-plugin-l10n-cs
INFO:Queue removal of reforis-diagnostics-plugin-l10n-cs
INFO:Queue removal of reforis-l10n-cs
INFO:Queue removal of reforis
INFO:Queue removal of python3-flask-babel
INFO:Queue removal of python3-babel
INFO:Queue removal of python3-pytz
INFO:Queue removal of python3-flask-session
INFO:Queue removal of python3-simplejson
INFO:Queue removal of python3-flask-seasurf
INFO:Queue removal of python3-flask
INFO:Queue removal of python3-click
INFO:Queue removal of python3-itsdangerous
INFO:Queue removal of python3-werkzeug
Press return to continue, CTRL+C to abort

I’ve changed my comment above slightly. Basically, the formatting of the pkglists config file has changed and that seems to have removed critical things from it which causes it to not pick up additional package lists correctly. The workaround I posted was mostly just to get updates working again, but it won’t pull in any additional package lists you had enabled. You can use my config file as reference to fix yours. Or you can post it here and I can fix it for you.

I had to rollback to snapshot created 24.4.2020 12:54 (pre-update), because of almost everything was broken,…
Will see if there will be some update and if it will break things again :space_invader:

Guys, you are using branch Here Be Lions (HBL). It contains the latest Turris development changes and there can be some bugs like this one from time to time. These details can be found in switch-branch and it requires to be able to login to your SSH and use that utility. More details about this branch can be found in this our documentation.

As I said to @michalko58 within 15 minutes which we received his report on support, we are working on improvements for package lists. The current behavior in Turris OS 3.x-5.0 is that you can choose in Updater tab package lists and they will install everything that it is included. We want to simplify it to our users and provide them with options to select parts from the package lists. Someone wants to install Honeypot as a Service on his router, but he does not want to use minipots and so on.

We know that Updater tabs in Foris and reForis are broken. It does not count with recently introduced changes, yet. For that, there is an issue in our Gitlab:

It has two labels. The first one is High. It means that it is required before releasing Turris OS 5.1, which won’t happen soon as we are working currently on Turris OS 5.0 and the second one is Doing.

Automatic updates in this branch are still working with CLI, DNS is working too. This just affects (re)Foris.

About this issue:

runtime: [string "file:////etc/updater/conf.d/turris-pkglists..."]:17: bad argument #1 to 'pairs' (table expected, got nil)

It should mean that you didn’t have enabled any package list and it was fixed yesterday.
More details in this issue:

Sorry, we can not fixed already migrated configuration. But your changes looks correct.
More details in this issue:

Thanks for the details! I’m aware HBL can be unstable and can contains bugs, and I’m perfectly fine with that. With my comments above I just wanted to outline my findings in case someone wanted to try and fix their config file manually. Personally I would say that if you’re not able to fix and/or diagnose what’s going on you should probably not be on these branches.

1 Like

Make switch-branch hbl again and wait for next fixes. Updater tab is still broken but pkgupdate works.

Well, after latest update, Foris and LuCi are accessible, but except AdBlock and MJPG streamer, all services have dissapeared.
I will try to switch to HBK, because I need my NAS to be working.

Edit:

On HBK, Foris and LuCI inaccessible, but at least NAS is working. reForis seems to be working OK.

@michalko58 that’s because as of now, update uninstalls all of those packages for some reason:

and because https://gitlab.labs.nic.cz/turris/turris-os-packages/-/merge_requests/320 hasn’t been merged yet, pkglists configuration does not migrate, therefore subsequent update will also uninstall all packages that were instaled throught foris from those package lists, including NAS package list

And does it mean, that if I switch back to HBL, that I can install NAS (at least) again from Foris and it will be working again?
I am at homeoffice so I can not afford to try it for long time. :slight_smile:

After last bigger update, Foris and reForis updater tabs still not working.

Foris

## Remote Exception: Internal error 'hidden'('<class 'KeyError'>')

### Remote request

{"module": "updater", "action": "get_settings", "kind": "request", "data": {"lang": "en"}}

### Stack trace

Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/foris_controller/message_router.py", line 117, in process_message File "/usr/lib/python3.7/site-packages/foris_controller/module_base.py", line 61, in perform_action File "/usr/lib/python3.7/site-packages/foris_controller_modules/updater/__init__.py", line 38, in action_get_settings File "/usr/lib/python3.7/site-packages/foris_controller/utils.py", line 113, in inner File "/usr/lib/python3.7/site-packages/foris_controller_modules/updater/handlers/openwrt.py", line 94, in get_user_lists File "/usr/lib/python3.7/site-packages/foris_controller_backends/updater/__init__.py", line 144, in get_user_lists File "/usr/lib/python3.7/site-packages/foris_controller_backends/updater/__init__.py", line 144, in <listcomp> KeyError: 'hidden'

reForis

An unknown API error occurred.

An error occurred while fetching data.

Current update pass with error. Foris and reForis updater tabs not working (device restarted).

> ##### Error notifications ##### Updater failed: runtime: [string ["file:////etc/updater/conf.d/switch-branch.l..."](file:////etc/updater/conf.d/switch-branch.l...)]:34: attempt to perform arithmetic on local 'stable' (a table value)

1 Like

Debug (pkgupdate -e DBG) producing:

DEBUG:src/lib/events.c:586 (run_util_init):Dumping busybox to: /tmp/updater-busybox-dkGEBD/busybox
DEBUG:src/lib/locks.c:45 (lua_acquire):Trying to get a lock at /var/lock/opkg.lock
DEBUG:backend.lua:361 (status_parse):Parsing status file /usr/lib/opkg/status
DEBUG:requests.lua:419 (Globals):Running script file:////etc/updater/conf.d/turris-pkglists.lua
DEBUG:requests.lua:419 (Globals):Running script file:////etc/updater/conf.d/switch-branch.lua
line not found
line not found
line not found
ERROR:src/pkgupdate/main.c:154 (main):
runtime: [string “file:////etc/updater/conf.d/switch-branch.l…”]:34: attempt to perform arithmetic on local ‘stable’ (a table value)
DEBUG:src/lib/locks.c:82 (lua_lock_release):Released lock at /var/lock/opkg.lock
DEBUG:src/lib/events.c:604 (run_util_clean):Removing temporally busybox from: /tmp/updater-busybox-dkGEBD/busybox

Looks like updater-ng 66.0-1.1 is messed up and since it prevents further updates only recovery is via rollback and wait for a fix.

Edit /etc/updater/conf.d/switch-branch.lua works.

2 Likes