Updater & opkg update fails (4.0.1) due to SSL certificate

[Mox]

I have a Turris Omnia model PTROM01-2G with “TurrisOS 4.0.1 80076f9 / LuCI branch (git-19.281.84184-0b4eebd)”.

Didn’t flash anything, just used the firmware as it came out of the box.

I configured it without issue (DHCP static leases, lots of Firewall rules, static routes for VPN, DynDNS, etc) mirroring my last OpenWRT router config. I left upgrading and installing additional packages as the last part, now having WAN connection, believing that Turris update just works out of the box (the reason I bought an Omnia, besides the powerful hardware). Alas, there are SSL certificate issues with both upgrade and opkg update.

Updater (both interface and “pkgupdate”) gives:

Updater failed:

runtime: [string "requests"]:395: [string "utils"]:427: URI download failed: SSL certificate problem: certificate has expired

“opkg update” gives this:

Downloading https://repo.turris.cz/hbs/omnia/packages/core/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/core/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/base/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/base/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/cesnet/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/cesnet/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/luci/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/luci/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/luci_theme_rosy/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/luci_theme_rosy/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/openwisp/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/openwisp/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/packages/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/packages/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/routing/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/routing/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/sidn/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/sidn/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/telephony/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/telephony/Packages.gz

Downloading https://repo.turris.cz/hbs/omnia/packages/turrispackages/Packages.gz
*** Failed to download the package list from https://repo.turris.cz/hbs/omnia/packages/turrispackages/Packages.gz

Collected errors:
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/core/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/base/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/cesnet/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/luci/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/luci_theme_rosy/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/openwisp/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/packages/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/routing/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/sidn/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/telephony/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://repo.turris.cz/hbs/omnia/packages/turrispackages/Packages.gz, wget returned 5.

wget code 5 means “SSL verification failure”.

Distribution feeds list is this:

src/gz turrisos_core https://repo.turris.cz/hbs/omnia/packages/core
src/gz turrisos_base https://repo.turris.cz/hbs/omnia/packages/base
src/gz turrisos_cesnet https://repo.turris.cz/hbs/omnia/packages/cesnet
src/gz turrisos_luci https://repo.turris.cz/hbs/omnia/packages/luci
src/gz turrisos_luci_theme_rosy https://repo.turris.cz/hbs/omnia/packages/luci_theme_rosy
src/gz turrisos_openwisp https://repo.turris.cz/hbs/omnia/packages/openwisp
src/gz turrisos_packages https://repo.turris.cz/hbs/omnia/packages/packages
src/gz turrisos_routing https://repo.turris.cz/hbs/omnia/packages/routing
src/gz turrisos_sidn https://repo.turris.cz/hbs/omnia/packages/sidn
src/gz turrisos_telephony https://repo.turris.cz/hbs/omnia/packages/telephony
src/gz turrisos_turrispackages https://repo.turris.cz/hbs/omnia/packages/turrispackages

First odd thing is that “luci_theme_rosy” and “openwisp” folders don’t exist in HBS. (side note: Rosy doesn’t show up in the interface either, only option is Bootstrap).

So, HBS doesn’t seem to match my OS version, doesn’t seem to match 4x either. Is it 5.x? I found latest 4.x in archives and it does contain luci_theme_rosy and openwisp here: Index of /archive/4.0.6/omnia/packages/. Maybe this problem is because the router shipped with HBS at the time, but it was months ago when I aquired it but hadn’t have the time to configure it and replace the old router, and now it’a a bit late to upgrade.

Anyway, that’s not the main problem. It’s the SSL certificate issue.

What I’ve tried/checked so far based on forum posts:

1. Time/date is synced via browser, so no issue of time discrepancies. Tried to set the time a little in the past, both through system’s “date” and “hwclock -w” to sync hardware clock to system.

2. openssl s_client -servername repo.turris.cz -connect repo.turris.cz:443 | openssl x509 -noout -dates
   outputs:

depth=3 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
notBefore=Oct 1 21:17:09 2021 GMT
notAfter=Dec 30 21:17:08 2021 GMT

So I think the main cert is ok, but sees the CA as expired?

3. wget --no-check-certificate https:// repo.turris.cz/hbs/omnia/packages/base/ca-bundle_20200601-1_all.ipk
   opkg install ca-bundle_20200601-1_all.ipk
   wget --no-check-certificate https:// repo.turris.cz/hbs/omnia/packages/base/ca-certificates_20200601-1_all.ipk
   opkg install ca-certificates_20200601-1_all.ipk
   There is no “update-ca-certificates” command, as I’ve seen on some posts. Tried
   opkg upgrade ca-bundle
   opkg upgrade ca-certificates
   but still nothing.

LATER EDIT:
cat /etc/ssl/certs/DST_Root_CA_X3.crt | openssl x509 -noout -enddate

notAfter=Sep 30 14:01:15 2021 GMT

So this is definitely the culprit. Installing the certificate packages wasn’t enough, as I suspected. What should I run to update CAs?

[Mox]

I found this. So it seems it’s a Let’s Encrypt CA issue and the openssl version I have installed on the router.

I don’t think openssl should use DST_Root_CA_X3.crt as CA, but R3 and ISRG Root X1 as Chrome shows for repo.turris.cz. There is no newer DST_Root_CA_X3.crt, it expired on September 30 as it was supposed to, so installing latest certs bundle was correct but didn’t fix the issue.

openssl version
OpenSSL 1.0.2t 10 Sep 2019

Just upgrading openssl without reflashing and losing config seems improbable. opkg whatdepends libopenssl shows too many package dependencies, including openssh-server and lighthttpd, might break something.

I’ll try to reflash the latest 5x medkit and hope restoring backed up config from 4.0.1 will work.

LE: I used the LED mode 6 method to download and flash latest HBS medkit, chose minimal config, setup password, then copied / overwritten config from 4.0.1 over WinSCP, as I learned that LuCi config backup/restore isn’t available in 5.x anymore. Reboot, and so far all works as expected.

I was surprised mode 6 worked, expecting it to run into the same SSL issue, and was also prepared to use mode 4. But it seems mode 6 it doesn’t use the https version of the repo, but probably the http one.

[Pepe]

Thanks for letting us know about this issue! We are investigating it, and we will get back to you with more details. In the meantime, reflash your router by using the latest medkit according to the documentation.

[Mox]

Thank you. I’m in luck and mode 6 worked (mode 4 would have worked also, but I was both lazy and curious), also restoring config over WinSCP worked. This should probably be documented somewhere, given that restoring config from LuCi isn’t an option anymore and everybody talks about snapshots, but they weren’t an option for me.

I am pleasantly surprised that upgrading from 4.0.1 to 5.2.7 without losing config went so smooth. I went through all that debugging because of stubborness and not trusting that an upgrade will be ok. I mistook 3.x → 5.x config problems with 4.x, had to read a lot on the forums to understand everything, but it was worth it. I also now find reForis useful with the new options. Your team does a good job.

[mvasilek]

For anyone who might have this problem in the future, here is a quick workaround: remove the content of /etc/ssl/certs/DST_Root_CA_X3.crt from /etc/ssl/certs/ca-certificates.crt.