Turris Omnia is munging SSL certificate ("Let's Encrypt" become "Turris" producing SEC_ERROR_UNKNOWN_ISSUER). Why? And how to fix?

Software SW tweaks
1

I have a webserver behind the Omnia. The Omnia has ports 80 and 443 open right now, and I used certbot to issue the certificate.

There’s a thread here describing the whole setup more or less:

but the key summary is that the certificate is issued fine by certbot and it validates fine byc ertbot and I can see that it has the issuer set right with a test that letsencrypt recommend:

$ openssl x509 -in /etc/letsencrypt/live/leaderboard.space/cert.pem -issuer -noout
issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3

and yet, if I try to access the web page the browser complains with SEC_ERROR_UNKNOWN_ISSUER and if I test form outside:

https://decoder.link/sslchecker/leaderboard.space/443

I can see that Issuer Organization is munged by the Omnia to: Turris

How? And why? And how can I get the Omnia firewall to pass through the certificate unmunged?

FYI the web site runs fine, all is good, just this certificate not recognized as trusted. It is accessed via a DDNS domain name, and lighttpd on the Omnia forwards that domain to my webserver:

$HTTP["host"] == "leaderboard.space" {
      proxy.server  = ( "" => ( ( "host" => "192.168.0.14" ) ) )
}

and on the webserver itself lighttpd serves the site and is configured:

$HTTP["host"] =~ "(leaderboard.space|arachne.lan)" {
		server.name             = "leaderboard.space"
		server.document-root    = "/var/www/html/leaderboard.space"

		$SERVER["socket"] == ":443" {
		    ssl.engine              = "enable"
		    ssl.ca-file             = "/etc/letsencrypt/live/leaderboard.space/chain.pem"
		    ssl.pemfile             = "/etc/letsencrypt/live/leaderboard.space/combined.pem"
		    ssl.honor-cipher-order  = "enable"
		    ssl.use-sslv2           = "disable"
		    ssl.use-sslv3           = "disable"
		}

		url.rewrite-once = ( "^/favicon\.ico$" => "/static/favicon.ico" )

		$HTTP["url"] !~ "^/static/" {
				scgi.protocol = "uwsgi"
				scgi.server = ( "/" => (( "socket" => "/var/run/lighttpd/uwsgi.socket-0", "check-local" => "disable" )), )
    }
}

More details on the letsencrypt forum if needed or provided if needed. But the issue is that I can see the issuer as “Let’s Encrypt” when I test it from the LAN side of the Omnia and as “Turris” when I test it from the WAN side. So it seems like the Omnia is intervening and altering the Issuer.

Any help here appreciated.

As an aside all the letsencrypt threads I found on the forum already related to certificates stored on the Omnia for the Omnia and I am using a webserver behind the Omnia.

2

Out of interest I put the certificate onto the Omnia and repeated the test form there:

# openssl x509 -in cert.pem -issuer -noout
issuer= /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

So the Omnia too can see that the Issuer is Let’s Encrypt. Conclusion the Omnia is not even sending my certificate, but some other certificate for some reason I don’t understand.

Appreciate all help here.

3

Indeed I just looked more closely at the diagnostics:

https://decoder.link/sslchecker/leaderboard.space/443

I can see:

OpenSSL Handshake

depth=0 C = CZ, ST = Prague, L = Prague, O = Turris
verify error:num=18:self signed certificate
verify return:1
depth=0 C = CZ, ST = Prague, L = Prague, O = Turris
verify return:1
CONNECTED(00000003)
OCSP response: no response sent
---
Certificate chain
 0 s:/C=CZ/ST=Prague/L=Prague/O=Turris
   i:/C=CZ/ST=Prague/L=Prague/O=Turris
-----BEGIN CERTIFICATE-----
MIIDUzCCAjugAwIBAgIJAPqNA/ft7T6FMA0GCSqGSIb3DQEBCwUAMEAxCzAJBgNV
BAYTAkNaMQ8wDQYDVQQIDAZQcmFndWUxDzANBgNVBAcMBlByYWd1ZTEPMA0GA1UE
CgwGVHVycmlzMB4XDTE3MDMyODEyMzExNloXDTM3MDMyMzEyMzExNlowQDELMAkG
A1UEBhMCQ1oxDzANBgNVBAgMBlByYWd1ZTEPMA0GA1UEBwwGUHJhZ3VlMQ8wDQYD
VQQKDAZUdXJyaXMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDvbWl/
Rz04G0E+rJO4miLVVd7X3dxeqZXRtgHt1RgXWmq8LWqkM/zbE0kHkc4hi5Qnl4+K
UEXpAKGEN+iBkDsrMGCAzIqUXQaN7jTGNmoTFifWEsgOhra/iWjTWC8pW47U3P6X
0HhT/2XVbMs5FMJHaSnlUWeDxlpE01UKxfFwLqO1p9FD3oK4ExO5jaMjL34wCGxD
RKdWXR+v9a5DxTIuGrWIIIxr/S4Xo0UqrePV364cFGHoEna7gyH0YyxQxNS/l4T1
R9qoEV4KaAdGnDFB5sTINU6JLHygwsowKfBWOMgNRdlbG9o0KjoOb3rxmdF3PnCi
ztM5Hy9OertO3QhNAgMBAAGjUDBOMB0GA1UdDgQWBBRNi8Gm32uEBPX9F49vBVu1
UiWl6TAfBgNVHSMEGDAWgBRNi8Gm32uEBPX9F49vBVu1UiWl6TAMBgNVHRMEBTAD
AQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCSHUX32zTfP7zywC/9u2KltyFsR88xZQM2
JJKYHbntL0f7+pC8qHAtSbR8PnjUgype4QNNPfRABY7WEuE0uPmvNhS7+WvER9XG
fUxYD9VH9d5AFLRPgDrBrhiv4Hi68kbaRDKktAnAI1nu4ScBRJTN4xLm44OQ4B+C
vdh/CDdazYMzmcxJGotoed6dx6VKvBzFsaU7xTUOKFQi6ofNKeI3mLppVWDUI7Br
FvQl0DnpWeY3vMewYsVsvtAGwZQYQbzCG/gxrXwhvL9izy+wBKEVovPTTfsggWwr
m6dKWjUPGFDUSijDtIExwHdL+soDiLqEqoVgO6h1LDXC+voRQ7h8
-----END CERTIFICATE-----
---
Server certificate
subject=/C=CZ/ST=Prague/L=Prague/O=Turris
issuer=/C=CZ/ST=Prague/L=Prague/O=Turris
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---

Which is clearly not my certificate which begins:

-----BEGIN CERTIFICATE-----
MIIFBTCCA+2gAwIBAgISA1FP6MicdisQQSycSiG1N/LAMA0GCSqGSIb3DQEBCwUA

So the Omnia is not even sending my certificate the rascal! What gives?

OK, so I did a 443 port forward to the webserver and then all is good. The link above checks out, and the page loads, without an self-signed warning. Great. I have a work around.

Problem is I don’t want to do a port forward as I want lighttpd on the router to farm out requests based on domain name.

So it’s looking like lighttpd on the Omnia is the thing I have somehow to coax into passing through the certificate?

4

So, moved this thread to here:

https://redmine.lighttpd.net/boards/2/topics/7585

But will still be mighty grateful if anyone out there has a webserver behind an Omnia who has done a SSL pass-through can pipe up, if such a person exists, which predicates it being possible!

5

I have done no investigation, but I suspect that your lighttps certificate was overriden by liggttpd-https-cert package. Check lighttpd configuration and if so remove lighttpd-https-cert package.

6

Thanks cynerd. Much appreciate the tip. I uninstalled that package, but didn’t work as hoped. Behaviour changed indeed, testing after installing it with the SSL certificate tester I’m using at https://decoder.link/sslchecker/leaderboard.space/443 reveals that no certificate is delivered afterwards. So half fixed. Lightttpd isn’t delivering its own, but isn’t passing through the one from my webserver.

And oddly, the Omnia reinstalled the package automatically shortly after. Odd.

I have a suspicion, frustratingly in a sense to my preference of keeping the website and its certificate together on the webserver, that the fastest fix will be for me to copy the certificate to the Omnia and have it serve the certificate. I’ll try and experiment with that in coming days. I live in hope that there is a way to ask lighttpd to pass-through tehc ertificate check. This is how I’m passing the web handling on:

$HTTP["host"] == "mydomain.tld" {
      proxy.server  = ( "" => ( ( "host" => "webserverip" ) ) )
}

which works a charm for the website over port 80 or 443, just not the SSL certificate, which lighttpd on the Omnia wants to negotiate on its own ;-). But in the docs:

https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_SSL

I’m not seeing a clear way of asking lighttpd to delegate the SSL certificate delivery to another host (behind it on the LAN).

Regards,

Bernd.

7

OK, did a quick test and can confirm that if I copy the certificate to the Omnia then and config lighttpd with:

$HTTP["host"] == "mydomain.tld" {
      $SERVER["socket"] == ":443" {
          ssl.engine              = "enable"
          ssl.ca-file             = "/etc/letsencrypt/live/mydomain.tld/chain.pem"
          ssl.pemfile             = "/etc/letsencrypt/live/mydomain.tld/combined.pem"
          ssl.honor-cipher-order  = "enable"
          ssl.use-sslv2           = "disable"
          ssl.use-sslv3           = "disable"
      }

      proxy.server  = ( "" => ( ( "host" => "myserverip" ) ) )
}

… so it’s only those two files (chain.pem and combined.pem) that I copy over to the Omnia, then:

  1. If lighttpd-https-cert is installed the Omnia still delivers the Turris certificate.
  2. If I remove liggttpd-https-cert then finally I get a successfull delivery of my certificate and my site works cleanly!

Now I remain curious to see if that package gets reinstalled somehow like it did yesterday. If so I have some other issue to resolve of course. in the interim, while not super happy (I’d prefer the Omnia’s lightttpd to pass-thru certificate management and delivery to my webserver) I have a working solution.

8

I believe that Lighttpd doesn’t support ssl pass trough with reverse proxy, at least not before 1.5 version. So you really have to have your certificate on Turris.

And why lighttpd-https-cert is returning is because it is part of base system. So updater always reinstalls it back unless you specify it explicitly. See this: https://www.turris.cz/doc/en/howto/updater#i_deleted_a_package_but_it_came_back

1 Like
9

Thanks heaps! It did come back again and so I have now added:

Uninstall("lighttpd-https-cert", { priority = 60 })

to /etc/updater/user.lua as recommended in that article. Most helpful.

Unfortunate that lighttpd doesn’t support SSL passthough, and hope it does some time in future as I do see a lot of articles about nginx and SSL passthrough it seems but not for lighttpd. It’s not a crisis of course, as the Omnia can serve it too, just not a clean division of responsibilities (Omnia as router, webserver behind it with websites and their SSL certificates).

So what I have done for now is written a small script that publishes the certificate from the webserver to the Omnia and will have to do so after every renewal.

1 Like