ISP issue with CDN Mozilla

Software SW help
,
1

How does the negative anchor work? Does it accept some wildcards, or do I have to type out exact domains?

I wanted to add addons.cdn.mozilla.net (which unfortunately has broken DNSSEC by using cloudfront). Without DNSSEC, it resolves to:

;; QUESTION SECTION:
;addons.cdn.mozilla.net.                IN      A

;; ANSWER SECTION:
addons.cdn.mozilla.net. 31      IN      CNAME   addons-cdn-geo.prod.mozaws.net.
addons-cdn-geo.prod.mozaws.net. 299 IN  CNAME   d1sp2sgy246t7c.cloudfront.net.
d1sp2sgy246t7c.cloudfront.net. 59 IN    A       13.249.9.62

I added

trust_anchors.negative = { 'addons.cdn.mozilla.net', 'cloudfront.net', 'mozaws.net' }

to /etc/kresd/custom.conf (and changed /etc/config/resolver to tell it about this file).

But even after resolver restart, I still get SERVFAIL with DNSSEC activated.

2

how is DNSSEC broken for addons.cdn.mozilla.net ?

Summary

; <<>> DiG 9.11.5-P1 <<>> addons.cdn.mozilla.net +dnssec
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29229
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;addons.cdn.mozilla.net. IN A

;; ANSWER SECTION:
addons.cdn.mozilla.net. 59 IN CNAME addons-cdn-geo.prod.mozaws.net.
addons.cdn.mozilla.net. 59 IN RRSIG CNAME 7 4 60 20190115013849 20190112003849 41873 mozilla.net. dBv7XEP3BKl/9RgtKF5/98ac6AIV9TPFV2WnGx2RrAqFCk273zsvtxGB 2pZZE07nyjFOZ8b3kG+dU4GPXyEgEVcEAHIqAyV+kUn/RunFVCw1x 57u /pnrFoZcanPR6R1YfTynuqGYs++CemxYhNM+BmtqtF7Kxio3ee05q9Lf Ez0=
addons-cdn-geo.prod.mozaws.net. 299 IN CNAME d1sp2sgy246t7c.cloudfront.net.
d1sp2sgy246t7c.cloudfront.net. 60 IN A 13.249.9.62

;; Query time: 976 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; MSG SIZE rcvd: 319

You sure it is not the upstream DNS server kresd is querying that is delivering a broken DNSSEC response?

3

That’s possible, because when I set kresd to not forward or to use another than my provider’s upstream DNS server, DNS resolution works. However, the question remains - if I know my provider breaks DNSSEC for this domain, how do I tell kresd to ignore it?

4

I find it likely that you suffer from exactly this problem, and consequently a noticeable fraction of names might be broken, mainly sites served from CDNs. I don’t think any approach with explicit “whitelist” will be pleasant.

Hmm, on second thought, I’m not convinced negative trust anchors in kresd will be usable to work around this forwarding problem. Explanation I originally wanted to send:

It turns off DNSSEC for the whole sub-tree. As described there, due to DNSSEC being designed to protect zones as wholes, in kresd it often takes effect only from zone cuts below the specified names – so if you want to cover cdn.mozilla.net., the name belongs directly into the mozilla.net zone and you need to have negative TA at that point already to be certain.

5

Thanks for pointing to the relevant thread. But asking my provider’s DNS servers directly works (at least from dig):

~# dig @217.77.165.81 addons.cdn.mozilla.net

; <<>> DiG 9.11.5-P1 <<>> @217.77.165.81 addons.cdn.mozilla.net
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27296
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;addons.cdn.mozilla.net.                IN      A

;; ANSWER SECTION:
addons.cdn.mozilla.net. 60      IN      CNAME   addons-cdn-geo.prod.mozaws.net.
addons-cdn-geo.prod.mozaws.net. 208 IN  CNAME   d1sp2sgy246t7c.cloudfront.net.
d1sp2sgy246t7c.cloudfront.net. 60 IN    A       13.32.100.227

;; Query time: 20 msec
;; SERVER: 217.77.165.81#53(217.77.165.81)
;; WHEN: Sat Jan 12 14:58:13 CET 2019
;; MSG SIZE  rcvd: 148
6

Yes, otherwise the ISP would’ve been pushed to fix their servers. ATM for kresd the combination of forwarding and negative trust anchors won’t result into a single direct query.

7

Okay. With

trust_anchors.negative = { 'mozilla.net', 'cloudfront.net', 'mozaws.net' }

DNS resolution for addons.cdn.mozilla.net works. @vcunat Could you please update wiki to include the information that you should add 2nd level domain names as trust anchors?

Though I still don’t understand why does the resolution not work without the negative trust anchor - does it mean my ISP screws up the DNSSEC records for this domain? Why would that happen? @vcunat feel free to split out this conversation to a separate topic, as I feel I kind of hijacked this topic with something not really relevant. Anyways, thanks for nice over-the-weekend support.

8

I don’t have rights to split topics. @Pepe: it would be nice to split this sub-thread starting with post 13.

As with the thread I linked, I expect this request will SERVFAIL:

dig @217.77.165.81 DS cdn.mozilla.net

kresd is using such intermediate queries during forwarding to discover the DNSSEC chain. “Normal” clients (e.g. non-validating) wouldn’t do such queries, so they wouldn’t be affected. ISPs often use old buggy resolver versions, I hear. Well for almost all their customers these bugs won’t be noticed in practice, so the ISPs aren’t pushed much to update, I guess…

9

Perfect, this query fails. I’ve reported the issue to Vodafone. Thanks very much.

1 Like