Kresd+wireguard configuration

bruteforce · September 9, 2019, 8:15am

Hello, trying to solve out this particullar scenario:

On turris router (version: Forris 100.3, TurrisOS 4.0-beta9):

knot-resolver (caching, recoursive server), version: 3.2.1-3.4
wireguard server, version: 0.0.20190601-1.0

Clients: laptos (widows, macos) and smartphones (iphone).

In both cases described bellow I am able to ping anything from connected wireguard client (router, DNS server and so on).

(A)
So here is the thing when you configure kresd as a forwarding DNS with option option forward_custom ‘99_cloudflare’ and connect wireguard client to wireguard server on turris router everything runs smootly. So basicaly you need to follow this KB arcticle which is about adding DoT servers: https://doc.turris.cz/doc/en/public/dns_knot_misc.
Again if you connect with your wireguard client to server everything is good including DNS resolving.

(B)
But look (in the link provided) for part called: Setup in Turris OS 3.9.6 through 3.10.8. Forwarding_upstream is acctually OFF and other settings are taken from custom.conf file which is clear. So I follow this advice so set forwarding_upstream to 0 and prepared some configuration in custom file. In custom configuration file is set of DNS servers where each supports DoT. The reason why I do this is to randomize selection of DNS server for clients queries. And this is the issue. So now if you connect with your client to wireguard server DNS resolution doesn’t work properly. Sometime works good and sometime not in other words if you try resolve name for instance bbc.com it resolves for first time, not for second, then after some time it resolves and so on…
Resolving is good ANYTIME (case A and B) for client on the LAN using the very same DNS server.

So the question is what actually has changed in version 3.11 and higher in resolver settings? What could be the problem of resolveng. Suppose that both configuration examples (in link provided) is basically the same so I can still use configuration style like in previous version of Turris resolver (apparently not fully), I am right?

Thank you for your help.

vcunat · September 9, 2019, 9:36am

Nitpick: if you just specify multiple servers, you do not get randomization. “Better” servers get preferred most of the time, based mainly on speed of answering (and presence of bad answers). There’s a new attempt to randomize in a more clever way, but that’s not packaged for Turris yet.

I’m not aware of any changes that would cause something like this. The best I can advise is to gather more information. For example, what do you mean that DNS does not work – no reply arrives to the client? Also, verbose logging often shows useful information, e.g. whether the problem is on the way in or on the way out (or inside resolver).

bruteforce · September 10, 2019, 10:46am

First thank you for your quick response. So:

“Randomization” - yes you are completely right it is not fully randomized (I shell take a look on that), I just randomizing selection from a list of dns servers. Will provide configuration which is in /etc/config/resolver and /etc/kresd/kresd.custom
[A - case]
Files:
/etc/config/resolver:

    config resolver 'common'
            list interface 'x.x.x.x' -> doesn't matter right now
            list interface 'x.x.x.x'
            list interface 'x.x.x.x'
            option port '53'
            option keyfile '/etc/root.keys'
            option verbose '0'
            option msg_buffer_size '4096'
            option msg_cache_size '50M'
            option net_ipv6 '0'
            option net_ipv4 '1'
            option prefered_resolver 'kresd'
            option prefetch 'yes'
            option static_domains '1'
            option dynamic_domains '0'
            option ignore_root_key '0'
            option forward_upstream '0'

config resolver 'kresd'
        option rundir '/tmp/kresd'
        option log_stderr '1'
        option log_stdout '1'
        option forks '1'
        option keep_cache '1'
        option include_config '/etc/kresd/custom.conf'

In file /etc/kresd/custom.conf is:

require 'math'
math.randomseed(os.time())
tls_bundle='/etc/ssl/certs/ca-certificates.crt'
dns_providers = {
      { -- CZ.NIC
        {'193.17.47.1', hostname='odvr.nic.cz', ca_file=tls_bundle},
        {'185.43.135.1', hostname='odvr.nic.cz', ca_file=tls_bundle},
      },
      { -- Quad9
        {'9.9.9.9', hostname='dns.quad9.net', ca_file=tls_bundle},
        {'149.112.112.112', hostname='dns.quad9.net', ca_file=tls_bundle},
        {'2620:fe::fe', hostname='dns.quad9.net', ca_file=tls_bundle},
      },
      { -- Cloudflare Resolver
        {'1.1.1.1', hostname='cloudflare-dns.com', ca_file=tls_bundle},
        {'1.0.0.1', hostname='cloudflare-dns.com', ca_file=tls_bundle},
        {'2606:4700:4700::1111', hostname='cloudflare-dns.com', ca_file=tls_bundle},
        {'2606:4700:4700::1001', hostname='cloudflare-dns.com', ca_file=tls_bundle},
      },
      { -- UncensoredDNS
        {'89.233.43.71 ', hostname='unicast.censurfridns.dk', ca_file=tls_bundle},
        {'2a01:3a0:53:53::0', hostname='unicast.censurfridns.dk', ca_file=tls_bundle},
      },
    }

    --get bigger chance to select cz.nic servers
    table.insert(dns_providers, dns_providers[1])

    policy.add(function (request, query)
      return policy.TLS_FORWARD(dns_providers[math.random(1, #dns_providers)])
    end)

------ Basically it is the same like here: https://www.ctrl.blog/entry/kresd-random-dns-forwarding.html -----

[B - case - works OK, with wireguard as well]
Here is what I have now (configurations which is fine and suffucient):
/etc/config/resolve

 config resolver 'common'
    	list interface 'x.x.x.x'
    	list interface 'x.x.x.x'
    	list interface 'x.x.x.x'
    	option port '53'
    	option keyfile '/etc/root.keys'
    	option verbose '0'
    	option msg_buffer_size '4096'
    	option msg_cache_size '50M'
    	option net_ipv6 '0'
    	option net_ipv4 '1'
    	option prefered_resolver 'kresd'
    	option prefetch 'yes'
    	option static_domains '1'
    	option dynamic_domains '0'
    	option ignore_root_key '0'
    	option forward_upstream '1'
         option forward_custom '00_odvr-cznic'

config resolver 'kresd'
	option rundir '/tmp/kresd'
	option log_stderr '1'
	option log_stdout '1'
	option forks '1'
	option keep_cache '1'

Focus on option forward_upstream ‘1’ in file /etc/config/resolver.
So in case B - custom configuration for kresd deamon is not used as I previously described. Now to your second point.
If you try configuration provided case A for instance in docker (I mean common knot-resolver docker image kindle provided by cz.nic) with this particular configuration you should see traffic on port 853 and client are able to resolver every single query. Also if you try this config on Turris (without wireguard) - yes, you got a functional configuration for the LAN clients. And here is the thing: if I try case A with wireguard clients there not able to resolve some queries anytime - I mean sometime they are, 1-yes; 2-no;3-yes;4-yes;5-no.

So question could be: I am right if say that configuration in case A and case B are very the same? Focus on kresd configuration because what it does in case A - forwards all the queries to particular dns server and in case B - the same forward traffic to to a particular server but in this case to only one - cz.nic ODVR. So it is the same or not?!?

Is it more clear?

vcunat · September 11, 2019, 7:30am

Since kresd 3.2.0 this makes it ignore it IPv6 addresses even in rules like TLS_FORWARD – I see at least one IPv4 in each set… but the third set has a wrong one with additional space,

target '89.233.43.71 " is not a valid IP address

so it makes sense to me that passing this third set makes things fail – and as you seem to have 25% chance of passing that set, it only fails sometimes.

bruteforce · September 11, 2019, 8:01am

Have nothing to say …
I’ll adjust config file and after some time will let you know.

bruteforce · September 12, 2019, 10:34am

OK, you solved the issue. I didn’t catch it. So problem was that in “ipaddress” in kred.config was an additional space so kresd was actually asking address like '89.233.43.71 ’ <- space is here.
Thank you for your help I’ve learned a lot. Possible to close this ticket.