Knot resolver and RPZ

Here you have version without temporary file:

#!/bin/bash

blacklistfile="/etc/kresd/blacklist.rpz"

cat > $blacklistfile <<EOF
\$TTL 60
@               IN      SOA  localhost. root.localhost.  (
            2   ; serial
            3H  ; refresh
            1H  ; retry
            1W  ; expiry
            1H) ; minimum
        IN      NS    localhost.

EOF

{
    wget -q 'http://malwaredomains.lehigh.edu/files/immortal_domains.txt' -O - | grep -v \#; \
    wget -q 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=1&startdate%5Bday%5D=&startdate%5Bmonth%5D=&startdate%5Byear%5D=&mimetype=plaintext' -O - ; \
    wget -q 'http://malwaredomains.lehigh.edu/files/BOOT' -O - | grep PRIMARY | cut -d " " -f 2; \
} | sort -u | sed -e 's/.*/\0\tCNAME\t.\n*.\0\tCNAME\t./' >> $blacklistfile

/etc/init.d/kresd reload

Hey All, I would love to get some adblocking working for my omnia turris.
Would it be possible for someone to either point me to a guide, or maybe even write one on here?

I would also really appreciate it, if they remembered that not everyone is a Unux whizz and the 3(?) (FCI,LUCI and Putty connect) different ways of accessing and changing things on the Omnia, aren’t a help, rather a hinderence for us basic users.
I read these suggestions and my brain starts to fizz!

I must admit to finding it really painful that there is an adblock app(plug-in whatever you want to call it), which doesn’t work with the basic system, yet is one that the update process seems dedicated to install.

I understand that people are maxed out, and this is the Christmas/new year holiday period. But this software has, I understand, been in development for ages now, yet it still seems to be full of contradictions and requirements for things to be done in 2 or 3 different places.

I thought the device was supposed to get getting regular updates and foolishly, I had hoped these updates would make things easier for me to use the device, but so far I’ve not seen it.

I also understand we’re at early days here, but at the moment it feels we are heading towards something you need a degree in Unix and its config to set up, as opposed to an end user friendly device which works neatly from a single interface

To me personally it doesn’t seem like ad-blocking is a basic router function that needs to work just by clicking a checkbox in Foris. AFAIK most (cheap) routers won’t let you do any advanced stuff at all; here you can do whatever you want with the system.

I would suggest to install ‘nano’, it is a very simple file editor.
Otherwise you can look at ‘vi’.

This is the script I use, very similar to @Cabal script.

#!/bin/bash

blacklistfile="/etc/kresd/blacklist.rpz"

cat > $blacklistfile <<EOF
\$TTL 60
@               IN      SOA  localhost. root.localhost.  (
                        2   ; serial
                        3H  ; refresh
                        1H  ; retry
                        1W  ; expiry
                        1H) ; minimum
                IN      NS    localhost.

EOF


{
    curl 'https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts' | gr                                           ep "^0" | cut -d " " -f 2; \
} | sort -u | sed -e 's/$/\tCNAME\t./' >> $blacklistfile


/etc/init.d/kresd reload

Be sure to add:
policy.add(policy.rpz(policy.DENY, '/etc/kresd/blacklist.rpz'))
to
/etc/kresd/custom.conf

1 Like

So for everyone willing to try this.
I took the guide jklaas posted and combined it with other info in the thread to make a beginner guide.

1). First open /etc/config/resolver with a text editor such as vi:

vi /etc/config/resolver

Look for the following:

config resolver 'kresd'
       option rundir '/tmp/kresd'
       option log_stderr '1'
       option log_stdout '1'
       option forks '1'
       #option include_config '/tmp/kresd.custom.conf'
       #option hostname_config '/etc/hosts'

And change it to:

config resolver 'kresd'
       option rundir '/tmp/kresd'
       option log_stderr '1'
       option log_stdout '1'
       option forks '1'
       option include_config '/etc/kresd/custom.conf'
       #option hostname_config '/etc/hosts'

Also look for the following:

config resolver 'common'
        list interface '0.0.0.0'
        list interface '::0'
        option port '53'
        option keyfile '/etc/root.keys'
        option verbose '0'
        option msg_buffer_size '4096'
        option msg_cache_size '20M'
        option net_ipv6 '1'
        option net_ipv4 '1'
        option forward_upstream '1'
        option prefered_resolver 'kresd'
        option ignore_root_key '0'
        option prefetch 'yes'

And change it to the following:

config resolver 'common'
        list interface '0.0.0.0'
        list interface '::0'
        option port '53'
        option keyfile '/etc/root.keys'
        option verbose '0'
        option msg_buffer_size '4096'
        option msg_cache_size '20M'
        option net_ipv6 '1'
        option net_ipv4 '1'
        option forward_upstream '0'
        option prefered_resolver 'kresd'
        option ignore_root_key '0'
        option prefetch 'yes'

2). Open /etc/kresd/custom.conf with a text editor:

vi /etc/kresd/custom.conf

Add the following lines in the specified to the file:

policy.add(policy.rpz(policy.DENY, '/etc/kresd/blacklist.rpz'))
policy.add(policy.all(policy.FORWARD('8.8.8.8')))
policy.add(policy.all(policy.FORWARD('8.8.4.4')))

3). Create a script file called “generate_blacklist.sh” in the following location /usr/local witht he following command:

vi /etc/kresd/generate_blacklist.sh

Add the following text/script:

#!/bin/bash

blacklistfile="/etc/kresd/blacklist.rpz"

cat > $blacklistfile <<EOF
\$TTL 60
@		IN	SOA  localhost. root.localhost.  (
			2   ; serial 
			3H  ; refresh 
			1H  ; retry 
			1W  ; expiry 
			1H) ; minimum 
		IN	NS    localhost.

EOF

{
    wget -q 'http://malwaredomains.lehigh.edu/files/immortal_domains.txt' -O - | grep -v \#; \
    wget -q 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=1&startdate%5Bday%5D=&startdate%5Bmonth%5D=&startdate%5Byear%5D=&mimetype=plaintext' -O - ; \
    wget -q 'http://malwaredomains.lehigh.edu/files/BOOT' -O - | grep PRIMARY | cut -d " " -f 2; \
} | sort -u | sed -e 's/.*/\0\tCNAME\t.\n*.\0\tCNAME\t./' >> $blacklistfile

/etc/init.d/kresd reload

4). Make the above script exacuteable with the following command:

chmod +x /etc/kresd/generate_blacklist.sh

5). Create a cronjob to exactue the above script, make a cronjob with the following command:

crontab -e

And add the following line:

6 1 * * 3 /etc/kresd/generate_blacklist.sh

That should be all…
Please give love/thanks to jklaas, cabal, Leonardo and all others that helped in this thread

P.S. If I made any mistakes please let me know and I’ll correct them asap

7 Likes

Hmmzz,
So I used the above to setup my adblocking.
But it doesn;t block anything.
I see the blacklists gets created and contains thousands of entries…
But I suspect requests aren’t checked against it.
Is there anything I’m forgetting?

Why it shouldnt be there? In Foris in updater we can find for example tor, web cam, tvheadend, majordomo and etc.
I think it’s good idea to have it there! :slight_smile:

In other case on some cheap router you can replace stock firmware with OpenWRT/DD-WRT/LEDE and then you can have ad-blocking :slight_smile:

I think that policy order is important.
In my case I have forwarding disabled (via foris or by setting 0 in /etc/config/resolver somewhere around forward.
And I’ve added forwarders to /etc/kresd/custom.conf (after that line with DENY):

policy.add(policy.rpz(policy.DENY, '/etc/kresd/blacklist.rpz'))
policy:add(policy.all(policy.FORWARD('8.8.8.8')))
policy:add(policy.all(policy.FORWARD('8.8.4.4')))

You can get those forwarders by typing (before disabling forwarding):

cat /tmp/kresd.conf
1 Like

Why does your first line begin with: policy.add
and the subsequent lines start with: policy:add
?

It’s quite funny, but looks like both versions are ok.

Docs (http://knot-resolver.readthedocs.io/en/latest/modules.html#query-policies) says to use dot, but /etc/init.d/kresd uses colon to generate forwarders.

I changed it to dot version as docs says.

policy:add(action) is equal with policy.add(policy, action). They are Knot Resolver 1.0.0 API calls and policy.add(action) is 1.1.0 API call. The support for policy:add(action) will be removed in 1.2.0 API.

1 Like

Working now!

Updated my “guide”

sort -u
It removes duplicates.

1 Like

Thank you all for this.

Now, while this works I do have a problem.
In order to include some own domains I use a second file and I added a policy
policy.add(policy.rpz(policy.DENY, '/etc/kresd/ownbl.rpz'))
While for almost all domains this works, there is at least one domain that this is not working.
For stoiximan.gr this is working but not for www.stoiximan.gr

#dig stoiximan.gr
; <<>> DiG 9.9.8-P4 <<>> stoiximan.gr
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 35433
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;stoiximan.gr.			IN	A

;; AUTHORITY SECTION:
blocked.		900	IN	SOA	blocked. . 0 3600 900 604800 900

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Jan 28 23:07:21 EET 2017
;; MSG SIZE  rcvd: 83

BUT

#dig www.stoiximan.gr
    ; <<>> DiG 9.9.8-P4 <<>> www.stoiximan.gr
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58848
    ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 4096
    ;; QUESTION SECTION:
    ;www.stoiximan.gr.		IN	A

    ;; ANSWER SECTION:
    www.stoiximan.gr.	242	IN	CNAME	stoiximan.gr.
    stoiximan.gr.		242	IN	A	104.16.102.199
    stoiximan.gr.		242	IN	A	104.16.99.199
    stoiximan.gr.		242	IN	A	104.16.98.199
    stoiximan.gr.		242	IN	A	104.16.101.199
    stoiximan.gr.		242	IN	A	104.16.100.199

    ;; Query time: 105 msec
    ;; SERVER: 127.0.0.1#53(127.0.0.1)
    ;; WHEN: Sat Jan 28 23:02:12 EET 2017
    ;; MSG SIZE  rcvd: 139

The policies are only applied to incoming queries, so if that RPZ only matches stoiximan.gr and not www.stoiximan.gr, this would be OK.

Thank you for the answer. Indeed it’s ok

Check my script.
I’ve updated sed parameters and now it creates entries for domain and wildcard for subdomains:

domain.com CNAME .
*.domain.com CNAME .

@Giblet - update Your guide

1 Like

Done, and thanks for the headsup!

Hi, can you please summarize the above described procedure step by step for a man who does not control the Linux commands. Thank you