I read the articles on doing an RPZ and I think it would be best to have this as something that can be done within the Knot resolver, it has that capability. However, the documentation is vague, especially with respect to how the Omnia runs.
to /etc/init.d/kresd. Any file like “blacklist.rpz” has to be put in $DEFAULT_RUNDIR which is “/tmp/kresd”. Otherwise a full path is needed. The Knot documentation states
rpz - implementes a subset of the RPZ format. Currently it can be used with a zonefile, a binary database support is on the way. Binary database can be updated by an external process on the fly.
zone "yieldmanager.net" { type master; notify no; file "null.zone.file"; };
and created a null.zone.file, also in /etc/init.d/kresd that is basically a wildcard pointing to 127.0.0.1
This doesn’t appear to work, but I think I have close to all the pieces. I expect my “bind 8 config” file wont really work and I need a different format.
Since I’m also doing local resolution with dnsmasq, I’ve added it as the last line in the “init_headers” stanza, just before the closing bracket.
On your suggestion, I tried just a list of hosts, but that didn’t seem to work either. Since knot was created by cz.nic, I was hoping someone there might chime in.
“Note: The module (and kres) expects domain names in wire format, not textual representation. So each label in
name is prefixed with its length, e.g. “example.com” equals to “\7example\3com”. You can use convenience
function todname(’example.com’) for automatic conversion.”
Tried both rpz file which than stops all addresses with dns conf error. And direct policy.add(policy.suffix(policy.DENY, {’\6badboy\2cz’})) type with no success.
OK, I figured it out. After perusing some of the links in the knot-resolver source, I played around with the files a little and came up with this blacklist.rpz file:
$ dig zantracker.com
; <<>> DiG 9.9.5-9+deb8u8-Debian <<>> zantracker.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 62428
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;zantracker.com. IN A
;; AUTHORITY SECTION:
blocked. 900 IN SOA blocked. . 0 3600 900 604800 900
;; Query time: 0 msec
;; SERVER: 192.168.4.1#53(192.168.4.1)
;; WHEN: Tue Dec 06 13:53:06 EST 2016
;; MSG SIZE rcvd: 85
So satisfying.
This should be trivial to automate with some quick scripting.
[edit] I also moved the “policy.add(policy.rpz(policy.DENY, ‘/etc/kresd/blacklist.rpz’))” line to just after the “modules” stanza and before the other “policy.add” lines.
to /etc/init.d/kresd, just after the “modules” stanza. So, that means after the
modules = {
...
}
block.
Create a script file called “generate_blacklist.sh” somewhere. There really should be a “/usr/local…” structure for this, but I just put it in “/root/rpz” for now.
#!/bin/bash
blacklistfile="/tmp/kresd/blacklist.rpz"
cat > $blacklistfile <<EOF
$TTL 60
@ IN SOA localhost. root.localhost. (
2 ; serial
3H ; refresh
1H ; retry
1W ; expiry
1H) ; minimum
IN NS localhost.
EOF
for host in wget --quiet -O - 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=1&startdate%5Bday%5D=&startdate%5Bmonth%5D=&startdate%5Byear%5D=&mimetype=plaintext'; do
echo “$host CNAME .” >> $blacklistfile
done
/etc/init.d/kresd reload
The “blacklistfile” must match the location listed in part 1. If there’s just a “blacklist.rpz”, then “blacklistfile” must be in “/tmp/kresd”. Otherwise specify the appropriate path. Also “/etc/init.d/kresd reload” is required since a file is being changed and kresd needs to reload the file. Don’t forget to make “generate_blacklist.sh” executable.
You want to make sure your blacklist file is up to date. Create a cron job for this. crontab -e
and add a line like the following: 6 1 * * 3 /root/rpz/generate_blacklist.sh
Please make sure you change the time you get the file. We don’t want to be bombarding the site with requests at the same time. A cron line looks like:
* * * * command to be executed
| | | | |
| | | | ±---- day of week (0 - 6) (Sunday=0)
| | | ±------ month (1 - 12)
| | ±-------- day of month (1 - 31)
| ±---------- hour (0 - 23)
±------------ min (0 - 59)
(from http://www.adminschoice.com/crontab-quick-reference).
I’m happy to explain the script to any who are interested. After all, you shouldn’t just be running a script you don’t understand on your router.
@jklaas thanks for the quick/simple guide…works just fine, but still does not filter enough stuff out.
Do you have any particular source in mind?
The most important thing for me is to block youtube adds (on my tv) and this yoyo list is not really doing that.
For the record, the new update that’s supposed to come soon will add support for a custom configuration file for Knot which wlll be read along with the generated one. (I don’t work for the team, but I backported some of their patches locally to get local host resolution working).
I don’t use Youtube much but there are a lot of ad blocking lists available you could try. You should look at the list supplied with https://github.com/notracking/hosts-blocklists to get ideas for sources.
My problem has been when have created a larger rpz file and used it, then blocking fails altogether. I don’t know if this is related to cache size or something else in my file, but this variable is set through a confusing automated process in init.d.
I keep messing with this and hope the update improves the process. It would be nice if the documentation was clearer too.
I also realized if the router reboots it wont find “/tmp/kresd/blacklist.rpz”. It might be better to specify a permanent location for the blacklist.rpz file, someplace not on the local storage that can stand to be written to a lot (depending on how much you udpate).
I just got through with fixing all the “updates”. One nice thing is a new entry in the config resolver 'kresd'
section, there’s a new entry option include_config '/tmp/kresd.custom.conf'
I wanted it to be persistent so I changed it to option include_config '/etc/kresd/custom.conf'
and put policy.add(policy.rpz(policy.DENY, 'blacklist.rpz'))
in there along with my other policy.add lines and they show up in my /tmp/kresd.config file. Still no way to edit via luci as far as I can tell, but it’s a step forward.
and the aforementioned one from yoyo. It ends up with about 17700 entries after duplicates are removed. I am interested in removing most ads but I’m more interested in removing links to malware.
I can share my updated script if anyone is interested.
I’m using the rpz file option. Using dig returns a “blocked” reply which is essentially an NXDOMAIN. If you know your way around setting up the hosts file it shouldn’t be difficult to modify the script.
It looks like I can’t upload a non-image file. Perhaps that can be adjusted. Meanwhile:
have you noticed any performance hit with that large a rpz file? i’m seeing some timeouts on the first query but it does immediately resolve on the second try.
I’m not sure. Sometimes a browser on a phone or other client is a little slow getting a site, but using dig on the command line works every time.
I also have turned forwarding off since my ISP likes to mess with my DNS “for security reason” (so they can try to sell things to me). Still too many ads getting through. I’m debating adding a “*” to the beginning of all entries in the list so it gets all the subdomains as well, but that will be detrimental without further editing.