Adblock package release for turris omnia


#21

that’s only a busybox builtin - at least only a compile/build option.


#22

There’s also the one shipped by coreutils available.


#23

Hi dibdot,

thank you very much for your manual how to set up adblock on Turris Omnia!

Unfortunately, when I followed your steps, it was not very straightforward for me – I’m not very experienced OpenWRT user though I have non-trivial Linux administration experience.That’s why I would like to share my step-by-step guide for other users. Feel free to update your first post if you like it.

So, here we go:

  1. [optional] Change current dir to /tmp, so you do not write to the Turris’ (precious) internal Flash memory. But you can use any dir you like…

    [turris]:/# cd /tmp

  2. Download AdBlock packages: adblock and luci-app-adblock. Find the latest version on these pages: adblock package and luci-app-adblock. Use proper links in wget commands below:

    [turris]:/tmp# wget https://downloads.lede-project.org/snapshots/packages/x86_64/packages/adblock_3.0.2-1_all.ipk
    [turris]:/tmp# wget https://downloads.lede-project.org/snapshots/packages/x86_64/luci/luci-app-adblock_git-17.257.70215-07593d2-1_all.ipk

  3. Install both packages:

    [turris]:/tmp# opkg install adblock_3.0.2-1_all.ipk
    [turris]:/tmp# opkg install luci-app-adblock_git-17.257.70215-07593d2-1_all.ipk

  4. Enable module. First of all, check the present state by running:

    [turris]:/tmp# /etc/init.d/adblock enabled; echo $?
    1

Please note that the script returned 1, which means there was some ‘error’ – this mean, the module is NOT enabled.

Let’s enable it and check the state again:

[turris]:/tmp# /etc/init.d/adblock enable
[turris]:/tmp# /etc/init.d/adblock enabled; echo $?
0

!!! Beware !!! Even though the /etc/init.d/adblock enable script (in version 3.0.2) returns 1 (error code), it seems that it does its job. You can verify it by running /etc/init.d/adblock enabled; echo $?. If the result is “0” (no error), the module is enabled.

  1. now, edit the kresd config file:

    vim /etc/config/resolver

and find the section config resolver 'kresd'. Add the following line:

list rpz_file "/etc/kresd/adb_list.overall"

just after the config resolver 'kresd' line. So the config may looks like this:

config resolver 'common'
    ...  // some global preferences
    .
    .
    .

config resolver 'kresd'
    list rpz_file "/etc/kresd/adb_list.overall"        
    option rundir '/tmp/kresd'
    option log_stderr '1'
    option log_stdout '1'
    option forks '1'
    option dynamic_domains '0'

config resolver 'unbound'
    option outgoing_range '60'
.
.
.
  1. Now, go to the LuCI web interface, Services -> Adblock page and make the following changes from the “default” pre-set:
  1. make sure “Enable Adblock” is checked
  • change “DNS Backend” to “kresd”
  • change “Startup Trigger” to “timed”
  • check additional Blocklist Sources, e.g.,: adaway, adguard, blacklist, disconnect, ransomware, winspy, yoyo, zeus
  • you can check Verbose Debug Logging (if you would like to check everything is working fine)
  • check Force Local DNS
  • finally, hit “Save & Apply” button
  1. Now, verify your setup:
  1. Check /etc/init.d/adblock status:

     [turris]:/tmp# /etc/init.d/adblock status
     ::: adblock runtime information
       + adblock_status  : enabled
       + adblock_version : 3.0.2
       + blocked_domains : 82457
       + fetch_utility   : wget (built-in)
       + dns_backend     : kresd (/etc/kresd)
       + last_rundate    : 17.09.2017 01:38:35
       + system_release  : Turris Omnia, OpenWrt omnia 15.05/3.8
    
  2. view log file:

     [turris]:/tmp# cat /var/log/messages | grep "adblock" 
    
  3. check the existence of new definitions-file for kresd:

     [turris]:/tmp# ls -lh /etc/kresd/adb_list.overall
      -rw-r--r--    1 root     root        1.0M Sep 16 23:51 /etc/kresd/adb_list.overall
    
  4. To be sure, it is better to restart the resolver now:

     [turris]:/tmp# /etc/init.d/resolver restart
    
  5. let’s try to resolve some Ad-domain:

     [turris]:/tmp# nslookup securepubads.g.doubleclick.net localhost
     Server:    127.0.0.1
     Address 1: 127.0.0.1 localhost
    
     nslookup: can't resolve 'securepubads.g.doubleclick.net': Name does not resolve
    

If you get “Name does not resolve”, the AdBlocking is working correctly.

Well, and that’s all! Happy ad-blocking! :slight_smile:


Network-level ad blocking
#24

EDIT: I resolved this – it’s not a bug, it is a “feature”…

@dibdot I believe there is some bug in processing manual blacklist.

Steps to reproduce:

  1. AdBlock web administration

  2. on the Overview tab, uncheck all Blocklist Sources

  3. hit Save & Apply

  4. go to the console and check, that the /etc/kresd/adb_list.overall file is empty (which is expected):

    wc -l /etc/kresd/adb_list.overall

    0 /etc/kresd/adb_list.overall
    
  5. then go back to the AdBlock web administration -> Advanced -> Edit Blacklis and add any records, e.g.:

    u.scdn.co
    *.u.scdn.co

  6. hit Save. You can check, that the content of /etc/adblock/adblock.blacklist file is correctly saved

  7. go back to Overview tab and hit Save & Apply (so that AdBlock will restart).

  8. check the /var/log/messages, that adblock restarted successfully:

    notice adblock-[3.0.2]: info : block list with overall 0 domains loaded successfully (Turris Omnia, OpenWrt omnia 15.05/3.8)

The problem: file /etc/kresd/adb_list.overall is still empty:

Expected behavior: file /etc/kresd/adb_list.overall would contain those 2 blacklisted domains.

Am I doing something wrong? Or is it a bug that could be fixed in the package?

EDIT: It’s not a bug – it was my fault :frowning:
The manual blacklist works only if you select blacklist among other Blocklist Sources – which is not very intuitive, but it is how it is.


#25

Well, the blacklist is a normal source, therefore you can enable/disable it like the others … regarding your domain input please note, that (as stated in the LuCI frontend) wildcards, regex or other weird encodings are not allowed. To block your example domain it’s enough to add the one liner u.scdn.co.

Thanks for your comprehensive installation instructions, I’ll add a link in the first post!


#26

I still see ads in youtube videos. Any help please?


#27

Please remember that you block always on (sub-)domain level (and not on page element level!) - the same applies to other projects like pihole. For remaining ads on advertising platforms like youtube, facebook etc. it’s likely that you need additional browser plugins which are hiding ads on DOM level, e.g. injecting JavaScript to skip the YouTube ads … anyway you’ll never get 100%.


#28

I tought I can take care of youtube on domain level. Never mind


#29

No that will not work.
< offtopic> On computer OS level uBlock Origin and on Android YouTube AdAway (Xposed) < /offtopic>


#32

Is Manual/backup mode needed? (Adblock package release for turris omnia)

I found, that forward_upstream '0' isnt needed, I have forwarding on and test went OK (nslookup securepubads.g.doubleclick.net localhost).

So now I am wondering about this backup method, if there is needed because lack of sha256sum, and how it is with updating domains… I dont see “Manual refresh” button in luci-app or something…


#33

Correct; forwarding should make no difference to the blocking. (As long as the forwarded-to servers are reasonable, but you need that even without any blocking.)


#34

correct, I fixed this in the first post and in the online doc for current release 3.1.

Backups will be used in case of download errors during adblock processing (e.g. a blocklist site is temporary not available) or during startup in backup mode. To refresh you blocklists setup an appropriate cron job (see online doc) or hit “Save & Apply” in LuCI.


#35

not very nice to " hijack" a software support thread for the adblock package with such useless poll. Open a separate forum thread for that, please.


#36

What does that forward_upstream '0' within /etc/config/resolver actually do?
I unfortunately cannot tell whether I disabled that option for adblock or not…


#37

That’s the checkbox “use forwarding” in Foris.


#38

Sorry for that and I apologise but I strongly disagree about uselessness of pool or question. IMHO many beginning users like me do not understand difference between solutions and what to choose


#39

Thanks for moving out your poll … and no worries, ask adblock related questions whenever you like …


#40

i also made a tutorial on how to do this in another, more general thread here:

Hopefully it will be useful for others as well. And FWIW, the aforementioned poll is here:


#41

@turris-admin I can’t no longer edit my first/initial post, that’s a pity for a package support thread - could you change that, please?!

There’s a new adblock update 3.1.1 with the following changes:

  • new function to set/delete options in external uci config files
    • kresd: automated ‘rpz_file’ handling in /etc/config/resolver
    • firewall: automated ‘force_dns’ handling if you enable or disable adblock
  • support sha256sum (default) and md5sum for blocklist comparison & conditional dns restarts
  • cosmetics

It’s no longer required to manually add the rpz_file entry and the md5sum utility is supported to for conditional kresd restarts. :relaxed:


#42

3.1.0

  • caution: config file update required!

Can you be more specific? Running 3.0.3 on Omnia, considering smooth update and thinking about best procedure :slight_smile: (aint see any checksum/restart switches for config section@readme.md)

Thank you for your work!