Adblock for DD-WRT (version 2)

Post new topic   Reply to topic    DD-WRT Forum Index -> General Questions
Goto page 1, 2, 3 ... 21, 22, 23  Next
Author Message
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Mon Feb 20, 2017 11:33    Post subject: Adblock for DD-WRT (version 2) Reply with quote
Adblock for DD-WRT
--------------------------------
Script for blocking ads on DD-WRT

Developed and tested on: NETGEAR R8500 / R7800
Hardware requirements: Minimum 4GB USB drive.
Partitioned, formatted, and mounted as swap (256-512MB), /jffs (1-2GB), and /opt (1-2GB)

NOTE: DO NOT try to run this script on your internal jffs.

Software requirements: DD-WRT , cURL (with entware if unavailable with your firmware)

Installation:
-------------


    1.
    Code:
    cd /jffs/dnsmasq


    2.
    Code:
    curl -o adblock.sh adblock.sh


    or
    Code:
    wget -O adblock.sh adblock.sh


    2.
    Code:
    chmod +x adblock.sh


    3. Use --update or -u command line option to update to the latest version.



Settings:
---------

1. Disable internal flash (JFFS2) if it's enabled.





2. Enable DNSMasq and local DNS for LAN and WAN. Add these lines under the additional options section.

Code:
conf-file=/jffs/dnsmasq/mpdomains
addn-hosts=/jffs/dnsmasq/mphosts




3. Enter additional options for dnsmasq if required, for example:

Code:
domain-needed
bogus-priv


4. Under Administration -> Cron, enter this or choose your own schedule (this executes this script on Mondays and Thursdays at 0600 hours):

Code:
0 6 * * 1,4 root /jffs/dnsmasq/adblock.sh




5. Reboot


Usage:
------



1. Add your custom blacklist or whitelist domains to myblacklist and mywhitelist files respectively.

2. Do not save anything in blacklist or whitelist files as they will be overwritten every
time the script is executed.

--


Changelog:

20200201:
MAJOR UPGRADE. Now supports remote operations as well. If your router is not powerful enough to run this script, you can now install this on a Linux system/Raspberry Pi and use the --remote=(router ip address) option to upload generated lists via SSH. Note that this mode is interactive and you will asked for your router's root password (not saved anywhere). To automate this, you can configure router with ssh keys.

Automatic fallback to wget if curl is unavailable. Not recommended though as BusyBox wget is severely limited in functionality and scope.

This release also supports DEBUG output if you face any errors. Check command line options.

adbhostgen (old version) is still functional and will remain so for a while. No new features will be added to this, so I recommend that you upgrade and change "adbhostgen.sh" to "adblock.sh" on all the relevant config pages.

20180331:
POINT RELEASE with all the bugfixes and new features from the alpha releases below.

20180329a2:
No more "bad name at line n" errors.

20180328a2:
Changed default ad destination IP from 0.0.0.0 to 0.1.2.3

20180327a1:
New options:
-i <ip-address>
--ip=<ip-address>
where <ip-address> is an IPv4 address of your own personal ad hell.

20180326a1:
New options:
-f, blocks Facebook.com and services
-F, blocks Facebook, Instagram, and WhatsApp

20180324a6:
Significant changes and bugfixes. The script should behave better as a cron job now.
A new option -q or --quiet suppresses output to stdout/console, yet retains log entires.
Considering that many users are still on ancient DD-WRT versions with outdated and/or
expired cURL certificates and exotic configuration that often doesn't play well with
this script, it now uses insecure cURL downloads (-k) as default mode of operation.
For experienced users, the new --secure command line parameter restores the secure
TLS/SSL communication with servers. After thinking long and hard and working with
users on solving their problems, offering sane defaults with switches to enable
advanced functionality seems like a good tradeoff. For now.


Website: https://adblock.sh
Github: https://github.com/m-parashar/adblock
Old version (adbhostgen): https://github.com/m-parashar/adbhostgen


See this post for detailed instructions.

Things to whitelist (mywhitelist file or adblock.sh -w <domainname> --wl=<domainname>)

* https://discourse.pi-hole.net/t/commonly-whitelisted-domains/212
* https://github.com/anudeepND/whitelist/blob/master/domains/whitelist.txt

_________________

Routers
NETGEAR R7800 x2: DD-WRT v3.0-r42847 std (04/06/20)
NETGEAR R8500: DD-WRT v3.0-r41517 std (11/11/19)

Scripts
Adblock.sh on DD-WRT (v2) -- Github | Website
Ad-blocking on DD-WRT (v1) -- Github


Last edited by Yamaraj on Sat Feb 01, 2020 13:28; edited 66 times in total
Sponsor
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Tue Feb 21, 2017 13:19    Post subject: Reply with quote
rev19: Cleaned up, added custom hosts.
rev22: Significant updates.
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Sat Feb 25, 2017 6:52    Post subject: Reply with quote
rev25: Speed optimizations. The script now gracefully handles blank lines and trailing whitespace in custom_hosts and whitelist.

rev26: Full path introduced to avoid cron failure. [2017/03/04]
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Fri Mar 17, 2017 9:50    Post subject: Reply with quote
rev28: script gracefully handles network downtime failures and DNS resolution issues. Automatically sorts whitelist and custom_hosts, and creates backup of the last hosts file.

rev29: whitelist updated. [2017/03/19]
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Thu May 04, 2017 10:17    Post subject: Reply with quote
rev30: whitelist updated. "custom_hosts" renamed to "blacklist" to avoid ambiguity.

Rock solid performance so far. Any feedback is welcome.
tweakradje
DD-WRT Novice


Joined: 25 Sep 2008
Posts: 34

PostPosted: Fri May 05, 2017 21:23    Post subject: Why so complicated? Reply with quote
I only use these 3 lines in startup for blocking:


sleep 20

# get hosts file with unix EOL
wget -qO- http://www.mvps.org/winhelp2002/hosts.txt|grep "^0.0.0.0" >> /tmp/hosts

# re-read hosts file
killall -HUP dnsmasq

together with dnsmasq as local dns and dhcp. You can use any inet hosts file you like of course. Disadvantage of hosts is it need to be hosts not domains. As extra I use the dnsmasq option field. I list the domains like:


domain-needed
bogus-priv
no-negcache
address=/2mdn.net/0.0.0.0
address=/2o7.net/0.0.0.0
address=/360yield.com/0.0.0.0
address=/ad-center.com/0.0.0.0
address=/ad-inside.com/0.0.0.0
address=/adbrite.com/0.0.0.0
address=/adinfuse.com/0.0.0.0
address=/admob.com/0.0.0.0
address=/admob.mobi/0.0.0.0
...


Can you explain what your method does more?

Cheers

_________________
Netgear R6400
Linksys wrt54G v5
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Mon May 08, 2017 19:02    Post subject: Reply with quote
1. The script merges 18 different hosts files into one for aggressive ad-blocking. This is obviously a custom solution since I use the same hosts file across different systems/networks.

2. Includes a whitelist/blacklist solution for custom needs.

3. Helps you start right away with minimum configuration. It's almost completely automated and portable.

4. Creates a backup of the last hosts file, just in case.

5. It's the best solution for my needs so far.

The script is actually quite straightforward and simple.

Best.
fatalhalt
DD-WRT Novice


Joined: 29 Oct 2015
Posts: 39

PostPosted: Tue May 09, 2017 3:10    Post subject: Reply with quote
Seems like a solid block list. I was only aware of somewhocares.org blocklist, but seeing your script there are a lot more lists out there.

You could silence the first curl command as well with '-s' (the "01. Mother of All Ad Blocks list")

11 megabyte text file is a pretty big list Surprised .
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Sat May 13, 2017 17:09    Post subject: Reply with quote
Thank you!

rev31: Renamed "createmphosts.sh" to "adbhostgen.sh", added silent option to the #01 list, as suggested by fatalhalt, updated the original post with fresh instructions and images.
genlish
DD-WRT Novice


Joined: 14 Jun 2017
Posts: 3

PostPosted: Wed Jun 21, 2017 11:27    Post subject: Re: Ad-blocking on Netgear R8500 Reply with quote
Yamaraj wrote:
System: Netgear R8500

Firmware: Kong's DD-WRT v3.0-r31870M (04/16/17)

Additional configuration: OpenVPN client and an 8GB pendrive partitioned and formatted as swap, /jffs, and /opt running Kong's optware.

Background: Needed a more elegant and basic setup than dealing with the nuances of OpenVPN and Privoxy enabled on the same router. Repurposing RPi3 had something to do with it, too.

Solution: hosts file & dnsmasq

Link: https://gist.github.com/m-parashar/ee38454c27f7a4f4e4ab28249a834ccc

Instructions:

1. Copy/create adbhostgen.sh in /jffs/dnsmasq directory. chmod +x it.

https://gist.github.com/m-parashar/ee38454c27f7a4f4e4ab28249a834ccc/raw/5968ae10fc16a0ea411c601e677e8118032ad4d9/adbhostgen.sh

2. Create/download the file "whitelist" without quotes in /jffs/dnsmasq and populate it with the domains you do not want blocked. ̶E̶n̶s̶u̶r̶e̶ ̶t̶h̶e̶r̶e̶'̶s̶ ̶n̶o̶ ̶t̶r̶a̶i̶l̶i̶n̶g̶ ̶n̶e̶w̶l̶i̶n̶e̶/̶c̶r̶.̶ [rev25]

https://gist.github.com/m-parashar/ee38454c27f7a4f4e4ab28249a834ccc/raw/5968ae10fc16a0ea411c601e677e8118032ad4d9/whitelist

3. Create/download file "blacklist" and populate it with the domains you want to block. [rev30]

https://gist.github.com/m-parashar/ee38454c27f7a4f4e4ab28249a834ccc/raw/5968ae10fc16a0ea411c601e677e8118032ad4d9/blacklist

4. Execute adbhostgen.sh in /jffs/dnsmasq to generate the hosts file. By default the hosts file is quite aggressive and approx 10MB in size, which is manageable enough for Netgear R8500. Comment out the hosts repos as you see fit.

5. Enable DNSMasq and local DNS for LAN and WAN

6. Enter this into the additional options field
Code:
addn-hosts=/jffs/dnsmasq/mphosts
domain-needed
bogus-priv




7. Under Administration -> Cron, enter this or choose your own schedule:
Code:
0 6 * * 1,4 root /jffs/dnsmasq/adbhostgen.sh
30 6 * * 1,4 root restart_dns




8. Reboot

Should work on R7000, R8000 and other >128MB RAM routers too. Any feedback is welcome.



It`s any way to make one computer (specified ip) or a IP range to be excluded from the filter? Like using iptables on router or using a different dns on the computer?
I've tried the solution of chjohans:
http://www.dd-wrt.com/phpBB2/viewtopic.php?p=740330
"Add the following to your "Startup" script:
dnsmasq -S 208.67.222.222 -R -i br0 -p 1054

This starts an dnsmasq instance on port 1054 using the OpenDNS DNS server (208.67.222.222)

Add the following to your "Firewall" script:
iptables -t nat -A PREROUTING -p tcp -i br0 -s 192.168.1.100 --dport 53 -j DNAT --to $(nvram get lan_ipaddr):1054
iptables -t nat -A PREROUTING -p udp -i br0 -s 192.168.1.100 --dport 53 -j DNAT --to $(nvram get lan_ipaddr):1054
"

but the router prioritizes his own host file instead skiping it.
does anyone have the solution?
Thanks.
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Sat Nov 25, 2017 6:36    Post subject: Re: Ad-blocking on Netgear R8500 Reply with quote
genlish wrote:
It`s any way to make one computer (specified ip) or a IP range to be excluded from the filter? Like using iptables on router or using a different dns on the computer?
I've tried the solution of chjohans:
http://www.dd-wrt.com/phpBB2/viewtopic.php?p=740330
"Add the following to your "Startup" script:
dnsmasq -S 208.67.222.222 -R -i br0 -p 1054

This starts an dnsmasq instance on port 1054 using the OpenDNS DNS server (208.67.222.222)

Add the following to your "Firewall" script:
iptables -t nat -A PREROUTING -p tcp -i br0 -s 192.168.1.100 --dport 53 -j DNAT --to $(nvram get lan_ipaddr):1054
iptables -t nat -A PREROUTING -p udp -i br0 -s 192.168.1.100 --dport 53 -j DNAT --to $(nvram get lan_ipaddr):1054
"

but the router prioritizes his own host file instead skiping it.
does anyone have the solution?
Thanks.


You will probably need to install optware and use another instance of dnsmasq to setup a separate DNS for those IPs. I haven't tried that yet. Anyone?


[rev33]: added mpdomains for additional protection. Lists updated.

SIZE:
mpdomains: ~2MiB
mphosts: ~14MiB


Status:




_________________

Routers
NETGEAR R7800 x2: DD-WRT v3.0-r42847 std (04/06/20)
NETGEAR R8500: DD-WRT v3.0-r41517 std (11/11/19)

Scripts
Adblock.sh on DD-WRT (v2) -- Github | Website
Ad-blocking on DD-WRT (v1) -- Github


Last edited by Yamaraj on Sat Nov 25, 2017 12:28; edited 1 time in total
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 6388
Location: UK, London, just across the river..

PostPosted: Sat Nov 25, 2017 8:06    Post subject: Reply with quote
personally i find privoxy as a little slow and buggy, although its working it add's a latency on browsing and sometimes blocks not listed sites ...
_________________
Atheros
TP-Link WR740Nv1 ---DD-WRT 55179 WAP
TP-Link WR1043NDv2 -DD-WRT 55303 Gateway/DoT,Forced DNS,AP Isolation,Ad-Block,Firewall
TP-Link WR1043NDv2 -DD-WRT 55303 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear R7800 --DD-WRT 55363 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55363 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55363 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Sat Nov 25, 2017 11:02    Post subject: Reply with quote
Alozaros wrote:
personally i find privoxy as a little slow and buggy, although its working it add's a latency on browsing and sometimes blocks not listed sites ...

Yeah, I played with it for a while and then settled for this solution for the same reasons. This script may not be perfect, but it can be tailored to do exactly what I want.

_________________

Routers
NETGEAR R7800 x2: DD-WRT v3.0-r42847 std (04/06/20)
NETGEAR R8500: DD-WRT v3.0-r41517 std (11/11/19)

Scripts
Adblock.sh on DD-WRT (v2) -- Github | Website
Ad-blocking on DD-WRT (v1) -- Github
Yamaraj
DD-WRT User


Joined: 25 Jan 2017
Posts: 110

PostPosted: Thu Nov 30, 2017 17:15    Post subject: Reply with quote
rev36: Added a secure version of the script which downloads cURL CA cert directly from the author's server and then uses secure transmission for downloading lists. Whitelist now applies to the domain list as well as hosts list.
_________________

Routers
NETGEAR R7800 x2: DD-WRT v3.0-r42847 std (04/06/20)
NETGEAR R8500: DD-WRT v3.0-r41517 std (11/11/19)

Scripts
Adblock.sh on DD-WRT (v2) -- Github | Website
Ad-blocking on DD-WRT (v1) -- Github
labo
DD-WRT Guru


Joined: 30 Jan 2015
Posts: 676
Location: Texas, USA

PostPosted: Tue Dec 05, 2017 16:20    Post subject: Reply with quote
Yamaraj,
I tried this and works great!. But seems bit aggressive list. Do you have a lite version of the blacklist?
With this current list I see issues with,
facebook and some deal sites such as Slickdeals.com and some legitimate redirect links.

_________________
ASUS GT-BE98 PRO Main: Fiber 5gbps up/down
ASUS AXE16000: AI Mesh node
2 X ASUS RT-AX89X: AI Mesh nodes
QNAP QSW-1208-8C 12-Port 10GbE Switch
XS712T ProSafe 12-Port 10GbE Switch
3 X R9000 DD-WRT Mesh
Goto page 1, 2, 3 ... 21, 22, 23  Next Display posts from previous:    Page 1 of 23
Post new topic   Reply to topic    DD-WRT Forum Index -> General Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum