My Ad/Malware block script

Post new topic   Reply to topic    DD-WRT Forum Index -> Advanced Networking
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1444
Location: Appalachian mountains, USA

PostPosted: Fri Jan 10, 2020 18:40    Post subject: Reply with quote
croweater wrote:
Is there a way to run this only on certain SSIDs? I ask because I prefer to not run this on my Guest wireless network.

The only easy way would be to bypass the router's own DNS system completely for the guest network. The procedure is not obvious, but it is not difficult. Suppose your guest network is unbridged on virtual wifi interface ath0.1 In GUI>Wireless>BasicSettings in the section where ath0.1 is created, under Advanced Settings (check the box) at the bottom, the IP Address of the subnet gateway is specified as, for example, 192.168.2.1, and just above it you can set an Optional DNS Target. If you set that DNS target to 192.168.2.1, DNS service for ath0.1 will be provided using whatever DNS we have set up for the router as a whole, so this is not what you are after. Instead you can set it to a different public DNS server like 1.1.1.1 for Cloudflare DNS, in which case DNS traffic for ath0.1 (only) would go to Cloudflare. If your guest network is on a bridge instead, say br1, then substitute br1 for ath0.1 in the above discussion, and look on the GUI>Setup>Networking page for a Network Configuration br1 section where you can enter the Optional DNS Target of your choice.

Note though that in this case, if you have DNSCrypt or DNS-through-TLS via Unbound set up for the router, DNS for this interface will bypass it. This will be ordinary, unencrypted DNS service. Also, DNS queries for this interface will not go through any vpn you have set up for the interface unless you are able to specify a DNS server with an IP address in the space that is routed through the VPN tunnel. (For AirVPN there's definitely an easy way. Do a browser find on "10.4.0.1" in the how-to linked in my signature below. I believe with NordVPN though you'd need to configure a route specifically for the DNS server. Not sure about other vpn providers.)

One option for a separate DNS system for guests is Adguard DNS, which provides substantial ad and malware blocking of its own. They have a "family" option as well: https://adguard.com/en/adguard-dns/overview.html. If you really don't want ad blocking for guests, look at Quad9 DNS https://quad9.net at 9.9.9.9 to at least get blocking of known malicious websites.

That's the easy way. If you want to do it the hard way in order to have the full flexibility of dnsmasq, see @eibgrad's long post on how to run two dnsmasqs so that a subnet can be given different DNS handling: Mon Jan 18, 2016 04:43 UTC post (shows here as Jan 17 late the night before, because of local time) in thread https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=288986.

_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Sponsor
croweater
DD-WRT User


Joined: 28 Dec 2019
Posts: 59

PostPosted: Fri Jan 10, 2020 20:51    Post subject: Reply with quote
Surpriseditworks,

You are a wealth of knowledge. I have an unbridged guest network. I will attempt to follow your instructions, and report back.
croweater
DD-WRT User


Joined: 28 Dec 2019
Posts: 59

PostPosted: Sat Jan 11, 2020 20:24    Post subject: Reply with quote
I must admit, after playing with adguard I am happy with the results. I think I will dispense with trying to setup the start script.

I still need to figure out how to setup my Guest network without using adguard though. I run an AirBnB and don't want my guests subjected to my filtering preferences.
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1444
Location: Appalachian mountains, USA

PostPosted: Sat Jan 11, 2020 20:53    Post subject: Reply with quote
croweater wrote:
I must admit, after playing with adguard I am happy with the results. I think I will dispense with trying to setup the start script.

I still need to figure out how to setup my Guest network without using adguard though. I run an AirBnB and don't want my guests subjected to my filtering preferences.

If it were me, as I mentioned, I'd probably either give them Quad9 9.9.9.9 for only malware filtering (and no significant logging: https://quad9.net/policy/), which conceivably might lower your liability risk, or Cloudflare 1.1.1.1 for no filtering at all (logging for four hours or so: https://blog.cloudflare.com/what-cloudflare-logs/). They're both big, fast, well known and well regarded, have servers all over, etc. I'm definitely no expert on DNS-server choices, though. Plenty of more well-informed opinion (and plenty of nonsense also) online.

FWIW, since you are exploring the smaller operation Adguard DNS more closely, my understanding is that they are a small Cyprus company with a mostly eastern European engineering staff. Some online say they are Russian and are instantly skeptical of them for that reason. Seems paranoid to me, but what do I know? It is certainly true that one really needs to trust one's DNS provider, as a malicious one could cause a lot of grief. I believe that when Adguard DNS servers (there are a fair number, worldwide) receive a DNS query, they NAT it and forward it to other servers like Cloudflare or Google DNS, etc. When they receive an IP address in reply, they screen that IP address against their giant ad and malware list and forward either NXDOMAIN (no such domain) or the IP address back to the enquiring computer. Perhaps this multistep process is why their response times are a bit longer than those of some others.

_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1444
Location: Appalachian mountains, USA

PostPosted: Sat Jan 11, 2020 21:08    Post subject: Reply with quote
By the way folks, I've posted an enhanced version of our adblocker in a thread on blocking youtube ads at https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=322606&start=36. The enhanced blocker lets you block just what we have here, or block youtube ads, or use some block lists that are more aggressive than what we use here, or block any combination of those depending on how you configure it. If you leave out the youtube parts it is very, very close to what we have here. (It just handles the recording of exit codes a wee bit differently and reverses the order of blacklist and whitelist application.)

The youtube ad blocking was quite a challenge (and still doesn't stop 100% of them) and was based on the work of @mkaand in the early posts of that thread. His work was for dd-wrt systems that use Unbound, however, whereas my version is for dnsmasq users. A third approach based on adding large numbers of iptables commands to the firewall is discussed in that thread as well.

Do note that discussion of general ad blocking that goes beyond youtube ad blocking in particular appears to be unwelcome there, so let's keep that in this thread, please.

_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
EnigmaSol
DD-WRT Novice


Joined: 07 Jan 2020
Posts: 5

PostPosted: Mon Jan 13, 2020 17:08    Post subject: Re: Adchoices Ads Reply with quote
[quote="EnigmaSol"]
SurprisedItWorks wrote:
EnigmaSol wrote:
Hi all,

I have copied and saved the script as startup (Thank you all who created/contributed to the script), DNSmasq is enabled but Local DNS is disabled.

The Adchoices ads and google ads are still able to slip through (yahoo.com). Am I doing something wrong (noob)? It seems all the sites I frequent are served by Adchoices/Google ads and there is no solution? even Ad Block plus extension works only sometimes.

Thanks again!

There is no 100% solution, as many ads are embedded now in the material you are trying to view and don't involve separate DNS lookups. All we can do is try to reduce the numbers of ads as best we can!


Not sure if it was the simple instructions on this page or the script, but this script (https://wiki.dd-wrt.com/wiki/index.php/Ad_blocking) very much works for me and stops the adschoices ads that I had been talking about. Thank you nonetheless SurprisedItWorks
EnigmaSol
DD-WRT Novice


Joined: 07 Jan 2020
Posts: 5

PostPosted: Mon Jan 13, 2020 18:09    Post subject: Re: Adchoices Ads Reply with quote
[quote="EnigmaSol"][quote="EnigmaSol"][quote="SurprisedItWorks"]
EnigmaSol wrote:
Hi all,

Not sure if it was the simple instructions on this page or the script, but this script (https://wiki.dd-wrt.com/wiki/index.php/Ad_blocking) very much works for me and stops the adschoices ads that I had been talking about. Thank you nonetheless SurprisedItWorks


Spoke too soon, that script mad the VPN super slow for some reason. Reverting back everything (no startup script, no Cron and no localDNS) fixed the issue. I am puzzled because the VPN tunnel should isolate everything from the local routing environment. Why would it become slow?
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1444
Location: Appalachian mountains, USA

PostPosted: Mon Jan 13, 2020 18:48    Post subject: Re: Adchoices Ads Reply with quote
EnigmaSol wrote:
Not sure if it was the simple instructions on this page or the script, but this script (https://wiki.dd-wrt.com/wiki/index.php/Ad_blocking) very much works for me and stops the adschoices ads that I had been talking about. Thank you nonetheless SurprisedItWorks.

Spoke too soon, that script mad the VPN super slow for some reason. Reverting back everything (no startup script, no Cron and no localDNS) fixed the issue. I am puzzled because the VPN tunnel should isolate everything from the local routing environment. Why would it become slow?

Yes, that is certainly strange! I have no idea why it happened. The simple script there is much like the one here but using one blocklist, no blacklist/whitelist, less logging, fewer safety checks, an no fancy youtube maneuvers. But really the idea is the same. Nothing in there really different except the localhost entries at the top of the downloaded file, which my script edits out. Anyone understand why that would matter? It's outside my very limited networking expertise.

_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 35

PostPosted: Tue Jan 14, 2020 1:47    Post subject: Reply with quote
@SurprisedItWorks

i don't get it to work, it's not injecting the host file since i ping everything from host (http://sbc.io/hosts/hosts) and get valid IP

i use one host file for testing and script:
Quote:
#Download targets from 7/8/18 Alozaros post at
#https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=315773
( cd /tmp; touch badhosts; sleep 30
sed 's/\./\\./g;s/.*/\/( |\\.)&$\/d/' <<'ENDWHITE' >badhosts.whitelist
hulu.com
huluad.com
ENDWHITE
awk '{print "0.0.0.0 "$1}' <<'ENDBLACK' >badhosts.blacklist
graph.facebook.com
ENDBLACK
( Download(){ { wget -O $1; EX="$EX $?"; } }; EX=
Download http://sbc.io/hosts/hosts
echo $EX > badhosts.codes ) 2>badhosts.log \
| sed 's/\t/ /g; /^0\.0\.0\.0 /!d; s/ *\#.*$//; s/\r//' \
| sort -u - badhosts.blacklist | sed -Ef badhosts.whitelist > badhosts
echo $(wc -l badhosts | awk '{print $2,"has",$1,"lines,"}') \
'exit codes' $(cat badhosts.codes) \
| logger -t "startup-code adblocker" ) && killall -HUP dnsmasq &


i got
Quote:
BusyBox v1.31.0 (2019-08-06 05:01:39 +04) built-in shell (ash)
root@DD-WRT:~# PrintIt(){ echo Printing $1 ;}
root@DD-WRT:~# PrintIt foo
Printing foo


for your youtube script did use curl -k and used wget and it's not getting the host file, used the host file from seanhandley which seems the only logical and maybe decent one, others are full of invalid host

before testing yours i was using this little functional script from Alozaros:

Quote:
wget -qO /tmp/mvps http://winhelp2002.mvps.org/hosts.txt
killall -HUP dnsmasq
stopservice dnsmasq && startservice dnsmasq


and was working OK.
in cron job i have:
0 12 * * * root /tmp/.rc_startup

i'm using it as start up script
in dnsmasq option i have:
Quote:
addn-hosts=/tmp/mvps
addn-hosts=/tmp/badhosts
address=/1e100.net/0.0.0.0

Local dns option is activated

any ideas??
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 6410
Location: UK, London, just across the river..

PostPosted: Tue Jan 14, 2020 5:43    Post subject: Reply with quote
Rickz wrote:
@SurprisedItWorks

i don't get it to work, it's not injecting the host file since i ping everything from host (http://sbc.io/hosts/hosts) and get valid IP

i use one host file for testing and script:
Quote:
#Download targets from 7/8/18 Alozaros post at
#https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=315773
( cd /tmp; touch badhosts; sleep 30
sed 's/\./\\./g;s/.*/\/( |\\.)&$\/d/' <<'ENDWHITE' >badhosts.whitelist
hulu.com
huluad.com
ENDWHITE
awk '{print "0.0.0.0 "$1}' <<'ENDBLACK' >badhosts.blacklist
graph.facebook.com
ENDBLACK
( Download(){ { wget -O $1; EX="$EX $?"; } }; EX=
Download http://sbc.io/hosts/hosts
echo $EX > badhosts.codes ) 2>badhosts.log \
| sed 's/\t/ /g; /^0\.0\.0\.0 /!d; s/ *\#.*$//; s/\r//' \
| sort -u - badhosts.blacklist | sed -Ef badhosts.whitelist > badhosts
echo $(wc -l badhosts | awk '{print $2,"has",$1,"lines,"}') \
'exit codes' $(cat badhosts.codes) \
| logger -t "startup-code adblocker" ) && killall -HUP dnsmasq &


i got
Quote:
BusyBox v1.31.0 (2019-08-06 05:01:39 +04) built-in shell (ash)
root@DD-WRT:~# PrintIt(){ echo Printing $1 ;}
root@DD-WRT:~# PrintIt foo
Printing foo


for your youtube script did use curl -k and used wget and it's not getting the host file, used the host file from seanhandley which seems the only logical and maybe decent one, others are full of invalid host

before testing yours i was using this little functional script from Alozaros:

Quote:
wget -qO /tmp/mvps http://winhelp2002.mvps.org/hosts.txt
killall -HUP dnsmasq
stopservice dnsmasq && startservice dnsmasq


and was working OK.
in cron job i have:
0 12 * * * root /tmp/.rc_startup

i'm using it as start up script
in dnsmasq option i have:
Quote:
addn-hosts=/tmp/mvps
addn-hosts=/tmp/badhosts
address=/1e100.net/0.0.0.0

Local dns option is activated

any ideas??


yep SurprisedItWorks script works well....the problem with it was it executes 2 times sometimes
it depends on start up times...if you execute it at last its not a problem...
also...curl is not present on all routers so stick to wget...
wget was corrupted on few build ago so stick to the last builds where it was fixed...

that old adblocking script you used was good but lacking of.. some vital stuff like sed the results and ect.

currently i use a short script based on SurprisedItWorks idea, that works for me as intended...

do keep in mind SurprisedItWorks adblocker has better construction, functionality and options to play with..
which make it an excellent work...

_________________
Atheros
TP-Link WR740Nv1 ---DD-WRT 55179 WAP
TP-Link WR1043NDv2 -DD-WRT 55303 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55460 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55460 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55363 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 35

PostPosted: Tue Jan 14, 2020 6:03    Post subject: Reply with quote
@Alozaros @SurprisedItWorks
it looks good work but don't know what i'm missing ?
simple script is working and newone does nothing Sad
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1444
Location: Appalachian mountains, USA

PostPosted: Tue Jan 14, 2020 15:52    Post subject: Reply with quote
Rickz wrote:
@Alozaros @SurprisedItWorks
it looks good work but don't know what i'm missing ?
simple script is working and newone does nothing Sad
Can you share with us some details of what happens? When I'm uncertain how things are with the script, I want to see the results of these:

cd /tmp
ls -l badhosts*
cat badhosts.codes
cat badhosts.log
wc -l badhosts
grep adblocker /var/log/messages


Unless you are really sure already that the trouble is at the Downloading stage, it would also be good, if you don't find the details of these to be private or excessively long, to share the whitelist and blacklist in their final, internal forms:

cat badhosts.whitelist
cat badhosts.blacklist


Also, can you share with us any of the actual Download lines that you've tweaked in any way? Especially include the Download definition, since it sounds like you needed to go with wget. That will give me things I can experiment with here.

And... you have the addn-hosts=/tmp/badhosts line in Additional Dnsmasq Options, right? I've seen two cases already (one was me Embarassed ) in which leaving off that line was the whole problem. Easy to lose sight of the mouse while wrestling with the elephant.

_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
EnigmaSol
DD-WRT Novice


Joined: 07 Jan 2020
Posts: 5

PostPosted: Tue Jan 14, 2020 15:58    Post subject: Step by Step? Reply with quote
Rickz wrote:
@Alozaros @SurprisedItWorks
it looks good work but don't know what i'm missing ?
simple script is working and newone does nothing Sad


@SurprisedItWorks @Alozaros

May be a post listing step by step (e.g. Enable these options, put these scripts here, save/reboot etc.) and options that work for all routers (e.g. wget for all), would enable better adoption and functionality for the script.

As a model, I, as a noob, found it extremely easy to implement these instructions: https://wiki.dd-wrt.com/wiki/index.php/Ad_blocking (sigh only if it didn't mess with my VPN)

Your patience and effort in creating such a post, I am sure, will be highly appreciated, by dd-wrt users such as myself. Thank you again!
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 35

PostPosted: Wed Jan 15, 2020 15:03    Post subject: Reply with quote
@SurprisedItWorks

i will do it later and report back, i have to enable telnet again and paste back your script,

the posted script used only was substituted the cult for get

i was having this issue:
Quote:
*** Can't find server name for address 192.168.1.1: Non-existent domain
*** Default servers are not available
Server: UnKnown
Address: 192.168.1.1

Name: googleads.g.doubleclick.net
Address: 0.0.0.0


so i assigned a host name/ip to ddwrt in dnsmasq option
Quote:
address=/DD-WRT.local/192.168.1.1
ptr-record=1.1.168.192.in-addr.arpa,"DD-WRT.local"


don't know if that has to do something for your script but i will reply back results,

now i ping a host and get normal reply:
>nslookup tpc.googlesyndication.com
Server: DD-WRT.local
Address: 192.168.1.1

Name: tpc.googlesyndication.com
Address: 0.0.0.0

for youtube case, if we find a way to block these site directories in ddwrt without blocking youtube itself then ads will be history,
these:

Quote:
https://www.youtube.com/pagead/
https://www.youtube.com/api/stats/ads?
https://www.youtube.com/pagead/adview?
https://www.youtube.com/api/stats/qoe?


do you think is possible with script to block those ?
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1444
Location: Appalachian mountains, USA

PostPosted: Wed Jan 15, 2020 16:07    Post subject: Reply with quote
Rickz wrote:
@SurprisedItWorks

i will do it later and report back, i have to enable telnet again and paste back your script,

the posted script used only was substituted the cult for get

i was having this issue:
Quote:
*** Can't find server name for address 192.168.1.1: Non-existent domain
*** Default servers are not available
Server: UnKnown
Address: 192.168.1.1

Name: googleads.g.doubleclick.net
Address: 0.0.0.0

I don't see a problem with that, actually. This looks like a reasonable output for nslookup googleads.g.doubleclick.net, assuming that domain is in one of the block lists (though on my laptop, nslookup doesn't even look for a name for the DNS server). It looks like the block lists are working. (But still lets try those tests I mentioned before, as seeing the results of those are good confidence builders. I always do those here when setting this up on a new router.)

If your router is named RouterName, what happens if you do nslookup RouterName? Does it return 192.168.1.1? That might be a test of your local-DNS setup, but it would have nothing to do with adblocking. Also, try nslookup 192.168.1.1 and see what you get. Here I get 1.1.168.192.in-addr.arpa name = TheRouterName where TheRouterName is the name I gave the router in GUI>Setup>BasicSetup.
Quote:
so i assigned a host name/ip to ddwrt in dnsmasq option
Quote:
address=/DD-WRT.local/192.168.1.1
ptr-record=1.1.168.192.in-addr.arpa,"DD-WRT.local"
don't know if that has to do something for your script but i will reply back results,

now i ping a host and get normal reply:
>nslookup tpc.googlesyndication.com
Server: DD-WRT.local
Address: 192.168.1.1

Name: tpc.googlesyndication.com
Address: 0.0.0.0

for youtube case, if we find a way to block these site directories in ddwrt without blocking youtube itself then ads will be history,
these:
Quote:
https://www.youtube.com/pagead/
https://www.youtube.com/api/stats/ads?
https://www.youtube.com/pagead/adview?
https://www.youtube.com/api/stats/qoe?


do you think is possible with script to block those ?

I can't see how we can do it with these tools. We are working with the DNS system, which only has www.youtube.com to work with. That's all the DNS system is given to look up. It never sees the rest of those URLs.

Search the forums though for how to block access to specific URLs (as opposed to domain names or IP addresses). It's a common dd-wrt topic, and perhaps there is a way unrelated to the DNS system.

_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Goto page Previous  1, 2, 3, 4, 5, 6  Next Display posts from previous:    Page 5 of 6
Post new topic   Reply to topic    DD-WRT Forum Index -> Advanced Networking All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum