My Ad/Malware block script

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Advanced Networking
Goto page Previous  1, 2, 3, 4  Next
Author Message
SurprisedItWorks
DD-WRT User


Joined: 04 Aug 2018
Posts: 443
Location: Appalachian mountains, USA

PostPosted: Wed Sep 25, 2019 23:18    Post subject: Reply with quote
So while I am in a fixing mood, and continuing my idiotic tradition of posting new versions way too soon and after minimal testing of changes (a dd-wrt tradition!), here's the latest adblocker code for GUI>Administration>Commands, Startup. Note the first comment re the other GUI edit required.

The main point here is just cleanup for readability. However, there are actual changes here to that earlier hurried kluge of syslog-entry machinery. This version is cleaner. Pointless group-command braces in the Download function have been removed as well. While I've badmouthed cats in pipelines, even in this very thread, I don't actually see a cleaner way here (in the syslog-entry code). So there it lurks.
Code:
#Hosts to block in dnsmasq using Add'l Config: addn-hosts=/tmp/badhosts
#Download targets from 7/8/18 Alozaros post at
#https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=315773
#See https://github.com/StevenBlack/hosts/blob/master/readme.md
#for alternative sbc.io files.  The one below is the first one there.
( cd /tmp; touch badhosts; sleep 30
  sed 's/\./\\./g;s/.*/\/( |\\.)&$\/d/' <<'ENDWHITE' >badhosts.whitelist
hulu.com
huluad.com
ENDWHITE
  awk '{print "0.0.0.0 "$1}' <<'ENDBLACK' >badhosts.blacklist
graph.facebook.com
ads.facebook.com
connect.facebook.net
ENDBLACK
  ( Download(){ curl -kf $1; EX="$EX $?"; }; EX=
    Download http://winhelp2002.mvps.org/hosts.txt
    Download http://sbc.io/hosts/hosts
    Download https://someonewhocares.org/hosts/zero/hosts
    echo $EX > badhosts.codes
  ) 2>badhosts.log \
  | sed 's/\t/ /g; /^0\.0\.0\.0 /!d; s/ *\#.*$//; s/\r//' \
  | sort -u - badhosts.blacklist \
  | sed -Ef badhosts.whitelist > badhosts
  wc -l badhosts \
  | awk '{printf "%s has %d lines, exit codes ",$2,$1}' \
  | cat - badhosts.codes \
  | logger -t "startup-code adblocker"
) && killall -HUP dnsmasq &

Note as before that the sleep may need lengthening for some router/build/config combinations. Try it and check the log entry, looking for 0 0 0 for exit codes (or look at /tmp/badhosts.codes in the CLI) and a decent-length badhosts file. As of today -- it can vary a bit day to day as the files posted at the URLs are edited -- I have 42475 lines. On the first try it's a good idea to look at /tmp/badhosts.log in the CLI to verify that the curls are doing something reasonable.

If you have an old or small build that doesn't have curl, you can replace curl -kf with wget -O - but be careful to include that final hyphen, and note that O is the letter "Oh" and not a zero. You'll likely get nothing but an error from the third Download line, as the wget in dd-wrt does not, by default anyway, handle https sites. You can just delete that third Download line and get by just fine on two, with a slightly smaller badhosts file resulting.

The whitelist entries here allow hulu ads, without which hulu will not stream, and the blacklist entries hamper some of facebook's spying and ad displays. The connect.facebook.net entry will likely break logging into other sites using facebook credentials in addition to hampering their spying. Edit: It appears that blacklisting graph.facebook.com kills facebook messenger, also in addition to hampering their spying.Boo hoo.

Whitelist entries can be either domains, as above, or linux extended regular expressions that expand to domains when each "." is escaped. The latter means the two entries here could be condensed to the one line hulu(ad)?.com or hulu(|ad).com if desired. A whitelisted domain foo.com implicitly whitelists x.foo.com and such as well. All that flexibility is absent for the blacklist, however. Those names must be domains. Either list can be empty, with zero lines.

_________________
Six of the Linksys WRT1900ACSv2 on r38159, r39144, r40009, and r40784. On various:
VLANs, client-mode travel router, two DNSCrypt servers (incl Quad9), multiple VAPs, USB/NAS, QoS, OpenVPN client/PBR (old=NordVPN, new=AirVPN).
Sponsor
pikasso
DD-WRT Novice


Joined: 06 Nov 2019
Posts: 2

PostPosted: Wed Nov 06, 2019 23:40    Post subject: Reply with quote
Hi there..
SurprisedItWorks.. I registered today to ddwrt just to say thank you.
This method is so simple and automatic, no need of flash drive, tuning something in addition, and just simply works.
I must say I had a lot of fun watching the posts updates, from one method then using curl.. then wget.. then optimizing the sequences.. really perfect mindset and script.

Thank you !
Pierrick
SurprisedItWorks
DD-WRT User


Joined: 04 Aug 2018
Posts: 443
Location: Appalachian mountains, USA

PostPosted: Thu Nov 07, 2019 5:13    Post subject: Reply with quote
Thank you, Pierrick! Unusual that people bother with that. It is appreciated.

And so here's a bonus, something I just added to my routers a couple of weeks ago. Add this as a new line after the sleep and before the first sed:

until ping -c 1 -w 10 &> /dev/null cloudflare.com; do sleep 110; done

This prevents things from going further until the router is able to ping cloudflare.com. Tries the ping every two minutes.

I was moved to add this when I had a power failure at home for an hour, and when the power came back up, the router booted before the modem. This meant the router was trying to do the Download steps before success was possible, and as a result all the Downloads failed and badhosts remained empty. This way it won't try the downloads until the network is up with DNS functioning.

_________________
Six of the Linksys WRT1900ACSv2 on r38159, r39144, r40009, and r40784. On various:
VLANs, client-mode travel router, two DNSCrypt servers (incl Quad9), multiple VAPs, USB/NAS, QoS, OpenVPN client/PBR (old=NordVPN, new=AirVPN).
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 2972
Location: UK, London, just across the river..

PostPosted: Thu Nov 07, 2019 6:39    Post subject: Reply with quote
does this line keeps pinging even when WAN is on....??

until ping -c 1 -w 10 &> /dev/null cloudflare.com; do sleep 110; done

instead I usually extend adblocker sleep time in startup script as it's placed in the custom section, so it loads independently and its the last thing to load at the end when WAN is available anyway ..

_________________
Atheros
TP-Link WR740Nv1 ------DD-WRT 33772 BS WAP/Switch (wired)
TP-Link WR1043NDv2 ----DD-WRT 41379 BS (AP,PPPoE,NAT,AD Blocking,AP Isolation,Firewall,Local DNS,Forced DNS,DoT)
TP-Link WR1043NDv2 ----DD-WRT 41517 BS (AP,NAT,AD Blocking,Firewall,Wi-Fi OFF,Local DNS,Forced DNS,DoT)
TP-Link WR1043NDv2 ----Gargoyle OS 1.11.0 (AP,NAT,QoS,Quotas)
Qualcomm/IPQ8065
2x Netgear R7800 -------DD-WRT 40270M 4.9 Kong (AP,NAT,AD-Blocking,AP&Net Isolation,VLAN's,Firewall,Local DNS,DNSCrypt-proxy v2 x2)
Broadcom
Netgear R7000 -------DD-WRT 40270M Kong (AP,NAT,VLAN,AD-Blocking,Firewall,Local DNS,Forced DNS,DoT)
------------------------------------------------------------------------------------------------
Stubby for DNS over TLS I DNSCrypt v2 via Entware by mac913
pikasso
DD-WRT Novice


Joined: 06 Nov 2019
Posts: 2

PostPosted: Thu Nov 07, 2019 10:12    Post subject: Reply with quote
SurprisedItWorks wrote:
Thank you, Pierrick! Unusual that people bother with that. It is appreciated.


Being around xda and other communities, I know that so well.. I have been having my router since one year but now, I need also more parental control and ads management that I switched recently to ddwrt.. I begin to have fun with it Very Happy

SurprisedItWorks wrote:

until ping -c 1 -w 10 &> /dev/null cloudflare.com; do sleep 110; done


Updated successfully.

For the records.. I had previously a host file on my windows machine.. but now, all my devices are under the same configuration, and with white listing, it saves also some time in case of future forced *inclusion*.

Thanks
Goto page Previous  1, 2, 3, 4  Next Display posts from previous:    Page 3 of 4
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Advanced Networking All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum