Is Ad-Blocking (Privoxy) Still Useful Nowadays?

Post new topic   Reply to topic    DD-WRT Forum Index -> General Questions
Goto page Previous  1, 2
View previous topic :: View next topic  
Author Message
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1447
Location: Appalachian mountains, USA
PostPosted: Sun Sep 01, 2019 14:13    Post subject: Reply with quote
PavelVD wrote:
Thanks SurprisedItWorks! I like your improvement!
I allowed a slight improvement for myself

Yes, a nice adaptation! Very Happy

When I cobble up these things for myself I sometimes forget, when I eventually post them, that people with different router setups are going to need to do things differently. My router reboots every night, and so reruns the script, at around the time yours runs the cron jobs, so in that way we are actually "on the same page" regarding updating the badhosts file. Your /jffs touch for continuity is of course ideal, but I don't use /jffs here and so am stuck with a minute (or less) of "vulnerability" to ads and malware in the middle of each night. Shocked
_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Back to top View user's profile Send private message
Sponsor
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 6447
Location: UK, London, just across the river..
PostPosted: Fri Sep 06, 2019 19:39    Post subject: Reply with quote
hmmmm...to run my adblocker from custom script
i do have "sh /tmp/custom.sh" in start up...and than again i restart it via cron "0 12 * * * root /tmp/custom.sh" to get the updates...
i also noticed in DDWRT for some reason there is a limit number of those host's files list it never D/L all of them...for that reason i run only 1...
currently i do have this one running..

_rogue=0.0.0.0
echo -e "n=1\nwhile ! wget -q -O /tmp/hsts http://winhelp2002.mvps.org/hosts.txt ; do\n\t[ \$n -gt 5 ] && break\n\tlet n+=1\n\tsleep 60\ndone\ngrep \"^0.0.0.0\" /tmp/hsts | grep \"^0.0.0.0\" | grep -v localhost | awk '{print \"$_rogue\\\t\"\$2}' | tr -d '\\\015' >/tmp/badhosts\nrm /tmp/hsts\nkillall -HUP dnsmasq" >/tmp/wh
sh /tmp/wh &

addn-hosts=/tmp/badhosts in DNSmasq

i guess in order to save space, it also does only 0.0.0.0 lines , those with 127.0.0.1 are not executed....so not much point to use that format...it needs revision i guess....
but i wouldn't mind to try this, jffs option...the only thing is, curl command is not present on few of my routers..so im limited to wget...

also there are many other adblocking scripts with or without whitelist/black list...
_________________
Atheros
TP-Link WR740Nv1 ---DD-WRT 55630 WAP
TP-Link WR1043NDv2 -DD-WRT 55723 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55779 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55819 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55779 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
Back to top View user's profile Send private message
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1447
Location: Appalachian mountains, USA
PostPosted: Sat Sep 07, 2019 0:14    Post subject: Reply with quote
Alozaros wrote:
i do have this one running..

_rogue=0.0.0.0
echo -e "n=1\nwhile ! wget -q -O /tmp/hsts http://winhelp2002.mvps.org/hosts.txt ; do\n\t[ \$n -gt 5 ] && break\n\tlet n+=1\n\tsleep 60\ndone\ngrep \"^0.0.0.0\" /tmp/hsts | grep \"^0.0.0.0\" | grep -v localhost | awk '{print \"$_rogue\\\t\"\$2}' | tr -d '\\\015' >/tmp/badhosts\nrm /tmp/hsts\nkillall -HUP dnsmasq" >/tmp/wh

So when I run that big echo in the CLI just to see more clearly what you have, I see that your /tmp/wh is this:
Code:
n=1
while ! wget -q -O /tmp/hsts http://winhelp2002.mvps.org/hosts.txt ; do
   [ $n -gt 5 ] && break
   let n+=1
   sleep 60
done
grep "^0.0.0.0" /tmp/hsts | grep "^0.0.0.0" | grep -v localhost | awk '{print "0.0.0.0\t"$2}' | tr -d '\015' >/tmp/badhosts
rm /tmp/hsts
killall -HUP dnsmasq

Makes sense, except you seem to have a redundant extra (from the Department of Redundancy Department? Surprised ) grep in there. And you might want to replace "^0.0.0.0" with "^0\.0\.0\.0" (to be super careful bordering on paranoid sometimes, like me), since the unescaped version will match things like 0000000 or 010 020, etc. at the beginning of a line. (Not that there should ever be any of those to match!)

I'm curious... Do you find the loop sometimes goes around a time or two before it succeeds? It seems a mystery that these wgets and curls work except when they don't. Shocked
_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Back to top View user's profile Send private message
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 6447
Location: UK, London, just across the river..
PostPosted: Sat Sep 07, 2019 2:30    Post subject: Reply with quote
SurprisedItWorks wrote:

I'm curious... Do you find the loop sometimes goes around a time or two before it succeeds? It seems a mystery that these wgets and curls work except when they don't. Shocked


nope i can see it ...cat/tmp/badhosts ... all there..
but yep sometimes is missing from syslog, but its fine
i thing is just a syslog issue

Thanks ill change those and see, and witch grep is not needed??
_________________
Atheros
TP-Link WR740Nv1 ---DD-WRT 55630 WAP
TP-Link WR1043NDv2 -DD-WRT 55723 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55779 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55819 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55779 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
Back to top View user's profile Send private message
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1447
Location: Appalachian mountains, USA
PostPosted: Sat Sep 07, 2019 15:19    Post subject: Reply with quote
Alozaros wrote:
but yep sometimes is missing from syslog, but its fine
i thing is just a syslog issue

Neither of us has anything in our scripts that would cause syslog to be written. If I wanted to print my little badhosts.errcodes file into the syslog, I believe I could add something like this to my script at or near the end:

logger -t "badhosts.errcodes [$$]" < badhosts.errcodes

Here the quotes contain a text tag that would go into the syslog entry to label it. Haven't tested this, so experimenting from the CLI would be appropriate.
Quote:
Thanks ill change those and see, and witch grep is not needed?
Either of the first two, as they select the same lines in the data. Need to keep the filename there, of course, so the easiest is to eliminate the second one:

grep "^0.0.0.0" /tmp/hsts | grep "^0.0.0.0" | grep -v localhost | awk '{print "0.0.0.0\t"$2}' | tr -d '\015' >/tmp/badhosts

For that matter, you could simplify more aggressively and replace all three greps with a single sed:

sed -n '/^0\.0\.0\.0/{/localhost/!p}' /tmp/hsts | awk '{print "0.0.0.0\t"$2}' | tr -d '\015' >/tmp/badhosts

That's the fun of Unix text processing... there are always many ways to do things!
_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Back to top View user's profile Send private message
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 1447
Location: Appalachian mountains, USA
PostPosted: Sun Sep 08, 2019 23:08    Post subject: Reply with quote
Hi folks...

Turns out I had a bug in my adblocking code above. Corrected and generally improved version in my Sunday Sep 8, 2019 (today) post at https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1176075. We've had these two threads going on one common topic for awhile. Time to join forces?
_________________
2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Back to top View user's profile Send private message
Goto page Previous  1, 2 Display posts from previous:    Page 2 of 2
Post new topic   Reply to topic    DD-WRT Forum Index -> General Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum