Posted: Fri Jul 06, 2018 6:50 Post subject: Is Ad-Blocking (Privoxy) Still Useful Nowadays?
Hi there,
I was recently wondering about this and couldn't find any information about the topic. Since most of the websites are encrypted through HTTPS (and pretty much everything is migrating towards that), is Privoxy in the DD-WRT implementation still worth it? It's my understanding that it inspects and modifies HTML requests to block ads and such, but since said connections are HTTPS, I'm guessing it can't do it.
I've read in some places found via Google about installing some HTTPS handlers (or something like that) for standalone Prixovy, but I don't know if that would be possible to implement in DD-WRT.
Could anyone please shed some light into this? Thanks in advance for your help!
Joined: 16 Nov 2015 Posts: 6446 Location: UK, London, just across the river..
Posted: Sat Jul 07, 2018 5:39 Post subject:
yep its working and its present on the high flash size units but there are much better scripts for
ad blocking that work almost on every DD-WRT unit, personally i find privoxy a bit buggy _________________ Atheros
TP-Link WR740Nv1 ---DD-WRT 55630 WAP
TP-Link WR1043NDv2 -DD-WRT 55723 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55779 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55819 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55779 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
I know that it actually works (for HTTP sites at least); I have it enabled for the past ~2 years on a Netgear R8000. What I was wondering is what happen with all the HTTPS sites, Privoxy on DD-WRT can't inspect and filter them, right?
Can you share the better scripts that you'd recommend over Privoxy? I've also found out it giving the gray "error" page a little bit too often; probably because it's filtering legal sites that it shouldn't be blocking (like "Unsuscribe" buttons that take you to ad networks when you actually want to unsuscribe from them or some not well-known e-commerce sites).
Great! Yes, I used one of those scripts on my previous, lower end router that didn't support the DD-WRT version that includes Privoxy. Actually, 75% of my upgrade to the R8000 was to get Privoxy built in .
I'm using Adguard for DNS filtering nowadays, so I don't need to filter them myself anymore. But I was happy to add another layer with Privoxy locally.
So, if anybody can share some thoughts on if it's useful with all the HTTPS connections (considering that Privoxy doesn't touch tem), I'll be really helpful. Also, if there is some way to make Privoxy process them, adding some certificates of something; again, like Adguard for Windows does with HTTPS.
Joined: 16 Nov 2015 Posts: 6446 Location: UK, London, just across the river..
Posted: Sun Jul 08, 2018 17:52 Post subject:
hmm as far as i know this script has nothing common with https modified headders
this script relys on hots/tmp file block list..
so far its much better than privoxy
for more on the subject https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1078933 _________________ Atheros
TP-Link WR740Nv1 ---DD-WRT 55630 WAP
TP-Link WR1043NDv2 -DD-WRT 55723 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55779 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55819 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55779 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
Joined: 03 Jan 2010 Posts: 7568 Location: YWG, Canada
Posted: Sun Jul 08, 2018 20:23 Post subject:
privoxy is a messy scripty thing thats too much for me, and it doesnt "just work" like dns filtering does for all devices. so i use ad and tracking blocking with dnsmasq with my own built list _________________ LATEST FIRMWARE(S)
BrainSlayer wrote:
we just do it since we do not like any restrictions enforced by stupid cocaine snorting managers
Joined: 16 Nov 2015 Posts: 6446 Location: UK, London, just across the river..
Posted: Mon Jul 09, 2018 6:06 Post subject:
tatsuya46 wrote:
privoxy is a messy scripty thing thats too much for me, and it doesnt "just work" like dns filtering does for all devices. so i use ad and tracking blocking with dnsmasq with my own built list
hmm its interesting what tracking blocking script you use ?? _________________ Atheros
TP-Link WR740Nv1 ---DD-WRT 55630 WAP
TP-Link WR1043NDv2 -DD-WRT 55723 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55779 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55819 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55779 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
privoxy is a messy scripty thing thats too much for me evertime, and it doesnt "just work" like dns filtering does for all devices. so find my results posted on this article ad and tracking blocking with dnsmasq with my own built list
Yes, you're right. Privoxy is mediocre at its best. Would you be kind enough to share your own curated list?
Last edited by JacobSeymour on Mon Sep 21, 2020 23:53; edited 1 time in total
It's a superb ad-blocker for Asus-Merlin and the owner of the code is thinking of porting it to DD-WRT if there's a demand for it. I've used it on my N66W for more than a year and highly recommend it.
Well, here's a really late reply! This is an update to Alazaros's code above to set up dnsmasq to return dummy IP address 0.0.0.0 for over 40,000 dubious domain names.
The touch is to ensure the file is there when dnsmasq looks for it early on. This may not be needed. The sleep is to give my two dnscrypt-proxy instances time to get certificates before asking them to look things up. The 60 sec is definitely overkill, but I'm too lazy to test to trim it down. One wget is commented out because it forwards to an https site that wget can't handle. The sed first eliminates the CR characters, since one of the files was apparently generated on Windows, and it eliminates the seemingly pointless line 0.0.0.0 0.0.0.0 as well. Most importantly, and this is the whole point of this report, it eliminates any line that does not begin with 0.0.0.0 so that slipping in a line like blah.di.blah.di goodbank.com, either at the source or in some mitm attack, can't steer your banking to evilbank.com at IP address blah.di.blah.di. The sort eliminates redundant entries; it reduced the file size about 20%.
Just the one line addn-hosts=/tmp/badhosts needs incuding in DNSMasq Additional Options. Doesn't seem to matter where it goes relative to other lines there. _________________ 2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
SurprisedItWorks or Alozaros
I am not using DNSCrypt, but Unbound (Recursive DNS Resolving) along with DNSMaq as suggested here. And your method does not work: If you take some real site from the /tmp/badhosts file, for example - "skgroup.kiev.ua", then it will still be available, albeit with some delay.
You have no idea how to win this?
Joined: 04 Aug 2018 Posts: 1447 Location: Appalachian mountains, USA
Posted: Sat Aug 31, 2019 23:36 Post subject:
PavelVD wrote:
SurprisedItWorks or Alozaros
I am not using DNSCrypt, but Unbound (Recursive DNS Resolving) along with DNSMaq as suggested here. And your method does not work: If you take some real site from the /tmp/badhosts file, for example - "skgroup.kiev.ua", then it will still be available, albeit with some delay.
You have no idea how to win this?
Actually, I have tested it many times with sites from /tmp/badhosts, and it works fine. Of note, however, is that my router has IPv6 disabled (my vpn provider requires that for its recommended dd-wrt setup), so the fact that some systems will obtain IPv6 addresses for these sites and allow them to be loaded does not affect me. One could modify the sed script to turn each line into two for the same domain, with one line mapping it to 0.0.0.0 for IPv4 and the other mapping it to :: for IPv6, but I have only done the most minimal experiments in that direction.
I have improved the script a bit since then, though the functioning is basically the same, but now with capturing of error information in case one of the sites fails to download. Also I now use curl instead of wget. Here is what I currently have in my startup commands for adblocking:
The hulu line near the end is to make an exception for the key hulu ad sites without which hulu streaming simply fails. They don't permit you to opt out of ads. One could modify the line to add other exceptions.
The file /tmp/badhosts.errcodes should show 0 0 0 when everything is working, and the file /tmp/badhosts.log should be empty. If /tmp/badhosts.errcodes has any nonzeros, its position will reveal which curl had problems, and /tmp/badhosts.log should clarify the nature of the error. I added these features assuming that eventually one or more of those three lists of "bad" domains will no longer be posted.
ALSO, VERY IMPORTANT! In GUI>Services>Services in Additional Dnsmasq Options, one must add the line addn-hosts=/tmp/badhosts or the script will indeed be useless! _________________ 2x Netgear XR500 and 3x Linksys WRT1900ACSv2 on 53544: VLANs, VAPs, NAS, station mode, OpenVPN client (AirVPN), wireguard server (AirVPN port forward) and clients (AzireVPN, AirVPN, private), 3 DNSCrypt providers via VPN.
Joined: 16 Nov 2015 Posts: 6446 Location: UK, London, just across the river..
Posted: Sun Sep 01, 2019 7:11 Post subject:
back in the days, when i tried unbound, i noticed those
adblocker scripts, ware not working as intended, but that was the unbound, recursive resolving option from basic set up page...i never tried unbound via entware..
I can also confirm those scripts are working with DNScrypt, and stubby via entware witch i have running on my units too... _________________ Atheros
TP-Link WR740Nv1 ---DD-WRT 55630 WAP
TP-Link WR1043NDv2 -DD-WRT 55723 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55779 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55819 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55779 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
Thanks SurprisedItWorks! I like your improvement!
I allowed a slight improvement for myself:
The command "0 12 * * * root /tmp/.rc_startup" from the section "Additional Cron Jobs" restarts services that I do not want to touch, and replaced it with:
Code:
30 02 * * * root /tmp/custom.sh
And your script placed in "Administration--Commands--Custom Script", and changed for yourself:
You need to press the "Run Commads" button once by placing "/tmp/custom.sh" in "Administration - Commands--Command Shell--Commands". (It is clear that you need to do this after rebooting the router after completing the setup.)
The advantage is that the "/jffs/badhosts" file does not disappear when the router is restarted, it is available very quickly and there is no need to wait for the certificate to be updated for DNSCrypt. (Just in case, I inform you that "jffs" is mounted automatically on an external USB flash card.)
Well and, of course, in "Additional Dnsmasq Options" include
Code:
addn-hosts=/jffs/badhosts
Everything works well!
The only pity is that there is no white list of sites for which badhosts would not work.
And, yes ... The Unbound malfunctioning error that I wrote about above ... Gone! I did not understand what it was.
.
