[Solution] Blocking all Youtube Video Advertisements

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Advanced Networking
Goto page Previous  1, 2, 3, 4  Next
Author Message
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 648
Location: Appalachian mountains, USA

PostPosted: Sat Jan 11, 2020 2:44    Post subject: Reply with quote
Hello all! After many, many experiments I finally have Startup code to set up a dnsmasq-based youtube adblocker that comes reasonably close to what @mkaand so kindly offered the Unbound users. It has the same three major components as @mkaand's system, and I found, just as he did, that all three seem quite necessary. It is not perfect. On some youtube platforms (Roku, Apple TV) I saw an initial ad or two, not once per video but once per youtube run. On my iPhone I saw one ad only, at the end of just one video out of many. On my Android TV I saw no ads at all. On the phone I did see the yellow dots on the progress line, but there were no ads as the dots were crossed! It seemed as though I might be seeing ads when clicking on the ridiculous video choices youtube offers on startup, cute cats or cartoon characters or whatever. Once I searched for videos to watch, however, ads seem to completely disappear.

Note that I do not use IPv6 and disable it in dd-wrt. If you use IPv6, you're going to need to modify this to block IPv6 addresses, etc. Also, I reboot dd-wrt every night, so the special IP address obtained for manifest.googlevideo.com is always less than a day old on my system. I don't know what will happen if you boot once and run for days or weeks or months. Will the IP address for manifest.googlevideo.com become "stale"? I posted above a bash function for updating it. It's still a little unclear to me which client devices need rebooting to clear DNS caches once this IP address is established.

The code in this post is based on an adblocker that I posted in https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=321121&start=42 but has been heavily tailored to the youtube-blocking mission. As stated in its opening comments, it does require that two lines be added to Additional DNSMasq Options:
Code:
#Adblocker.  Requires two dnsmasq Add'l Config lines:
#addn-hosts=/tmp/badhosts
#address=/1e100.net/0.0.0.0
( STARTED=/tmp/root/StartedAdBlocker
  [[ -f $STARTED ]] && exit || touch $STARTED
  ( cd /tmp; touch badhosts; sleep 30
    until ping -c 1 -w 1 &>/dev/null cloudflare.com; do sleep 120; done
    #for youtube ad blocker: https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1187268
    #and https://www.reddit.com/r/pihole/comments/9w5swx/i_think_ive_managed_to_block_youtube_ads_with/
      D=manifest.googlevideo.com; A='^Address [0-9]+: '; I='[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'
      { nslookup $D 2>/dev/null; echo -n " $?" >> badhosts.codes; } \
      | sed -nE "/$A$I/{s/$A($I).*/\1 $D/p;q}" | tee -a badhosts \
      | awk '{print "youtube entry:",$0}' \
      | logger -t "startup-code adblocker"
    sed 's/\./\\./g;s/.*/\/( |\\.)&$\/d/' <<'ENDWHITE' >badhosts.whitelist
manifest.googlevideo.com
ENDWHITE
    awk '{print "0.0.0.0 "$1}' <<'ENDBLACK' >badhosts.blacklist
ads.facebook.com
connect.facebook.net
ENDBLACK
    { Download(){ { curl -kf $1; echo -n " $?" >> badhosts.codes; } };
      #youtube from Handley, stuart engineering, barcelona: https://gist.github.com/seanhandley
        Download https://gist.githubusercontent.com/seanhandley/d76c51099a32287c0a9074d5f15ac7d0/raw/2bde83a93e68208585d602df787d782036271438/gistfile1.txt
      #other youtube URLs: https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1186953
        Download https://api.hackertarget.com/hostsearch/?q=googlevideo.com \
        | awk -F , '{print "0.0.0.0",$1}'
        Download https://raw.githubusercontent.com/anudeepND/youtubeadsblacklist/master/domainlist.txt \
        | sed '/---/d;s/^/0.0.0.0 &/'
      echo '' >> badhosts.codes
    } 2>badhosts.log \
    | sed 's/\t/ /g; /^0\.0\.0\.0 /!d; s/ *\#.*$//; s/\r//' \
    | sort -u | sed -Ef badhosts.whitelist \
    | cat - badhosts.blacklist >> badhosts
    wc -l badhosts \
    | awk '{printf "%s has %d lines, exit codes",$2,$1}' \
    | cat - badhosts.codes \
    | logger -t "startup-code adblocker"
  ) && killall -HUP dnsmasq & )

The whole approach is to extend /etc/hosts with a new file /tmp/badhosts in the same format that mostly (except for one line) gives various domains IP addresses of 0.0.0.0, which are DNS dead ends, more or less. Online lists of sites to block are downloaded with a bash function, Download, created here. That function when run also adds a few lines to /tmp/badhosts.log and two characters, a space and an exit code, to /tmp/badhosts.codes. If downloading goes as intended, the exit codes will all be zero and the log will show ordinary downloading statistics. There is a delay before the code does anything meaningful, just to allow time for networking and the DNS system to come up first, as both are needed, and a zero-length file /tmp/root/StartedAdBlocker is created as a "note to self" that this code has been run and should not be run again if an Apply in the GUI reruns the Startup code.

The code's commenting convention is that comment lines apply to the code indented under them, which might be many lines. The code block early on that begins with #for youtube ad blocker creates the one badhosts line, for manifest.googlevideo.com, that does not have 0.0.0.0 as the IP address. Instead, the IP address entered on that line is obtained as the IP listed for that domain in the public DNS system when this Startup code is run. It will come from whatever DNS server the router is using and so will most often be from a server reasonably nearby geographically. Do understand that this is NOT a fixed IP address! It appears to change every few minutes at most! Much has been made in this thread of the fact that inverse lookups on this IP address lead back to domains with coded names often beginning with an airport code, with IAD for Dulles International Airport in Washington DC getting perhaps more attention in the discussion that it really deserves. There is really nothing special about IAD. I've seen LAX just as often, and others besides. In any case, once this line is in badhosts, looking up manifest.googlevideo.com in the router's world yields this IP address saved from boot time, and an inverse lookup on that IP address yields just manifest.googlevideo.com, not the magic coded filenames with airport codes. This is as close as we can get to creating an actual PTR record, but it seems close enough.

The short code block nearer the end that begins with #youtube from Handley downloads @mkaand's fixed list of domains to block, and the short block that begins with #other youtube URLs downloads his dynamic list of domains to block.

In addition to the fixed and dynamic lists and some special treatment for manifest.googlevideo.com, just as @mkaand said, it turns out to be necessary to block all domains *.1e100.net, something that must be handled, per a comment at the top of the code, by adding a line address=/1e100.net/0.0.0.0 to Additional DNSMasq Options. It had to be done there, as the badhosts file cannot block based on wildcards.

The code also includes whitelist and blacklist facilities. As presented above, only manifest.googlevideo.com is whitelisted, but more lines can be added as needed. Whitelisting does not apply to the special manifest.googlevideo.com line we created but affects only the downloaded material. It turned out that the fixed list includes manifest.googlevideo.com, so if it were not whitelisted, it would be given an IP address of 0.0.0.0 and effectively override the special line and defeat its purpose. So this one line of whitelisting is required. More whitelisting lines can be added as needed. (More below on the subtleties of the format of whitelisting lines, as each is really a variant form of regular expression.)

Here I have blacklisted ads.facebook.com and connect.facebook.net just to illustrate. Blacklisting connect.facebook.net may effectively kill facebook messenger, so don't be surprised if you need to remove that line. The format of blacklist lines is straightforward. One line, one domain blocked. No wild cards. Add as many lines as you like within reason. (If you want to add thousands, better perhaps to use the Download facility?)

Entries are made to the system log (search for "adblocker") detailing the IP address assigned to manifest.googlevideo.com, giving the number of lines (one domain per line) in badhosts, and showing the list of Download exit codes (which we hope are zero).

You can add other adblocking, besides for youtube, by adding Downloads, either just before or just after the #youtube from Handley block. Any download list that is just lines like 0.0.0.0 some.host.blah and comments beginning with #, blank lines, extra spaces (or tabs), is fine, and line endings in either linux or Windows form will work. Other formats will require sed or awk cleverness to adjust things, as was needed for the youtube fixed and dynamic lists above. Redundant lines will be eliminated automatically, so duplications are not an issue.

My adblocker in https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=321121&start=42 uses these two blocks in particular:
Code:
      #sbc.io URL:  https://github.com/StevenBlack/hosts/blob/master/readme.md
        Download http://sbc.io/hosts/hosts
      #Alozaros URLs: 7/8/18 at https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=315773
        Download http://winhelp2002.mvps.org/hosts.txt
        Download https://someonewhocares.org/hosts/zero/hosts

and here is another that supposedly targets domains related to mass surveillance (the word "supposedly" is actually all I know about it):
Code:
      #CHEF-KOCH trackers: https://github.com/CHEF-KOCH/NSABlocklist/tree/master/HOSTS/Trackers
        CK=https://raw.githubusercontent.com/CHEF-KOCH/NSABlocklist/master/HOSTS/Trackers
        Download $CK/trackers.txt
        Download $CK/Canvas%20font%20fingerprinting.txt
        Download $CK/audio%20fingerprinting%20pages.txt
        Download $CK/canvas%20fingerprinting%20pages.txt
        Download $CK/webrtc%20tracking.txt

The badhost lines created by the latter block ipleak.net, so you may want to whitelist the latter. And one of these Downloads, I forget which, ended up putting a line in badhosts with a domain name containing special characters. I found that it printed as secret.Google.com but with the G reduced in height to the height of an x. This is evil, and it caused dnsmasq to have trouble, so I actually had to allow it using the whitelist line secret.[^g]+oogle.com using [^g]+ to represent one or more occurences of characters that are not the letter g.

Yes, the whitelist lines are mostly just extended regular expressions like used by sed -E or grep -E. Two exceptions. First, the "." is not a wild card. It really is a period. Second, implicit is that whatever it matches comes either at the beginning of a domain name or just after a period in the domain name. So hulu(ad)?.com represents, using the more familiar "*" wildcard notation, hulu.com, *.hulu.com, huluad.com, and *.huluad.com. All get whitelisted. If you are a hulu streamer and use all the Download lists above, you will need this line, as hulu won't work if you allow its ads to be blocked.

Consider another whitelist example. After finding I could not access news sources nypost.com or nymag.com after including all the Downloads above, I went into the dd-wrt CLI and used grep appropriately on /tmp/badhosts and discovered that both were blocked, along with adcache.nymag.com, stats.nymag.com, and ads.nypost.com. I needed to whitelist nypost.com and nymag.com, so I added the whitelist line ny(mag|post).com only to realize that I was whitelisting nymag.com, *.nymag.com, nypost.com, and *.nypost.com, which meant I was whitelisting adcache.nymag.com, stats.nymag.com, and ads.nypost.com as well as the two domains I wanted to whitelist! So I had to then blacklist the latter three domains separately. Whitelist processing proceeds blacklist processing, so the combination of this one whitelist entry and the three blacklist entries did what I needed. Whew! I certainly can't say this is the ideal scheme. I can only say its the best one I have the patience to create.

Most likely this post will get edited over the next few days to clean it up. But I'm going ahead and posting it just to get things moving. Hope the approach works as well for you as it seems to work for me (for now Shocked ).

_________________
Six Linksys WRT1900ACSv2 (39144/40009/41954):
VLANs, multiple VAPs, NAS, QoS, client-mode travel router, OpenVPN client/PBR (AirVPN), two DNSCrypt servers (incl Quad9) routed through vpn.
Sponsor
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Sat Jan 11, 2020 3:57    Post subject: Reply with quote
i was getting random ads and initial ads, i have seen some videos longer than 10 and 20 minutes and no ads by now, i tested with channels i follow that i know display ads,

i have simplified and removed some duplicated IP, only google/youtube ads ip related are added, there are IP used globally not only local for my case, redirected manifest to @mkaand IP supplied, and this is my final - so far - firewall list

iptables -I FORWARD -s 172.217.0.33 -j DROP
iptables -I FORWARD -s 172.217.0.46 -j DROP
iptables -I FORWARD -s 172.217.12.226 -j DROP
iptables -I FORWARD -s 172.217.12.238 -j DROP
iptables -I FORWARD -s 172.217.13.228 -j DROP
iptables -I FORWARD -s 172.217.13.238 -j DROP
iptables -I FORWARD -s 172.217.13.66 -j DROP
iptables -I FORWARD -s 172.217.13.78 -j DROP
iptables -I FORWARD -s 172.217.14.196 -j DROP
iptables -I FORWARD -s 172.217.15.110 -j DROP
iptables -I FORWARD -s 172.217.15.65 -j DROP
iptables -I FORWARD -s 172.217.15.78 -j DROP
iptables -I FORWARD -s 172.217.15.97 -j DROP
iptables -I FORWARD -s 172.217.15.98 -j DROP
iptables -I FORWARD -s 172.217.15.99 -j DROP
iptables -I FORWARD -s 172.217.161.66 -j DROP
iptables -I FORWARD -s 172.217.164.129 -j DROP
iptables -I FORWARD -s 172.217.164.142 -j DROP
iptables -I FORWARD -s 172.217.164.164 -j DROP
iptables -I FORWARD -s 172.217.164.174 -j DROP
iptables -I FORWARD -s 172.217.164.98 -j DROP
iptables -I FORWARD -s 172.217.164.99 -j DROP
iptables -I FORWARD -s 172.217.167.66 -j DROP
iptables -I FORWARD -s 172.217.194.153/29 -j DROP
iptables -I FORWARD -s 172.217.2.205 -j DROP
iptables -I FORWARD -s 172.217.25.130 -j DROP
iptables -I FORWARD -s 172.217.25.131 -j DROP
iptables -I FORWARD -s 172.217.25.142 -j DROP
iptables -I FORWARD -s 172.217.25.161 -j DROP
iptables -I FORWARD -s 172.217.25.196 -j DROP
iptables -I FORWARD -s 172.217.25.227 -j DROP
iptables -I FORWARD -s 172.217.26.2 -j DROP
iptables -I FORWARD -s 172.217.27.78 -j DROP
iptables -I FORWARD -s 172.217.28.2 -j DROP
iptables -I FORWARD -s 172.217.29.1 -j DROP
iptables -I FORWARD -s 172.217.29.226 -j DROP
iptables -I FORWARD -s 172.217.3.32 -j DROP
iptables -I FORWARD -s 172.217.3.33 -j DROP
iptables -I FORWARD -s 172.217.3.35 -j DROP
iptables -I FORWARD -s 172.217.30.34 -j DROP
iptables -I FORWARD -s 172.217.30.78 -j DROP
iptables -I FORWARD -s 172.217.5.238 -j DROP
iptables -I FORWARD -s 172.217.5.97 -j DROP
iptables -I FORWARD -s 172.217.5.98 -j DROP
iptables -I FORWARD -s 172.217.5.99 -j DROP
iptables -I FORWARD -s 172.217.6.34 -j DROP
iptables -I FORWARD -s 172.217.6.78 -j DROP
iptables -I FORWARD -s 172.217.7.130 -j DROP
iptables -I FORWARD -s 172.217.7.132 -j DROP
iptables -I FORWARD -s 172.217.7.162 -j DROP
iptables -I FORWARD -s 172.217.7.163 -j DROP
iptables -I FORWARD -s 172.217.7.174 -j DROP
iptables -I FORWARD -s 172.217.7.206 -j DROP
iptables -I FORWARD -s 172.217.7.238 -j DROP
iptables -I FORWARD -s 172.217.7.246 -j DROP
iptables -I FORWARD -s 172.217.8.14 -j DROP
iptables -I FORWARD -s 172.217.9.195 -j DROP
iptables -I FORWARD -s 172.217.9.206 -j DROP
iptables -I FORWARD -s 209.85.202.148 -j DROP
iptables -I FORWARD -s 209.85.202.149 -j DROP
iptables -I FORWARD -s 209.85.232.148 -j DROP
iptables -I FORWARD -s 209.85.232.149 -j DROP
iptables -I FORWARD -s 216.58.193.66 -j DROP
iptables -I FORWARD -s 216.58.194.194 -j DROP
iptables -I FORWARD -s 216.58.195.66 -j DROP
iptables -I FORWARD -s 216.58.195.68 -j DROP
iptables -I FORWARD -s 216.58.196.130 -j DROP
iptables -I FORWARD -s 216.58.197.161 -j DROP
iptables -I FORWARD -s 216.58.197.226 -j DROP
iptables -I FORWARD -s 216.58.200.98 -j DROP
iptables -I FORWARD -s 216.58.202.130 -j DROP
iptables -I FORWARD -s 216.58.202.131 -j DROP
iptables -I FORWARD -s 216.58.202.164 -j DROP
iptables -I FORWARD -s 216.58.203.100 -j DROP
iptables -I FORWARD -s 216.58.210.227 -j DROP
iptables -I FORWARD -s 216.58.217.33 -j DROP
iptables -I FORWARD -s 216.58.217.78 -j DROP
iptables -I FORWARD -s 216.58.218.238 -j DROP
iptables -I FORWARD -s 64.233.186.148 -j DROP
iptables -I FORWARD -s 64.233.186.149 -j DROP
iptables -I FORWARD -s 74.125.130.100 -j DROP
iptables -I FORWARD -s 74.125.130.101 -j DROP
iptables -I FORWARD -s 74.125.130.102 -j DROP
iptables -I FORWARD -s 74.125.130.113 -j DROP
iptables -I FORWARD -s 74.125.130.138 -j DROP
iptables -I FORWARD -s 74.125.130.139 -j DROP
iptables -I FORWARD -s 74.125.135.148 -j DROP
iptables -I FORWARD -s 74.125.135.149 -j DROP
iptables -I FORWARD -s 74.125.142.148 -j DROP
iptables -I FORWARD -s 74.125.142.149 -j DROP
iptables -I FORWARD -s 74.125.193.100 -j DROP
iptables -I FORWARD -s 74.125.193.101 -j DROP
iptables -I FORWARD -s 74.125.193.102 -j DROP
iptables -I FORWARD -s 74.125.193.103 -j DROP
iptables -I FORWARD -s 74.125.193.104 -j DROP
iptables -I FORWARD -s 74.125.193.105 -j DROP
iptables -I FORWARD -s 74.125.193.106 -j DROP
iptables -I FORWARD -s 74.125.193.113 -j DROP
iptables -I FORWARD -s 74.125.193.132 -j DROP
iptables -I FORWARD -s 74.125.193.138 -j DROP
iptables -I FORWARD -s 74.125.193.139 -j DROP
iptables -I FORWARD -s 74.125.193.147 -j DROP
iptables -I FORWARD -s 74.125.193.154 -j DROP
iptables -I FORWARD -s 74.125.193.155 -j DROP
iptables -I FORWARD -s 74.125.193.156 -j DROP
iptables -I FORWARD -s 74.125.193.157 -j DROP
iptables -I FORWARD -s 74.125.193.99 -j DROP
iptables -I FORWARD -s 74.125.200.154 -j DROP
iptables -I FORWARD -s 74.125.200.155 -j DROP
iptables -I FORWARD -s 74.125.200.156 -j DROP
iptables -I FORWARD -s 74.125.200.157 -j DROP
iptables -I FORWARD -s 74.125.23.148 -j DROP
iptables -I FORWARD -s 74.125.23.149 -j DROP
iptables -I FORWARD -s 74.125.24.103 -j DROP
iptables -I FORWARD -s 74.125.24.104 -j DROP
iptables -I FORWARD -s 74.125.24.105 -j DROP
iptables -I FORWARD -s 74.125.24.106 -j DROP
iptables -I FORWARD -s 74.125.24.132 -j DROP
iptables -I FORWARD -s 74.125.24.147 -j DROP
iptables -I FORWARD -s 74.125.24.94 -j DROP
iptables -I FORWARD -s 74.125.24.99 -j DROP
iptables -I FORWARD -s 74.125.68.148 -j DROP
iptables -I FORWARD -s 74.125.68.149 -j DROP
iptables -I FORWARD -s 74.125.68.154 -j DROP
iptables -I FORWARD -s 74.125.68.155 -j DROP
iptables -I FORWARD -s 74.125.68.156 -j DROP
iptables -I FORWARD -s 74.125.68.157 -j DROP
iptables -I FORWARD -s 74.125.90.66 -j DROP
iptables -I FORWARD -s 74.125.90.99 -j DROP
iptables -I FORWARD -s 172.217.2.195 -j DROP
iptables -I FORWARD -s 172.217.9.194 -j DROP
iptables -I FORWARD -s 172.217.9.198 -j DROP
iptables -I FORWARD -s 172.217.7.226 -j DROP
iptables -I FORWARD -s 172.217.7.225 -j DROP
iptables -I FORWARD -s 172.217.164.130 -j DROP
iptables -I FORWARD -s 172.217.164.162 -j DROP
iptables -I FORWARD -s 172.217.15.66 -j DROP
iptables -I FORWARD -s 172.217.15.70 -j DROP
iptables -I FORWARD -s 172.217.8.2 -j DROP
iptables -I FORWARD -s 172.217.13.226 -j DROP
iptables -I FORWARD -s 172.217.7.194 -j DROP
iptables -I FORWARD -s 172.217.5.226 -j DROP
iptables -I FORWARD -s 216.239.38.129 -j DROP
iptables -I FORWARD -s 172.217.7.129 -j DROP
iptables -I FORWARD -s 74.125.67.1 -j DROP
iptables -I FORWARD -s 216.239.37.1 -j DROP
iptables -I FORWARD -s 172.217.2.206 -j DROP
iptables -I FORWARD -s 216.239.36.4 -j DROP
iptables -I FORWARD -s 172.217.164.129 -j DROP
iptables -I FORWARD -s 172.217.9.193 -j DROP
iptables -I FORWARD -s 172.217.164.132 -j DROP
iptables -I FORWARD -s 172.217.15.100 -j DROP
iptables -I FORWARD -s 172.217.15.67 -j DROP
iptables -I FORWARD -s 172.217.12.227 -j DROP

optional AD's IP ranges to block
#iptables -I FORWARD -s 172.217.164.129/26 -j DROP
#iptables -I FORWARD -s 172.217.3.32 -j DROP
#iptables -I FORWARD -s 172.217.3.33/27 -j DROP
#iptables -I FORWARD -s 216.239.38.113/28 -j DROP
#iptables -I FORWARD -s 216.239.38.15 -j DROP
#iptables -I FORWARD -s 216.239.38.21 -j DROP
#iptables -I FORWARD -s 216.239.38.127 -j DROP
#iptables -I FORWARD -s 216.239.38.128 -j DROP
#iptables -I FORWARD -s 216.239.38.130 -j DROP
#iptables -I FORWARD -s 172.217.13.65/27 -j DROP
#iptables -I FORWARD -s 172.217.13.225/27 -j DROP
#iptables -I FORWARD -s 172.217.15.65/28 -j DROP
#iptables -I FORWARD -s 172.217.7.128/25 -j DROP
#iptables -I FORWARD -s 172.217.2.192/26 -j DROP
#iptables -I FORWARD -s 172.217.2.192 -j DROP
#iptables -I FORWARD -s 172.217.194.153/29 -j DROP
#iptables -I FORWARD -s 172.217.8.1/27 -j DROP
#iptables -I FORWARD -s 172.217.8.0 -j DROP
#iptables -I FORWARD -s 172.217.5.225/27 -j DROP
#iptables -I FORWARD -s 172.217.1.1/27 -j DROP
#iptables -I FORWARD -s 172.217.1.0 -j DROP
#iptables -I FORWARD -s 172.217.1.31 -j DROP
#iptables -I FORWARD -s 172.217.1.193/27 -j DROP


Dnsmasq additional Option in Services:

address=/gstaticadssl.l.google.com/0.0.0.0
address=/dynamicads.g.doubleclick.net/0.0.0.0
address=/i4.ytimg.com/0.0.0.0
address=/s.ytimg.com/172.217.8.110
address=/gmail.com/172.217.2.69
address=/googlemail.l.google.com/172.217.8.101
address=/drive.google.com/172.217.1.110
address=/i.ytimg.com/172.217.8.86
address=/www.google.com.do/172.217.15.195
address=/google.com.do/172.217.2.131
address=/i9.ytimg.com/172.217.3.142
address=/i1.ytimg.com/172.217.2.78
address=/yt3.ggpht.com/172.217.2.65 #172.217.8.65
address=/photos-ugc.l.googleusercontent.com/172.217.2.65
address=/youtube.com/172.217.2.78
address=/www.youtube.com/172.217.2.78 #172.217.3.142
address=/photos.google.com/172.217.2.142
address=/www.google.com/172.217.15.195 #172.217.2.142
address=/google.com/172.217.8.142 #cache.google.com
address=/pagead.l.google.com/0.0.0.0
address=/clients1.google.com/0.0.0.0
address=/googleadservices.com/0.0.0.0
address=/manifest.googlevideo.com/172.217.19.238
address=/ads.us.criteo.com/0.0.0.0
address=/bidder.criteo.com/0.0.0.0
address=/r1---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.238
address=/r2---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.238
address=/r3---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.238
address=/r4---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.238
address=/r5---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.238
address=/r6---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.238
address=/r1---sn-ugpjvh-nups.googlevideo.com/172.217.19.238
address=/r4---sn-ugpjvh-nups.googlevideo.com/172.217.19.238
address=/r5---sn-ugpjvh-nups.googlevideo.com/172.217.19.238
address=/r6---sn-ugpjvh-nups.googlevideo.com/172.217.19.238
address=/r3---sn-ugpjvh-nups.googlevideo.com/172.217.19.238
address=/r2---sn-ugpjvh-nups.googlevideo.com/172.217.19.238
address=/r3---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.238
address=/r1---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.238
address=/r2---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.238
address=/r3---sn-ugpjvh-nupl.googlevideo.com/172.217.19.238
address=/r2---sn-ugpjvh-nupl.googlevideo.com/172.217.19.238
address=/r1---sn-ugpjvh-nupl.googlevideo.com/172.217.19.238

as i wrote before, i also use a host file in start up script for other ads but that doesn't stop youtube videos ads so that's why i did not post details about it.
MrPickles
DD-WRT Novice


Joined: 14 Jan 2017
Posts: 40

PostPosted: Sat Jan 11, 2020 14:04    Post subject: Reply with quote
Thanks guys, this is an invaluable topic.

Especially with Apple products.

Going to give these scripts a try!

Cheers,
MP
mkaand
DD-WRT User


Joined: 06 Jan 2008
Posts: 295
Location: Istanbul

PostPosted: Sat Jan 11, 2020 15:17    Post subject: Reply with quote
@SurprisedItWorks Excellent job! But I have some suggestions for you. Please simplified your script (like an idiot guide) and prepare like a tutorial (check first message of this topic) Peoples who wants to block Youtube ADs doesn't care about long paragraphs and long sentences. Less words more codes are always better.

Another suggestion, please -ONLY- post your DNSmasq Youtube AD block. I open this topic for this purpose ONLY. For generic AD blocks everyone has their own solution and it is not the purpose of this topic.

For my Unbound script (first post) works perfectly on LG Smart TV Youtube App (WebOS), Android TV, iPhone and iPad.

You're quite welcome @MrPickles.

_________________
Kaan's World | @mkaand | PLEX Archive | Trakt.tv
mkaand
DD-WRT User


Joined: 06 Jan 2008
Posts: 295
Location: Istanbul

PostPosted: Sat Jan 11, 2020 15:20    Post subject: Reply with quote
@Rickz, you don't need to write firewall rules for blocking Youtube video ADs. These rules only make your router more slower and eats memory. I watch youtube without any ADs on every platform and I do not use any firewall rule for that.
_________________
Kaan's World | @mkaand | PLEX Archive | Trakt.tv
SurprisedItWorks
DD-WRT Guru


Joined: 04 Aug 2018
Posts: 648
Location: Appalachian mountains, USA

PostPosted: Sat Jan 11, 2020 16:34    Post subject: Reply with quote
mkaand wrote:
@SurprisedItWorks Excellent job! But I have some suggestions for you. Please simplified your script (like an idiot guide) and prepare like a tutorial (check first message of this topic) Peoples who wants to block Youtube ADs doesn't care about long paragraphs and long sentences. Less words more codes are always better.

Yes, I understand of course. But writing something shorter takes longer, and this was a quick write up last night. I'll keep an abbreviated version on the to-do list though.
Quote:
Another suggestion, please -ONLY- post your DNSmasq Youtube AD block. I open this topic for this purpose ONLY. For generic AD blocks everyone has their own solution and it is not the purpose of this topic.

Indeed, that's why I redid my script -- this took a few hours actually, with all the testing -- to allow a youtube-only version with other adblocking then being strictly optional. It's why I presented the youtube part separately, in the first part of the post. Still though, there was no way around, even for youtube only, needing at least a minimal whitelist, and the simplist thing for the moment was to keep the whitelist and blacklist mechanisms I already had. Given that starting point, it added very modestly to the post (and not at all to the script) to include the option of adding other block lists.

And yes, the script as presented is long, and I should be able to go back and create a streamlined version. Certainly a fair bit of the length is for secondary things, like the prevention of accidental rerunning, the fairly extensive logging, and the delay until networking and DNS are up. None of those things were there when I started this in the nonyoutube world, but all got added over time to solve problems that came up. These are easily removed though, and it should also be easy to hardwire in a special-purpose whitelisting of manifest.googlevideo.com so that a general whitelisting mechanism is not needed. With those things in mind then, I'll aim for a streamlined version next.

The catch is that I won't plan on fully testing that streamlined version here. I have a version that works on my router, and I'm not going to set it aside to debug code that I don't need. All I can promise is that I'll take reasonable care in stripping down the big version to make the small version. Basically this is a labor of love and a volunteer effort. It's not reasonable for any of us to start accepting homework assignments.

Expect a minimalist version then in perhaps a few days. I do also understand that for many people in the forums, English is a second language, and I can try to be more careful to remember that when writing. It just takes a bit more time and energy, and I haven't had enough of either lately.

_________________
Six Linksys WRT1900ACSv2 (39144/40009/41954):
VLANs, multiple VAPs, NAS, QoS, client-mode travel router, OpenVPN client/PBR (AirVPN), two DNSCrypt servers (incl Quad9) routed through vpn.
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Sat Jan 11, 2020 17:52    Post subject: Reply with quote
@mkaand
so far is working OK, of course this is for router with enough memory, even you see the list is longer because i don't use many ranges, the ip ranges used before were longer than this, these ip are used globally, not only for 1 region and it's good to have more alternatives,
i suggest to try your method without the host file you use since there are many nonexistence host there and also i believe that blocks some good hosting videos in other regions, blocking *.googlevideos.com is not the best or ideal way to block ads i guess
we are sharing different methods to block youtube ads so i don't see the problem with firewall, best way to block is by IP since many different ad's host could use same ip, ADS's IP does not host good boy videos
portsup
DD-WRT User


Joined: 20 Oct 2018
Posts: 83

PostPosted: Sun Jan 12, 2020 0:16    Post subject: Reply with quote
For android use Youtube Vanced.

https://www.xda-developers.com/youtube-vanced-apk/
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Sun Jan 12, 2020 21:29    Post subject: Reply with quote
@portsup it's ok but only for android

@mkaand my method will be ok for more powerful router or maybe hardware based firewall
i restored to original default backup since was getting connections drop while using wireless , maybe because low space or memory issues, also was getting some ads unfiltered time to time, this equipment was faulty while having firewall + host
i'm using WRT54G-TM with firmware DD-WRT v3.0-r40559 mega (08/06/19)
can i use your method with this firmware ?
which router do you use ?
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Mon Jan 13, 2020 4:40    Post subject: Reply with quote
@SurprisedItWorks
a simplified version of your script will be great idea, and please, only for youtube without using other ad's host list
MrPickles
DD-WRT Novice


Joined: 14 Jan 2017
Posts: 40

PostPosted: Mon Jan 13, 2020 10:22    Post subject: Reply with quote
I've tried all three. Rickz did mention his was for Android.

No success with any for YouTube ads via Roku or AppleTV.

MP
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Mon Jan 13, 2020 14:05    Post subject: Reply with quote
@MrPickles
blocking IP list was working for me but router got bricked somehow, wireless was not working at all Sad maybe due reasons i explained, for Vizio TV with cromecast was ok and only saw some initial 10 seconds ads by youtube, can't add all ip ranges needed due to lack of this router resources, had to recover backup of all config and it's working without any script or firewall rules
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Tue Jan 14, 2020 8:02    Post subject: Reply with quote
here's another interesting discussion about youtube ads, worth checking out.

discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/435
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Sun Jan 19, 2020 19:06    Post subject: Reply with quote
@SurprisedItWorks @mkaand @MrPickles


we just can't black list the *.googlevideo.com host/ip because it's shared between other videos, this is demonstrated and commented in some other forums, so i came to the conclusion to redirect youtube video host's to other that didn't give any ads while playing, this time i used netstat to find all host used while playing a video. found like 5 or 4 *.1e100.net used, some in standby and other with connection established, tested each ip associated with those host and got the one that didn't display any ADS

you have to find out which of the cache servers don't display ads and redirect all other host like i have it in dnsmasq options

to find ip redirector 1e100.net ranges use: hxxps://www.robtex.com/cidr/172.217.19.46-24
there are many.. not only .19 range, you could test with other ranges like 172.217.21.xx etc etc

my static DNS are:
see screen shot

in the advanced routing options, just in case, i blocked google's public dns:
Quote:
8.8.8.8 and 8.8.4.4


i have seem some videos longer than 20 minutes and the video is not interrupted with ADS in the middle or end, the worst ADS are those has 3 or 4 minutes of video duration, haven't seen any for days

in Services/dnsmasq Options i have:

Quote:
address=/1e100.net/0.0.0.70
ptr-record=70.0.0.0.in-addr.arpa,"1e100.net"

address=/DD-WRT.local/192.168.1.1
ptr-record=1.1.168.192.in-addr.arpa,"DD-WRT.local"
address=/cache.google.com/172.217.19.206
ptr-record=206.19.217.172.in-addr.arpa,"cache.google.com"

address=/pagead.l.google.com/0.0.2.0
address=/gstaticadssl.l.google.com/0.0.1.1
address=/cloudfront.net/0.0.1.2
address=/clients1.google.com/0.0.0.9
address=/manifest.googlevideo.com/172.217.19.46
ptr-record=46.19.217.172.in-addr.arpa,"manifest.googlevideo.com"

address=/criteo.com/0.0.0.8
address=/ads.facebook.com/0.0.0.7
address=/adsense.com/0.0.0.6
address=/doubleclick.net/0.0.0.5
address=/googlesyndication.com/0.0.0.4
address=/googleadsserving.cn/0.0.0.3
address=/metric.gstatic.com/0.0.0.2
address=/www.googletagservices.com/0.0.0.30
address=/googleadservices.com/0.0.0.20
address=/r1---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.46
address=/r2---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.46
address=/r3---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.46
address=/r4---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.46
address=/r5---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.46
address=/r6---sn-ugpjvh-nq9s.googlevideo.com/172.217.19.46
address=/r1---sn-ugpjvh-nups.googlevideo.com/172.217.19.46
address=/r4---sn-ugpjvh-nups.googlevideo.com/172.217.19.46
address=/r5---sn-ugpjvh-nups.googlevideo.com/172.217.19.46
address=/r6---sn-ugpjvh-nups.googlevideo.com/172.217.19.46
address=/r3---sn-ugpjvh-nups.googlevideo.com/172.217.19.46
address=/r2---sn-ugpjvh-nups.googlevideo.com/172.217.19.46
address=/r3---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.46
address=/r1---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.46
address=/r2---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.46
address=/r4---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.46
address=/r5---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.46
address=/r6---sn-ugpjvh-nq9l.googlevideo.com/172.217.19.46
address=/r3---sn-ugpjvh-nupl.googlevideo.com/172.217.19.46
address=/r2---sn-ugpjvh-nupl.googlevideo.com/172.217.19.46
address=/r1---sn-ugpjvh-nupl.googlevideo.com/172.217.19.46
address=/r4---sn-ugpjvh-nupl.googlevideo.com/172.217.19.46
address=/r5---sn-ugpjvh-nupl.googlevideo.com/172.217.19.46
address=/r6---sn-ugpjvh-nupl.googlevideo.com/172.217.19.46

address=/bc.googleusercontent.com/0.0.0.60
address=/r1---sn-ugpjvh-nupz.googlevideo.com/172.217.19.46
address=/r2---sn-ugpjvh-nupz.googlevideo.com/172.217.19.46
address=/r1---sn-ugpjvh-ixme.googlevideo.com/172.217.19.46
address=/r2---sn-ugpjvh-ixme.googlevideo.com/172.217.19.46
address=/r1---sn-ugpjvh-ixml.googlevideo.com/172.217.19.46
address=/r2---sn-ugpjvh-ixml.googlevideo.com/172.217.19.46



Also from this site/info: hxxps://unix.stackexchange.com/questions/47601/how-can-i-block-a-specific-url-on-a-dd-wrt-router i got the idea to block some youtube path by keyword..maybe useless and maybe some don't work, just in case i leave them there by now..
keywords:

Quote:
/ads?
/generate_204?
/adview?
/aclk?
/qoe?
www.youtube.com/mac_204?action_fcts=1
www.youtube.com/api/stats/qoe?
www.youtube.com/api/stats/ads?
www.youtube.com/generate_204?
www.google.com/pagead/lvz?
www.google.com.do/pagead/lvz?
www.youtube.com/ptracking?
www.youtube.com/pagead/paralleladview?
www.youtube.com/csi_204?
www.youtube.com/get_midroll
www.youtube.com/pagead/conversion/?


if someone want to test don't forget to reboot router after those changes and clear your browser and dns cache from windows or other devices


Last edited by Rickz on Sun Jan 26, 2020 7:35; edited 14 times in total
Rickz
DD-WRT Novice


Joined: 01 Jan 2020
Posts: 27

PostPosted: Sun Jan 19, 2020 19:12    Post subject: Reply with quote
dnsmasq options i use
Goto page Previous  1, 2, 3, 4  Next Display posts from previous:    Page 3 of 4
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Advanced Networking All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum