Posted: Tue Feb 20, 2018 10:59 Post subject: Re: quick question from the beginner
kkdu wrote:
Thanks Yamaraj and others for testing this.
From the beginners point of reading, the steps isnt clear yet (atleast to me), or i overlooked the critical steps.
I have a router Dir868L with DDWRT v24sp-2-std build 25974. A 8GB USB is formatted as /swap and ready to be plugged-in.
My questions are
1. Do we need to ENABLE or DISABLE jffs in the router. Screenshot shows disabled.
2. If its disabled how to proceed ; do I need to create the /dnsmasq folder in /jffs path. will this work
3. If /jffs is enable, will the /dsmasq, /mphosts, /mpdomains still need to created.
4. will the UUID for mounting the partition be explicitly added or automatically detected.
@kkdu
I'll admit the instructions aren't quite beginner friendly. I hope to update them soon as I can.
1. If you already have a USB drive formatted as /jffs and swap (optionally /opt as well), then you can and should disable internal flash storage, as shown in the screenshot. Those without a pendrive have no choice but to keep it enabled.
2. If your pendrive does not have a /jffs partition, I recommend you create one. Any and all standard Linux tools should work, e.g. fdisk. You can SSH to the router and use fdisk to create /jffs. Google for details on creating a partition using fdisk. Use ext2 or ext3 for /jffs and /opt. Once it is done and your router recognizes the /jffs partition on the pendrive, you can create a directory named "dnsmasq" on partition /jffs.
Code:
cd /jffs
mkdir dnsmasq
3. Whether you use external /jffs or internal, you will need to create the directory "dnsmasq" on partition /jffs and then copy all the files inside the 7z archive to this directory. Alternatively, you could use cURL to download the files directly from Github.
4. Once you create /jffs, swap, (and optionally /opt) on your pendrive and reboot, you will have the UUIDs on Services>USB page if you have enabled USB support. You can, then, copy the UUID of /jffs to the field shown in screenshot.
In short, UUID is automatically created, but you need to explicitly tell the router to map it to a specific mount point.
Thanks Yamaraj.
Installed as instructed and it started to work as expected. Was able to enjoy the difference the way the page loads now.
I would still have couple of questions that need your input.
1. when executing the script #time ./abhostgen.sh the first time, the script runs as expected generating all hosts files and saved to the mphosts.log. I see that there are 157 ad domains blocked. However , when checking the file sizes of mpdomains and mphosts , its reported as 0B and 4kB respectively.
2. I then overwrite these files from the original to get the original sizes of 3MB and 9MB. Again executing the script, this reduces the size of these files. Is there anything im missing
3. Most sites riddled with ads are tested and are blocked successfully. ex. http://www.angelfire.com/alt2/entertainment/ad_block_test.html .However, when trying to run the same site in the Chrome incognito mode, some ads appear again. Test on FireFox - no issue. is this a behavior of Chrome and have you experienced it.
4. Youtube ads most time are killed, however, some come back. Not sure if this is related to the issue 1 where the mpdoamins and mphosts are not updated.
PS: I am also finishing the manual/documentation for the setup , will pm you once i have all things checked. if okayed, you can choose to publish for all users
1 & 2. You needn't time the script when running it. "./adbhostgen.sh" without the quotes is enough. Sounds like an issue with configuration. Check and confirm that the script and other files are in /jffs/dnsmasq directory. You can PM me with screenshots of your router config and an ls -al of /jffs/dnsmasq.
3. You may need to clear Chrome cache. Google have more than a few fallback options for serving ads even if we attempt to disable most of them. Blocking most ads without breaking functionality is what I had in mind when creating this script. As it is, hosts files and dnsmasq have their own limitations. They can't block canvas and other js ads with clever and sneaky servers.
4 & 5. Could be the problem addressed above. I'll look into that list and add servers that are missing in my blocklist. Thank you.
--
VERSION: 20180221
The script now automatically downloads latest whitelist and blacklist files from the official Github repository if not found in /jffs/dnsmasq. What it means: You no longer need to download and extract the 7z archive for the script to work. Just get the script and everything it needs to function will be downloaded automatically. If you mess up your whilelist/blacklist files, just delete them and they will be auto downloaded.
EXPERIMENTAL: a self-updating[1] strain of the official script now exists for you to play with. mpadblock.sh This version will attempt to update itself if there's a new version available on Github.
[1]: self-modifying/self-updating scripts are not POSIX compliant and the desired outcome is not guaranteed.
Posted: Sun Feb 25, 2018 1:51 Post subject: My travails
Hi Yamaraj
I finally got adblocking working a few minutes ago. I followed your instructions, and had to do a bit of digging around to get it working. I have captured my learnings hoping others will benefit too.
Preparing USB Stick
- Used 32GB USB stick for my purpose
- [Could not get fdisk to work when I login to the router via telnet]
- Downloaded and followed instructions from: http://www.techovity.com/how-to-create-ext2ext3-linux-partition-from-windows/
- Used "MiniTool Partition Wizard" to create below partitions on Win10 PC (Took a few hours!! to complete the format & partioning)
/jffs in ext2 format
/opt in ext2 format
/data in ext2 format
[Linux Swap]
- Services--> USB enabled USB
- Plugged in USB-drive but was not recognized; Router Reboot Fixed it
- Mapped created UUID to /jffs and /opt in Services--> USB (/data had no option!!)
- telnet to router (username: root; passwd: <WebUI Passwd>)
- cd /jffs
- mkdir dnsmasq
- cd dnsmasq
- [Unsuccessful in getting wget and curl to download https://raw.githubusercontent.com/m-parashar/adbhostgen/master/adbhostgen.sh ]
- vi adbhostgen.sh (copy/paste contents of the script file, and saved it)
- chmod +x adbhostgen.sh
- Run the script ...
================================================================================
Code:
root@NetgearAC1750_DDWRT:/jffs/dnsmasq# ./adbhostgen.sh
Network up. Generating the hosts file now...
Downloading / updating cURL cacert for secure communication...
Whitelist and Blacklist files not found. Downloading latest defaults...
BLITZ mode OFF.
StevenBlack list
notracking list
Cameleon list
hosts-file ad_servers list
Disconnect.me list
quidsup/notrack list
MalwareDomains list
Securemecca list
abuse.ch blocklist
adaway list
Easylist & others
Processing whitelist and blacklist files...
Size of /tmp/mphosts.tmp before formatting: 13.2M
Size of /tmp/mpdomains.tmp before formatting: 6.1M
Removing duplicates and formatting the list of domains...
Removing temporary lists...
Size of /jffs/dnsmasq/mphosts after formatting: 9.3M
Size of /jffs/dnsmasq/mpdomains after formatting: 5.0M
335827 ad domains blocked.
Restarting DNS server (dnsmasq)...
I added it to the whitelist but it is not working, do we need to reboot for this to take effect?
Thanks in advance
First things first, thanks for writing instructions. I'll update the first post with a link to yours.
Just add dickssportinggoods.com to whitelist and run the script again. It should work. _________________ Routers
NETGEAR R7800 x2: DD-WRT v3.0-r42847 std (04/06/20)
NETGEAR R8500: DD-WRT v3.0-r41517 std (11/11/19)
Also, you should be using SSH, not telnet. _________________ Routers
NETGEAR R7800 x2: DD-WRT v3.0-r42847 std (04/06/20)
NETGEAR R8500: DD-WRT v3.0-r41517 std (11/11/19)
Some of these ads are nearly impossible to quash using only hosts and domain level blocking. JS and canvas based sneaky ads often and frequently even dodge browser level ad blockers. However, I have blacklisted a few of their servers in the past two days. I recommend that you update blacklist and whitelist files, or just get the latest 7z archive and update everything.
If you have a list of hosts/domains that I or others have missed, or would prefer to include them in the official distribution, please submit them here: https://github.com/m-parashar/adbhostgen/issues
mac913 wrote:
Yamaraj wrote:
VERSION: 20180222
Added file size information for logging and debugging. Cleaned up code.
Read first post for instructions and download links.
I've jumped from the customized simple adblocker to a customized version of this script that fits my needs.
TIA for helping to filter the $hit!
So glad to hear this little script is making web and browsing experience a little more tolerable. _________________ Routers
NETGEAR R7800 x2: DD-WRT v3.0-r42847 std (04/06/20)
NETGEAR R8500: DD-WRT v3.0-r41517 std (11/11/19)
If we add a TLD (e.g. instagram.com) to the whitelist, will all subdomains be whitelisted as well? Or do we have to go through the hosts file to find every mention of *.instagram.com to add it to the whitelist?
If we add a TLD (e.g. instagram.com) to the whitelist, will all subdomains be whitelisted as well? Or do we have to go through the hosts file to find every mention of *.instagram.com to add it to the whitelist?
Thanks!
Add to whitelist...
instagram.com
and run the script again. _________________ Home Network on Telus 1Gb PureFibre - 10GbE Copper Backbone
2x R7800 - Gateway & WiFi & 3xWireGuard - DDWRT r54203 Std k6.1
Off Site 1
R7000 - Gateway & WiFi & WireGuard - DDWRT r54079 Std
E3000 - Station Bridge - DDWRT r49626 Mega K4.4
Off Site 2
R7000 - Gateway & WiFi - DDWRT r52330 Std
E2000 - Wired ISP IPTV PVR Blocker - DDWRT r35531
Posted: Sun Mar 04, 2018 21:18 Post subject: Re: Outbrain, Taboola, etc
Yamaraj wrote:
So glad to hear this little script is making web and browsing experience a little more tolerable.
This is a very welcome script and can easily be customized to your needs, THANKS! _________________ Home Network on Telus 1Gb PureFibre - 10GbE Copper Backbone
2x R7800 - Gateway & WiFi & 3xWireGuard - DDWRT r54203 Std k6.1
Off Site 1
R7000 - Gateway & WiFi & WireGuard - DDWRT r54079 Std
E3000 - Station Bridge - DDWRT r49626 Mega K4.4
Off Site 2
R7000 - Gateway & WiFi - DDWRT r52330 Std
E2000 - Wired ISP IPTV PVR Blocker - DDWRT r35531