how to use oisd.nl/downloadsXtra for matching / blocking

Post new topic   Reply to topic    DD-WRT Forum Index -> Advanced Networking
Goto page 1, 2  Next
Author Message
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Wed Oct 27, 2021 23:42    Post subject: how to use oisd.nl/downloadsXtra for matching / blocking Reply with quote
Title: how to use https://oisd.nl/downloadsXtra for matching the blocking on DDWRT of yourself. And it is fastest and simple, stable.

(Nothing related with keweon DNS)

Verified Device: Netgear R6250
Key Hardware Info: 800MHZ dual core with 256M memories and 128M flash
Firmware : DD-WRT v3.0-r47596 std (10/25/21)
Kernal : Linux 4.4.289 #4369 SMP Mon Oct 25 03:56:19 +07 2021 armv7l
Mode: Gatewat, wireless, unbound (no needs USB disk insert), enable CTF & FA.
WAN speed : 110M ordered and stable.

I prefer unbound to avoid ... anyway, you can try DNSCrypt in low-level DDWRT.
(if you follow me with unbound, you must config unbound 1st to verify it works BEFORE any script in *.sh and additional dnsmasq options !
Code:

root@DD-WRT:/tmp# unbound-checkconf /jffs/etc/unbound.conf
unbound-checkconf: no errors in /jffs/etc/unbound.conf


DDWRT->Setup->Basic
Ignore WAN DNS
Shortcut Forwarding Engine CTF
Flow Acceleration CTF & FA
STP Enable
Use DNSMasq for DNS Enable
DHCP-Authoritative Enable
Recursive DNS Resolving (Unbound) Enable
Forced DNS Redirection Enable
NTP server IP 162.159.200.123, 216.239.35.0 # it is VIP before using unbound, that is time.cloudflare and time.google.com

DDWRT->Services->Services
SmartDNS Disable
Dnsmasq Enable
Encrypt DNS Disable
Cache DNSSEC data Disable
Validate DNS Replies (DNSSEC) Disable
Check unsigned DNS replies Disable
No DNS Rebind Enable
Query DNS in Strict Order Enable
Add Requestor MAC to DNS Query Disable
RFC4039 Rapid Commit support Enable
Maximum Cached Entries 1500
Additional Dnsmasq Options:
Code:
# conf-file=/jffs/etc/E # comment this option if you dislike the extra blocking; uncomment this option only when /jffs/etc/E ready !
addn-hosts=/tmp/B
domain-needed
bogus-priv
no-negcache
dhcp-option=43,01:04:00:00:00:02
dhcp-host=00:11:22:33:44:55,192.168.1.6,infinite # if you want to lock one special device with fixed IP

Turning off radio Enable
SSHd Enable
SSH TCP Forwarding Disable
Password Login Enable
Port 22
Syslogd Enable # if cannot surf, the trouble is that NTP servers both cloudflare and time.google downing! So you can be easy to visit https://192.168.1.1/Syslog.asp focusing contents BackGround in RED

DDWRT->Administration->Management
Additional Cron Jobs
Code:
0 6 * * * root /jffs/etc/config/B127001.startup

Internal Flash Storage Enable

root@DD-WRT:~# chmod +x /jffs/etc/config/B127001.startup
Code:
root@DD-WRT:~# cat /jffs/etc/config/B127001.startup
#!/bin/sh
curl -sS -L --compressed "https://dnsmasq.oisd.nl" | sed 's/address=\//127.0.0.1 /;s/\/#//;/^#/d;/^$/d;/your_favour_keeping.web/d;/.facebook.com$/d' > /tmp/B
killall -HUP dnsmasq

# My explanation: that is full list about 372,200 domains, 12Mib, Must not be 0.0.0.0 here, allow your *.facebook.com keep on working, daily schedule updating in mem only, very fast almost full for you !

root@DD-WRT:~# chmod +x /jffs/E0000.sh # the E is only an option, if you dislike, no need to keep it and remove conf-file=/jffs/etc/E in Additional Dnsmasq Options
root@DD-WRT:~# cat /jffs/E0000.sh
Code:
#!/bin/sh
clear
echo " Download EXTRA (Danger! need2manuel once a month because save in internal ) DNSMasq hosts (<9K) from oisd.nl/extra then replace all /# with /0.0.0.0 then delete all comment lines or empty "
echo " Reference additional Dnsmasq Options: conf-file=/jffs/etc/E"
curl -sS -L --compressed "https://dnsmasq.oisd.nl/extra" | sed 's/\/#/\/0.0.0.0 /;s/\/#//;/^#/d;/^$/d' > /tmp/E
echo -n " Total domains ( setas 0.0.0.0 ) : $(cat /tmp/E | wc -l -c)"
cat /jffs/etc/myExtra.defined >> /tmp/E
echo " -=>>> DONE."
mv -f /tmp/E /jffs/etc/
killall -HUP dnsmasq

# My explaination here: 127.0.0.1 completed equal with 0.0.0.0 verified in E! myExtra.defined is your private ALWAYS block, whose format is like "address=/jsc.mgid.com/0.0.0.0"

# My special explanation here: do not try to use block.conf in unbound.conf, I found it will be UNEXPECTED ...
root@DD-WRT:~# cat /jffs/etc/unbound.conf
Code:
server:
verbosity: 1
interface: 0.0.0.0@7053
tls-cert-bundle: "/etc/ssl/ca-bundle.crt"
outgoing-num-tcp: 16
incoming-num-tcp: 16
msg-buffer-size: 8192
msg-cache-size: 1m
num-queries-per-thread: 30
rrset-cache-size: 2m
infra-cache-numhosts: 400
username: ""
pidfile: "/var/run/unbound.pid"
root-hints: "/etc/unbound/named.cache"
target-fetch-policy: "2 1 0 0 0 0"
harden-short-bufsize: yes
harden-large-queries: yes
key-cache-size: 100k
neg-cache-size: 10k
so-reuseport: yes
num-threads: 2
msg-cache-slabs: 2
rrset-cache-slabs: 2
infra-cache-slabs: 2
key-cache-slabs: 2
outgoing-range: 462
access-control: 127.0.0.0/8 allow
access-control: 192.168.1.1/24 allow
local-data: "localhost A 127.0.0.1"
local-data: "DD-WRT A 192.168.1.1"
cache-max-ttl: 14400
cache-min-ttl: 1200
prefetch: yes
prefetch-key: yes
minimal-responses: yes
serve-expired: yes
serve-expired-ttl: 43200
ip-ratelimit: 0
so-rcvbuf: 256k
udp-upstream-without-downstream: yes
hide-identity: yes
hide-version: yes
do-not-query-localhost: no
qname-minimisation: yes
harden-glue: yes
harden-below-nxdomain: yes
rrset-roundrobin: yes
aggressive-nsec: yes
deny-any: yes
auto-trust-anchor-file: "/etc/unbound/root.key"
python:
remote-control:
forward-zone:
name: "."
forward-tls-upstream: yes
forward-addr: 9.9.9.9@853#dns.quad9.net         # you can choose anyone or combination listed, consider law and privacy, quad9 maybe a better even slow than cloudflare.
forward-addr: 149.112.112.112@853#dns.quad9.net # 2nd of quad9
forward-addr: 1.1.1.1@853#cloudflare-dns.com    # cloudflare is fastest than others in my case
# forward-addr: 176.103.130.130@853#dns.adguard.com # if your DDWRT performance is pool or you are lazy to do above complex ... you can try only adguardDNS, but it seems of Russia ?
# forward-addr: 176.103.130.131@853#dns.adguard.com
# # forward-addr: 8.8.8.8@853#dns.google
# # forward-addr: 8.8.4.4@853#dns.google

auth-zone:
name: "."
master: 192.203.230.10
master: 192.5.5.241
master: 192.33.4.12
fallback-enabled: yes
for-downstream: no
for-upstream: yes


Best wish all of you enjoy!
Sponsor
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Thu Oct 28, 2021 3:19    Post subject: Reply with quote
Sorry, there is a bug.

Please use two command both "stopservice dnsmasq" and "startservice dnsmasq" instead of original "killall -HUP dnsmasq" to work with updating B and/or E immediately.
the-joker
DD-WRT Developer/Maintainer


Joined: 31 Jul 2021
Posts: 2146
Location: All over YOUR webs

PostPosted: Thu Oct 28, 2021 9:13    Post subject: Reply with quote
Hello saphirely =)

Thank you for sharing your setup how-to

This posts was moved to advanced networking instead.

Regarding the bug you mentioned, feel free to edit your first post and replace killall -HUP dnsmasq with the stop/start service you posted.

Thanks again for your contribution.

Be well.

_________________
Saving your retinas from the burn!🔥
DD-WRT Inspired themes for routers
DD-WRT Inspired themes for the phpBB Forum
DD-WRT Inspired themes for the SVN Trac & FTP site
Join in for a chat @ #style_it_themes_public:matrix.org or #style_it_themes:discord

DD-WRT UI Themes Bug Reporting and Discussion thread

Router: ANus RT-AC68U E1 (recognized as C1)
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Thu Oct 28, 2021 18:53    Post subject: Reply with quote
@the-joker
DD-WRT Developer/Maintainer

Thank you!
But I found new solution for facebook, how ever, I cannot post here, always rejected...
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Thu Oct 28, 2021 18:54    Post subject: Reply with quote
Code:

root@DD-WRT:~# cat /jffs/E0000.sh
#!/bin/sh
clear
echo " Download EXTRA (Danger! need2manuel once a month because save in internal ) DNSMasq hosts (<9K) from oisd.nl/extra then replace all /# with /0.0.0.0 then delete all comment lines or empty "
echo " Reference additional Dnsmasq Options: conf-file=/jffs/etc/E"
curl -sS -L --compressed "https://dnsmasq.oisd.nl/extra" | sed 's/\/#/\/0.0.0.0 /;s/\/#//;/^#/d;/^$/d;/address=\/graph.facebook.com/d' > /tmp/E
echo
echo -n " Total domains ( setas 0.0.0.0 ) : $(cat /tmp/E | wc -l -c)"
cat /jffs/etc/myExtra.defined >> /tmp/E
echo " -=>>> DONE."
mv -f /tmp/E /jffs/etc/
stopservice dnsmasq
startservice dnsmasq

# Here ";/address=\/graph.facebook.com/d" is the key for Facebook App, if included it, you can touch without too much white holes...
# Bonus ! if U are using some/unknown/oem cheap(?) mobile phone, U'd better check https://192.168.1.1/Syslog.asp focusing contents BackGround in YELLOW, U will find some special communication UNEXPECTED, then U can block in myExtra.defined, for exmaple: address=/tclclouds.com/0.0.0.0
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Thu Oct 28, 2021 18:56    Post subject: Reply with quote
Do not import ".facebook.com" in /jffs/etc/config/B127001.startup, it does not work.
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 14126
Location: Texas, USA

PostPosted: Thu Oct 28, 2021 19:34    Post subject: Reply with quote
What was rejected? Any tips and tricks are always welcome; unless, of course, they are already covered and up-to-date in a wiki article or other post Cool

Always have to pay attention to the forest, not the trees.

_________________
"Life is but a fleeting moment, a vapor that vanishes quickly; All is vanity"
Contribute To DD-WRT
Pogo - A minimal level of ability is expected and needed...
DD-WRT Releases 2023 (PolitePol)
DD-WRT Releases 2023 (RSS Everything)

----------------------
Linux User #377467 counter.li.org / linuxcounter.net
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Thu Oct 28, 2021 19:44    Post subject: Reply with quote
what I posted three parties above should be in one, how ever, my firefox (V93.0) met so many blocked from original content, I do not know why, so wasted three parties. Please do not mis-understand.
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 14126
Location: Texas, USA

PostPosted: Thu Oct 28, 2021 20:04    Post subject: Reply with quote
Ah. Well, if I were able to fix this, I would if asked. But I cannot edit your posts.
_________________
"Life is but a fleeting moment, a vapor that vanishes quickly; All is vanity"
Contribute To DD-WRT
Pogo - A minimal level of ability is expected and needed...
DD-WRT Releases 2023 (PolitePol)
DD-WRT Releases 2023 (RSS Everything)

----------------------
Linux User #377467 counter.li.org / linuxcounter.net
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Thu Oct 28, 2021 23:11    Post subject: Reply with quote
if you follow my solution and met Google Playstore cannot update your Apps, you need to uncomment this line "address=/www-google-analytics.l.google.com/0.0.0.0" in /jffs/etc/E.

Or

Code:

root@DD-WRT:~# cat /jffs/E0000.sh
#!/bin/sh
clear
echo " Download EXTRA (Danger! need2manuel once a month because save in internal ) DNSMasq hosts (<9K) from oisd.nl/extra then replace all /# with /0.0.0.0 then delete all comment lines or empty "
echo " Reference additional Dnsmasq Options: conf-file=/jffs/etc/E"
curl -sS -L --compressed "https://dnsmasq.oisd.nl/extra" | sed 's/\/#/\/0.0.0.0 /;s/\/#//;/^#/d;/^$/d;/address=\/graph.facebook.com/d;/address=\/address=/www-google-analytics.l.google.com/d' > /tmp/E
echo
echo -n " Total domains ( setas 0.0.0.0 ) : $(cat /tmp/E | wc -l -c)"
cat /jffs/etc/myExtra.defined >> /tmp/E
echo " -=>>> DONE."
mv -f /tmp/E /jffs/etc/
stopservice dnsmasq
startservice dnsmasq


That's all.
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Sat Oct 30, 2021 18:19    Post subject: Reply with quote
Sorry, pasted wrong above.
It should be
Code:
curl -sS -L --compressed "https://dnsmasq.oisd.nl/extra" | sed 's/\/#/\/0.0.0.0/;s/\/#//;/^#/d;/^$/d;/address=\/graph.facebook.com/d;/address=\/www-google-analytics.l.google.com/d' > /tmp/E


At last, is there anyone can help me to boot and make sure the dnsmasq working normal while dnsmasq extra option conf-file=/tmp/E which it is not exist during the beginning of boot? I hope it can be added later like /tmp/B
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Sun Nov 07, 2021 21:05    Post subject: Reply with quote
Follow up...
The above solution is not stable enough, yes, verified.
Because there will be Memory->Buffers less than 800K while Free over 100M after a long time of using.
So it is recommended that users observe http(s)://192.168.1.1/Info.htm, private suggestion: restart daily by Admin->Keep Alive-> Schedule Reboot->Enable, At a set time ....
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Sun Nov 07, 2021 21:07    Post subject: Reply with quote
Or you may choose adhole.org instead of quad9.net
Code:

forward-zone:                                                                                                                                                                       
name: "."
forward-tls-upstream: yes
forward-addr: 107.155.83.188@853#us-central.adhole.org
forward-addr: 194.124.76.14@853#us-east.adhole.org
# forward-addr: 9.9.9.9@853#dns.quad9.net
# forward-addr: 149.112.112.112@853#dns.quad9.net

That is ADhole.org in USA, you can decide by your location.
I found that is stabler but losing private controlling, maybe it is better to block on unbound.conf instead of blocking in dnsmasq, which will force to be equipped on /jffs/etc/blockhost.conf and will be shorten the life if daily midnight updating on internal storage. I dislike to insert any USB disk...
saphirely
DD-WRT User


Joined: 13 Dec 2020
Posts: 266

PostPosted: Sun Nov 07, 2021 21:08    Post subject: Reply with quote
One more thing is that NTP Server format is "IP1 IP2" not "IP1,IP2" so that I choose 78.138.17.129 216.239.35.0 now.
Also you can ssh then try to
Code:
nvram show | grep ntp
to find yours. especially ntp_timer=3600 1hour too shorter, you can expand to like 3600*24*N days then
Code:
submit nvram
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 14126
Location: Texas, USA

PostPosted: Sun Nov 07, 2021 23:46    Post subject: Reply with quote
Are you sure it's not 'nvram commit'....
_________________
"Life is but a fleeting moment, a vapor that vanishes quickly; All is vanity"
Contribute To DD-WRT
Pogo - A minimal level of ability is expected and needed...
DD-WRT Releases 2023 (PolitePol)
DD-WRT Releases 2023 (RSS Everything)

----------------------
Linux User #377467 counter.li.org / linuxcounter.net
Goto page 1, 2  Next Display posts from previous:    Page 1 of 2
Post new topic   Reply to topic    DD-WRT Forum Index -> Advanced Networking All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum