Tutorial: How to monitor DNS traffic in real-time

Post new topic   Reply to topic    DD-WRT Forum Index -> Advanced Networking
Author Message
eibgrad
DD-WRT Guru


Joined: 18 Sep 2010
Posts: 9157

PostPosted: Sat Apr 02, 2022 20:18    Post subject: Tutorial: How to monitor DNS traffic in real-time Reply with quote
The following script allows for real-time monitoring of DNS on the router for the purposes of knowing what DNS servers are in use, and which network interfaces are being used by those DNS servers.

ddwrt-dns-monitor.sh

Overview

One of the most difficult aspects of the router for users is managing DNS. DNS configuration is distributed across the GUI, involving the WAN (w/ ISP and custom options), DNSCrypt, DHCP server (with its own custom servers), the VPNs, etc. Trying to understand exactly how DNS is being handled becomes quite difficult, esp. for those less familiar w/ the underlying system. It's darn near impossible to just look at GUI and all the relevant configuration pages and be sure it works the way you assume it works. There's a good chance you're wrong! And things can change over time, so what appears correct NOW may not be the case 30 minutes or an hour from now.

I'm aware that many ppl depend on online DNS leak testing tools for these purposes. But I don't trust them. Having a third-party *reliably* determine the actual DNS server(s) in use seems a near impossible task. And these websites rarely if ever explain their methodology. I assume it involves some Javascript "trickery" and/or WebRTC (which many ppl keep disabled, myself included). So you just have to take it on faith it's correct (good or bad results). Frankly, I can't understand how it could be accurate given the LAN clients are accessing DNS via the router's proxy, DNSMasq (at least by default).

That's why I threw together this simple utility.

How It Works

The most accurate means to determine which DNS servers are in use by the router, and how they are being routed, is to use connection tracking.

Code:
egrep 'dport=(53|853) ' /proc/net/nf_conntrack


The above one-liner will do more to keep you *accurately* informed about what's happening on the router than all the online DNS leaking testing tools put together.

The connection tracking system is constantly being updated w/ *all* the traffic from the LAN clients and the router. That's how the router maps outgoing traffic w/ incoming replies (and vice versa). NOTHING is more reliable than this part of the subsystem for such purposes.

All my script does is just put some fancy (but still useful) ornamentation around this one-liner. It's now easier to determine when you have a DNS leak, even if you don't understand connection tracking (which anyone w/ an interest in this topic should investigate; it's NOT difficult).

Compatibility

As of this moment, the script only supports IPv4, NOT IPv6. I may be able to add IPv6 support in the future, but I just don't have access to it at the moment.

Installation

The script installs into /tmp by default, since it's not expected the utility will be used on a regular basis, but just for temporary diagnostic purposes. But if you want to have it available and/or running permanently, you could install it to /jffs or wherever you prefer.

For immediate execution and monitoring (accepting all the defaults) ...

Code:
curl -kLs -H 'Cache-Control: no-cache' pastebin.com/raw/i37RqaSJ | tr -d '\r' | sh


Alternatively, to download the script and make changes ...

Code:
script=/tmp/tmpfile.$$.sh
curl -kLs -H 'Cache-Control: no-cache' pastebin.com/raw/i37RqaSJ | tr -d '\r' > $script
chmod +x $script
vi $script


Then execute the script to begin monitoring.

Code:
$script


Since the script is installed in /tmp, it will NOT survive a reboot.

Definitions

I've never found consensus on the definition of a DNS leak. So for the purpose of this post and the utility, here is *my* definition. A DNS leak is anytime you're using the ISP's DNS server(s), or any other DNS servers which the ISP can either eavesdrop on, or worse, hijack back to his own servers. Given this definition, simply avoiding the ISP's DNS server(s) is NOT sufficient, if all you end up doing is routing your alternative DNS server(s) (e.g., Cloudflare) "in the clear" over the WAN. OTOH, DNSCrypt is NOT considered a DNS leaks since it's encrypted, even if routed over the WAN.

I realize others may quibble w/ that definition, but again, for the purposes discussed herein, those are the underlying assumptions.

Understanding the Results

The output is color-coded to make identification of problematic connections more obvious to the untrained eye. For the most part, the only connections you need to concern yourself w/ are those in RED, since they represent traditional Do53 (in the clear) queries sent over the WAN. But NOT all such connections are necessarily a problem. For example, if you're double NAT'd and the router is getting *local* name resolution from a *local* upstream DNS server, this is probably safe. OTOH, if your landlord is offering his own internet connection to you on his private network, perhaps that's a concern. But the script has no way to know one way or the other. YOU have to properly interpret the results in that case.

I also include other relevant information in the header to make interpretation easier, such as WAN/LAN ip, DNS configuration on the WAN vs. LAN (DHCP server), DNSCrypt, configuration of the OpenVPN and/or WireGuard clients, etc.

As the system configuration changes (e.g., due to the starting/stopping of OpenVPN or WireGuard clients), the output will change accordingly. You may find the LAN clients and/or router using different DNS servers and/or network interfaces. But beware; the old connections will NOT disappear instantaneously! They'll persist for a short period (perhap several minutes) until they're finally starved of all activity. So don't jump to the conclusion that something isn't working properly until you give the changes time to settle.

Practical Limitations

In order to keep the display limited to the available screen size (or as best possible), the script ignores duplicates. IOW, what you're seeing on the screen is NOT all the connections from a given LAN client or the router, but only those that are unique, since the purpose of the script is not to monitor the *amount* of DNS activity, but simply how it's being handled. Even so, you'll probably have to expand your terminal window vertically to prevent the header from scrolling off the screen if you have a particularly busy system.

Preserving Results for Later Review

I provided a logging option (disabled by default) to record these problematic connections. But in order to keep the file a manageable size, it too ignores duplicates. If you hit Ctrl-C to exit the utility, it will report the log file by name (since it varies each time you run it). You can easily browse it using the less command.

Alternative Persistent Setup

If you wish to have the utility run continuously, even when you log off ssh, I recommend installing Entware and including the screen utility.

Code:
opkg install screen


The screen utility allows you to run the script in the background, detached from your current ssh session.

Code:
/opt/sbin/screen -dmS dnsmon </path/script-name>


You can then reattach to the running script at any time, from any ssh session, on any client machine!

Code:
/opt/sbin/screen -r dnsmon


And detach w/ Ctrl-A D.

You could even configure the startup script to call the screen utility to have DNS monitoring (and logging) available at the earliest possible moment after bootup!

Code:
(
until [ -f /opt/sbin/screen ]; do sleep 10; done
/opt/sbin/screen -dmS dnsmon /jffs/ddwrt-dns-monitor.sh
) &


Differences w/ AsusWRT-Merlin Version

As some of you may already know, I've been supporting a version of this same script on the AsusWRT-Merlin forum since January 30, 2022. I strongly urge you to visit that forum if you have questions, since many of them have probably already been addressed there. Functionally, there isn't all that much difference. The two most obvious differences are 1) the two platforms support different secure DNS options (e.g., AsusWRT-Merlin supports DoT, whereas DD-WRT support DNSCrypt) and 2) different VPN options (e.g., there is no WireGuard support on AsusWRT-Merlin), which can result in slightly different header information. Less obvious, only the DD-WRT version of the script works in a bridged configuration (in that case, the WAN ip identified in the script is actually the bridge's default gateway).

Menu Options

Most are self-explanatory, but here's a bit more detail for specific options.

The interval ('i') option has a min/max value of 3/30 secs. Anything outside those boundaries would likely either overtax the router or produce misleading results.

By default, if the number of lines of output exceed the vertical limits of the display, it will require responding to the More prompt to continue. You can disable that behavior by enabling the scroll ('s') option.

The dupes ('d') option hides/shows the duplicates count for a given connection. As I said previously, the fact there are duplicates is NOT necessary/relevant to understanding how DNS is being handled, but I included it for those who might find it useful/meaningful for other less obvious diagnostic purposes.

Feedback

I'm happy to take questions, listen to feedback, and consider enhancements on either forum. All I ask is that you check the AsusWRT-Merlin forum before asking here since there's a good chance it's been asked and answered. I'm just trying to avoid repeating myself unnecessarily. Of course, if you have DD-WRT specific issues, by all means, ask/discuss it here.

Enjoy!

_________________
ddwrt-ovpn-split-basic.sh (UPDATED!) * ddwrt-ovpn-split-advanced.sh (UPDATED!) * ddwrt-ovpn-client-killswitch.sh * ddwrt-ovpn-client-watchdog.sh * ddwrt-ovpn-remote-access.sh * ddwrt-ovpn-client-backup.sh * ddwrt-mount-usb-drives.sh * ddwrt-blacklist-domains.sh * ddwrt-wol-port-forward.sh * ddwrt-dns-monitor.sh (NEW!)


Last edited by eibgrad on Sat Apr 02, 2022 20:49; edited 2 times in total
Sponsor
eibgrad
DD-WRT Guru


Joined: 18 Sep 2010
Posts: 9157

PostPosted: Sat Apr 02, 2022 20:51    Post subject: Reply with quote
Reserved for future use.
_________________
ddwrt-ovpn-split-basic.sh (UPDATED!) * ddwrt-ovpn-split-advanced.sh (UPDATED!) * ddwrt-ovpn-client-killswitch.sh * ddwrt-ovpn-client-watchdog.sh * ddwrt-ovpn-remote-access.sh * ddwrt-ovpn-client-backup.sh * ddwrt-mount-usb-drives.sh * ddwrt-blacklist-domains.sh * ddwrt-wol-port-forward.sh * ddwrt-dns-monitor.sh (NEW!)
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 12812
Location: Netherlands

PostPosted: Sun Apr 03, 2022 7:35    Post subject: Reply with quote
Many thanks a very useful addition for DDWRT.

I will make this a sticky in this forum.

_________________
Routers:Netgear R7000, R6400v1, R6400v2, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800/XR500: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
ho1Aetoo
DD-WRT Guru


Joined: 19 Feb 2019
Posts: 2899
Location: Germany

PostPosted: Sun Apr 03, 2022 17:41    Post subject: Reply with quote
thank you very much!

tested and found to be good

very well implemented and very helpful

(a bit OT : but I still don't understand what problems some people have with "strict-order" - it works fine for me and as it should)
eibgrad
DD-WRT Guru


Joined: 18 Sep 2010
Posts: 9157

PostPosted: Sun Apr 03, 2022 18:04    Post subject: Reply with quote
ho1Aetoo wrote:
(a bit OT : but I still don't understand what problems some people have with "strict-order" - it works fine for me and as it should)


Like so much when it comes DNS, it gets complicated.

The issue w/ strict-order is relevant to the OpenVPN client. On certain platforms like Merlin and FT (FreshTomato), you have the option to use Strict mode for configuring DNS w/ the OpenVPN client(s), which maps down to use of the strict-order directive in DNSMasq. And this is usually implemented by combining the ISP's DNS servers (or custom servers if you choose to ignore the ISP's DNS servers) w/ those of the VPN provider. They are ordered in such a way as to make sure the VPN's DNS servers are used *first* (and ideally, exclusively). But it doesn't work. Given enough time and activity, eventually ALL the available DNS servers will be used, leading to DNS leaks.

DD-WRT can be configured to behave similarly by enabling the strict-order directive w/ DNSMasq when using the OpenVPN clients. And would normally suffer from the same problem of DNS leaks. But @egc some time ago changed the OpenVPN clients to *remove* the ISP's DNS servers (or custom servers) from DNSMasq, leaving only the VPN provider's DNS servers, hence no DNS leaks.

_________________
ddwrt-ovpn-split-basic.sh (UPDATED!) * ddwrt-ovpn-split-advanced.sh (UPDATED!) * ddwrt-ovpn-client-killswitch.sh * ddwrt-ovpn-client-watchdog.sh * ddwrt-ovpn-remote-access.sh * ddwrt-ovpn-client-backup.sh * ddwrt-mount-usb-drives.sh * ddwrt-blacklist-domains.sh * ddwrt-wol-port-forward.sh * ddwrt-dns-monitor.sh (NEW!)
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 12812
Location: Netherlands

PostPosted: Sun Apr 03, 2022 19:34    Post subject: Reply with quote
It is exactly like @eibgrad said.
The described problems became even worse when DNSMasq 2.85 came into production.

Problems with strict order are also worse when using PBR and the vpn DNS servers are routed via the VPN and other DNS servers are using the much faster WAN.

Bottom line regarding DNS leaks you can not rely on strict order.
So that was indeed changed in WireGuard and OpenVPN (with help and advice from @eibgrad).

So when using DDWRT you should not experience DNS leaks if and that is a big if you setup correctly and the VPN provider is pushing the correct things.
So despite our best effort @eibgrads utility is very welcome and useful to check for DNS leaks.

_________________
Routers:Netgear R7000, R6400v1, R6400v2, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800/XR500: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
eibgrad
DD-WRT Guru


Joined: 18 Sep 2010
Posts: 9157

PostPosted: Wed Apr 27, 2022 4:58    Post subject: Reply with quote
UPDATE: Version 1.1.0 is now available.

1. Due to a parsing error, TCP connections were NOT being displayed, only UDP. While TCP is used much less than UDP for DNS, it would still be worth using the latest script and rechecking for DNS leaks.

2. Due to a parsing error, the option to hide the public IP didn't always work (very obscure error only likely to occur when the router was in a bridged configuration).

_________________
ddwrt-ovpn-split-basic.sh (UPDATED!) * ddwrt-ovpn-split-advanced.sh (UPDATED!) * ddwrt-ovpn-client-killswitch.sh * ddwrt-ovpn-client-watchdog.sh * ddwrt-ovpn-remote-access.sh * ddwrt-ovpn-client-backup.sh * ddwrt-mount-usb-drives.sh * ddwrt-blacklist-domains.sh * ddwrt-wol-port-forward.sh * ddwrt-dns-monitor.sh (NEW!)
idlehands
DD-WRT Novice


Joined: 09 Nov 2012
Posts: 12

PostPosted: Sat Jun 04, 2022 14:15    Post subject: Thank You! Reply with quote
Thanks eibgrad and egc for the killer user friendly script and loads of learnin'! DNS has made my head spin since 'nam; believe me when I say there's not enough residual agent orange in Lieutenant Dan's foxhole to clear that foliage. Without you guys and others volunteer networkers like you, a networking neophyte such as myself would leave behind a digital footprint that would make Sasquatch blush. Embarassed

Thank you for helping us maintain a toehold on our privacy in the mass surveillance age.

_________________
Netgear R7000 Updated ≈ Monthly
Wireguard, PBR, VAP
Adblocking & Authoritative, Validating, Recursive Caching DNS Server with DNSSEC via Unbound Verified with ddwrt-dns-monitor.sh and dig

Tutorial: How to monitor DNS traffic in real-time
Display posts from previous:    Page 1 of 1
Post new topic   Reply to topic    DD-WRT Forum Index -> Advanced Networking All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum