route traffic by domain with dnsmasq and ipset

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Advanced Networking
Goto page Previous  1, 2
Author Message
James2k
DD-WRT Guru


Joined: 23 Oct 2011
Posts: 549

PostPosted: Sat Dec 17, 2016 14:06    Post subject: Reply with quote
Thanks for the guide, it helped me implement some ipset policies via Dnsmasq which need to bypass a VPN connection.

I've noticed with Netflix however that it appears the initial domain list provided is enough to prevent the dreaded "VPN/Proxy error", but streaming errors can still occur with the following error codes Q8226 and Q8227 respectively.

I observed streams getting past the 7% stage (usually when any proxy or VPN detection occurs), hang on 20% and then error with the above codes (mostly Q8226), labelled as "Unknown error has occurred". I contacted Netflix and they informed me that error is specific to routing, which makes sense. They aren't in a position to troubleshoot knowing a VPN is in use (though I did explain I am ensuring traffic DOES NOT use the VPN), but they did confirm that my WAN IP was visible on my account/connection logs, so the traffic was being routed through the WAN. Though a traceroute will confirm this also.

What appears to be the problem is there are some netflix domains that may not initially be part of the policy but subsequently get added after the error occurs, as IP addresses get added as requests are made in some cases. It seems that Netflix might be seeing that, some connections are from my WAN IP, while others are not (initially), and that's probably why the app detects something strange is going on.

Streaming another item immediately after the error again works no problem, likely because the ipset policy now has the IPv4 address that was previously not going through the WAN, I guess the initial domain policy might need to be expanded to prevent this error completely.

This is what I'm using currently:

Code:
ipset=/netflix.net/netflix.com/nflxext.com/nflximg.com/nflxvideo.net/amazonaws.com/NETFLIX

_________________
James

Main router:

Netgear R7000 overclocked to 1.2GHz - DD-WRT v3.0-r35965M kongac

IPv6 6in4 (HE.net), OpenVPN (with PBR and split tunnelling), Entware, dnsmasq with ipset

Easy ipset support for the R7000

VPN speed: Download: 77.96 Mbps Upload: 5.00 Mbps (AES-128-CBC HMAC-SHA1)

Yes you can get 50 Mbps+ with OpenVPN on a R7000 if you configure it properly!

Previous routers:

ASUS RT-N66U - The Dark Knight
WNR2000v3 - Bought on the cheap for someone else, neutered crap
WNR3500Lv1 - First venture into the DD-WRT world
Sponsor
batteryhorsestaple
DD-WRT Novice


Joined: 31 Dec 2016
Posts: 1

PostPosted: Sat Dec 31, 2016 0:37    Post subject: Reply with quote
Hello,

Thanks for the interesting walkthrough & sorry to bump an old thread, this is still the best resource for this solution that I can find.

I have a Maxwell-based Linksys WRT1900ACS(V2), so (I think that's why) the files don't work for me. Can anyone please give me a hand getting this to work? At the moment I get:

At the moment terminal hangs on this command
Quote:
insmod /jffs/usr/lib/modules/xt_set.ko


Router NameDD-WRT
Router ModelLinksys WRT1900ACS
Firmware Version DD-WRT v3.0-r30796 std (10/25/16)
Kernel VersionLinux 3.18.42 #102 SMP Fri Oct 14 01:08:44 CEST 2016 armv7l
James2k
DD-WRT Guru


Joined: 23 Oct 2011
Posts: 549

PostPosted: Sat Dec 31, 2016 9:15    Post subject: Reply with quote
Its a bit of a pain. You need to compile the xt_set module with the correct toolchain for your router against the kernel sources of the build you are running because its not included by default.

In your case you would need to use the Linux 3.18 kernel source based on the info in your post:

Shameless, plug, I did write a very basic guide a while ago for compiling IPv6 modules for DD-WRT, back when the K3.x builds were first arriving on the scene, but the same principles apply for xt_set.

https://blog.jmwhite.co.uk/2013/08/10/compiling-ipv6-modules-from-source-for-dd-wrt

- All mips references will need to be replaced by arm equivalents, as this was written for a MIPSEL ASUS RT-N66U.

You'll ideally need a Linux machine to checkout the SVN source for the Linux kernel source and go from there in order to compile the module. You then need move the compiled module onto your router, usually JFFS is a good place and then insmod, unfortunately, DD-WRT doesn't have modprobe so it gets tricky to debug if the module fails or crashes.

_________________
James

Main router:

Netgear R7000 overclocked to 1.2GHz - DD-WRT v3.0-r35965M kongac

IPv6 6in4 (HE.net), OpenVPN (with PBR and split tunnelling), Entware, dnsmasq with ipset

Easy ipset support for the R7000

VPN speed: Download: 77.96 Mbps Upload: 5.00 Mbps (AES-128-CBC HMAC-SHA1)

Yes you can get 50 Mbps+ with OpenVPN on a R7000 if you configure it properly!

Previous routers:

ASUS RT-N66U - The Dark Knight
WNR2000v3 - Bought on the cheap for someone else, neutered crap
WNR3500Lv1 - First venture into the DD-WRT world
Xentrk
DD-WRT Novice


Joined: 03 Jun 2016
Posts: 45

PostPosted: Wed Mar 01, 2017 0:56    Post subject: Reply with quote
Router Model: Dlink-DIR880L
Firmware Version: DD-WRT v3.0-r29837 std (06/06/16)
Kernel Version: Linux 4.4.12 #883 SMP Fri Jun 3 13:48:18 CEST 2016 armv7l

I also support some ASUS RT-AC88U routers running Merlin’s Firmware 380.64_2 release. Swetoast in snbforums.com has written some malware-filter and privacy-filter scripts using ipset that I want to run on the DD-WRT router listed above. But I need ipset to run the scripts. On an attempt to run the malware-filter script, I did see this error:

Code:

insmod: ip_set.ko: module not found
insmod: ip_set_hash_net.ko: module not found
insmod: ip_set_hash_ip.ko: module not found
insmod: xt_set.ko: module not found


I did follow the instructions in the OP and had mixed results. The first and subsequent attempts stopped DHCP Service from running. I have some ad blocking files in the Additional DNSMasq Options as follows:

conf-file=/jffs/dns/dnsmasq.adblock.conf
addn-hosts=/jffs/dns/dlhosts
conf-file=/jffs/dns/personal-ads-list.conf

When issuing the
Code:

insmod /jffs/usr/lib/modules/xt_set.ko

command, It would hang at the command prompt. Sometimes, I would then lose access to the webgui too, as others have described. I found the DHCP service had stopped. I then had to set a static IP in my client to reconnect. If I lost webgui, I would issue the command in a SSH session:
Code:

httpd -p 80 or httpd -h /www and sometimes the command reboot at the command prompt.

What I found is many strange (hex?) characters where inserted into one or both of the .conf files when trying to insmod xt_set.ko. When I removed the reference to the files in Additional DNSMasq Options, I could get DHCP service back up. Sometimes, I would have to do a restore of configuration and restore the .conf files from backup.

I wonder if the xt_set module contained in ipset_ipt_libmnl.K3.Arm.tar is not compatible with my kernel?

Code:

lsmod | grep "xt_set"

returns nothing on DD-WRT. On my ASUS RT-AC88U running Merlin Firmware 380.64_2, it returns
Code:

xt_set,ip_set_hash_ip,ip_set_hash_net



When I issue the command on DD-WRT
Code:

ipset –v, it returns: ipset v6.21.1, protocol version: 6


On Merlin FW:
Code:

ipset v6.29, protocol version: 6


So ipset appears to work on DD-WRT.

I am hesitant to try any further attempts as I have brought down internet for a many users on my several attempts.

Any suggestions are greatly appreciated. Thank you!

EDIT: I found this post http://www.dd-wrt.com/phpBB2/viewtopic.php?t=303984

which is xt_set for kernel 4.4.14. I will give it a try later tonight.
Nocturnal42
DD-WRT Novice


Joined: 25 Apr 2017
Posts: 3

PostPosted: Tue Apr 25, 2017 16:11    Post subject: Reply with quote
Sadly this doesn't work for me. The traffic that should be bypassing the VPN is going nowhere (I get a lot of connection timed out messages). I follow all the steps without getting any error messages. I can see that ip addresses are being added using ipset -L.

Beyond that, I don't really know enough to find where the packets are going astray.

my prerouting mangle table looks like this
Code:
Chain PREROUTING (policy ACCEPT 10536 packets, 4960K bytes)
 pkts bytes target     prot opt in     out     source               destination
  578 98066 MARK       all  --  any    any     anywhere             anywhere             match-set NETFLIX dst MARK set 0x1
   40 15060 MARK       all  --  !vlan2 any     anywhere             XXXXXXXXXXXXXXX  MARK or 0x80000000
19905 8231K CONNMARK   all  --  any    any     anywhere             anywhere             CONNMARK and 0x0
Nocturnal42
DD-WRT Novice


Joined: 25 Apr 2017
Posts: 3

PostPosted: Thu Apr 27, 2017 15:05    Post subject: Reply with quote
Watching traffic on vlan2 with tcpdump, I can see that ARP requests are being generated for the matched domains... which seems odd.

00:56:06.464475 ARP, Request who-has s3-1.amazonaws.com tell XXXXXXXXXXXXXXXXXXX, length 28
Nocturnal42
DD-WRT Novice


Joined: 25 Apr 2017
Posts: 3

PostPosted: Fri May 05, 2017 16:34    Post subject: Reply with quote
Still beating my head against this. If I change

Code:
ip route add table 100 default dev $(nvram get wan_ifname)


to

Code:
ip route add table 100 default via $(nvram get wan_gateway) dev $(nvram get wan_ifname)


I no longer get the invalid arp requests, and packets start appearing on the wan interface. Unfortunately, they aren't being NATed.
Goto page Previous  1, 2 Display posts from previous:    Page 2 of 2
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Advanced Networking All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum