Default route kicking in too soon
Builds after 43031 have extra wait time before the default route kicks in, so those should not be affected.
On some builds/routers/setup the default route can kick in too soon after a reboot, one symptom can be that
the time is not correct this also can happen if you have manually specified a slow time server (DDWRT works
best if you leave the time server field empty). This is under investigation.
If you suspect this is the case then it is easy to check and mitigate.
Disable the Route Allowed IP's via tunnel and add the following in Administration/Commands, Save as Startup
(reboot after each change).:
Code:
sleep 60
ip route add 0.0.0.0/1 dev oet1
ip route add 128.0.0.0/1 dev oet1
I disabled Route Allowed IPs and created the recommended Startup script. Now the tunnel is working for all traffic!
But, when I setup PBR it seems to have no effect, all clients are routed through the tunnel. I'm not well versed on linux networking commands, but I'm guessing `ip route add`] is conflicting with PBR?
Any advice on how to get my WireGuard tunnel setup with PBR? Either by fixing the route cause of the default route timing problem or adjusting the script to accomidate PBR?
Example: Setting Policy Based Routing to 192.168.11.64/26. Computer 192.168.11.83 is routed through the tunnel but so is 192.168.11.5!
Last edited by superTaco2 on Sun May 30, 2021 10:16; edited 2 times in total
Joined: 18 Mar 2014 Posts: 12915 Location: Netherlands
Posted: Sat May 29, 2021 7:32 Post subject:
You do not have to add anything manually (default route kicking in too soon should not be happening anymore and if it is happening you will see a warning in syslog)
The IP addresses you fill in in the PBR field are using the VPN.
You use CIDR notation as described in the guide, the address range you are using: 192.168.11.64/26
means all IP addresses from 192.168.11.64 until and including 192.168.11.127 (so 64 IP addresses) will use the VPN other IP addresses will use the WAN
I spent a bunch more time on this, but the only way I'm able to get the WireGuard tunnel working is with the a startup script that adds the ip routes. And this solution doesn't work with PBR.
Any troubleshooting suggestions?
I tried a bunch of configurations (listed below).
I then thought on the timing problem and checked my syslogs and was surprised they start at 1/1/1970 (date is 0?). There are some issues resolving 2.pool.nto.org but it eventually gets there. What's odd to me is the WireGuard tunnel + routing is setup BEFORE the current date is resolved. Would that have something to do with it??
Logs Excerpt (timezone: Zulu) (IP addresses modified to protect the innocent. 3.3.188.150 is ISP ip)
Quote:
Jan 1 00:00:20 r6700 user.info root: Enable WireGuard interface oet1 on port 51820
Jan 1 00:00:20 r6700 user.info root: Establishing WireGuard tunnel with peer endpoint us-wa1.wg.ivpn.net:2050
Jan 1 00:00:21 r6700 daemon.info httpd[1080]: httpd : httpd server started at port 80
Jan 1 00:00:21 r6700 user.info : httpd : http daemon successfully started
Jan 1 00:00:22 r6700 daemon.info dnsmasq-dhcp[1040]: DHCPDISCOVER(br0) b4:aa:aa:aa:aa:aa
Jan 1 00:00:22 r6700 daemon.info dnsmasq-dhcp[1040]: DHCPOFFER(br0) 192.168.11.5 b4:aa:aa:aa:aa:aa
Jan 1 00:00:22 r6700 daemon.info dnsmasq-dhcp[1040]: DHCPREQUEST(br0) 192.168.11.5 b4:aa:aa:aa:aa:aa
Jan 1 00:00:22 r6700 daemon.info dnsmasq-dhcp[1040]: DHCPACK(br0) 192.168.11.5 b4:aa:aa:aa:aa:aa PleaseHelpMeEgc
Jan 1 00:00:25 r6700 user.info root: WireGuard setting route for oet1 to endpoint us-wa1.wg.ivpn.net:2050 via 0.0.0.0 dev vlan2
Jan 1 00:00:25 r6700 user.info root: WireGuard 172.28.**.***/255.255.255.255 added to oet1
....
Jan 1 00:00:30 r6700 daemon.err ntpclient[1657]: Failed resolving address to hostname 2.pool.ntp.org: Name does not resolve
Jan 1 00:00:30 r6700 daemon.err ntpclient[1657]: Failed resolving server 2.pool.ntp.org: Network is down
Jan 1 00:00:30 r6700 daemon.notice ntpclient[1657]: Network up, resolved address to hostname 212.18.3.19
Jan 1 00:00:30 r6700 daemon.debug ntpclient[1657]: Connecting to 212.18.3.19 [212.18.3.19] ...
May 29 18:36:49 r6700 daemon.info ntpclient[1657]: Time set from 212.18.3.19 [212.18.3.19].
May 29 18:36:50 r6700 user.info root: Enable WireGuard interface oet1 on port 51820
May 29 18:36:50 r6700 user.info root: Establishing WireGuard tunnel with peer endpoint us-wa1.wg.ivpn.net:2050
May 29 18:36:50 r6700 user.info root: WireGuard setting route for oet1 to endpoint us-wa1.wg.ivpn.net:2050 via 3.3.188.1 dev vlan2
May 29 18:36:50 r6700 user.info root: WireGuard 172.28.**.***/255.255.255.255 added to oet1
Wire Guard Tunnel Configuration Experiments
Quote:
DOESN'T WORK - no traffic through tunnel (duckduckgo "what is my ip" reports isp ip, not vpn tunnel ip)
Startup Script: <empty>
PBR: 192.168.11.64/26
Allowed IPs:0.0.0.0/1,128.0.0.0/1
Route Allowed IPs via Tunnel: Enable
Wire Guard Status:
endpoint: 23.19.87.237:2050
latest handshake: 51 years, 161 days, 17 hours, 32 minutes, 58 seconds ago
transfer: 2.96 KiB received, 548 B sent
(on f5 in browser)
endpoint: 23.19.87.237:2050
latest handshake: 1 minute, 49 seconds ago
transfer: 8.48 KiB received, 960 B sent
Troubleshoot:
d/c wifi, reconnect
wait 3 minutes
**********************************************
DOESN'T WORK - no traffic through tunnel (ddg "what is my ip" reports isp ip, not vpn tunnel ip)
Startup Script: <empty>
PBR: 192.168.11.64/26
Allowed IPs:0.0.0.0/1,128.0.0.0/1
Route Allowed IPs via Tunnel: Disable
Wire Guard Status:
endpoint: 23.19.87.237:2050
latest handshake: 51 years, 161 days, 17 hours, 42 minutes, 10 seconds ago
transfer: 92 B received, 212 B sent
(on f5 in browser)
endpoint: 23.19.87.237:2050
latest handshake: 15 seconds ago
transfer: 184 B received, 520 B sent
Troubleshoot:
d/c wifi, reconnect
wait 3 minutes
**********************************************
DOESN'T WORK - no traffic through tunnel (ddg "what is my ip" reports isp ip, not vpn tunnel ip)
Startup Script: <empty>
PBR: <empty>
Allowed IPs:0.0.0.0/1,128.0.0.0/1
Route Allowed IPs via Tunnel: Enable
Wire Guard Status:
endpoint: 23.19.87.237:2050
latest handshake: 1 minute, 31 seconds ago
transfer: 92 B received, 2.15 KiB sent
Troubleshoot:
d/c wifi, reconnect
wait 3 minutes
**********************************************
DOESN'T WORK - no traffic through tunnel (ddg "what is my ip" reports isp ip, not vpn tunnel ip)
Startup Script: <empty>
PBR: <empty>
Allowed IPs:0.0.0.0/1,128.0.0.0/1
Route Allowed IPs via Tunnel: Disable
Wire Guard Status:
endpoint: 23.19.87.237:2050
latest handshake: 1 minute, 31 seconds ago
transfer: 92 B received, 2.15 KiB sent
via ssh `ip route`: (IP addresses modified to protect the innocent. 3.3.188.150 is ISP ip)
default via 3.3.188.1 dev vlan2
23.19.87.237 via 3.3.188.1 dev vlan2
3.3.188.0/22 dev vlan2 scope link src 3.3.188.150
127.0.0.0/8 dev lo scope link
192.168.11.0/24 dev br0 scope link src 192.168.11.1
Troubleshoot:
d/c wifi, reconnect
wait 2 minutes
via ssh run:
ip route add 0.0.0.0/1 dev oet1
ip route add 128.0.0.0/1 dev oet1
All clients now routed through vpn tunnel.
*********************************************
WORKS - all traffic through tunnel (ddg "what is my ip" reports vpn tunnel ip)
Startup Script:
sleep 60
ip route add 0.0.0.0/1 dev oet1
ip route add 128.0.0.0/1 dev oet1
PBR: <empty>
Allowed IPs:0.0.0.0/1,128.0.0.0/1
Route Allowed IPs via Tunnel: Disable
*********************************************
DOESN'T WORK - client 192.168.11.5 traffic still routed through tunnel
Startup Script:
sleep 60
ip route add 0.0.0.0/1 dev oet1
ip route add 128.0.0.0/1 dev oet1
PBR: 192.168.11.64/26
Allowed IPs:0.0.0.0/1,128.0.0.0/1
Route Allowed IPs via Tunnel: Disable
Joined: 18 Mar 2014 Posts: 12915 Location: Netherlands
Posted: Sat May 29, 2021 19:28 Post subject:
You always have to Enable Route allowed IP's via tunnel.
To test do not put anything in the PBR Field.
The route kicks in after NTP time is up but there is timeout of 90 seconds, the syslog will tell you if everything is working, it appears to work but the best way to see everything is from the CLI (telnet/Putty):
Code:
grep -E -i 'oet|wireguard' /var/log/messages
So
Remove all scripting you have done
Enable Route allowed IP's via tunnel (otherwise nothing is routed via the tunnel)
Clear the PBR field
Save and Apply
Wait two minutes and send output of the above command
You can check your routing also from CLI with:
traceroute 8.8.8.8
Thanks for the help, followed your instructions: disabled scripts & pbr, enabled Rouate Allowed IPs, rebooted and waited 2 mintutes . Unfortunately traffic is still not routing through the Tunnel:
After Reboot and 2 minutes: grep -E -i 'oet|wireguard' /var/log/messages
(IP addresses modified to protect the innocent. 3.3.188.150 is ISP ip)
Quote:
Jan 1 00:00:20 r6700 user.info root: Enable WireGuard interface oet1 on port 51820
Jan 1 00:00:20 r6700 user.info root: Establishing WireGuard tunnel with peer endpoint us-wa1.wg.ivpn.net:2050
Jan 1 00:00:25 r6700 user.info root: WireGuard setting route for oet1 to endpoint us-wa1.wg.ivpn.net:2050 via 0.0.0.0 dev vlan2
Jan 1 00:00:25 r6700 user.info root: WireGuard 172.**.**.**6/255.255.255.255 added to oet1
Jan 1 00:00:28 r6700 user.info root: Enable WireGuard interface oet1 on port 51820
Jan 1 00:00:29 r6700 user.info root: Establishing WireGuard tunnel with peer endpoint us-wa1.wg.ivpn.net:2050
Jan 1 00:00:29 r6700 user.info root: WireGuard setting route for oet1 to endpoint us-wa1.wg.ivpn.net:2050 via 3.3.188.1 dev vlan2
Jan 1 00:00:29 r6700 user.info root: WireGuard 172.**.**.**6/255.255.255.255 added to oet1
May 29 20:18:10 r6700 user.info root: Enable WireGuard interface oet1 on port 51820
May 29 20:18:11 r6700 user.info root: Establishing WireGuard tunnel with peer endpoint us-wa1.wg.ivpn.net:2050
May 29 20:18:11 r6700 user.info root: WireGuard setting route for oet1 to endpoint us-wa1.wg.ivpn.net:2050 via 3.3.188.1 dev vlan2
May 29 20:18:11 r6700 user.info root: WireGuard 172.**.**.**6/255.255.255.255 added to oet1
ip route show
(IP addresses modified to protect the innocent. 3.3.188.150 is ISP ip)
Quote:
default via 3.3.188.1 dev vlan2
23.19.87.237 via 3.3.188.1 dev vlan2
3.3.188.0/22 dev vlan2 scope link src 3.3.188.150
127.0.0.0/8 dev lo scope link
192.168.11.0/24 dev br0 scope link src 192.168.11.1
traceroute 8.8.8.8
Route clearly going through isp, several hops mention ISP by name.
That was it!! I didn't realize the IP Address/Netmask needed to be in CIDR format!! Though it needed to be /32 rather than 24:.
nvram show | grep oet1_ipaddrmask
Code:
oet1_ipaddrmask=172.**.***.**6/32
I had initially followed the IVPN Setup Guide which was written against an older v39715 when Subnet Mask was a separate field and was in Dotted Decimal Netmask format.
I'll have to ping the IVPN team and let them know their guide is quite out of date.
Joined: 18 Mar 2014 Posts: 12915 Location: Netherlands
Posted: Sun May 30, 2021 10:59 Post subject:
Glad it is solved
I will make some extra checks so that this will also be caught and specifically mention it in the label.
Actually using /24 is smarter as that is another safeguard against routing problems, if you add an /24 address to an interface a route is automatically added