[seanPH]

Router: R7000
Firmware: DD-WRT v3.0-r47074 std (07/20/21)
Tunnels: 2 x wireguard

My configuration was originally setup on r46974 and had the same problems on that version. Upgraded yesterday to r47074 and symptoms are same.

Background: all but 1 or 2 devices on my main subnet automatically route via oet1. Oet1 connection is good, fast and robust - no complaints there. VPN provider blocks (new) incoming traffic on that connection. The IP is shared, and theoretically can change. The problem is oet2. Oet2 connects to a dedicated/static IP and port-forwarding (only above 5000) is enabled. Although both connections are from same VPN provider they are (obviously) different endpoints.


Problems:

1. The Tunnel status window for both ( oet1 and oet2) show identical text. That is they both show the status details of oet1. e.g. endpoint, connect time, etc. This is minor/cosmetic because the details of both oet1 and oet2 show separately in the syslog - and I never see any problem with either oet1 or oet2 connecting.

2. (main problem) All - incoming from the internet - connections to oet2 correctly route to destination/port (eg: WOL, SSH, RDP and HTTP/HTTPS), nothing is dropped or blocked. Connections are made via oet2, but then the connection drops out after only 10-15 seconds. For WOL there is no ongoing connection needed, and so WOL works perfectly. For SSH, I can log on (stored key) and execute 3 or 4 times something like "ls -al" and then the connection drops out (ie about 15 seconds). For serving static webpage I *was* seeing the same thing - with only half the page rendered the connection was lost. But that connection (http and https only) is working much better (better, not perfect) after I made these changes: MTU changed 1440 -> 1296 and keepalive changed 25 -> 15 and the following in Firewall:

#
# tell server to use (my) MTU setting of 1296 not the default
iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Those changes: MTU, keepalive made no difference to the SSH connections.

[egc]

What VPN provider are you using?

To get some more insight show pictures of the settings page of oet1 and oet2, (of course black out keys)

What extra manual firewall rules are you using?

MTU problems can occur, how to track those is described in the WG server setup guide under troubleshooting.

Running two tunnels to the same provider can be problematic depending on the backend of the provider and your own settings

If you want to use just the second tunnel to get to your router consider using to setup a WG server besides your WG client

[seanPH]

[seanPH]

ummm. this does not seem related to running 2 tunnels to me. Please not be distracted by that. Firstly - both tunnels run fine (connect, speed, etc ), it is only the "incoming connections" that are the problem. Secondly - when I completely disable oet1 - and reboot router - (so there is only one tunnel) the symptoms are exactly the same. This problem is about - maintaining a connection for incoming (from the other side of the tunnel ) for more than 15-20 seconds...

I have no evidence to say that this an MTU problem. Maybe it is, maybe not. What I meant to do was just to describe an MTU change that made a small difference. It did not fix the problem (e.g. SSH still identical symptoms after that MTU change), and for static http it seem to *improve* the connection for my test page, but certainty did not fix it. If the expert opinion here is - it really does look like an MTU problem - then I will pursue that line of investigation. But I just mentioned that one thing I had tried, and the result. I would be surprised if my (non expert) first guess, being "MTU", turned out to be correct....

[egc]

Question first, Status for a Peer is fetched by its unique peerkey.
Are you using different peers but with the same key?

Second question which provider are you using?

You describe a speed problem, like slow or hanging connections that could be related to an MTU problem, see the troubleshooting section.

You want incoming connections but enabled the incoming firewall, so first step is to disable the incoming firewall.

Furthermore disable CVE mitigation (it probably is not necessary to disable it but it depends on what your provider is doing)

Do you use any manual firewall rules?

As we do not know the precise setup of your provider (and they are of course not telling us) it is sometimes very difficult to track/mitigate these problems

[seanPH]

to limit the size of the problem lets just focus on SSH. I can CONNECT from the outsude.. I then login. I can then execute commands etc.. everything good at this point (ie two way communication over the tunnel). Then after about 15-20 seconds, the SSH connection drops outs... Of course - I can reconnect - log in again etc... But the problem is that the connection is only 15 seconds long each time.

Here are the routes (in "save firewall") that I use to connect by SSH from the outside world:


# SSH port-forwarding on the OET2 tunnel
iptables -t nat -A PREROUTING -i oet2 -p tcp --destination-port xxxx -j DNAT --to-destination 192.168.33.11:22
iptables -t nat -A PREROUTING -i oet2 -p udp --destination-port xxxx -j DNAT --to-destination 192.168.33.11:22
#
# push RELATED/ESTABLISHED rule back to NEAR top of chain (after incoming redirects)
iptables -D FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -D FORWARD -m state --state RELATED,ESTABLISHED -j logaccept
iptables -I FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
#
# ACCEPT SSH from anyone, going to batopi
iptables -I FORWARD -p tcp --dst 192.168.33.11 --dport 22 -j logaccept
iptables -I FORWARD -p udp --dst 192.168.33.11 --dport 22 -j logaccept
#
# tell server to use (my) MTU setting of 1296 not the default
iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

[seanPH]

Just to be clear - the incoming SSH connection hits the expected/intended rules and no packets that should be forwarded are dropped. The incoming packets successfully arrive at the correct destination... But then after happily doing 4 or 5 "ls -al" commands on my SSH connection...for about 15-20 seconds, the connection drops. Really am not so concerned about the routing.

In case my wording is bad - I am saying the Oet2 tunnel STAYS UP - it is only the connection eg my ssh session (initiated from the other side of the tunnel ) that drops after 15-20sec. The oet2 tunnel is rock solid, and is processing lots of data and many connections.

Just retested: "incoming firewall"
Just retested: "CVE-2019-14899 Mitigation"

Neither setting made any difference to this problem.

[egc]

I was hoping it was a firewall problem on the DDWRT side i.e. if you have the inbound firewall enabled you can get a connection from outside as the router initiates a connection and then the RELATED,ESTABLISHED rule will allow a connection, but the connection closes automatically and that is where the keep alive comes into play, if this process is not working properly you will have disconnections which will resolve with the frequency of the keep-alive setting.

However your firewall rules probably open up the firewall so it is open for inbound connections even if the connection is not initiated with keepalive (to be sure I have to see the firewall rules working with:
iptables -vnL FORWARD , but I am pretty sure from your description this works)

So the problem is perhaps not on the DDWRT side.
The same problem as described above can also happen on the providers side so it might be just a problem on the providers side

I sort of duplicated your situation, I have a central VPN server, my own router is connected as a WG client (it is a site-to-site setup so slightly different from your setup)

Usually when I want to connect to my home I use my phone with the WG client to connect to the server and the server connects everything (there are more clients/family members attached also from abroad).

To mimic your situation I added port forwarding to forward an SSH port via the server to the client and from the client to my QNAP NAS and I had no problem to SSH into my NAS from my phone via the internet just using Connectbot on my phone pointing to the DDNS address of my server.

So your kind of setup can work with DDWRT controlled WG server and WG client.

Unfortunately that does not help you

So at this moment I am out of options (but I will sleep on it tonight ).
The only thing I can think of now is the nuclear option aka reset to defaults and start fresh.
In some cases it miraculously starts to work then (https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=329682 )

As said if you cannot resolve it then consider setting up a WG server on the router (besides the already running WG client) and connect to that or simply SSH into your router (although that is not considered very safe, if you do it, at least use SSH with keys and not password)

P.S.
You did not answer my question if are you using the same key for both peers.
I want to know that because that could be the cause of the status window displaying the same for both peers.
In earlier code I also parsed for the tunnel number but as peerkeys are normally unique I left it out to save some space

[seanPH]

Thanks for your detailed reply !! And thanks for testing similar setup in your environment ! I do feel we are getting closer....

1. Yes - they are the same keys. And yes - that is clearly why I am seeing the same status window in oet1 and oet2. HOWEVER - that does not prevent 2 clearly different tunnels operating fully and successfully - with independent routing rules on each one.. that is proven. both operate well.. seemingly without interfering with each other. THIS POINT NOT RELATED TO THE PROBLEM.. because now operating with one tunnel.

2. As I mentioned in earlier post - to avoid confusion/distraction (about 2 tunnels / same keys ) while I am working on this problem - I have DISABLED oet1 and then REBOOTED... So - let me be clear - I am running with ONE and ONLY ONE tunnel... it is called oet2. Correct status (endpoint) shows in the status window as there is only one.

So - back to the problem of the connection dropouts - after many more hours testing it today ... and process of elimination.. I am starting to think it is the router (either dd-wrt sw or the config ) that is to blame for these dropouts.

FIRST - I know it is NOT the "two tunnels" because I am running with ONE tunnel and still have the problem.

SECOND - Until today I have blamed the VPN provider in my mind. However - today's testing seems to clear them (at least a little). Connecting smartphone to LTE network only, and running WG client to same VPN endpoint of oet2 ( oet2 is disabled during this test) , I can then SSH from laptop into my smartphone (same IP and port I was using in tests thru the ddwrt router) - and the SSH terminal session stays up for hours.... Not just 15 seconds. So the VPN provider is not automatically dropping the port/connection, and the endpoint is not setup badly. So.... I am asking myself, what is different ? What is keeping the session active (same endpoint) when I do like that to smartphone? that is different/missing when doing it behind a router.

Running WG in separate server here - not really suitable (thats another topic) . It'd have to run as router anyway, given the incoming ports (ssh, web, ftp, wol, rdp, etc) all go to different devices.... But - again that is another distraction - I am JUST trying to connect by SSH to a server attached to the R7000 by ethernet cable... from the outside. IT authenticates, it connects, there is two way traffic, commands entered, output shown... there are no (obvious) packet drops anywhere.. it is **just** that the SSH connection (note: NOT the tunnel) drops after 15-20 seconds... Note: the wg tunnel does NOT drop out. Oet2 stays up the whole time... Other people continue using it (different ports) ..and it is always UP. But my SSH connection .... from internet ==== over tunnel==== to r7000 and then on local lan to SSH server..... that connection goes down after approx 15 second.

Im going sleep now too

[egc]

I never implied having the same peerkey with different tunnels could cause the trouble you are having.
I only implied that it could be the cause of the status window problem and I am glad that you can confirm it.

I will put this on my list to fix but as it is minor it can take a while before it is patched upstream.

sleep well

[seanPH]

(thanks @egc for the suggestion)
I changed my SSH server (inside my LAN) to listen on 5xxx instead of 22. Then I changed the routes ( both -t nat, and FORWARD in the r7000 router ) so that the port always stays at 5xxx for SSH during entire connection-session ... and not convert to 22 for part of the journey.

Anyway - it did not change the symptoms. It still connects from outside, still it stays up about 15-20 seconds... still the connect drops. Exactly same.

I think I should try to SSH just to the router - if that works - then I would know for sure that it is the journey from router <-> 192.168.33.11 that is the problem...

But it was good idea (even did not work) and I will leave my config like that - it is cleaner and simpler, with less moving parts. Thanks again....

But I still need solution.... I would like to also test different VPN provider, but I am trying to think how to do that ? And which one ? (need wireguard and portforwarding and works well with dd-wrt )

[egc]

You have to research carefully not all providers support WireGuard on the router and those who do not often support port forwarding.

I think Mullvad does also TorGuard.

I use Keepsolid which was very cheap via a deal with StackSocial (lifetime for 30 bucks) but they do not support port forwarding

[seanPH]

I think Torguard....

They say "Yes we do, port forwarding is limited to ports above 2048 and is supported through tunnel types OpenVPN, WireGuard and OpenConnect."

And it seems they are saying that portforwarding is available even on standard plan (ie without a dedicated IP) which makes it substantially cheaper.

My browser (honey) added a coupon which half the price. Making it only $30 per year ... including PF ... hmmm.... only a 7 day money back on it ...

[egc]

Their setup instructions look quite good, it is not using the latest build but it should work

https://torguard.net/knowledgebase.php?action=displayarticle&catid=55&id=263

[seanPH]

yeah ... I have only 3 month (paid) left on my current IP address. But I have 3 years (paid) on regular VPN access... hahahaha... so I trying to avoid going to Torguard