That was a much older and buggier version of OpenVPN. When I raised this issue, it resulted in the default MTU being set in OpenVPN config of 1400. That ended up fixing it for my use case (and probably others as well)
Since then I've replaced the 2 routers involved with ones with faster CPU's and newer DD-wrt, currently version r58568 which have newer OpenVPN versions.
I revisited my original testing and it appears the original bug in OpenVPN - that you could send certain UDP packets with DF set through the VPN tunnel and OpenVPN would trash them, NOT replying with an ICMP control packet, seems to have been corrected.
I have tried setting "Tunnel MTU Setting" to any value and cannot get the bug to reappear. I also don't have problems with SIP phones registering over the VPN anymore. However, there is ANOTHER bug in OpenVPN that now is a problem.
That is this. If you try turning OFF the "tunnel MTU setting" by setting it to 0, dd-wrt merely sets it to 1500 and passes the option
"tun-mtu 1500"
to OpenVPN. Unfortunately, there is another bug in OpenVPN. Basically, not only is tun-mtu used to calculate some things internally in OpenVPN, it is ALSO used to send ICMP control messages back to the sender stating the maximum MTU size allowable. For example note the following with "tunnel MTU setting" set to 0 on both ends of the VPN:
Code:
tedm@beachserver:~$ ping -M do -c 1 -s 1473 172.16.1.16
PING 172.16.1.16 (172.16.1.16) 1473(1501) bytes of data.
ping: local error: message too long, mtu=1500
^^^^^^^^
That is the ping under Debian reporting 2 things. First is that the ICMP packet, with the Do Not Fragment bit set, of 1473 bytes, is too big and thus ping received an ICMP control message saying that the packet was too big. Second it is another ICMP control message saying that the max MTU is 1500. Which, obviously, is baloney since the packet got rejected with a MTU of 1473 . Clearly that is yet another bug in OpenVPN.
If you think "I'll be a smart guy and fix this" by setting "tunnel MTU setting" to 1472, then it is true that now OpenVPN will send back the correct ICMP control message. But, it also recalculates the internal MTU and will disallow larger MTU's in the VPN. So now, you will get the same error as before - except that it will block at 1444 (which is 28 bytes lower then 1472) while reporting MTU is supposed to be 1472.
Now, it MAY BE possible to work around this IF you set the Tunnel UDP MSS Fix turned on, on both sides of the VPN. However, setting this on or off in DD-WRT seems to do absolutely nothing. The options passed to OpenVPN _do_not_ seem to change at all. At least, not by simply clicking the Apply Settings button. (I haven't tried rebooting the router yet, I will later)
But, it would be better I think if we could fix the "Tunnel MTU Setting" so that setting it to 0 removes the option entirely from being passed to OpenVPN.
There's a discussion here on this that seems pretty knowledgeable and the biggest comment on it recommends against setting these options at all:
Especially if you DON'T have control over both sides of the VPN because some of the options MUST be symmetrically set.
We had the tunnel MTU setting set to 1400 due to an earlier bug in OpenVPN that's now been corrected, apparently, so now we need the option to unset it completely.
Or maybe we can just say hey, stick any old option you want in the dd-wrt GUI - it won't actually DO anything - but it will make the operator FEEL like they are doing something. <eyeroll>
This isn't "dredging" no matter what you think - I have application testing I'm doing and unlike TCP, where the stack takes care of fragmentation and black hole path discovery, with UDP if the application developer decides they must use that protocol, then they have to handle all the MTU stuff in the app - and doing that depends on proper ICMP messages being sent back from the intermediate routers. Not some invalid "we'll-pull-some-number-out-of-our-ass-for-the-MTU-size" baloney that's going on now. How am I supposed to figure out if the app is bolloxing it up or the intermediate router is bolloxing it up if DD-wrt isn't handing passing the options properly?
I posted links to all of the networking discussions that explain all of this there's no point in reiterating it when others have already put the time into explaining it. If YOU don't understand the problem, and you don't want to read the history to understand it, or read others' explanations that explain it, then why on earth are you saying to RTFM when DD-WRT does not appear to follow the M?
Joined: 08 May 2018 Posts: 15233 Location: Texas, USA
Posted: Sat Nov 30, 2024 3:04 Post subject:
I completely understand that you aren't following prescribed practices. I removed part of my original comment, but shall now reinstate it. Consider forwarding this "report" upstream, as it's related to OpenVPN itself (if there is actually a bug). That is probably exactly the response I would get if I forwarded your report to Sebastian himself. _________________ "Life is but a fleeting moment, a vapor that vanishes quickly; All is vanity"
Contribute To DD-WRT Pogo - A minimal level of ability is expected and needed... DD-WRT Releases 2023 (PolitePol)
DD-WRT Releases 2023 (RSS Everything)
----------------------
Linux User #377467 counter.li.org / linuxcounter.net
Joined: 18 Mar 2014 Posts: 13429 Location: Netherlands
Posted: Sat Nov 30, 2024 9:21 Post subject:
The companion text shows: (Default: 1400, 0 Not Set)
So 0 means not set, meaning there is no mtu setting in the config.
I know because I put it there and wrote the code to remove the mtu setting if it was set to 0
For client
Quote:
if (nvram_invmatch("openvpncl_mtu", "") && nvram_invmatchi("openvpncl_mtu", 0))
fprintf(fp, "tun-mtu %s\n", nvram_safe_get("openvpncl_mtu"));
For server:
Quote:
if (nvram_invmatch("openvpn_mtu", "") && nvram_invmatchi("openvpn_mtu", 0))
fprintf(fp, "tun-mtu %s\n", nvram_safe_get("openvpn_mtu"));
To be sure I double checked and if you set it to 0 there indeed is no mtu setting in the config (server:/tmp/openvpn/openvpn.conf, client:/tmp/openvpncl/openvpn.conf) and OpenVPN should calculate it, yes they have been working on it but still do a shitty job so you will often see that it calculates to its default of 1500 which might not always be correct. _________________ Routers:Netgear R7000, R6400v1, R6400v2, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399 Install guide R7800/XR500:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614 Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
Last edited by egc on Sat Nov 30, 2024 10:30; edited 1 time in total
Which, obviously, is baloney since the packet got rejected with a MTU of 1473 . Clearly that is yet another bug in OpenVPN.
wrong the MTU is 1501
Quote:
PING 172.16.1.16 (172.16.1.16) 1473(1501) bytes of data.
tedm wrote:
If you think "I'll be a smart guy and fix this" by setting "tunnel MTU setting" to 1472, then it is true that now OpenVPN will send back the correct ICMP control message. But, it also recalculates the internal MTU and will disallow larger MTU's in the VPN. So now, you will get the same error as before - except that it will block at 1444 (which is 28 bytes lower then 1472) while reporting MTU is supposed to be 1472.
I'm not sure what you're trying to tell us, but I think you're making a mistake.
You define the PAYLOAD size in the ping command and not the total size.
You must add an overhead of 28 for ICMP to the payload
payload+overhead=MTU
1444+28=1472
so everything is correct and I don't see a bug there
otherwise I have no idea what you want to tell us.
The companion text shows: (Default: 1400, 0 Not Set)
So 0 means not set, meaning there is no mtu setting in the config.
I know because I put it there and wrote the code to remove the mtu setting if it was set to 0
To be sure I double checked and if you set it to 0 there indeed is no mtu setting in the config (server:/tmp/openvpn/openvpn.conf, client:/tmp/openvpncl/openvpn.conf) and OpenVPN should calculate it,
Here is MY /tmp/openvpncl/openvpn.conf (destination IP obscured)
Yes, as you see - the 1500 setting is NOT in there. HOWEVER, here is the output of the log from http://172.16.100.1/Status_OpenVPN.asp, immediately after rebooting:
That tun-mtu is changed IF a number is inserted into the DD-WRT config. That is, if 1400 is put into the GUI config - then 1400 appears in the log, etc.
The tun-mtu 1500 is coming from SOMEWHERE. I don't think it's being generated internally by OpenVPN although maybe it could be? But it sure looks like there's an invocation of openvpn in DD-WRT that has it. Is there possibly anywhere else within DD-WRT that may still have that entry, possibly left over from long ago?
If this is an OpenVPN bug, I'll gladly submit it upstream. But, if the reception there is anything like here (no, it couldn't possibly be our code) then they are going to blame DD-WRT claiming it's doing it.
Joined: 18 Mar 2014 Posts: 13429 Location: Netherlands
Posted: Sat Nov 30, 2024 12:37 Post subject:
Quote:
The tun-mtu 1500 is coming from SOMEWHERE. I don't think it's being generated internally by OpenVPN although maybe it could be? But it sure looks like there's an invocation of openvpn in DD-WRT that has it. Is there possibly anywhere else within DD-WRT that may still have that entry, possibly left over from long ago?
The config is just that the config to run OpenVPN, you can also manually make it and run OpenVPN with it outside DDWRT, so no DDWRT does not interfere with it after the config is made.
The default is 1500 so yes I think it is coming from OpenVPN.
To be honest the whole MTU is a can of worms with --mssfix --fragment --mtu-disc etc.
Which, obviously, is baloney since the packet got rejected with a MTU of 1473 . Clearly that is yet another bug in OpenVPN.
wrong the MTU is 1501
Quote:
PING 172.16.1.16 (172.16.1.16) 1473(1501) bytes of data.
tedm wrote:
If you think "I'll be a smart guy and fix this" by setting "tunnel MTU setting" to 1472, then it is true that now OpenVPN will send back the correct ICMP control message. But, it also recalculates the internal MTU and will disallow larger MTU's in the VPN. So now, you will get the same error as before - except that it will block at 1444 (which is 28 bytes lower then 1472) while reporting MTU is supposed to be 1472.
I'm not sure what you're trying to tell us, but I think you're making a mistake.
You define the PAYLOAD size in the ping command and not the total size.
You must add an overhead of 28 for ICMP to the payload
payload+overhead=MTU
1444+28=1472
so everything is correct and I don't see a bug there
OK well technically you can't have an MTU of 1501 on an Ethernet network, but let's assume OpenVPN is using it's internal fragmentation/reassembly code so that a packet with DF header goes through. If that is the case then I am incorrect. I am going off the MAN page for ping under Debian which says:
-s packetsize
Specifies the number of data bytes to be sent. The default is 56,
which translates into 64 ICMP data bytes when combined with the 8
bytes of ICMP header data.
I assumed that "packetsize" meant literally what it says - _PACKET_SIZE_. As in "total packetsize" Damn man pages! So yes, then the total packetsize would be 64 not 56 if you specify 56. Arrg!!
Technically, though, the overhead here is 8 + 20, where 20 is the OpenVPN header. In this case I'm using TCP for the VPN protocol so the header is 20 bytes, which is where the 28 is coming from. There is also the HMAC header which is determined by the HMAC algorithm but in this case, is apparently taken care of by OpenVPN's internal fragmentation code.
I will do some more research on this and report back. There is still the matter of the fact that apparently, upstream fixed SOMETHING because I'm no longer seeing the black hole that I was seeing back in 2021, and thus the 1400 probably needs to be defaulted to something else now.
And the DD-WRT implementation of WG would also have to be cudgeled into doing subnet-to-subnet VPN routing for my purposes, just as I had to do with OpenVPN.
I think the root problem is the action of a VPN.
The idea of taking every packet, and encrypting it, then decrypting it, along with possibly fragmenting it if it's larger than the VPN overhead, then you add in the fact that packet delivery on the Internet is NOT guaranteed, means that VPN issues are ALWAYS going to happen if you use VPNs. Then you add to that some VPN implementations (Untangle's and the auto setup scripts on https://github.com/hwdsl2/setup-ipsec-vpn, as well as DD-WRT) by default enable NAT and/or proxy arping as a cheap hack for proper routing, and you really throw a monkey wrench into the mix.
And it's not like MSS hasn't been an issue before. Check this out from 5 years ago - nothing has changed at all:
Far too many apps out there are written by developers who simply DO NOT understand the TCP/IP protocol and just read the Chat-GPT generated definition of the TCP and the UDP protocol and go by that when selecting a protocol for their application to use.
The VoIP industry got BADLY burned by this 20 years ago which is why EVERY softphone out there written TODAY has the option of registering SIP either over TCP or UDP. For example with the default DD-WRT settings for OpenVPN, and MTU set to 1400, Linphone registration over the VPN has an error if UDP is selected, while it does not if TCP is selected. Don't ask me why this is.
But so many other applications (such as tftp) don't.
I really don't think WG would be any faster for me. The biggest speedup came from replacing the routers at both ends with Netgear R7000's which have a 1Ghz CPU clock. If I FTP down a file from a server on the other end that is behind my lan2lan VPN, I get the exact same xfer speed as if I FTP down a file from a server at the other end that is on a public IP address, outside of the VPN. I think the answer isn't that one VPN protocol is faster than the other, the answer is how much horsepower your network devices at each end of the VPN have.
I am still messing with DD-WRT running the VPN because I'm "old school" I guess and a network device is supposed to be a box without a spinning fan or a screensaver that runs at 50% of the CPU (thank you "pipes" and "fire" screensavers, LOL). The world is full of PC's and I have access to PILES of 4th generation Intel G5s that need merely a PCI ethernet card and a Unix server boot stick to be perfectly serviceable routers. But, I still am messing with these SOHO routers.
of course there are also MTUs larger than 1500 called jumbo frames or baby jumbo frames
Yes if you have jumbo frames enabled on your gigE switches (most don't) or are running WAN links on OC3s and such that support those esoteric frame sizes. I'd say the percentage of OpenWRT devices with that capability are less than 0.01%
My openwrt router supports a maximum MTU of 9856 (At least that's what the driver says)
actually every router with 10Gbit ports should support jumbo frames
After all, it is necessary to reduce overhead and increase throughput and was already specified with 1000Base-T
