SMB v3 file integrity issues

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware
Goto page Previous  1, 2, 3, 4  Next
Author Message
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 7453
Location: Netherlands

PostPosted: Wed Jan 13, 2021 8:08    Post subject: Reply with quote
Did you enable encryption as described n the first post?
_________________
Routers:Netgear R7800, R6400v1, R6400v2, Linksys EA8500, EA6900 (XvortexCFE), E2000 (converted WRT320N), WRT54GS v1.
WireGuard Documents & Guides:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327397
OpenVPN Documents & Guides: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327398
IPSET: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327261
Install guide R6400v2:http://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
Sponsor
sierratango
DD-WRT Novice


Joined: 22 Jul 2014
Posts: 48

PostPosted: Wed Jan 13, 2021 18:54    Post subject: Reply with quote
egc wrote:
Did you enable encryption as described n the first post?


The tests were performed without enabling encryption as the additional integrity checks were not required.
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Wed Jan 13, 2021 20:06    Post subject: Reply with quote
I have moved my r7000 to only act as NAS host (no WAN, no wireless). I reset r44700 to factory defaults and only configured the basics. So far I cannot reproduce the issue, but it's still early.

I compared the nvram show output of the fully configured router with the factory defaults. Interestingly, this line is in the default config:
nvram_ver=4
vs. this line from when I originally setup the router in September 2020 (r44467)
nvram_ver=5
Not sure what this means or if this is an issue, but I would have expected an increase in versions instead of a decrease.
I didn't see any other immediately obviously strange differences.

Transfer speed to NAS has approx. doubled (~4MB/s to ~8MB/s). Not sure why.
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Thu Jan 14, 2021 16:39    Post subject: Reply with quote
Ran a series of tests on my Netgear r7000 router running r44700 (reset to factory, as described above) with SMB v3 and encryption enabled using the test script supplied by @sierratango. My PC (current CPU, Windows 10 with Ubuntu subsystem) is connected on the LAN side with gigabit ethernet. There is no other client connected to the router.

I had fewer transfer failures than before, but they are occurring (about 1 per 20GB written). The script exits with this error when there is a failure: "cp: error writing '/mnt/z/Test/testfile': Input/output error". This exit seems to occur due to the enabled integrity checks in SMB v3.

Also noticed that NAS transfer speed increases further with Shortcut Forwarding Engine (SFE) enabled, to ~11MB/s with router CPU% going to 70%. This is unexpected as I have no QoS, filters, etc. as my PC is connected on the LAN side. Failures occurred with and without SFE enabled.

EDIT: Updated to r45420 (latest) and reset to factory. Noticed the Samba version was updated to v3.3.1. Running tests now ...
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Fri Jan 15, 2021 3:28    Post subject: Reply with quote
Ran into a strange issue with Windows 10 transfer speed to NAS being very slow (~20 Mbit/s). This was after the installation of Windows Subsystem for Linux (WSL2). So same machine running a Ubuntu subsystem virtually on Windows 10 base. The strange thing is, WSL2 transfer speed is normal (~65 Mbit/s).

Also, saw frequent transfer integrity issues when transferring from Windows, but none from WSL2. So no conclusive results wrt transfer corruptions yet.

Tried reverting back to r44700, but no difference in results.
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Fri Jan 15, 2021 20:44    Post subject: Reply with quote
I think I figured out the Windows slowness issue: Changing the 802.1x and/or Routing items under Administration -> Management to disabled seems to cause a change that is not reversible in the GUI. Windows transfer speeds dropped from averaging about 65Mbps to averaging 20 Mbps (on r45420). Changing it back to enabled, rebooting, even power-cycling had no effect. I was only able to restore higher speed by resetting to factory defaults.

This leads me to believe that the transfer integrity issue may not be related to Samba. Further testing ongoing...
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Sat Jan 16, 2021 19:18    Post subject: Reply with quote
Never mind the crap I wrote above. Captured a transfer with Wireshark and found lots of dup acks and retransmissions. Then I found this post https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=945793 and change port speed from auto (gigabit/full) to 100/Full on my PC. Fixed the issues, steady 96 Mbps SMB transfer rates (r7000 running dd-wrt r44715).

I'm using Cat6 patch cable and have tried different ones. Maybe an issue lurking in the network stack?

Will continue testing SMB transfers to make sure the issue is resolved.
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Sat Jan 16, 2021 20:21    Post subject: Reply with quote
Taking this further, there seems to be an issue with the auto-negotiation. After rebooting the router it now negotiates 100/Half when I fix the client to 100/Full. Not sure why I got 100/Full before, maybe just chance. Trying to set port speed in the GUI doesn't work, it reverts to auto-neg.

Is there a way to set port speed using the command line?
MrPickles
DD-WRT Novice


Joined: 14 Jan 2017
Posts: 49

PostPosted: Sun Jan 17, 2021 5:49    Post subject: Reply with quote
I'm also running a R7000 using dd-wrt 45454 with a connected USB NAS drive. Drive is formatted EXFAT with write access needed across multiple manufacturer devices.

After formatting & reformatting the USB to EXFAT, I can copy a percentage of files before receiving a file manager error "could not write to ...".

MP
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Mon Jan 18, 2021 1:33    Post subject: Reply with quote
Update: Upgraded to r45454.
I get reliable Samba transfers when I set my PC ethernet adapter speed/duplex to 100/Half. at 100/Full or 1000/Full, transferred files are sometimes corrupted. I haven't found a way to fix the port settings for the router yet, maybe this is not possible.

I tried different network cables, and ports on the gateway, and another client machine. Short of some kind of router hardware defect, which is unlikely as others are reporting issues, this issue seems to be located in the dd-wrt network stack. Maybe it occurs only when services on the router are accessed, like Samba, idk. Any ideas for further tests?
sierratango
DD-WRT Novice


Joined: 22 Jul 2014
Posts: 48

PostPosted: Mon Jan 18, 2021 6:34    Post subject: Reply with quote
I've been on r41986 for a while now and have not had a single file corruption even after more tests transferring tens of gigabytes of data from 2 Linux machines over Gigabit Ethernet and Wi-Fi.

The corruptions start happening in the release right after r41986 when ksmbd is introduced.

I am going to take egc's word for it about ksmbd being a work-in-progress and will stick to Samba3 until ksmbd is more mature.
CanadaMoose
DD-WRT User


Joined: 12 Apr 2017
Posts: 50

PostPosted: Mon Jan 18, 2021 20:53    Post subject: Reply with quote
Installed r41986 on my r7000 router and wow, what a difference. I get consistent Samba throughput of 240 Mbps with CPU 15% idle when connected via gigabit ethernet.
On r44467 and higher (up to r45454) I got around 25 Mbps, and 52 Mbps when dropping my adapter speed from auto to 100/full. (Got 95 Mbps once during my testing, but that didn't survive a reboot).

So looks great. I will run transfers with file integrity checks overnight. Only caveat is that SMB v3 is not yet supported.
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 7453
Location: Netherlands

PostPosted: Fri Jan 22, 2021 16:46    Post subject: Reply with quote
Next build will have updated ksmbd with some bugs fixed
_________________
Routers:Netgear R7800, R6400v1, R6400v2, Linksys EA8500, EA6900 (XvortexCFE), E2000 (converted WRT320N), WRT54GS v1.
WireGuard Documents & Guides:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327397
OpenVPN Documents & Guides: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327398
IPSET: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327261
Install guide R6400v2:http://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
sierratango
DD-WRT Novice


Joined: 22 Jul 2014
Posts: 48

PostPosted: Fri Jan 22, 2021 19:53    Post subject: Reply with quote
egc wrote:
Next build will have updated ksmbd with some bugs fixed


Thanks, I can see updates in the source code to 'src/router/smbd-next' but not in 'src/router/smbd'. Do you know which one gets compiled into each new build?
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 7453
Location: Netherlands

PostPosted: Fri Jan 22, 2021 20:01    Post subject: Reply with quote
smbd-next is ksmbd is the one which is compiled in.
_________________
Routers:Netgear R7800, R6400v1, R6400v2, Linksys EA8500, EA6900 (XvortexCFE), E2000 (converted WRT320N), WRT54GS v1.
WireGuard Documents & Guides:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327397
OpenVPN Documents & Guides: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327398
IPSET: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=327261
Install guide R6400v2:http://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
Goto page Previous  1, 2, 3, 4  Next Display posts from previous:    Page 2 of 4
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum