VLAN config, Updated Guide: How to correctly configure

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware
Goto page 1, 2, 3  Next
Author Message
MrPete
DD-WRT User


Joined: 09 Jul 2013
Posts: 74

PostPosted: Wed Nov 01, 2017 2:10    Post subject: VLAN config, Updated Guide: How to correctly configure Reply with quote
UPDATE: As Of Sep 1, 2018, the wiki page has been updated to incorporate everything in this guide.
Please see: "Switched Ports"


This post is intended to update the "Switched Ports" wiki page as of October 2017, particularly for R7000-type gigabit routers. If you want to do VLAN configuration, do read that page first. It's mostly correct!

Note that simply using the GUI will work well for many setups, because many setups do not fully exercise the capabilities of the router/switch.

Unfortunately, the initialization and driver code hasn't been refactored in (forever), so some internal settings are multiply-initialized, while others are not auto-initialized at all. The result: certain features and settings will absolutely be unstable or non-working, unless you take the time to properly configure the underlying nvram settings. (No, we don't have a list of what will break and when.)

The following instructions help you set all of the appropriate nvram variables in the correct way.

(This has been tested on a variety of Kong builds from r33575 and forward, and via source code examination.)

1. DO use the DD-WRT Setup->VLANs page to make your initial configuration changes.
  1. If your only changes involve the checkboxes at the bottom of the page (e.g. Auto-Negatiate...Enabled), your job is done!
  2. If you make other changes, by setting it up here first, most of your work is already done.
  3. While some changes made on the VLANs page do require additional manual configuration, you've finished the hard part!


2. Once you have made your desired changes on the VLANs page, ssh/telnet to the router and record its current state using these commands:
Code:
nvram show | grep vlan.*ports | sort
nvram show | grep port.*vlans | sort
nvram show | grep vlan.*hwname | sort


3. Here's additional documentation beyond the current "Switched Ports" information:
  1. Many popular Gigabit routers (e.g. Netgear R7000) still use port 5 for the CPU, not port 8.
  2. In the port*vlans section, #18 changes to be: 18 = 100 Mbit is unchecked or greyed because Auto-Negotiate or Gigabit is checked
  3. Add "21 = Gigabit is unchecked or greyed because Auto-Negotiate is checked"
  4. While "t" indicates ports where an 802.1q header tag should be added (e.g. for trunking), "u" is built into the default configuration (i.e. vlan2ports=0 5u) to ensure traffic to the WAN port is untagged.


4. VERY IMPORTANT For a fully working VLAN Config, you must make the following manual changes because the current GUI doesn't do it!
  1. Add vlan N hwname for every VLAN other than #1 and #2. Each one has a value of "et0". ie
    Code:
    nvram set vlan3hwname=et0
  2. Add a correct vlan N ports value for each new VLAN, and correct any existing ones to remove/change ports no longer in those VLANs. Each entry contains the ports in that VLAN, plus the CPU port (5 or 8 -- just use the same CPU port as in vlan1ports). For example, if port 4 is now in VLAN 3 instead of VLAN 1, you might need:
    Code:
    nvram set vlan1ports="1 2 3 5*"
    nvram set vlan3ports="4 5"
  3. (You have made the changes but they are not yet active nor saved!) Use the above nvram show commands to view your updated settings. Are you sure they're correct? If so, move ahead. Just know that once you commit, your router will follow the new rules, for better or worse Smile
  4. Commit the changes.
    Code:
    nvram commit


5. Further down in this thread, quarkysg provides instructions for enhancing VLANs to support ID's up to 4096. (Thank you!!) NOTE: As of revs 35079-35165 (depending on the kernel), 4096 VLAN ID support is now built in, for Broadcom-based devices.

Further Notes:
- On my own r7000, I recently added two VLANs, bridged one of the new VLANs with a WiFi virtual interface, added extra DHCP servers, etc... and did so without making the currently-needed underlying nvram changes. It all kinda-sorta seemed to work... except I kept losing wifi connections, the router would crash and need reboot multiple times a day, and in general nothing worked perfectly (until I performed the needed manual edits.) I urge you not to ignore these steps if you add/change VLAN assignments!


Last edited by MrPete on Sat Sep 01, 2018 20:33; edited 18 times in total
Sponsor
mac913
DD-WRT Guru


Joined: 02 May 2008
Posts: 1403
Location: Canada

PostPosted: Wed Nov 01, 2017 3:58    Post subject: Reply with quote
Ever since I've had my R7000s the GUI vlan setup has always worked for me and I have one port that has as many as 9 tagged vlans. Currently i'm on Kongac Build 33525M.

I know some people tend to use CLI to configure their vlans (maybe because the GUI isn't supported for their SoC or very custom setups) but I never had use it since the GUI always worked on the R7000 and network.

I guess you're trying to understand what the vlan GUI does by checking the CLI commands. Many Wiki's are out dated too and I relay more on web searches for answers. There are many knowable users but I guess like myself just don't have the time to even answer questions. I do try to at least post about issues I have run into and solutions that fixed it. Sometimes, I do find it frustration when I come across a post where someone fixed their their own issue because they didn't get any help and never state what they did to fix it and becomes an empty post, too me this junk and the whole post should be removed. IMHO, DD-WRT Forums is one of the best for information and support.

Thank-you for your findings!

_________________
Home Network on Telus PureFibre - Wired GigaBit Backbone
2x R7800 - Gateway WiFi 3xVPN - KONGAT 39855M
E3000 - Gateway Wired IPTV - DDWRT r35531 Mega
WHR-HP-G54 - Internal Routing - DDWRT r35531 std-special

______________________
Off Site 1

R7000 - Gateway & WiFi & VPN - KONGAC 39855M
WRT610Nv1 - Repeater Bridge - DDWRT r33679 Mega K2.4

__________________________
Off Site 2

R7000 - Gateway & WiFi - KONGAC 39855M
4x E3000 - Gateway & 3xWDS Wired IPTV - DDWRT r35531 Mega
E2000 - IPTV PVR Blocker - DDWRT r35531 Mega

__________________
YAMon 3.4 | Ad-Blocking for DD-WRT | DNSCrypt-Proxy V2
MrPete
DD-WRT User


Joined: 09 Jul 2013
Posts: 74

PostPosted: Wed Nov 01, 2017 20:56    Post subject: Reply with quote
mac913, thanks for your response. I'm not trying to discern the GUI from the nvram. I'm trying to get the router properly configured by checking the nvram. Smile

You've made a specific claim: that your vlan setup is fine, based only on config via the GUI.

Would you be willing to take a moment to grab your vlan nvram values (see my tutorial) and post them? That would be a great confirmation that your particular VLAN config DOES work properly... which is a great step toward resolving some of the anomalies!

I can think of a few reasons for the differences:

a) Perhaps certain specific VLAN setups may actually configure properly, while others do not. Apparently yours is fine and mine is not Smile

b) A given config may *appear* to work ok w/o additional nvram work behind the scenes... but in reality is unstable, depending on how hard it is pressed and the exact configuration.

AFAIK, the only way to properly test is to check the nvram values after doing a config change. So far, I have not yet had a VLAN setup that works properly *without* manual adjustments.

I'm running the latest (v3.0-r33575M) on my R7000. I just performed a simple test on a spare port: I changed the vlan from 3 (existing) to 5 (unused so far.) Result: misconfigured, and needs additional settings in exactly the way I described above.
JAMESMTL
DD-WRT Guru


Joined: 13 Mar 2014
Posts: 856
Location: Montreal, QC

PostPosted: Wed Nov 01, 2017 22:13    Post subject: Reply with quote
Unless I am missing something, your statement that vlans are misconfigured is based on *your experience of unstable wifi etc* and you relate this to some missing nvram vars.

The presence of an nvram var does not necessarily correlate to the underlying system being misconfgured. Is the nvram var necessary to configure the vlan? if so, does ddwrt use a default when missing?

As ddwrt is mature on the r7000 and many of us have been configuring vlans strictly via the GUI, you would be seeing countless non-functioning vlan + wifi posts which is not the case.

I have personally been running multiple vlans + vlan trunks on a number of r7000s for years with absolutely 0 issues.

There is however a serious issue concerning vlans and broadcom units that ddwrt really needs to address and that is the handling vlans > 15. We really need access to the full vlan 0 - 4095 spectrum even if that is limited to 16 total concurrent vlans.

_________________
IPv6 Ready - HE IPv6 Tunnel
http://test-ipv6.com (10/10)
http://ipv6-test.com (20/20)
http://test-ipv6.netiter.dk (20/20)

MrPete
DD-WRT User


Joined: 09 Jul 2013
Posts: 74

PostPosted: Thu Nov 02, 2017 0:32    Post subject: Reply with quote
Good questions!

I'm downloading the source code to find answers.

My gut sense (I've got a little experience with FW, drivers, real time, etc... ie 40+ years Wink )... uninitialized variables. It all depends...

Those who have rich VLAN configs with no issues, it would be helpful to know what your nvram vlan values are. I can examine the source w/ more understanding that way...

ie assuming your config works AND you have a variety of unset nvram values, knowing which values are set correctly or wrongly for you will help us gain confidence about why your config should be stable... or not.
MrPete
DD-WRT User


Joined: 09 Jul 2013
Posts: 74

PostPosted: Sat Nov 04, 2017 5:38    Post subject: Reply with quote
Answers! (esp for mac913 and jamesmtl)

I now know a lot more than I did a few days ago, having done a fast but pretty deep dive on the DDWRT and driver code, and some simple experiments. (I also have a deeper appreciation for the volume of code and drivers in DDWRT Smile )

A summary of what I learned (some details below):

1) The Broadcom drivers 100% depend on nvram (in particular the two sets of variables NOT being kept up to date -- vlan*hwname and vlan*ports) for initialization.

2) Higher level DDWRT code uses the GUI nvram (port*vlans) to initialize the /proc/* vlan tree info, making things *appear* ok for code that uses /proc/*

3) Depending on the situation, DDWRT code performs some of its own hardware level settings using ifconfig (haven't found the source for that yet.) Not surprising that some scenarios DO work ok w/o proper initialization and updates.

4) Examples of situations that fail w/o proper Broadcom initializations: a variety of ARP scenarios fail completely. Placing an uninitialized vlan in a bridge will almost certainly fail (ie attempt to duplicate the built-in default w/ a second wifi and vlan bridge.) Etc.

I've reconfigured my R7000 both with and without the correct nvram values. Results:
* Without correct config (using an extra vlan bridged with a wifi virtual interface, paralleling the default setup) ARP is broken, the bridged devices work badly, the whole box quickly degrades.
* After restoring correct nvram values (and no other change), the box stabilizes and everything works correctly. HOWEVER, it can take 1-2 complete power cycles and some patience for it to settle down! (My guess: network devices get confused when things don't work as expected.)

A Few Details

Code you can examine for yourself:

start_config_vlan() -- initializes vlan HW for all vlan's where vlan*hwname exists and vlan*ports has values

BCM driver methods:
chipattach() initializes chip...
bcm_robo_config_vlan() does low-level broadcom config of each vlan with ports in vlan*ports

Other Observations
I am impressed by the amount of low level detail our intrepid dev's must be keeping in their heads Smile

Sadly, it's also clear that various bits of code have crept in over time that are essentially patches on patches... and attempts to cover over little-understood issues. (e.g. some things get re-initialized many times just due to the way the loops are set up, while others never get set. The former likely causes no harm, the latter...not so much.)

None of this is particularly surprising. However, one of the basics I learned long ago was that ensuring HW is in a well-known state is more than a little important to having a reliable and predictable platform. So, not at all surprising that various squirrely issues show up on occasion.

NOT a critique of the dev's at all. This is a huge undertaking. I need to step back and consider what (if anything) I can do over time to help improve the overall system.
MrPete
DD-WRT User


Joined: 09 Jul 2013
Posts: 74

PostPosted: Sat Nov 04, 2017 5:48    Post subject: Reply with quote
JAMESMTL wrote:
Is the nvram var necessary to configure the vlan? if so, does ddwrt use a default when missing?


I can say this now with some authority:

* The nvram vars (that are not being updated) ARE necessary for the Broadcom drivers to properly initialize the vlan in the switch.
* If the vars are missing, those vlans and ports on the switch are not properly initialized by the driver -- it skips over them.
* Yes there will be some kind of default value... but for example, the vlans and ports will not be connected, at least by the driver.

Now, DDWRT does do a bit of its own manual configuration, under some circumstances.

Without having a completely configured live code trace / diagnostic system it would be quite difficult to determine exactly which scenarios will succeed and which will fail.

At this point, I'm not surprised some simple setups work ok. I'm also not surprised that other setups fail. That's the nature of uninitialized hardware Smile
MrPete
DD-WRT User


Joined: 09 Jul 2013
Posts: 74

PostPosted: Sat Nov 04, 2017 5:52    Post subject: Reply with quote
JAMESMTL wrote:
We really need access to the full vlan 0 - 4095 spectrum even if that is limited to 16 total concurrent vlans.


Unfortunately, this limitation is not just in DDWRT. The Broadcom drivers have built-in assumptions about VLAN numbers.

I'm sure a workaround can be devised. Given our limited resources, I would want to invent something that requires zero to minimal adjustment to the drivers maintained by the vendor(s).

I'll sleep on it... Zzzzzzz
quarkysg
DD-WRT User


Joined: 03 May 2015
Posts: 297

PostPosted: Sat Nov 04, 2017 7:55    Post subject: Reply with quote
MrPete wrote:
JAMESMTL wrote:
We really need access to the full vlan 0 - 4095 spectrum even if that is limited to 16 total concurrent vlans.


Unfortunately, this limitation is not just in DDWRT. The Broadcom drivers have built-in assumptions about VLAN numbers.

I'm sure a workaround can be devised. Given our limited resources, I would want to invent something that requires zero to minimal adjustment to the drivers maintained by the vendor(s).

I'll sleep on it... Zzzzzzz


The 16 VID limit is artificially set in the switch robo driver in the Linux. The firmware I’ve build for the 868 and 880 D-Link routers allows full 4096 VIDS and VLANs to be configured. You just have to do it using scripts.

For my use case, as my ISP have high VID numbers configured for WAN and IPTV which is impossible to configure using GUI, I just use a startup script to reconfigure my VLANS and its working great for me.
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 3921
Location: Netherlands

PostPosted: Sat Nov 04, 2017 8:37    Post subject: Reply with quote
@Quarkysg, could you also compile a robo switch module like you did for the Shortcut Forwarding Engine (shortcut-fe.ko, which solves the problem with Policy Based Routing and VPN) so that user can test it.

Your work is very much appreciated

_________________
Routers:Netgear R7800, Netgear R6400v1, Netgear R6400v2, Linksys EA6900 (XvortexCFE), Linksys E2000 (converted WRT320N), WRT54GS v1.
Install guide Linksys EA6900: http://www.dd-wrt.com/phpBB2/viewtopic.php?t=291230
Simple PBR (Policy Based Routing) script: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=318662
Install guide R6400v2:http://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
OpenVPN Server Setup:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=318795
Install guide R7800: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
diesel2k
DD-WRT User


Joined: 28 Dec 2009
Posts: 59

PostPosted: Sat Nov 04, 2017 9:39    Post subject: Reply with quote
Hi,

Thanks for the detailed write up. I just added the missing nvram variables to my setup. I have switch port 4 in VLAN 3 used for guest wifi.

This is how my config looks now:

root@DD-WRT:~# nvram show | grep vlan.*ports | sort
size: 37535 bytes (28001 left)
vlan1ports=1 2 3 5*
vlan2ports=0 5u
vlan3ports=4 5
root@DD-WRT:~# nvram show | grep port.*vlans | sort
size: 37535 bytes (28001 left)
port0vlans=2 18 19 21
port1vlans=1 18 19 21
port2vlans=1 18 19 21
port3vlans=1 18 19 21
port4vlans=3 18 19 21
port5vlans=1 2 3 16
root@DD-WRT:~# nvram show | grep vlan.*hwname | sort
size: 37535 bytes (28001 left)
vlan1hwname=et0
vlan2hwname=et0
vlan3hwname=et0
root@DD-WRT:~#

_________________
Internet Router: Edgerouter ER-X v.1.10
Acces Point: R7000 v. Latest Kong
kammerma
DD-WRT Novice


Joined: 27 Dec 2016
Posts: 24

PostPosted: Sat Nov 04, 2017 12:01    Post subject: Reply with quote
To add a data point to the discussion: Configured VLANs exclusively via GUI on my AC68U, which is deployed as an access point. Screenshot attached. Everything working as expected.

root@wap2:~# nvram show | grep vlan.*ports | sort
size: 40776 bytes (24760 left)
vlan1ports=1 2 3 4 5*
vlan2ports=0 5u
root@wap2:~# nvram show | grep port.*vlans | sort
size: 40776 bytes (24760 left)
port0vlans=2 18 19 21
port1vlans=1 11 12 13 16 18 19 21
port2vlans=13 18 19 21
port3vlans=1 18 19 21
port4vlans=1 18 19 21
port5vlans=1 2 11 12 13 16
root@wap2:~# nvram show | grep vlan.*hwname | sort
size: 40776 bytes (24760 left)
vlan1hwname=et0
vlan2hwname=et0



Picture1.png
 Description:
 Filesize:  100.03 KB
 Viewed:  16950 Time(s)

Picture1.png


MrPete
DD-WRT User


Joined: 09 Jul 2013
Posts: 74

PostPosted: Sat Nov 04, 2017 12:30    Post subject: Reply with quote
kammerma wrote:
To add a data point to the discussion: Configured VLANs exclusively via GUI on my AC68U, which is deployed as an access point. Screenshot attached. Everything working as expected.

vlan1ports=1 2 3 4 5*
vlan2ports=0 5u

port0vlans=2 18 19 21
port1vlans=1 11 12 13 16 18 19 21
port2vlans=13 18 19 21
port3vlans=1 18 19 21
port4vlans=1 18 19 21
port5vlans=1 2 11 12 13 16

vlan1hwname=et0
vlan2hwname=et0


GREAT example! Thanks. Can you briefly describe what use is made of port 2, and VLANs 11, 12 and 13?

Those are the elements that are inconsistent in nvram:
* vlan*ports says port 2 is in VLAN1, and has no info on VLANs 11, 12, 13
* vlan*hwname knows nothing about 11, 12, 13
kammerma
DD-WRT Novice


Joined: 27 Dec 2016
Posts: 24

PostPosted: Sat Nov 04, 2017 12:48    Post subject: Reply with quote
The VLANs are primarily used to distributed VPNs to WLANs on various access points.

VLAN11: Connecting WLAN to an OpenVPN client running on the main router
VLAN12: Guest WLAN
VLAN13: Connecting WLAN to an OpenVPN client running on an access point (and port 2 on this access point, connecting to a NAS)

I see the inconsistencies ... never was an issue Smile
quarkysg
DD-WRT User


Joined: 03 May 2015
Posts: 297

PostPosted: Sat Nov 04, 2017 23:03    Post subject: Reply with quote
egc wrote:
@Quarkysg, could you also compile a robo switch module like you did for the Shortcut Forwarding Engine (shortcut-fe.ko, which solves the problem with Policy Based Routing and VPN) so that user can test it.

Your work is very much appreciated


The switch-robo.ko and it's one liner patch is attached in this post. Do try it out on your Broadcom based ARM routers running DD-WRT with Linux kernel version 4.4.x. switch-robo is a kernel driver for Broadcom based SoCs (specifically the BCM53xx) network switch.

Once the kernel module is installed, you will notice that /sys/switch/eth0/vlan will contains 4096 directories instead of just 16 (0-15). Use those to configure your VLANs.

I use the following startup scripts to configure my VLANs. I store the scripts in the JFFS partitions. You can use a USB thumb drive as well. Avoid storing too much in NVRAM (i.e. scripts) as NVRAM space is precious.

# Clear VLAN 0 & 2
echo "" > /proc/switch/eth0/vlan/0/ports
echo "" > /proc/switch/eth0/vlan/2/ports

# Configure VLAN 1 with LAN port 1, 2, 3 and CPU port
echo "0 1 2 5t*" > /proc/switch/eth0/vlan/1/ports

# Configure VLAN 1000 with WAN port and CPU port (both tagged)
echo "4t 5t" > /proc/switch/eth0/vlan/1000/ports

# Configure VLAN 2000 with LAN port 4, WAN port (tagged) and CPU port (tagged)
echo "3 4t 5t" > /proc/switch/eth0/vlan/2000/ports

# Setting up VLAN interfaces ...

# We don't need the vlan2 interface now ...
/sbin/ifconfig vlan2 down
/sbin/vconfig rem vlan2

# Setup vlan1000 interface
/sbin/vconfig add eth0 1000
/sbin/ifconfig vlan1000 up
/sbin/ifconfig vlan1000 txqueuelen 0

# Setup vlan2000 interface
/sbin/vconfig add eth0 2000
/sbin/ifconfig vlan2000 up
/sbin/ifconfig vlan2000 txqueuelen 0

# Tell DD-WRT PPPoE startup code to use the new WAN interface
/usr/sbin/nvram set pppoe_wan_ifname=vlan1000

I've arbitrarily chose VID 1000 and 2000 for illustrative purposes. You should customise the above scripts for your own use.

Of course you can continue to use the NVRAM method to interface with the DD-WRT configure subsystems, but I find using scripts the most straight forward and flexible.

HTH.



linux-4.4-switch-robo.zip
 Description:
DD-WRT Linux 4.4x switch-robo.ko kernel module (ARM)

Download
 Filename:  linux-4.4-switch-robo.zip
 Filesize:  9.21 KB
 Downloaded:  160 Time(s)

Goto page 1, 2, 3  Next Display posts from previous:    Page 1 of 3
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum