MrPete
DD-WRT User
Joined: 09 Jul 2013
Posts: 82
|
 Posted: Sat May 18, 2019 16:20 Post subject: HOW TO: R7000 w/ 4 tagged VLAN Bridged to Nx VAP |
 |
UPDATE Dec 12, 2019: Added more details...
Thanks to a great bugfix from BrainSlayer and other help, I've got this working reliably. Many lessons learned shared here...
Summary (what you'll find below)
- Diagnosing/fixing stability issues with multiple WiFi interfaces
- Actual unique-MAC requirements
- Enabling many WiFI interfaces, vLANs, Bridges, etc
- Diagnosing performance limits of DD-WRT, and a solution
- Changing from an all-DD-WRT solution, to DD-WRT as WAP
- VLAN Trunking: undocumented req'ts
- A clean/simple multi-interface, multi-subnet WAP example
Goals for my own setup
- Five WiFI SSID's on multiple subnets (2 public, 2 private, 1 for IoT)
- OpenVPN server linking to remote small office with net-to-net SMB networking
- Internal gigabit LAN for work, email hosting, 2 NAS boxes, etc
- Upgrade from 40mbit fiber to gigabit fiber
Initial Equipment
- R7000 with Kong firmware
- Multiple not-so-smart switches
Initial Challenges
- Couldn't get the WiFi's to stay up with stability over time
- R7000 CPU overload at gigabit speed, let alone OpenVPN
Diagnostics
I learned the hard way that the instability was somehow related to MAC address uniqueness. But exactly what was the issue wasn't clear until recently. Now I know:
Instability: All WiFi interface MAC addresses MUST be unique.
Here are (Shell/SSH) commands to understand your reality: | Code: | #ifconfig|grep HWaddr|sort
(this shows overview incl MAC for each interface)
#wl -i <interface> bssid
(this shows at a low level) |
Important: Bridges can all share one MAC. VLANs likewise. They are not the issue.
Performance: How to determine overload in ddwrt etc?
| Code: |
#top (shows per-process loads, and overall interrupt loads)
#iperf3 (throughput to/from router. Usually you want *through* router tests between two endpoints. Download iperf for your various boxen ;) )
|
Result: Neither R7000 nor other "affordable" routers can offload much of the packet processing or encryption to hardware.
Stable Solution
* As of BS changeset 39780, the MAC address issue is fixed. It bumps the 5G WiFi MAC to be another 16 addresses bigger than 2.4G MAC. No danger of overlap.
* You can manually poke this into your own nvram for an older release: | Code: | nvram show|grep pci/2/1/macaddr
(add one to the upper nibble of the last byte. E.g. if E7 change to F7. Should be ok to change F7 to 07.)
nvram set "pci/2/1/macaddr"=(your fixed MAC)
nvram commit
[b]reboot[/b] |
IMPORTANT: If you manually set this, unless you clear nvram first, this will NOT fully propagate everywhere! For example, the Setup->Networking page still shows the wrong MAC for eth2 (5G Wifi)...
* In addition, I'm using the startup "workaround" script others have used. In my experience, making changes to the WiFi interfaces is NOT effective until I reboot and allow this script to run. I do NOT know if all of these steps are necessary.
| Code: |
sleep 10;stopservice nas;stopservice wlconf;startservice wlconf;startservice nas
wlconf eth1 down
wlconf eth1 up
wlconf eth2 down
wlconf eth2 up
|
Again: reboot after any change involving Wifi, VLAN and/or Bridge interfaces.
My High Performance Solution
Let me say up front... if it were not for my OpenVPN requirement, I'm not at all sure that all of this was necessary. It was cheap and I could do it  ...
HOWEVER, what follows is far more complex than a single DD-WRT Wifi Router/Switch/OpenVPN Server. DD-WRT by itself is incredibly simpler to configure and maintain. INCREDIBLY simpler .
First, I spent $50 on two things:
* $40 on a Netgear GSS108e 8 port smart switch (what a deal, lifetime warranty)
* $10 on a good gigabit ethernet PCIe card
I have a spare old quad core mini-desktop case computer, and a spare SSD. Combine the computer, the spare ethernet card, and the SSD... voila a great platform for a pfSense firewall/router!
New configuration:
(ISP) <--> (pfSense) <--> (Smart Switch) <--> (DDWRT)
* DDWRT only handles WiFi, VLAN trunking and bridging
* pfSense handles OpenVPN, DNSmasq, DHCP, etc
* 7 trunked VLANs, pfSense to Smart Switch
* 4 trunked VLANs, Smart Switch to DDWRT
* 2 VLANs uniquely Smart Switch to wired subnets
* 1 VLAN for DD-WRT management
* 2 VLANs both wired and WiFi, handled at the Smart switch
* 2 VLANs are pure WiFi
* *.1 subnet gateways are at pfSense
* *.250 are subnet access to DD-WRT (SSH and browser)
* Pretty much all PC-type ethernet cards easily offload network I/O, checksums, VLAN tagging and more
DD-WRT Configuration
- First, ensure you can use wired ethernet to configure DD-WRT cleanly, on an UNtagged port. I created VLAN 10, bridged to combine br0, VLAN1, VLAN2, VLAN10. That plugs into an untagged port on my smart switch. yes, a "wasted" extra cable... but it simply works while I configure everything else
For each VLAN-WiFi combination:
- Configure one new bridge (I have br9, br11, br12, br15)
- Set up each WiFi interface as Bridged
- Attach one VLAN and one or more WiFI's to each bridge
- Define IP info on each bridge (Setup->Networking), this defines the IP of that subnet at your DD-WRT box. Leave things like Masq/NAT, Net Isolation, etc OFF. Those happen at pfSense in my configuration.
- (For my config, my DD-WRT firewall is off, dnsmasq off, etc.)
- Remember to manually configure the nvram as described in the Wiki entry for Switched Ports: https://wiki.dd-wrt.com/wiki/index.php/Switched_Ports
Once that's all set up, from SSH you should see everything looking good using "brctl show"
Then I enable the VLAN trunk:
- Configure the desired VLANs on one DD-WRT port (I use port 4. VLANs 9, 11, 12, 15 are all there, and tagged) (And fix up the nvram again)
- Add the trunked wire
Result:
* Both pfSense box and DDWRT are coasting
* Full performance to the limit of my networks
* (My sense: Quad core is overkill except for OpenVPN)
* pfSense supports multiple MAC for one IP. VERY nice for laptops and such!
Lessons Learned
* Don't use VLAN 1 to connect to other boxes. It usually has special meaning.
* Always preserve a simple non-tagged-VLAN way to connect to a port
* It helps to have a spare port that can be configured for one untagged VLAN
* When trunking VLANs, the trunk port MUST only use tagged VLANs. NO UNTAGGED. It will cause grief. This is true with DD-WRT, with pfSense, and with a Smart switch.
* For diagnostics, local tcpdump is your friend. I still don't have WireShark showing VLAN info. On tcpdump:
tcpdump -env VLAN [nn] -- shows all VLAN tagged traffic (optionally for a single vlan)
tcpdump -env VLAN and port 67 -- all VLAN tagged traffic for a given port
tcpdump -env port 67 and VLAN -- DOES NOT WORK. VLAN must come first, because it strips part of the header! (Multiple VLAN entries assume you want to look at VLAN-in-VLAN encapsulation!?!!) |
|
