Update2:
Wondering if the issue is that no where is the gateway 192.168.200.1 specified in the DD-WRT setup, so clients don't know where the route to requesting a lease.
This suspicion proved to be correct.
I appreciate my application is somewhat niche, but I do believe it has lots of value for home users like me.
Thanks again for the help, Per! I have split the details of achieving my goal into a separate thread for clarity & to make it easier to find.
Joined: 13 Aug 2013 Posts: 6870 Location: Romerike, Norway
Posted: Mon Jan 25, 2021 15:58 Post subject:
chaski wrote:
Update2:
Wondering if the issue is that no where is the gateway 192.168.200.1 specified in the DD-WRT setup, so clients don't know where the route to requesting a lease.
DHCP must be disabled on dd-wrt and the DHCP server at OPNsense will provide IP address and gateway.
That is if the OPNsense is connected to port 1 (labelled 4 on the casing)
First, Thank you very much to everyone who contributed in this thread. With your help, I was able to set up my network as I wished
What was my goal?
I have a little Server in my LAN that runs a bunch of Docker Containers (Nextcloud, Jellyfish, Caddy...) and I wanted to expose Nextcloud to the Internet, so I can use it from everywhere. Usually people just port forward and that´s it. But I wanted to isolate Nextcloud and Caddy each into their separate VLANs, make them accessible from my LAN + Wifi and also from the Internet. On the other hand, those VLANs should not have access to my LAN or WiFi and I think I just made it. On top of that I still want to use that Server to host Containers for my LAN, so the Interface has to be available in the LAN + for VLANs and all should be isolated.
This is in the Startup Commands section to create the VLANs:
Code:
swconfig dev switch0 set enable_vlan 1
swconfig dev switch0 vlan 1 set ports "1 2 3 4 6"
swconfig dev switch0 vlan 10 set ports "3t 6t"
swconfig dev switch0 vlan 20 set ports "3t 6t"
swconfig dev switch0 set apply
vconfig add eth1 10
vconfig add eth1 20
ifconfig eth1.10 192.168.1.1 netmask 255.255.255.0
ifconfig eth1.20 192.168.2.1 netmask 255.255.255.0
Pay attention to Port #3. This is included in the VLAN 1 and additionally and tagged in VLAN 10 and 20. Why? On Port 3 is the Server with my Docker Containers connected. I still want to run various Containers for LAN usage, but want this Interface to have additional VLANs. In Docker I defined the VLANs with the macvlan driver, so the Containers can use them.
In the Firewall Commands I included these IPTable rules:
Code:
iptables -t nat -I POSTROUTING -o `get_wanface` -j MASQUERADE
iptables -I FORWARD -i eth1.10 -o `get_wanface` -m state --state NEW -j ACCEPT
iptables -I FORWARD -i eth1.20 -o `get_wanface` -m state --state NEW -j ACCEPT
The first one I got from here and I wonder if this is too open. The default MASQUERADE rule created by DD-WRT on Startup is much more strict, reading
Code:
-A POSTROUTING -s 192.168.0.0/24 -d 192.168.0.0/24 -o br0 -j MASQUERADE
and since my Rule sits on the top, I think it makes the one from DD-WRT useless. It would probably be more secure to create that MASQUERADE rule including the VLAN Subnets, but I have yet to try. I think stricter is better, right?
The 2 following IPTables rules I created myself were needed so the Containers in the VLANs can communicate outwards into the Internet.
I made various tests from within those Containers and none was able to connect anywhere else than to his own VLAN and the Internet.
Now if I want to allow Traffic from my Caddy Proxy (192.168.1.2) to the Nextcloud Apache Instance (192.168.2.4), usually I would set this rule:
But since the Nextcloud Containers do not have fixed IP Addresses, I left this rule out and just included the Nextcloud VLAN in the compose file for the Caddy Container. This way Name resolution between those 2 VLANs works and the Containers can reach each other.
I also did not define additional DHCP Servers in the DD-WRT GUI, since both VLANs are only used for Docker Containers and Docker takes care of assigning IP addresses to the containers itself.
To make Nextcloud available via the Caddy Reverse Proxy, I just had to create a Port forwarding rule in the DD-WRT Web Interface to Port 443 of the Caddy Container IP.
One hairy thing though! After I tested the whole setup, I just deactivated the Port Forwarding Rule in the Web Interface and clicked APPLY. The Nextcloud Instance was still working from external, although testing for Port 443 failed. I have no real explanation for that, to be honest. I even created / uploaded a new file in Nextcloud and it was working. I then rebootet DD-WRT and Nextcloud stopped working. This was a bit concerning, but now I know if I enable / disable Port Forwarding Rules, I better restart DD-WRT, because things can get wonky.
Thanks again to everyone for your examples. This made it possible for me to figure out how to setup my VLANs
Joined: 18 Mar 2014 Posts: 12921 Location: Netherlands
Posted: Fri Mar 31, 2023 12:59 Post subject:
You can save yourself some trouble by using bridges e.g. creata br1 and br2 and then assigning the VLANs to their respective bridges.
When creating the bridges in the GUI the necessary firewall rules for masquerading are automatically made.
It is not the rule you mention, for MASQUERADING traffic out of the WAN an SNAT rule is used as SNAT is much faster than MASQUERADING (but not always possible) e.g.:
iptables -t nat -I POSTROUTING -s 192.168.2.1 -o $(get_wanface) -j SNAT --to $(nvram get wan_ipaddr)
MASQUERADING also works but is slower and as you do not specify an interface now you are MASQUERADING all traffic over all interfaces and that makes it very difficult to troubleshoot or log adequately.
One advantage of MASQUERADING is that all traffic from outside now comes from the router so your servers will allow that by default otherwise you might need to tweak the firewall of the server (port forwarding, open ports etc)
Thank you very much egc for your comments! You made me dive deeper into the MASQUERADE vs SNAT topic and I did quite some reading and I think I understand now why SNAT is quicker. One downside usually is that with SNAT you have to define the IPs and it "should" be problematic when the WAN IP of the router changes, but I suspect that DD-WRT triggers a bunch of stuff anyway when the WAN IP changes and SNAT is not an issue?
egc wrote:
You can save yourself some trouble by using bridges e.g. creata br1 and br2 and then assigning the VLANs to their respective bridges.
I tried that by
* Removing my iptables rules from Admin/Commands
* Created 2 new bridges br1 and br2 in Setup/Networking
* Assigned VLAN10 to br1 and VLAN20 to br2 in the same page
* Scrolled down to Interface setup and set VLAN10 and VLAN20 back from Unbridged to Default
* Put in the Subnets I had previously on VLAN10 and 20 to br1 and br2
* Enabled Network isolation on the br1 and br2 interfaces
Rebooted and somehow was not able to get to or out of the 2 VLANs now on the br1 and br2 interfaces. I could not figure out what iptables rules to set to get back to the previous state, so I reverted everything back and just substituted my previous MASQUERADE rule to an snat rule like this:
This brought me back to my previous state (access to the VLANs and access to the Internet from the VLANs + Port forwarding to the VLANs possible).
I am still curious about 2 things:
* Will adding bridging as an additional layer have a positive effect on performance or was it just a suggestion to get rid of my 2 iptables FORWARD rules? Because it did less as it does now, so I suspect I would have neede additional iptables rules also when using bridge interfaces?
* I do have a performance problem when accessing services from the Internet in my LAN / VLAN. My upload speed usually is 50 Mbit/s, but when I access a service from the outside, it maxes out at around 12 Mbit/s and I can not figure out yet what the problem is. Since it probably has nothing to do with VLAN, I think I will create another post, since I was not able to see this discusses previously yet.
egc wrote:
Unless you are placing the Servers/VLAN/BRIDGE IP's in the DMZ you still have to port forward.
Unfortunately the DD-WRT Web Interface for DMZ defaults to the LAN Network, the VLANs do not show up, so I was not able to test the DMZ feature. I was curious to see if performance would be better than the Port forwarding, which seems to be really slow for me, unfortunately.
Joined: 18 Mar 2014 Posts: 12921 Location: Netherlands
Posted: Sun Apr 02, 2023 12:50 Post subject:
SNAT works in this case for DDWRT because a lot of things are triggered when the WAN ip changes so you should be good
I actually have it running with VLAN3 assigned to a bridge (br1) so something went wrong when you tried it, the mistake often happens is if you are impatient, working with bridges requires patience, when a bridge is made a lot of things need to be adapted, so after creating a bridge (and between all other steps) wait at least a minute and when you are done reboot.
A bridge is not needed per se but some firewall processes take only brX into account and not unbridged VLANs of VAPs.
Besides you need a bridge if you want to add a more interfaces like a VAP.
But as said not needed per se, if you use the GUI and unbridge the VLAN then it will work in your case as well without needing any extra iptables rules.
I did a quick test and deleted my VLAN 3 from my bridge, I unbridged VLAN 3, gave it its own subnet and setup a DHCP server.
Note in the GUI I have Masquerade / NAT enabled (although that actually SNATs )
See attachement
All these iptables you see are automatically created no need to manually do anything:
I will give it another shot, but there is another equation in the mix I should take care of first probably. And this is my DD-WRT build is 3.0-r49626 std which might be a bit outdated. So I will upgrade first to the latest before I do more testing. Currently stuff works (apart from the performance), but I think updating first makes sense