# Routing between subnets (VLAN-s) [SOLVED]

## mocsokmike

Hi,

I am struggling with this problem for days now. I am planning to reform our network by splitting it to four different subnets, each using their own VLAN, in the following way:

VLAN 10 (servers), 192.168.10.0/24

VLAN 20 (company 1), 192.168.20.0/24

VLAN 30 (company 2), 192.168.30.0/24

VLAN 40 (guests), 192.168.40.0/24

I have several virtual servers, they are connected to the necessary VLAN-s now. This part works.

My main gateway is connected to all VLAN-s except 40 (because guests should use a separate gateway in the future).

What I would like to achieve is to allow some admins from VLAN 20 to access VLAN 10 via my main gateway. My plan is to give them fixed IP-s with the DHCP server, and to allow these IP-s access the other network. However, all my previous attempts in routing failed, and I believe it is because the VLAN tags.

Is this even possible? If yes, what do I need to do to allow certain IP-s in VLAN 20 into VLAN 10?

Since a lot of configuration could be involved, I am not posting all of them now, please ask what you need and I will paste their respective parts here.

----------

## chiefbag

Are all the VLANS on the same physical network card.

Whats the output of your "ifconfig" on the gateway and "route -n"

Have you any iptables rules specifically limiting ingress of traffic?

I assume net.ipv4.ip_forward = 1 must be set as you have access from all subnets through gateway to external?

----------

## mocsokmike

Yes, all VLAN-s are on eth0.

The VLAN-specific part of my ifconfig output:

```
eth0_com1   Link encap:Ethernet  HWaddr f2:6c:be:22:9b:7f

          inet addr:192.168.20.253  Bcast:192.168.20.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:16659 errors:0 dropped:764 overruns:0 frame:0

          TX packets:22517 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:2253720 (2.1 MiB)  TX bytes:25614035 (24.4 MiB)

eth0_srv  Link encap:Ethernet  HWaddr f2:6c:be:22:9b:7f

          inet addr:192.168.10.253  Bcast:192.168.10.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:33964 errors:0 dropped:0 overruns:0 frame:0

          TX packets:36079 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:8076734 (7.7 MiB)  TX bytes:7877643 (7.5 MiB)

eth0_com2   Link encap:Ethernet  HWaddr f2:6c:be:22:9b:7f

          inet addr:192.168.30.253  Bcast:192.168.30.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
```

VLAN-related part of route -n:

```
Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0_srv

192.168.20.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0_com1

192.168.30.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0_com2
```

I allow traffic from these VLAN-s in iptables (related part of my firewall script):

```
iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT

iptables -A FORWARD -s 192.168.10.0/24 -m state --state NEW -j ACCEPT

iptables -A FORWARD -s 192.168.20.0/24 -m state --state NEW -j ACCEPT

iptables -A FORWARD -s 192.168.30.0/24 -m state --state NEW -j ACCEPT
```

The gateway works as a router:

```
cat /proc/sys/net/ipv4/ip_forward

1
```

----------

## chiefbag

That looks ok to me, I have tested this with a similar setup without any iptables rules and all works.

Perhaps there is some blanket iptables rules blocking, aside from that some kernel config issues.

Included is for example the settings for PRI.94 VLAN

```
cat /usr/src/linux/.config | grep VLAN | grep -v "is not set"

CONFIG_BRIDGE_EBT_VLAN=m

CONFIG_BRIDGE_VLAN_FILTERING=y

CONFIG_VLAN_8021Q=y
```

```

sysctl -a | grep -w "PRI/94"

net.ipv4.conf.PRI/94.accept_local = 0

net.ipv4.conf.PRI/94.accept_redirects = 0

net.ipv4.conf.PRI/94.accept_source_route = 1

net.ipv4.conf.PRI/94.arp_accept = 0

net.ipv4.conf.PRI/94.arp_announce = 0

net.ipv4.conf.PRI/94.arp_filter = 0

net.ipv4.conf.PRI/94.arp_ignore = 0

net.ipv4.conf.PRI/94.arp_notify = 0

net.ipv4.conf.PRI/94.bootp_relay = 0

net.ipv4.conf.PRI/94.disable_policy = 0

net.ipv4.conf.PRI/94.disable_xfrm = 0

net.ipv4.conf.PRI/94.force_igmp_version = 0

net.ipv4.conf.PRI/94.forwarding = 1

net.ipv4.conf.PRI/94.igmpv2_unsolicited_report_interval = 10000

net.ipv4.conf.PRI/94.igmpv3_unsolicited_report_interval = 1000

net.ipv4.conf.PRI/94.log_martians = 1

net.ipv4.conf.PRI/94.mc_forwarding = 0

net.ipv4.conf.PRI/94.medium_id = 0

net.ipv4.conf.PRI/94.promote_secondaries = 0

net.ipv4.conf.PRI/94.proxy_arp = 0

net.ipv4.conf.PRI/94.proxy_arp_pvlan = 0

net.ipv4.conf.PRI/94.route_localnet = 0

net.ipv4.conf.PRI/94.rp_filter = 1

net.ipv4.conf.PRI/94.secure_redirects = 1

net.ipv4.conf.PRI/94.send_redirects = 0

net.ipv4.conf.PRI/94.shared_media = 1

net.ipv4.conf.PRI/94.src_valid_mark = 0

net.ipv4.conf.PRI/94.tag = 0

net.ipv4.neigh.PRI/94.anycast_delay = 100

net.ipv4.neigh.PRI/94.app_solicit = 0

net.ipv4.neigh.PRI/94.base_reachable_time_ms = 30000

net.ipv4.neigh.PRI/94.delay_first_probe_time = 5

net.ipv4.neigh.PRI/94.gc_stale_time = 60

net.ipv4.neigh.PRI/94.locktime = 100

net.ipv4.neigh.PRI/94.mcast_solicit = 3

net.ipv4.neigh.PRI/94.proxy_delay = 80

net.ipv4.neigh.PRI/94.proxy_qlen = 64

net.ipv4.neigh.PRI/94.retrans_time_ms = 1000

net.ipv4.neigh.PRI/94.ucast_solicit = 3

net.ipv4.neigh.PRI/94.unres_qlen = 31

net.ipv4.neigh.PRI/94.unres_qlen_bytes = 65536
```

----------

## mocsokmike

CONFIG_BRIDGE_VLAN_FILTERING was missing in my kernel - I added it now, but I am not in the office at the moment and won't be until Monday. I will test it then.

I don't have CONFIG_BRIDGE_EBT_VLAN available in my kernel (4.4.8-hardened-r1).

Now it looks like this:

```
cat /usr/src/linux/.config | grep VLAN | grep -v "is not set"

CONFIG_BRIDGE_VLAN_FILTERING=y

CONFIG_VLAN_8021Q=m

CONFIG_MACVLAN=m
```

My VLAN setting also differ in a few places, most likely because of my different kernel version:

```
sysctl -a | grep -w "eth0_com1"

net.ipv4.conf.eth0_com1.accept_local = 0

net.ipv4.conf.eth0_com1.accept_redirects = 0

net.ipv4.conf.eth0_com1.accept_source_route = 1

net.ipv4.conf.eth0_com1.arp_accept = 0

net.ipv4.conf.eth0_com1.arp_announce = 0

net.ipv4.conf.eth0_com1.arp_filter = 0

net.ipv4.conf.eth0_com1.arp_ignore = 0

net.ipv4.conf.eth0_com1.arp_notify = 0

net.ipv4.conf.eth0_com1.bootp_relay = 0

net.ipv4.conf.eth0_com1.disable_policy = 0

net.ipv4.conf.eth0_com1.disable_xfrm = 0

net.ipv4.conf.eth0_com1.force_igmp_version = 0

net.ipv4.conf.eth0_com1.forwarding = 0

net.ipv4.conf.eth0_com1.igmpv2_unsolicited_report_interval = 10000

net.ipv4.conf.eth0_com1.igmpv3_unsolicited_report_interval = 1000

net.ipv4.conf.eth0_com1.ignore_routes_with_linkdown = 0

net.ipv4.conf.eth0_com1.log_martians = 0

net.ipv4.conf.eth0_com1.mc_forwarding = 0

net.ipv4.conf.eth0_com1.medium_id = 0

net.ipv4.conf.eth0_com1.promote_secondaries = 0

net.ipv4.conf.eth0_com1.proxy_arp = 0

net.ipv4.conf.eth0_com1.proxy_arp_pvlan = 0

net.ipv4.conf.eth0_com1.route_localnet = 0

net.ipv4.conf.eth0_com1.rp_filter = 1

net.ipv4.conf.eth0_com1.secure_redirects = 1

net.ipv4.conf.eth0_com1.send_redirects = 1

net.ipv4.conf.eth0_com1.shared_media = 1

net.ipv4.conf.eth0_com1.src_valid_mark = 0

net.ipv4.conf.eth0_com1.tag = 0

net.ipv4.neigh.eth0_com1.anycast_delay = 100

net.ipv4.neigh.eth0_com1.app_solicit = 0

net.ipv4.neigh.eth0_com1.base_reachable_time_ms = 30000

net.ipv4.neigh.eth0_com1.delay_first_probe_time = 5

net.ipv4.neigh.eth0_com1.gc_stale_time = 60

net.ipv4.neigh.eth0_com1.locktime = 100

net.ipv4.neigh.eth0_com1.mcast_resolicit = 0

net.ipv4.neigh.eth0_com1.mcast_solicit = 3

net.ipv4.neigh.eth0_com1.proxy_delay = 80

net.ipv4.neigh.eth0_com1.proxy_qlen = 64

net.ipv4.neigh.eth0_com1.retrans_time_ms = 1000

net.ipv4.neigh.eth0_com1.ucast_solicit = 3

net.ipv4.neigh.eth0_com1.unres_qlen = 31

net.ipv4.neigh.eth0_com1.unres_qlen_bytes = 65536
```

----------

## mocsokmike

Still not working. I see no blocking firewall rules - for example, this is in the beginning of my firewall script, and my pings are not going from VLAN 20 to VLAN 10:

```
iptables -A INPUT -p icmp -j ACCEPT

iptables -A FORWARD -p icmp -j ACCEPT
```

Do you have a network bridge?

----------

## chiefbag

No I don't have a br setup.

Have you tried to telnet a port across the VLANs instead of ping?

----------

## chiefbag

Example of /etc/conf.d/net

```
config_PRI="172.10.90.1 netmask 255.255.255.0"

vlans_PRI="94 95"

config_PRI_94="172.10.94.1 netmask 255.255.255.0"       

config_PRI_95="172.10.95.1 netmask 255.255.255.0" 
```

----------

## chiefbag

Have you rebooted the gateway considering "CONFIG_BRIDGE_VLAN_FILTERING" was not built as a module?

 *Quote:*   

> CONFIG_BRIDGE_VLAN_FILTERING was missing in my kernel - I added it now, but I am not in the office at the moment and won't be until Monday. I will test it then. 
> 
> I don't have CONFIG_BRIDGE_EBT_VLAN available in my kernel (4.4.8-hardened-r1). 
> 
> Now it looks like this: 
> ...

 

----------

## mocsokmike

Still not working.

I have rebooted the gateway after the kernel modification. My test using telnet didn't work either.

So you say this should be OK, as your settings are very much similar and you can access another VLAN from a client, via your gateway?

The client I am trying with only sees VLAN 20. It is connecting to a wifi AP and that SSID can access only VLAN 20. Its gateway can access VLAN 10.

From this client, I can ping the gateway's other interfaces (like its VLAN 10 IP address, for example), but no other IP-s from the other IP ranges.

----------

## chiefbag

If you can reach the VLAN 10 gateway ip from the VLAN20 client then it must be down to an iptables/ebtables rule, the only other thing I could think of is the possibility that some type of filtering is occurring on your AP, there is a "client isolation" option on most AP's but this should only effect clients on the same subnet/SSID/VLAN ( or so I would imagine ).

----------

## mocsokmike

OK, I was testing it in a wrong way. I can ping other network endpoints, like a printer, which is in a different IP segment. The only things I cannot ping are my Gentoo VM-s. The problem was I tried these first.

This means the gateway works, but I have a firewall or network configuration problem on all my VM-s, disallowing incoming network traffic from one VLAN to another VLAN interface of that server.

For example, the IP addresses of my DHCP server are:

VLAN 10: 192.168.10.231

VLAN 20: 192.168.20.231

From a client in VLAN 20, I can ping only its VLAN 20 IP address.

I can ping one client on another VLAN, and this should go through my gateway.

I use the same iptables rules for all VLAN at the moment.

My Gentoo VM-s only answer a ping from their VLAN network interface in the same VLAN, no another.

Can you help me what setting can cause this?

----------

## chiefbag

Not sure, perhaps this is a different question as such.

When you say VM's what platform are you using, is it VMWare Workstation, ESXi?

----------

## mocsokmike

I use Citrix XenServer. I don't think it would limit my VM-s network traffic in any way.

I am starting to believe this phenomenon is a function of my kernel (I use hardened sources).

What I described earlier sounds like a security feature.

Do you use hardened kernel?

----------

## chiefbag

Not using hardened kernel. 

I imagine it must be to do with the Citrix setup as you say you can reach other physical devices such as printer inter VLAN

Maybe there is some security on the Citrix dom0 setup but I'm not familiar with it.

----------

## mocsokmike

OK, I got it!

I needed to SNAT the cross-VLAN communication using iptables. This is an example line, to SNAT VLAN 20 to VLAN 10 after routing decision was made:

```
iptables -t nat -A POSTROUTING -o eth0_srv -s 192.168.20.0/24 -j SNAT --to 192.168.10.253
```

I will mark this thread solved. Thank you for all your help, chiefbag! It seems your firewall-suspicion was correct.

----------

