# [Solved] Need help on bridge

## redwood

Hi,

I'm trying to connect an lxc container to a bridge.

On my host computer I have 2 RJ45 ports on the motherboard, but I only use one of them.

In the past, I've used the following configuration for running qemu images without network problems:

```

# grep -v "^$" /etc/conf.d/net |grep -v "^#"

config_eth0="null" #disable dhcp on eth0

config_br0="192.168.1.40 netmask 255.255.255.0 brd 192.168.1.255"

routes_br0="default via 192.168.1.1"

RC_NEED_br0="net.eth0"

brctl_br0="setfd 0 

           sethello 1 

           stp off"

bridge_br0="eth0"

config_eth1="noop" #don't need this RJ45 port

```

```

# route -n

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

0.0.0.0         192.168.1.1     0.0.0.0         UG    57     0        0 br0

127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo

127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo

169.254.0.0     0.0.0.0         255.255.0.0     U     6      0        0 eth0.1

192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br0

# brctl show

bridge name     bridge id               STP enabled     interfaces

br0             8000.003048f6faf3       no              eth0

                                                        vethBAA72P

```

On my lxc container I have the follwing config:

```

# cat /var/lib/lxc/mail/config 

# Template used to create this container: /usr/share/lxc/templates/lxc-gentoo

# Parameters passed to the template:

# For additional config options, please look at lxc.container.conf(5)

lxc.utsname = mail 

lxc.autodev=1

lxc.tty = 1

lxc.pts=1024

lxc.cap.drop=sys_module mac_admin mac_override sys_time

lxc.kmsg=0

lxc.stopsignal=SIGRTMIN+4

# networking

lxc.network.type  = veth

lxc.network.link  = br0 

lxc.network.flags = up

lxc.network.name  = eth0

lxc.network.ipv4  = 192.168.1.41/24 

lxc.network.ipv4.gateway = 192.168.1.1

lxc.network.mtu = 1500

# DHCP

#lxc.network.ipv4 = 0.0.0.0

lxc.network.hwaddr = 02:3f:65:58:3c:02

#inet 192.168.1.41  netmask 255.255.255.0  broadcast 192.168.1.255

#inet6 fe80::3f:65ff:fe58:3c02  prefixlen 64  scopeid 0x20<link>

#inet6 fd8f:f36f:b732:0:3f:65ff:fe58:3c02  prefixlen 64  scopeid 0x0<global>

#ether 02:3f:65:58:3c:02  txqueuelen 1000  (Ethernet)

#lxc.mount = /etc/lxc/mail.fstab 

lxc.rootfs = /var/lib/lxc/mail

lxc.console=/var/log/lxc/mail.console

lxc.rootfs = /var/lib/lxc/mail

### lxc-gentoo template stuff starts here

# sets container architecture

# If desired architecture != amd64 or x86, then we leave it unset as

# LXC does not oficially support anything other than x86 or amd64.

lxc.arch = amd64

#container set with shared portage

lxc.mount.entry=/usr/portage usr/portage none ro,bind 0 0

lxc.mount.entry=/usr/portage/distfiles usr/portage/distfiles none rw,bind 0 0

#If you use eix, you should uncomment this

lxc.mount.entry=/var/cache/eix var/cache/eix none ro,bind 0 0

lxc.include = /usr/share/lxc/config/gentoo.common.conf

#cgroups

lxc.cgroup.devices.deny = a

lxc.cgroup.devices.allow = c *:* m

lxc.cgroup.devices.allow = b *:* m

lxc.cgroup.devices.allow = c 1:3 rwm

lxc.cgroup.devices.allow = c 1:5 rwm

lxc.cgroup.devices.allow = c 1:7 rwm

lxc.cgroup.devices.allow = c 1:8 rwm

lxc.cgroup.devices.allow = c 1:9 rwm

lxc.cgroup.devices.allow = c 4:1 rwm

lxc.cgroup.devices.allow = c 5:0 rwm

lxc.cgroup.devices.allow = c 5:1 rwm

lxc.cgroup.devices.allow = c 5:2 rwm

lxc.cgroup.devices.allow = c 136:* rwm

```

The host and lxc guest can ping each other, 

but the lxc guest cannot ping other computers on my 192.168.1.0/24 lan 

and can't ping ip's on the wan.

My host runs shorewall for its firewall

and I have ip forwarding on:

```

# grep IP_FORW /etc/shorewall/shorewall.conf 

IP_FORWARDING=On

```

And ip_forward'ing is enabled:

```

# cat  /proc/sys/net/ipv4/ip_forward

1

```

Any ideas?

Thanks.Last edited by redwood on Mon Mar 09, 2015 2:05 pm; edited 3 times in total

----------

## redwood

The problem is due to my HOST's shorewall configuration:

```

Shorewall:loc2net:REJECT:IN=br0 OUT=br0 PHYSIN=vethIA5FC3 ...

Shorewall:loc2fw:ACCEPT:IN=br0 OUT= PHYSIN=eth0

```

```

# brctl show

bridge name     bridge id               STP enabled     interfaces

br0             8000.003048f6faf3       no              eth0

                                                        vethIA5FC3

```

My SOHO setup is this:

       WAN

        |

      DSL

        |

=============

| OpenWRT router  |

=============

        |

    ====================

    | Switch  (192.168.1.0/24 )     |

    ====================

        |     |         |         |        |

        |     Host2  Host3  Host4  Switch2 (192.168.1.0/24)

        |                                       |

        |                                     More pc's

        |

        |

        |

        br0 (eth0) ----------------------------------------------------------------------------------------------------

        |                                                         |                                                      |                         |

===================                  ======================       =========      =========

|        Host1 (192.168.1.40)  |                  |    LXC1  mail (192.168.1.41)     |       | LXC2 www  |      | LXC3 pbx   |

===================                  ======================       =========      =========

I want to put move public side services 

such as mail, web, pbx, etc.,

from Host 1  into  lxc containers

I could put the lxc conainers on a separate subnet from the Host, but I'm not sure I need that at this point.

I just want basically virtual lxc computers to appear to be part of my network and accessible from the net

----------

## redwood

Found part of the solution here:

[url]

http://serverfault.com/questions/445991/bridging-lxc-containers-to-host-eth0-so-they-can-have-a-public-ip

[/url]

Apparently, I had CONFIG_BRIDGE_NETFILTER=y compiled into my kernel, and therefore the kernel was directing all ip traffic from the bridge through netfilter for routing.

The quick solution to test this out: 

```

HOST# cd /proc/sys/net/bridge

HOST# ls

bridge-nf-call-arptables  bridge-nf-call-iptables        bridge-nf-filter-vlan-tagged

bridge-nf-call-ip6tables  bridge-nf-filter-pppoe-tagged  bridge-nf-pass-vlan-input-dev

HOST# for f in bridge-nf-*; do echo 0 > $f; done

```

And for the future, to set this up at boot:

```

# cat >> /etc/sysctl.d/99-bridge-nf-dont-pass.conf <<EOF

net.bridge.bridge-nf-call-ip6tables = 0

net.bridge.bridge-nf-call-iptables = 0

net.bridge.bridge-nf-call-arptables = 0

net.bridge.bridge-nf-filter-vlan-tagged = 0

bridge-nf-filter-pppoe-tagged = 0

bridge-nf-pass-vlan-input-dev = 0

EOF

# service procps start [RHEL command --- what's GENTOO equivalent for openrc?]

```

After disabling bridge netfilter routing, I am now able to ping from my container to my lan computers and my lan computers are now able to ping my lxc container, so my lxc container appears as just another computer on my lan. I did not have to touch my shorewall configuration files.

However, I am still not able to ping to the WAN.:

```

LXC# ping www.google.com

PING www.google.com (74.125.137.106) 56(84) bytes of data.

From social.acjlaw.net (192.168.1.40): icmp_seq=7 Destination Host Unreachable

HOST# dmesg

Shorewall:loc2net:REJECT:IN=br0 OUT=br0 MAC=XXX SRC=192.168.1.41 DST=74.125.137.106 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=35300 DF PROTO=ICMP TYPE=8 CODE=0 ID=220 SEQ=10

```

----------

## redwood

I couldn't ping to the WAN because in my 

network settings in /var/lib/lxc/{container}/config

I had 

```
lxc.network.ipv4.gateway = auto
```

which might've worked for DHCP but not for a static IP.

The solution was to specify the gateway explicitly:[/quote]

```

# networking

lxc.network.type  = veth

lxc.network.link  = br0 

lxc.network.flags = up

lxc.network.name  = eth0

lxc.network.ipv4  = 192.168.1.41/24 

lxc.network.ipv4.gateway = 192.168.1.1

#lxc.network.ipv4.gateway = auto

lxc.network.mtu = 1500

```

----------

## Meet Joe Black

Well, it's definitely better to try static network settings in the LXC before you would like to try any DHCP set.

P.S. And yes, you usually need to have default gateway set in LXC and also sometimes smth like "nameserver 8.8.8.8" inside your LXC's /etc/resolv.conf as shown on the wiki page https://wiki.gentoo.org/wiki/LXC#Adjusting_guest_config_of_the_container_after_using_template_script .

Network problems are also may be related to your router/iptables/nftables config. See /home/rt/scripts/nft.sh as an example config on the wiki:

https://wiki.gentoo.org/wiki/LXC#Host_configuration_for_VLANs_inside_the_bridge_wich_are_connected_to_container.27s_virtual_ethernet_pair_device

P.S.P.S. I haven't ever stumbled myself on such a problem with in-kernel bridge or smth like that. I have the following kernel bridge-related config:

# grep BRIDGE /usr/src/linux/.config

CONFIG_BRIDGE_NETFILTER=m

# CONFIG_NF_TABLES_BRIDGE is not set

# CONFIG_BRIDGE_NF_EBTABLES is not set

CONFIG_BRIDGE=m

CONFIG_BRIDGE_IGMP_SNOOPING=y

CONFIG_BRIDGE_VLAN_FILTERING=y

Your problem is most probably related to NF_TABLES_BRIDGE or BRIDGE_NF_EBTABLES option (or their in-kernel incorrect configs). I also have BRIDGE_NETFILTER=m set as module (not compiled into the kernel).

----------

## redwood

I haven't emerge'd nftables yet, though I have read up a little on it.  

It's supposed to offer a more concise syntax for the rules as well as being stateful 

and designed in such a way that new filters can be written in userspace instead of 

requiring new kernel modules. All well and good I suppose, but I've never

written firewall rules by hand.

I guess the nftables rules in legacy mode should be backward compatible with iptables 

so that I could switch from iptables to nftables but still let shorewall write my firewall rules?

----------

## Meet Joe Black

Well, I haven't ever used shorewall. I migrated to nftables myself from iptables rules wich I wrote by hand earlier. I prefer nftables syntax and other features over iptables.

P.S. It really doesn't matter wich one do you use of those. Your rules must be correct on both. That's what has to be your priority.   :Laughing: 

----------

