# net.ppp0 (adsl pppoe) network problem with xen domU started

## bodek_

Hi,

this has been bothering me for longer time and I have found no satisfying solution yet. if you had any *i mean really - any* ideas they woud be definetely appreciated.

in my setup I have one central server, which is main mail server, web, firewall etc and virtual machines are running under xen (mainly testing stuff, nothing too important)

i have gentoo 2006.1 with all latest updates (emerge finished yesterday) and xen 3.02-r4 running on it

kernel: 2.6.16.28-xen-r1

recently after emerges old type of adsl pppoe seemed to be broken in conf.d/net, so i switched according to net.example to the new "proper way" of setting up pppoe connection (i can provide conf.d/net if requested but i checked it with different sources some dozen times, so one can almost assume it is ok)

the virtual network is bound to interface dummy0

internet---(eth3/ppp0)-dom0-(dummy0)--------(all the xen machines)

                                   |

                                  eth2(internal net)

probably most common setup to be found - in /etc/xen/xend-config.sxp only one change

(network-script network-bridge 'netdev=dummy0')

(vif-script vif-bridge)

now what happens - bootup works fine, all network cards start (except dummy0), xend/xendomains starts OK (bringing dummy0 up). i then try to start centos domU with the same kernel as above only domU version, and as soon as it tries to access the network through ppp0 interface, the ppp0 dies in very strange way, meaning:

interface ppp0 and eth3 are still up

there are still default routes set for ppp0

tcpdump on ppp0 shows no packets coming-in/out, from internal network, shows only some some LCP exchange with pppoe gateway

now i can still access internal networks from domU and vice-versa and also before problem happens i can access networks (except ppp0) and it does not cause any problems

what else - lets say i dont do anything with domU, but just start it and then restart named in dom0 (can be any other daemon) which binds separately to all network cards, then the same problem happens again - net.ppp0 gets somehow "disjointed" from the rest of kernel network stack and rest of OS does not see it (forgets about it?)

problem has nothing to do with iptables - tried to restart it / stop it (leaving just forwarding on) - all the same - no packets coming to net.ppp0 (although it is there and UP and routes are set)

what i made so far:

- reinstall xen completely including kernel redownload/recompile

- emerge -e x2 (system, then world)

- reinstall and recompile xen + kernel again

in the process after that last thing i noticed something very interesting:

- if I enable in the kernel  "bridged ip/arp packets filtering" to have "physdev" match support then the whole problem above dissapears!

- however my life would be too good - with following options enabled packets coming out from dummy0 network don't seem to be hitting the POSTROUTING chain in iptables in NAT table and thus i have packets with private ip address leaving ppp0 (and we know what internet routers do with such packets, dont we?). WHY DOES IT HAPPEN?

- it would also be great to know why before even without bridged filtering all worked perfect and now it does not anymore (i disabled bridged filtering intentionally - i didnt need any l2 filtering and it was nightmare to configure)

- the other main question remains - why the hell the packets from dummy0 dont go through POSTROUTING chain? Where could I NAT them? With ebtables? This is illogical to have two separate functions in kernel to do the same thing.

Any ideas would be greatly appreciated! I can provide *any* config files/logfiles you request. (btw. dmesg,messages shows nothing, pppd is set to debug - also nothing)

Let me know if you saw anything even remotely similarrrr....

Regards,

Bogdan

----------

## bodek_

as i didnt get any reply i had to work a bit harder on it...

1. net.ppp0 aka new way of calling pppoe for adsl - this is broken with xen, for details read post above (i might log defect for it, we will see...) - if you use 'the old way' aka config_ethX=( "adsl" ) and run pppoe setup everything will work fine

2. for the POSTROUTING chain problem - this is just the way iptables is engineered when used in conjuction with ebtables - most people describe the problem as impossible to be solved as packets get to netfilter framework much earlier than on dummy0 interface and the POSTROUTING chain does not get hit twice with the same connection (packet already been there), sbd advised to use NOTRACK target/option with raw table - that did not work in my environment the way expected, so i found something else:

http://en.opensuse.org/Xen3_yet_another_Virtual_Network_Concept ---> i took network-virtual script from there

which instead of creating 'network card mess' in dom0 just creates xenbrX and treats it as just another network card, meaning in the end you have only one used nic in dom0 and no problems with iptables/ebtables/netfilter. solution is a bit 'unclean' (e.g look at mac of xenbrX) but at the end of the day i have no need for nothing 'more fancy'

hope it helps somebody.

Rgds,

Bogdan

----------

