# Routing Problems...

## Utoxin

I've got a box with 2 network cards. Until yesterday, it was working perfectly. Then I had to reboot it a few times to fix some issues with the harddrives. Now, it has some routing problems. It is accessible from outside the local subnet at both IP addresses, but from inside the subnet, it can /not/ be reached.

As far as I can tell, nothing in the configuration has changed. Anyone have any idea how to fix this?

----------

## pjp

Can you go out from the card in question?

----------

## Utoxin

Both cards can reach the internet. Neither card can reach any IP (other than their own) on the local net.

----------

## pjp

Is the gateway still correct?

----------

## Utoxin

Yes. The gateway is our main T1 router. (What's really odd is that the cards can reach /that/ local IP, but not any of the others.)

----------

## pjp

/etc/hosts and /etc/resolv.conf haven't changed?

----------

## Utoxin

Nope. I've seen this happen once before, and it's /definately/ a routing issue. But last time I solved it by luck, and I don't know how to solve it again. Last time it happened, I also wasn't running Gentoo.

It seems to me that what happens is that the box gets confused about which interface it's trying to route stuff through for the local net. It knows how to route for the internet, because it has a default route pointing to the gateway. 

*hmmm*

I'm going to fiddle with it for a minute, and see if I can give it a default route for the local net as well.

----------

## Utoxin

No luck.

----------

## Naan Yaar

What does /sbin/route give you?

----------

## pjp

'man route' mentions route and route_cache under /proc/net.  I also have an rt_cache_stat file.  Maybe something there.

----------

## Utoxin

Okay, I've got a copy of the output of route up at http://utoxin.kydance.net/route.txt

That's on the box that's actually having the problem, so if you have trouble getting to it, let me know, and I'll find another way to get the text somewhere viewable.

----------

## Utoxin

 *kanuslupus wrote:*   

> 'man route' mentions route and route_cache under /proc/net.  I also have an rt_cache_stat file.  Maybe something there.

 

Couldn't see anything useful in there, though I didn't look real hard.

----------

## Naan Yaar

Oops, can you do below as "/sbin/route -n" to get the numeric output.

 *Utoxin wrote:*   

> Okay, I've got a copy of the output of route up at http://utoxin.kydance.net/route.txt
> 
> That's on the box that's actually having the problem, so if you have trouble getting to it, let me know, and I'll find another way to get the text somewhere viewable.

 

----------

## Utoxin

 *Naan Yaar wrote:*   

> Oops, can you do below as "/sbin/route -n" to get the numeric output.
> 
>  *Utoxin wrote:*   Okay, I've got a copy of the output of route up at http://utoxin.kydance.net/route.txt
> 
> That's on the box that's actually having the problem, so if you have trouble getting to it, let me know, and I'll find another way to get the text somewhere viewable. 

 

Done.

----------

## Naan Yaar

Shouldn't the Destination column for the second row be 216.190.203.0?  Also, are the interface assignments correct?  That is, hosts with IP's 216.190.203.x, where x is > 128, are connected to eth0 and the IPs with x < 128 connected to eth1?

EDIT: reading your routing again, the gateway is on eth1 and it has IP > 128.  This means that the first row should be eth1 and the second row should be eth0 (with the destination set to 216.190.203.0).

```

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

216.190.203.128 0.0.0.0         255.255.255.128 U     0      0        0 eth0

216.190.203.128 0.0.0.0         255.255.255.128 U     0      0        0 eth1

0.0.0.0         216.190.203.129 0.0.0.0         UG    1      0        0 eth1

```

----------

## Utoxin

Let me explain our network.

We have the block 216.190.203.128-255.

My server has two nics in it, on with IP 133 and one with IP 134. They should both grant access to the local subnet, and should both be able to access the internet.

In the past, this has worked, with the exact configuration I have now.

EDIT: Well, not quite that exact config. But if I use the exact config, it can't even be seen outside the subnet. The original config used eth0 for the default route to the gateway.

----------

## Naan Yaar

OK.  The second entry will never fire because it is a duplicate of the first in terms of genmask & address.

If you (temporarily) remove the first route and ifconfig down eth0, can you then ping internal hosts?  It should then be a plain vanilla box behind a gateway network setup...

 *Utoxin wrote:*   

> Let me explain our network.
> 
> We have the block 216.190.203.128-255.
> 
> My server has two nics in it, on with IP 133 and one with IP 134. They should both grant access to the local subnet, and should both be able to access the internet.
> ...

 

----------

## Utoxin

Mmkay. That fixed the issue locally, can you still see it? If so, does this mean that it's a Bad Idea (tm) to have two nics access the same network? We've got two other servers, one windows, and one Linux that are also doing this, and they don't seem to have problems.

----------

## Naan Yaar

I can still see the box fine.  Does the card on eth0 work?  You can probably do the single route experiment by swapping around eth0 for eth1 to check.

 *Utoxin wrote:*   

> Mmkay. That fixed the issue locally, can you still see it? If so, does this mean that it's a Bad Idea (tm) to have two nics access the same network? We've got two other servers, one windows, and one Linux that are also doing this, and they don't seem to have problems.

 

----------

## Utoxin

It does work. However, I just investigated the output of 'route' on our other linux box, and noticed something interesting. Take a look:

```

[root@helium /root]# route -n

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

216.190.203.128 216.190.203.131 255.255.255.128 UG    0      0        0 eth0

216.190.203.128 0.0.0.0         255.255.255.128 U     0      0        0 eth0

216.190.203.128 0.0.0.0         255.255.255.128 U     0      0        0 eth1

127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo

0.0.0.0         216.190.203.129 0.0.0.0         UG    0      0        0 eth0

```

Notice the first route line. It's IP addresses are 131 and 132. It appears that it is routing all local traffic from either of it's interfaces through eth0. Is there any way to get Gentoo to do that?

----------

## Naan Yaar

Looks like you have hit a nasty issue.  Googling gives me this:

http://www.uwsg.iu.edu/hypermail/linux/kernel/0204.2/1694.html

To add the gateway for the local network, you can use "route add" in /etc/conf.d/local.start, I imagine.

PS. The configuration that works on your other box seems to route everything through eth0 anyway (?), making the two ethernet card set up seemingly redundant.

----------

## Utoxin

Very interesting. So I'm not even supposed to do that. I suppose I'll pull the extra interface then, and just alias the other IP onto eth0. What a pain.

----------

