# Unusual Router

## CDLM

I've got a wierd question - keeping in mind im just getting into linux routers and such so if this just isnt possible let me know.

i have a box with say, three nics. two (eth1 and eth2)  are connected to cable modems. the other (eth0)  to the internal network. eth0 has two ips, 10.0.0.1 and 10.0.0.2.

some clients on the internal network have 10.0.0.1 as their router, some have 10.0.0.2 as their router.

now is it possible on the router box to have traffic on eth0 on IP 10.0.0.1 to be routed to eth1 and traffic on eth0 on IP 10.0.0.2 be routed to eth2?

and if i've confused the hell out of all you, let me know and ill draw up a diagram

- Dave -

----------

## Arasi

Fundamentally no you can not do this.

You have traffic coming in from a private subnet to either private ip but it is still all the same private subnet, you then want that traffic to route through, routing uses either known networks to route traffic or a routing protocol to learn networks and best path or they forward to a default gateway.

You can't have 2 default gateways and then have the router look at the source ip to determine the default gateway.  Because your routing tables are not isolated for those externel nics.

If you had 2 physical boxes...or say went with user mode linux...another topic...then you would have 2 seperate routing tables on the machines and therefore could route traffic that way.

Question:  Why are you looking to route like this? is there a practical application your trying to obtain that could be done another way?

Arasi

----------

## Dracnor

That shouldn't be a problem with IP tables.  You can just set up a rule that says any traffic coming from 10.0.0.1 be routed to eth1 and 10.0.0.2 to the other one.  You can just set up masquerading.  You'd want to make sure that the clients use the external DNS servers of the appropriate nic.

----------

## Arasi

 *Quote:*   

> That shouldn't be a problem with IP tables. You can just set up a rule that says any traffic coming from 10.0.0.1 be routed to eth1 and 10.0.0.2 to the other one.

 

But what about traffic not coming from 10.0.0.1 or 10.0.0.2?  Can you set an iptable rule that says if the packet was sent to 10.0.0.1 and is destined externally to go to such and such gateway?

See I agree with you if its traffic from just that machine as a router but I'm thinking about traffic from hosts going to that single internal card to either ip and being routed accordingly, thus the packets are not from 10.0.0.1 or 10.0.0.2  and then the question becomes is there a pattern in range he could use to set an iptables entry or will it be a nightmare to administer this way.

Arasi

----------

## Dracnor

Well you can set up different subnets to route to different ethernet cards.  So you could put one host on one subnet and another host on another subnet and when creating the subnet mask be able to span the subnets such that the hosts can communicate with each other.  I can't remember if you can split a subnet (IIRC you can).  But, if he's going to manually (statically) assign IP addresses he could easily create a rule each time.  There should be multiple ways to do this.

----------

## Arasi

 *Quote:*   

> I can't remember if you can split a subnet (IIRC you can). But, if he's going to manually (statically) assign IP addresses he could easily create a rule each time. There should be multiple ways to do this.

 

Yeah splitting the subnet is where I was pretty sure it would be fundamentally wrong....I understand with iirc but then again your also involving an application layer into the mix to support this, here he just has the good ol' layer 3 to work around and the basics of how that works looks either messy imo or will not work the way he's asked.

I suppose knowing better what he wishes this to do for him (the application of said configuration) would go a long way to suggesting the best way to configure it.

Arasi

----------

## CDLM

alright, fundamentally what I'm trying to do is have a router box for multiple machines at a LAN party. I have four to five cable modems and I want it so that the network is basically divided so that certain IPs use certain cable modems:

ie first 30 ips go thru modem 1

next 30 ips go thru modem 2

etc

and optimally i'd like to only have 1 internal nic (5internal + 5external = 10nics which is gonna be hard to pull off on one box)

now what would be the easiest (or most correct) way to accomplish this?

alternatively if there's a way to "combine" the internet connections (spread the load across all the external connections) that would work fine for me too.

- Dave -

----------

## Arasi

Not sure what kind of costing you get, but for that many cable lines I could get a fibre line in to simplify the whole process.

Admittedly fibre is a matter of availability and costs are dictated by said availability so it may not be feasible in your area.

On the other hand if you want to run 5 cable modems and use it to the extent you've described I'd recommend getting a 3Com NetBuilder Super Stack II used off ebay for 40.00 US and configure it up to segment and direct the gateways as you have said.  Then you could share a local ip subnet, and direct outbound traffic without having such a massive configuration to set up in software.

The application you are describing sounds more to me like a candidate for a hardware solution rather than a software one even though I'm sure with enough configuration you could manage something in software.

Arasi

----------

## CDLM

cost is not an issue, the local cable provider is sponsoring us by providing hookup/modems/service for the LAN

i've seen other lan groups do this with linux, ill ask around. if i find a solution ill post it but i'd still appreciate if someone could point me in the general direction of how to do this.

- Dave -

----------

## jhmartin

I believe some usage of 'policy routing' will allow what you are trying to do. Also, if your isp doesn't do egress filtering it should be possible to round-robin the outgoing packets to get what amounts to a really big outgoing pipe.

----------

## Arasi

 *Quote:*   

> cost is not an issue, the local cable provider is sponsoring us

 

Thats nice  :Smile: 

Well, ultimately it is doable but heres the thing, your going to have 5 inputs into one card...then 5 outputs...right?

is that as well one card out?

If so, then your going to actually really limit that bandwidth by forceing it all through one device.  The thing is that only one system can techincally talk to your internal nic at a time...if you now have 30 hosts sending data (regardless of what subnets and splitting you do it will ultimately cause lots of lag at that point in your network.

I'd still look at picking up a used netbuilder or layer 3 switch and do your routing from hardware. Anything where you can at least return the ratio of internal connections to external at 1:1

Good luck though, let us know how you make out.

Arasi

----------

## jhmartin

Although it is true that they'll all be going through one machine, but what is the capacity of that machine vs the up/down capacity of the combined cable modems?  If it is at all a decent-sized machine it should be able to handle  the traffic.

----------

## Arasi

 *Quote:*   

> but what is the capacity of that machine vs the up/down capacity of the combined cable modems?

 

Perhaps a better questions is, how much data is actually being transferred externally to take advantage of a pipe that wide...if its not using the combined bandwidth, but theres a lot of little traffic than even though the capacity may be higher there will still be latency due to having a single point of entry/exit from the lan.

Ultimately adding capacity is usually the band-aid fix to network traffic jams when traffic shaping and setting up seperate broadcast domains (subnets) is the real answer.

I had a company once go from a simple dial up connection, then to an isdn then to dsl and now they have fibre...they still had slow internet...Why...because the gateway is off a hub connected to all their users and those systems all wanted to talk at once...when they thought fixing it with a switch might help they soon realized most of that traffic was directed to go to the gateway.

Thats when we looked at setting up segments and additional gateways, while downgrading the fibre to 3 seperate dsl links.

They've never imagined their network could go that fast....but most people assume if you have the ferrari that you'll burn rubber...then you get stuck in rush hour traffic because theres not enough lanes to simultaneously pass through.  :Smile: 

Arasi

----------

## CDLM

looks like its going to be four cable modems like the last LAN - some windows app was used for the last one that round-robin-ized all four connections

the reason we need a lot of bandwidth is for a few reasons, all of which crop up at the start of the lan and in the wee hours of the morning/when no tourneys are going:

1. patch downloading - while I make a local ftp for most patches, there's always people wanting other mods/maps/etc

2. online gaming - for some reason people like to play online at a LAN... don't ask me why, i have no idea  :Very Happy: 

3. massive amounts of downloading of "other stuff" just because the internet is there - we try to stop warez/movie/porn downloading but there's always someone sucking bandwidth

all of which are a detriment to normal internet surfing/IM-ing

the rest of the network is all switched, intel and cisco switches, so its not that the network is badly arranged.

i'd like to use a linux solution, but the pro-windows staff (basically everyone else) is pushing for the windows solution for ease of administration

- Dave -

----------

## CDLM

side-note - anyone know of some windows network monitoring software (monitor traffic by interface?)

----------

## Mnemia

While I agree that you might run into bandwidth/latency issues due to having the single egress point, I strongly disagree that what you want to do is difficult or anything with Linux.

All you have to do to round-robin the connections is simply set up an iptables rule (in the PREROUTING table) that uses the "nth" packet match module to mark packets sequentially with 4 different FWMARKS. This method of selecting packets neatly avoids the problem of static vs. dynamic IPs etc. However, you might want to check to make sure you have similar latency across the connections before you do this if you want that to be consistent within connections. If so a slightly more complicated packet selection method might be order. 

Then you can use the iproute utility to set up some policy routing to send packets with each of the FWMARKS (1-4) to different cable modems. You can in fact have multiple different routing tables and default gateways in Linux; you just can't access them through the old "route" utility. Emerge iproute and you can use the "ip" command which allows access to the much more advanced routing capabilities as well as offering a nicer user interface for controlling regular routing.

----------

## oegat

Since we are talking NAT and not routing of "real" IP:s, I see no problems other than bandwidth with your solution. It would be perfectly doable with 2 outgoing nics as in your first example, I have friends doing exactly that in their apartment, each user there chooses gateway from which outgoing line they want.

4-5 modems might be heavy for a single pci machine to handle, but a linux solution can't be after all be doing worse that the windows solution you mentioned earlier. Can you afford to have 2 machines NAT:in 2-3 modems each? That would surely do.

----------

## Arasi

 *Quote:*   

> side-note - anyone know of some windows network monitoring software (monitor traffic by interface?)

 

Sure, but you don't need it when you can fire up comparable linux options.

My personal preference is using snort (snort.org) to watch the network, however to actively watch whats happening you'll have to look at a graphical front end, because snort likes to log in the backend or display to the terminal window...but it displays a lot and fast so you tend to miss things.

Best bet is go to freshmeat (freshmeat.net) and search for "network monitor" to find either a source cvs for something or a package name that you can check the portage tree for.

As to the other posts:

 *Quote:*   

>  I strongly disagree that what you want to do is difficult or anything with Linux. 

 

I suppose this comes down to perspective, based on someones knowledge setting this up with iptables can be daunting...although I tend to agree that whats been proposed seems very easy to configure...I wasn't aware how easily you could round robin outbound traffic...but then again I never had a need to.

Arasi

----------

## hummus

 *Quote:*   

> That shouldn't be a problem with IP tables. You can just set up a rule that says any traffic coming from 10.0.0.1 be routed to eth1 and 10.0.0.2 to the other one. You can just set up masquerading. You'd want to make sure that the clients use the external DNS servers of the appropriate nic.

 

i'm not sure if you'd want to do this if all the computers were onthe same switch. Not all cable modems implement STP(spanning tree protocol) and when you connect them to the same hub, or with crossover, etc you could get some nasty broadcast loops if they're assigned IP's onthe same subnet.

----------

## CDLM

alright, could someone point me at a good doc for actually understanding iptables?  :Very Happy:  Thanks.

- Dave -

----------

## Dracnor

I used the  tutorial on yolinux when I set up my gateway.  This should give you a starting point into setting up the complex situation you have  :Wink: 

----------

## Crg

 *CDLM wrote:*   

> 
> 
> all of which are a detriment to normal internet surfing/IM-ing
> 
> 

 

Why don't you use traffic shapping - that would make more efficient use of all your bandwidth.

With linux routing you could routing certain groups of ips over one modem, and others over another, along with proper QoS to ensure low latency for games - priority for websurfing, and ftp to be able to download lots.

Also it seems people aren't very aware of routing/advanced routing capabilities in Linux and have a limited experience with iptables so always think of iptables first, when really routing issues are often better solved with routing.Last edited by Crg on Wed Jan 14, 2004 9:28 pm; edited 1 time in total

----------

## Crg

 *Mnemia wrote:*   

> 
> 
> Then you can use the iproute utility to set up some policy routing to send packets with each of the FWMARKS (1-4) to different cable modems.
> 
> 

 

You could just get linux routing to load balance the traffic over multiple modems.  Either way I'm not sure that it would work without the other end being setup to handle it, unless ip routing send all the packets for a session out the same interface, (or iptables does).

Edit: You can set up routing to do load balancing across all interfaces keeping interface affinity.  Let me know if you're interested in doing this, if you are I'll look at creating and pasting up a script that balances and does QoS.

----------

## Crg

 *Arasi wrote:*   

> 
> 
> You can't have 2 default gateways and then have the router look at the source ip to determine the default gateway.
> 
> 

 

/sbin/iprule add from 10.0.0.0/25 to 0/0 table 101 pref 102

/sbin/iprule add from 10.0.0.128/25 to 0/0 table 102 pref 102

/sbin/iproute add table 101 via eth1

/sbin/iproute add table 102 via eth2

Note: With this setup there is a couple of ip address around middle you won't be able to use, which shouldn't be an issue with a LAN setup (won't need that many addresses and can easily be avoid by allocating ips from one end (.2, .3, .4...) to some clients and the other end (.254, .253, .252...) to others).

Note: You don't need 2 ip's on eth0 either all clients can have the same ip address as the gateway.

----------

