# DIY virtualized router with real hardware

## 1clue

Renamed from "Need hardware recommendations for DIY router" since that no longer defines the project.

Really, what I need is a router. If somebody knows of off-the-shelf hardware for under $300 then that would be fine too.

This is for my home office, and I need big router features and little router price.

I just bought a Cisco EA6500 (to put behind my other EA6500) but now, before I get it out of the box, I realize that's not going to work, I still don't get the features I want.

Here's what I need:

802.1q (VLANs)

VPN support

Gigabit speeds full duplex

IPV6

firewall that can do rules based on VLANs

A good DHCP+DNS that's better than what normally comes on SOHO equipment.

Low latency

It would be neat to have something like Gentoo on it rather than DD-WRT.  I would just stick DD-WRT on what I just bought, but it's not supported.

All the PC-based linux routers I've done before had a high latency for some reason.  Of course last time I did it was almost a decade ago.

Any ideas would be appreciated.Last edited by 1clue on Mon Oct 28, 2013 5:06 am; edited 2 times in total

----------

## frostschutz

I'd use a cheap TP-Link router with OpenWRT on it, but maybe that's just me...

----------

## 666threesixes666

i started collecting and documenting technologies revolving around your plan....

802.1q (VLANs)

VPN support 

Gigabit speeds full duplex

IPV6

firewall that can do rules based on VLANs

A good DHCP+DNS that's better than what normally comes on SOHO equipment.

Low latency 

vlans didnt get 2, vpn only got to hamachi/haguichi...  gigabit is hardware problem need 2 gigabit add ins, if motherboard doesnt support it, gigabit switch....  ipv6 supported out of kernel, firewall iptables & "webmin" are good sauce, based on vlans no idea though i imagine so...  dnsmasq wiki page handles dhcp + dns, defacto openwrt/ddwrt dhcp dns server...  low latency, just build your own box and make your serious linux machine act as router.

so i mean its over 50% of what your post is asking for though not 100%  the spirit of the open source community is take what others have done, and build upon it.

----------

## Atom2

I'd see two possible routes to go down - although none involves gentoo:

1.) go with pfsense and buy an ALIX board: The pfsense software is freely available, very comprehensive, easily configurable through a web interface and probably one of the best firewall and router distributions available. There's even a dedicated book available covering all the configuration; the latest version based upon the 2.1 release is about to be released in due course. BTW pfsense is based on BSD.

ALIX is one recommeneded hardware partner. You'll find more info about the ALIX hardware and other hardware options here and if I recall correctly you should be able to find something within your budget.

2.) Use the Asus RT-N66U and reflash its firmware with dd-wrt. The (wireless) router hardware is very good, serves the 5 GHz and 2.4 GHz band at the same time and provides upt to 450 MBit/s wireless speed. All ports are Gigabit LAN and dd-wrt runs very stable. Pricewise the Asus hardware is definitely well within your budget.

----------

## 1clue

My old "inside" router died, and I want to get something better.  I tried DD-WRT on that thing, and it was flaky.  I'll use DD-WRT again if I have to but so far none of the hardware I'm interested in is listed in their database.

My existing device (which I'm going to keep) has 802.11ac, which is almost a gigabit in wireless, dual band.  I have that part covered, I'm not experiencing any sort of latency there.  Problem is, this one doesn't have any VLAN support, so I'm kind of in trouble.  My old one had VLANs.

I've got a whole lot of home devices, and I have some hardware for my work that I want to isolate from all that.  The nature of my work, I need a clean network for my job.  I also need some isolation within the work environment for testing purposes.

For this piece, if I get anything with a radio on it I will probably turn the radio off.  I want a secure zone behind the normal router, and I want it to actually be somewhat secure, so wires only.

I'm looking at the ALIX stuff now.  This is looking really promising.  These boards, they say Linux so maybe Gentoo would be an option.  I don't know anything about pfSense yet, so I've got some reading to do.  Building my own switch out of boards is ideal, I just haven't been able to figure out where to get hardware.

Keep the ideas coming.

Thanks.

----------

## 666threesixes666

i was looking at some interesting articles about vlans and trunking.

http://www.techrepublic.com/article/use-vlan-trunking-and-gigabit-ethernet-for-smarter-multihoming/

http://wiki.openwrt.org/doc/networking/network.interfaces

http://www.linuxjournal.com/article/7268?page=0,0

gentoo can do ospf bgp & rip via the packages quagga & bird.  id really like to get some collaboration on working out corporate grade routing / networking on gentoo.  i knew it at some point on corporate grade equipment, but i wanted to migrate away from that way back when.

----------

## 1clue

Right now, I'm really curious about pfSense, but I don't think it will work on a SOHO router appliance.  I've browsed their forum a bit and don't see much.

I don't like the ALIX boards for this because I want more VLANs than the board has NICs.  I have 4 VLANs planned.  I also can't really find anything promising as hardware for my price point.

I'm currently fixated on what seems to be my best option:  DD-WRT is on the verge of releasing a build for an EA6500.  I have 2 of them, one of which is still in the box since I'm still trying to figure out if there's a better solution out there.

I'm hoping that DD-WRT has improved somewhat since I tried it last.  Or better yet, I can find a 4+nics (not including uplink) board that takes a real Linux distro or pfSense for under $300.

----------

## Atom2

 *1clue wrote:*   

> Right now, I'm really curious about pfSense, but I don't think it will work on a SOHO router appliance.  I've browsed their forum a bit and don't see much.

 

That's my understanding as well.

 *1clue wrote:*   

> I don't like the ALIX boards for this because I want more VLANs than the board has NICs.  I have 4 VLANs planned.  I also can't really find anything promising as hardware for my price point.
> 
> [...]
> 
>  Or better yet, I can find a 4+nics (not including uplink) board that takes a real Linux distro or pfSense for under $300.

 

Have a look at standard PC/server hardware that can be expanded with a dual link (or quad link) Ethernet card. An example might be this. Bearing in mind that this is from ebay.de you should be able to get something similar much cheaper your side of the pond (assuming you are in the US as your budget was quoted in $).

Add a Dual Link Intel PCIe Ethernet card like this one plus a cheaper PCI uplink card (in case you need 4 Gigabit links for VLANs plus 1 additional Ethernet link for the internet uplink) and you should be ready to go.

Another option might be a Quad Link Gigabit Ethernet card, although and unfortunately those are usually in another leauge price-wise so might not be within your budget.

Having said that, I am confident that the former solution (dual link PCIe plus single link PCI) should still work out within your budget.

Also have a read here. There is an idea about sharing the WAN uplink with one of the VLANs through a VLAN capable switch which then goes untagged to the ADSL modem.  I haven't tried that myself, but that might work as well. In that post, they also discuss a number of hardware alternatives with more than 2 LAN ports.

I can only vouch for pfSense - it is indeed a great and very stable solution.

----------

## 1clue

Yes, I'm in the USA.

The budget is a relatively soft one, but I don't see why I should have to pay much more for what I'm looking at.

The hardware on the EA6500 is adequate to what I need, I just need one that works the way I want it to.

I haven't really found a regular PC box suitable for this sort of thing.  All the barebones systems seem to have no more than 2 NICs on them, but I think if I went to mini-itx or something I might find better, there's really no need to be super compact I guess.

For some reason I'm getting irritable about this.  I want something like a switch or SOHO router that can run a real distro.  Gentoo or maybe a firewall/router-specific distro.  I've already spent 2 days on this, which means I should have bought a USD$600 pfSense pre-installed machine to begin with and spent the two days making money instead of unsuccessfully trying not to spend it.

Maybe I'm just tired.

----------

## Carnildo

 *1clue wrote:*   

> I haven't really found a regular PC box suitable for this sort of thing.  All the barebones systems seem to have no more than 2 NICs on them, but I think if I went to mini-itx or something I might find better, there's really no need to be super compact I guess.

 

Is there some reason why you can't take something with a full-sized mainboard, stick a half-dozen single-port Ethernet cards in it, and call it good?

----------

## 1clue

Other than the size of it, not really.  I don't have a whole lot of room, and I don't want to fill up that space with an AT-sized 6-port switch.  I'd rather put in another server or five.   :Smile: 

I'm sorry but I can't get the image of a giant antique out of my head.  I'd rather push my budget and get something that's better suited to the task.

While I haven't tried pfSense yet, I'm interested in seeing what it does.  Maybe I'll put it on a VM?  Or I guess I never looked for a demo app.

----------

## 1clue

Actually at this point it's looking pretty nice to get a couple of these:  http://www.newegg.com/Product/Product.aspx?Item=N82E16833114037

and then find a small board to plug them into.

Just so happens that DD-WRT got a beta of the EA6500 firmware going a couple days ago, so I wasted a couple days trying to make that work.  Very buggy and inconsistent, which is what I thought about it before with my wrt610n.

So I got fed up with that, and downloaded a pfSense image.  md5 didn't match up, and half the mirrors seem to be dead.  So I got fed up with that too.

I think I've blown a week looking for something.

Carnildo is sounding pretty darn smart right now.  I KNOW I could do this with Linux on a full board.

Maybe what I should do is buy my new workstation and stuff a bunch of nics in my old i7/12g.  Just a little bit overkill, but it's hardware I have, mostly.

----------

## AngelKnight

 *1clue wrote:*   

> Actually at this point it's looking pretty nice to get a couple of these:  http://www.newegg.com/Product/Product.aspx?Item=N82E16833114037
> 
> and then find a small board to plug them into.
> 
> Just so happens that DD-WRT got a beta of the EA6500 firmware going a couple days ago, so I wasted a couple days trying to make that work.  Very buggy and inconsistent, which is what I thought about it before with my wrt610n.
> ...

 

Did this lately on a home build, using a Soekris Net6501.  Having 4 1gbps ports onboard is nice.  I think it ticks every box on your list.  It ran up a bit higher than $300 since I insisted on having 2x spinning media rather than booting from USB/flash, along with getting 802.11 from the vendor.

Soekris and ALIX boards can run any Linux you want.  Even Gentoo or something else source-based if you don't mind the wait.

If you haven't done so already, you might try posting on the Kernel & Hardware forum.

----------

## 666threesixes666

i agree with the philosophy of reusing old hardware, you could even get a full on work station out of the router its self, not needing to turn on another system to make a quick post to gentoo forums.  i disagree with that philosophy as old equipment usually power hogs.  old equipment does 1 thing though, gives you experience and time to hunt down micro devices that will fit your specifications.  one of my friends on facebook uses a micro pc as a router, ill ask em what kind it is.  in the mean time, ill leave you with this....

http://en.wikipedia.org/wiki/Mini-ITX#History

----------

## 1clue

@AngelKnight, this is looking promising.

I'd seen ALIX before from earlier in the thread, but not yet Soekris.  I'll go fixate on that for awhile.

Ideally, I would have enough to be my router and nothing else on the system, so I would actually prefer a non-spinning disk.  But so far everything I look at is double or triple my budget, or more.  Breaking it by a hundred bucks or so isn't going to kill me.  I'm beginning to realize that my budget isn't very realistic.

I'm still interested in any small boards anybody knows about with lots of ethernet ports and an ability to run real Linux.  Especially handy would be a way to add a 4x ethernet in addition to whatever is on the board.

Thanks.

----------

## NeddySeagoon

1clue,

I'm using a HP Microserver with a 4 port intel NIC.  I got my Microserver in the days of £100 cashback in the UK, which made it a very good deal.

It runs gentoo-hardened and host about half a dozen KVMs.  I did max out the RAM to do that.  One of the KVMs is my media server, so its full of HDDs too.

One KVM is my router.

I don't know what costs are like today in USD

----------

## 1clue

 *NeddySeagoon wrote:*   

> 1clue,
> 
> I'm using a HP Microserver with a 4 port intel NIC.  I got my Microserver in the days of £100 cashback in the UK, which made it a very good deal.
> 
> It runs gentoo-hardened and host about half a dozen KVMs.  I did max out the RAM to do that.  One of the KVMs is my media server, so its full of HDDs too.
> ...

 

So, about that:

Don't you consider it a security risk to have a high-function computer as the Internet gateway?

About using a VM as a router:  What do you do, give the VM the extra NICs?  All the NICs?

Can you/do you use VLANs and trunking?

If that sort of thing works, then I'm all for it.  My old i7 is perfect, I'm already using it for a bunch of virtualization and all I'll need to do is add some NICs.

----------

## 1clue

I could get a couple of those four-port nics and that would be the end of my purchases for that.   :Smile: ))

----------

## zeronullity

I've been down this road before myself.. although your performance will increase significantly, so will your power consumption. Even a OLD pentium 4 system with custom compiled kernel/OS will smoke a retail based router any day. 1 to 2ms response. times vs. 0.1 ms or less response times. You might have other options other then DD-WRT.. tomato,OpenWRT, etc. If you can custom compile for your retail router thats the best way to go.. only install packages that you need like with OpenWRT. There is no need to spend a lot of money unless your upgrading to AC wireless. $20 or old scrap pentium 4 configured for power savings.. should do the trick.. just expect to pay in the long run in energy cost.. get a few decent ethernet cards like e1000, and a wireless N/AC card.. and do a minimum low-latency kernel setup. I've yet to find a retail router that out performs my P4 box thats sitting in the garage due to energy cost.. and it's simply loaded with e1000 cards with custom Gentoo/kernel.

FYI - I found that performance wise DD-WRT is extremely buggy especially on routers like Belkin.. compared to other firmware like Tomato by shibby.Last edited by zeronullity on Thu Oct 17, 2013 8:44 pm; edited 1 time in total

----------

## NeddySeagoon

1clue,

The bare metal install is gentoo-hardened with only enough functionality to support KVMs.

Applications are run in KVMs.

I did intend to use PCI pass through to donate the 4 port NIC to the router KVM but there is a hardware bug in its PCIe bridge, which means its not safe to do that.  Its fixed in later versions of the card.  Instead the interfaces have to be donated to bridges for the router to pick up.

The bare hardware has no internet access at all.  The internet is provided by a PPPoE link which has its endpoint in the router VM.

I don't see it as a security risk but its quite a single point of failure risk.  My setup may help you get started but be warned that the link is out of date.

There is a mail KVM that runs qmail, http-replicator and a portage mirror, so the bare hardware can't even update unless both the mail server and router KVMs are both running. On boot it complains about not being able to find any timeservers.

The router is shorewall, set up so that everything that is not explicity permitted is denied.

There is an Apache webserver in the mix too but its on a Raspberry Pi running Gentoo rather than a KVM, since Intel shell code won't run on an ARM system.

The Pi is powered from a USB port on the HP Microserver.

I don't use VLANs yet but the support is there if I need to.  I've recently got fed up enough with telephone cold callers to get a SIP gateway.

To use that properly I expect I will need to use VLANs for QoS.  The idea is to set up POTS as outgoing only.

----------

## 1clue

@zeronullity,

I already have a Linksys EA6500AC, which is wireless AC.  I messed with DD-WRT on that and decided it has nothing for me.  DD-WRT is not only unstable but inconsistent even with itself.  Messing with the VLANs page, I experimented with plugging and unplugging cables that the label for the WAN was correct, but the "status light" under it was way over on the other side.  When messing with VLAN settings, the check under the WAN matched to the WAN and the other ports were reversed in order.

I tried for the command line at that point, figured out that there's not enough room in there to do anything, and there's no real organization to what's going on in there anyway.

At least with a mainstream Linux kernel everything makes sense and there are tons of examples.

I started on this quest because my existing VLAN-capable router broke, and my linksys was not up to the task.  After having looked at all the open source firmwares out there I'm convinced that that hardware is not adequate.  I would like a few more ports anyway.

----------

## 1clue

@NeddySeagoon,

My intent at this point is to put the new box (whatever it is) on the outside, dedicate a port with its own VLAN over to the Linksys, so I can use that, stock, for home use.  Everything else will go to my office equipment.  That will have no wireless access and will not be accessible from "home" except for maybe a home-only DMZ.

I have another box on the horizon, I'll have 3 virtualization hosts at that point and one of them is a laptop, they'll need to have VLAN support.  Maybe I could figure out some sort of failover VM on the other side, if the main box breaks I just move the cable...

I will DEFINITELY give your link a read when I get the chance.  I thought about a virtualized router but didn't think it would be practical or really useful.  Maybe I need to bone up on my networking and virtualization knowledge.  But the idea of it being a VM means I'm not limited in any real way regarding hardware which is great.

I've been staring at raspberry pi for awhile now, haven't had time to mess with it.  Aside of its namesake being one of my favorite dessert foods, I think they're really cool gadgets as well.  I can't think how many times I've wanted a controller like that.

Things are getting way too complicated right now, I need to slow down and take baby steps.  First to read your link and then maybe to set up a hardened Gentoo VM, then maybe add a 4-port NIC to it.  Never done hardened anything before, but maybe it's time.

As usual, you've expanded my horizon a lot, changed a bunch of my parameters and I need to re-think this with more information.

Thanks.

----------

## zeronullity

I'd give Tomato by shibby a try first maybe with a thumbdrive for extra space.. not sure if it has all the support

 your looking for based on flash size/ram.. etc. As far as I know EA6500 is currently supported.  Yeh, hard to beat custom hardware /w custom kernel in both security

& performance, providing you set it up correctly that is. Old scrap system with Asus AC wireless card and Intel gigabit cards should cost less then that retail AC router.

Retail routers are horrible for anything cpu intensive.. i.e.  file transfers from usb, etc. can easily halt all performance on the network, what little there is already.

----------

## 1clue

I think I've given up on SOHO hardware for what I'm thinking about.  What I have is one of the higher performance appliances out there, and it's great as far as home use but everything I try on it comes up lacking for work.  I'm all about leaving that in place to handle the wireless and digital cockroaches.

I don't even want wireless access on the business side of things.  Everything has ethernet, and will continue to have it.  If you have wireless access to a network, it is not secure.  The end.  I'm freaking right now because my work equipment is wirelessly accessible.

Frankly I was going down the SOHO firmware route because I didn't want to pay for a real switch/router, and I didn't think a virtualized one would work.

NeddySeagon's idea really sounds perfect for me.  I'll try to figure out a failover method to reduce the chance of a disaster, but I can't imagine it getting much better than this.

I'm still interested in small appliance hardware that can handle a full distro.  Here are some specs about what I'm interested in:

4g minimum memory

16g flash

2 usb ports

one or more PCIe slots

As another point, I'm also interested in a screaming fast barebones, with an i7 and capable of 32g ram.  Needs at least 3 monitor ports, DVI or HDMI or display port.  No analog.  Sata3 and lots of usb3.  This is obviously not the router, I want a sick workstation that might be luggable.

Thanks.

----------

## zeronullity

I don't know about luggable but I like my x8dal-i.. although you can get a workstation with even more 

then 24 cores now if needed.. it does support up to 96GB of ram. It's all about how much $$$$ you wan

t to dish out. I was only suggesting installing wireless card to replace the EA6500  wireless AC fuction.. 

however if not needed, don't install, and I agree.. wired is much more secure especially when combined 

with encryption. Although no system/network is 100% secure.. It's all about time/methods of cracking it..

which can include physical access. I believe at some time in US history we used electromagnetic surveillance 

to spy on Russian telecommunications that way they couldn't say we were taping their physical lines.. so even 

phyiscal lines are not secure. =)  Any ways you have  plenty of x86 based systems to choose from, that should

work for your needs. Just do your research on specs/performance & you should be fine. - GL

----------

## 1clue

@NeddySeagoon,

I'm currently running a setup very much like what you're describing in that doc.  I have a /boot and then the rest of it is LVM2.  The host_root volume is in the same pool as the guests.

So I guess I need to ask, what can I do in a VM to set this up and have it actually work for a real install?  I have VMs running on this thing that would be difficult to move.

I'm using Ubuntu 12.04 as the host OS right now because I don't want to have to keep tinkering with it and it's the lowest maintenance thing I've used recently.  But that's not what I had on there at first, I've reinstalled the host OS several times now without touching any of the VMs.  Back up the VM non-disk files, restore them and you're done.

I know the hardware drivers will change, but can I emulate the actual CPU and build a real kernel at least?  That would be the real clincher for me, it takes a long time to get through that.  And I've never even attempted a hardened anything.  I would think I could add support for the real hardware AND the virtio drivers, and maybe get it working that way.  Am I correct?

Here's the deal:  I'm using an ssd as the boot drive, I have /boot as a regular partition and everything else is LVM2.

I have the SSD, a plain hard disk and a raid1 array.

```

root@chronos:~# vgs

  VG     #PV #LV #SN Attr   VSize   VFree  

  hddvg1   1   5   0 wz--n- 698.63g 435.15g

  rd1vg1   1   1   0 wz--n-   2.73t   2.44t

  ssdvg1   1   5   0 wz--n- 223.07g 159.07g

root@chronos:~# lvs

  LV                VG     Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert

  host_spinner      hddvg1 -wi-ao-- 153.13g                                           

  host_swap         hddvg1 -wi-ao--  22.35g                                           

  popeye            hddvg1 -wi-a---  60.00g                                           

  y_root            hddvg1 -wi-ao--  20.00g                                           

  z_root            hddvg1 -wi-ao--   8.00g                                           

  freenas_files     rd1vg1 -wi-a--- 300.00g                                           

  host_root         ssdvg1 -wi-ao--  30.00g                                           

  ...

and about 10 more.                                     

root@chronos:~# 

```

I was originally just going to do a test run on your instructions so I know what to expect.  Then I started wondering, why not just use it if it works?

Edit:

I guess I'm thinking something along these lines:

Install Gentoo using a throw-away /boot on lvm and a "keeper" /.

Build the kernel for the real hardware, which I guess I can do using lspci and all that from the real box.

Probably have to support virtio devices too?

Boot it and test it for actual functionality.

Merge the throwaway /boot into the real /boot, update grub to support dual boot and reboot.

I don't see why this wouldn't work, but I'd like to have some sort of reinforcement that I'm not half baked.

----------

## NeddySeagoon

1clue,

It should be fine.  KVM performace is best if you tell the KVM to use the same CPU as the real hardware.

I did it the other way round to you, I did the real bare metal hardware install. When that worked, I tarred it as a stage4 to make a master KVM, which isn't used, it just gets cloned before I customise the cline for the application at hand.

That reminds me.  My master KVM is about 18 months out of date.

Going the other way should work too but your KVM starter kernel may need three sets of drivers.

1. Virtio Drivers (Only used in the KVMs)

2. Drivers for the emulated hardware in the KVM (may not be used ever - but see below)

3. Drivers for your real bare metal hardware (so you can move the KVM to the bare metal).

What will you use for a boot loader in the KVMs ?

If you use grub and a kernel with the virtio support, you will need to pass grub a device.map file as the virtio partitions will have no BIOS equivilents.

If you install grub using the KVM hardware drivers, you will have  /dev/sda and friends in the KVM and grub will install normally

----------

## 1clue

NeddySeagoon,

The box has a real /boot partition.  It's using grub2 I think right now.

I guess I'm not sure what makes most sense for me.  I'm guessing that a few VMs will be based on Gentoo, and then the bare metal box.  So maybe virtio for speed, and then real hardware drivers in case I use one on the bare metal.  Does that sound reasonable?

For VMs, I typically install a minimal VM of a distro and then clone it for whatever purpose.  Actually the one I started is my "base" for new Gentoo, if it all works out right.  What you just said about kernels reinforces this, I think once I get it right I won't need to have hardly any down time, I can compile the kernel on a base vm and try it as I need. 

I made the partition map pretty much as the wiki page suggests, and added a volume for /boot as well.  I was going to set up the boot volume for use when it's a VM, and then copy stuff over when I want to boot the bare metal from it.

----------

## 1clue

I just ordered one of these:  http://www.newegg.com/Product/Product.aspx?Item=33-114-037

I'm not sure why they're so cheap, but it says they work on Linux.

If it works well I'll probably order another one.

----------

## 1clue

OK I have another question.  Maybe I'm just nervous about this being a VM that will become the core OS.

When I map these logical volumes to the VM which will become the Gentoo host, I figure I'll define them as a virtio filesystem in the VM, but then I have a question.

Is it best to leave the device paths exactly as they are?  Or make them appear to be /dev/vg/host_whatever?

In VMs I typically use a minimal partition strategy, just a / and maybe a swap, and maybe tmpfs here and there.  It makes it easier to move things.

But with Gentoo I don't really like that idea, not really sure why that makes a difference though.  Except that with this installation I intend to have a copy of it as the real host.

Is it very difficult to make a single-partition VM and then split it out when I know it's working?  Never really had luck with that when actually running the machine, but here I could use another OS (maybe the installer) to deal with it.

----------

## NeddySeagoon

1clue,

That startech 4 post NIC looks to only have a single lane PCIe bus, so you get at most, 1.5Gbit/sec over the PCIe interface.

That means it won't support all four LAN ports going flat out.

My Intel 4 port NIC needs a PCIeX4 slot.

Not being able to drive your network ports flat out may not be a problem.  It wouldn't be for me as I have mt internet connection in one, at 80/20 Mbit, 54 Mbit wireless on another.

A raspberry pi on a 3rd. and my protected wired network, that is all 1G everywhere on the fourth.

The virtio drivers are in two parts.  The host part, which you need in the bare metal kernel and the VM part, which you need in the VMs.

You must have the right real hardware drivers in the bare metal kernel since there is nothing to provide the other half of the virtio drivers.

I can post kernel configs if you want.

Splitting up a single partition install when you move it is trivial.  Tar it up.  Make the new filesystem tree.  untar it in its new home.  Fix /etc/fstab, grub.conf hostname and the ssh host key.

Install grub to the new MBR and it may boot first time.  You may need a new kernel if the hardware has changed - it will have when you move from a VM to real hardware but you can make a kernel that works in both places.

----------

## 1clue

What a drag.  I knew there was something wrong with that card.  I will need to have some high speed ports, in fact when I get my next box up I might need to bond a couple ethernet channels between the existing i7 and the other.  Or maybe I can figure a way to segment the network by high traffic connections between VMs.

Well, I guess I'll use that one for the low-speed stuff when I get another card.  Live and learn.  I can use one of them for the cable anyway, give it priority over everything and let it go at that.

I will be interested in your kernel configs.

There's got to be a better way of looking at this.  Regular hardware I'm fine with, VMs I'm fine with.  This VM-as-real-router-with-real-devices thing makes my head hurt.  Once I get it up I'll undoubtedly think it was easy, but it's one step to the left of black magic right now.

----------

## 1clue

It seems it's pcix*4.  I'm in luck.  They're just on sale for $100 off or so.  The reviews say they're getting full speed out of them.  Maybe it's not a bad deal after all.

http://www.startech.com/Networking-IO/Adapter-Cards/4-Port-PCI-Express-Gigabit-Ethernet-NIC-Network-Adapter-Card~ST1000SPEX4

----------

## NeddySeagoon

1clue,

I couidn't find any data, so I was judging by the PCIe connector size.

For real hardware in a VM the bare metal hardware kernel needs PCI pass through support.

Like I said, my hardware has a bug so I never made it work.

VM kernel

Bare metal kernel

----------

## 1clue

You're using multilib on your host kernel.  Is that necessary?  Can't I just do that in the guests?

I'm going to want multilib on guests, I just thought it was purely the OS, in which case the guest would be fine.

This is taking a lot longer than I'd anticipated.  I'm having to do real work, and trying to wedge an install of Gentoo in between the spaces.

I've finally gotten around to kernel config.

By the time I actually install it my @#%@ network card will probably be here.   :Sad: 

----------

## NeddySeagoon

1clue,

I was not aware I was using multilib at all. On that box or anywhere else.

I'll need to look into that.

----------

## 1clue

Your kernel config has it as one of the first settings.

I will definitely want one or more of the guests on there to be multilib, but I don't think I want the host to be multilib unless it actually needs to be.

OK thanks.  I'll get through it sooner or later, hopefully before the @#$% nic gets here.

----------

## 1clue

I lost the race.  I just got my network card, and I haven't had time to get Gentoo working.   :Sad: 

----------

## 1clue

The good news is the existing xubuntu install recognized it with no problems, so there's no driver issue.

I still need to do real work before I can play with this more.   :Sad: 

----------

## 1clue

I sort of abandoned the link to your system and went the normal handbook route.

Is there a reason to use hardened+selinux on the host image?  More importantly, if I have hardened can I then add selinux later?

I've never done either of these before, so I don't know.

Thanks.

----------

## 1clue

emerge-webrsync doesn't seem to work, so I downloaded a portage and extracted it myself.  I hope that's all emerge-webrsync was supposed to do.

----------

## Mad Merlin

It looks like you already got past the hardware selection process, but I've been very pleased with the EdgeRouter Lite. It meets all of your requirements for $100, ships with software based on Vyatta, but can run Gentoo too. I have several of them running in a variety of roles, including as a router for my home network. Highly recommended.

----------

## 1clue

@Mad Merlin,

It looks interesting for normal circumstances, but given NeddySeagoon's virtual router I think that's the way for me.  This network I'm making is a few pieces of hardware and a bunch of VMs anyway.

Well, let me amend that:  I have a boatload of home devices with whatever random junk we buy or my wife downloads from the net.  That's going to be partitioned off by this device, over to the router appliance with wireless.  Aside of that, it's going to be a few virtualization servers and a workstation.  Trying to figure out a way to do this with only one 4-port NIC on the host, but so far I think I'm going to need another one.

It's odd, lspci shows EXACTLY the same hardware type for the 4-way NIC as what's on my motherboard.  Realtek RTL8111/8168.  Go figure.

It's been a _LONG_ time since I've started a Gentoo install.  I've never done a hardened anything on any distro, and so this has a lot of newness to me, and I detect a lot of learning on the virtualized router end too.

I was just thinking, this box I'm installing on now is the first Gentoo install I ever made.  I bought the hardware all new and installed Gentoo as the first OS.  So this is homecoming in a way, even though it's been wiped and reinstalled with a BUNCH of different distros since.

I got lost in the infinity of kernel options, then reloaded Neddy's bare metal kernel and fixed it for my hardware, ignoring all else.

This thing is dragging on way too long, I have to get it done.

Question to anyone:  When I'm making a KVM host, do I need filesystem support for all the guest operating systems as well?  I'm using LVM mapping to manage filesystems directly on the host.

I have:

/boot is a real partition.

ssd volume group

hdd volume group (just a disk, no RAID or anything, for things like downloads.

raid 1 volume group

A removable slot-load sata drive for backups.

----------

## 1clue

Renamed from "Need hardware recommendations for DIY router" since that no longer defines the project.

I realized the title no longer matches the project.  Rather than start a new thread I renamed the existing one.

Thanks to everyone so far for your input.  It's been educational and interesting.

----------

## NeddySeagoon

1clue,

I'm only using hardened, not SELinux

I pass logical volumes directly to the KVMs  To the bare metal, they are just block devices. The bare metal has no knowledge of the filesystems im use by the KVMs.

Thats true if you use file systems in a file too.

My 4 port NIC is used for

1. The internet my public /29

2. The protected wired network 192.168.100.0/24

3. The protected wireless network 192.168.54.0/24

4. Internet facing services 192.168.10.0/24

----------

## 1clue

NeddySeagoon,

I compiled in some extra filesystem support then.  :Smile: 

I have the kernel for the "bare metal" compiled.  I ran out of free time and it's sitting there, I'll get started again later.

Thanks.

----------

