# 2 of 3 PCs on same switch ... [SOLVED]

## dufeu

edit: Sun Nov 26 12:55:19 EST 2017

TL:DR version: Verizon FiOS {now Frontier} provided router assigned multiple IP addresses to both PCs. Could not clear multiple assigned IP addresses from router's admin. Could not clear addresses using a hard reboot. Resolved by resetting router to factory defaults. I'm writing up a more complete post now and will link to it from here when it is complete.

end edit

I have 3 PCs and a printer on the same switch. Everything was working fine up until the last upgrade{from more than 4 weeks ago}. I didn't immediately notice the issue because I've only updated the kernel on my main {working}, not @world. 

I now have two of the three PC unable to ping the router or the internet. The 3rd PC is still behaving perfectly fine.

The two problem PCs cannot ping the router. The router cannot ping the 2 problem PCs.

All the devices on the switch can see each other. The working PC and the printer are freely available to the rest of my local network.

Each of the same switch PCs can freely access the other PCs NFS and SAMBA shares. 

In order to simplify diagnostics, I replaced the original configured bridging I had on one of the PCs and set it up with 

```
config_eth0="dhcp"
```

This is the result:

```
firelizard ~ # /etc/init.d/net.eth0 restart

 * Bringing down interface eth0

 *   Stopping dhcpcd on eth0 ...

sending signal TERM to pid 21811

waiting for pid 21811 to exit                                                                                                                    [ ok ]

RTNETLINK answers: No such file or directory

Error talking to the kernel

 * Bringing up interface eth0

 *   dhcp ...

 *     Running dhcpcd ...

eth0: waiting for carrier

eth0: carrier acquired

DUID 00:01:00:01:0f:41:8e:37:00:e0:4d:0e:ee:e1

eth0: IAID 5c:54:eb:49

eth0: adding address fe80::a0aa:87da:a1ee:65d1

eth0: soliciting an IPv6 router

eth0: rebinding lease of 192.168.1.201

eth0: probing address 192.168.1.201/24

eth0: leased 192.168.1.201 for 86400 seconds

eth0: adding route to 192.168.1.0/24

eth0: adding default route via 192.168.1.1

forked to background, child pid 22367                                                                                                            [ ok ]

 *     received address 192.168.1.201/24                                                                                                         [ ok ]

firelizard ~ # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

From 192.168.1.201 icmp_seq=1 Destination Host Unreachable

From 192.168.1.201 icmp_seq=2 Destination Host Unreachable

From 192.168.1.201 icmp_seq=3 Destination Host Unreachable

^C

--- 192.168.1.1 ping statistics ---

4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3035ms

pipe 4
```

I set this PC up with a reserved IPv4 address at the router. Note that after receiving the reserved IP address, assigning the route and forking to the background, , the resulting ping no longer sees the router that dhcpd just got the IP address from.

One of the non-communicating PCs is a virtual clone of the working PC. All three PC are configured the same consistent with the differences in their hardware.

I've never seen anything like this before. Anyone have any thoughts? All the posts I've seen regarding being unable to ping the router through a switch seem to be in regards to Cisco managed switches. That's not the case here. Some additional info:

```
firelizard ~ # ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    link/ether 40:8d:5c:54:eb:49 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.201/24 brd 192.168.1.255 scope global eth0

       valid_lft forever preferred_lft forever

    inet6 fe80::a0aa:87da:a1ee:65d1/64 scope link 

       valid_lft forever preferred_lft forever

firelizard ~ # ip route

default via 192.168.1.1 dev eth0 src 192.168.1.201 metric 3 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.201 metric 3 

firelizard ~ # ping 192.168.1.200

PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data.

64 bytes from 192.168.1.200: icmp_seq=1 ttl=64 time=0.130 ms

64 bytes from 192.168.1.200: icmp_seq=2 ttl=64 time=0.199 ms

64 bytes from 192.168.1.200: icmp_seq=3 ttl=64 time=0.222 ms

^C64 bytes from 192.168.1.200: icmp_seq=4 ttl=64 time=0.274 ms

64 bytes from 192.168.1.200: icmp_seq=5 ttl=64 time=0.345 ms

c64 bytes from 192.168.1.200: icmp_seq=6 ttl=64 time=0.140 ms

^C

--- 192.168.1.200 ping statistics ---

6 packets transmitted, 6 received, 0% packet loss, time 5048ms

rtt min/avg/max/mdev = 0.130/0.218/0.345/0.075 ms
```

```
pyrogyro linux # ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000

    link/ether fc:aa:14:5a:13:c9 brd ff:ff:ff:ff:ff:ff

    inet6 fe80::feaa:14ff:fe5a:13c9/64 scope link 

       valid_lft forever preferred_lft forever

4: br0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000

    link/ether fc:aa:14:5a:13:c9 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.200/16 brd 192.168.255.255 scope global br0

       valid_lft forever preferred_lft forever

    inet6 fe80::feaa:14ff:fe5a:13c9/64 scope link 

       valid_lft forever preferred_lft forever

pyrogyro linux # ip route

default via 192.168.1.1 dev br0 metric 4 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.0.0/16 dev br0 proto kernel scope link src 192.168.1.200 

pyrogyro linux # ping 192.168.1.201

PING 192.168.1.201 (192.168.1.201) 56(84) bytes of data.

64 bytes from 192.168.1.201: icmp_seq=1 ttl=64 time=0.296 ms

64 bytes from 192.168.1.201: icmp_seq=2 ttl=64 time=0.288 ms

^C

--- 192.168.1.201 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1000ms

rtt min/avg/max/mdev = 0.288/0.292/0.296/0.004 ms

pyrogyro linux # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.371 ms

64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.417 ms

64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=4.61 ms

^C

--- 192.168.1.1 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2001ms

rtt min/avg/max/mdev = 0.371/1.801/4.615/1.989 ms
```

----------

## SP2340

 *dufeu wrote:*   

> 
> 
> ```
> firelizard ~ # ip addr
> 
> ...

 

Here is the relevant information.  You will see you are using the same network but have 2 different masks.

192.168.1.201 and 192.168.1.200 are on the same network.  The problem is the mask you cannot have 2 systems setting on the same network with different masks.  Since 192.168.1.200 is working I believe you need to change 192.168.1.201's mask to /16 and it should work too.

----------

## dufeu

 *SP2340 wrote:*   

> Here is the relevant information.  You will see you are using the same network but have 2 different masks.
> 
> 192.168.1.201 and 192.168.1.200 are on the same network.  The problem is the mask you cannot have 2 systems setting on the same network with different masks.  Since 192.168.1.200 is working I believe you need to change 192.168.1.201's mask to /16 and it should work too.

 

All these years and I did not know that the masks in a subnet had to match.

Your suggestion resulted in a big improvement all around. Very helpful! And thank you!!

I don't recall reading anything about matching masks in any of the documentation I first learned from. Just that the portions of a subnet that would be visible to a particular machine were controlled by it's own mask setting. But hey, whatever works.

It smoothed out most of my samba shares issues with some of the Windows boxen on our network in both directions. Now to finish taking all the Window boxen out of their 'homegroups' that a well intentioned network user set on our network ... 

Not everything is fixed yet. What is fixed is that all the PCs on the network can now see each other. Oddly enough, neither of the previously borked PCs can yet ping the router even though the router is visible to one of them.

I'm still trying different things but I'm certainly open to more suggestions! This is what things look like now:

```
pyrogyro etc # ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000

    link/ether fc:aa:14:5a:13:c9 brd ff:ff:ff:ff:ff:ff

    inet6 fe80::feaa:14ff:fe5a:13c9/64 scope link 

       valid_lft forever preferred_lft forever

5: br0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000

    link/ether fc:aa:14:5a:13:c9 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.200/24 brd 192.168.1.255 scope global br0

       valid_lft forever preferred_lft forever

    inet6 fe80::feaa:14ff:fe5a:13c9/64 scope link 

       valid_lft forever preferred_lft forever

pyrogyro etc # ip route

default via 192.168.1.1 dev br0 metric 5 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.200
```

```
firelizard ~ # ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    link/ether 40:8d:5c:54:eb:49 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.201/24 brd 192.168.1.255 scope global eth0

       valid_lft forever preferred_lft forever

    inet6 fe80::a0aa:87da:a1ee:65d1/64 scope link 

       valid_lft forever preferred_lft forever

firelizard ~ # ip route

default via 192.168.1.1 dev eth0 src 192.168.1.201 metric 3 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.201 metric 3 

firelizard ~ # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

^C

--- 192.168.1.1 ping statistics ---

2 packets transmitted, 0 received, 100% packet loss, time 1023ms

firelizard ~ # ip neigh

192.168.1.196 dev eth0 lladdr 00:0a:cd:1f:4a:dd STALE

192.168.1.156 dev eth0 lladdr e8:40:f2:b9:82:8d STALE

192.168.1.200 dev eth0 lladdr fc:aa:14:5a:13:c9 REACHABLE

192.168.1.1 dev eth0  INCOMPLETE

192.168.1.203 dev eth0 lladdr 00:25:90:d1:82:a7 STALE

192.168.1.159 dev eth0 lladdr 40:8d:5c:05:46:16 STALE

fe80::d0b6:afc2:cabe:57d0 dev eth0 lladdr 40:8d:5c:05:46:16 STALE

firelizard ~ # ping 192.168.1.159

PING 192.168.1.159 (192.168.1.159) 56(84) bytes of data.

64 bytes from 192.168.1.159: icmp_seq=1 ttl=128 time=0.909 ms

64 bytes from 192.168.1.159: icmp_seq=2 ttl=128 time=0.745 ms

^C

--- 192.168.1.159 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1001ms

rtt min/avg/max/mdev = 0.745/0.827/0.909/0.082 ms

firelizard ~ # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

From 192.168.1.201 icmp_seq=1 Destination Host Unreachable

From 192.168.1.201 icmp_seq=2 Destination Host Unreachable

^C

--- 192.168.1.1 ping statistics ---

3 packets transmitted, 0 received, +2 errors, 100% packet loss, time 2013ms

pipe 2

```

```
blaze /etc # ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: enp9s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    link/ether 00:25:90:d1:82:a6 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.202/24 brd 192.168.1.255 scope global enp9s0

       valid_lft forever preferred_lft forever

    inet6 fe80::225:90ff:fed1:82a6/64 scope link 

       valid_lft forever preferred_lft forever

4: enp10s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    link/ether 00:25:90:d1:82:a7 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.203/24 brd 192.168.1.255 scope global enp10s0

       valid_lft forever preferred_lft forever

    inet6 fe80::225:90ff:fed1:82a7/64 scope link 

       valid_lft forever preferred_lft forever

blaze /etc # ip route

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev enp10s0 proto kernel scope link src 192.168.1.203 

192.168.1.0/24 dev enp9s0 proto kernel scope link src 192.168.1.202 

blaze /etc # ip neigh

192.168.1.156 dev enp9s0 lladdr e8:40:f2:b9:82:8d STALE

192.168.1.156 dev enp10s0 lladdr e8:40:f2:b9:82:8d STALE

192.168.1.1 dev enp10s0 lladdr d4:a9:28:13:de:ca STALE

192.168.1.1 dev enp9s0 lladdr d4:a9:28:13:de:ca STALE

192.168.1.201 dev enp10s0 lladdr 40:8d:5c:54:eb:49 STALE

192.168.1.159 dev enp10s0 lladdr 40:8d:5c:05:46:16 STALE

192.168.1.159 dev enp9s0 lladdr 40:8d:5c:05:46:16 STALE

192.168.1.200 dev enp9s0 lladdr fc:aa:14:5a:13:c9 STALE

192.168.1.200 dev enp10s0 lladdr fc:aa:14:5a:13:c9 REACHABLE

192.168.1.196 dev enp10s0 lladdr 00:0a:cd:1f:4a:dd STALE

blaze /etc # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.439 ms

64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.414 ms

64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.418 ms

^C

--- 192.168.1.1 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2034ms

rtt min/avg/max/mdev = 0.414/0.423/0.439/0.026 ms

blaze /etc # ping gentoo.org

ping: unknown host gentoo.org

blaze /etc # cat /etc/resolv.conf 

# Generated by net-scripts for interface enp9s0

nameserver 8.8.4.4

nameserver 8.8.8.8

nameserver 208.67.222.222

nameserver 192.168.1.1
```

----------

## ct85711

Well, going by what you have posted, I am assuming pyrogyro is the one that can ping the default gateway.  The interesting part, I am specifically seeing, is that the 3 machines have different routing tables.

pyrogyro 

```
pyrogyro etc # ip route 

default via 192.168.1.1 dev br0 metric 5

127.0.0.0/8 via 127.0.0.1 dev lo

192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.200
```

firelizard

```
firelizard ~ # ip route 

default via 192.168.1.1 dev eth0 src 192.168.1.201 metric 3

127.0.0.0/8 via 127.0.0.1 dev lo

192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.201 metric 3 
```

blaze

```
blaze /etc # ip route 

127.0.0.0/8 via 127.0.0.1 dev lo

192.168.1.0/24 dev enp10s0 proto kernel scope link src 192.168.1.203

192.168.1.0/24 dev enp9s0 proto kernel scope link src 192.168.1.202 
```

Now ignoring the differences in ip addresses and interface names; blaze has the most significant differences in the routing table.  First being that there is no default route specified.  Note, you can only specify ONE default route, so you have to pick one of the network cards as the default.  If a computer doesn't have a default route, then it doesn't know where to send packets outside of the network; as the default route is there to tell the computer that if the destination isn't known send it to someone who does (i.e. the router, to forward onto another router and eventually the destination).  As far as firelizard, the main difference I am seeing, is that on the default route contains src 192.168.1.201 where pyrogyro doesn't.  The metrics part, you can safely ignore as that only affects duplicate paths.  For fallback routes (which is what default routes are), there is only one route to go...

----------

## LIsLinuxIsSogood

 *Quote:*   

> 5: br0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 

 

Oddly enough, it also looks like this machine (the working one) is running a different setup that I had to look up just now, called "promiscuous" mode...It refers to the reading of all network packets regardless of whether destined to that card or another address.  Perhaps it is nothing?   

I also concur that the default routes on all the machines should be looking more or less similar.

----------

## eccerr0r

Promiscuous mode is normal and required on soft bridges.

As for the question on hand I still don't get how the interfaces got configured the way they did - they look very weird.  Perhaps best to rip up all the network configuration and start over - configuring all the same way, unless some details on how they were configured would be helpful.

BTW, yes, if you have your netmasks wrong, it's still a toss up whether your network...works.  You'll get lucky if all of them just so happen to have IP addresses on the same subnet anyway despite having incorrect masks.  It's just when getting out of the subnet when things start breaking, and yes, there is no reason to have incorrect netmasks... don't do it!

----------

## LIsLinuxIsSogood

Reset the default gateway on each of them to the router...that could really help a lot.  I noticed at least one of the three machines doesn't have one set.  Then what it does is probably forward all those packets to some other PC, or randomly assigned or something and then the other PC has to reply back to the non-working one to say, "hey there I'm no gateway router" so that won't be good.  If only there were some automatic way to configure network settings, oh wait NetworkManager.

----------

## SP2340

[quote="dufeu"] *SP2340 wrote:*   

> 
> 
> All these years and I did not know that the masks in a subnet had to match.

 

You learn something everyday.

 *Quote:*   

> Your suggestion resulted in a big improvement all around. Very helpful! And thank you!!

 

You are welcome.

 *Quote:*   

> Not everything is fixed yet. What is fixed is that all the PCs on the network can now see each other. Oddly enough, neither of the previously borked PCs can yet ping the router even though the router is visible to one of them.
> 
> I'm still trying different things but I'm certainly open to more suggestions! This is what things look like now:
> 
> ```
> ...

 

Looking at your original post you had pyrogyro as a '/16' and said it was working.  The question now is what does your router thing the mask is of the connected network?  /16 or /24?  You need all devices to be on the same mask.

----------

## LIsLinuxIsSogood

 *Quote:*   

> You need all devices to be on the same mask.

 

This is not actually true, and it doesn't make any sense.  However if you said on the same subnet then you would be accurate in saying so.

The way you can ensure that all PC's are on the same subnet is to provide a similar subnet mask, and make sure that there addresses (logical or IP) are within the same subnet.

If you have never looked at it, there are subnet calculators everywhere ont he web.  While you shouldn't need it, it may help to know how a subnet mask gets applied.

For pinging between machines, the answer is as simple as:

1. IP addresses on the same subnet (masks will help, but they don't need to be the same necessarily... but it is worth a check in case the mask puts you into a different subnet, like 255.0.0.0 will not work for 192.168.1.0 which is a class C address space, where as depending on the number of hosts Class A has the most, then B then C, you can use higher subnets for the larger classes, but it will limit your options in terms of communicating between them.  I don't know if this is what is happening for you.  But more than likely it is a question of one of the next two items)

2. Routes for communicating between each other...this is and can be especially tricky, but the answer is always a basic functioning piece, whether looking at MAC addresses and Address Resolution Protocol (ARP, which again you souldn't have to do), or just working with IP using the route command.  Personally I think that what you are in need of is to check the routing tables.

3. Default routes (for accessing internet)

----------

## LIsLinuxIsSogood

Why does pyrogyro and blaze have two interfaces, are they separate network interface cards, so you are actually attempting to have how many total addresses be able to "ping" one another...3 or 5 addresses?

EDIT: I take that back, looks like 4 NIC's, since on pyrogyro the interfaces share same ethernet address

Also, I can't tell what it is about blaze that isn't working -- other than a stale route table, the ping to router is working.  What else are you trying to do other than that with it?

Since you've gone ahead and changed the subnets already, for pyrogyro, which was already working before you started making changes, I just want to point out the differences between the previous (old) and current settings...

 *Quote:*   

> pyrogyro linux # ip route
> 
> default via 192.168.1.1 dev br0 metric 4
> 
> 127.0.0.0/8 via 127.0.0.1 dev lo
> ...

 

Under this setup, the use of a default route for dev br0 is sending anything outside the subnet to your router (192.168.1.1)

The reason it was working fine, despite the last line looking wrong, was that is just an additional route for the larger subnet that directs all traffic going to this larger subnet, meaning it was pretty much harmless unless your router happened to be located on yet another larger subnet, which isn't the case.

So that's why this worked to begin with, pinging the router from pyrogyro...

 *Quote:*   

> pyrogyro linux # ping 192.168.1.1
> 
> PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
> 
> 64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.371 ms
> ...

 

Based on your more recent post, it looks like you changed the addressing to place them all onto the same subnet from an addressing standpoint.  That should make the routing a bit easier, but it still doesn't solve the problem.  Please if possible be very clear and state which machines cannot ping which other machines. It helps to have it laid out very basic when looking to diagnose it from not right in front of the machine.

Also, I myself have rarely needed to do this but I think you need to flush your routing table:

 *Quote:*   

> blaze /etc # ip neigh
> 
> 192.168.1.156 dev enp9s0 lladdr e8:40:f2:b9:82:8d STALE
> 
> 192.168.1.156 dev enp10s0 lladdr e8:40:f2:b9:82:8d STALE
> ...

 

Same for here:

 *Quote:*   

> firelizard ~ # ip neigh
> 
> 192.168.1.196 dev eth0 lladdr 00:0a:cd:1f:4a:dd STALE
> 
> 192.168.1.156 dev eth0 lladdr e8:40:f2:b9:82:8d STALE
> ...

 

Specifically, the INCOMPLETE to 192.168.1.1 is the reason for notbeing able to ping the router.  Therefore, have you used ARP before.  Check it out,

It is actually not that difficult to use once you get used to it.  It works on the MAC address layer, which is the link layer of or physical addressing layer, which in the model for networking is one step below the logical addressing that of IP (and two steps below TCP, and UDP and all those other P's)  

If you've flushed your route tables, re-linked to the router, and still can't ping, then the next thing would be to selectively (or not) remove and add physical addresses to the arp table.

Take a look here for some help:  https://en.wikipedia.org/wiki/Address_Resolution_Protocol

----------

## LIsLinuxIsSogood

 *Quote:*   

> Oddly enough, neither of the previously borked PCs can yet ping the router even though the router is visible to one of them. 

 

Not sure exactly what it is that being visible refers to if it isn't to ping it, then what does this mean?  Do you mean address solicitation from router?

By the way, I too like you have a way of wanting to control the addresses in the ancient methods of the most primitive tools, but one of the better uses for things like either NetworkManager or else another net util like dhcp (looks like you must have a client installed) is that some of those will also take care of this step of having to clear old settings.

Other software is really bad at it, so I guess since you are sort of asking for a particular answer (or more like not just any answer) I would go about it this way...

This is my machine running from a live linux environment (sysrecsueCD)

 *Quote:*   

> root@sysresccd /root % ip route
> 
> default via 192.168.1.1 dev enp3s0  proto static  metric 100 
> 
> 192.168.1.0/24 dev enp3s0  proto kernel  scope link  src 192.168.1.8  metric 100 
> ...

 

Then, to get it working again, I just used the NM applet in the taskbar to disconnect and reconnect, proving that some of the utilities show great value in being able to handle things like this automatically...

 *Quote:*   

> root@sysresccd /root % ip route
> 
> default via 192.168.1.1 dev enp3s0  proto static  metric 100 
> 
> 192.168.1.0/24 dev enp3s0  proto kernel  scope link  src 192.168.1.8  metric 100 
> ...

 

EDIT: I should add, the command you want to run is,ip route flush table main 

```
#ip route flush table main 
```

----------

## ct85711

First off, generally you should not ever have to mess with the arp table, as it is automatically updated on it's own.  The way IPv4 works is that the computer will send a arp request out (a broadcast) onto the network, asking who has that IP address.  The computer has that IP address will respond back with it's mac address, which is added to the first computer's arp table.  Then it sends the packet of data out.  As the age grows (since the last time it communicated to that mac address), the system will first mark it as STALE, meaning it is old.  After a while, the system will automatically clean out old entires from the arp table (it also waits for a minimum amount of entires too, to minimize wasting cpu time).

Beyond that, the arp table is generally volatile, meaning when you restart your computer it is wiped out and nothing is saved.  Having a static arp table entry can cause you issues if/when the network changes (as the system won't know of the change nor try finding out if it has changed on those ip addresses).  The routing table on the other hand is saved.  In generally, you should have at least 3 entries in it.  First you will have one for the loopback, 127.0.0.1, next you will have the general network one for the subnet that computer is on.  Last is the default gateway, in that anything that isn't on your network (which is based off your subnet mask using a bit-wise multiplication of the subnet mask and the computer's ip address), is sent there.

In this case where your subnet masks were incorrectly setup, has caused your arp cache to be corrupted (as the computer thinks it knows how to get there, but has the wrong information.  So as LIsLinux suggested and verify everything have the correct subnet mask first; then flush the arp tables (or restart the devices).  Otherwise I highly recommend you DO NOT add any static arp entries yourself, let the system manage the arp tables on it's own.

----------

## eccerr0r

Bridges have been notoriously annoying to set up, do the other two machines work if the machine with the bridge completely disconnected from the network?

And agreed, I don't see a reason for needing static ARP in this network configuration, it should work just fine (I think my network is similar...  I have a machine with a bridge for VMs and a whole mess of other machines connected to switches, hubs and another bridge in pfSense...)

----------

## LIsLinuxIsSogood

 *Quote:*   

> Otherwise I highly recommend you DO NOT add any static arp entries yourself, let the system manage the arp tables on it's own.

 

Or just restart the computer, which wipes out the arp table again, ha ha

Anyway, I only said to try it AFTER doing everything else mentioned, so I really wasn't trying to tell the OP to go messing with that first.  Much the opposite, and the example I provided goes to show that the bad routing table can be corrected with a simple re-linking through the use of a common tool like Network Manager.

The OP has not confirmed what kind of software he is running on each of the machines, whether it is something like nm, or dhclient or dhcpcd whatever...this is a problem because regardless of the "difficultyness" (yes that is a word I think for this situation) the bridge and however many PC's are designed to function there are going to have to work independent of one another.  What I mean by that is it is possible for the setups to be different on each one and the entire network still work.  That is how networking and interconnected network devices work.  They each have a ton of stuff going on behind them, but when the time comes to broadcast a message, whether that message is a TCP handshake for address resolution, the three way handshake, or the subsequent delivery of many more packets filled with the payload.  

To LP (last poster) I'm not sure how big your apartment is, or what kind of setup you have, but if you are suggesting that anyone with a question about bridging connections could turn to you in this case, then be my guest.  However, I don't like to take that approach...the because it works for me it should work for you.  Or else clearly our OP wouldn't be in this situation.  Which is why I tried to provide a bit of background.  And in reality, while there is littly reason to imagine that the problem would need any arp entries for the resolution, there is still a problem of not knowing the precise networking utilities at use here on each PC.  

My suggested solution therefore is, please include for each machine, something about the way the device interfaces are set up, beyond the IP address. Like, dmesg output for each  with grep * where (where * is the named interface of each of those interfaces).  I know this seems/sounds like a ridiculous request, but that could be useful to see.

----------

## eccerr0r

You know, it's kind of funny I asked about the exact software configuration of all the machines on the OP's network, it is quite important as it would explain why the routing tables look so different to each other.  If they were all configured by hand, that would explain that.

I don't know how much bridging experience people have, but a badly configured bridge can be an explanation for ARPs getting eaten.  As they are in promisc mode, they are processing every packet that comes by and if they happen to respond to a packet they aren't supposed to respond to, it will cause weird things to happen like what we see here.  It's worth an experiment to remove the bridge temporarily and reintroduce it to the network once others are fixed.  We know bridging SHOULD work, and my example proves that point, but there's too many variables here.  Taking one out will simplify things and help with debug.  I'm not sure what circumstance would generate state "INCOMPLETE" but "FAILED" means no responders to an ARP and is the usual response to a missing machine.

Are there any firewall/iptables rules that could have masked some of the ARP responses too?

Also another thing is if possible reboot the switch, the switch does save state though by now it should have cleared...

----------

## SP2340

 *LIsLinuxIsSogood wrote:*   

>  *Quote:*   You need all devices to be on the same mask. 
> 
> This is not actually true, and it doesn't make any sense.  However if you said on the same subnet then you would be accurate in saying so.

 

So to be on the same network they need to have the same mask.

Since I'm looking at the mask as being the problem I'm directing the OP to look at the mask thus my statement.

You are welcome to disagree all you want but if the mask is not the same they are not on the same network.

----------

## LIsLinuxIsSogood

SP2340, this is not a matter of disagreement, so let's just wait for the OP to return with the progress on all three machines.

----------

## dufeu

 *LIsLinuxIsSogood wrote:*   

> SP2340, this is not a matter of disagreement, so let's just wait for the OP to return with the progress on all three machines.

 

I haven't been able to be responsive these last two weeks for RL issues. 

I'm back and I'm still having the same problems in that 2 of my PCs still can't 'see' beyond my local switch. It's really weird.

Before we go any further, I'll be re-collecting information on all three systems so that I can post it here. If anyone has suggestions for commands to run they would like to see the output of, please let me know.

Also, I've added a Mellanox 10Gb card to 2 of the PCs along with swapping my local 8-port 1Gb switch with a Quanta L4M 48-port (2 SPF+) switch.

Don't be distracted by the  new hardware. My problem with the two PCs hasn't changed at all. I wanted to confirm the new equipment works and is properly recognized by the kernel. I needed to confirm their state before the return period ran out.

PC1: works as expected:

```
dns_domain_lo="lamasondufeu"

config_eth0=null

config_br0="192.168.1.200/24"

bridge_br0="eth0"

rc_net_br0_need="net.eth0"

rc_net_lo_provide="!net"

rc_net_eth0_provide="!net"

dns_servers_br0="8.8.4.4 8.8.8.8 208.67.222.222"

routes_br0="default via 192.168.1.1"
```

This configuration supported when I used to run 3 VMs.

Further information:

```
pyrogyro conf.d # ip link

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT group default qlen 1000

    link/ether fc:aa:14:5a:13:c9 brd ff:ff:ff:ff:ff:ff

4: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/ether 00:02:c9:54:bc:48 brd ff:ff:ff:ff:ff:ff

5: br0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

    link/ether fc:aa:14:5a:13:c9 brd ff:ff:ff:ff:ff:ff

pyrogyro conf.d # ip neigh

192.168.1.1 dev br0 lladdr d4:a9:28:13:de:ca REACHABLE

192.168.1.202 dev br0 lladdr 00:25:90:d1:82:a6 REACHABLE

192.168.1.15 dev br0 lladdr 60:f1:89:74:3a:4d STALE

192.168.1.18 dev br0 lladdr e8:40:f2:b9:82:8d STALE

192.168.1.13 dev br0 lladdr a0:48:1c:91:41:66 STALE

192.168.1.101 dev br0 lladdr 74:f6:12:c1:c6:ec STALE

192.168.1.6 dev br0  FAILED

192.168.1.64 dev br0  FAILED

192.168.1.9 dev br0  FAILED

192.168.1.2 dev br0  FAILED

192.168.1.26 dev br0 lladdr 00:0a:cd:1f:4a:f8 STALE

192.168.1.201 dev br0 lladdr 40:8d:5c:54:eb:49 STALE

192.168.1.102 dev br0  FAILED

192.168.1.100 dev br0 lladdr 70:7e:43:66:98:bd STALE

192.168.1.5 dev br0  FAILED

192.168.1.3 dev br0  FAILED

pyrogyro conf.d # ip route

default via 192.168.1.1 dev br0 metric 5 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.200
```

PC2:Not working as expected:

```
dns_domain_lo="lamasondufeu"

config_eth0="192.168.1.201/24"

routes_eth0="default via 192.168.1.1"

dns_servers_eth0="192.168.1.1 8.8.4.4"
```

Further information:

```
firelizard ~ # ip link

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000

    link/ether 40:8d:5c:54:eb:49 brd ff:ff:ff:ff:ff:ff

firelizard ~ # ip neigh

192.168.1.99 dev eth0 lladdr 30:05:5c:2d:20:35 REACHABLE

192.168.1.200 dev eth0 lladdr fc:aa:14:5a:13:c9 REACHABLE

192.168.1.1 dev eth0  INCOMPLETE

192.168.1.98 dev eth0  FAILED

192.168.1.202 dev eth0 lladdr 00:25:90:d1:82:a6 STALE

firelizard ~ # ip route

default via 192.168.1.1 dev eth0 metric 3 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.201
```

PC3:Not working as expected:

```
dns_domain_lo="lamasondufeu"

config_enp9s0="dhcp"

config_enp10s0=null
```

When open-rc starts net, dhcp successfully pulls the assigned IP address from the router and then advertises to see if anyone else has that IP address already. The router (192.168.1.1) has reserved 192.168.1.202 for PC3.

PC3 further information:

```
blaze ~ # ip link

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: enp9s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000

    link/ether 00:25:90:d1:82:a6 brd ff:ff:ff:ff:ff:ff

4: enp10s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000

    link/ether 00:25:90:d1:82:a7 brd ff:ff:ff:ff:ff:ff

5: enp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/ether 00:02:c9:54:94:c0 brd ff:ff:ff:ff:ff:ff

blaze ~ # ip route

default via 192.168.1.1 dev enp9s0 src 192.168.1.202 metric 3 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev enp9s0 proto kernel scope link src 192.168.1.202 metric 3 

blaze ~ # ip neigh

192.168.1.18 dev enp9s0 lladdr e8:40:f2:b9:82:8d STALE

192.168.1.6 dev enp9s0  FAILED

192.168.1.201 dev enp9s0 lladdr 40:8d:5c:54:eb:49 STALE

192.168.1.99 dev enp9s0 lladdr 30:05:5c:2d:20:35 STALE

192.168.1.1 dev enp9s0 lladdr d4:a9:28:13:de:ca STALE

192.168.1.200 dev enp9s0 lladdr fc:aa:14:5a:13:c9 REACHABLE

192.168.1.98 dev enp9s0  FAILED
```

PC2 ping results:

```
firelizard ~ # ping 192.168.1.200

PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data.

64 bytes from 192.168.1.200: icmp_seq=1 ttl=64 time=0.102 ms

64 bytes from 192.168.1.200: icmp_seq=2 ttl=64 time=0.171 ms

^C

--- 192.168.1.200 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1007ms

rtt min/avg/max/mdev = 0.102/0.136/0.171/0.036 ms

firelizard ~ # ping 192.168.1.202

PING 192.168.1.202 (192.168.1.202) 56(84) bytes of data.

64 bytes from 192.168.1.202: icmp_seq=1 ttl=64 time=0.222 ms

64 bytes from 192.168.1.202: icmp_seq=2 ttl=64 time=0.315 ms

^C

--- 192.168.1.202 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1000ms

rtt min/avg/max/mdev = 0.222/0.268/0.315/0.049 ms

firelizard ~ # ping 192.168.1.99

PING 192.168.1.99 (192.168.1.99) 56(84) bytes of data.

64 bytes from 192.168.1.99: icmp_seq=1 ttl=255 time=2.93 ms

64 bytes from 192.168.1.99: icmp_seq=2 ttl=255 time=0.546 ms

^C

--- 192.168.1.99 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1001ms

rtt min/avg/max/mdev = 0.546/1.741/2.936/1.195 ms

firelizard ~ # ping 192.168.1.6

PING 192.168.1.6 (192.168.1.6) 56(84) bytes of data.

^C

--- 192.168.1.6 ping statistics ---

2 packets transmitted, 0 received, 100% packet loss, time 1000ms

firelizard ~ # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

^C

--- 192.168.1.1 ping statistics ---

3 packets transmitted, 0 received, 100% packet loss, time 2026ms

firelizard ~ # ip neigh

192.168.1.6 dev eth0  FAILED

192.168.1.99 dev eth0 lladdr 30:05:5c:2d:20:35 REACHABLE

192.168.1.200 dev eth0 lladdr fc:aa:14:5a:13:c9 REACHABLE

192.168.1.1 dev eth0  FAILED

192.168.1.98 dev eth0  FAILED

192.168.1.202 dev eth0 lladdr 00:25:90:d1:82:a6 STALE
```

PC3 is similar.

PC1 ping results:

```
pyrogyro conf.d # ping 192.168.1.202

PING 192.168.1.202 (192.168.1.202) 56(84) bytes of data.

64 bytes from 192.168.1.202: icmp_seq=1 ttl=64 time=0.179 ms

64 bytes from 192.168.1.202: icmp_seq=2 ttl=64 time=0.171 ms

^C

--- 192.168.1.202 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1000ms

rtt min/avg/max/mdev = 0.171/0.175/0.179/0.004 ms

pyrogyro conf.d # ping 192.168.1.201

PING 192.168.1.201 (192.168.1.201) 56(84) bytes of data.

64 bytes from 192.168.1.201: icmp_seq=1 ttl=64 time=0.265 ms

64 bytes from 192.168.1.201: icmp_seq=2 ttl=64 time=0.261 ms

^C

--- 192.168.1.201 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 999ms

rtt min/avg/max/mdev = 0.261/0.263/0.265/0.002 ms

pyrogyro conf.d # ping 192.168.1.99

PING 192.168.1.99 (192.168.1.99) 56(84) bytes of data.

64 bytes from 192.168.1.99: icmp_seq=1 ttl=255 time=0.437 ms

64 bytes from 192.168.1.99: icmp_seq=2 ttl=255 time=0.415 ms

^C

--- 192.168.1.99 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 999ms

rtt min/avg/max/mdev = 0.415/0.426/0.437/0.011 ms

pyrogyro conf.d # ping 192.168.1.100

PING 192.168.1.100 (192.168.1.100) 56(84) bytes of data.

64 bytes from 192.168.1.100: icmp_seq=1 ttl=255 time=3.74 ms

64 bytes from 192.168.1.100: icmp_seq=2 ttl=255 time=7.37 ms

^C

--- 192.168.1.100 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1000ms

rtt min/avg/max/mdev = 3.744/5.557/7.370/1.813 ms

pyrogyro conf.d # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=12.5 ms

64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=1.30 ms

^C

--- 192.168.1.1 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 999ms

rtt min/avg/max/mdev = 1.302/6.948/12.594/5.646 ms

pyrogyro conf.d # neigh

-su: neigh: command not found

pyrogyro conf.d # ip neigh

192.168.1.1 dev br0 lladdr d4:a9:28:13:de:ca REACHABLE

192.168.1.202 dev br0 lladdr 00:25:90:d1:82:a6 REACHABLE

192.168.1.15 dev br0 lladdr 60:f1:89:74:3a:4d STALE

192.168.1.18 dev br0 lladdr e8:40:f2:b9:82:8d STALE

192.168.1.13 dev br0 lladdr a0:48:1c:91:41:66 STALE

192.168.1.101 dev br0 lladdr 74:f6:12:c1:c6:ec STALE

192.168.1.99 dev br0 lladdr 30:05:5c:2d:20:35 REACHABLE

192.168.1.6 dev br0  FAILED

192.168.1.64 dev br0  INCOMPLETE

192.168.1.9 dev br0  FAILED

192.168.1.2 dev br0  FAILED

192.168.1.26 dev br0 lladdr 00:0a:cd:1f:4a:f8 STALE

192.168.1.21 dev br0 lladdr fc:aa:14:b3:b3:f0 STALE

192.168.1.201 dev br0 lladdr 40:8d:5c:54:eb:49 STALE

192.168.1.102 dev br0  FAILED

192.168.1.100 dev br0 lladdr 70:7e:43:66:98:bd REACHABLE

192.168.1.98 dev br0  FAILED

192.168.1.5 dev br0  FAILED

192.168.1.3 dev br0  FAILED
```

Devices 98, 200, 201 and 202 are local to my switch. All other devices are on other switches in my network. PC2 and PC3 cannot ping anything outside my local switch.

As a reminder, both PC2 and PC3 have undergone an "emerge @world" while PC1 has not. PC1 did not get it's emerge @world run because I haven't had time to resolve all the blockers. At the present time, if I want to update any packages on PC2 or PC3, I'm currently 'fetching' with PC1.

Oddly enough, local PCs lanwide CAN see the samba shares on PC3. I only have samba shares on PC1 and PC3.

edit to include some additional info:

```
firelizard ~ # /etc/init.d/net.eth0 restart

 * Bringing down interface eth0

RTNETLINK answers: No such file or directory

Error talking to the kernel

 * Bringing up interface eth0

 *   192.168.1.201/24 ...                                                                                                                        [ ok ]

 *   Adding routes

 *     default via 192.168.1.1 ..
```

PC3 is similar though I won't demonstrate it since it's my running NAS. PC2 I can test until the cows come home.

There is no reason for this error message from the kernel. All the appropriate modules are loaded and I've made no changes in either ethernet hardware or software for over a year. Except for just adding the Mellanox cards today.

----------

## dufeu

 *LIsLinuxIsSogood wrote:*   

>  *Quote:*   5: br0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000  
> 
> Oddly enough, it also looks like this machine (the working one) is running a different setup that I had to look up just now, called "promiscuous" mode...It refers to the reading of all network packets regardless of whether destined to that card or another address.  Perhaps it is nothing?   
> 
> I also concur that the default routes on all the machines should be looking more or less similar.

 

If I recall correctly - I need to look this up to confirm - promiscuous is required for bridge networking for virtual machines depending on how you want ethernet access to work for your VMs. i.e. If you're permitting a given VM to access the Internet, other VMs, other virtual networks on your LAN or even see the host itself. My setup required my VMs be able to access the Internet which meant access to my physical LAN etc.

----------

## eccerr0r

Yes, bridges are supposed to be in promiscuous mode.

So they can ping each other, just nothing outside of the switch?

Technically all are on the same network as you have everything on 192.168.1.0/24.  Just that these are all on switch1.

Switch1 (brand? model?):

PC1: br0:192.168.1.200/24 - can ping 192.168.1.1

PC2: eth0:192.168.1.201/24 - problems with 192.168.1.1

PC3: enp9s0:192.168.1.202/24 - problems with 192.168.1.1

Uplink...to somewhere...

Remote receiver of uplink... :

192.168.1.1/?? - router

You have an extensive network here it seems?  How many switches/bridges?

You definitely have a weird problem here.  Did you update the kernel recently as well?  I can't think of any user space programs minus perhaps iptables that could do something like this.  Tried flushing all your iptables/... firewall configuration, if you have any, just to be sure, including router?

What if you change the ip address from 201 / 202 to something else, maybe 21 / 22 if you're not already using them?  And unplugging the working pc with the bridge didn't do anything?  swapping ports?  physically connecting to elsewhere on the network?

----------

## Ralphred

 *dufeu wrote:*   

> 
> 
> PC2:Not working as expected:
> 
> ```
> ...

 

I'm sure I remember reading about not using CDIR notation in conf.d/net anymore*, something to do with a kernel upgrade maybe, anyway for PC2 try:

```
config_eth0="192.168.1.201 netmask 255.255.255.0 brd 192.168.1.255"

routes_eth0="default via 192.168.1.1"
```

*my suspicion about this bears out, my machine on 4.x kernel uses full notation, not something I would do unless forced to, my 3.x machine still uses CDIR.

----------

## LIsLinuxIsSogood

 *Quote:*   

> I'm sure I remember reading about not using CDIR notation in conf.d/net anymore

 

Where did you read that, it seems highly improbable that a net script of this sort would bnot accomodate the CIDR standard??

Anyway, you can try that because it won't hurt to specify the correct network masks but I am still repeating that the issue with these routing tables is that I'm seeing some bigger problems with them.  Which yes are affected by things like default routes, among other aspects of routes as specified in the routing table.   My suggestions still to proceed with flushing routing tables (why? there is nothing to lose with starting fresh, and the modern day protocols for networking with IP addresses etc. have a very intelligent/smart way of discovering hosts by themselves so you won't really have much else to do after that).  The other thing you can do to make things easier is limit the number of unnecessary gobbledy-gook in there by switching the VM's OFF.

Until you get the machines (physical PC's with real hardware connected among themselves) wait until later to add any virtualized networking layer to the mix.  But, if need be to set it up for later as well, and at the same time keep it from becoming too confused from he get go, tou may continue to PLAN ON needing it later, but also by keeping the virtual network adapters turned off it is going to help with things like default routes, and make later troubleshooting once it is needed a lot easier.

Please also make sure to post back some interface related driver information, such as are all these machines using same or different hardware, and assuming they are working (due to DHCP address assignments) can you also login to the router if you have access and ping them all from there?

----------

## LIsLinuxIsSogood

 *Quote:*   

> Tried flushing all your iptables... firewall configuration, if you have any, just to be sure, including router?

 

Just to emphasize this point it seems like a routing concern could be a concern of rputers and switches which is why I am suggestive of logging onto the  router and attempt that first (ping each Pc with whatever the tools on there are to do so).

----------

## dufeu

 *eccerr0r wrote:*   

> You have an extensive network here it seems?  How many switches/bridges?
> 
> You definitely have a weird problem here.  Did you update the kernel recently as well?  I can't think of any user space programs minus perhaps iptables that could do something like this.

 

I going to concentrate on just fixing 'firelizard' for now. It's a test box with no work or data on it. I'm going to ignore 'blaze' since it's at least performing it's intended function. Lack of internet access means I can't really run things like 'emerge' as I should.

At this point, I'm pretty sure the problem was introduced during my last attempt to perform an "emerge @world". I succeeded with 'firelizard' and 'blaze'. I failed with 'pryogyro' due to unresolved blocker issues. I bring this up because I initially built pyrogyro back in April of 2007. In that time, pyrogyro has successively migrated via ddrescue to 3 generations of PCs. firelizard was built using a stage4 tarball based on pyrogyro {virtually identical hardware} and blaze was built from a combined stage3/rsync approach less than 6 months ago. I bring this all this up because the settings on all three boxen are rich in gentoo portage history. I'd not be surprised that some unexpected historical artifact/interaction is in play here.

I manually keep the kernel in sync for all three PCs. Note:

```
pyrogyro ~ # uname -a

Linux pyrogyro 4.13.8-gentoo #3 SMP PREEMPT Wed Nov 22 10:08:00 EST 2017 x86_64 AMD A10-7850K Radeon R7, 12 Compute Cores 4C+8G AuthenticAMD GNU/Linux

firelizard ~ # uname -a

Linux firelizard 4.13.8-gentoo #1 SMP PREEMPT Thu Oct 19 07:12:49 EDT 2017 x86_64 AMD A10-7890K Radeon R7, 12 Compute Cores 4C+8G AuthenticAMD GNU/Linux

blaze /etc/conf.d # uname -a

Linux blaze 4.13.8-gentoo #3 SMP Wed Nov 22 10:17:40 EST 2017 x86_64 Intel(R) Xeon(R) CPU E5-2603 0 @ 1.80GHz GenuineIntel GNU/Linux
```

All three boxen are running the same kernel version.

Additionally, I always keep the last working kernel version as backup. On firelizard, I'm still having the "can't ping the router" problem regardless of which kernel I boot with:

```
firelizard ~ # ls -l /boot/

total 21674

lrwxrwxrwx 1 root root       1 Mar 14  2017 boot -> .

drwxr-xr-x 6 root root    1024 Nov 23 08:32 grub

-rw-r--r-- 1 root root 3872696 Sep 24 08:31 initramfs-genkernel-x86_64-4.12.14-gentoo

-rw-r--r-- 1 root root 3959900 Oct 19 13:27 initramfs-genkernel-x86_64-4.13.8-gentoo

-rw-r--r-- 1 root root 7131920 Sep 24 08:30 kernel-4.12.14-gentoo

-rw-r--r-- 1 root root 7213328 Oct 19 13:26 kernel-4.13.8-gentoo

drwx------ 2 root root   12288 Mar 14  2017 lost+found

drwxr-xr-x 2 root root    1024 Sep 24 07:46 memtest86plus
```

These two items together suggest to me that the kernel is not an issue. I'm not saying my issue(s) is(are) not in these areas. I'm just saying I think it's unlikely.

As for possible switch configuration issues, I'm not looking there first either. Up until 2 days ago, the switch in my room was a TP-Link TL-SG108. This is your bog standard mass produced from China, unmanaged {read: stupid} switch. The replacement is a Quanta L4M. My two problem children can only ping PCs/devices within the 'switch', regardless of which switch I use. I also checked this with a ZyXEL GS1100-16. Same behavior.

This is what firelizard looks like now:

```
firelizard conf.d # cat net

config_eth0="192.168.1.201/24"

routes_eth0="default via 192.168.1.1"

dns_servers_eth0="8.8.4.4 8.8.8.8 192.168.1.1"

firelizard conf.d # ip route

default via 192.168.1.1 dev eth0 metric 3 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.201 

firelizard conf.d # ip link

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/sit 0.0.0.0 brd 0.0.0.0

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000

    link/ether 40:8d:5c:54:eb:49 brd ff:ff:ff:ff:ff:ff

firelizard conf.d # udevadm test-builtin net_id /sys/class/net/eth0 2>/dev/null

ID_NET_NAME_MAC=enx408d5c54eb49

ID_OUI_FROM_DATABASE=GIGA-BYTE TECHNOLOGY CO.,LTD.

ID_NET_NAME_PATH=enp1s0

firelizard ~ # ip route

default via 192.168.1.1 dev eth0 metric 3 

127.0.0.0/8 via 127.0.0.1 dev lo 

192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.201 

firelizard ~ # ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.

^C

--- 192.168.1.1 ping statistics ---

3 packets transmitted, 0 received, 100% packet loss, time 2021ms

firelizard ~ # ping 192.168.1.200

PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data.

64 bytes from 192.168.1.200: icmp_seq=1 ttl=64 time=0.080 ms

64 bytes from 192.168.1.200: icmp_seq=2 ttl=64 time=0.253 ms

64 bytes from 192.168.1.200: icmp_seq=3 ttl=64 time=0.250 ms

64 bytes from 192.168.1.200: icmp_seq=4 ttl=64 time=0.240 ms

^C

--- 192.168.1.200 ping statistics ---

4 packets transmitted, 4 received, 0% packet loss, time 3030ms

rtt min/avg/max/mdev = 0.080/0.205/0.253/0.074 ms

firelizard ~ # ping 192.168.1.202

PING 192.168.1.202 (192.168.1.202) 56(84) bytes of data.

64 bytes from 192.168.1.202: icmp_seq=1 ttl=64 time=0.370 ms

64 bytes from 192.168.1.202: icmp_seq=2 ttl=64 time=0.349 ms

64 bytes from 192.168.1.202: icmp_seq=3 ttl=64 time=0.298 ms

^C

--- 192.168.1.202 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2023ms

rtt min/avg/max/mdev = 0.298/0.339/0.370/0.030 ms

firelizard ~ # ping 192.168.1.99

PING 192.168.1.99 (192.168.1.99) 56(84) bytes of data.

64 bytes from 192.168.1.99: icmp_seq=1 ttl=255 time=0.874 ms

64 bytes from 192.168.1.99: icmp_seq=2 ttl=255 time=2.94 ms

64 bytes from 192.168.1.99: icmp_seq=3 ttl=255 time=0.726 ms

^C

--- 192.168.1.99 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2021ms

rtt min/avg/max/mdev = 0.726/1.515/2.947/1.014 ms
```

I get the "RTNETLINK answers: No such file or directory" message on both firelizard and blaze. This disturbs me.  :Wink:  :

```
firelizard conf.d # ../init.d/net.eth0 restart

 * Caching service dependencies ...                                                                                                              [ ok ]

 * Bringing down interface eth0

RTNETLINK answers: No such file or directory

Error talking to the kernel

 * Bringing up interface eth0

 *   192.168.1.201/24 ...                                                                                                                        [ ok ]

 *   Adding routes

 *     default via 192.168.1.1 ... 
```

As others have noted, it appears on all interfaces. I see this on blaze. This is one of the things I think lends weight to the idea that my issues were somehow introduced with the 'emerge @world' on fireliard and blaze. While the RTNETLINK message may be meaningless and not have anything to do with my issues, my issues and the message seem to have appeared in the same timeline. On the surface, they also appear to involve networking hence the same group of programs or related programs.

Regarding our local network topolgy: from my Quanta L4M, I have a single cat5 cable across the house to another TP-Link TL-SG108. From there is a short {2 feet} hop to a FiOS G1100 "Designed by GreenWave" router. There is only one other room besides mine plugged into the remote TP-Link switch. The remainder of the house plugs into an ABS NW-111-8P. This is yet another unmanaged 8-port switch. The FiOS router has the two switches and 1 computer plugged in it. Other than a few computers directly plugged into the ABS switch, all other computers in the network plug into local room switches with then plug into one of the two switches by the router. Pretty much your usual small office star configuration.

We avoid using wireless for all in-house computers. Wireless is reserved solely for our smartphones. 

I'm going to putz around and try some random things. Among other things, I'll try different configurations for 'config_eth0="$COMMAND"'. I've been refreshing my memory and re-reading the Gentoo Handbook's Networking: Advanced.

I'm open to suggestions on different things to try.

BTW: The 10G subnet is up and sweeeet!.

edit: addendum!

One of the things that just occurred to me that we haven't really been paying enough attention to is that both blaze and firelizard successfully have initial conversations with the router.

blaze uses 'config_enp9s0="dhcp"' and gets the reserved address every time.

firelizard's 'routes_eth0="default via 192.168.1.1"' command seems to work and doesn't generate any failure message.

It's after initialization is complete that they can no longer see outside the switch.

----------

## eccerr0r

It does not make sense any userland programs, except those that influence the kernel like iptables, could cause such behavior.

Try 

```
# iptables -F
```

This is the only userland thing I could think of, if perhaps the saved file format was changed and now blocking all packets from your router for whatever reason.

----------

## dufeu

 *eccerr0r wrote:*   

> It does not make sense any userland programs, except those that influence the kernel like iptables, could cause such behavior.
> 
> Try 
> 
> ```
> ...

 

```
firelizard ~ # iptables -F

firelizard ~ #
```

 nothing reported.

Actually, I spent quite a bit of time trying different things. Ultimately, I set:

```
config_eth0="null"
```

and executed 'ip' manually:

```
# ip addr add 192.168.1.201/24 brd + dev eth0

# ip route add 192.168.1.0/24 via 192.168.1.1
```

I was able to ping the router on one occasion. I tried to execute

```
# dns_servers_eth0="8.8.8.8 192.168.1.1"
```

 and then I couldn't ping the router again.

When i tried to clear all route and IP address information from eth0, I'd get results indicating eth0 successfully cleared. Yet when I'd tried to re-enter a default route, it wouldn't let me do it saying it already existed.

I finally set the router to reserve the address and then set /etc/conf.d/net to

```
config_eth0="dhcp"
```

I watched it reboot several times. The messages are normal - i.e. query the router for an IP assignment, receive the reserved IP, query the network to see of anyone else is using this IP address. Timeout and accept the IP address. It's either right at the last steps of this process or it's one of the follow on boot processes which seem to turn everything pear shaped. I'm almost thinking it's at the end of the "dns_servers" command. /etc/resolv.conf seems to get updated properly. It's after that that I can no longer see router.

The other thing is that I don't seem to be able to completely clear all the route entries. After I think the entries are cleared, I go back to re-add routes. Then I get messages telling the route I want to add is already there. I've been having to reboot each time in order to test different approaches. Same goes for the arp cache. The issues I'm having with the two problem PCs could be table entries not being cleared properly. I need to find out more about where the tables are stored.

When I google RTNETLINK messages, what I find are almost always suggestions that there are missing kernel modules. Yet when I see RTNETLINK messages on my problem PCs, it's invariably telling me something about "no such file or directory". I'm beginning to believe the messages I'm seeing have to do with network table entries and possible missing or already existing elements.

I'm too tired to continue this tonight. 'Laters!

----------

## SP2340

Might I suggest a step by step process for fixing this?

Step one: ensure that both the switch/route and your box are using the same subnet mask.

Step two: stop iptables on your box to ensure that it is not interferring with communication.

IPTAVBLES can be stopped with one of the following depending on what system you are running;

```
openrc:

/etc/init.d/iptables stop

systemd:

systemctl stop iptables

```

Step three: ping from system to router and from router to system to ensure they are talking

Step four: post entire iptables rule set

This can be done using

```
iptables -S
```

Step five: does your router/switch have any settings that would not allow communication between devices?

If it does post those settings

Step six: if ping still is not working with the firewall turned off then post all network settings from both the router/switch and the box.

I know a few posts back you had different masks on both devices.

----------

## eccerr0r

I don't think iptables -F should report anything... it only clears all your rules, then you need to test pinging again.

You should always be able to ping the router regardless if you have a default route set, or even if you have the wrong route set.  If the wrong route was set, it just means you can't get out of your subnet.

The missing file thing still seems to imply something wrong with kernel or modules or something related...but doesn't make sense if you didn't touch the kernel...

----------

## dufeu

 *SP2340 wrote:*   

> Might I suggest a step by step process for fixing this?
> 
> Step one: ensure that both the switch/route and your box are using the same subnet mask.
> 
> Step two: stop iptables on your box to ensure that it is not interferring with communication.
> ...

 

Step 1:Router is a commercial device provided by Frontier. No changes there. Local switches are/were unmanaged 1G switches. Replaced room switch 3 days ago but no changes in problem behavior. All devices on room switch use same subnet mask: 192.168.1.0/24

Step 2: iptables is not running on any device connected to room switch.

```
blaze ~ # iptables -S

-P INPUT ACCEPT

-P FORWARD ACCEPT

-P OUTPUT ACCEPT

firelizard ~ # iptables -S

-P INPUT ACCEPT

-P FORWARD ACCEPT

-P OUTPUT ACCEPT

pyrogyro ~ # iptables -S

-P INPUT ACCEPT

-P FORWARD ACCEPT

-P OUTPUT ACCEPT

blaze ~ # rc-update show

               binfmt | boot                                   

             bootmisc | boot                                   

               cronie |      default                           

                devfs |                                 sysinit

                dmesg |                                 sysinit

                 fsck | boot                                   

             hostname | boot                                   

              hwclock | boot                                   

              keymaps | boot                                   

            killprocs |                        shutdown        

    kmod-static-nodes |                                 sysinit

...
```

rc-update show list similar results for firelizard and pryogyro. i.e. I don't start iptables/ip6tables at boot up as I don't feel a need for a firewall behind the router.

step 3: I can't ping the router from blaze nor firelizard. I can't ping blaze or firelizard from the router. pryrogyro had no problems in either direction. blaze and firelizard will both talk with the router during boot-up when aquiring their respective reserved IP addresses from the router. They stop communicating with the router sometime during the boot process.

step 4: posted - see step 1

step 5: no such settings {unmanaged switches}. Router configuration has been unchanged for 9 months.

step 6: Oh wow! The router is royally fscked. Going AFK to try to resolve this. Somehow, the router has assigned 4 IP address to blaze and 3 to firelizard. And I can't clear the assignments from admin ... shutting down all computers on net work and rebooting router now. More later.

----------

## eccerr0r

I think it should be OK to assign four different DHCP leases to a specific machine because a client ID is also associated with them, however all four should not be in its arp cache.  But you may not be able to view its arp cache...

----------

## dufeu

It turns out the ISP provided FiOS-G1100 Quantum Gateway was the sole source of the problem. When you login to the router via 'admin', there are four different places where you can check to see what devices are connected to the router. These are the ARP table listing {under the Advanced tab}. the DNS Server {under the Advanced tab - list DNS Server known connections}, IP Address Distribution {under the Advanced tab - Connection List} and several different connection lists under Firewall. Depending on which list you're looking at, it may make sense to have multiple entries for a single connected device.

Example: Under port forwarding, the connection list includes all active connections {IP:port#} including those created automatically for UPnP. Think: transmission bittorrent. The connection list also includes all inactive but manually added port forwarding connections as well.

Of interest here is the DNS Server Connection Listing. It is supposed to be a listing of all router IP assignments via DHCP plus all manually defined static connections. Unless the client is doing IP address aliasing, there should never be more than a single IP address per client in this listing. For my client PC 'blaze', there were 4 IP addresses listed. One was the reserved static IP address {201}. Two were router assigned IP addresses {19 + 25}. These three addresses all had the same host name of 'blaze'. The last IP address {153} was a amalgamation of two different samba server host names apparently when I changed /etc/samba/smb.conf on that PC. For my set up, the router should only ever have a single IP address for 'blaze'. It is interesting to note that the reserved static IP address never appeared in the ARP table. Unfortunately, it didn't occur to me until after I forced a factory reset of the router to check the ARP table for one of the bogus IP addresses.

Behaviorally, upon booting, blaze would properly negotiate getting an IP address. Upon IP address receipt, the router would apparently only recognize communications from one of the bogus IP addresses. The router basically ignored all packets with the correct IP address in favor of non-existent packets with the wrong IP address. Go figure.

It seems that setting up reserved IP assignments on this router has some potential for getting screwed up.

There is no way, or I couldn't find it, of clearing bad IP addresses that the router thinks it knows about. hence the factory reset.  Since I ended up doing a factory reset, I also reconfigured the built-in DHCP and reduced it's dynamic IP assignment range. I simplified my devices needing static IP addresses to manual self assignment. I'm hoping this avoids the router confusing itself in the future. Only my boxes and the printers need static assignment. 

Also of note: the RTNETLINK messages have disappeared from both computers. The usual recommendation that a kernel module didn't properly load was misleading in this case. I'm going to guess that once the router decided not to talk to the assigned address, this may have left something not properly set in the data used by the nic driver in the kernel. I recall from my days of installing multiple nics with the same chip that the data the kernel loaded could be funky if the nics weren't initialized correctly. This was one of the reasons why we used to compile the nic drivers as modules instead of compiling them in directly. Basically, I'm suggesting the driver got properly loaded but it's working data structure got crapped at the end of initial negotiation with the router.

To be honest, I've never before seen a built-in DHCP daemon in a router do unasked for multiple IP assignments for a single device. Note these were not simultaneous assignments but sequentially happening through prior assignments never being expired. I consider this to be multiple bugs with the router.

Problem resolved through factory reset.

I'd like to thank everyone whom offered really excellent advice and suggestions. Everything is running really sweet now.  :Wink: 

----------

## LIsLinuxIsSogood

 *Quote:*   

> To be honest, I've never before seen a built-in DHCP daemon in a router do unasked for multiple IP assignments for a single device.  Note these were not simultaneous assignments but sequentially happening through prior assignments never being expired.

 

While tricky, yes, now that youve identified the source as your ISP provided router, it is worth noting that a good router (yes better ones exist) can do a lot to improve not just the kind of trouble here you have with  lower level communications involved in connections over LAN, but additionally it could improve usability, security and even provide superior results in performance too.

 *Quote:*   

> I consider this to be multiple bugs with the router. 

 

I used to experience some “incompatibility” in the past with the isp provided equipment from Verizon but that was in the olden days of  dsl routers. (I cant say i am shocked though they are doing some funny stuff with their equipment).  Calling it buggy though may not actually be the most accurate as is likely these are limited aspects or features provided for low  price and production costs.  I suggest that you go by a new Router to improve things, when it comes to managing connections,, since a switch is only going to perform up to the capability of the router anyhow and it is the Router that needs to make up and processes the basic rules for routing on your network.

----------

## Ralphred

 *LIsLinuxIsSogood wrote:*   

> I used to experience some “incompatibility” in the past with the isp provided equipment from Verizon but that was in the olden days of  dsl routers. (I cant say i am shocked though they are doing some funny stuff with their equipment).  Calling it buggy though may not actually be the most accurate as is likely these are limited aspects or features provided for low  price and production costs.  I suggest that you go by a new Router to improve things, when it comes to managing connections,, since a switch is only going to perform up to the capability of the router anyhow and it is the Router that needs to make up and processes the basic rules for routing on your network.

 

Yeah, I can second this, routers trying to be "clever" and tripping over themselves in the attempt to make them easier to use, all on hardware worth less than the cost of shipping it to you. Do yourself a favour, go out and buy something, er, nice.

----------

