Banana Pi BPI-R2 network only 1 interface working

Banana Pi Router design BPI-R4(MT7988)
21

it does reply, but the receiver doesn’t get the ARP packet… very strange

14:36:23.658884 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
14:36:24.683151 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
14:36:25.706895 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
14:36:26.730856 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
14:36:27.755067 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
14:36:28.778893 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
14:36:29.802890 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
14:36:30.827204 d0:50:99:ae:82:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Request who-has 172.16.203.1 tell 172.16.201.1, length 46
22

I see requests and replies…

But it looks like you query hostname and not ip…

23

hi frank, if i use tcpdump -n it shows the ip address, tcpdump resolves the ip to the hostname

24

As layer2 seems to work,what says tcpdump if you do a ping?

25

this was the output when i was doing a ping, so we see arp packets but not icmp

26

Does tcpdump -n only show 1 ip-adress for r2?

27

no sorry, only when i do tcpdump -n -i wan.100

28

tested on my test-r2 and kernel 5.4.2-main without vlan:

root@bpi-r2:~# ip link set wan up
[  248.140287] mtk_soc_eth 1b100000.ethernet wan: PHY [mdio-bus:00] driver [Generic PHY]
[  248.148205] mtk_soc_eth 1b100000.ethernet wan: configuring for phy/rgmii link mode
[  252.322395] mtk_soc_eth 1b100000.ethernet wan:x
[  252.330774] IPv6: ADDRCONF(NETDEV_CHANGE): wan: link becomes ready

root@bpi-r2:~# ip addr add 192.168.0.11/24 dev wan                                                   
root@bpi-r2:~# ping 192.168.0.10
PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data.
64 bytes from 192.168.0.10: icmp_seq=1 ttl=64 time=0.844 ms
64 bytes from 192.168.0.10: icmp_seq=2 ttl=64 time=0.477 ms
^C
--- 192.168.0.10 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 2ms
rtt min/avg/max/mdev = 0.477/0.660/0.844/0.185 ms
root@bpi-r2:~#

on first quick test with vlan i see same result…only arp on ping

main-router (using lan0.500):

[17:47] root@bpi-r2-e:~ (526)# ping 192.168.50.10
PING 192.168.50.10 (192.168.50.10) 56(84) bytes of data.
17:47:41.654384 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
17:47:42.731661 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
17:47:43.771639 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
From 192.168.0.10 icmp_seq=1 Destination Host Unreachable
From 192.168.0.10 icmp_seq=2 Destination Host Unreachable
From 192.168.0.10 icmp_seq=3 Destination Host Unreachable
17:47:44.812358 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
17:47:45.861630 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
17:47:46.891643 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
From 192.168.0.10 icmp_seq=4 Destination Host Unreachable
From 192.168.0.10 icmp_seq=5 Destination Host Unreachable
From 192.168.0.10 icmp_seq=6 Destination Host Unreachable
17:47:47.932064 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
17:47:48.971675 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
17:47:50.011675 ARP, Reply 192.168.50.10 is-at 02:01:02:03:04:18 (oui Unknown), length 42
^C
--- 192.168.50.10 ping statistics ---
9 packets transmitted, 0 received, +6 errors, 100% packet loss, time 8357ms
pipe 4
[17:47] root@bpi-r2-e:~ (527)# ip addr show lan0.500
121: lan0.500@lan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:10:12:03:04:08 brd ff:ff:ff:ff:ff:ff
    inet 192.168.50.1/24 scope global lan0.500
       valid_lft forever preferred_lft forever
    inet6 fe80::10:12ff:fe03:408/64 scope link 
       valid_lft forever preferred_lft forever
[17:48] root@bpi-r2-e:~ (528)#

test-router:

root@bpi-r2:~# ip addr show wan.500
9: wan.500@wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen0
    link/ether 02:01:02:03:04:18 brd ff:ff:ff:ff:ff:ff
    inet 192.168.50.10/24 scope global wan.500
       valid_lft forever preferred_lft forever
    inet6 fe80::1:2ff:fe03:418/64 scope link 
       valid_lft forever preferred_lft forever
root@bpi-r2:~#

i see some strange on my main router after recognizing that untagged packets no more working after adding vlan:

17:51:21.051644 ARP, Reply 192.168.0.11 is-at 02:01:02:03:04:18 (oui Unknown), length 42

ip address is from untagged interface, but mac is from vlan 500

root@bpi-r2:~# ip addr show wan
3: wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 16:2a:bf:ed:8c:93 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.11/24 scope global wan
       valid_lft forever preferred_lft forever
    inet6 fe80::142a:bfff:feed:8c93/64 scope link 
       valid_lft forever preferred_lft forever
root@bpi-r2:~# ip addr show wan.500
9: wan.500@wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 
1000
    link/ether 02:01:02:03:04:18 brd ff:ff:ff:ff:ff:ff
    inet 192.168.50.10/24 scope global wan.500
       valid_lft forever preferred_lft forever
    inet6 fe80::1:2ff:fe03:418/64 scope link 
       valid_lft forever preferred_lft forever
root@bpi-r2:~#

thats indeed strange…my switch is configured on both ports in trunk mode 1U500T, so it should not mix up vlans / drop vlan-id

i also needed a reboot to get wan working again

29

hi frank, i’ve changed my setup again to 2 different interfaces. so without vlan. i can get my config working, but when i reboot, sometimes one of the interfaces shows the same result that arp is not working, when i reboot, it sometimes works again.

30

maybe you can test with following config:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
 pre-up ifconfig $IFACE up
 post-down ifconfig $IFACE down

auto wan
iface wan inet dhcp
 pre-up /bin/ip link set wan address 26:bf:b5:59:15:d9
 pre-up ifconfig $IFACE up
 post-down ifconfig $IFACE down

    auto lan0 
    iface lan0 inet manual
     address 172.16.203.1
     netmask 255.255.252.0
     pre-up /bin/ip link set lan0 address 26:bf:b5:59:15:d5
     pre-up ifconfig $IFACE up
     post-down ifconfig $IFACE down
    # vlan-raw-device wan

    root@bananapir2:~# cat /etc/netplan/10-fre.yaml 
    network:
        version: 2
        ethernets:
           lan0:
              addresses: 
               - 172.16.203.1/22
               - 172.16.203.2/22

and then reboot a few times when pinging both interfaces

31

you should drop the ifconfig lines for normal interfaces…you need them only for cpu-ports (ethx) and using ip command only

i guess this does not fix the issue, but it may interfere

32

ok ill give it a try

33

it seems not depending on vlan…after some time wan gets inactive (not link status, but i cannot ping)

34

that’s exactly the problem i have

35

have now tried pinging long time without issue

64 bytes from 192.168.0.10: icmp_seq=2552 ttl=64 time=0.398 ms

will try in some minutes…ping still working…added vlan…and can ping both :thinking:

root@bpi-r2:~# ping 192.168.0.10
PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data.
64 bytes from 192.168.0.10: icmp_seq=1 ttl=64 time=0.552 ms
64 bytes from 192.168.0.10: icmp_seq=2 ttl=64 time=0.464 ms
64 bytes from 192.168.0.10: icmp_seq=3 ttl=64 time=0.457 ms
64 bytes from 192.168.0.10: icmp_seq=4 ttl=64 time=0.461 ms
64 bytes from 192.168.0.10: icmp_seq=5 ttl=64 time=0.451 ms
^C
--- 192.168.0.10 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 155ms
rtt min/avg/max/mdev = 0.451/0.477/0.552/0.037 ms
root@bpi-r2:~# ping 192.168.50.1
PING 192.168.50.1 (192.168.50.1) 56(84) bytes of data.
64 bytes from 192.168.50.1: icmp_seq=1 ttl=64 time=0.928 ms
64 bytes from 192.168.50.1: icmp_seq=2 ttl=64 time=0.465 ms
64 bytes from 192.168.50.1: icmp_seq=3 ttl=64 time=0.484 ms
64 bytes from 192.168.50.1: icmp_seq=4 ttl=64 time=0.473 ms
^C
--- 192.168.50.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 94ms
rtt min/avg/max/mdev = 0.465/0.587/0.928/0.198 ms
root@bpi-r2:~# ip addr show wan
3: wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether aa:79:02:2c:bb:92 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.11/24 scope global wan
       valid_lft forever preferred_lft forever
    inet6 fe80::a879:2ff:fe2c:bb92/64 scope link 
       valid_lft forever preferred_lft forever
root@bpi-r2:~# ip addr show wan.500
9: wan.500@wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 
1000
    link/ether 02:01:02:03:04:18 brd ff:ff:ff:ff:ff:ff
    inet 192.168.50.10/24 scope global wan.500
       valid_lft forever preferred_lft forever
    inet6 fe80::1:2ff:fe03:418/64 scope link 
       valid_lft forever preferred_lft forever
root@bpi-r2:~#

basicly i added vlan this way to wan:

ip link add link wan name wan.500 type vlan id 500
ip link set wan.500 address 02:01:02:03:04:18 up
ip addr add 192.168.50.10/24 dev wan.500

after some time both don’t get working…i see arp packets from vlan, answered now with right mac-adress, and no packets from wan-ip

will now try with lan0

36

frank, found something more. When i ping from outside to bpi-r2 i receive answers. When i start pinging from the bpi-r2 to an address in the range, the connection is lost and the outside pings also stops responding.

37

seems vlan on lan0 is not working…but do not break untagged packets

38

i dont think its a vlan issue, problem is there on native interface also.

39

this is what i get on tcpdump when the pings stop responding. this happens when i ping from the bpi-r2 to another ip on the subnet.

19:50:37.422435 IP 172.16.201.1 > 172.16.203.1: ICMP echo request, id 3515, seq 3633, length 64
19:50:37.422623 IP 172.16.203.1 > 172.16.201.1: ICMP echo reply, id 3515, seq 3633, length 64
19:50:38.286551 IP 172.16.201.1 > 172.16.203.1: ICMP echo request, id 6142, seq 235, length 64
19:50:38.287202 IP 172.16.203.1 > 172.16.201.1: ICMP echo reply, id 6142, seq 235, length 64
19:50:38.446464 IP 172.16.201.1 > 172.16.203.1: ICMP echo request, id 3515, seq 3634, length 64
19:50:38.446772 IP 172.16.203.1 > 172.16.201.1: ICMP echo reply, id 3515, seq 3634, length 64
19:50:39.287892 IP 172.16.201.1 > 172.16.203.1: ICMP echo request, id 6142, seq 236, length 64
19:50:39.288532 IP 172.16.203.1 > 172.16.201.1: ICMP echo reply, id 6142, seq 236, length 64
19:50:39.470434 IP 172.16.201.1 > 172.16.203.1: ICMP echo request, id 3515, seq 3635, length 64
19:50:39.470622 IP 172.16.203.1 > 172.16.201.1: ICMP echo reply, id 3515, seq 3635, length 64
19:50:40.283296 ARP, Request who-has 172.16.200.1 tell 172.16.203.1, length 28
19:50:41.306815 ARP, Request who-has 172.16.200.1 tell 172.16.203.1, length 28
19:50:42.330811 ARP, Request who-has 172.16.200.1 tell 172.16.203.1, length 28
19:50:43.358403 ARP, Request who-has 172.16.200.1 tell 172.16.203.1, length 28
19:50:44.378986 ARP, Request who-has 172.16.200.1 tell 172.16.203.1, length 28
19:50:45.402889 ARP, Request who-has 172.16.200.1 tell 172.16.203.1, length 28

whenever there is a second mac address involved on the switch cpu the problems appear…

40

first tried on 4.14, there seems to be no issue, on my 4.19 it looks like vlan over wan basicly not working,no ping over vlan, but ping over wan stable so far, after moving ip-address to lan0 i cannot ping my other host. after reboot lan0 is working again, added vlan (does not work, i see arp requests, but not receved on the other side), lan0 still working

  • so vlan-functionality in 4.19 is broken, and ip-address cannot be moved to another device
  • at least in 5.4 it breaks host-interface

@ryder.lee any idea why this happen?