Banana Pi BPI-R2 network only 1 interface working

Banana Pi Router design BPI-R4(MT7988)
1

Hi i have a problem to activate more than one interface. If i configure 2 interfaces, 1 is not working. I have tried with networkd, ifupdown… same problem.

Tried with w-frank’s kernel and armbian, always the same issue. Even on different BPI-R2 boards.

i see both interfaces up. but only 1 is accessible. can somebody point me in the right direction?

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
   valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether fe:5b:d2:67:d4:62 brd ff:ff:ff:ff:ff:ff inet6 fe80::fc5b:d2ff:fe67:d462/64 scope link valid_lft forever preferred_lft forever 3: wan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether fe:5b:d2:67:d4:62 brd ff:ff:ff:ff:ff:ff inet 172.16.201.1/24 brd 172.16.201.255 scope global wan valid_lft forever preferred_lft forever inet6 fe80::fc5b:d2ff:fe67:d462/64 scope link valid_lft forever preferred_lft forever 4: lan0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether fe:5b:d2:67:d4:62 brd ff:ff:ff:ff:ff:ff inet 10.130.0.188/24 brd 10.130.0.255 scope global lan0 valid_lft forever preferred_lft forever inet6 fe80::fc5b:d2ff:fe67:d462/64 scope link valid_lft forever preferred_lft forever 5: lan1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether fe:5b:d2:67:d4:62 brd ff:ff:ff:ff:ff:ff 6: lan2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether fe:5b:d2:67:d4:62 brd ff:ff:ff:ff:ff:ff 7: lan3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether fe:5b:d2:67:d4:62 brd ff:ff:ff:ff:ff:ff 8: br0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether a2:88:4a:18:ec:da brd ff:ff:ff:ff:ff:ff

2

Please use code-tags.

I see only ip-address for wan and lan0

3

Hi Frank, i’m only using wan and lan0. if i ping both interfaces at the same time, i sometimes receive answers for a minute or 2, and then it times out. When i enable only 1 interface, everything is fine.

4

dumb questions…how do you ping? from 1 client? how is your topology?

as dsa-interfaces inheriting mac from cpu-port, and all ports are connected to eth0 you have same mac on all interfaces…

ip-adress (L3) is translated to mac (L2) before going to physical line (L1). both ip-adresses are translated to same mac on client…

so try changing mac-address of any of the 2 interfaces to another one

5

frank, i tested with different clients on my network, on the different subnets. (vlans) via ifupdown i tried to give a different mac on 1 or both interfaces… same result.

6

client1 - wan R2 lan0 - client2

with wan-mac != lan0-mac does not work?

if you define vlans on physical ports imho the mac is also inherited from parent device…

can you post output of “ip a” with different mac-adresses if its not working? and formatted as code to be readable (select block and press “<>”)

maybe ethtool -S wan/lan0 is also helpful

also which kernel from me you have used…i know there is a trgmii-issue, but not how to reproduce it…maybe you can change trgmii in dts to rgmii for testing and recompile kernel

7

this is my setup:

lan0 dhcp client. (this IF is now stable) wan static ip with dhcp server. (clients get IP but can not ping to static ip) both ports are untagged on switch to different vlan.

if i disable lan0 on startup, clients on wan get ip and can ping wan interface.

this is my /etc/network/interfaces:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
pre-up ifconfig $IFACE up
post-down ifconfig $IFACE down

auto wan
iface wan inet static
 address 172.16.201.1
 netmask 255.255.255.0
 hwaddress ether de:4f:fb:d5:3c:fb
 pre-up ifconfig $IFACE up
 post-down ifconfig $IFACE down

auto lan0
iface lan0 inet dhcp
 hwaddress ether de:4f:fb:d5:3c:fa
 pre-up ifconfig $IFACE up
 post-down ifconfig $IFACE down


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 66:f7:dd:f2:e3:68 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::64f7:ddff:fef2:e368/64 scope link 
       valid_lft forever preferred_lft forever
3: wan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether de:4f:fb:d5:3c:fb brd ff:ff:ff:ff:ff:ff
    inet 172.16.201.1/24 brd 172.16.201.255 scope global wan
       valid_lft forever preferred_lft forever
    inet6 fe80::dc4f:fbff:fed5:3cfb/64 scope link 
       valid_lft forever preferred_lft forever
4: lan0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether de:4f:fb:d5:3c:fa brd ff:ff:ff:ff:ff:ff
    inet 192.168.88.153/24 brd 192.168.88.255 scope global lan0
       valid_lft forever preferred_lft forever
    inet6 fe80::dc4f:fbff:fed5:3cfa/64 scope link 
       valid_lft forever preferred_lft forever
5: lan1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 66:f7:dd:f2:e3:68 brd ff:ff:ff:ff:ff:ff
6: lan2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 66:f7:dd:f2:e3:68 brd ff:ff:ff:ff:ff:ff
7: lan3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 66:f7:dd:f2:e3:68 brd ff:ff:ff:ff:ff:ff

root@bananapir2:~# ethtool -S wan
NIC statistics:
     tx_packets: 51
     tx_bytes: 3268
     rx_packets: 392
     rx_bytes: 57979
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 26
     TxMulticast: 19
     TxBroadcast: 13
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 31
     TxPktSz65To127: 26
     TxPktSz128To255: 0
     TxPktSz256To511: 1
     TxPktSz512To1023: 0
     Tx1024ToMax: 0
     TxBytes: 4540
     RxDrop: 0
     RxFiltering: 419
     RxMulticast: 306
     RxBroadcast: 86
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 244
     RxPktSz65To127: 394
     RxPktSz128To255: 40
     RxPktSz256To511: 133
     RxPktSz512To1023: 0
     RxPktSz1024ToMax: 0
     RxBytes: 104474
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0

root@bananapir2:~# ethtool -S lan0
NIC statistics:
     tx_packets: 318
     tx_bytes: 46429
     rx_packets: 1822
     rx_bytes: 166173
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 302
     TxMulticast: 20
     TxBroadcast: 5
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 8
     TxPktSz65To127: 213
     TxPktSz128To255: 79
     TxPktSz256To511: 19
     TxPktSz512To1023: 5
     Tx1024ToMax: 3
     TxBytes: 48511
     RxDrop: 0
     RxFiltering: 0
     RxMulticast: 751
     RxBroadcast: 611
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 955
     RxPktSz65To127: 522
     RxPktSz128To255: 161
     RxPktSz256To511: 182
     RxPktSz512To1023: 0
     RxPktSz1024ToMax: 2
     RxBytes: 198969
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0
8

try removing this for both interfaces…

9

hi frank, issue is the same.

10

I still think it’s a mac-adress issue and mac-change happens too late…if both ports working alone but not simultanously…

Switch sees mac twice (with vlan it is different lan segment but i guess it is not enough) and do not know where to “route” packets…

Or you can try the trgmii-change (maybe using 5.4-main).

11

strange thing is that i see the correct mac address in the arp table. my kernel uses trgmii, should i change it?

12

You can try it but it does not explain why 1 port works but not 2

13

hi frank, even if i try with one interface and when i add vlan tag interface to it, problem is the same… i’m pulling my hair out for the moment :slight_smile:

14

which kernel do you use? on which OS? how do you configure vlan? Where is your vlan?

On wan i see this (maybe because of vlan):

RxFiltering: 419
15

Hi Frank, im using following kernel on armbian:

Linux bananapir2 4.19.20-mt7623 #5.75 SMP Mon Feb 11 08:08:34 CET 2019 armv7l armv7l armv7l GNU/Linux

This is my network setup: auto lo iface lo inet loopback

auto eth0
iface eth0 inet manual
 pre-up ifconfig $IFACE up
 post-down ifconfig $IFACE down

auto wan
iface wan inet dhcp

# pre-up ifconfig $IFACE up
# post-down ifconfig $IFACE down

auto wan.100
iface wan.100 inet static
 address 172.16.203.1
 netmask 255.255.252.0

# pre-up ifconfig $IFACE up
# post-down ifconfig $IFACE down
 vlan-raw-device wan

root@bananapir2:~# ethtool -S wan
NIC statistics:
     tx_packets: 714
     tx_bytes: 110567
     rx_packets: 422313
     rx_bytes: 116795210
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 627
     TxMulticast: 91
     TxBroadcast: 23
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 340
     TxPktSz65To127: 148
     TxPktSz128To255: 0
     TxPktSz256To511: 253
     TxPktSz512To1023: 0
     Tx1024ToMax: 0
     TxBytes: 121233
     RxDrop: 0
     RxFiltering: 940
     RxMulticast: 325982
     RxBroadcast: 95714
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 118029
     RxPktSz65To127: 122113
     RxPktSz128To255: 69233
     RxPktSz256To511: 26693
     RxPktSz512To1023: 56519
     RxPktSz1024ToMax: 30666
     RxBytes: 124690224
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0
16

Don’t know this option…can you try dropping this?

your wan gets ip over dhcp? You have still rxfiltering which is afair from vlan definition (you’ve run ethtool on wan only…can you check wan.100 too?

17

frank, i’ve dropped the option, same result.

i can not get stats from the vlan interface:

root@bananapir2:~# ethtool -S wan.100
no stats available
18

the vlan interface does not announce its mac address, same issue that i have on the second interface before, without the vlan…

So when there is more than 1 interface, problems appear with mac addresses.

19

seems like ethtool does not support vlans

maybe “ip -s link” shows any drops on vlan…

i have set mac on vlan this way (because i have 2 and they need to be different) over a ppp running over it

auto wan.140
iface wan.140 inet manual

auto internet
iface internet inet ppp
  pre-up /bin/ip link set wan.140 address 02:11:02:03:04:05 up
  provider internet
20

frank, i’m now using your way to configure the network, and now i see arp packets. But it doesn’t reply

14:28:52.069993 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:28:53.094192 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:28:53.094405 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:28:54.117957 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:28:54.118167 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:28:55.141959 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:28:55.142169 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:28:56.166263 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:28:56.166470 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:28:57.190012 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:28:57.190225 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:28:58.213975 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:28:58.214184 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:28:59.238306 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:28:59.238521 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28
14:29:00.262062 ARP, Request who-has bananapir2 tell 172.16.201.1, length 46
14:29:00.262263 ARP, Reply bananapir2 is-at 02:11:02:03:04:05 (oui Unknown), length 28