BPI-R64 - network performance

Hi all!

At the beginning of this post, I would like to thank everybody in this forum and especially @frank-w for all the extraordinary work in order to improve the software support of R64 and R2.

Initially I wanted to ask for help, since the routing performance of my BPI-R64 was quite bad, but recently I found that this is a thermal issue (CPU cooling is definitely needed) and therefore I would like to only share my test results and discuss some possible optimization here.

I general I would like to use the BPI-R64 as router/firewall in my home network. The following things are important for me:

  • high network throughput “LAN to LAN” and decent throughput “LAN to WAN”
  • iptables support

In order to verify my requirements, I did some network performance tests with iperf3 and scp. Since I’m only interested in routing scenarios, I connected 3 fairly good machines to WAN, LAN0 and LAN1 in order to test the performance (MASQUERADE/NAT is not needed).

I used one of the latest kernels from @frank-w and the user land from the official Ubuntu 18.04. (with the kernel from the official BPI release 2020-05-08, the performance was definitely worse)

build.conf (excerpt)

...
#board=bpi-r2
board=bpi-r64
#mainline uboot for r64 needs 64bit uImage
#uimagearch=arm64

Kernel

[root@bpi-r64 ~]# uname -a
Linux bpi-r64 5.4.77-bpi-r64-main #2 SMP PREEMPT Sat Nov 14 19:35:18 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

Network config is quite simple:

auto eth0
iface eth0 inet manual
auto eth1
iface eth1 inet manual
auto lan0
iface lan0 inet static
 address 10.2.0.2
 netmask 255.255.255.252
 post-up echo 1 > /sys/class/net/lan0/queues/rx-0/rps_cpus
auto lan1
iface lan1 inet static
 address 10.2.0.6
 netmask 255.255.255.252
 post-up echo 1 > /sys/class/net/lan1/queues/rx-0/rps_cpus

results LAN to LAN - before I found the thermal issue

date; iperf3 -n 5G -i 2 -c 10.2.0.5
So 15 Nov 2020 15:49:44 CET
Connecting to host 10.2.0.5, port 5201
[  5] local 10.2.0.1 port 54348 connected to 10.2.0.5 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-2.00   sec   200 MBytes   837 Mbits/sec    0   1.04 MBytes       
[  5]   2.00-4.00   sec   180 MBytes   755 Mbits/sec    0   1.04 MBytes       
[  5]   4.00-6.00   sec   174 MBytes   729 Mbits/sec    0   1.04 MBytes       
[  5]   6.00-8.00   sec   151 MBytes   634 Mbits/sec    0   1.04 MBytes       
[  5]   8.00-10.00  sec   121 MBytes   509 Mbits/sec    0   1.04 MBytes       
[  5]  10.00-12.00  sec   101 MBytes   425 Mbits/sec    0   1.04 MBytes       
[  5]  12.00-14.00  sec  76.2 MBytes   320 Mbits/sec    0   1.04 MBytes       
[  5]  14.00-16.00  sec  75.0 MBytes   315 Mbits/sec    0   1.04 MBytes       
[  5]  16.00-18.00  sec  56.2 MBytes   236 Mbits/sec    0   1.04 MBytes       
[  5]  18.00-20.00  sec  18.8 MBytes  78.6 Mbits/sec    0   1.04 MBytes       
[  5]  20.00-22.00  sec  47.5 MBytes   199 Mbits/sec    0   1.04 MBytes       
[  5]  22.00-24.00  sec  33.8 MBytes   142 Mbits/sec    0   1.04 MBytes       
[  5]  24.00-26.00  sec  45.0 MBytes   189 Mbits/sec    0   1.04 MBytes       
[  5]  26.00-28.00  sec  35.0 MBytes   147 Mbits/sec    0   1.04 MBytes          

The whole time, I thought it is not a thermal issue, since the CPU temperature is not really increasing during the test, but now I’m quite sure the this is not measuring the CPU temperature.

[root@bpi-r64 ~]# date; while true; do cat /sys/class/thermal/thermal_zone0/temp; sleep 2; done;
Sun Nov 15 15:49:43 CET 2020
46300
49500
49500
49700
49400
48700
49000
48300
47900
47600
47100
47500
46800
47400

And the issue can be easily found by checking the actual CPU frequency:

[root@bpi-r64 ~]# while true; do cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq; sleep 2; done;
1262500
1262500
1137500
1025000
1025000
812500
600000
437500
437500
30000
30000
437500
437500
437500
437500
437500

As you can see, the CPU frequency sometimes drops to even 30MHz.

results LAN to LAN - with professional cooling device (my fingers)

date; iperf3 -n 5G -i 2 -c 10.2.0.5
So 15 Nov 2020 15:58:49 CET
Connecting to host 10.2.0.5, port 5201
[  5] local 10.2.0.1 port 54362 connected to 10.2.0.5 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-2.00   sec   200 MBytes   838 Mbits/sec    0   1.04 MBytes       
[  5]   2.00-4.00   sec   208 MBytes   870 Mbits/sec    0   1.04 MBytes       
[  5]   4.00-6.00   sec   206 MBytes   865 Mbits/sec    0   1.04 MBytes       
[  5]   6.00-8.00   sec   205 MBytes   860 Mbits/sec    0   1.04 MBytes       
[  5]   8.00-10.00  sec   211 MBytes   886 Mbits/sec    0   1.04 MBytes       
[  5]  10.00-12.00  sec   191 MBytes   802 Mbits/sec    0   1.04 MBytes       
[  5]  12.00-14.00  sec   206 MBytes   865 Mbits/sec    0   1.04 MBytes       
[  5]  14.00-16.00  sec   212 MBytes   891 Mbits/sec    0   1.04 MBytes       
[  5]  16.00-18.00  sec   212 MBytes   891 Mbits/sec    0   1.04 MBytes       
[  5]  18.00-20.00  sec   212 MBytes   891 Mbits/sec    0   1.04 MBytes       
[  5]  20.00-22.00  sec   212 MBytes   891 Mbits/sec    0   1.04 MBytes       
[  5]  22.00-24.00  sec   212 MBytes   891 Mbits/sec    0   1.04 MBytes       
[  5]  24.00-26.00  sec   211 MBytes   886 Mbits/sec    0   1.04 MBytes       
[  5]  26.00-28.00  sec   204 MBytes   855 Mbits/sec    0   1.04 MBytes       
[  5]  28.00-30.00  sec   204 MBytes   855 Mbits/sec    0   1.04 MBytes       
[  5]  30.00-32.00  sec   195 MBytes   818 Mbits/sec    0   1.04 MBytes       
[  5]  32.00-34.00  sec   196 MBytes   823 Mbits/sec    0   1.04 MBytes       
[  5]  34.00-36.00  sec   205 MBytes   860 Mbits/sec    0   1.04 MBytes       
[  5]  36.00-38.00  sec   206 MBytes   865 Mbits/sec    0   1.04 MBytes       
[  5]  38.00-40.00  sec   198 MBytes   828 Mbits/sec    0   1.04 MBytes       
[  5]  40.00-42.00  sec   210 MBytes   881 Mbits/sec    0   1.04 MBytes       
[  5]  42.00-44.00  sec   210 MBytes   881 Mbits/sec    0   1.04 MBytes       
[  5]  44.00-46.00  sec   212 MBytes   891 Mbits/sec    0   1.04 MBytes       
[  5]  46.00-48.00  sec   212 MBytes   891 Mbits/sec    0   1.04 MBytes       
[  5]  48.00-49.57  sec   168 MBytes   892 Mbits/sec    0   1.04 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-49.57  sec  5.00 GBytes   867 Mbits/sec    0             sender
[  5]   0.00-49.57  sec  5.00 GBytes   866 Mbits/sec                  receiver

iperf Done.

So all-in-all I’m quite happy with the performance and I will do further test with iptables and maybe NAT next week.

I also used the following optimization from bpi-r2-improved-network-performance-by-rps since it increased the performance a bit, even with the thermal issue (i.e. when I didn’t cool the CPU).

echo 1 > /sys/class/net/lan0/queues/rx-0/rps_cpus
echo 1 > /sys/class/net/lan1/queues/rx-0/rps_cpus

Further optimizations

I have seen that the second GMAC (eth1) is available, but seems not to be used (WAN and LAN0-4 interfaces use eth0). Did anybody try this for R64 already (I have seen work for R2 only). Is this because of my /etc/network/interfaces config or does it need to be changes in e.g. mt7622-bananapi-bpi-r64.dts?

BR Michael

Hi,

I tested only client to lan-port and then i got always >900mbit/s with iperf3 (default settings),but i measured not traffic through device. have took a look on cpu-usage and interrupts?

have you tried disabling nat?

One gmac1 should be enought as on r64 this works in sgmii-mode (in theory 2,5gbit/s bidirectional). Dsa core only supports 1 cpu-port, so second gmac is not easy possible (by adding to dts). On r2 i (more rene) moved wan outside dsa and changed mode of switch to route wan to gmac2. This is no more possible with mt7531 driver because mt7531 itself does not support this (afaik only a vlan-based mapping).

Hi!

I didn’t use any iptables in my previous test.

However I did some tests with iptables and NAT today.

With IP tables

  • chains set to DROP and around 30-40 rules including LOG rules
  • speed decreased quite a lot
  • ksoftirqd uses ~90% CPU in top
  • interrupts is hard to say, since “GIC-0 30 Level arch_timer”, “Rescheduling interrupts” and “IRQ work interrupts” do increase all the time (even when there is no network load). it seems that the counter of the first two items increase by around 100-200 per second, when the network load is applied and the 3rd one does not increase so fast

date; iperf3 -n 5G -i 2 -c 10.2.0.5

Do 19 Nov 2020 22:37:06 CET
Connecting to host 10.2.0.5, port 5201
[  5] local 10.2.0.1 port 51830 connected to 10.2.0.5 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-2.00   sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]   2.00-4.00   sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]   4.00-6.00   sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]   6.00-8.00   sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]   8.00-10.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  10.00-12.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  12.00-14.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  14.00-16.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  16.00-18.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  18.00-20.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  20.00-22.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  22.00-24.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  24.00-26.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  26.00-28.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  28.00-30.00  sec   136 MBytes   571 Mbits/sec    0   1.02 MBytes       
[  5]  30.00-32.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  32.00-34.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  34.00-36.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  36.00-38.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  38.00-40.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  40.00-42.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  42.00-44.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  44.00-46.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  46.00-48.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  48.00-50.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  50.00-52.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  52.00-54.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  54.00-56.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  56.00-58.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  58.00-60.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  60.00-62.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  62.00-64.00  sec   138 MBytes   577 Mbits/sec    0   1.02 MBytes       
[  5]  64.00-66.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  66.00-68.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  68.00-70.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  70.00-72.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  72.00-74.00  sec   139 MBytes   582 Mbits/sec    0   1.02 MBytes       
[  5]  74.00-74.09  sec  6.25 MBytes   615 Mbits/sec    0   1.02 MBytes       

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-74.09  sec  5.00 GBytes   580 Mbits/sec    0             sender
[  5]   0.00-74.09  sec  5.00 GBytes   579 Mbits/sec                  receiver

iperf Done.

iptables + MASQUERADE on lan1

  • iperf3 client on lan0 is accessing server on lan1
  • iptables -t nat -A POSTROUTING -o lan1 -j MASQUERADE
  • CPU and interrupts seem similar to the previous test

date; iperf3 -n 5G -i 2 -c 10.2.0.5

Do 19 Nov 2020 22:54:47 CET
Connecting to host 10.2.0.5, port 5201
[  5] local 10.2.0.1 port 52196 connected to 10.2.0.5 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-2.00   sec   124 MBytes   522 Mbits/sec    0   1.02 MBytes       
[  5]   2.00-4.00   sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]   4.00-6.00   sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]   6.00-8.00   sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]   8.00-10.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  10.00-12.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  12.00-14.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  14.00-16.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  16.00-18.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  18.00-20.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  20.00-22.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  22.00-24.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  24.00-26.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  26.00-28.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  28.00-30.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  30.00-32.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  32.00-34.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  34.00-36.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  36.00-38.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  38.00-40.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  40.00-42.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  42.00-44.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  44.00-46.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  46.00-48.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  48.00-50.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  50.00-52.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  52.00-54.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  54.00-56.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  56.00-58.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  58.00-60.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  60.00-62.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  62.00-64.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  64.00-66.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  66.00-68.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  68.00-70.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  70.00-72.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  72.00-74.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  74.00-76.00  sec   126 MBytes   530 Mbits/sec    0   1.02 MBytes       
[  5]  76.00-78.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  78.00-80.00  sec   125 MBytes   524 Mbits/sec    0   1.02 MBytes       
[  5]  80.00-81.48  sec  92.5 MBytes   525 Mbits/sec    0   1.02 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-81.48  sec  5.00 GBytes   527 Mbits/sec    0             sender
[  5]   0.00-81.48  sec  5.00 GBytes   527 Mbits/sec                  receiver

iperf Done.

MASQUERADE only

  • first delete: iptables -F
  • first delete: iptables -P FORWARD ACCEPT
  • first delete: iptables -t nat -D POSTROUTING -o lan1 -j MASQUERADE
  • after iptables have been delete, speed is again about 800, but when NAT is enabled, speed drops to around 500-600
  • add again: iptables -t nat -A POSTROUTING -o lan1 -j MASQUERADE
  • CPU and interrupts seem similar to the previous tests

date; iperf3 -n 5G -i 2 -c 10.2.0.5

Do 19 Nov 2020 23:10:01 CET
Connecting to host 10.2.0.5, port 5201
[  5] local 10.2.0.1 port 52222 connected to 10.2.0.5 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-2.00   sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]   2.00-4.00   sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]   4.00-6.00   sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]   6.00-8.00   sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]   8.00-10.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  10.00-12.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  12.00-14.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  14.00-16.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  16.00-18.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  18.00-20.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  20.00-22.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  22.00-24.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  24.00-26.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  26.00-28.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  28.00-30.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  30.00-32.00  sec   125 MBytes   524 Mbits/sec    0   1.00 MBytes       
[  5]  32.00-34.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  34.00-36.00  sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]  36.00-38.00  sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]  38.00-40.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  40.00-42.00  sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]  42.00-44.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  44.00-46.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  46.00-48.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  48.00-50.00  sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]  50.00-52.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  52.00-54.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  54.00-56.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  56.00-58.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  58.00-60.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  60.00-62.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  62.00-64.00  sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]  64.00-66.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  66.00-68.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  68.00-70.00  sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]  70.00-72.00  sec   126 MBytes   530 Mbits/sec    0   1.00 MBytes       
[  5]  72.00-74.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  74.00-76.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  76.00-78.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  78.00-80.00  sec   128 MBytes   535 Mbits/sec    0   1.00 MBytes       
[  5]  80.00-80.55  sec  35.0 MBytes   535 Mbits/sec    0   1.00 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-80.55  sec  5.00 GBytes   533 Mbits/sec    0             sender
[  5]   0.00-80.55  sec  5.00 GBytes   533 Mbits/sec                  receiver

iperf Done.

When configuring and testing, I noticed that some LAN-interfaces do not come up after boot and cannot be brought up with:

[root@bpi-router ~]# ifup lan0
RTNETLINK answers: File exists
Failed to bring up lan0.

This issue seems to happen randomly and flushing the interface, fixes the problem:

ip addr flush dev lan0

Did anybody experience similar issues?

BR Michael

How do you bring them up? You need to bring up eth0 first,then lan/wan

My understanding is that /etc/network/interfaces controls the order in the official BPI Ubuntu. Do you have the same understanding?

In /etc/network/interfaces, the order of the entries is: lo, eth0, eth1, wan, lan0, lan1, lan2, lan3

I don’t know if bpi ubuntu uses this file as ubuntu 18.4 uses networkd/netplan which needs to be disabled first (netcfg/do_not_use_netplan=true in kernel cmdline). Maybe this is configuring lanx before ethx or tries to assign ip-config to eth0