As you have the issue also with wan-sfp it is more mtk driver issue because sfp is not connected to the airoha phy.
I can maybe force with ethtool my transceiver to set 2500 as speed and see if the issue is still there… will do more tests later, I have a 92mm fan spinning on it with a custom designed case xD
Could you do local speedtest,just to limit way to your endpoint which can also cause speeddrops?
1 Like
Did a test with my local truenas 10.10.1.20 with 2.5gbit interface.
The switch is still in the middle.
BPI 10g airoha wan PHY → 2,5g switch port → 2,5g truenas port.
Connecting to host 10.10.1.20, port 5201
[ 5] local 10.10.1.3 port 50018 connected to 10.10.1.20 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 128 KBytes 1.05 Mbits/sec 2 1.41 KBytes
[ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 3.00-4.00 sec 128 KBytes 1.05 Mbits/sec 36 1.41 KBytes
[ 5] 4.00-5.00 sec 128 KBytes 1.05 Mbits/sec 13 24.0 KBytes
[ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 2 1.41 KBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 9.00-10.01 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 384 KBytes 314 Kbits/sec 55 sender
[ 5] 0.00-13.28 sec 259 KBytes 160 Kbits/sec receiver
iperf Done.
BPI 10g rj45 transceiver WAN SPF+ → 2,5g switch port → 2,5g truenas port.
root@bpi-r4:~# iperf3 -c 10.10.1.20 -p 5201
Connecting to host 10.10.1.20, port 5201
[ 5] local 10.10.1.3 port 57464 connected to 10.10.1.20 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 256 MBytes 2.14 Gbits/sec 1327 103 KBytes
[ 5] 1.00-2.00 sec 251 MBytes 2.11 Gbits/sec 1365 94.7 KBytes
[ 5] 2.00-3.00 sec 170 MBytes 1.42 Gbits/sec 766 87.7 KBytes
[ 5] 3.00-4.00 sec 245 MBytes 2.06 Gbits/sec 1113 133 KBytes
[ 5] 4.00-5.00 sec 235 MBytes 1.97 Gbits/sec 1043 94.7 KBytes
[ 5] 5.00-6.00 sec 217 MBytes 1.82 Gbits/sec 1113 107 KBytes
[ 5] 6.00-7.00 sec 245 MBytes 2.05 Gbits/sec 1020 110 KBytes
[ 5] 7.00-8.00 sec 240 MBytes 2.01 Gbits/sec 1136 103 KBytes
[ 5] 8.00-9.00 sec 224 MBytes 1.88 Gbits/sec 837 96.2 KBytes
[ 5] 9.00-10.01 sec 240 MBytes 1.99 Gbits/sec 1165 72.1 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 2.27 GBytes 1.95 Gbits/sec 10885 sender
[ 5] 0.00-10.01 sec 2.27 GBytes 1.95 Gbits/sec receiver
iperf Done.
Edit:
Loaded the BPI 24.10 image with 1.9.1 firmware results are the same.
So R4Pro can send and truenas can receive with 2Gbit/s basicly (with many retransmitts maybe cause of wrong flowcontrol maybe also configurable on your switch?),but over the phy traffic is more worse than against your speedtest server. And it seems that mac is not the problem here.
My switch is very stupid. No config options.
For sure software is not mature, still I am a little bit confused why my SFP is seen as 10gbit when I can tell for sure that it can sync 1000/2,5/5/10gbit.
Indeed running
ethtool -s mxl_lan5 speed 2500 duplex full autoneg off
root@OpenWrt:~# iperf3 -c 10.10.1.20 -p 5201
Connecting to host 10.10.1.20, port 5201
[ 5] local 10.10.1.3 port 49230 connected to 10.10.1.20 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 283 MBytes 2.37 Gbits/sec 0 1.41 MBytes
[ 5] 1.00-2.00 sec 281 MBytes 2.36 Gbits/sec 0 1.48 MBytes
[ 5] 2.00-3.00 sec 279 MBytes 2.34 Gbits/sec 0 1.85 MBytes
[ 5] 3.00-4.00 sec 281 MBytes 2.36 Gbits/sec 857 1.35 MBytes
[ 5] 4.00-5.00 sec 281 MBytes 2.35 Gbits/sec 0 1.47 MBytes
[ 5] 5.00-6.00 sec 281 MBytes 2.36 Gbits/sec 0 1.63 MBytes
[ 5] 6.00-7.00 sec 280 MBytes 2.35 Gbits/sec 734 1.20 MBytes
[ 5] 7.00-8.00 sec 281 MBytes 2.36 Gbits/sec 0 1.37 MBytes
[ 5] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 36 802 KBytes
[ 5] 9.00-10.00 sec 280 MBytes 2.35 Gbits/sec 198 775 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.74 GBytes 2.36 Gbits/sec 1825 sender
[ 5] 0.00-10.00 sec 2.74 GBytes 2.35 Gbits/sec receiver
iperf Done.
root@OpenWrt:~# ethtool mxl_lan5
Settings for mxl_lan5:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseSR/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10000baseSR/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: off
Port: FIBRE
PHYAD: 0
Transceiver: internal
Supports Wake-on: d
Wake-on: d
Link detected: yes
Autoneg is off, speeds are better and less retries, still showing as 10gbit
For sure the aeonsemi (or airoha) has issues related to the real synced speed or whatever because i have the sentiment that keeps pushing at max speed.
Removing the SFP, switched back to PHY
[ 1354.765470] sfp sfp1: SM: enter empty:up:down event dev_down
[ 1354.771121] sfp sfp1: SM: exit empty:down:down
[ 1355.160683] Aeonsemi AS21xxx mdio-bus:18: failed to send ipc msg for 1: -14
[ 1357.195675] Aeonsemi AS21xxx mdio-bus:18: aeon_ipc_send_cmd fail to polling status failed: -110
[ 1357.204360] Aeonsemi AS21xxx mdio-bus:18: failed to send ipc msg for 1: -14
[ 1357.226544] Aeonsemi AS21xxx mdio-bus:18: failed to send ipc msg for 1: -14
[ 1366.076144] Aeonsemi AS21xxx mdio-bus:18: failed to send ipc msg for 1: -14
[ 1366.108678] Aeonsemi AS21xxx mdio-bus:18: Firmware Version: 1.9.1
[ 1366.115149] mxl862xx mdio-bus:10 mxl_lan5: PHY [mdio-bus:18] driver [Aeonsemi AS21xxx] (irq=POLL)
[ 1366.124021] mxl862xx mdio-bus:10 mxl_lan5: configuring for phy/10gbase-r link mode
[ 1366.132061] mxl862xx mdio-bus:10: dsa mux: switch to channel1
[ 1370.316209] mxl862xx mdio-bus:10 mxl_lan5: Link is Up - 2.5Gbps/Full - flow control rx/tx
[ 1370.316231] br-lan: port 7(mxl_lan5) entered blocking state
[ 1370.329950] br-lan: port 7(mxl_lan5) entered forwarding state
configured for 10gbase-r but link 2,5
10gbaser is imho right as this is the link between phy and mxl port. But i guess the issue is that it does not do inband management…a problem we try to fix the right way…so connecting the phy in usxgmii mode and let mac/dsa port config in real linkspeed (here 2500basex).
flashed the new image:
results are much better now:
BPI 10g airoha wan PHY → 2,5g switch port → 2,5g truenas port.
root@OpenWrt:~# ethtool eth1
Settings for eth1:
Supported ports: [ TP MII ]
Supported link modes: 100baseT/Full
1000baseT/Half 1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 100baseT/Full
1000baseT/Half 1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: Symmetric Receive-only
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 2500Mb/s
Duplex: Full
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 1c
Transceiver: external
MDI-X: Unknown
Current message level: 0x000000ff (255)
drv probe link timer ifdown ifup rx_err tx_err
Link detected: yes
root@OpenWrt:~# iperf3 -c 10.10.1.20 -p 5201
Connecting to host 10.10.1.20, port 5201
[ 5] local 10.10.1.3 port 48432 connected to 10.10.1.20 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 284 MBytes 2.38 Gbits/sec 0 990 KBytes
[ 5] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 0 990 KBytes
[ 5] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 1.01 MBytes
[ 5] 3.00-4.00 sec 281 MBytes 2.36 Gbits/sec 0 1.06 MBytes
[ 5] 4.00-5.00 sec 281 MBytes 2.36 Gbits/sec 0 1.06 MBytes
[ 5] 5.00-6.00 sec 280 MBytes 2.35 Gbits/sec 0 1.24 MBytes
[ 5] 6.00-7.00 sec 280 MBytes 2.35 Gbits/sec 0 1.24 MBytes
[ 5] 7.00-8.00 sec 280 MBytes 2.35 Gbits/sec 0 1.24 MBytes
[ 5] 8.00-9.00 sec 282 MBytes 2.36 Gbits/sec 0 1.24 MBytes
[ 5] 9.00-10.00 sec 280 MBytes 2.35 Gbits/sec 0 1.24 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.74 GBytes 2.36 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.74 GBytes 2.35 Gbits/sec receiver
iperf Done.
root@OpenWrt:~# iperf3 -c it1.speedtest.aruba.it -p 5201
Connecting to host it1.speedtest.aruba.it, port 5201
[ 5] local 10.10.1.3 port 36154 connected to 31.14.130.68 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 92.1 MBytes 772 Mbits/sec 0 3.77 MBytes
[ 5] 1.00-2.00 sec 108 MBytes 909 Mbits/sec 0 3.77 MBytes
[ 5] 2.00-3.00 sec 108 MBytes 909 Mbits/sec 0 3.77 MBytes
[ 5] 3.00-4.00 sec 109 MBytes 913 Mbits/sec 0 3.77 MBytes
[ 5] 4.00-5.00 sec 107 MBytes 894 Mbits/sec 1 2.69 MBytes
[ 5] 5.00-6.00 sec 108 MBytes 905 Mbits/sec 0 2.93 MBytes
[ 5] 6.00-7.00 sec 108 MBytes 906 Mbits/sec 0 3.13 MBytes
[ 5] 7.00-8.00 sec 110 MBytes 920 Mbits/sec 0 3.29 MBytes
[ 5] 8.00-9.00 sec 103 MBytes 863 Mbits/sec 1 2.42 MBytes
[ 5] 9.00-10.00 sec 98.9 MBytes 829 Mbits/sec 0 2.55 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.03 GBytes 882 Mbits/sec 2 sender
[ 5] 0.00-10.03 sec 1.03 GBytes 879 Mbits/sec receiver
iperf Done.
tested the lan 10gbit phy and has same results. wondering if this “Added support for 1000BASE-T and 2.5GBASE-T for 10G RJ45 WAN/LAN” means inband management
@zuowei8 from which source is this image created? i’ve seen that in MTK-SDK there are much changes to phy driver (seems like completely new codebase)