[BPI-R4] and SFP

For more info from kernel, do this before inserting sfp:

echo "file drivers/net/phy/* +p" > /sys/kernel/debug/dynamic_debug/control

Hopefully dynamic debugging is enabled…

Then it shows more on how mac is configured…

1000Mbps sgmii leave autoneg on. It is only broken for 2500base-x. Disabling it may break it.

plugin:

[ 2328.551015] sfp sfp1: tx-fault 1 -> 0
[ 2328.554698] sfp sfp1: SM: enter empty:down:down event tx_clear
[ 2328.560561] sfp sfp1: SM: exit empty:down:down
[ 2328.565013] sfp sfp1: mod-def0 0 -> 1
[ 2328.568667] sfp sfp1: los 1 -> 0
[ 2328.571891] sfp sfp1: SM: enter empty:down:down event insert
[ 2328.577538] sfp sfp1: SM: exit probe:down:down
[ 2328.581974] sfp sfp1: SM: enter probe:down:down event los_low
[ 2328.587707] sfp sfp1: SM: exit probe:down:down
[ 2328.879965] sfp sfp1: SM: enter probe:down:down event timeout
[ 2328.897013] sfp sfp1: module OEM              GLC-BX-U-C       rev 1.0  sn 1152216730       dc 150602  
[ 2328.906412] mtk_soc_eth 15100000.ethernet eth2: optical SFP: interfaces=[mac=2-4,22-24,27,29, sfp=22]
[ 2328.915631] mtk_soc_eth 15100000.ethernet eth2:  interface 22 (1000base-x) rate match none supports 6,10,13-14,41
[ 2328.925889] mtk_soc_eth 15100000.ethernet eth2: optical SFP: chosen 1000base-x interface
[ 2328.933974] mtk_soc_eth 15100000.ethernet eth2: requesting link mode inband/1000base-x with support 00,00000000,00000200,00006440
[ 2328.945615] sfp sfp1: SM: exit present:down:down
[ 2328.961077] hwmon hwmon2: temp1_input not attached to any thermal zone
[ 2333.878157] sfp sfp1: los 0 -> 1
[ 2333.881405] sfp sfp1: SM: enter present:down:down event los_high
[ 2333.887401] sfp sfp1: SM: exit present:down:down
[ 2333.892043] sfp sfp1: los 1 -> 0
[ 2333.895260] sfp sfp1: SM: enter present:down:down event los_low
[ 2333.901174] sfp sfp1: SM: exit present:down:down

plugin + plugout:

[ 2328.551015] sfp sfp1: tx-fault 1 -> 0
[ 2328.554698] sfp sfp1: SM: enter empty:down:down event tx_clear
[ 2328.560561] sfp sfp1: SM: exit empty:down:down
[ 2328.565013] sfp sfp1: mod-def0 0 -> 1
[ 2328.568667] sfp sfp1: los 1 -> 0
[ 2328.571891] sfp sfp1: SM: enter empty:down:down event insert
[ 2328.577538] sfp sfp1: SM: exit probe:down:down
[ 2328.581974] sfp sfp1: SM: enter probe:down:down event los_low
[ 2328.587707] sfp sfp1: SM: exit probe:down:down
[ 2328.879965] sfp sfp1: SM: enter probe:down:down event timeout
[ 2328.897013] sfp sfp1: module OEM              GLC-BX-U-C       rev 1.0  sn 1152216730       dc 150602  
[ 2328.906412] mtk_soc_eth 15100000.ethernet eth2: optical SFP: interfaces=[mac=2-4,22-24,27,29, sfp=22]
[ 2328.915631] mtk_soc_eth 15100000.ethernet eth2:  interface 22 (1000base-x) rate match none supports 6,10,13-14,41
[ 2328.925889] mtk_soc_eth 15100000.ethernet eth2: optical SFP: chosen 1000base-x interface
[ 2328.933974] mtk_soc_eth 15100000.ethernet eth2: requesting link mode inband/1000base-x with support 00,00000000,00000200,00006440
[ 2328.945615] sfp sfp1: SM: exit present:down:down
[ 2328.961077] hwmon hwmon2: temp1_input not attached to any thermal zone
[ 2333.878157] sfp sfp1: los 0 -> 1
[ 2333.881405] sfp sfp1: SM: enter present:down:down event los_high
[ 2333.887401] sfp sfp1: SM: exit present:down:down
[ 2333.892043] sfp sfp1: los 1 -> 0
[ 2333.895260] sfp sfp1: SM: enter present:down:down event los_low
[ 2333.901174] sfp sfp1: SM: exit present:down:down
[ 2342.464426] sfp sfp1: los 0 -> 1
[ 2342.467658] sfp sfp1: SM: enter present:down:down event los_high
[ 2342.473675] sfp sfp1: SM: exit present:down:down
[ 2342.478315] sfp sfp1: los 1 -> 0
[ 2342.481540] sfp sfp1: SM: enter present:down:down event los_low
[ 2342.487447] sfp sfp1: SM: exit present:down:down
[ 2346.762508] sfp sfp1: los 0 -> 1
[ 2346.765743] sfp sfp1: SM: enter present:down:down event los_high
[ 2346.771758] sfp sfp1: SM: exit present:down:down
[ 2346.776397] sfp sfp1: los 1 -> 0
[ 2346.779614] sfp sfp1: SM: enter present:down:down event los_low
[ 2346.785528] sfp sfp1: SM: exit present:down:down
[ 2351.057402] sfp sfp1: los 0 -> 1
[ 2351.060649] sfp sfp1: SM: enter present:down:down event los_high
[ 2351.066645] sfp sfp1: SM: exit present:down:down
[ 2351.071288] sfp sfp1: los 1 -> 0
[ 2351.074505] sfp sfp1: SM: enter present:down:down event los_low
[ 2351.080417] sfp sfp1: SM: exit present:down:down
[ 2355.349203] sfp sfp1: los 0 -> 1
[ 2355.352448] sfp sfp1: SM: enter present:down:down event los_high
[ 2355.358443] sfp sfp1: SM: exit present:down:down
[ 2355.363084] sfp sfp1: los 1 -> 0
[ 2355.366301] sfp sfp1: SM: enter present:down:down event los_low
[ 2355.372214] sfp sfp1: SM: exit present:down:down
[ 2359.640797] sfp sfp1: los 0 -> 1
[ 2359.644033] sfp sfp1: SM: enter present:down:down event los_high
[ 2359.650042] sfp sfp1: SM: exit present:down:down
[ 2359.654677] sfp sfp1: los 1 -> 0
[ 2359.657894] sfp sfp1: SM: enter present:down:down event los_low
[ 2359.663808] sfp sfp1: SM: exit present:down:down
[ 2363.933623] sfp sfp1: los 0 -> 1
[ 2363.936858] sfp sfp1: SM: enter present:down:down event los_high
[ 2363.942866] sfp sfp1: SM: exit present:down:down
[ 2363.947499] sfp sfp1: los 1 -> 0
[ 2363.950724] sfp sfp1: SM: enter present:down:down event los_low
[ 2363.956629] sfp sfp1: SM: exit present:down:down
[ 2368.224957] sfp sfp1: los 0 -> 1
[ 2368.228193] sfp sfp1: SM: enter present:down:down event los_high
[ 2368.234204] sfp sfp1: SM: exit present:down:down
[ 2368.238838] sfp sfp1: los 1 -> 0
[ 2368.242062] sfp sfp1: SM: enter present:down:down event los_low
[ 2368.247969] sfp sfp1: SM: exit present:down:down
[ 2372.519252] sfp sfp1: los 0 -> 1
[ 2372.522497] sfp sfp1: SM: enter present:down:down event los_high
[ 2372.528494] sfp sfp1: SM: exit present:down:down
[ 2372.533132] sfp sfp1: los 1 -> 0
[ 2372.536349] sfp sfp1: SM: enter present:down:down event los_low
[ 2372.542263] sfp sfp1: SM: exit present:down:down
[ 2376.475029] sfp sfp1: los 0 -> 1
[ 2376.478264] sfp sfp1: SM: enter present:down:down event los_high
[ 2376.484269] sfp sfp1: SM: exit present:down:down
[ 2376.488893] sfp sfp1: mod-def0 1 -> 0
[ 2376.492551] sfp sfp1: tx-fault 0 -> 1
[ 2376.496202] sfp sfp1: SM: enter present:down:down event remove
[ 2376.502329] sfp sfp1: module removed
[ 2376.505905] sfp sfp1: SM: exit empty:down:down
[ 2376.510360] sfp sfp1: SM: enter empty:down:down event tx_fault
[ 2376.516185] sfp sfp1: SM: exit empty:down:down

big update, the first sfp port is eth2 and i got now an IP from my ISP, yay :slight_smile: i changed only the sgmii thingen as you mentioned in the .dts. Can i apply the same to openwrt and it should work?

Should work, but who knows…

Can i trigger dynamic debug from bootup (to avoid hotplugging sfp while system running)? I guess else i miss some messages when only enable dyndebug and the set interface up to look for advertising (afair i did this already and did not see any advertising).

I was thinking about the same recently, but still have to look it up…

Why avoid hotplug? Should be ok.

plugged one of my sfp in a 2.5/10G-switch i ordered recently, the other is in r4 v1.1 on wan-sfp

root@bpi-r4-v11:~# echo "file drivers/net/* +p" > /sys/kernel/debug/dynamic_debug/control
root@bpi-r4-v11:~# ip link set eth2 up                                                                                                      
[   82.516418] mtk_soc_eth 15100000.ethernet eth2: configuring for inband/10gbase-r link mode                                               
[   82.524753] mtk_soc_eth 15100000.ethernet eth2: major config 10gbase-r                                                                   
[   82.531831] mtk_soc_eth 15100000.ethernet eth2: phylink_mac_config: mode=inband/10gbase-r/none adv=00,00000000,00000800,00006440 pause=04
[   82.544175] mtk_soc_eth 15100000.ethernet: mux mux_gdm1_to_gmac1_esw isn't present on the SoC                                            
[   82.552770] mtk_soc_eth 15100000.ethernet: mux mux_gmac2_gmac0_to_gephy isn't present on the SoC                                         
[   82.561555] mtk_soc_eth 15100000.ethernet: mux mux_u3_gmac2_to_qphy isn't present on the SoC                                             
[   82.569990] mtk_soc_eth 15100000.ethernet: mux mux_gmac1_gmac2_to_sgmii_rgmii isn't present on the SoC                                   
[   82.579279] mtk_soc_eth 15100000.ethernet: mux mux_gmac12_to_gephy_sgmii isn't present on the SoC                                        
[   82.588150] mtk_soc_eth 15100000.ethernet: path gmac3_usxgmii in set_mux_gmac123_to_gephy_sgmii updated = 0                              
[   82.597889] mtk_soc_eth 15100000.ethernet: path gmac3_usxgmii in set_mux_gmac123_to_usxgmii updated = 1                                  
[   82.609463] sfp sfp1: SM: enter present:down:down event dev_up                                                                           
[   82.615298] sfp sfp1: tx disable 1 -> 0                                                                                                  
[   82.619142] sfp sfp1: SM: exit present:up:wait                                                                                           
root@bpi-r4-v11:~# [   82.669782] sfp sfp1: SM: enter present:up:wait event timeout                                                         
[   82.675539] sfp sfp1: SM: exit present:up:link_up                                                                                        
                                                                                                                                            
root@bpi-r4-v11:~# ethtool eth2                                                                                                             
Settings for eth2:                                                                                                                          
        Supported ports: [ FIBRE ]                                                                                                          
        Supported link modes:   10000baseSR/Full                                                                                            
        Supported pause frame use: Symmetric Receive-only                                                                                   
        Supports auto-negotiation: Yes                                                                                                      
        Supported FEC modes: Not reported                                                                                                   
        Advertised link modes:  10000baseSR/Full                                                                                            
        Advertised pause frame use: Symmetric Receive-only                                                                                  
        Advertised auto-negotiation: Yes                                                                                                    
        Advertised FEC modes: Not reported                                                                                                  
        Speed: Unknown!                                                                                                                     
        Duplex: Unknown! (255)                                                                                                              
        Auto-negotiation: on                                                                                                                
        Port: FIBRE                                                                                                                         
        PHYAD: 0                                                                                                                            
        Transceiver: internal                                                                                                               
        Current message level: 0x000000ff (255)                                                                                             
                               drv probe link timer ifdown ifup rx_err tx_err                                                               
        Link detected: no                                                                                                                   
root@bpi-r4-v11:~#

strange are the pinmuxes, right?? seems these are defines for the packet-routing (path)

$ grep -Rni 'mux_gdm1_to_gmac1_esw'
drivers/net/ethernet/mediatek/mtk_eth_path.c:53:static int set_mux_gdm1_to_gmac1_esw(struct mtk_eth *eth, u64 path)
drivers/net/ethernet/mediatek/mtk_eth_path.c:276:		.name = "mux_gdm1_to_gmac1_esw",
drivers/net/ethernet/mediatek/mtk_eth_path.c:277:		.cap_bit = MTK_ETH_MUX_GDM1_TO_GMAC1_ESW,
drivers/net/ethernet/mediatek/mtk_eth_path.c:278:		.set_path = set_mux_gdm1_to_gmac1_esw,
drivers/net/ethernet/mediatek/mtk_eth_soc.h:1065:	MTK_ETH_MUX_GDM1_TO_GMAC1_ESW_BIT,
drivers/net/ethernet/mediatek/mtk_eth_soc.h:1112:#define MTK_ETH_MUX_GDM1_TO_GMAC1_ESW		\
drivers/net/ethernet/mediatek/mtk_eth_soc.h:1113:	BIT_ULL(MTK_ETH_MUX_GDM1_TO_GMAC1_ESW_BIT)
drivers/net/ethernet/mediatek/mtk_eth_soc.h:1158:#define MTK_MUX_GDM1_TO_GMAC1_ESW (MTK_ETH_MUX_GDM1_TO_GMAC1_ESW | MTK_MUX)
drivers/net/ethernet/mediatek/mtk_eth_soc.h:1195:		      MTK_MUX_GDM1_TO_GMAC1_ESW | \
drivers/net/ethernet/mediatek/mtk_eth_soc.h:1204:		      MTK_GDM1_ESW | MTK_MUX_GDM1_TO_GMAC1_ESW | \

error-mesage is also printed there:

$ grep -Rni "isn't present on the SoC"
drivers/net/ethernet/mediatek/mtk_eth_path.c:330:			dev_dbg(eth->dev, "mux %s isn't present on the SoC\n",

in else branch for condition

if (MTK_HAS_CAPS(eth->soc->caps, mtk_eth_muxc[i].cap_bit)) {

one point i see is that eth->soc->caps is u64 and the cap_bit is int…so maybe a unsigned/size issue in MTK_HAS_CAPS macro??

i added some debug, but it seems not related to usxgmii…only sgmii and rgmii…

noticed another thing…link_up event seems to run into timeout and later changed without updating state in ethtool…

[   31.428589] sfp sfp1: SM: enter present:down:down event dev_up                                                                           
[   31.434413] sfp sfp1: tx disable 1 -> 0                                                                                                  
[   31.438267] sfp sfp1: SM: exit present:up:wait                                                                                           
root@bpi-r4-v11:~# [   31.496164] sfp sfp1: SM: enter present:up:wait event timeout                                                         
[   31.501919] sfp sfp1: SM: exit present:up:link_up                                                                                        
                                                                                                                                            
root@bpi-r4-v11:~# ethtool eth2                                                                                                             
Settings for eth2:                                                                                                                          
        Supported ports: [ FIBRE ]                                                                                                          
        Supported link modes:   10000baseSR/Full                                                                                            
        Supported pause frame use: Symmetric Receive-only                                                                                   
        Supports auto-negotiation: Yes                                                                                                      
        Supported FEC modes: Not reported                                                                                                   
        Advertised link modes:  10000baseSR/Full                                                                                            
        Advertised pause frame use: Symmetric Receive-only                                                                                  
        Advertised auto-negotiation: Yes                                                                                                    
        Advertised FEC modes: Not reported                                                                                                  
        Speed: Unknown!                                                                                                                     
        Duplex: Unknown! (255)                                                                                                              
        Auto-negotiation: on                                                                                                                
        Port: FIBRE                                                                                                                         
        PHYAD: 0                                                                                                                            
        Transceiver: internal                                                                                                               
        Current message level: 0x000000ff (255)                                                                                             
                               drv probe link timer ifdown ifup rx_err tx_err                                                               
        Link detected: no

maybe i need a quirk to extend this wait-time? in this case this is my sfp VENDOR/PN

[    3.447503] sfp sfp1: module CISCO-FINISAR    FTLX8571D3BCL-C2 rev A    sn S2209167650      dc 220916

maybe someone is able to decode sfps eeprom i got via

ethtool -m eth2 raw on > hifibre10gsfp.bin

hifibre10gsfp.bin (512 Bytes)

root@bpi-r4-v11:~# hexdump -C hifibre10gsfp.bin                                                                                             
00000000  03 04 07 10 00 00 00 00  00 00 00 06 67 00 00 00  |............g...|                                                              
00000010  08 02 00 1e 43 49 53 43  4f 2d 46 49 4e 49 53 41  |....CISCO-FINISA|                                                              
00000020  52 20 20 20 00 00 90 65  46 54 4c 58 38 35 37 31  |R   ...eFTLX8571|                                                              
00000030  44 33 42 43 4c 2d 43 32  41 20 20 20 03 52 00 a5  |D3BCL-C2A   .R..|                                                              
00000040  00 1a 00 00 53 32 32 30  39 31 36 37 36 35 30 20  |....S2209167650 |                                                              
00000050  20 20 20 20 32 32 30 39  31 36 20 20 68 80 03 72  |    220916  h..r|                                                              
00000060  00 00 11 0e da 88 47 c6  c3 4f ee b2 d6 1b 47 06  |......G..O....G.|                                                              
00000070  ae 0d 17 00 00 00 00 00  00 00 00 00 4e 10 e9 bf  |............N...|                                                              
00000080  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|                                                              
*                                                                                                                                           
000000f0  ff ff ff ff ff ff ff ff  00 00 00 00 00 00 00 00  |................|                                                              
00000100  50 00 f6 00 4b 00 fb 00  8c a0 75 30 88 b8 79 18  |P...K.....u0..y.|                                                              
00000110  1d 4c 00 00 17 70 03 e8  31 2d 04 97 27 10 05 c7  |.L...p..1-..'...|                                                              
00000120  31 2d 02 86 27 10 03 2d  00 00 00 00 00 00 00 00  |1-..'..-........|                                                              
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|                                                              
00000140  00 00 00 00 3f 80 00 00  00 00 00 00 01 00 00 00  |....?...........|                                                              
00000150  01 00 00 00 01 00 00 00  01 00 00 00 00 00 00 15  |................|                                                              
00000160  19 6a 82 eb 00 00 00 00  00 00 1c 00 00 00 82 00  |.j..............|                                                              
00000170  01 40 00 00 05 40 00 00  00 00 00 00 08 11 55 00  |.@[email protected].|                                                              
00000180  43 4f 55 49 41 38 4e 43  41 41 31 30 2d 32 34 31  |COUIA8NCAA10-241|                                                              
00000190  35 2d 30 33 56 30 33 20  01 00 46 00 00 00 00 c6  |5-03V03 ..F.....|                                                              
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|                                                              
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 aa aa  |................|                                                              
000001c0  53 46 50 2d 31 30 47 2d  53 52 20 20 20 20 20 20  |SFP-10G-SR      |                                                              
000001d0  20 20 20 20 32 33 00 00  00 00 00 00 00 00 00 35  |    23.........5|                                                              
000001e0  15 1a 20 24 2a 30 20 30  00 00 00 00 00 00 00 00  |.. $*0 0........|                                                              
000001f0  00 00 00 00 00 1d 00 00  ff ff ff ff 00 00 00 00  |................|                                                              
00000200

I get messages like these on R3 also. It may be normal, as debugging info only.

thx for confirming, yes i think too they can be ignored as they do not represent the mode i want to use (10Gbase-r/usxgmii)…but strange that link_up is reported, but ethtool does not show the link

Then it is the Mac/PCs not reporting link up…

Check how I fix the autoneg issue now for 2500base-x.

Maybe 10Gbase-r/usxgmii needs similar fix?

As far as i see the patch is independ of the mac/pcs and in phylink core…maybe i need to change this line to include 10gbase-r right?

if (interface != PHY_INTERFACE_MODE_2500BASEX)
		return false;

To something like this:

if (interface != PHY_INTERFACE_MODE_2500BASEX && interface != PHY_INTERFACE_MODE_10GBASER)
		return false;

Or can i force autoneg off (but afair i tried and got a traceback)

of_device_is_compatible(node, "mediatek,eth-mac")

So it only works on the mediatek mac…

Yes saw this…and because of this it will not be upstreamable :stuck_out_tongue:

Is my assumption right with the change for 10g? Any additional way to trace where the problem is located?

Yep, maybe use pcs->ops->supports_inband(interface) or something.

Check state->link from phylink_mac_pcs_get_state() I guess…

1 Like

thanks eric very much for this info…noticed the pcs driver (CONFIG_PCS_MTK_USXGMII) was not enabled (though i had looked for it already, but seems not)…

root@bpi-r4-v11:~# ip link set eth2 up                                                                                                                                                  
[   20.646238] mtk_soc_eth 15100000.ethernet eth2: configuring for inband/10gbase-r link mode                                                                                           
root@bpi-r4-v11:~# [   20.751857] mtk_soc_eth 15100000.ethernet eth2: Link is Up - 10Gbps/Full - flow control off                                                                       
                                                                                                                                                                                        
root@bpi-r4-v11:~# ethtool eth2                                                                                                                                                         
Settings for eth2:                                                                                                                                                                      
        Supported ports: [ FIBRE ]                                                                                                                                                      
        Supported link modes:   10000baseSR/Full                                                                                                                                        
        Supported pause frame use: Symmetric Receive-only                                                                                                                               
        Supports auto-negotiation: Yes                                                                                                                                                  
        Supported FEC modes: Not reported                                                                                                                                               
        Advertised link modes:  10000baseSR/Full                                                                                                                                        
        Advertised pause frame use: Symmetric Receive-only                                                                                                                              
        Advertised auto-negotiation: Yes                                                                                                                                                
        Advertised FEC modes: Not reported                                                                                                                                              
        Speed: 10000Mb/s                                                                                                                                                                
        Duplex: Full                                                                                                                                                                    
        Auto-negotiation: on                                                                                                                                                            
        Port: FIBRE                                                                                                                                                                     
        PHYAD: 0                                                                                                                                                                        
        Transceiver: internal                                                                                                                                                           
        Current message level: 0x000000ff (255)                                                                                                                                         
                               drv probe link timer ifdown ifup rx_err tx_err                                                                                                           
        Link detected: yes

it seems i fall into this (which not nice without error-message btw):

will make some basic tests with my other r4 and then continue work on RSS/LRO

basicly it works this is speedtest without RSS/LRO:

root@bpi-r4-v11:~# iperf3 -bidir -c 192.168.90.10                                                                                                                                       
Connecting to host 192.168.90.10, port 5201                                                                                                                                             
[  5] local 192.168.90.11 port 43602 connected to 192.168.90.10 port 5201                                                                                                               
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd                                                                                                                        
[  5]   0.00-1.00   sec   541 MBytes  4.54 Gbits/sec   38    706 KBytes                                                                                                                 
[  5]   1.00-2.00   sec   539 MBytes  4.52 Gbits/sec   11    936 KBytes                                                                                                                 
[  5]   2.00-3.00   sec   541 MBytes  4.54 Gbits/sec   53    867 KBytes                                                                                                                 
[  5]   3.00-4.00   sec   541 MBytes  4.54 Gbits/sec   30   1015 KBytes                                                                                                                 
[  5]   4.00-5.00   sec   541 MBytes  4.54 Gbits/sec   43    943 KBytes                                                                                                                 
[  5]   5.00-6.00   sec   540 MBytes  4.53 Gbits/sec   33    813 KBytes                                                                                                                 
[  5]   6.00-7.00   sec   542 MBytes  4.55 Gbits/sec   36    731 KBytes                                                                                                                 
[  5]   7.00-8.00   sec   540 MBytes  4.53 Gbits/sec   28    977 KBytes                                                                                                                 
[  5]   8.00-9.00   sec   540 MBytes  4.53 Gbits/sec   45    865 KBytes                                                                                                                 
[  5]   9.00-10.00  sec   540 MBytes  4.53 Gbits/sec   66    764 KBytes                                                                                                                 
- - - - - - - - - - - - - - - - - - - - - - - - -                                                                                                                                       
[ ID] Interval           Transfer     Bitrate         Retr                                                                                                                              
[  5]   0.00-10.00  sec  5.28 GBytes  4.54 Gbits/sec  383             sender                                                                                                            
[  5]   0.00-10.00  sec  5.28 GBytes  4.53 Gbits/sec                  receiver                                                                                                          
                                                                                                                                                                                        
iperf Done.

i guess retransmitts are caused by cpu overload due to missing rss/lro

with RSS+LRO patches link takes ~1min to get up and after it is up throughput is same (but not enabled with ethtool yet) - so not working yet

Thanks for sharing, I also needed that for my defconfig.

I can’t start MIKROTIK S+RJ10. It seems to me that he is not receiving power.

Can I force power on SFP1 SFP2 is it controlled by GPIO82-83?

root@OpenWrt:/sys/devices/platform/1001f000.pinctrl/gpio/gpiochip428/power# cat /sys/kernel/debug/gpio
gpiochip0: GPIOs 428-511, parent: platform/1001f000.pinctrl, pinctrl_moore:
 gpio-428 (                    |tx-disable          ) in  lo
 gpio-430 (                    |los                 ) in  hi IRQ
 gpio-432 (                    |asm_sel             ) in  hi
 gpio-433 (                    |pca9545_rst         ) in  hi
 gpio-441 (                    |reset               ) in  lo IRQ ACTIVE LOW
 gpio-442 (                    |wps                 ) in  hi IRQ ACTIVE LOW
 gpio-482 (                    |los                 ) in  hi IRQ
 gpio-491 (                    |bpi-r4:pio:blue     ) out lo
 gpio-498 (                    |tx-disable          ) in  lo
 gpio-507 (                    |bpi-r4:pio:green    ) out lo
 gpio-510 (                    |mod-def0            ) in  hi IRQ ACTIVE LOW
 gpio-511 (                    |mod-def0            ) in  hi IRQ ACTIVE LOW

Here I see that both GPIOs (mod-def0) are occupied for control. Is it possible to free them?

These are bound by dts,but afaik they are needed by driver. You can assign different gpio (e.g. from 26pin header) to the moddef0.

Are you sure the moddef-pins are used for powering your sfp? The 3v3 is fixed passed to the sfp.

Can you read the i2c of the sfp? What do you see with ethtool? I guess you need a quirk (e.g. for disabling the tx-disable or adding more modes) instead of changing the moddef0 gpio.

What says dmesg? Can you read sfp data with ethtool -m?

Nothing happens when I install the module.

According to the diagram, power appears on the module after the mod-def0 signal changes to LO. The module needs power to switch this signal.

Above you can see its installed state and it is HI.

I have to run this module every time, bypassing Q12. I need a power boost on the module for it to start. After this, the module changes mod-def0 to LO and the BPI circuit continues to power it.

root@OpenWrt:~# cat /sys/kernel/debug/gpio
gpiochip0: GPIOs 428-511, parent: platform/1001f000.pinctrl, pinctrl_moore:
 gpio-428 (                    |tx-disable          ) out lo
 gpio-430 (                    |los                 ) in  lo IRQ
 gpio-432 (                    |asm_sel             ) in  hi
 gpio-433 (                    |pca9545_rst         ) in  hi
 gpio-441 (                    |reset               ) in  lo IRQ ACTIVE LOW
 gpio-442 (                    |wps                 ) in  hi IRQ ACTIVE LOW
 gpio-482 (                    |los                 ) in  hi IRQ
 gpio-491 (                    |bpi-r4:pio:blue     ) out lo
 gpio-498 (                    |tx-disable          ) out lo
 gpio-507 (                    |bpi-r4:pio:green    ) out lo
 gpio-510 (                    |mod-def0            ) in  lo IRQ ACTIVE LOW
 gpio-511 (                    |mod-def0            ) in  lo IRQ ACTIVE LOW

A vicious circle, the module cannot change the mod-def0 signal from HI to LO because there is no power. BPI-R4 cannot supply power because mod-def0 HI.

[   86.268015] sfp sfp@1: SM: enter empty:up:down event insert
[   86.273596] sfp sfp@1: SM: exit probe:up:down
[   86.603175] sfp sfp@1: SM: enter probe:up:down event timeout
[   86.619969] sfp sfp@1: module MikroTik         S+RJ10           rev 2.16 sn XXXXXXXX     dc 210425
[   86.629444] sfp sfp@1: sfp: support mode 00,00000400,00006040
[   86.635194] mtk_soc_eth 15100000.ethernet eth1: requesting link mode inband/10gbase-kr with support 00,00000400,000060c0
[   86.659622] sfp sfp@1: Module switched to 1.5W power level
[   86.665102] sfp sfp@1: SM: exit waitpwr:up:down
[   86.987178] sfp sfp@1: SM: enter waitpwr:up:down event timeout
[   86.993006] sfp sfp@1: tx disable 1 -> 0
[   86.996943] sfp sfp@1: SM: exit present:up:wait
[   87.051174] sfp sfp@1: SM: enter present:up:wait event timeout
[   87.056998] sfp sfp@1: probing phy device through the [MDIO_I2C_NONE] protocol
[   87.064215] sfp sfp@1: SM: exit present:up:link_up
[   87.095191] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 10Gbps/Full - flow control off
[   87.114504] br-lan: port 4(eth1) entered blocking state
[   87.119721] br-lan: port 4(eth1) entered forwarding state
[   91.771718] sfp sfp@0: SM: enter empty:up:down event insert
[   91.777297] sfp sfp@0: SM: exit probe:up:down
[   92.107173] sfp sfp@0: SM: enter probe:up:down event timeout
[   92.123970] sfp sfp@0: module MikroTik         S+RJ10           rev 2.16 sn XXXXXXXXX     dc 210319
[   92.133446] sfp sfp@0: sfp: support mode 00,00000400,00006040
[   92.139195] mtk_soc_eth 15100000.ethernet eth2: requesting link mode inband/10gbase-kr with support 00,00000400,000060c0
[   92.163627] sfp sfp@0: Module switched to 1.5W power level
[   92.169105] sfp sfp@0: SM: exit waitpwr:up:down
[   92.491175] sfp sfp@0: SM: enter waitpwr:up:down event timeout
[   92.496998] sfp sfp@0: tx disable 1 -> 0
[   92.500929] sfp sfp@0: SM: exit present:up:wait
[   92.555173] sfp sfp@0: SM: enter present:up:wait event timeout
[   92.560996] sfp sfp@0: probing phy device through the [MDIO_I2C_NONE] protocol
[   92.568211] sfp sfp@0: SM: exit present:up:link_up
[   92.599187] mtk_soc_eth 15100000.ethernet eth2: Link is Up - 10Gbps/Full - flow control off
[   92.618504] br-wan: port 2(eth2) entered blocking state
[   92.623721] br-wan: port 2(eth2) entered forwarding state
[  117.266056] TCP: br-lan: Driver has suspect GRO implementation, TCP performance may be compromised.