BPI-R3 SFP Module compatibility

holger.hees · January 30, 2023, 9:07pm

Vielen Dank!

I installed a fresh version of openwrt (master), this time together with kernel modules (sfp and mtk_soc_eth) with debug symbols… I hope it will give me some more insights.

frank-w · January 30, 2023, 10:34pm

Debug symbols allowing running debugger (gdb) in linux…i’m not sure this will help you here.

holger.hees · January 31, 2023, 7:48am

ok. My understanding was that a kernel without debug option compiled, will strip out all debug related stuff lik “dev_*dbg/pr_debug()”.

I will try out “/sys/kernel/debug/dynamic_debug/control” and check if this will show me some more messages.

just another idea regarding the topic “BRI-R3 SFP Module compatibility”. Maybe it will be helpful to have a page with a “compatibility matrix”. Currently you have to read the full topic to get an idea which modules are supported and which not. And you have to read also unrelated messages like my messages.

Sandokan · January 31, 2023, 11:06am

Hi! Any chance to solve this? I’m not found any way to make this module working…

It is a very strange behaviour cause the module is declared full compatible with Cisco GLC-T/SFP-GE-T, Meraki MA-SFP-1GB-TX, Mikrotik, Ubiquiti UniFi UF-RJ45-1G, Intel, Netgear… My question is if anyone have a 1000base-T RJ45 working with R3. Thanks

frank-w · January 31, 2023, 3:08pm

I can add such matrix in my repo,but i need to know which modules are tested and working not working, maybe detailed in “recognized” and “really working”. I guess we need a way for find out the correct module informations like “how to find on a shop” and some kind of technical part number for identification. Maybe some kind of class like copper/fibre/ont/1g/2g(only)

something like this:

https://www.fw-web.de/dokuwiki/doku.php?id=en:bpi-r3:start#network_connections

have now only added mine, maybe i can add a column for comments (working after patch xyz or something) or maybe ethtool output. For speed i can use the 1000baseX/T names

prdtabim · January 31, 2023, 11:13pm

I need some help. This XPON sfp adapter ( DFP-34X-2C2 ) support 2500Base-x

 sfp sfp1: module ODI              DFP-34X-2C2      rev      sn XPON22XXXXXX     dc 221206  
mtk_soc_eth 15100000.ethernet eth1: switched to inband/1000base-x link mode
mtk_soc_eth 15100000.ethernet eth1: configuring for inband/1000base-x link mode

I have tried LAN_SDS_MODE ( 0,4,5,6 ) followed by reboot and the result is always the same.

Anyone have a clue ? what can be changed to put the module in 2500Base-x ?

 ethtool -m eth1
        Identifier                                : 0x03 (SFP)
        Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
        Connector                                 : 0x01 (SC)
        Transceiver codes                         : 0x00 0x00 0x00 0x02 0x22 0x00 0x01 0x00 0x00
        Transceiver type                          : Ethernet: 1000BASE-LX
        Transceiver type                          : FC: intermediate distance (I)
        Transceiver type                          : FC: Longwave laser (LC)
        Transceiver type                          : FC: Single Mode (SM)
        Encoding                                  : 0x01 (8B/10B)
        BR, Nominal                               : 1300MBd
        Rate identifier                           : 0x00 (unspecified)
        Length (SMF,km)                           : 20km
        Length (SMF)                              : 20000m
        Length (50um)                             : 0m
        Length (62.5um)                           : 0m
        Length (Copper)                           : 0m
        Length (OM3)                              : 0m
        Laser wavelength                          : 1310nm
        Vendor name                               : ODI
        Vendor OUI                                : 00:00:00
        Vendor PN                                 : DFP-34X-2C2
        Vendor rev                                : 
        Option values                             : 0x00 0x1a
        Option                                    : RX_LOS implemented
        Option                                    : TX_FAULT implemented
        Option                                    : TX_DISABLE implemented
        BR margin, max                            : 0%
        BR margin, min                            : 0%
        Vendor SN                                 : XPON22XXXXXX
        Date code                                 : 221206

frank-w · February 1, 2023, 7:04am

Which slot do you use and which image/kernel do you use? My kernel has reworked phylink support. I’m not sure this is already merged to recent openwrt snapshots.

Can you provide ethtool for parent interface too? Maybe we need to add 2500baseX to adverticed modes.

prdtabim · February 1, 2023, 11:18am

Hi, Thanks for your help. I tried a dozen or more snapshots in the last 3 weeks. Now R3 is using this release.

  OpenWrt SNAPSHOT r21962-d649a8dcef

I’m testing recently in wan port ( left port = eth1 ). Already tested some weeks ago in lan port ( right port = sfp2 ) with the same results. If the port is empty ethtool shows 2500Base-x as available. The moment that plugged the xpon module it shows that message “mtk_soc_eth 15100000.ethernet eth1: switched to inband/1000base-x link mode” and doesn’t revert back to 2500base-x in any way. Unplugging the xpon module and after plugging a 2.5gbase-t module ( that works ok if plugged directly ) doesn’t solve this.

By parent interface do you are refering to what ? at night I can test and post the ethtool results.

frank-w · February 1, 2023, 2:06pm

I don’t know if openwrt use the phylink-changes currently mainline. Have no 2g5 sfp at the moment…only 1g fibre.

I meant eth1/lan4,you had posted only ethtool of the module itself.but the parent itself maybe should show both (maybe more) modes…eth1 itself can be 1g and 2g5,but sfp limits this to modes it supports

For 2g5 mode we have to look if speed needs to be set too…here for sfp1/eth1 sfp slot.

github.com

frank-w/BPI-Router-Linux/blob/6.1-main/drivers/net/ethernet/mediatek/mtk_sgmii.c#L139


	struct mtk_pcs *mpcs = pcs_to_mtk_pcs(pcs);
	unsigned int sgm_mode;


	if (!phylink_autoneg_inband(mode)) {
		/* Force the speed and duplex setting */
		if (speed == SPEED_10)
			sgm_mode = SGMII_SPEED_10;
		else if (speed == SPEED_100)
			sgm_mode = SGMII_SPEED_100;
		else
			sgm_mode = SGMII_SPEED_1000;


		if (duplex == DUPLEX_FULL)
			sgm_mode |= SGMII_DUPLEX_FULL;


		regmap_update_bits(mpcs->regmap, SGMSYS_SGMII_MODE,
				   SGMII_DUPLEX_FULL | SGMII_SPEED_MASK,
				   sgm_mode);
	}
}

Or maybe this is wrong beginning here:

github.com

frank-w/BPI-Router-Linux/blob/6.1-main/drivers/net/ethernet/mediatek/mtk_sgmii.c#L46


static int mtk_pcs_config(struct phylink_pcs *pcs, unsigned int mode,
			  phy_interface_t interface,
			  const unsigned long *advertising,
			  bool permit_pause_to_mac)
{
	struct mtk_pcs *mpcs = pcs_to_mtk_pcs(pcs);
	unsigned int rgc3, sgm_mode, bmcr;
	int advertise, link_timer;
	bool changed, use_an;


	if (interface == PHY_INTERFACE_MODE_2500BASEX)
		rgc3 = RG_PHY_SPEED_3_125G;
	else
		rgc3 = 0;


	advertise = phylink_mii_c22_pcs_encode_advertisement(interface,
							     advertising);
	if (advertise < 0)
		return advertise;


	link_timer = phylink_get_link_timer_ns(interface);

At least this looks strange to me:

if (interface == PHY_INTERFACE_MODE_2500BASEX)
		rgc3 = RG_PHY_SPEED_3_125G;
	else
		rgc3 = 0;
...
regmap_update_bits(mpcs->regmap, mpcs->ana_rgc3,
			   RG_PHY_SPEED_3_125G, rgc3);

Either RG_PHY_SPEED_3_125G is a genmask or a value,but using it as both in the regmap looks strange

Maybe you can add printks on these positions to figure out which mode is requested and if options are set right

prdtabim · February 1, 2023, 9:45pm

eth1 after a reboot of bpi-r3.

ethtool eth1
Settings for eth1:
        Supported ports: [ MII ]
        Supported link modes:   2500baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  2500baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Auto-negotiation: on
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: no

After plugging the XPON SFP

at logread
Wed Feb  1 18:25:05 2023 kern.info kernel: [  151.432958] sfp sfp1: module ODI             DFP-34X-2C2      rev      sn XPON22120052     dc 221206
Wed Feb  1 18:25:05 2023 kern.info kernel: [  151.442363] mtk_soc_eth 15100000.ethernet eth1: switched to inband/1000base-x link mode

ethtool eth1
Settings for eth1:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  1000baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Auto-negotiation: on
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes

frank-w · February 2, 2023, 10:31am

Thx,so basicly eth1 is initially in 2500baseX/Full mode and change when sfp is inserted. You can try to force mode to 2500baseX in code i posted above

frank-w · February 4, 2023, 12:10pm

@prdtabim

can you try if this series from @dangowrt fixes your 2g5-problem?

https://patchwork.kernel.org/project/linux-mediatek/cover/[email protected]/

?

see comment in coverletter as it requires additional patch to 6.2 for apply

prdtabim · February 4, 2023, 2:48pm

Hi

Thanks for the tips. I will try this weekend.

frank-w · February 4, 2023, 6:49pm

note series is based on net-next (patch 8 will fail, so apply it with reject and make changes manually)

have applied series here, but have no 2g5 sfp to test: https://github.com/frank-w/BPI-Router-Linux/tree/6.2-netnext-pcs

dangowrt · February 8, 2023, 11:12pm

Updated series is here https://patchwork.kernel.org/project/linux-mediatek/cover/[email protected]/

ericwoud · February 28, 2023, 2:54pm

This one has an option to order for banana pi

€ 14,24 59%OFF | SFP-2.5GE-RJ45 Gigabit 2.5G Module SFP To RJ45 Optical Interface Expansion 2500M Rate Compatible Huawei H3C Cisco TPLINK Switch https://a.aliexpress.com/_EyiBS4l

Anyone have any experience?

Sandokan · March 1, 2023, 8:45pm

Just tested a couple of Huawei MA5671A GPON sticks with standard rooted firmware and Carlito firmware. Both seems to works at 2.5Gbps. Some reeboot attempts are needed to be recognized by R3. I get EEPROM -6 reading error (the EEPROM is simulated on MA5671). Maybe due to the fact that this device is slow to start (fyi it uses an old OpenWRT version). On my TP-LInk MC220L media converter the sticks works without issues. Probably a longer timeout somewhere on R3 startup procedure may be useful…

frank-w · March 11, 2023, 1:08pm

I got the 2g5 sfps i ordered from erics link above.

They work well with v12 of daniels patches on 6.3. v13 does not work as he needs to drop a patch (or 2) for 2500base-X support. This needs to be done via sfp quirks. It is a software issue we try to solve…hardware works well…i get ~ 2.4 gbit/s with iperf3

Here is a new sale for 13.22€ and without shipping costs if interested:

https://m.de.aliexpress.com/item/1005005052139284.html

Sandokan · March 11, 2023, 6:59pm

Ordered a pair. I hope these will be made compatible soon with OpenWRT.

frank-w · March 11, 2023, 7:04pm

I guess daniel will add the rejected patches to openwrt till a better solution is available