I have a problem with the 10G LAN port of the BPI-R4 Pro 8X (lan-phy variant, i.e. the AS21010 RJ45 in front of MxL86252C port 13) and I would like to know if someone else see the same.
Frames received on this port are lost depending of the frame length modulo 16 (FCS included). Lengths of 10 and 14 mod 16 are fine. With 2 mod 16 it loses a bit, with 4, 5 and 6 mod 16 TCP collapses. It happens only when frames arrive back to back (UDP with gaps between packets is clean). No counter moves on the MxL (RMON of port 13, no FCS errors, no discards) or in the AS21010 debug counters.
The other end is a BPI-R4 with a SFP-10G-T module (BCM84891), 10GBASE-T, MTU 9000. Both run frank-w 7.3-main with some patches, AS21010 firmware 1.9.2 and MxL firmware 1.0.85. It is easy to reproduce from the R4 Pro with iperf3 in reverse mode, so the R4 sends:
iperf3 -c <R4> -R -M 1460
iperf3 -c <R4> -R -M 1452
The first one gives 1518 byte frames and ~3 Gbps with 3 retransmits. The second gives 1510 byte frames and ~310 Mbps with ~1000 retransmits in 8 s. With jumbo frames it is the same: MSS 8892 does 3-4 Gbps with 200 retransmits, MSS 8900 does 9.8 Gbps with 0-5. The other direction (R4 Pro sends) is clean at every size.
What I tried: the same cable and module on the WAN RJ45 of the R4 Pro (same AS21010 but connected directly to the SoC MAC) gives no loss at all. Port 13 in 10GBASE-R instead of USXGMII, so no rate adaptation in the AS21010, gives the same loss. So I think the problem is between the AS21010 and the port 13 receiver of the MxL, which are both firmware I can’t change.
Has anyone seen this on the lan-sfp variant (SFP+ module on port 13 instead of the AS21010)? Is there a MxL862xx firmware newer than 1.0.85? For now I clamp the MSS so the traffic through my tunnels lands on a good length, however this does not help UDP.