BPI-R4: 100Mbit broken

The problem: i am facing another problem thats a bit wierd. i have a “old” voip base station that uses 100mbit rj45. but somehow it does not get an ip address. the request does not even go into the router. i have all four rj45 ports configured as br-lan and all work (tested with my laptop), but somehow… when i try to connect that base station, its somehow incompatible. anyone an idea why is that? the only difference is this 100mbit thingen, could this be the problem?

thats all i can see when i unplug and plugin the power supply of that device:

Fri Jan 12 23:04:02 2024 daemon.notice netifd: Network device 'lan2' link is down
Fri Jan 12 23:04:02 2024 kern.info kernel: [ 3921.471764] mt7530-mmio 15020000.switch lan2: Link is Down
Fri Jan 12 23:04:02 2024 kern.info kernel: [ 3921.477779] br-lan: port 3(lan2) entered disabled state
Fri Jan 12 23:04:07 2024 kern.info kernel: [ 3926.893551] mt7530-mmio 15020000.switch lan2: Link is Up - 100Mbps/Full - flow control rx/tx
Fri Jan 12 23:04:07 2024 kern.info kernel: [ 3926.902442] br-lan: port 3(lan2) entered blocking state
Fri Jan 12 23:04:07 2024 kern.info kernel: [ 3926.907681] br-lan: port 3(lan2) entered forwarding state
Fri Jan 12 23:04:07 2024 daemon.notice netifd: Network device 'lan2' link is up
Fri Jan 12 23:04:22 2024 kern.info kernel: [ 3941.363112] mt7530-mmio 15020000.switch lan2: Link is Down
Fri Jan 12 23:04:22 2024 kern.info kernel: [ 3941.369108] br-lan: port 3(lan2) entered disabled state
Fri Jan 12 23:04:22 2024 daemon.notice netifd: Network device 'lan2' link is down
Fri Jan 12 23:04:24 2024 kern.info kernel: [ 3942.948551] mt7530-mmio 15020000.switch lan2: Link is Up - 100Mbps/Full - flow control rx/tx
Fri Jan 12 23:04:24 2024 kern.info kernel: [ 3942.957397] br-lan: port 3(lan2) entered blocking state
Fri Jan 12 23:04:24 2024 kern.info kernel: [ 3942.962615] br-lan: port 3(lan2) entered forwarding state
Fri Jan 12 23:04:24 2024 daemon.notice netifd: Network device 'lan2' link is up

When i use a netgear GS108T between the voip station and the r4, it works. I used lan2 on the r4 for the switch. For testing i used: ethtool -s lan2 autoneg off speed 100 duplex full - now i have the same behaviour - no IP on the base station. Same with

ethtool -s lan2 autoneg on (if switched off before)
ethtool -s lan2 advertise 0x008
Settings for lan2:
	Supported ports: [ TP MII ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  100baseT/Full 
	Advertised pause frame use: No
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  10baseT/Half 10baseT/Full 
	                                     100baseT/Half 100baseT/Full 
	Link partner advertised pause frame use: No
	Link partner advertised auto-negotiation: Yes
	Link partner advertised FEC modes: Not reported
	Speed: 100Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 2
	Transceiver: external
	Auto-negotiation: on
	MDI-X: Unknown
	Supports Wake-on: d
	Wake-on: d
	Link detected: yes

@ericwoud so we can continue here. Sorry, thought the old thread is the “main topic for anything related to r4”. Can we move all those messages from the old thread to this one or just delete them? So it does not flood the other thread? And if i can help in any way with that problem (like testing or gathering information), just let me know, would love to help.

You could just remove the messages as all info is here. It will be deleted after 24 hours.

1 Like

I’ll check on my latest image, but that will be somewhere next week unfortunately.

The problem is when everything about a board is in one thread it is hard to find a specific problem later (if someone hits it too) and steps to reproduce and possible fix as different questions mixing at some time :slight_smile:

so better make separate threads for such things (specific issues) and leave the plain board discussion (is it supporting x or y, release dates,new version etc.) in the vendor thread.

So now if some one have an issue with 100mbit/s mode on r4, he will find this thread and only related answers (except mine here :p)

Back to topic:

Outut of ethtool basicly looks good:

Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 

Tells you your local interface supprts these modes and

Link partner advertised link modes:  10baseT/Half 10baseT/Full 
	                                     100baseT/Half 100baseT/Full 

Is what the other side (your voip box) supports…these modes are intersected and the highest mode is taken

Advertised link modes:  100baseT/Full 

This is applied to interface

	Speed: 100Mb/s
	Duplex: Full
	....
	Link detected: yes

And link is basicly recognized…maybe connection runs out of sync or there are crc errors at ethernet or ip layer so that link is dropped on one side…

You can look into the interfaces statistics with

ip -s link show dev lan2

or

ethtool -S lan2

ip -s does not work, sure it is -s?

root@OpenWrt:~# ip -s link show dev lan2
BusyBox v1.36.1 (2024-01-12 12:45:39 UTC) multi-call binary.

Usage: ip [OPTIONS] address|route|link|neigh|rule [ARGS]

OPTIONS := -f[amily] inet|inet6|link | -o[neline]

ip addr add|del IFADDR dev IFACE | show|flush [dev IFACE] [to PREFIX]
ip route list|flush|add|del|change|append|replace|test ROUTE
ip link set IFACE [up|down] [arp on|off] [multicast on|off]
	[promisc on|off] [mtu NUM] [name NAME] [qlen NUM] [address MAC]
	[master IFACE | nomaster] [netns PID]
ip neigh show|flush [to PREFIX] [dev DEV] [nud STATE]
ip rule [list] | add|del SELECTOR ACTION
root@OpenWrt:~# ip link show dev lan2
7: lan2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether ae:22:45:c4:db:74 brd ff:ff:ff:ff:ff:ff
NIC statistics:
     tx_packets: 60847
     tx_bytes: 24900469
     rx_packets: 2072
     rx_bytes: 340558
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 1550856
     TxMulticast: 54219
     TxBroadcast: 7873
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 9792
     TxPktSz65To127: 1525386
     TxPktSz128To255: 28694
     TxPktSz256To511: 7230
     TxPktSz512To1023: 36706
     Tx1024ToMax: 5140
     TxBytes: 148262075
     RxDrop: 0
     RxFiltering: 0
     RxUnicast: 1750
     RxMulticast: 301
     RxBroadcast: 21
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 1296
     RxPktSz65To127: 111
     RxPktSz128To255: 6
     RxPktSz256To511: 574
     RxPktSz512To1023: 73
     RxPktSz1024ToMax: 12
     RxBytes: 377854
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0

Openwrt uses busybox to emulate iproute2 command and possible does not support all flags from it,but you got some information…and it looks good so far. See no cause why should be dropped from this side…can you see interface stats from the other side (voip box)? You do the test without the switch in between?

no, i have no connection to that base without the switch. the base also has no wifi connection, so there is no way i can think of to gather more information. what i could do is use the switch, try to get some information, only change the cable from base->switch to base->r4, wait a few seconds, then back to base->switch and see whats the difference. but guess that wont help much.

You need this informations in error case else it will not help to nail down the root cause :slight_smile:

Commands look at ethernet layer,so we do not need a connection on higher layer…we want to know why the link is established and then dropped from any side

its not dropped, it is never established. the drops happen when i unplug the cable. that was to show that there is “nothing else” happening. i have dhcp on verbose, i see every request (e.g. from my mobile phone or my workstation), for testing purposes to see if the device even gets that far. it does not.

unfortunately, i guess the base station is a very bad device for such testing. and i dont have any other device with 100mbit :frowning:

i gave away my raspberry pi 3b a few weeks ago. i guess that would be a good candidate for testing.

Any device will do, cause you can set the speed to 100MB with the advertisement in 1 side.

Maybe first reset the statistics and then plug in.

oh that sounds perfect. i have a laptop, i will test it with that

Your log above says that links gets up and after 15seconds down…i guess you talk about dhcp/ip layer…i’m still on ethernet :slight_smile:

So posssibly link came up but have no (valid) traffic (errors,no keepalive/dhcp frames). But this is not advertising error, it is handling this advertised mode in ethernet driver.

yay, i have now my laptop connected to wifi. with that i am able to connect via ssh from my workstation to it. and the rj45-usb-c-adapter is connected. i have the same problem now on the laptop. so if anyone could tell me what i should check, i can provide that output

oli@fedora:~$ ethtool -S enp3s0f3u1
NIC statistics:
     tx_packets: 79
     rx_packets: 0
     tx_errors: 0
     rx_errors: 0
     rx_missed: 0
     align_errors: 0
     tx_single_collisions: 0
     tx_multi_collisions: 0
     rx_unicast: 0
     rx_broadcast: 0
     rx_multicast: 0
     tx_aborted: 0
     tx_underrun: 0

i forgot to clean stats on the r4 but i guess thats not a problem, because there are no errors

root@OpenWrt:~# ethtool -S lan2
NIC statistics:
     tx_packets: 66108
     tx_bytes: 27059435
     rx_packets: 2072
     rx_bytes: 340558
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 1617668
     TxMulticast: 58992
     TxBroadcast: 8668
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 10761
     TxPktSz65To127: 1588679
     TxPktSz128To255: 31703
     TxPktSz256To511: 7954
     TxPktSz512To1023: 39874
     Tx1024ToMax: 6357
     TxBytes: 159209767
     RxDrop: 0
     RxFiltering: 0
     RxUnicast: 1750
     RxMulticast: 301
     RxBroadcast: 21
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 1296
     RxPktSz65To127: 111
     RxPktSz128To255: 6
     RxPktSz256To511: 574
     RxPktSz512To1023: 73
     RxPktSz1024ToMax: 12
     RxBytes: 377854
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0

Look in statistics from both sides of this broken connection…if lan on r4 is clean it is possible that gmac (eth0) cannot handle 100mbit…afaik the connection between switchports and the ethernet controller (mac) is also ethernet (inside sgmii) with vlan-similar tagging

First output shows tx,but no rx (so also no errors) on your laptop…btw. wonder why ethtool talks about “packets” as it is on ethernet layer where it is called frames…not noticed before :stuck_out_tongue:

Could you bring down the interface to reset statistics on r4 before? You still see traffic geberated over working connection before…same for eth0

If ip link set eth0/lan2 down/up does not clear the stats you should reboot to do it

this is now after a reboot of the router:

root@OpenWrt:~# ethtool -S lan2
NIC statistics:
     tx_packets: 192
     tx_bytes: 88532
     rx_packets: 57
     rx_bytes: 7956
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 199
     TxMulticast: 213
     TxBroadcast: 83
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 37
     TxPktSz65To127: 226
     TxPktSz128To255: 71
     TxPktSz256To511: 40
     TxPktSz512To1023: 94
     Tx1024ToMax: 27
     TxBytes: 148584
     RxDrop: 0
     RxFiltering: 0
     RxUnicast: 4
     RxMulticast: 44
     RxBroadcast: 9
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 7
     RxPktSz65To127: 16
     RxPktSz128To255: 29
     RxPktSz256To511: 5
     RxPktSz512To1023: 0
     RxPktSz1024ToMax: 0
     RxBytes: 8982
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0

crap, forgot to do ethtool to set it to 100mbit. can i somehow configure this in /etc/config/network? or do i have to use some /etc/init.d/rc.local? because otherwise when i reboot, its automatically on auto negotiation and that works with 1000, then the stats are wrong.

Does packets increase in both directions when you try to do something on the laptop generating traffic?

So for switch it looks good…that seems to confirm that issue needs to be searched in mac driver

So also post stats for eth0

statistics after i configured everything back to 100mbit (router):

root@OpenWrt:~# ethtool -S lan2
NIC statistics:
     tx_packets: 1355
     tx_bytes: 683552
     rx_packets: 95
     rx_bytes: 11200
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 6053
     TxMulticast: 1433
     TxBroadcast: 192
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 491
     TxPktSz65To127: 4739
     TxPktSz128To255: 569
     TxPktSz256To511: 243
     TxPktSz512To1023: 1227
     Tx1024ToMax: 409
     TxBytes: 1970320
     RxDrop: 0
     RxFiltering: 0
     RxUnicast: 133
     RxMulticast: 72
     RxBroadcast: 10
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 30
     RxPktSz65To127: 105
     RxPktSz128To255: 61
     RxPktSz256To511: 13
     RxPktSz512To1023: 6
     RxPktSz1024ToMax: 0
     RxBytes: 32332
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0

statistics on the laptop (fedora 39):

oli@fedora:~$ ethtool -S enp3s0f3u1
NIC statistics:
     tx_packets: 1056
     rx_packets: 330
     tx_errors: 0
     rx_errors: 0
     rx_missed: 0
     align_errors: 0
     tx_single_collisions: 0
     tx_multi_collisions: 0
     rx_unicast: 19
     rx_broadcast: 155
     rx_multicast: 156
     tx_aborted: 0
     tx_underrun: 0

statistics after a dhclient enp3s0f3u1 laptop:

root@fedora:/home/oli# ethtool -S enp3s0f3u1
NIC statistics:
     tx_packets: 1097
     rx_packets: 330
     tx_errors: 0
     rx_errors: 0
     rx_missed: 0
     align_errors: 0
     tx_single_collisions: 0
     tx_multi_collisions: 0
     rx_unicast: 19
     rx_broadcast: 155
     rx_multicast: 156
     tx_aborted: 0
     tx_underrun: 0

router:

root@OpenWrt:~# ethtool -S lan2
NIC statistics:
     tx_packets: 1434
     tx_bytes: 716356
     rx_packets: 95
     rx_bytes: 11200
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 6411
     TxMulticast: 1509
     TxBroadcast: 202
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 503
     TxPktSz65To127: 5017
     TxPktSz128To255: 607
     TxPktSz256To511: 261
     TxPktSz512To1023: 1284
     Tx1024ToMax: 450
     TxBytes: 2100815
     RxDrop: 0
     RxFiltering: 0
     RxUnicast: 133
     RxMulticast: 72
     RxBroadcast: 10
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 30
     RxPktSz65To127: 105
     RxPktSz128To255: 61
     RxPktSz256To511: 13
     RxPktSz512To1023: 6
     RxPktSz1024ToMax: 0
     RxBytes: 32332
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0

whats raising mainly on the router is: TxBytes, TxPktSz65To127 and TxUnicast. They keep raising even with that “broken” connection. on the laptop, the tx_packets is slowly raising (every second 1 packet, more or less)

how looks eth0 on r4? there are similar RX/TX keys and some with pX which representing the switch entry (also on r4 with builtin switch - i checked)

root@OpenWrt:~# ethtool -S eth0
NIC statistics:
     tx_bytes: 15462064
     tx_packets: 21174
     tx_skip: 0
     tx_collisions: 0
     rx_bytes: 2125766
     rx_packets: 10254
     rx_overflow: 0
     rx_fcs_errors: 0
     rx_short_errors: 0
     rx_long_errors: 0
     rx_checksum_errors: 0
     rx_flow_control_packets: 0
     rx_xdp_redirect: 0
     rx_xdp_pass: 0
     rx_xdp_drop: 0
     rx_xdp_tx: 0
     rx_xdp_tx_errors: 0
     tx_xdp_xmit: 0
     tx_xdp_xmit_errors: 0
     rx_pp_alloc_fast: 112543
     rx_pp_alloc_slow: 9
     rx_pp_alloc_slow_ho: 0
     rx_pp_alloc_empty: 9
     rx_pp_alloc_refill: 1802
     rx_pp_alloc_waive: 0
     rx_pp_recycle_cached: 0
     rx_pp_recycle_cache_full: 0
     rx_pp_recycle_ring: 113841
     rx_pp_recycle_ring_full: 0
     rx_pp_recycle_released_ref: 0
     p06_TxDrop: 0
     p06_TxCrcErr: 0
     p06_TxUnicast: 10254
     p06_TxMulticast: 0
     p06_TxBroadcast: 0
     p06_TxCollision: 0
     p06_TxSingleCollision: 0
     p06_TxMultipleCollision: 0
     p06_TxDeferred: 0
     p06_TxLateCollision: 0
     p06_TxExcessiveCollistion: 0
     p06_TxPause: 0
     p06_TxPktSz64: 0
     p06_TxPktSz65To127: 0
     p06_TxPktSz128To255: 0
     p06_TxPktSz256To511: 0
     p06_TxPktSz512To1023: 0
     p06_Tx1024ToMax: 0
     p06_TxBytes: 2125766
     p06_RxDrop: 0
     p06_RxFiltering: 15
     p06_RxUnicast: 21174
     p06_RxMulticast: 0
     p06_RxBroadcast: 0
     p06_RxAlignErr: 0
     p06_RxCrcErr: 0
     p06_RxUnderSizeErr: 0
     p06_RxFragErr: 0
     p06_RxOverSzErr: 0
     p06_RxJabberErr: 0
     p06_RxPause: 0
     p06_RxPktSz64: 0
     p06_RxPktSz65To127: 0
     p06_RxPktSz128To255: 0
     p06_RxPktSz256To511: 0
     p06_RxPktSz512To1023: 0
     p06_RxPktSz1024ToMax: 0
     p06_RxBytes: 15462064
     p06_RxCtrlDrop: 0
     p06_RxIngressDrop: 0
     p06_RxArlDrop: 0

have you any kind of hw offloading activated?

else it looks good…traffic in both directions without errors