[BPI-R4] any plans for jumbo frame support?

Yes,patches are only in this 6.18-jumbo branch.

See this patch in mtk sdk for fixing the dsa warnings:

https://git01.mediatek.com/plugins/gitiles/openwrt/feeds/mtk-openwrt-feeds/+/91956213e315348639dc350ce567b893e1d85ed3^!/#F0

updated 6.18-jumbo branch, but only compilation tested yet…did first bootup, and git an oops…still looking why this happens…seems like happen in probe (i guess i found it but need to think how to solve correctly).

edit: have fixed crash in 6.19-jumbo (and dropped that update from 6.18 tree as it breaks), but jumbo-frames handling is now completely different…i have to set mtu for eth0 first before i can set dsa-user-port mtu

#BPI-R4
# ip link set eth0 mtu 9000 up
# ip link set dev lan3 mtu 9000 up
# ip a a 192.168.90.1/24 dev lan3

#other side (laptop):
$ sudo ip link set dev enx00e04c68001b mtu 9000 up
$ sudo ip a a 192.168.90.2/24 dev enx00e04c68001b
$ ping -M do -s 8972  192.168.90.1
PING 192.168.90.1 (192.168.90.1) 8972(9000) bytes of data.
8980 bytes from 192.168.90.1: icmp_seq=1 ttl=64 time=0.738 ms
...

mhm, it seems that the mac down/up is still needed, but mt533x error seems gone

root@bpi-r4-v11:~
# ip a a 192.168.90.1/24 dev lan3
root@bpi-r4-v11:~
# ip link set eth0 mtu 9000 up
Error: mtu greater than device maximum.
root@bpi-r4-v11:~
# ip link set eth0 up
[   51.854102] mtk_soc_eth 15100000.ethernet eth0: configuring for fixed/internal link mode
[   51.862259] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 10Gbps/Full - flow control rx/tx
root@bpi-r4-v11:~
# ip link set eth0 down
[   54.964902] mtk_soc_eth 15100000.ethernet eth0: Link is Down
root@bpi-r4-v11:~
# ip link set eth0 mtu 9000 up
[   95.284815] mtk_soc_eth 15100000.ethernet eth0: configuring for fixed/internal link mode
[   95.293009] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 10Gbps/Full - flow control rx/tx
root@bpi-r4-v11:~
# ip link set dev lan3 mtu 9000 up
[  131.224631] mt7530-mmio 15020000.switch lan3: configuring for phy/internal link mode
[  131.233282] mt7530-mmio 15020000.switch lan3: Link is Up - 1Gbps/Full - flow control rx/tx
root@bpi-r4-v11:~
# 
root@bpi-r4-v11:~
# dmesg | grep mt753
[    3.359175] mt7530-mmio 15020000.switch: configuring for fixed/internal link mode
[    3.369729] mt7530-mmio 15020000.switch: Link is Up - 10Gbps/Full - flow control rx/tx
[    3.370382] mt7530-mmio 15020000.switch wan (uninitialized): PHY [mt7530-0:00] driver [Generic PHY] (irq=POLL)
[    3.371955] mt7530-mmio 15020000.switch lan1 (uninitialized): PHY [mt7530-0:01] driver [Generic PHY] (irq=POLL)
[    3.373384] mt7530-mmio 15020000.switch lan2 (uninitialized): PHY [mt7530-0:02] driver [Generic PHY] (irq=POLL)
[    3.374836] mt7530-mmio 15020000.switch lan3 (uninitialized): PHY [mt7530-0:03] driver [Generic PHY] (irq=POLL)
[  131.224631] mt7530-mmio 15020000.switch lan3: configuring for phy/internal link mode
[  131.233282] mt7530-mmio 15020000.switch lan3: Link is Up - 1Gbps/Full - flow control rx/tx

Thanks! I saw your newest commits and I’ve been trying to backport them to 6.12 (since I’m running OpenWrt). The mtk_eth_soc driver has changed a lot in the past year or so – I’ll need a while to figure out how.

CMIIW: we’re looking at ci: fix R3 filename (used in images-repo)…net: mediatek: fix crash after jumbo-patch correct?

I reverted my fix and simply moved the mtk_set_max_mtu more down in mtk_add_mac in later commit (6.19-jumbo)

so basicly you need these commits:

a3207a383794 2025-12-28 net: mtk_eth_soc: try to fix crash the right way Frank Wunderlich  (HEAD -> 6.19-jumbo, origin/6.19-jumbo)
ca4c976dce22 2025-11-12 net: ethernet: mtk_eth_soc: add dynamic rx buffer adjustment support Mason Chang 
b4c6cb1d5532 2025-10-17 net: ethernet: mtk_eth_soc: add 9k jumbo frame support Mason Chang 
92bfdbdd4de1 2025-10-08 net: ethernet: mtk_eth_soc: change default rx buffer length Mason Chang

most changes are because RSS/LRO support, but it should be no much problem to adapt the changes for 6.12

i will merge the top 2 into one tomorrow

Thanks for the update. I’ve gotten those commits in my build – and just managed to compile the kernel fixing a few merge conflicts (as you mentioned, was mostly for RSS and LRO). I’ll figure out how to roll it out to test in the next few days.

Unfortunately when I compiled and run the updated kernel on my bpi I could no longer talk to it over Ethernet while plugged into the 1gbps eth switch. It booted – does the change only work for the 10gbps SFP ports, or?

I don’t have a UART either so that may be the next thing I get to figure out what’s wrong.

Hello everyone, I’ve committed 4 more commits to my repo (synced with latest Frank’s 6.18-jumbo,at least few days ago) 1st 3 addresses the same issue as the

Last one should add some performance.

I didn’t heavily testerd it yet, and didn’t test the 4th commit at all.

Feel free to join the testing. I’m planning to do it by the end of the week. Will update on any result

Moderated: added link

Frank, thanks for adding the link, +1 commit

short description - these commits address the next issues:

  • Global MTU config issue: Changing the MTU on a single port can lead to DMA ring buffer reading memory corruption

  • coordinated MTU change on the port - as all ports are on the same RX RMA buffers - all port stopping and re-initialization is required for safe MTU changing

  • Some performance issues with mixed MTU ports: after enabling jumbo frames on a single port the whole system, including MT-1500 ports are switching to jumbo frame mode to be consistent with DMA ring buffers, it causes cache allocation/cleaning overhead and reduces 1-flow RX performance on a 10G port from ~5G to ~2.2G

and

addresses the performance drop - with these 2 commits, the avg RX speed for a single flow is ~2.7G

before:
[  5]  20.00-21.00  sec   256 MBytes  2.15 Gbits/sec    1   1.17 MBytes        
[  5]  21.00-22.00  sec   259 MBytes  2.17 Gbits/sec    0   1.33 MBytes        
[  5]  22.00-23.00  sec   260 MBytes  2.18 Gbits/sec    0   1.46 MBytes        
[  5]  23.00-24.00  sec   259 MBytes  2.17 Gbits/sec    0   1.59 MBytes        
[  5]  24.00-25.00  sec   260 MBytes  2.18 Gbits/sec    1   1.25 MBytes        
[  5]  25.00-26.00  sec   261 MBytes  2.19 Gbits/sec    0   1.39 MBytes        
[  5]  26.00-27.00  sec   260 MBytes  2.18 Gbits/sec    0   1.53 MBytes        
[  5]  27.00-28.00  sec   258 MBytes  2.16 Gbits/sec    3   1.17 MBytes    

after:
[  5]   9.00-10.00  sec   321 MBytes  2.69 Gbits/sec    0   1.40 MBytes        
[  5]  10.00-11.00  sec   328 MBytes  2.75 Gbits/sec    0   1.56 MBytes        
[  5]  11.00-12.00  sec   325 MBytes  2.73 Gbits/sec   26   1.24 MBytes        
[  5]  12.00-13.00  sec   318 MBytes  2.66 Gbits/sec    0   1.40 MBytes        
[  5]  13.00-14.00  sec   324 MBytes  2.72 Gbits/sec    0   1.56 MBytes        
[  5]  14.00-15.00  sec   329 MBytes  2.76 Gbits/sec   35   1.25 MBytes        
[  5]  15.00-16.00  sec   326 MBytes  2.74 Gbits/sec    0   1.43 MBytes   

CAUTION changing MTU under heavy load (iperf) may halt the system

Imho mtu should be changed on system startup which is currently broken because of the flow with enable mac first (to set max-mtu),set mtu and then reset mac again.

Could you add description to the corresponsing commit? I saw that you did many cleanups (removing spaces linebreaks etc.) so hard to find real changes. Maybe cleanup first (separately).

Thanks for working on getting this in better shape!

yes it makes sense to split the cleanup and functionality changing

Sure, it will take some time

Consider it as a draft version now :slight_smile:

If I understood correctly the runtime mtu change halts under the load due to DMA racing during the reset. Likely need to make sure that all buffers are flushed before changing MTU

+2

commit fc44b11c6870c9a3f8e87d2d8eb2c86105d8de56
commit af1b4adcffa8cfdc6781696a6f9f68f673000a68

Have improved the 10G port throughput from

to:

root@pve1:~# iperf3 -c 10.0.1.1
Connecting to host 10.0.1.1, port 5201
[  5] local 10.0.1.10 port 52804 connected to 10.0.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   386 MBytes  3.24 Gbits/sec   69   1.40 MBytes       
[  5]   1.00-2.00   sec   391 MBytes  3.28 Gbits/sec    0   1.59 MBytes       
[  5]   2.00-3.00   sec   374 MBytes  3.14 Gbits/sec    8   1.31 MBytes       
[  5]   3.00-4.00   sec   362 MBytes  3.04 Gbits/sec    0   1.48 MBytes       
[  5]   4.00-5.00   sec   389 MBytes  3.26 Gbits/sec   15   1.19 MBytes       
[  5]   5.00-6.00   sec   368 MBytes  3.08 Gbits/sec    0   1.39 MBytes       
[  5]   6.00-7.00   sec   370 MBytes  3.10 Gbits/sec    0   1.56 MBytes       
[  5]   7.00-8.00   sec   379 MBytes  3.18 Gbits/sec   18   1.28 MBytes       
[  5]   8.00-9.00   sec   356 MBytes  2.99 Gbits/sec    0   1.44 MBytes       
[  5]   9.00-10.00  sec   381 MBytes  3.20 Gbits/sec   18   1.13 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  3.67 GBytes  3.15 Gbits/sec  128             sender
[  5]   0.00-10.00  sec  3.66 GBytes  3.15 Gbits/sec                  receiver

with mixed MTU ports

perf top:

Samples: 421K of event 'cycles:P', 4000 Hz, Event count (approx.): 38034979025 lost: 0/0 drop: 0/0
Overhead  Shared Object                                          Symbol
  23.85%  [kernel]                                               [k] __pi_dcache_clean_poc
  15.10%  [kernel]                                               [k] __arch_copy_to_user
   7.90%  [kernel]                                               [k] __pi_dcache_inval_poc
   6.04%  [kernel]                                               [k] page_pool_alloc_pages
   3.87%  [kernel]                                               [k] _copy_to_iter
   3.30%  [kernel]                                               [k] gro_receive_skb
   2.40%  [kernel]                                               [k] default_idle_call
   1.95%  [kernel]                                               [k] mtk_poll_rx
   1.84%  [kernel]                                               [k] finish_task_switch.isra.0

It shows that __pi_dcache_clean_poc is consuming ~24% of cpu time on RX is still spent in cache cleaning for DMA.

changed: typos fixed :slight_smile:

I’ve renamed the branch to 6.18-jumbo-sandbox

I’ll continue here.

At this point i’ll appreciate any testing. especially on MTU-9000 enabled port, as in my current config i’m unable to test it any time soon.

The expected performance for RX should be close to wire-speed.