Hello, new user here, completely new to OpenWRT and BPI. Hoping to get some help and hope you don’t mind noob questions.
I’m trying to troubleshoot a strange issue.
I have my desktop linked to the BPI-R4 set up as an OpenWRT router via a 10GBps SFP+ DAC. The R4 is connected to my ISP’s ONT via a Cat8 ethernet cable with an SFP adapter into the SFP-WAN port.
When I ssh into the R4 and do speedtest, I get around 4089.55 Mbit/s Down and 2080.85 Mbit/s Up.
When I perform the same speed test on my desktop PC, I get 2439.02 Mbit/s Down, and 2073.75 Mbit/s Up.
If I directly connect my desktop to the ONT and perform the same speed measurement, I get around the same 4 GBps speed.
What could be causing the slowdown? I know that at least from one unit to another I am at least getting pretty close to the 10GBps top speed, so it’s not an issue with my hardware. Is there anything I could change in the configurations?
Also, my ISP advertises 10 GBps speed, but the R4 and my desktop’s 10G NIC both get only up to a max of 4Mbit/s speed. Is that to be expected?
Are you sure your ISP allows you to download at 10gbe speed?
Even if they sell you at that speed, a speedtest may not be able to get that speed from the speedtest servers.
Aside from that, if the NIC is not accelerated, there may be a performance issue.
Try to run top on the R4 while you’re executing the speedtest and check if the CPU load rises above 1 … if so, you need to enable packet steering
Current mainline (and openwrt) does not support a feature called RSS (and lro is not complete) for aggregating rx packages. Tx should be 10G,but maybe because of ack packages need to be handled. Also for traffic between both sfp you have to enable flow ofload,else all traffic goes over cpu.
Hi jpsollie - Thanks for the suggestion. I’ll check with my ISP again about it, though last time I talked to them they said that they had checked remotely and the ONT was capable of delivering 10G. How would I enable packet steering?
Frank - thanks! I have read about other people trying flow offload, so I tried this myself. When I went into the settings in LuCI and checked “Software Offload” and “Hardware Offload”, speed actually dropped to a fraction of the normal speed. I was getting around 0.22 MBps down.
Thanks! Upgrading to the latest snapshot and then checking “Software Offload” > “Hardware Offload” in Network>Firewall> General Settings>Routing/NAT Offsetting did improve things a bit!
First of all it’s important to know which version of OpenWrt you are using. I can only help you with official OpenWrt from openwrt.org, for the R4 you can download the SD card image here.
Second of all, at this point you can only obtain full 10G routing performance if you enable hardware flow offloading.
If you are using the R4 as (NAT-) router, doing this should fix the speed issues:
uci set firewall.@defaults[0].flow_offloading='1'
uci set firewall.@defaults[0].flow_offloading_hw='1'
uci commit firewall
reload_config
If you are having a bridge between the ISPs ONT and the interface your client machine is connected to you will in addition to the above also need to install bridger:
opkg update
opkg install bridger
To obtain full speed in both directions simultaneously you may need to build from source and include @Dale’s patch for utilizing more than one PPE. I will integrate it with OpenWrt once it gets accepted upstream, hopefully in a couple of days.
OT but bridger has always been a mystery to me.
From a layman’s pov what does it do when is it needed?
the term bridge is confusing to me at least. does it apply in a lan-bridge, vlan-bridge in network config? or when the bpi-r4 acts as a bridge(and not a NAT router) between ONT and the client machine?
Bridger is a Linux user space daemon which includes and eBPF tc classifier
program that offloads the bridge network datapath.
Bridger creates connection tracking flows for the connections which happen within a Linux bridge between different interfaces. In this way, those flows can be added to PPE and get offloaded in hardware instead of having to travel through the CPU.
From the cases you describe, the “lan bridge” and the “bpi-r4 acts as a bridge” are the exact same. LAN/WAN terminology doesn’t matter when all interfaces involved are bridged.
bridger is not needed for hardware-offloading bridged traffic flowing between different ports of a DSA switch, such as the 4x 1GE RJ-45 ports of the R4.
However, the SFP cages are not part of that DSA switch, but are considered separate network interfaces. Hence, the only way to offload traffic between those is via the PPE, and that works based on flows which need to be tracked.
In both cases, VLANs can differ and tagging/untagging would always be taken care of in hardware, as both, the DSA switch and the PPE support doing that.
Thank you for the crystal clear explanation.
bridger is not needed in my use case. as both sfp+ ports and dsa switch (eth0, eth1, eth2) are not bridged.
as there is no major traffic originating or terminating on the bpi-r4 in my use case. i expect when multiple ppe comes online it will have ticked all my checkbox.
yes. if you have a bridge consisting of eth1 + ports on the dsa switch.
no if eth1 is an independent routable interface and not part of the bridge consisting of lan1+lan2+lan3+lan4
this just gave me an idea that may simplify the management of the interfaces and vlans as I manually define vlans base on ports and not through the bridge interface.
with bridger, i can actually create a big bridge consisting of eth1+ 4x1Gbe then use luci to create and manage vlans… interesting…
