It is not related to the RSS/LRO, but as I am testing everything on nvme or raid arrays, and next tests only on this raid 6 array from these days, I would like share with everyone interested thinking about using bpi-r4 as its own nas.
Here is the on-going re-check performance of my raid6 array:
I have to say, that (as I created the array about more than 10 years ago) the array have probably NOT well optimalised stripe and stride, therefore performance of re-sync and working in degraded mode is lower. But I think that re-check is going quite fine.
working with a kernel built from 6.16-rsslro.
I have two questions.
a) ethtool -K eth0 lro on gets me an error “Could not change any device features”, is there some magic I need first? a kernel CONFIG_* flag? does the order in which ethtool flags are set matter?
b) if I try to set more than two RSS rules [e.g., use loc 2 or loc 3] I get EINVAL. Do I just need to choose the two dst-ips that matter the most for performance, or is there a way I can enable RSS for each VLAN?
Afair the “lro on” via ethtool was dropped at some state because needs to be defined by dest-ip. So afaik lro is working on layer 3 (mapping fastpath via ip header).
I have only tested with 1 rule,but afaik 4 should be possible
6.14-rsslro2 compiling with the same config I used for 6.14-rsslro & 6.16-rsslro did not produce a kernel that sees the network interfaces. it’s 4AM here so I may not fix the config til later.
using the mt7988a_bpi-r4_defconfig [admittedly filtered through the Armbian build framework] still has not produced a usable kernel. it’s now quarter to 5AM EDT.
You should use defconfig in the same branch. In 6.16 the sram was changed from allocating in ethernet driver to dedicated sram-mmio driver. I guess this is missing when you use older defconfig.
Not sure if i tested the 6.16 tree again after rebasing rss/lro patches on the irq/sram changes.
for this post, I did use that branch’s defconfig, as noted filtered through Armbian’s build framework.
Meanwhile, this post [different thread] suggests I’m barking up the wrong features. Assuming my primary/expected use-case for this board is router, which tree and which CONFIG_ should I be using?
Afair routing part (ppe+flowtable offload) should be mainline. But you have to define a flowtable with nftables…i did this only for hwnat some time ago and not with r4 yet as i have not much 10G equipment for testing routing.
sorry for the delay as I had working on hardware issues with my sfp nas solution…
I wrote script which is running as a systemd service to configure rss/lro every startup.
It should work for more IP aliases, and because several issues appeared, I was trying to comment every non-stright solution.
Script should produce easy readable output like:
OK irq 104 smp_affinity set to 1
OK irq 105 smp_affinity set to 2
OK irq 106 smp_affinity set to 4
OK irq 107 smp_affinity set to 8
OK rps value set to 0
OK HW LRO RX rule for sfp2 with dest ip 192.168.2.161 created
OK LRO for sfp2 enabled
OK HW LRO RX rule for sfp1 with dest ip 192.168.2.129 created
OK LRO for sfp1 enabled
or also ERROR in case of any problem.
rsslro_set.sh (4.7 KB)
Hmm, just looked into the script shortly, and instead of parsing dmesg output, I would rather use /sys structure today. I did not know how to work with it when the script is created.
it has issues for sure… but it does use the /sys/devices [and no part of dmesg] and I hope it’s a bit more readable. BUT… it also has an issue or 3 in that it more or less operates via assumed-state rather than rsslro_set.sh’s better state verification.
But I also don’t see how it gets around how many rules you can have. Mind you, my assumptions are much more complex. I’m trying to have half a dozen VLANs.
e.g.
tabris@ratatosk:~$ ip a|grep -A2 ethLAN|egrep '(ethLAN|inet )'
4: ethLAN: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
inet 192.168.88.253/23 brd 192.168.89.255 scope global ethLAN
10: guest@ethLAN: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.80.253/24 brd 192.168.80.255 scope global guest
11: iot@ethLAN: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.82.253/24 brd 192.168.82.255 scope global iot
12: modem@ethLAN: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.100.253/24 brd 192.168.100.255 scope global modem
13: surv@ethLAN: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.81.253/24 brd 192.168.81.255 scope global surv
