[Banana Pi BPI-R64] Mainline OpenWRT image

Could you point me to hwnat patches,did not find them. Afair they were in staging of blogics/felix’ tree,but i do not see them anymore…just need author to ask him about it

The only related i’ve found is this offloading patch in generic/pending

Seems to be this patch as the the code from bug report is from there

https://git.openwrt.org/?p=openwrt/openwrt.git;a=blob;f=target/linux/generic/pending-5.4/770-16-net-ethernet-mediatek-mtk_eth_soc-add-flow-offloadin.patch;h=f63ed2899884c735e3cd007a0c54143a7713b176;hb=HEAD#l166

It looks like oops occours on (f0+d0=1c0) second rcu_assign_pointer. I don’t see implementation of mtk_foe_entry_commit in the patch (strange because header-entry is created),but i guess it generates a hash for the src-dest relation and saves it to eth->foe_flow_table with hash as index. I guess the hash is created and returned,but entry in flow-table is not created so there is a out-of-bounds access Found implementation of mtk_foe_entry_commit here,but strange access to foe_table after creating hash in mtk_ppe_hash_entry

btw. it looks like offloading is only supported for mt7621 and 7622 (r64), not mt7623 (r2) but always compiled in…try to add patches 15+16 to my repo for testing

seems it needs also basic offload patches (640+), and i cannot apply on my 5.11-base because of missing FLOW_OFFLOAD_* in nf_flow_table.h i did not found in any other patches…no chance to get openwrt-patches into a normal linux tree ;(

have got it merged with 5.10-patches from https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=tree;f=target/linux/generic/pending-5.10;hb=HEAD

but it seems there is no Kconfig-option there…which option needs to be set to get the offloading?

maybe NFT_FLOW_OFFLOAD and/or NETFILTER_XT_TARGET_FLOWOFFLOAD ?

@lcsaszar @dixyes how did you disabled hwnat? unloading any module? can you trigger the issue anyhow?

I guess hwnat related asic should be almost the same in mt762x, so they share the same hwnat code for mt762{0,1,2,3,8,9}?

I enable/disable hwnat via luci: Network->Firewall->{enable “Software flow offloading" then enable/disable “Hardware flow offloading”}

When it’s enabled, just use it as gateway, it will crash at any time.

I still use that openwrt image I built before without enabling hwnat, now my r64 is used as my main router.

I bought r64 because my r6220(mt7621) will panic randomly with SMT and HWNAT both enabled. I disabled SMT, things seems to be OK, but there’s a strange slowly kernel memory leaking, that make r6220 crash in two day to two weeks period.

So I think the panic in SMT and HWNAT enabled mt7621 may be related to the panic in 2-cores and HWNAT enabled mt7622.

However I’m getting busy these days, I cannot confirm that. If I got some time, I will try trigger panic in mt7621 to compare with mt7622 panic.

Maybe,but it is only activated for mt7622 and 7621

Struct mtk_soc_data is modified by adding offload_version…and this has no default value and checked for existance/valid value (all in mtk_eth_soc.c

Reproducing with my code will be useful. Can you compare loaded mpdules when enabled/disabled and look in your kernelconfig,if both above options are set? Seems there is no mtk specific one

PPE offloading codes is located in mtk_soc_eth module, that driver is always built-in (in my own build), so there’s nothing changed in modules list (it’s even not shown in lsmod).

Openwrt use nftables kernel api to enable/disable offloading, I have few knowledge about it, so I donot know what you do to use offloading in common distros. You may check Openwrt fw3 releated code (mostly shell scripts) to find it out.

here’s my iptables -L:

[email protected]:~# iptables -L | grep -i offload
# no hwnat setted
FLOWOFFLOAD  all  --  anywhere             anywhere             /* !fw3: Traffic offloading */ ctstate RELATED,ESTABLISHED FLOWOFFLOAD
# hwnat enabled
FLOWOFFLOAD  all  --  anywhere             anywhere             /* !fw3: Traffic offloading */ ctstate RELATED,ESTABLISHED FLOWOFFLOAD hw

I have CONFIG_NFT_FLOW_OFFLOAD=m and CONFIG_NETFILTER_XT_TARGET_FLOWOFFLOAD=m, the two modules xt_FLOWOFFLOAD and nft_flow_offload is always loaded while enabling/disabling hwnat.

Thanks this looks like we need to know how flowcontrol in nftables need to be enabled…maybe nftables tools are need to patched too. Can you export nftables to file (maybe nftables-save) so we see the command to apply flowtables rule? List does not show it

It crashes too quick that I cannot save it all… maybe I should omit “treat oops as panic” option.

but I managed to get this:

iptables -A FORWARD -m comment --comment "!fw3: Traffic offloading" -m conntrack --ctstate RELATED,ESTABLISHED -j FLOWOFFLOAD --hw

Don’t know if nft or xt userspace tools (nft, iptables command) need to be patched, but seems that xt_FLOWOFFLOAD and nft_flow_offload kmod must be loaded.

So these packets are put in separate chain “flowoffload” with a hw flag…it will be interesting what happens in this chain…have you serial console working? Then you could remove rj45 cables and look on it,maybe there is no crash because no traffic on lan/ports. If serial not working you could try removing only wan,because i guess offloading only happens only if nat is activated on wan.

I think we ahould create new topic for it

Sure, we’d better discuss in standalone thread only about this hwnat issue

could you summarize it? as i don’t see the issue i cannot create opening-post…just trace you’ve got in openwrt, bug-report and iptables/nftables setting…

We’re going to have a whole new HWNAT subsystem in mainline kernel. MTK will be the first vendor to adapt it.

1 Like

Will this work for IPV6 flows as well? Currently in openwrt, it can do full gigabit at 0% CPU load, but only on IPV4.

That chain is processed by xt_OFFLOAD, in this patch: https://github.com/openwrt/openwrt/blob/master/target/linux/generic/hack-5.4/650-netfilter-add-xt_OFFLOAD-target.patch

Will mt7623 also supported too? Can this also be used with “legacy” iptables?

work for all hwnat platforms.

I’ve updated my branch with 5.10 support.

You can enable it from: menuconfig -> Global build settings -> use the testing kernel version.

Hi guys. I have build image for R64 with kernel 5.10 https://cloud.mail.ru/public/cq89/AVqxejYqM. During building I have got one issue. Midnight Commander can’t compile.

1 Like

You can try to enable ext4 journaling (CONFIG_TARGET_EXT4_JOURNAL), it should help in cases of power loss.

Have you seen/fixed this bug with hw offload: [R64] Mainline OpenWRT image ?

Support for BananaPi BPi-R64

OpenWrt UBI installer image generator

is there a mainline-series to add hw-nat?

in openwrt it is pending, and i do not see the previous selection to mt7622 (and 7621/7629) only