[Banana Pi BPI-R64] Mainline OpenWRT image

Banana Pi Router design BPI-R64(MT7622)
102

@frank-w my command order is:

  1. echo 1 > /sys/kernel/debug/ieee80211/phy0/mt76/dbdc
  2. iw reg set ISO_3166-1_alpha-2
  3. iw reg set DE
  4. iw reg get result below:

root@OpenWrt:/# iw reg get global country DE: DFS-ETSI (2400 - 2483 @ 40), (N/A, 20), (N/A) (5150 - 5250 @ 80), (N/A, 23), (N/A), NO-OUTDOOR, AUTO-BW (5250 - 5350 @ 80), (N/A, 20), (0 ms), NO-OUTDOOR, DFS, AUTO-BW (5470 - 5725 @ 160), (N/A, 26), (0 ms), DFS (5725 - 5875 @ 80), (N/A, 13), (N/A) (57000 - 66000 @ 2160), (N/A, 40), (N/A)

root@OpenWrt:/#

103

In your log you did

iwinfo wlan1 freqlist

And htinfo before the dbdcā€¦thats why i askedā€¦

But your system is capable of 5ghz now after the region configā€¦does an additional wlanX come up after the echo dbdc? This new one is the 5g deviceā€¦the older phy is 2g4 only

104

@frank-wļ¼Œ

I donā€™t get your point. but now wlan0 is MT7615, wlan1 is board wifi.

105

From line 43 of your log (freqlist was before it but long output)

root@OpenWrt:/# iwinfo wlan1 htmodelist
HT20 HT40
root@OpenWrt:/# echo 1 > /sys/kernel/debug/ieee80211/phy0/mt76/dbdc

So if 7615 phy0 is wlan0 and mt7622 is wlan1 after the echo to dbdc you should see a wlan2 interface in ā€œip aā€ (mt7615 phy1) which should be capable of 5ghz

So it was looking on my bpi-r2 (wlan0 is mt7615 phy0 and wlan1 is phy1): 802.11ac 4x4 standard size mini pcie card is launched

107

@frank-w,

ip a: still only show wlan0 and wlan1. seems didnā€™t take effect.

108

As i said earlier, dbdc is merged with 5.7 linux kernel,maybe openwrt uses newer mt76 driver for 5.4, here you need to look in source if dbdc is implemented or only the sys-file exists. Maybe next lts (5.10) comes in openwrt soon. Maybe you can try my debian-image (need to add firmware files) to test your card regardibg the dbdc mode

109

ok, thanks for your reply.

but itā€™s confuse for me the BPI kernel 4.19 branch it is support the all function. https://github.com/BPI-SINOVOIP/BPI-R64-openwrt

110

Maybe openwrt always links latest mt76 driver into it (then it should work),but mainline linux native starts supporting dbdc in mt76 driver from 5.7 (no external linking to openwrts mt76).

Have you tried disabling mt7622? Maybe wlan1 used by mt7622 blocks dbdc anyhow. Normally it should be created by a formatstring like wlan%d so it should create the next free wlanX,but i see in buggy mt6625 (not really comparable i know) driver that access is done to fixed interface name

111

I have the same random crash at the same address. I will disable HW NAT and monitor for a few days.

Update (20.01.2021): no crash since I disabled hardware NAT.

112

Could you point me to hwnat patches,did not find them. Afair they were in staging of blogics/felixā€™ tree,but i do not see them anymoreā€¦just need author to ask him about it

The only related iā€™ve found is this offloading patch in generic/pending

Seems to be this patch as the the code from bug report is from there

https://git.openwrt.org/?p=openwrt/openwrt.git;a=blob;f=target/linux/generic/pending-5.4/770-16-net-ethernet-mediatek-mtk_eth_soc-add-flow-offloadin.patch;h=f63ed2899884c735e3cd007a0c54143a7713b176;hb=HEAD#l166

It looks like oops occours on (f0+d0=1c0) second rcu_assign_pointer. I donā€™t see implementation of mtk_foe_entry_commit in the patch (strange because header-entry is created),but i guess it generates a hash for the src-dest relation and saves it to eth->foe_flow_table with hash as index. I guess the hash is created and returned,but entry in flow-table is not created so there is a out-of-bounds access Found implementation of mtk_foe_entry_commit here,but strange access to foe_table after creating hash in mtk_ppe_hash_entry

btw. it looks like offloading is only supported for mt7621 and 7622 (r64), not mt7623 (r2) but always compiled inā€¦try to add patches 15+16 to my repo for testing

seems it needs also basic offload patches (640+), and i cannot apply on my 5.11-base because of missing FLOW_OFFLOAD_* in nf_flow_table.h i did not found in any other patchesā€¦no chance to get openwrt-patches into a normal linux tree ;(

have got it merged with 5.10-patches from https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=tree;f=target/linux/generic/pending-5.10;hb=HEAD

but it seems there is no Kconfig-option thereā€¦which option needs to be set to get the offloading?

maybe NFT_FLOW_OFFLOAD and/or NETFILTER_XT_TARGET_FLOWOFFLOAD ?

@lcsaszar @dixyes how did you disabled hwnat? unloading any module? can you trigger the issue anyhow?

113

I guess hwnat related asic should be almost the same in mt762x, so they share the same hwnat code for mt762{0,1,2,3,8,9}?

I enable/disable hwnat via luci: Network->Firewall->{enable ā€œSoftware flow offloading" then enable/disable ā€œHardware flow offloadingā€}

When itā€™s enabled, just use it as gateway, it will crash at any time.

I still use that openwrt image I built before without enabling hwnat, now my r64 is used as my main router.

I bought r64 because my r6220(mt7621) will panic randomly with SMT and HWNAT both enabled. I disabled SMT, things seems to be OK, but thereā€™s a strange slowly kernel memory leaking, that make r6220 crash in two day to two weeks period.

So I think the panic in SMT and HWNAT enabled mt7621 may be related to the panic in 2-cores and HWNAT enabled mt7622.

However Iā€™m getting busy these days, I cannot confirm that. If I got some time, I will try trigger panic in mt7621 to compare with mt7622 panic.

114

Maybe,but it is only activated for mt7622 and 7621

Struct mtk_soc_data is modified by adding offload_versionā€¦and this has no default value and checked for existance/valid value (all in mtk_eth_soc.c

Reproducing with my code will be useful. Can you compare loaded mpdules when enabled/disabled and look in your kernelconfig,if both above options are set? Seems there is no mtk specific one

115

PPE offloading codes is located in mtk_soc_eth module, that driver is always built-in (in my own build), so thereā€™s nothing changed in modules list (itā€™s even not shown in lsmod).

Openwrt use nftables kernel api to enable/disable offloading, I have few knowledge about it, so I donot know what you do to use offloading in common distros. You may check Openwrt fw3 releated code (mostly shell scripts) to find it out.

hereā€™s my iptables -L:

root@BPIR64:~# iptables -L | grep -i offload
# no hwnat setted
FLOWOFFLOAD  all  --  anywhere             anywhere             /* !fw3: Traffic offloading */ ctstate RELATED,ESTABLISHED FLOWOFFLOAD
# hwnat enabled
FLOWOFFLOAD  all  --  anywhere             anywhere             /* !fw3: Traffic offloading */ ctstate RELATED,ESTABLISHED FLOWOFFLOAD hw

I have CONFIG_NFT_FLOW_OFFLOAD=m and CONFIG_NETFILTER_XT_TARGET_FLOWOFFLOAD=m, the two modules xt_FLOWOFFLOAD and nft_flow_offload is always loaded while enabling/disabling hwnat.

116

Thanks this looks like we need to know how flowcontrol in nftables need to be enabledā€¦maybe nftables tools are need to patched too. Can you export nftables to file (maybe nftables-save) so we see the command to apply flowtables rule? List does not show it

117

It crashes too quick that I cannot save it allā€¦ maybe I should omit ā€œtreat oops as panicā€ option.

but I managed to get this:

iptables -A FORWARD -m comment --comment "!fw3: Traffic offloading" -m conntrack --ctstate RELATED,ESTABLISHED -j FLOWOFFLOAD --hw

Donā€™t know if nft or xt userspace tools (nft, iptables command) need to be patched, but seems that xt_FLOWOFFLOAD and nft_flow_offload kmod must be loaded.

118

So these packets are put in separate chain ā€œflowoffloadā€ with a hw flagā€¦it will be interesting what happens in this chainā€¦have you serial console working? Then you could remove rj45 cables and look on it,maybe there is no crash because no traffic on lan/ports. If serial not working you could try removing only wan,because i guess offloading only happens only if nat is activated on wan.

I think we ahould create new topic for it

119

Sure, weā€™d better discuss in standalone thread only about this hwnat issue

120

could you summarize it? as i donā€™t see the issue i cannot create opening-postā€¦just trace youā€™ve got in openwrt, bug-report and iptables/nftables settingā€¦

121

Weā€™re going to have a whole new HWNAT subsystem in mainline kernel. MTK will be the first vendor to adapt it.

1 Like
122

Will this work for IPV6 flows as well? Currently in openwrt, it can do full gigabit at 0% CPU load, but only on IPV4.