BPI-R2 Kernel 4.14 HNAT

Yes, state=UNBIND means it is not leveraging the hardware.
I do not know how to make an entry state=BIND manually.
I only can try to observe network stack to make sure the LAN <–> WAN stream can pass the checking in reasonable nf_hook registered by mtkhnat, and then reach this line.

[Reasonable path]

eth0 --> lanX --> xxx_pre_routing() hook --> Network stack & SW NAT --> xxx_post_routing() hook --> wan ---> eth0 or eth1, depending on dsa setting.

In 4.14 (with 2nd gmac) the path imho should be this:

Client => lanx => (eth0) => nfprerouting-hook => sw/hwnat (postrouting) => (eth1) => wan/ppp => internet

reverse way (answer from internet-server):

Server => wan/ppp => (eth1) => sw/hw nat (prerouting) => postrouting => (eth0) => lanx => client

I updated the file drivers/net/ethernet/mediatek/mtk_hnat/hnat.c with

cr_set_field(host->ppe_base + PPE_BNDR, BIND_RATE, 0x20);

However now iptables is very slow and all sys entries are still set as UNBIND.

# cat /sys/kernel/debug/hnat/all_entry

Is there anything else require?

as i understand @rainfall83 right with this:

then this change is not needed…and all traffic should be bound. i tested only init-procedure (will be called if module loaded), because i don’t know how to debug it deeper

OK then i will revert my change.

i can confirm, that on 4.14-hnat also all entries are UNBIND:

[email protected]:~# cat /sys/kernel/debug/hnat/all_entry

how can we debug this?

[  380.129873] [HNAT] hnat_probe: (drivers/net/ethernet/mediatek/mtk_hnat/hnat.c
:252) of_node=hnat
[  380.129900] [HNAT] hnat_probe: (drivers/net/ethernet/mediatek/mtk_hnat/hnat.c
:265) res:-559325376
[  380.129966] [HNAT] hnat_probe: (drivers/net/ethernet/mediatek/mtk_hnat/hnat.c
:273) host-fe_base:-503775232
[  380.130236] [HNAT] hnat_probe: (drivers/net/ethernet/mediatek/mtk_hnat/hnat.c
:281) err (debugfs):0
[  380.133039] [HNAT] hnat_probe: (drivers/net/ethernet/mediatek/mtk_hnat/hnat.c
:293) err (hnat_start):0
[  380.133044] [HNAT] hnat_probe: (drivers/net/ethernet/mediatek/mtk_hnat/hnat.c
:299) register reached
[  380.323695] [HNAT] hnat_probe: (drivers/net/ethernet/mediatek/mtk_hnat/hnat.c
:303) ret (hnat_register):0

As far as i know,traffic is received by hnat-module,but i don’t know why it is unbind…there are several unclear checks (mtk_hnat_nf_post_routing) before bind is set (skb_to_hnat_info)

i’ve added some printk’s to mtk_hnat_nf_post_routing, and looked which are printed…

first result: they are less than i thought…only 2 entries (i thought that this is done by each packet received from driver)

Jul 18 12:03:25 bpi-r2-ubuntu kernel: [  115.123949] [HNAT] mtk_hnat_nf_post_rou
ting: (drivers/net/ethernet/mediatek/mtk_hnat/hnat_nf_hook.c:171) ct
Jul 18 12:03:25 bpi-r2-ubuntu kernel: [  115.124015] [HNAT] mtk_hnat_nf_post_rou
ting: (drivers/net/ethernet/mediatek/mtk_hnat/hnat_nf_hook.c:178) help

my code:

167     ct = nf_ct_get(skb, &ctinfo);
168     if (!ct)
169         return 0;
170 printk(KERN_WARNING "[HNAT] %s: (%s:%i) ct",
171  __FUNCTION__, __FILE__, __LINE__);
173     /* rcu_read_lock()ed by nf_hook_slow */
174     help = nfct_help(ct);
175     if (help && rcu_dereference(help->helper))
176         return 0;
177 printk(KERN_WARNING "[HNAT] %s: (%s:%i) help",
178  __FUNCTION__, __FILE__, __LINE__);
180     if ((FROM_GE_WAN(skb) || FROM_GE_LAN(skb)) &&
181         skb_hnat_is_hashed(skb) &&
182         (skb_hnat_reason(skb) == HIT_BIND_KEEPALIVE_DUP_OLD_HDR))
183         return -1;
184 printk(KERN_WARNING "[HNAT] %s: (%s:%i) from_wan/lan",
185  __FUNCTION__, __FILE__, __LINE__);

as you see the last message does not came up…so that condition seems to be the Problem…after adding a printk for the check-parts…i get much more messages (also the missing “from_wan/lan”)…seems previous test has cached something on client-side

180 printk(KERN_WARNING "[HNAT] %s: (%s:%i) check: from wan: %i,from lan:%i,skb-hashed:%i,skb-reason:%i",
181  __FUNCTION__, __FILE__, __LINE__,FROM_GE_WAN(skb), FROM_GE_LAN(skb),skb_hnat_is_hashed(skb),skb_hnat_reason(skb));

what i wonder: i did not get any message where from “lan:1”

everytime the “from lan/wan”-message comes up wan=1…maybe its a detection-problem of lan-ports

Jul 18 12:21:24 bpi-r2-ubuntu kernel: [  440.135087] [HNAT] mtk_hnat_nf_post_routing: (drivers/net/ethernet/mediatek/mtk_hnat/hnat_nf_hook.c:181) check: from wan: 1,from lan:0,skb-hashed:1,skb-reason:14
Jul 18 12:21:24 bpi-r2-ubuntu kernel: [  440.135156] [HNAT] mtk_hnat_nf_post_routing: (drivers/net/ethernet/mediatek/mtk_hnat/hnat_nf_hook.c:189) from_wan/lan

skb-hashed is mostly 1, and skb-reason is 12,13,14,15, 23 (here all values are 0).

drivers/net/ethernet/mediatek/mtk_hnat/hnat.h:371:#define FROM_GE_LAN(skb)	(HNAT_SKB_CB(skb)->iif == FOE_MAGIC_GE_LAN)
drivers/net/ethernet/mediatek/mtk_hnat/hnat.h:375:#define FOE_MAGIC_GE_LAN	0x7272
drivers/net/ethernet/mediatek/mtk_hnat/hnat_nf_hook.c:223:		HNAT_SKB_CB(skb)->iif = FOE_MAGIC_GE_LAN;

the flag is set in prerouting-nf-hook:

224     if (IS_WAN(state->in))
225         HNAT_SKB_CB(skb)->iif = FOE_MAGIC_GE_WAN;
226     else if (IS_LAN(state->in))
227         HNAT_SKB_CB(skb)->iif = FOE_MAGIC_GE_LAN;

which leads us to IS_LAN-macro:

drivers/net/ethernet/mediatek/mtk_hnat/hnat.h:402:#define IS_LAN(dev)		(!strncmp(dev->name, "lan", 3))

lol…after i added a printk to pre_routing-function i see my problem:

Jul 18 12:48:11 bpi-r2-ubuntu kernel: [  117.792015] [HNAT] mtk_hnat_nf_pre_routing: (drivers/net/ethernet/mediatek/mtk_hnat/hnat_nf_hook.c:221) in-device: ap0

i tested only over wifi-device :wink: because i need my laptop to build kernel…i think IS_LAN should be changed to include ap0 and wlan* devices or maybe !wan (but how about the unknown-devices - last step in prerouting)…

there are maybe some other problems:


121     if (IS_LAN(dev))
122         gmac = NR_GMAC1_PORT;
123     else if (IS_WAN(dev))
124         gmac = NR_GMAC2_PORT;

imho i cannot append ap0 to IS_LAN because then traffic is routed to GMAC1_PORT…maybe we need a IS_WIFI :slight_smile:

if i test over lan0 i have also BIND and FIN-states :slight_smile:

cat /sys/kernel/debug/hnat/all_entry | grep -v 'UNBIIND'

also tested with 4.14-hnat-branch…same result

uname -r
cat /sys/kernel/debug/hnat/all_entry | grep -v 'UNBIND'

so i assume that hnat works, @rainfall83 am i right? how can we add this behaviour to ap0 and wlanx?

so everything that is not wan (dts-setting), lan (first 3 letters from interface name, so also lanbrx) and bridge-device (first 2 letters “br”) get tagging invalid…maybe we can create a additional tag (e.g. FOE_WIFI) and add a IS_WIFI-macro (check if devicename is ap0 or contains “wlan”).

btw. in /proc/interrupts i see no decisive difference between with and without mtkhnat-module

ok, it takes some time…i started a download (ubuntu-image) and after some seconds, the interrupts counting ~4/second…in debugfs i see more bind than unbind-entries

to get it running for wifi maybe we can change the code to check only for wan (because traffic should not go from wan to wan…and wan is always part of the check):

if (FROM_GE_WAN(skb) || IS_WAN(out)) {

instead of

if ((FROM_GE_WAN(skb) || FROM_GE_LAN(skb)) &&


if (FROM_GE_WAN(skb) || IS_WAN(out)) {

instead of

if ((IS_LAN(out) && FROM_GE_WAN(skb)) || (IS_WAN(out) && FROM_GE_LAN(skb))) {

but first i want to merge current version, if someone confirm it is working

maybe then we can merge the block to only check once for wan-port and make additional checks inside

only check for wan seems not to work at least without tagging in mtk_hnat_nf_pre_routing

this is what i tried (no traffic with this patch on top of 4.14-hnat-branch): hnat_test_wifi.diff (2,5 KB) i have bind-entries for ap0-connection, but traffic seems to routed wrong or dropped

@jackzeng please use this thread if you are working on hnat (wifi/bridge-support).

Ok,frank,I am going to try your repo to see the xhci problem.

Hello, Frank, I’m trying to port Hnat to kernel 4.4, but after I ported one version code, eth will be disabled. do you have any suggestions?

current hnat-code depends on second gmac. I don’t know if it is implemented same as in 4.9/4.14. also forwarding saves sourceport which needs name lanx or wan (needs dsa-driver).

Why do you put such effort in old kernel 4.4?

can you help me with uboot (mmc-offset) and bluetooth (also not working in 4.4) and wifi (wmt-tools)?

What is status of official hnat-patch (new framework)? Last info i have is currently under review…

@Ryder.Lee can you give me an update on this?

framework should be this: https://www.kernel.org/doc/Documentation/networking/nf_flowtable.txt right? how far is hnat/hwqos implementation (4.19)?

Per my understanding, framework support software acceleration and openwrt community enhanced it to support hardware acceleration in openwrt trunk. we should wait until upstream framework support hw acceleration before sending patch, thanks.

Can i see the progress anywhere?

We are studying upstream framework and just found hnat feature can only be supported in openwrt trunk. After we have overview about framework, we will discuss with netfilter core developer about hw nat plan, and hope mt7623 can be the first platform to support upstream hnat feature, thanks.

Any news about this feature being implemented soon ?

We have sent 3 patches to openwrt and kernel.org to fix mt7621 hnat issues. All patches were merged, but OpenWRT user reported system become unstable and we are trying to find the root cause, thanks.

How many connection supported by hnat? I tests BPI-R2 by ixia, one UDP flow it works good. But I saw only 4 records in debug/hnat/all_entry when run test with 128 UDP flows.