[BPI-R2] Kernel Development

Banana Pi Router design
170

A bit too late actually but I think I might have something.

mtk_btif.c > references inc/mtk_btif.h > references ../plat_inc/btif_pub.h > which itself is referenced in mtk_btif.c

so might be a circular dependency that is messing things up, because running the compile with

./build > complete.log

reveals that the entry point for the error is one step before:

  ccache arm-linux-gnueabihf-gcc -Wp,-MD,drivers/misc/mediatek/btif/common/.mtk_btif.o.d  -nostdinc -isystem /usr/lib/gcc-cross/arm-linux-gnueabihf/7/include -I../arch/arm/include -I./arch/arm/include/generated -I../include -I./include -I../arch/arm/include/uapi -I./arch/arm/include/generated/uapi -I../include/uapi -I./include/generated/uapi -include ../include/linux/kconfig.h -include ../include/linux/compiler_types.h -D__KERNEL__ -mlittle-endian -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 -fno-dwarf2-cfi-asm -fno-omit-frame-pointer -mapcs -mno-sched-prolog -fno-ipa-sra -mabi=aapcs-linux -mfpu=vfp -marm -Wa,-mno-warn-deprecated -D__LINUX_ARM_ARCH__=7 -march=armv7-a -msoft-float -Uarm -fno-delete-null-pointer-checks -Wno-frame-address -Wno-format-truncation -Wno-format-overflow -O2 --param=allow-store-data-races=0 -Wframe-larger-than=1024 -fstack-protector-strong -Wno-unused-but-set-variable -Wimplicit-fallthrough -Wno-unused-const-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-var-tracking-assignments -g -pg -fno-inline-functions-called-once -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -fno-strict-overflow -fno-merge-all-constants -fmerge-constants -fno-stack-check -fconserve-stack -Werror=date-time -Werror=incompatible-pointer-types -Werror=designated-init -Werror -I../drivers/misc/mediatek/include/mt-plat/ -I../drivers/misc/mediatek/include/mt-plat/mt7623/include -I../arch/arm/mach-mt7623/include/mach -I../drivers/misc/mediatek/include/mt-plat -Idrivers/misc/mediatek/btif/common/inc -Idrivers/misc/mediatek/btif/common/plat_inc -I ../drivers/misc/mediatek/btif/common -I ./drivers/misc/mediatek/btif/common    -DKBUILD_BASENAME='"mtk_btif"' -DKBUILD_MODNAME='"btif"' -c -o drivers/misc/mediatek/btif/common/mtk_btif.o ../drivers/misc/mediatek/btif/common/mtk_btif.c
In file included from ../drivers/misc/mediatek/btif/common/mtk_btif.c:40:0:
../drivers/misc/mediatek/btif/common/inc/mtk_btif.h:37:10: fatal error: btif_pub.h: No such file or directory
 #include "btif_pub.h"
          ^~~~~~~~~~~~
compilation terminated.
../scripts/Makefile.build:265: recipe for target 'drivers/misc/mediatek/btif/common/mtk_btif.o' failed
../scripts/Makefile.build:509: recipe for target 'drivers/misc/mediatek/btif/common' failed
../scripts/Makefile.build:509: recipe for target 'drivers/misc/mediatek/btif' failed
../scripts/Makefile.build:509: recipe for target 'drivers/misc/mediatek' failed
../scripts/Makefile.build:509: recipe for target 'drivers/misc' failed
make[6]: *** [drivers/misc/mediatek/btif/common/mtk_btif.o] Error 1
make[5]: *** [drivers/misc/mediatek/btif/common] Error 2
make[4]: *** [drivers/misc/mediatek/btif] Error 2
make[3]: *** [drivers/misc/mediatek] Error 2
make[2]: *** [drivers/misc] Error 2
/home/pi/BPI-R2-4.14/Makefile:1649: recipe for target 'drivers' failed
make[1]: *** [drivers] Error 2
make[1]: *** Waiting for unfinished jobs....
171

It’s mid-week so hadn’t had a chance to take a good peek at it. Been busy with doing stuff at - you guessed it - main place of employment. :frowning:

172

Reproduced it. I’m wondering is it a first time you attempt to do an out-of-tree build for kernel tree with wifi driver?

Quick fix: replace all occurrences of “-I$(src)” in driver makefiles with “-I$(srctree)/$(src)”.

TL/DR: Problem happens due to $(src) being always a relative path to the location of the currently processed makefile from the source tree root. Thus produced C preprocessor search path directives end up to contain relative paths too. It works OK for in-tree build as something like ./drivers/misc/mediatek/inc is resolved to the expected in-tree directory containing required includes. But it does not work for out-of-tree build as PWD at compile time equals to KBUILD_OUTPUT and you won’t expect that required include files are available in there. It means that for include paths you should use -I$(srctree)/$(src)/... instead or simple -I$(src)/.... Constructs like this are in process of being updated all over the kernel - you may want to check the output of this command pipeline:
# git grep 'cflags-y' | grep -- '-I' | grep -- '$(src)'

1 Like
173

It is first time i do this (regardless of wifi-driver)…to take care of my ssd i want to build in ram only.

It seems you have not done this before :slight_smile: or is this already fixed in one of your last commits?

174

Yes, I wasn’t using out of tree build for linux kernel as traditionally it always was an in-tree build and my devops experience with other projects tell that moving from in tree to out of tree builds is an error-prone process - which you don’t want when you’re building such a significant thing as a kernel.

My PR to your 5.4-wifi branch had been updated with this commit:

1 Like
175

thanks, tried adding your commits to my merged tree, but ran into “no space left” and “echo: I/O error”…seems like 2G ramdisk is not enough :stuck_out_tongue: after resize ramdisk i see build is consuming 2.2G :wink: build was working so far

updated 5.4-merged and uploaded compiled kernel here

2 Likes
176

What is new in this build?

177

I have included lexa2s patches and upgraded to rc5

178

I mean in regards to wifi :stuck_out_tongue: BTW that was a quick response. Thanks

Reason for asking: If there is significant changes then maybe I will make a new build for arch.

179

You should use this for testing wifi only…

180

Two main highlights:

  1. Fixed debugging functionality so it is possible to compile driver with -DDBG=1 patched into Makefile and enable dynamic printk for various prints in driver code. Without fixes debug building was broken.
  2. Fixed building as a module. This is the main target for testing. In theory driver should work the same be it module or built-in but in practice I’ve seen some nasty problems from time to time when running wifi driver as a module. Sometimes problems looks like a driver hang, other times it might be something failing in another part of the kernel, like getting fake ext4 errors reported while fs is OK in reality. Bottom line is: don’t build wifi as a module if you’re targeting wide user audience. But if you want to test it on your own in non-prod environment - feel free to do it and report your findings back please. Your help is really appreciated.
181

kernel 5.4 is now finally released so i have updated my repo and added 5.4-main and uploaded binary-kernels on my gdrive

https://drive.google.com/open?id=1WnmFOaHjXp8oSYinGaL7khKlA4l21Dmc

3 Likes
182

This kernel version contains a module for wireguard ?

183

imho only the basic layer for encryption was merged in 5.5, 5.4 needs wireguard manually added

184

Hi frank, thank you for your very good job for this development.

If I can just add 2 comments. I compiled the 5.4-main kernel and installed it on the debian buster distribution (yours).

First, on the buster debian version, the iptables command returns the following:

root@bpi-r2:~# iptables -nL
iptables: Operation not supported.

On Buster, netfilter uses the new standard module named nftables instead of iptables legacy. To make continuity on all distributions, the iptables command pointed on new nft iptables:

root@bpi-r2:~# update-alternatives --config iptables
There are 2 choices for the alternative iptables (providing /usr/sbin/iptables).

  Selection    Path                       Priority   Status
------------------------------------------------------------
* 0            /usr/sbin/iptables-nft      20        auto mode
  1            /usr/sbin/iptables-legacy   10        manual mode
  2            /usr/sbin/iptables-nft      20        manual mode

Press <enter> to keep the current choice[*], or type selection number:

When you change the alternative to iptables-legacy, iptables command works. When I checked the kernel config, the NF_TABLES config is not available (as module or installed).

I compiled a new kernel with NF_TABLES as module and now, iptables command (based on iptables-nft) works. Since NFT is the new standard for network filtering command, I think that It is good to be activated by default on all new kernels.

Second, I have problem with fixed IP address on lan when I installed the new kernel (deb package) on R2. 9 of 10 times, ping the network gateway doesn’t work on the first boot on the kernel. When I reboot, generally the ping to gateway works.

I never have a network lost when the gateway was pinged one time, but one time I have never pinged the gateway after 10 reboots…

My network configuration file:

root@bpi-r2:~# cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
  pre-up ip link set $IFACE up
  post-down ip link set $IFACE down

auto lan0
iface lan0 inet static
  hwaddress ether 08:00:00:00:00:01 # if you want to set MAC manually
  address 192.168.1.46
  netmask 255.255.255.0
  gateway 192.168.1.254
  pre-up ip link set $IFACE up
  post-down ip link set $IFACE down

source-directory /etc/network/interfaces.d
185

Is only CONFIG_NF_TABLES needed? I guessed i had them included in defconfig…seems not…so i will add

For network issue…how old is your 5.4-main kernel? I fixed a wrong dts setting yesterday which may affect lan-ports

186

yesterday too (in afternoon, CEST timezone). I’ve just pull from your git, I have the following changes:

remote: Enumerating objects: 18, done.
remote: Counting objects: 100% (18/18), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 18 (delta 10), reused 16 (delta 9), pack-reused 0
Dépaquetage des objets: 100% (18/18), fait.
Depuis https://github.com/frank-w/BPI-R2-4.14
   cef42756642e..09639ebc2ff4  5.4-main   -> origin/5.4-main
Mise à jour cef42756642e..09639ebc2ff4
Fast-forward
 arch/arm/boot/dts/mt7623n-bananapi-bpi-r2.dts |  3 +--
 arch/arm/configs/mt7623n_evb_fwu_defconfig    |  1 +
 build.sh                                      | 22 ++++++++++++++++++----
 3 files changed, 20 insertions(+), 6 deletions(-)

I will try with this new version.

I think (I have not tested) that CONFIG_NF_TABLES is the minimum to run nft. Personnaly I have the following configuration (depending of filter utilization):

CONFIG_NF_TABLES=m
CONFIG_NF_TABLES_SET=m
CONFIG_NF_TABLES_INET=y
CONFIG_NF_TABLES_NETDEV=y
CONFIG_NFT_NUMGEN=m
CONFIG_NFT_CT=m
# CONFIG_NFT_FLOW_OFFLOAD is not set
CONFIG_NFT_COUNTER=m
CONFIG_NFT_CONNLIMIT=m
CONFIG_NFT_LOG=m
CONFIG_NFT_LIMIT=m
CONFIG_NFT_MASQ=m
CONFIG_NFT_REDIR=m
CONFIG_NFT_NAT=m
CONFIG_NFT_TUNNEL=m
CONFIG_NFT_OBJREF=m
CONFIG_NFT_QUOTA=m
CONFIG_NFT_REJECT=m
CONFIG_NFT_REJECT_INET=m
CONFIG_NFT_COMPAT=m
CONFIG_NFT_HASH=m
CONFIG_NFT_XFRM=m
CONFIG_NFT_SOCKET=m
CONFIG_NFT_OSF=m
CONFIG_NFT_TPROXY=m
CONFIG_NFT_SYNPROXY=m
CONFIG_NFT_DUP_NETDEV=m
CONFIG_NFT_FWD_NETDEV=m

CONFIG_NF_TABLES_IPV4=y
CONFIG_NFT_REJECT_IPV4=m
# CONFIG_NFT_DUP_IPV4 is not set
# CONFIG_NFT_FIB_IPV4 is not set
# CONFIG_NF_TABLES_ARP is not set

CONFIG_NF_TABLES_IPV6=y
CONFIG_NFT_REJECT_IPV6=m
# CONFIG_NFT_DUP_IPV6 is not set
# CONFIG_NFT_FIB_IPV6 is not set

# CONFIG_NF_TABLES_BRIDGE is not set

but I’m not a kernel specialist ^^

187

is the dts-change i’ve mentioned

git log -1 --oneline -- arch/arm/boot/dts/mt7623n-bananapi-bpi-r2.dts
03705c843af0 arm: dts: fix phy-mode and flowcontrol between gmac0 and port6

defconfig is the add of CONFIG_NF_TABLES

just try it out and tell me, which options are needed :slight_smile:

188

I have this error on build with the new version :

  YACC    scripts/dtc/dtc-parser.tab.[ch]
  UPD     include/generated/uapi/linux/version.h
"5.4.2-bpi-r2COPYING CREDITS Documentation Kbuild Kconfig LICENSES MAINTAINERS Makefile README README.md arch block build.conf build.sh certs crypto drivers fs include init ipc kernel lib mm net samples scripts security sound tools usr virt 5.4-main" exceeds 64 characters
make: *** [Makefile:1165: include/generated/utsrelease.h] Error 1
make: *** Waiting for unfinished jobs....
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
189

Try adding double quotes around $branches here: