I’m rather new to Banana Pies and would like to contribute to driver testing/development (mainly testing) of the relevant components.
Can anyone share experience on how to organize development environment for efficient Linux kernel testing/debugging with BPIs? I’ve seen GitHub - frank-w/BPI-Router-Linux: Linux kernel 4.14+ for BPI-R2, 5.4+ for R64, 6.1+ for R2Pro and R3 repo which can be a starting point to build custom kernel binaries. I’m wondering on how it is best to organize smooth transfers of freshly built kernel binary and kernel modules between the build-machine and the Pi. Installing that manually by taking out the SD-card and replacing boot files every time looks like a lot of churn.
Pulling the files from TFTP server in U-boot is probably better though maybe there are some other variants? I’ve tried to follow How to create initrd/initramfs for use with tftp. It seems OK but I guess initrd is the only convenient option for packing kernel modules and not messing everytime with transfering them to the target filesystem… Is there any good guide on how to organize that (buildroot part as well)?
Another problem is the size of the kernel image. It grows significantly with debugging options enabled (like sanitizers and other stuff) and attached initrd. And may be well greater than CONFIG_SYS_BOOTM_LEN limit defined by U-boot. Has anyone coped with big kernel image sizes, or is this a strange idea overall? Are there any good methods/strategies other than redefining the constant?
For quick tests i mostly use tftp,but it requires at least a bootloader with network working and configured and a tftp server.
I have dnsmasq as tftp server running on my main-router with an inotify script which generates a file list. This filelist is downloaded in my uboot when choosing the tftp option in my uboot menu for bpi-routers. In my kernel repo i handle compile with build.conf+build.sh and here after compiling i choose upload which asks for filename and does the upload (then inotify updates the filelist).
This way it works till you need modules (only builtin)…these must be present in rootfs (sdcard),but when linux has network support you can install once and then push the module from compile pc to board via sftp/scp (manual step,but without removing the sdcard)…i did this for some mt76 wifi tasks.
I also can start openwrt recovery itb which are nearly 30MB in my uboot…
Yeah, I’m poking around with tftp now. It boots okay with a self-compiled kernel with default bpi-r4 config and even initrd.
But when I toggle an extra kernel debugging option CONFIG_KASAN, it doesn’t boot even without initrd. The size of the image of course increases significantly (though not extremely). This should probably deal with the defined memory map. U-boot prints out that fdt has been overwritten. Any ideas how this can be adjusted to be able to boot debug-enabled kernel eventually? (if you need some extra configuration info or logs, please tell)
Using ethernet@15110100 device
TFTP from server 192.168.0.112; our IP address is 192.168.0.19
Filename 'r4-itbfiles.lst'.
Load address: 0x46000000
Loading: *
TFTP error: 'file /tmp/ftpd/r4-itbfiles.lst not found for 192.168.0.19' (1)
Not retrying...
tftp download failed
enter kernel name: bpi-r4.itb
kernelinput=bpi-r4.itb
## Error: "initrd" not defined
PARTITION_CONFIG only exists on eMMC
sd available
bootconf=#conf-base#ov-sd
Using ethernet@15110100 device
TFTP from server 192.168.0.112; our IP address is 192.168.0.19
Filename 'bpi-r4.itb'.
Load address: 0x46000000
Loading: #################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#############################
4.7 MiB/s
done
Bytes transferred = 21405700 (146a004 hex)
## Loading kernel (any) from FIT Image at 46000000 ...
Using 'conf-base' configuration
Trying 'kernel-1' kernel subimage
Description: Linux Kernel 6.12.47-main
Type: Kernel Image
Compression: gzip compressed
Data Start: 0x460000e8
Data Size: 21323859 Bytes = 20.3 MiB
Architecture: AArch64
OS: Linux
Load Address: 0x44000000
Entry Point: 0x44000000
Hash algo: sha1
Hash value: 62eca42bb956fde19f9d9032d05d1d9a69bc31e0
Verifying Hash Integrity ... sha1+ OK
## Loading fdt (any) from FIT Image at 46000000 ...
Using 'conf-base' configuration
Trying 'fdt-base' fdt subimage
Description: Flattened Device Tree blob
Type: Flat Device Tree
Compression: uncompressed
Data Start: 0x47456238
Data Size: 36736 Bytes = 35.9 KiB
Architecture: AArch64
Load Address: 0x47000000
Hash algo: sha1
Hash value: a141981f607a7f7a89b7328300d2b81661150ee1
Verifying Hash Integrity ... sha1+ OK
Error: fdt overwritten
Could not find a valid device tree
Kernel does not boot here because you have overwritten fdt. Try using different loadaddr. For openwrt recovery i use 0x50000000,but here initrd is inside fit image…my bootchain uses external initrd.
It is not easy to find a memory map to work with large kernels and initrd.
For extra arguments like debug=7 i use bootopts which are used in buildargs
I build the kernel as an archlinux package. Updating the kernel is as updating the package. See latest posts of archlinuxarm on R4.
Only when linux does not boot anymore, I can pull out sdcard, or boot linux-recovery from nand and chroot to rootfs.
Basically I only use U-Boot to do a distroboot from nand for booting from nvme (extlinux.conf) and do everything else in linux rootfs or recovery initrd, both running bash.