Driver development environment for Banana Pies

Hi,

I’m rather new to Banana Pies and would like to contribute to driver testing/development (mainly testing) of the relevant components.

Can anyone share experience on how to organize development environment for efficient Linux kernel testing/debugging with BPIs? I’ve seen GitHub - frank-w/BPI-Router-Linux: Linux kernel 4.14+ for BPI-R2, 5.4+ for R64, 6.1+ for R2Pro and R3 repo which can be a starting point to build custom kernel binaries. I’m wondering on how it is best to organize smooth transfers of freshly built kernel binary and kernel modules between the build-machine and the Pi. Installing that manually by taking out the SD-card and replacing boot files every time looks like a lot of churn.

Pulling the files from TFTP server in U-boot is probably better though maybe there are some other variants? I’ve tried to follow How to create initrd/initramfs for use with tftp. It seems OK but I guess initrd is the only convenient option for packing kernel modules and not messing everytime with transfering them to the target filesystem… Is there any good guide on how to organize that (buildroot part as well)?

Another problem is the size of the kernel image. It grows significantly with debugging options enabled (like sanitizers and other stuff) and attached initrd. And may be well greater than CONFIG_SYS_BOOTM_LEN limit defined by U-boot. Has anyone coped with big kernel image sizes, or is this a strange idea overall? Are there any good methods/strategies other than redefining the constant?

Thanks for your help!

For quick tests i mostly use tftp,but it requires at least a bootloader with network working and configured and a tftp server.

I have dnsmasq as tftp server running on my main-router with an inotify script which generates a file list. This filelist is downloaded in my uboot when choosing the tftp option in my uboot menu for bpi-routers. In my kernel repo i handle compile with build.conf+build.sh and here after compiling i choose upload which asks for filename and does the upload (then inotify updates the filelist).

This way it works till you need modules (only builtin)…these must be present in rootfs (sdcard),but when linux has network support you can install once and then push the module from compile pc to board via sftp/scp (manual step,but without removing the sdcard)…i did this for some mt76 wifi tasks.

I also can start openwrt recovery itb which are nearly 30MB in my uboot…

Frank, thank you for the feedback!

Yeah, I’m poking around with tftp now. It boots okay with a self-compiled kernel with default bpi-r4 config and even initrd.

But when I toggle an extra kernel debugging option CONFIG_KASAN, it doesn’t boot even without initrd. The size of the image of course increases significantly (though not extremely). This should probably deal with the defined memory map. U-boot prints out that fdt has been overwritten. Any ideas how this can be adjusted to be able to boot debug-enabled kernel eventually? (if you need some extra configuration info or logs, please tell)

Using ethernet@15110100 device
TFTP from server 192.168.0.112; our IP address is 192.168.0.19
Filename 'r4-itbfiles.lst'.
Load address: 0x46000000
Loading: *
TFTP error: 'file /tmp/ftpd/r4-itbfiles.lst not found for 192.168.0.19' (1)
Not retrying...
tftp download failed
enter kernel name: bpi-r4.itb
kernelinput=bpi-r4.itb
## Error: "initrd" not defined
PARTITION_CONFIG only exists on eMMC
sd available
bootconf=#conf-base#ov-sd
Using ethernet@15110100 device
TFTP from server 192.168.0.112; our IP address is 192.168.0.19
Filename 'bpi-r4.itb'.
Load address: 0x46000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #############################
         4.7 MiB/s
done
Bytes transferred = 21405700 (146a004 hex)
## Loading kernel (any) from FIT Image at 46000000 ...
   Using 'conf-base' configuration
   Trying 'kernel-1' kernel subimage
     Description:  Linux Kernel 6.12.47-main
     Type:         Kernel Image
     Compression:  gzip compressed
     Data Start:   0x460000e8
     Data Size:    21323859 Bytes = 20.3 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x44000000
     Entry Point:  0x44000000
     Hash algo:    sha1
     Hash value:   62eca42bb956fde19f9d9032d05d1d9a69bc31e0
   Verifying Hash Integrity ... sha1+ OK
## Loading fdt (any) from FIT Image at 46000000 ...
   Using 'conf-base' configuration
   Trying 'fdt-base' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x47456238
     Data Size:    36736 Bytes = 35.9 KiB
     Architecture: AArch64
     Load Address: 0x47000000
     Hash algo:    sha1
     Hash value:   a141981f607a7f7a89b7328300d2b81661150ee1
   Verifying Hash Integrity ... sha1+ OK
Error: fdt overwritten
Could not find a valid device tree

Kernel does not boot here because you have overwritten fdt. Try using different loadaddr. For openwrt recovery i use 0x50000000,but here initrd is inside fit image…my bootchain uses external initrd.

It is not easy to find a memory map to work with large kernels and initrd.

For extra arguments like debug=7 i use bootopts which are used in buildargs

I build the kernel as an archlinux package. Updating the kernel is as updating the package. See latest posts of archlinuxarm on R4.

Only when linux does not boot anymore, I can pull out sdcard, or boot linux-recovery from nand and chroot to rootfs.

Basically I only use U-Boot to do a distroboot from nand for booting from nvme (extlinux.conf) and do everything else in linux rootfs or recovery initrd, both running bash.

It is just what method you prefer.

Worked for me with kaddr=0x50000000, thanks)

I actually wasn’t aware of external initrd scheme. Will consider and research this method, too.

I prefer having initrd locally on usb or such and only loading kernel via tftp and not having multiple kernels with inird in them