[BPI-R3 Mini] [SOLVED] Debian installer (upstream) stuck

Banana Pi Router design BPI-R3/R3 Mini(MT7986)
41

Sure, that part is clear. U-boot can pass board specific devicetree to GRUB (EFI loader) correctly. But GRUB then not pass it to the kernel when it loads it. So the bootlog provided above pretty much what one get if try to load the kernel without devicetree.

Yes, right now the installer can boot successfully if I manually configure a valid devicetree path in GRUB. It successfully recognized the NVMe and installed Ubuntu into it.

1 Like
42

https://docs.kernel.org/admin-guide/efi-stub.html

Can add the commandline options initrd= and dtb= to the vmlinuz.efi if I understand correctly, using the correct path.

1 Like
43

Quick status update. I enabled bootefi command on your fork. Temporarly disabled non-R3 mini targets just to speed up the CI. The resulting u-boot is able to boot GRUB and then Linux from the NVMe.

The boot commands at u-boot:

usb start
pci enum
nvme scan
setenv kernel_addr_r 0x40008000
setenv fdt_addr_r 0x40000000
fatload usb 0:1 ${fdt_addr_r} mt7986a-bananapi-bpi-r3-mini.dtb
fatload nvme 0:1 ${kernel_addr_r} EFI/ubuntu/shimaa64.efi
bootefi ${kernel_addr_r} ${fdt_addr_r}

Before booting straight to the NVMe, I copied to devicetree into th NVMe rootfs (/boot/mt7986.dtb). This might not required as pointed out Eric in the previous comment. Contrary to my prevoius comments, GRUB might actually pass devicetree to the kernel, and the issue actually solved by the clk_ignore_unused pd_ignore_unused cma=128M argument. To be foolproof, I added it regardless.

Then I edited GRUB menuentry just to boot into the system:

menuentry 'Ubuntu BananaPi R3 mini' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-0bc688a8-9da8-4167-9a53-fff27df09ebe' {
	recordfail
	load_video
	gfxmode $linux_gfx_mode
	insmod gzio
	if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
	insmod part_gpt
	insmod ext2
	search --no-floppy --fs-uuid --set=root 0bc688a8-9da8-4167-9a53-fff27df09ebe
	devicetree /boot/mt7986.dtb
	linux	/boot/vmlinuz root=UUID=0bc688a8-9da8-4167-9a53-fff27df09ebe ro  console=ttyS0,115200n8 crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M clk_ignore_unused pd_ignore_unused cma=128M
	initrd	/boot/initrd.img
}

This boot straight to the system. Some TODO/missing pieces:

  • wifi driver not loaded at startup, the module have to loaded manually
  • eMMC not visible, lsblk only see the NVMe
  • ethernet not working, probably due to the missing firmware (the driver looking at the eMMC)

Hopefully I will have time to investigate these.

44

Both (wifi+ethernet) on r3mini need firmware in /lib/firmware on defined rootfs (nvme) to work and of course kernel modules in /lib/modules/$(uname -r). For modules you can add the module name to /etc/modules.conf or afair in modprobe.d

45

Quick status update: Debian works as well. The clk_ignore_unused is a required kernel parameter. With that, it is reaches the initramfs phase, and successfully load every required clk driver. So =y is not needed indeed.

However, when the installer finished the job, it is required to include that parameter in /etc/defaults/grub and in /boot/grub/grub.cfg before reboot. After reboot, update-grub needed to make sure the line will be part of the generated grub.cfg.

Unfortunately the NIC driver not builded for Debian, not even as a module. I had to compile it for myself (following this guide). I had to tweak the .config a little bit after make module prepare, to match as close as possible with the built config (e.g.: CONFIG_CC_VERSION_TEXT was different). And as you said, linux-firmware package required (even the installer DVD has a trimmed list of firmwares).

Wifi worked out of the box, NVMe as well. Some stuff like netplan, networkd, timesync etc. missing or not configured by default. Also I used a Realtek USB Ethernet dongle for bootstrap the system.

Overall, it was not much effort after everything sorted out. Even with that, probably not worth the hassle compared flashing your prebuilt image. But I wanted to check how good the upstream support :slight_smile:

46

Userspace application and firmware can be installed afterwards. Ethernet and switch driver ahould be included (afaik not working as modules). Have you checked in /proc/config.gz if they are really not enabled?

47

Yes, I checked and it said “not selected” nor as a module or builtin. But it was not a big deal, maybe I send another PR to Debian upstream, to enable it as a module. This Airoha NIC air8811 driver looks like it could be easy to compile out-of-tree as well, for example with dkms.

48

You need this as module to allow loading firmware from rootfs (else you have to include firmware in kernel binary).

1 Like