[BPI-R3 Mini] [SOLVED] Debian installer (upstream) stuck

I have not enabled it,maybe it is enabled by default. If you boot efi stack,maybe it is part of this. Virtio is because of qemu,right?

Is it required that efi file is on an efi partition when directly booting it?

Yes, virtio because of the QEMU, othewise this would be simply usb.

Is it required that efi file is on an efi partition when directly booting it?

Good question. In this case, the EFI payload bootaa64.efi memory mapped from the ISO to the 0x4b000000 address. My guess this is not required for the installer, but might be required if you boot from NVMe or eMMC.

Do we need efi at all? Basicly kernel+dtb+initrd should be enough,except installer requires efi,but on arm(64) efi is not standard.

No, I just dumped my notes into this comment if you or someone else want to experiment with EFI booting. ā€œLegacyā€ (kernel+dtb+initrd) boot work just as fine.

I just tried it and no luck :frowning:

I confirmed the clock drivers all in their place but we does not reach initrd loading without the clock driver init. So looks like it makes no sense to build these as modules…

I’m not sure autoloading modules works with clock drivers and in initrd. Afaik they have to be builtin.

Maybe it makes aense for debian team providing a per-vendor kernel binary to not increase the size too much if this is the argument for make clock drivers as modules.

Or just use my kernel and booting only the initrd to have installer (for e.g. different partition scheme than my images)…could this work?

Or just use my kernel and booting only the initrd to have installer (for e.g. different partition scheme than my images)…could this work?

I like the idea. This should be enough to install Debian but then we still have the same problem when we want to boot it if I understand correctly. I’m may be wrong about this, but the installer built from the upstream kernel, so it may fail to load rootfs at the first boot…

I think I ask them to have this module builtin. This is around ~13kb which should not be an issue. Also 7988, 7981, 8195 and 8173 clock drivers are all builtins. This justify to have 7986 builtin as well in my opinion.

I mean not using debian kernel entirely…only the rootfs

I continued the experiments with diferent installers. I expected little bit more responsiveness or quicker merge of the =y clockdriver change at Debian, lets hope they merge the PR sonner than later. Until that happens, I checked on which major distro config have it:

https://oracle.github.io/kconfigs/?config=UTS_RELEASE&config=COMMON_CLK_MT7986&config=COMMON_CLK_MT7986_ETHSYS&config=PINCTRL_MT7986

As turned out, Ubuntu since 24.10 and Oracle Linux 10 have it. So I tried both.

Ubuntu 25.04 successfully boot with bootefi. (Consider enabling bootefi for your u-boot builds as well). Unfortunately it is unable to initialize the a PCI subsystem, so lspci’s output literally empty. At least it can reach init, unpack initramfs and I only stuck at the point when it probe the block devices. eMMC not visible as a install target disk (even though lsblk see it), and NVMe not visible since the PCI subsystem not initialized. Your u-boot build see the NVMe, list its content, etc. so the device is fine…

Oracle Linux boot up as well and it is also reach the init successfully. Well, then something fails, the installer abort with a huge error banner in infinite loop.

Regarding the PCI: in your config, there were some extra config to bring up the PCI subsystem? My assumption I’ll run into this in the future with Debian installer as well.

Pcie nodes are still missing in upstream uboot because they want to switch dts to of_upatream first. But this requires having dts in linux also for mt7988 and some driver changes in uboot to be compatible with linux dts

I’m curious about PCIe in Linux, this is not in upstream for MT7986?

Linux should work. Any messages in dmesg different to working system?

Yes there were some anomalies. I’ll get back to that tomorrow I did not saved the dmesg output. thanks for the confirmation about the PCI

I managed to save out a bootlog, here it is: ubuntu_bootlog.txt (48.3 KB). Also included some output from the init’s output, but there are some stackdump from UBSAN.

The output of lsmod:

root@ubuntu-server:/# lsmod
Module                  Size  Used by
qrtr                   49152  2
mt7915e               229376  0
mt76_connac_lib       106496  1 mt7915e
mt76                  159744  2 mt7915e,mt76_connac_lib
mac80211             1658880  3 mt76,mt7915e,mt76_connac_lib
cfg80211             1306624  4 mt76,mt7915e,mac80211,mt76_connac_lib
libarc4                12288  1 mac80211
at24                   32768  0
ofpart                 16384  0
crypto_safexcel       172032  0
cmdlinepart            16384  0
spinand                86016  0
authenc                12288  1 crypto_safexcel
nandcore               69632  1 spinand
bch                    28672  1 nandcore
mtd                   106496  5 cmdlinepart,ofpart,nandcore,spinand
pwm_fan                24576  0
libdes                 36864  1 crypto_safexcel
auxadc_thermal         24576  0
ramoops                40960  0
pwm_mediatek           12288  1
reed_solomon           28672  1 ramoops
leds_gpio              12288  0
uio_pdrv_genirq        16384  0
uio                    28672  1 uio_pdrv_genirq
binfmt_misc            28672  1
sch_fq_codel           24576  1
nvme_fabrics           36864  0
nvme_keyring           20480  1 nvme_fabrics
nvme_core             225280  1 nvme_fabrics
nvme_auth              28672  1 nvme_core
efi_pstore             12288  0
dm_multipath           49152  0
nfnetlink              24576  1
dmi_sysfs              24576  0
ip_tables              36864  0
x_tables               65536  1 ip_tables
autofs4                61440  2
overlay               208896  1
nls_iso8859_1          12288  0
raid10                 81920  0
raid456               217088  0
async_raid6_recov      24576  1 raid456
async_memcpy           16384  2 raid456,async_raid6_recov
async_pq               16384  2 raid456,async_raid6_recov
async_xor              16384  3 async_pq,raid456,async_raid6_recov
async_tx               16384  5 async_pq,async_memcpy,async_xor,raid456,async_raid6_recov
xor                    12288  1 async_xor
xor_neon               16384  1 xor
raid6_pq              110592  3 async_pq,raid456,async_raid6_recov
raid1                  61440  0
raid0                  28672  0
linear                 16384  0
uas                    32768  0
usb_storage            90112  2 uas
phy_mtk_tphy           49152  3
xhci_mtk_hcd           40960  0
nvmem_mtk_efuse        12288  0
spi_mt65xx             32768  0
pcie_mediatek_gen3     32768  0
mtk_rng                12288  0
polyval_ce             12288  0
polyval_generic        12288  1 polyval_ce
i2c_mt65xx             36864  0
ghash_ce               24576  0
sm4                    12288  0
sha2_ce                20480  0
sha256_arm64           24576  1 sha2_ce
sha1_ce                12288  0
gpio_keys              24576  0
fixed                  16384  4
aes_neon_bs            24576  0
aes_neon_blk           28672  1 aes_neon_bs
aes_ce_blk             36864  0
aes_ce_cipher          12288  1 aes_ce_blk

For verification, here is a good bootlog with a recent OpenWRT snapshot: openwrt_bootlog.txt (26.5 KB)

And the lsmod from OpenWRT:

air_en8811h            16384  2 
aquantia               24576  0 
at24                   16384  0 
authenc                12288  2 crypto_safexcel,authencesn
authencesn             12288  0 
cfg80211              315392  4 mt7915e,mt76_connac_lib,mt76,mac80211
compat                 12288  2 mac80211,cfg80211
crypto_safexcel        98304  0 
des_generic            12288  0 
geniv                  12288  1 seqiv
gpio_button_hotplug    16384  0 
leds_gpio              12288  0 
libcrc32c              12288  1 nf_tables
libdes                 20480  2 crypto_safexcel,des_generic
mac80211              618496  3 mt7915e,mt76_connac_lib,mt76
md5                    12288  1 crypto_safexcel
mt76                   77824  2 mt7915e,mt76_connac_lib
mt76_connac_lib        49152  1 mt7915e
mt7915e               139264  0 
nf_conntrack           98304  7 nft_redir,nft_nat,nft_masq,nft_flow_offload,nft_ct,nf_nat,nf_flow_table
nf_defrag_ipv4         12288  1 nf_conntrack
nf_defrag_ipv6         16384  1 nf_conntrack
nf_flow_table          28672  2 nf_flow_table_inet,nft_flow_offload
nf_flow_table_inet     12288  0 
nf_log_syslog          16384  0 
nf_nat                 36864  4 nft_redir,nft_nat,nft_masq,nft_chain_nat
nf_reject_ipv4         12288  2 nft_reject_ipv4,nft_reject_inet
nf_reject_ipv6         12288  2 nft_reject_ipv6,nft_reject_inet
nf_tables             196608208 nft_fib_inet,nf_flow_table_inet,nft_reject_ipv6,nft_reject_ipv4,nft_reject_inet,nft_reject,nft_redir,nft_quota,nft_numgen,nft_nat,nft_masq,nft_log,nft_limit,nft_hash,nft_flow_offload,nft_fib_ipv6,nft_fib_ipv4,nft_fib,nft_ct,nft_chain_nat
nfnetlink              16384  1 nf_tables
nft_chain_nat          12288  2 
nft_ct                 16384  4 
nft_fib                12288  3 nft_fib_inet,nft_fib_ipv6,nft_fib_ipv4
nft_fib_inet           12288  0 
nft_fib_ipv4           12288  1 nft_fib_inet
nft_fib_ipv6           12288  1 nft_fib_inet
nft_flow_offload       12288  0 
nft_hash               12288  0 
nft_limit              12288  5 
nft_log                12288  0 
nft_masq               12288  1 
nft_nat                12288  0 
nft_numgen             12288  0 
nft_quota              12288  0 
nft_redir              12288  0 
nft_reject             12288  3 nft_reject_ipv6,nft_reject_ipv4,nft_reject_inet
nft_reject_inet        12288  2 
nft_reject_ipv4        12288  0 
nft_reject_ipv6        12288  0 
ppp_async              16384  0 
ppp_generic            40960  3 pppoe,ppp_async,pppox
pppoe                  20480  0 
pppox                  12288  1 pppoe
pwm_fan                12288  0 
seqiv                  12288  0 
sha1_ce                12288  0 
sha1_generic           12288  2 crypto_safexcel,sha1_ce
sha512_arm64           16384  0 
slhc                   12288  1 ppp_generic
usb_common             12288  3 xhci_plat_hcd,xhci_hcd,usbcore
usbcore               192512  4 xhci_plat_hcd,xhci_pci,xhci_mtk_hcd,xhci_hcd
xhci_hcd              135168  3 xhci_plat_hcd,xhci_pci,xhci_mtk_hcd
xhci_mtk_hcd           20480  0 
xhci_pci               16384  0 
xhci_plat_hcd          12288  0 

Yeah something is strange with clkmux,not sure where the 255 shift comes from

[    0.445882] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.464264] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    0.476942] ------------[ cut here ]------------
[    0.476961] UBSAN: shift-out-of-bounds in /build/linux-VHJeXi/linux-6.14.0/drivers/clk/mediatek/clk-mux.c:44:8
[    0.476971] shift exponent 255 is too large for 64-bit type 'long unsigned int'
[    0.476978] CPU: 2 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.14.0-15-generic #15-Ubuntu

Possibly ubuntu changed something in kernel which break here.

Can you try without usbsan (idk what exactly this is atm without googling for)?

Why mixing ubuntu with debian?

Is it possible this come from a incompatible device tree? I’m using the Debian devicetree, since I cannot find any devicetree binaries for Ubuntu.

At the moment, I’m not mixing it, only the devicetree. This is stock Ubuntu 25.04 ARM64 ISO, and it’s contenct copied to an Ext4 partiton in a USB stick. So R3mini booting from that, and both initrd and kernel are what provided by Ubuntu.

Device tree normally should be from upstream linux except fixes done by distribution itself. As i said no idea where the error is caused,but it happens in clk-mux driver and so posibly break more than only the unknown usbsan.

I see, thank you. Peeking into the installer’s grub.cfg they actually provide some dtb-s for a few platform so I keep searching.

cat boot/grub/grub.cfg 
set timeout=30

loadfont unicode

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray

set dtb=
set cmdline=
smbios --type 1 --get-string 5 --set system_product_name
smbios --type 1 --get-string 6 --set system_product_version
smbios --type 1 --get-string 19 --set system_sku_number
regexp "ThinkPad X13s.*" "$system_product_version"
if [ $? = 0 ]; then
        cmdline="clk_ignore_unused pd_ignore_unused arm64.nopauth"
        dtb="devicetree /casper/dtbs/sc8280xp-lenovo-thinkpad-x13s.dtb"
fi
if [ "$system_product_version" == "ThinkPad T14s Gen 6" ]; then
        # Workaround for 64GB crashes on T14s
        cutmem 0x8800000000 0x8fffffffff
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e78100-lenovo-thinkpad-t14s.dtb"
fi
if [ "$system_product_version" == "Yoga Slim 7 14Q8X9" ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-lenovo-yoga-slim7x.dtb"
fi
if [ "$system_product_name" == "XPS 13 9345" ]; then
        # Workaround for 64GB crashes. Sigh...
        cutmem 0x8800000000 0x8fffffffff
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-dell-xps13-9345.dtb"
fi
if [ "$system_product_name" == "Galaxy Book4 Edge" ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-samsung-galaxy-book4-edge.dtb"
fi
if [ "$system_product_name" == "Swift SF14-11" ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1p64100-acer-swift-sf14-11.dtb"
fi
if [ "$system_product_name" == "Swift SF14-11T" ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1p64100-acer-swift-sf14-11.dtb"
fi
regexp "ASUS Vivobook S 15.*" "$system_product_name"
if [ $? == 0 ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-asus-vivobook-s15.dtb"
fi
regexp "ASUS Zenbook A14 UX3407Q.*" "$system_product_name"
if [ $? == 0 ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1p42100-asus-zenbook-a14.dtb"
fi
regexp "ASUS Zenbook A14 UX3407R.*" "$system_product_name"
if [ $? == 0 ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-asus-zenbook-a14.dtb"
fi
regexp "HP OmniBook X Laptop 14-.*" "$system_product_name"
if [ $? == 0 ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-hp-omnibook-x14.dtb"
fi
if [ "$system_sku_number" == "Surface_Laptop_7th_Edition_2036" ]; then
        # Workaround for 64GB crashes. Sigh...
        cutmem 0x8800000000 0x8fffffffff
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-microsoft-romulus13.dtb"
fi
if [ "$system_sku_number" == "Surface_Laptop_7th_Edition_2037" ]; then
        # Workaround for 64GB crashes. Sigh...
        cutmem 0x8800000000 0x8fffffffff
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-microsoft-romulus15.dtb"
fi
if [ "$system_product_name" == "CRD" ]; then
        cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
        dtb="devicetree /casper/dtbs/x1e80100-crd.dtb"
fi

menuentry "Try or Install Ubuntu Server" {
	set gfxpayload=keep
	linux	/casper/vmlinuz $cmdline  --- console=tty0
	initrd	/casper/initrd
	$dtb
}
menuentry 'Boot from next volume' {
	exit 1
}
menuentry 'UEFI Firmware Settings' {
	fwsetup
}

I was able to install Ubuntu 25.04 into the NVMe (screenshot below). The issue was not the kernel, but the missing devicetree. It turned out, GRUB cannot pass the devicetree (loaded by u-boot) to the kernel. So you have to manually configure the devicetree as done for ARM based notebooks in my previous comment. For some reason I assumed the devicetree passed througth all the way to the kernel like u-boot → GRUB → kernel but thats NOT the case. Interestingly, GRUB has a command to list efi memory maps which show the devicetree, but still not passed automatically.

So I made the following grub menuentry into the grub.cfg:

menuentry "Try or Install Ubuntu Server on MT7986" {
    set gfxpayload=keep
    cmdline="clk_ignore_unused pd_ignore_unused cma=128M"
    linux	/casper/vmlinuz $cmdline  --- console=ttyS0,115200n8
    initrd	/casper/initrd
    devicetree /casper/dtbs/mt7986.dtb
}

The mt7986.dtb is just the copied/renamed from Debian, probably extracted from OpenWRT FIT image would do the trick as well. I made the casper/dtbs/ folder (playing safe, I was not sure what is GRUB’s root or workdir, the casper folder was straightforward).

So I have Ubuntu installed on the NVMe, but unfortunately I can’t load it. OpenWRT u-boot does not support PCIe NVMe, and bootefi missing from your u-boot fork. I’ll try to load it manually with booti but it would be really convenient to use EFI for both installing and loading the installed Linux just like on a PC :slight_smile: I sent a PR (not tested) to your fork enabling the EFI options, consider merge it if its makes sense, thanks in advance!

Devicetree is board specific so it should not have the name of soc only…but your system is now boooting correctly?