Hi everyone, I’m currently working on the BPI-R3 Mini (MT7986, eMMC DDR4 variant) and trying to run a full Debian or Ubuntu system on it instead of OpenWrt/ImmortalWrt. The main issue I’m facing is PCIe initialization — specifically missing clock definitions in the device tree (pcie, pcie_pipe, pcie_aux, top_axi). Because of this, the PCIe controller probes incorrectly and NVMe SSDs cannot bind. Before I start rebuilding the kernel and DTS from scratch, I’d like to ask: Has anyone already built a working Debian or Ubuntu kernel specifically for the R3 Mini? Is there a maintained device tree for MT7986 BPI-R3 Mini that fully enables PCIe (including NVMe)? Does any prebuilt image exist with fully working PCIe/NVMe support? I’m currently using: ImmortalWrt preloader + FIP Debian root filesystem MT7986 5.15/6.x kernel attempts Any pointers to a working kernel config, DTS patch, or full image would be greatly appreciated. Thanks in advance!
You could use my buildscript (which does not yet support r3mini) with r3mini boot file (im.gz) for emmc
And build for R3 to download the right kernel file from my linux repo.
I have it here:
https://ftp.woudstra.mywire.org/images/
Instructions here:
You may want to check that whole page about the buildscript. It is now more menu-driven, so it is quit easy to use it. No need to use commandline options anymore for basic setup.
The script has had a major overhaul, and I’m still ironing out the bugs out of the script/images, caused by this overhaul. Should basically be ok, maybe some small issues, so you may need a uart cable.
Also Ubuntu is quite new, and I do not test that very much. It started out as an ArchlinuxARM installer, but it can do more now. Debian I’ve just added.
=Hi Eric,
I’m working with a Banana Pi R3 Mini and trying to install Ubuntu on the eMMC using the available Ubuntu images (bpir3m-ubuntu-emmc-rt.img.gz / ap.img.gz).
Here is what I observed:
-
I flashed the Ubuntu image directly to eMMC using:
gunzip -c bpir3m-ubuntu-emmc-rt.img.gz | dd of=/dev/mmcblk0 bs=4M conv=fsync
-
After reboot, the boot process fails very early in BL2 with:
ERROR: Partition ‘fip’ not found ERROR: FIP boot source initialization failed PANIC at PC
-
From the logs it looks like BL2 (OpenWrt v2025.07.11) expects a GPT layout containing a “fip” partition, but the Ubuntu image overwrites the disk layout and removes that partition.
-
I attempted to restore the bootloader by manually writing the OpenWrt FIP:
dd if=openwrt-25.12.0-…-emmc-bl31-uboot.fip of=/dev/mmcblk0 bs=512 seek=13312
After that, BL2 → BL31 → U-Boot loads correctly and the OpenWrt U-Boot menu appears.
-
However, if I then write the Ubuntu image again (either with dd or U-Boot gzwrite), the same issue returns because the GPT layout containing the “fip” partition gets overwritten again.
So the boot chain becomes:
BootROM → BL2 → search for “fip” partition → panic
My questions:
- Does the Ubuntu image need to be installed into a specific partition instead of writing the entire disk?
- Does your build/install script keep the OpenWrt bootloader partition layout when installing Ubuntu?
- Is there a recommended method to install Ubuntu to eMMC on the R3 Mini without breaking the FIP partition?
I do have UART access and can test different procedures if needed.
Thanks for the work on the build system.
When using my image, please follow my readme, specially these steps:
You are missing the steps to write my bl2 to mmcblk0boot0. (You do not need the mmc bootpart command, this is already set correctly it would seem)
In my images, I use a boot-partition instead of a fip-partition (default, but can change to fip anytime if you really want). My fork of ATF can handle fat32 as well as fip
Hi Eric,
I’m trying to install your Ubuntu image on a Banana Pi R3 Mini (2GB RAM) using the method described in your README:
R3-MINI & R4 Build/Install emmc version using image (openwrt on nand)
I booted into OpenWrt from NAND and followed the exact steps.
USB contents:
bpir3m-ubuntu-emmc-ap.img.gz
bpir3m-ubuntu-emmc-rt.img.gz
Commands executed:
echo 0 > /sys/block/mmcblk0boot0/force_ro
gunzip -c /mnt/sda1/bpir3m-ubuntu-emmc-ap.img.gz | dd of=/dev/mmcblk0 bs=4M conv=fsync
dd if=/dev/mmcblk0 of=/dev/mmcblk0boot0 bs=17K seek=1 count=32 conv=fsync
mmc bootpart enable 1 1 /dev/mmcblk0
Verification:
mmc extcsd read /dev/mmcblk0 | grep PARTITION_CONFIG
Boot configuration bytes [PARTITION_CONFIG: 0x48]
Partition layout:
mmcblk0
├─mmcblk0p1 1007K
├─mmcblk0p2 255M
└─mmcblk0p3 7G
However when switching DIP to eMMC boot (only switch D down) the board halts immediately after BL2:
F0: 102B 0000
FA: 1040 0000
FA: 1040 0000 [0200]
F9: 103F 0000
F3: 0000 0000 [0200]
V0: 7027 6006 [0001]
00: 1017 0000
F3: 4002 0000 [0200]
01: 102A 0001
02: 4002 0000
BP: 2000 00C0 [0001]
EC: 0000 0000 [2000]
T0: 0000 0183 [010F]
System halt!
NAND OpenWrt boots normally and eMMC is detected correctly.
Do you know what could cause BL2 to halt like this on the R3 Mini?
Is there any additional step needed for the emmc boot image or is this possibly an image issue?
Thanks for your work on this project.
Looks right, so I’ll need to check it in the coming week…
This is not bl2…this is bootrom. Have you written bl2 to boot0 and have set bootpart enable / partition config and maybe bootbus in uboot?
Did you download a fresh copy? I did recently fix some problems.
If that also fails you could try manually extract the emmc.bin from
And write to boot0… Plain dd, no skip
Hi Eric,
I followed your suggestion and extracted the ATF package and wrote emmc.bin directly to /dev/mmcblk0boot0.
Steps I did:
-
Extracted from bpir3m-atf-git-2-2.13r46.17174.3c1f864e2-1-aarch64.pkg.tar.xz
-
Wrote to boot0: dd if=emmc.bin of=/dev/mmcblk0boot0 bs=512 conv=fsync
-
Boot configuration: mmc extcsd read /dev/mmcblk0 | grep PARTITION_CONFIG → PARTITION_CONFIG: 0x48
-
After reboot the board loads: BL2 v2.13.0 Alarm ATF BPIR3M EMMC
Then it loads kernel and initramfs from the eMMC image and Ubuntu 24.04 boots successfully.
Boot chain now works: BootROM → BL2 → BL31 → Linux kernel → Ubuntu
Thanks for the help!
Hi
I managed to get Ubuntu 24.04 booting from eMMC on the BPI-R3 Mini by writing the ATF bootloader directly to the eMMC boot0 partition.
The working boot chain is now:
BootROM → BL2 (ATF in boot0) → BL31 → U-Boot → Linux → Ubuntu
Steps that fixed the boot:
dd if=emmc.bin of=/dev/mmcblk0boot0 bs=512 conv=fsync
PARTITION_CONFIG shows 0x48 and the system now boots reliably from eMMC.
Current system:
• Ubuntu 24.04 • Kernel: 6.18.15-bpir • Docker running • CasaOS installed • RM520N 5G modem working via QMI (router mode)
My goal for this setup is to use the R3 Mini as:
• 5G router • Docker host • CasaOS server • NVMe NAS (Immich photo backup)
The current blocker is PCIe/NVMe.
Kernel logs show:
mtk-pcie-gen3 11280000.pcie: PCIe link down probe with driver mtk-pcie-gen3 failed with error -110
So the PCIe controller initializes but no device is detected.
Questions:
- Does the Ubuntu kernel I’m using lack the correct DTB for NVMe on the R3 Mini?
- Is there a recommended kernel build that already enables PCIe/NVMe for this board?
- Does the WiFi PCIe device need to be disabled to use the NVMe slot?
If helpful I can provide full dmesg or test patched DTBs.
Thanks again for your build system work.
So there was a small typo in the readme, it should be skip instead of seek.
The correct commands:
echo 0 > /sys/block/mmcblk0boot0/force_ro
gunzip -c /mnt/sda1/bpir.img.gz | dd of=/dev/mmcblk0 bs=4M conv=fsync,sparse
dd if=/dev/mmcblk0 of=/dev/mmcblk0boot0 bs=17K skip=1 count=32 conv=fsync
mmc bootpart enable 1 1 /dev/mmcblk0
I have on the R3 (not mini)
[root@bpir3 ~]# lspci
00:00.0 PCI bridge: MEDIATEK Corp. Device 1f32 (rev 01)
01:00.0 Non-Volatile memory controller: Sandisk Corp PC SN740 NVMe SSD (DRAM-less) (rev 01)
[root@bpir3 ~]# uname -r
6.18.13-bpir
The R3 mini should be the same PCIe wise. I did have the same nvme in the R3mini before, it did work (the kernel was older, but also rolling-stable).
I do not think the difference is 13 to 15. Maybe your hardware is not compatible? How about after a reboot?
Is your nvme in the compatibility list?
https://github.com/frank-w/BPI-Router-Linux/wiki/BPI%E2%80%90Router%E2%80%90Linux-Wiki
Hi Eric,
I investigated the PCIe initialization issue on the BPI-R3 Mini (MT7986). Previously the kernel failed during PCIe probe with:
mtk-pcie-gen3 … Failed to get clk index: 0 mtk-pcie-gen3 … failed to get clocks
To resolve this I modified the device tree used by the system.
What I changed:
Original PCIe node:
clocks = <0x04 0x34 0x04 0x33 0x04 0x35 0x04 0x36>; clock-names = “pl_250m”, “tl_26m”, “peri_26m”, “top_133m”;
Modified PCIe node:
clocks = <0x04 0x34 0x04 0x33 0x04 0x35 0x04 0x36>; clock-names = “pcie”, “pcie_peri”, “pcie_pipe”, “pcie_aux”;
Steps performed:
• Booted BananaWRT from SPI-NAND • Mounted the Ubuntu eMMC boot partition • Extracted the DTB (mt7986a-bananapi-bpi-r3-mini.dtb) • Decompiled it with dtc to DTS • Edited the PCIe node clock-names • Recompiled the DTS to DTB • Replaced the original DTB with the modified one
Result after reboot:
The PCIe controller now initializes successfully and the driver loads:
mtk-pcie-gen3 11280000.pcie: host bridge /soc/pcie@11280000 ranges
The link training proceeds but currently reports:
PCIe link down, LTSSM state: detect.quiet
This test was done without an NVMe device installed, so the controller is active but no endpoint is detected. I will test again with the NVMe SSD installed to confirm link-up and device enumeration.
I worked on the PCIe initialization issue on the BPI-R3 Mini (MT7986) where the kernel previously failed with the clock error and the PCIe driver probe did not start.
Steps I performed:
- Booted the system from SPI-NAND (BananaWRT) so I could safely modify the eMMC boot files.
- Mounted the eMMC boot partition (
/mnt/mmcblk0p2) which contains the kernel image and DTB used by Ubuntu. - Copied the original DTB:
dtbs/linux-bpir-git/mt7986a-bananapi-bpi-r3-mini.dtb
- Decompiled it using
dtc:
dtc -I dtb -O dts -o r3mini.dts mt7986a-bananapi-bpi-r3-mini.dtb
- Located the PCIe node
/soc/pcie@11280000in the DTS. - Adjusted the clock configuration in the PCIe node to avoid the clock index error during initialization.
- Recompiled the DTS back to DTB:
dtc -I dts -O dtb -o mt7986a-bananapi-bpi-r3-mini-fixed.dtb r3mini.dts
- Backed up the original DTB and replaced it with the modified one.
After booting with the modified DTB, the kernel now initializes the PCIe controller correctly and reaches the link training stage.
Current dmesg output shows:
mtk-pcie-gen3 11280000.pcie: host bridge /soc/pcie@11280000 ranges mtk-pcie-gen3 11280000.pcie: PCIe link down, LTSSM state: detect.quiet
This is expected because the NVMe device was not installed during the test. The controller is now active and attempting link training.
Next step is to test again with the NVMe SSD installed to confirm the PCIe link comes up and the NVMe device enumerates.
So the original clock/probe failure is resolved and PCIe initialization now proceeds to link detection.
[root@bpir3 ~]# dtc -fI fs -O dts /proc/device-tree/soc/pcie\@11280000/
<stdout>: ERROR (name_properties): /: "name" property is incorrect ("pcie" instead of base node name)
Warning: Input tree has errors, output forced
/dts-v1/;
/ {
pinctrl-names = "default";
#address-cells = <0x03>;
phy-names = "pcie-phy";
bus-range = <0x00 0xff>;
pinctrl-0 = <0x17>;
clock-names = "pl_250m", "tl_26m", "peri_26m", "top_133m";
reg-names = "pcie-mac";
interrupts = <0x00 0xa8 0x04>;
clocks = <0x04 0x34 0x04 0x33 0x04 0x35 0x04 0x36>;
interrupt-map = <0x00 0x00 0x00 0x01 0x16 0x00 0x00 0x00 0x00 0x02 0x16 0x01 0x00 0x00 0x00 0x03 0x16 0x02 0x00 0x00 0x00 0x04 0x16 0x03>;
#size-cells = <0x02>;
device_type = "pci";
interrupt-map-mask = <0x00 0x00 0x00 0x07>;
compatible = "mediatek,mt7986-pcie", "mediatek,mt8192-pcie";
ranges = <0x82000000 0x00 0x20000000 0x00 0x20000000 0x00 0x10000000>;
#interrupt-cells = <0x01>;
status = "okay";
phys = <0x15 0x02>;
reg = <0x00 0x11280000 0x00 0x4000>;
phandle = <0x4a>;
name = "pcie";
interrupt-controller {
#address-cells = <0x00>;
#interrupt-cells = <0x01>;
phandle = <0x16>;
interrupt-controller;
};
};
[root@bpir3 ~]# lspci
00:00.0 PCI bridge: MEDIATEK Corp. Device 1f32 (rev 01)
01:00.0 Non-Volatile memory controller: Sandisk Corp PC SN740 NVMe SSD (DRAM-less) (rev 01)
[root@bpir3 ~]# dmesg | grep pci
[ 0.350045] /soc/pcie@11280000: Fixed dependency cycle(s) with /soc/pcie@11280000/interrupt-controller
[ 3.282450] mtk-pcie-gen3 11280000.pcie: host bridge /soc/pcie@11280000 ranges:
[ 3.289792] mtk-pcie-gen3 11280000.pcie: MEM 0x0020000000..0x002fffffff -> 0x0020000000
[ 3.427927] mtk-pcie-gen3 11280000.pcie: PCI host bridge to bus 0000:00
[ 3.434582] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 3.457959] pci_bus 0000:00: root bus resource [mem 0x20000000-0x2fffffff]
[ 3.475217] pci 0000:00:00.0: [14c3:1f32] type 01 class 0x060400 PCIe Root Port
[ 3.482572] pci 0000:00:00.0: BAR 0 [mem 0x00000000-0x00007fff 64bit]
[ 3.499267] pci 0000:00:00.0: PCI bridge to [bus 00]
[ 3.504236] pci 0000:00:00.0: bridge window [io 0x0000-0x0fff]
[ 3.510320] pci 0000:00:00.0: bridge window [mem 0x00000000-0x000fffff]
[ 3.527422] pci 0000:00:00.0: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 3.535229] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[ 3.552626] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 3.568543] pci 0000:01:00.0: [15b7:5015] type 00 class 0x010802 PCIe Endpoint
[ 3.582360] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
[ 3.591886] pci 0000:01:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x2 link at 0000:00:00.0 (capable of 63.012 Gb/s with 16.0 GT/s PCIe x4 link)
[ 3.630032] pci 0000:01:00.0: ASPM: default states L1
[ 3.640241] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 3.655924] pci 0000:00:00.0: bridge window [mem 0x20000000-0x200fffff]: assigned
[ 3.669089] pci 0000:00:00.0: BAR 0 [mem 0x20100000-0x20107fff 64bit]: assigned
[ 3.681594] pci 0000:01:00.0: BAR 0 [mem 0x20000000-0x20003fff 64bit]: assigned
[ 3.696254] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 3.708220] pci 0000:00:00.0: bridge window [mem 0x20000000-0x200fffff]
[ 3.718727] pci_bus 0000:00: resource 4 [mem 0x20000000-0x2fffffff]
[ 3.728960] pci_bus 0000:01: resource 1 [mem 0x20000000-0x200fffff]
[ 3.743495] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[ 3.754760] pcieport 0000:00:00.0: PME: Signaling with IRQ 139
[ 3.759063] mt7986a-pinctrl 1001f000.pinctrl: pin GPIO_4 already requested by 11280000.pcie; cannot claim for pinctrl_moore:521
[ 3.763187] pcieport 0000:00:00.0: AER: enabled with IRQ 139
[ 3.780692] nvme nvme0: pci function 0000:01:00.0
The pcie node of the R3 and R3-mini are from the same source-file.
I really do not see why the clock names need to change…
This is the dtb that is being used at boot:
[root@bpir3 ~]# cat /boot/bootcfg/atfdtb
/boot/dtbs/linux-bpir-git/mt7986a-bananapi-bpi-r3-atf.dtb
Except yours is r3m-atf.dtb.
It is constructed from /boot/dtbs/linux-bpir-git/mt7986a-bananapi-bpi-r3.dtb and the dts’s at
[root@bpir3 ~]# ls /boot/dtbos/
bootargs.dts emmc-clock-fix.dts memory.dts nand-enable.dts sdmmc-enable.dts
with:
bpir-toobox --write2dtb
See:
bpir-toobox --help
So if you need to change something, add a dts in /boot/bootcfg and run
bpir-toobox --write2dtb
So the dtb you used to boot did not change, if r3m-atf.dtb was not changed by the toolbox.
Hi Eric,
I did some additional testing on the BPI-R3 Mini (MT7986) regarding the PCIe / NVMe initialization.
During normal boot the PCIe controller fails early with:
mtk-pcie 11280000.pcie: PCIe link down, ltssm reg val: 0x1000001 mtk-pcie: probe of 11280000.pcie failed with error -110
Relevant boot log section:
mtk-pcie 11280000.pcie: host bridge /pcie@11280000 ranges mtk-pcie 11280000.pcie: Parsing ranges property phy [email protected]: u3 efuse - intr 29, rx_imp 10, tx_imp f mtk-pcie 11280000.pcie: PCIe link down, ltssm reg val: 0x1000001
Clock status shows the PCIe clocks exist but are not active yet:
cat /sys/kernel/debug/clk/clk_summary | grep pcie
infra_ipcier 26000000 pcie_phy_sel 26000000 infra_pcie 125000000 infra_ipcie_pipe 40000000
To investigate further I entered OpenWrt failsafe mode and manually reinitialized the PCIe controller.
Commands executed:
echo 1 > /sys/bus/pci/rescan dmesg | grep nvme
ls /sys/bus/platform/devices | grep pcie
Result:
10003000.wed_pcie 11280000.pcie pcie-phy@11c00000
Then I manually re-bound the PCIe driver:
echo 11280000.pcie > /sys/bus/platform/drivers/mtk-pcie/unbind echo 11280000.pcie > /sys/bus/platform/drivers/mtk-pcie/bind
After rebinding the driver the PCIe controller initializes correctly.
Kernel output:
mtk-pcie 11280000.pcie: PCI host bridge to bus 0000:00 pci_bus 0000:00: scanning bus pci 0000:00:00.0: [14c3:1f32] type 01 class 0x060400
Then the NVMe device is detected:
pci 0000:01:00.0: [c0a9:5426] type 00 class 0x010802 nvme nvme0: pci function 0000:01:00.0 nvme nvme0: allocated 64 MiB host memory buffer nvme nvme0: 4/0/0 default/read/poll queues nvme0n1: p1 p2
The disk appears correctly in the system:
lsblk
nvme0n1 1.8T ├─nvme0n1p1 16M └─nvme0n1p2 1.8T
Bandwidth negotiation:
8.000 Gb/s available PCIe bandwidth, limited by 5 GT/s x2 link
So this matches the expected PCIe Gen2 x2 limit of the MT7986 controller.
Hardware used:
• BPI-R3 Mini (2GB RAM) • OpenWrt 21.02.3 running from eMMC • Kernel 6.18.x bpir build • NVMe SSD 2TB (PCIe 4.0 x4 drive negotiating to Gen2 x2) • RM520N-GL 5G modem working via QMI
PCIe and NVMe are therefore confirmed working after rebinding the driver.
The remaining issue appears to be that the initial PCIe probe during early boot fails link training, while rebinding the driver later works correctly.
Images and logs used for testing are available here:
https://drive.google.com/drive/folders/1lENckxpiZa5bdkPAA6FfbGx1IlLJqHql?usp=sharing
My goal for this setup is:
• 5G router • Docker host • CasaOS server • NVMe Home NAS
If useful I can provide full dmesg logs or the modified DTB I used.
Thanks again for your work on the build system.
Try adding a quirk like this, but then for your hardware:
From bcb06dbc8dfb2418a6694f8d9650f7bfcd2b376a Mon Sep 17 00:00:00 2001
From: Eric Woudstra <[email protected]>
Date: Fri, 12 Jan 2024 15:01:30 +0100
Subject: [PATCH] Add quirk for WD-SN740
On cold-boot sometime get:
nvme nvme0: Device not ready; aborting initialisation, CSTS=0x0
This quirk fixes that.
Changes to be committed:
modified: drivers/nvme/host/pci.c
---
drivers/nvme/host/pci.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index d86f2565a92cac..b184f5d553470f 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -4036,6 +4036,8 @@ static const struct pci_device_id nvme_id_table[] = {
.driver_data = NVME_QUIRK_BOGUS_NID, },
{ PCI_DEVICE(0x10ec, 0x5765), /* TEAMGROUP MP33 2TB SSD */
.driver_data = NVME_QUIRK_BOGUS_NID, },
+ { PCI_DEVICE(0x15b7, 0x5015), /* TEST QUIRK FOR MY WD SN740 */
+ .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0x0061),
.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0x0065),
Or maybe there is another thing to quirk, as your hardware causes errors even earlier then this nvme
Otherwise you need to add into the hook in the initramfs, to reinitialize the pci-bus
Or just get another nvme and use this one for something else