Best boot strategy for BPI-R4: eMMC/NAND -> NVMe with SD fallback + Frank’s U-Boot?

I believe my changes were not propagated. I’ve created a new patch and I’m rebuilding everything. I’ll update about the result.

Btw. I’ve found wrong/missing config: PCIE_MEDIATEK_GEN3. PCIE_MEDIATEK supports only Gen2.

I have good and bad news.

My changes work. Kind of :wink: At least nvme is an available command.

MT7988> pci enum; nvme scan;nvme info

returns nothing.

MT7988> nvme detail

as well.

MT7988> dm tree
 Class     Seq    Probed  Driver                Name
-----------------------------------------------------------
 root          0  [ + ]   root_driver           root_driver
 clk           0  [   ]   fixed_clock           |-- dummy40m
 clk           1  [ + ]   fixed_clock           |-- dummy208m
 firmware      0  [   ]   psci                  |-- psci
 syscon        0  [   ]   syscon                |-- hwver
 syscon        1  [ + ]   syscon                |-- watchdog@1001c000
 clk           2  [ + ]   mt7988-clock-fixed-p  |-- apmixedsys@1001e000
 clk           3  [ + ]   mt7988-clock-topckge  |-- topckgen@1001b000
 pinctrl       0  [ + ]   mt7988_pinctrl        |-- pinctrl@1001f000
 gpio          0  [ + ]   mediatek_gpio         |   |-- mediatek_gpio
 pinconfig     0  [   ]   pinconfig             |   |-- i2c1-pins
 pinconfig     1  [   ]   pinconfig             |   |   `-- mux
 pinconfig     2  [   ]   pinconfig             |   |-- pwm-pins
 pinconfig     3  [   ]   pinconfig             |   |   `-- mux
 pinconfig     4  [ + ]   pinconfig             |   |-- spi0-pins
 pinconfig     5  [   ]   pinconfig             |   |   `-- mux
 pinconfig     6  [   ]   pinconfig             |   |-- mmc0default
 pinconfig     7  [   ]   pinconfig             |   |   |-- mux
 pinconfig     8  [   ]   pinconfig             |   |   |-- conf-cmd-dat
 pinconfig     9  [   ]   pinconfig             |   |   |-- conf-clk
 pinconfig    10  [   ]   pinconfig             |   |   |-- conf-dsl
 pinconfig    11  [   ]   pinconfig             |   |   `-- conf-rst
 pinconfig    12  [ + ]   pinconfig             |   `-- mmc1default
 pinconfig    13  [   ]   pinconfig             |       |-- mux
 pinconfig    14  [   ]   pinconfig             |       |-- conf-cmd-dat
 pinconfig    15  [   ]   pinconfig             |       `-- conf-clk
 clk           4  [   ]   mt7988-clock-sgmiisy  |-- syscon@10060000
 clk           5  [   ]   mt7988-clock-sgmiisy  |-- syscon@10070000
 syscon        2  [ + ]   syscon                |-- syscon@10080000
 syscon        3  [   ]   syscon                |-- syscon@10081000
 clk           6  [   ]   fixed_clock           |-- dummy12m
 usb           0  [   ]   xhci-mtk              |-- xhci@11200000
 phy           0  [   ]   mtk-tphy              |-- usb-phy@11c50000
 syscon        4  [ + ]   syscon                |-- syscon@11f20000
 syscon        5  [   ]   syscon                |-- syscon@11f30000
 syscon        6  [ + ]   syscon                |-- syscon@11f40000
 syscon        7  [ + ]   syscon                |-- topmisc@11d10000
 clk           7  [ + ]   mt7988-clock-infracf  |-- infracfg@10001000
 serial        0  [ + ]   serial_mtk            |-- serial@11000000
 pwm           0  [   ]   mtk_pwm               |-- pwm@10048000
 spi           0  [ + ]   mtk_spim              |-- spi@1100a000
 mtd           0  [ + ]   spi_nand              |   `-- spi_nand@0
 mmc           0  [ + ]   mtk_sd                |-- mmc@11230000
 blk           0  [ + ]   mmc_blk               |   |-- [email protected]
 partition     0  [ + ]   blk_partition         |   |   |-- [email protected]:1
 partition     1  [ + ]   blk_partition         |   |   |-- [email protected]:2
 partition     2  [ + ]   blk_partition         |   |   |-- [email protected]:3
 partition     3  [ + ]   blk_partition         |   |   |-- [email protected]:4
 partition     4  [ + ]   blk_partition         |   |   |-- [email protected]:5
 partition     5  [ + ]   blk_partition         |   |   |-- [email protected]:6
 partition     6  [ + ]   blk_partition         |   |   `-- [email protected]:7
 bootdev       0  [   ]   mmc_bootdev           |   `-- [email protected]
 clk           8  [ + ]   mt7988-clock-ethdma   |-- syscon@15000000
 reset         0  [ + ]   mediatek_reset        |   |-- reset
 syscon        8  [ + ]   syscon                |   `-- syscon@15000000
 clk           9  [ + ]   mt7988-clock-ethwarp  |-- syscon@15031000
 reset         1  [ + ]   mediatek_reset        |   |-- reset
 syscon        9  [ + ]   syscon                |   `-- syscon@15031000
 ethernet      0  [ + ]   mtk-eth               |-- ethernet@15110100
 bootdev       1  [   ]   eth_bootdev           |   `-- [email protected]
 regulator     0  [ + ]   regulator_fixed       |-- regulator-3p3v
 regulator     1  [ + ]   regulator_fixed       |-- regulator-1p8v
 button        0  [ + ]   button_gpio           |-- keys
 button        1  [ + ]   button_gpio           |   `-- wps
 nop           0  [   ]   gpio_led_wrap         |-- leds
 led           0  [   ]   gpio_led              |   |-- led-green
 led           1  [   ]   gpio_led              |   `-- led-blue
 bootstd       0  [   ]   bootstd_drv           |-- bootstd
 bootmeth      0  [   ]   bootmeth_extlinux     |   |-- extlinux
 bootmeth      1  [   ]   bootmeth_efi_mgr      |   |-- efi_mgr
 bootmeth      2  [   ]   bootmeth_efi          |   |-- efi
 bootmeth      3  [   ]   bootmeth_pxe          |   |-- pxe
 bootmeth      4  [   ]   vbe_simple            |   `-- vbe_simple
 efi           0  [   ]   EFI block driver      |-- efi
 simple_bus    0  [   ]   cpu_bus               `-- cpus
 cpu           0  [   ]   mtk-cpu                   |-- cpu@0
 cpu           1  [   ]   mtk-cpu                   |-- cpu@1
 cpu           2  [   ]   mtk-cpu                   |-- cpu@2
 cpu           3  [   ]   mtk-cpu                   `-- cpu@3

My changes: package/boot/uboot-mediatek/patches/450a-add-bpi-r4-nvme-pcie.patch:

diff --git a/configs/mt7988a_bananapi_bpi-r4-emmc_defconfig b/configs/mt7988a_bananapi_bpi-r4-emmc_defconfig
index 87c0932..a2b81ad 100644
--- a/configs/mt7988a_bananapi_bpi-r4-emmc_defconfig
+++ b/configs/mt7988a_bananapi_bpi-r4-emmc_defconfig
@@ -16,6 +16,9 @@ CONFIG_DEBUG_UART_BASE=0x11000000
 CONFIG_DEBUG_UART_CLOCK=40000000
 CONFIG_ENV_OFFSET_REDUND=0x440000
 CONFIG_PCI=y
+CONFIG_NVME=y
+CONFIG_NVME_PCI=y
+CONFIG_BLK=y
 CONFIG_DEBUG_UART=y
 CONFIG_AHCI=y
 CONFIG_FIT=y
@@ -73,6 +76,7 @@ CONFIG_CMD_FS_UUID=y
 CONFIG_CMD_UBI=y
 CONFIG_CMD_UBI_RENAME=y
 CONFIG_CMD_MEMSIZE=y
+CONFIG_CMD_NVME=y
 CONFIG_OF_EMBED=y
 CONFIG_ENV_OVERWRITE=y
 CONFIG_ENV_IS_IN_MMC=y
@@ -112,6 +116,7 @@ CONFIG_MTD_UBI_FASTMAP=y
 CONFIG_PHY_FIXED=y
 CONFIG_MEDIATEK_ETH=y
 CONFIG_PCIE_MEDIATEK=y
+CONFIG_PCIE_MEDIATEK_GEN3=y
 CONFIG_PHY=y
 CONFIG_PHY_MTK_TPHY=y
 CONFIG_PINCTRL=y
@@ -123,6 +128,7 @@ CONFIG_DM_REGULATOR=y
 CONFIG_DM_REGULATOR_FIXED=y
 CONFIG_DM_REGULATOR_GPIO=y
 CONFIG_DM_PWM=y
+CONFIG_DM_PCI=y
 CONFIG_PWM_MTK=y
 CONFIG_RAM=y
 CONFIG_SCSI=y
diff --git a/configs/mt7988a_bananapi_bpi-r4-sdmmc_defconfig b/configs/mt7988a_bananapi_bpi-r4-sdmmc_defconfig
index 027b100..325e2ad 100644
--- a/configs/mt7988a_bananapi_bpi-r4-sdmmc_defconfig
+++ b/configs/mt7988a_bananapi_bpi-r4-sdmmc_defconfig
@@ -16,6 +16,9 @@ CONFIG_DEBUG_UART_BASE=0x11000000
 CONFIG_DEBUG_UART_CLOCK=40000000
 CONFIG_ENV_OFFSET_REDUND=0x440000
 CONFIG_PCI=y
+CONFIG_NVME=y
+CONFIG_NVME_PCI=y
+CONFIG_BLK=y
 CONFIG_DEBUG_UART=y
 CONFIG_AHCI=y
 CONFIG_FIT=y
@@ -72,6 +75,7 @@ CONFIG_CMD_FS_GENERIC=y
 CONFIG_CMD_FS_UUID=y
 CONFIG_CMD_UBI=y
 CONFIG_CMD_UBI_RENAME=y
+CONFIG_CMD_NVME=y
 CONFIG_OF_EMBED=y
 CONFIG_ENV_OVERWRITE=y
 CONFIG_ENV_IS_IN_MMC=y
@@ -111,6 +115,7 @@ CONFIG_MTD_UBI_FASTMAP=y
 CONFIG_PHY_FIXED=y
 CONFIG_MEDIATEK_ETH=y
 CONFIG_PCIE_MEDIATEK=y
+CONFIG_PCIE_MEDIATEK_GEN3=y
 CONFIG_PHY=y
 CONFIG_PHY_MTK_TPHY=y
 CONFIG_PINCTRL=y
@@ -122,6 +127,7 @@ CONFIG_DM_REGULATOR=y
 CONFIG_DM_REGULATOR_FIXED=y
 CONFIG_DM_REGULATOR_GPIO=y
 CONFIG_DM_PWM=y
+CONFIG_DM_PCI=y
 CONFIG_PWM_MTK=y
 CONFIG_RAM=y
 CONFIG_SCSI=y
diff --git a/configs/mt7988a_bananapi_bpi-r4-snand_defconfig b/configs/mt7988a_bananapi_bpi-r4-snand_defconfig
index fb7924f..b296574 100644
--- a/configs/mt7988a_bananapi_bpi-r4-snand_defconfig
+++ b/configs/mt7988a_bananapi_bpi-r4-snand_defconfig
@@ -13,6 +13,9 @@ CONFIG_PRE_CON_BUF_ADDR=0x4007EF00
 CONFIG_DEBUG_UART_BASE=0x11000000
 CONFIG_DEBUG_UART_CLOCK=40000000
 CONFIG_PCI=y
+CONFIG_NVME=y
+CONFIG_NVME_PCI=y
+CONFIG_BLK=y
 CONFIG_DEBUG_UART=y
 CONFIG_AHCI=y
 CONFIG_FIT=y
@@ -70,6 +73,7 @@ CONFIG_CMD_FS_UUID=y
 CONFIG_CMD_UBI=y
 CONFIG_CMD_UBI_RENAME=y
 CONFIG_CMD_MEMSIZE=y
+CONFIG_CMD_NVME=y
 CONFIG_OF_EMBED=y
 CONFIG_ENV_OVERWRITE=y
 CONFIG_ENV_IS_IN_UBI=y
@@ -112,6 +116,7 @@ CONFIG_MTD_UBI_FASTMAP=y
 CONFIG_PHY_FIXED=y
 CONFIG_MEDIATEK_ETH=y
 CONFIG_PCIE_MEDIATEK=y
+CONFIG_PCIE_MEDIATEK_GEN3=y
 CONFIG_PHY=y
 CONFIG_PHY_MTK_TPHY=y
 CONFIG_PINCTRL=y
@@ -123,6 +128,7 @@ CONFIG_DM_REGULATOR=y
 CONFIG_DM_REGULATOR_FIXED=y
 CONFIG_DM_REGULATOR_GPIO=y
 CONFIG_DM_PWM=y
+CONFIG_DM_PCI=y
 CONFIG_PWM_MTK=y
 CONFIG_RAM=y
 CONFIG_SCSI=y

Any suggestions?

is the mtk_pcie_gen3 driver enabled in uboot?

else check dt nodes if all except the pcie2 (key-b) are enabled

Also enable dm_pci and cmd_pci so you can look at lower layer

I do not see enabling of pcie nodes in the r4 uboot dts…also there is no patch adding the pcie support to mt7988,but maybe it is already upstream.

Make sure this is there in your uboot source and enabling also for the right r4 dts (here only rfb)

1 Like

if you just want to be able to boot from NVME - my suggestion is to take an existing image (it could be with openwrt, or linux , doesnt matter ), find the one that supports nvme , update bootloader from this image or flash it to nand/emmc , adjust uboot config with your specific boot sequence for your specific setup

btw. i try to get nvme working in openwrt for R4+R4Pro

my own uboot works so far here for R4pro with nvme in cn14 (cn13 depends on pcie2 with special phy so not available in uboot)… looks like i miss something in openwrt yet (possibly the gpio-hog)

BPI-R4P> pci enum
BPI-R4P> pci scan
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x14c3     0x7988     Bridge device           0x04
01.00.00   0x1e4b     0x1202     Mass storage controller 0x08
BPI-R4P> nvme scan
BPI-R4P> nvme info
Device 0: Vendor: 0x1e4b Rev: GTdecda1 Prod: 1OXUW7Y39WJPT7YGL4QE
            Type: Hard Disk
            Capacity: 122104.3 MB = 119.2 GB (250069680 x 512)
BPI-R4P>

works now in my openwrt-uboot too :slight_smile: and linux now too :wink:

here i have added nvme patch for R4 too (but untested), so you can try building R4 tree from here:

mhm, have flashed my uboot now (previously i did a chainload of same u-boot.bin) and now pcie does not find anything again ;( something strange, but at least openwrt works now