[BPI-R4] Failed attempt: add NVMe support on uboot

I was able to patch the latest version of U-boot:

commit fd46ea0e701920eb205c2bce9d527bf0dec10b59 (HEAD -> master, origin/master, origin/HEAD)
Merge: 18908395ed 7d22317a18
Author: Tom Rini <[email protected]>
Date:   Fri Jul 5 16:36:43 2024 -0600

And confirm that is still working.

Hi @frank-w

Seems that U-boot 202407 (not 202407-rc5) has already pushed PCIe 3.0 for Mediatek MT7988 driver in mainline. But still missing DTS patch for PCIe. I was able to build 202407 with partial patch from original patch, just to add missing devices in DTS and is working.

1 Like

Yes,dts patches are rejected because boards are not of_upstream yet,but good to know that dts patches still working…christian made a bit different handling of the optional phy…only tested it with of_upstream conversion

It is merged for 2024.10…not in 2024.07 release

1 Like

Hi all,

I was playing around with latest version of U-Boot 2025.01. I’ve noticed that the PCIe 3.0 driver is already in U-Boot but DTS for PCIe is not yet there. So I took the DTS PCIe part from 202410 from Frank-W and I’ve ported to the new U-Boot version.

Then, success!

## Starting application at 0x41E00000 ...


U-Boot 2025.01-rc3-EasyOS+ (Dec 03 2024 - 02:11:51 +0100)

CPU:   MediaTek MT7988
Model: mt7988-rfb
DRAM:  4 GiB
Core:  54 devices, 22 uclasses, devicetree: separate
MMC:   mmc@11230000: 0
Loading Environment from nowhere... OK
In:    serial@11000000
Out:   serial@11000000
Err:   serial@11000000
Net:   
Warning: ethernet@15100000 (eth0) using random MAC address - aa:88:d9:66:02:88
eth0: ethernet@15100000
LED 'blue:status' not found (err=-19)
LED 'green:status' not found (err=-19)
Starting EasyOS ..

EEEEEEEEEEEEEEEEEEEEEE                                                             OOOOOOOOO        SSSSSSSSSSSSSSS 
E::::::::::::::::::::E                                                           OO:::::::::OO    SS:::::::::::::::S
E::::::::::::::::::::E                                                         OO:::::::::::::OO S:::::SSSSSS::::::S
EE::::::EEEEEEEEE::::E                                                        O:::::::OOO:::::::OS:::::S     SSSSSSS
  E:::::E       EEEEEE  aaaaaaaaaaaaa      ssssssssssyyyyyyy           yyyyyyyO::::::O   O::::::OS:::::S            
  E:::::E               a::::::::::::a   ss::::::::::sy:::::y         y:::::y O:::::O     O:::::OS:::::S            
  E::::::EEEEEEEEEE     aaaaaaaaa:::::ass:::::::::::::sy:::::y       y:::::y  O:::::O     O:::::O S::::SSSS         
  E:::::::::::::::E              a::::as::::::ssss:::::sy:::::y     y:::::y   O:::::O     O:::::O  SS::::::SSSSS    
  E:::::::::::::::E       aaaaaaa:::::a s:::::s  ssssss  y:::::y   y:::::y    O:::::O     O:::::O    SSS::::::::SS  
  E::::::EEEEEEEEEE     aa::::::::::::a   s::::::s        y:::::y y:::::y     O:::::O     O:::::O       SSSSSS::::S 
  E:::::E              a::::aaaa::::::a      s::::::s      y:::::y:::::y      O:::::O     O:::::O            S:::::S
  E:::::E       EEEEEEa::::a    a:::::assssss   s:::::s     y:::::::::y       O::::::O   O::::::O            S:::::S
EE::::::EEEEEEEE:::::Ea::::a    a:::::as:::::ssss::::::s     y:::::::y        O:::::::OOO:::::::OSSSSSSS     S:::::S
E::::::::::::::::::::Ea:::::aaaa::::::as::::::::::::::s       y:::::y          OO:::::::::::::OO S::::::SSSSSS:::::S
E::::::::::::::::::::E a::::::::::aa:::as:::::::::::ss       y:::::y             OO:::::::::OO   S:::::::::::::::SS 
EEEEEEEEEEEEEEEEEEEEEE  aaaaaaaaaa  aaaa sssssssssss        y:::::y                OOOOOOOOO      SSSSSSSSSSSSSSS   
                                                           y:::::y                                                  
                                                          y:::::y                                                   
                                                         y:::::y                                                    
                                                        y:::::y                                                     
                                                       yyyyyyy                                                      
Scanning NVME..
Scanning USB..
starting USB...
Bus xhci@11200000: xhci-mtk xhci@11200000: hcd: 0x0000000011200000, ippc: 0x0000000011203e00
xhci-mtk xhci@11200000: ports disabled mask: u3p-0x0, u2p-0x0
xhci-mtk xhci@11200000: u2p:1, u3p:1
Register 200010f NbrPorts 2
Starting the controller
USB XHCI 1.10
scanning bus xhci@11200000 for devices... 4 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
EasyOS(BPi-R4)> pci   
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x14c3     0x7988     Bridge device           0x04
01.00.00   0x1ed0     0x2283     Mass storage controller 0x08
EasyOS(BPi-R4)> nvme info
Device 0: Vendor: 0x1ed0 Rev: APF1M3R1 Prod: 702172310010561     
            Type: Hard Disk
            Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)
EasyOS(BPi-R4)> ls nvme 0:1
            ./
            ../
            lost+found/

0 file(s), 3 dir(s)

To implement PCIe 3.0 DTS I’ve created 2 additional .dtsi files: mt7988-pcie-enable.dtsi and mt7988-pcie.dtsi. Then I’ve just included in mt7988.dtsi mt7988-pcie.dtsi like this:

#include <dt-bindings/interrupt-controller/irq.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/clock/mt7988-clk.h>
#include <dt-bindings/reset/mt7988-reset.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/pinctrl/mt65xx.h>
#include <dt-bindings/phy/phy.h>
#include "mt7988-pcie.dtsi"

/ {
..

And in mt7988-rfb.dts mt7988-sd-rfb.dts like this:

/dts-v1/;
#include "mt7988.dtsi"
#include "mt7988-pcie-enable.dtsi"
#include <dt-bindings/gpio/gpio.h>

I’m attaching the files in case somebody wants to try it.

mt7988-pcie.dtsi (4.8 KB)

mt7988-pcie-enable.dtsi (216 Bytes)

mt7988.dtsi.patch (337 Bytes)

mt7988-rfb.dts.patch (272 Bytes)

mt7988-sd-rfb.dts.patch (278 Bytes)

Files mt7988-pcie-enable.dtsi and mt7988-pcie.dtsi just copy them to arch/arm/dts. The other 3 patches you can use them with patch -p1 -i ... I’ve preferred to keep them in a non-patch format to be able to modify fast.

Or you can use my U-boot 202501 fork in my git hub where I’ve applied these patches: GitHub - EasyNetDev/u-boot at mt7988

1 Like

You could leave the status=disabled for the 3 pcie and drop the enable file…to make things easier.

1 Like

Indeed I could do it. But I went on this approach to not enable globally the PCIe just for the specific .dtb

if you do it for you only, why not :wink:

currently there are only rfb and the bpi-r4…and imho all here having the r4 and no other mt7988 based board, right?

for mainlining of course the ā€œdisable first and enable in boardā€ aproach is the way to go

1 Like

Yes, was trying to leave close to mainline and in the same time to try to touch as little as possible the original dtsi and dts file.

In this way I can apply the 2 files easier to next U-boot release and make the changes than have a full patch over the 3 main dts/dtsi files.

Only wanted to show an easier way till it is not yet supported in mainline uboot (dts changes are stopped till OF_UPSTREAM changes for which i currently upstream linux dts and we need fdtoverlay command built in uboot and merging of dtso to basedt to have the dtbs for linking into uboot).

1 Like

Thanks for the tip. In the end I was building a custom .dts for the board and I’m compiling it and used in U-boot.

Hello Gentlemen, kudos on your work so far!

I’ve been scouring the internet for anything similar, and came across this active forum post, so thought I’d throw myself out there early rather than regret it later.

I’m working on a product using a mt8395 (same as mt8195 from software perspective), for which we would like to boot into an OS on a NVMe drive. If it would take too many engineering hours, we may swap over to using a UFS drive, but the hardware engineer and I prefer NVMe. Right now I’m doing a little prototyping using the eval board, and at least confirmed the kernel level drivers work well with NVMe out-of-the-box. But, since I’m not seeing native U-boot support for NVMe (at least in MediaTek’s demo branch) I got curious about what’s already out there to address it.

Would you two (or any others) happen to have any suggestions, knowledge and/or advice? I’m a complete novice when it comes to U-boot (several coworkers historically take care of all the U-boot development, but I’ve been looking for a chance to get into it for a while), but I’m sure it’s not going to be simple. Still, I’m hoping I can port the existing upstream PCIe driver and/or your own work to the mt8195 fairly painlessly.

Are either of you aware of a better starting point and/or any major stumbling blocks I should consider before attempting to do so? Or conversely, if there is a better starting point/other project or resource out there which would be a better starting point or additional reference for my use case? Do you think it would be better to port the related files into MediaTek’s branch, or are the changes extensive enough that it would be better to just setup the eval board for upstream U-boot? I saw at least one of the Github links in this forum is dead, (perhaps because it was ported into mainline?), do you guys have all the source contained within one location? If you have any general guidance before I get too deep, please do share!

Thank you in advance and apologies if this is off topic! Good luck on your further developments!

please create an own topic as this is not related to bpi-r4, but i’m not sure anyone can help you here as mt8195 is completely different chipset where afaik bpi has no board with

hi there, just for me to be sure to understand correctly…this will allow to boot directly on the nvme like for the vanilla openwrt instead of the risky sd card?

You cannot boot directly from nvme. You can only boot from spi (nand/nor) or mmc (sd/emmc) and then in uboot continue from nvme (linux kernel,initrd,rootfs).

1 Like

Only if someone ever finished this idea. It’s just an idea now and would need someone to build the uboot image for it and the right configuration.

Then technically, yes, it would be possible to boot ā€œstraightā€ (boots from NAND or NOR the uboot, which then activates the nvme and chains to that) from NVME.

My uboot has most parts prepared

But for my testing i added a prompt for asking the kernel file,but this could also fixed value or something configurable with env var.

Not sure if openwrt has something similar.

if i switch to vanilla openwrt would that be possible then? i am just wondering if i go with an sd card or if i push a bit further to make it more robust with an nvme.