Heck yes 
Awesome, @frank-w, I appreciate it 
I’ll pull/cherry-pick and give it a try - thanks a ton, @sam33 
I’m assuming the menu interface for provisioning the nvme volume/configuring uboot env is still needing some implementation - once I’ve gotten a build that’s working + manually installed, I’ll take a crack at replicating these steps with the process that ends up working for me/adjusting the existing menu interface items to include nvme as an option: sdmmc, snand, emmc
Pcie2 is the keyb slot and empty in my case…have not tried without mpcie card but these are after the key-m so i guess this should not break
Have not done anything with menu for nvme yet,but if you use my precompiled you can add menuitems with uEnv.txt
btw. steps that have to be done (my nvme is currently empty, so i’m stuck here atm…need to create partitiontable first 
):
BPI-R4> if pci enum; then echo success;fi
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card
set trans table 0: 0x28200000 0x28200000, 0x7e00000
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card
set trans table 0: 0x30200000 0x30200000, 0x7e00000
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card
set trans table 0: 0x38200000 0x38200000, 0x7e00000
success
BPI-R4> if nvme scan; then echo success;fi
success
BPI-R4> if nvme info; then echo success;fi
Device 0: Vendor: 0x1cc1 Rev: VC0S036H Prod: 2M232LAJ8HNT
Type: Hard Disk
Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
success
BPI-R4> ls nvme 0:1
** No partition table - nvme 0 **
Couldn't find partition nvme 0:1
after creating partitions i see my testfile 
BPI-R4> if nvme info;then echo success;fi
Device 0: Vendor: 0x1cc1 Rev: VC0S036H Prod: 2M232LAJ8HNT
Type: Hard Disk
Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
success
BPI-R4> ls nvme 0:1
0 bpi-boot-vfat
1 file(s), 0 dir(s)
BPI-R4> ls nvme 0:2
<DIR> 4096 .
<DIR> 4096 ..
<DIR> 16384 lost+found
0 bpi-root-ext
BPI-R4>
BPI-R4> if fatload nvme 0:1 $loadaddr bpi-boot-vfat;then echo load success;fi
0 bytes read in 0 ms
load success
BPI-R4>
if the loaded file (e.g. from environment - fit var) you can continue with the boot-chain (bootm $loadaddr#bootconf). maybe you can copy useusb as usenvme and change related parts (that would my way).
seems nvme info returns also “success” if no device is found…mhm, imho we need to access partition to check if nvme works via script
have it now defined like this:
usenvme=pci enum; nvme scan;nvme info; if ls nvme 0:1;then setenv device nvme;setenv partition 0:1;else echo nvme partition not found;fi
seems to work
but it would be interesting, if nvme also works if there is nothing in mpcie slots
added nvme boot in my uboot 2024-04-bpi tree for r3 and r4 (currently tested only on r3mini, but should work across all devices)
Cool, thanks for the update. Will try on my R4 the next days. Currently working on a windows build setup using Powershell and Docker to build things my way. Though I also got your build.sh working in WSL.
What are the chances of this patch making it to mainline U-Boot, since what I understood was, that this was more a workaround or hack than a final fix to the issue?
John will try to upstream nvme driver next week,not sure about mt7988 support as there is the problem with the key-b slot i currently have disabled. So from driver view it will be upstreamed.
The builtin environment is my specific way to use this driver and this is not upstreamable…maybe @dangowrt add nvme boot in a similar way. If you do not want the ask for filename you have to patch it out…at this point i want it configurable
If you have it working in docker/wsl please share way how to setup…maybe anybody else will use this too
I’ve sent first version of uboot pcie driver to get first responses as rfc
https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
Hi frank-w,
I was looking in U-boot for MT7988 NVME boot and I notice that is not working. I checked this thread and I’ve applied the patch to my U-boot and I got this:
MT7988> pci enum
detected a card at 0x11290000
no card detected at 0x11300000
no card detected at 0x11310000
MT7988> pci
BusDevFun VendorId DeviceId Device Class Sub-Class
_____________________________________________________________
00.00.00 0x14c3 0x7988 Bridge device 0x04
01.00.00 0x1ed0 0x2283 Mass storage controller 0x08
no card detected at 0x11300000
MT7988> nvme scan
MT7988> nvme info
Device 0: Vendor: 0x1ed0 Rev: APF1M3R1 Prod: 702172310010561
Type: Hard Disk
Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)
MT7988> nvme detail
Blk device 0: Optional Admin Command Support:
Namespace Management/Attachment: no
Firmware Commit/Image download: yes
Format NVM: yes
Security Send/Receive: yes
Blk device 0: Optional NVM Command Support:
Reservation: yes
Save/Select field in the Set/Get features: no
Write Zeroes: yes
Dataset Management: yes
Write Uncorrectable: no
Blk device 0: Format NVM Attributes:
Support Cryptographic Erase: No
Support erase a particular namespace: Yes
Support format a particular namespace: No
Blk device 0: LBA Format Support:
LBA Format 0 Support: (current)
Metadata Size: 0
LBA Data Size: 512
Relative Performance: Better
Blk device 0: End-to-End DataProtect Capabilities:
As last eight bytes: No
As first eight bytes: No
Support Type3: No
Support Type2: No
Support Type1: No
Blk device 0: Metadata capabilities:
As part of a separate buffer: No
As part of an extended data LBA: No
MT7988> nvme device
IDE device 0: Vendor: 0x1ed0 Rev: APF1M3R1 Prod: 702172310010561
Type: Hard Disk
Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)
MT7988> nvme part
Partition Map for nvme device 0 -- Partition Type: EFI
Part Start LBA End LBA Name
Attributes
Type GUID
Partition GUID
1 0x00008000 0x1dcf2fff ""
attrs: 0x0000000000000000
type: 0fc63daf-8483-4772-8e79-3d69d8477de4
(linux)
guid: 110b369b-ed3e-4244-9e89-d59fa1864e8b
MT7988> ls nvme 0:1
<DIR> 4096 .
<DIR> 4096 ..
<DIR> 16384 lost+found
So I was able to see the FS on NVME in U-boot.
I’m using a BananaPI R4 brought 1-2 weeks ago.
U-boot version is:
commit fe2ce09a0753634543c32cafe85eb87a625f76ca (HEAD -> master, origin/master, origin/HEAD)
Merge: 16324b43db 1fd754cebd
Author: Tom Rini <[email protected]>
Date: Tue Jun 18 08:34:56 2024 -0600
looks like you got it working, and yes, it is not yet merged as we have to do OF_UPSTREAM first which is very much work (requires complete clock driver change and some other driver patches to be compatible with upstream DTS)…
I hope you will be able to push it to the mainline.
Now I will be able to use NVME SSD to build my Booting and root FS on it and give it a try 
I was able to patch the latest version of U-boot:
commit fd46ea0e701920eb205c2bce9d527bf0dec10b59 (HEAD -> master, origin/master, origin/HEAD)
Merge: 18908395ed 7d22317a18
Author: Tom Rini <[email protected]>
Date: Fri Jul 5 16:36:43 2024 -0600
And confirm that is still working.
Hi @frank-w
Seems that U-boot 202407 (not 202407-rc5) has already pushed PCIe 3.0 for Mediatek MT7988 driver in mainline. But still missing DTS patch for PCIe. I was able to build 202407 with partial patch from original patch, just to add missing devices in DTS and is working.
Yes,dts patches are rejected because boards are not of_upstream yet,but good to know that dts patches still working…christian made a bit different handling of the optional phy…only tested it with of_upstream conversion
It is merged for 2024.10…not in 2024.07 release
Hi all,
I was playing around with latest version of U-Boot 2025.01. I’ve noticed that the PCIe 3.0 driver is already in U-Boot but DTS for PCIe is not yet there. So I took the DTS PCIe part from 202410 from Frank-W and I’ve ported to the new U-Boot version.
Then, success!
## Starting application at 0x41E00000 ...
U-Boot 2025.01-rc3-EasyOS+ (Dec 03 2024 - 02:11:51 +0100)
CPU: MediaTek MT7988
Model: mt7988-rfb
DRAM: 4 GiB
Core: 54 devices, 22 uclasses, devicetree: separate
MMC: mmc@11230000: 0
Loading Environment from nowhere... OK
In: serial@11000000
Out: serial@11000000
Err: serial@11000000
Net:
Warning: ethernet@15100000 (eth0) using random MAC address - aa:88:d9:66:02:88
eth0: ethernet@15100000
LED 'blue:status' not found (err=-19)
LED 'green:status' not found (err=-19)
Starting EasyOS ..
EEEEEEEEEEEEEEEEEEEEEE OOOOOOOOO SSSSSSSSSSSSSSS
E::::::::::::::::::::E OO:::::::::OO SS:::::::::::::::S
E::::::::::::::::::::E OO:::::::::::::OO S:::::SSSSSS::::::S
EE::::::EEEEEEEEE::::E O:::::::OOO:::::::OS:::::S SSSSSSS
E:::::E EEEEEE aaaaaaaaaaaaa ssssssssssyyyyyyy yyyyyyyO::::::O O::::::OS:::::S
E:::::E a::::::::::::a ss::::::::::sy:::::y y:::::y O:::::O O:::::OS:::::S
E::::::EEEEEEEEEE aaaaaaaaa:::::ass:::::::::::::sy:::::y y:::::y O:::::O O:::::O S::::SSSS
E:::::::::::::::E a::::as::::::ssss:::::sy:::::y y:::::y O:::::O O:::::O SS::::::SSSSS
E:::::::::::::::E aaaaaaa:::::a s:::::s ssssss y:::::y y:::::y O:::::O O:::::O SSS::::::::SS
E::::::EEEEEEEEEE aa::::::::::::a s::::::s y:::::y y:::::y O:::::O O:::::O SSSSSS::::S
E:::::E a::::aaaa::::::a s::::::s y:::::y:::::y O:::::O O:::::O S:::::S
E:::::E EEEEEEa::::a a:::::assssss s:::::s y:::::::::y O::::::O O::::::O S:::::S
EE::::::EEEEEEEE:::::Ea::::a a:::::as:::::ssss::::::s y:::::::y O:::::::OOO:::::::OSSSSSSS S:::::S
E::::::::::::::::::::Ea:::::aaaa::::::as::::::::::::::s y:::::y OO:::::::::::::OO S::::::SSSSSS:::::S
E::::::::::::::::::::E a::::::::::aa:::as:::::::::::ss y:::::y OO:::::::::OO S:::::::::::::::SS
EEEEEEEEEEEEEEEEEEEEEE aaaaaaaaaa aaaa sssssssssss y:::::y OOOOOOOOO SSSSSSSSSSSSSSS
y:::::y
y:::::y
y:::::y
y:::::y
yyyyyyy
Scanning NVME..
Scanning USB..
starting USB...
Bus xhci@11200000: xhci-mtk xhci@11200000: hcd: 0x0000000011200000, ippc: 0x0000000011203e00
xhci-mtk xhci@11200000: ports disabled mask: u3p-0x0, u2p-0x0
xhci-mtk xhci@11200000: u2p:1, u3p:1
Register 200010f NbrPorts 2
Starting the controller
USB XHCI 1.10
scanning bus xhci@11200000 for devices... 4 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
EasyOS(BPi-R4)> pci
BusDevFun VendorId DeviceId Device Class Sub-Class
_____________________________________________________________
00.00.00 0x14c3 0x7988 Bridge device 0x04
01.00.00 0x1ed0 0x2283 Mass storage controller 0x08
EasyOS(BPi-R4)> nvme info
Device 0: Vendor: 0x1ed0 Rev: APF1M3R1 Prod: 702172310010561
Type: Hard Disk
Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)
EasyOS(BPi-R4)> ls nvme 0:1
./
../
lost+found/
0 file(s), 3 dir(s)
To implement PCIe 3.0 DTS I’ve created 2 additional .dtsi files:
mt7988-pcie-enable.dtsi and mt7988-pcie.dtsi. Then I’ve just included in mt7988.dtsi mt7988-pcie.dtsi like this:
#include <dt-bindings/interrupt-controller/irq.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/clock/mt7988-clk.h>
#include <dt-bindings/reset/mt7988-reset.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/pinctrl/mt65xx.h>
#include <dt-bindings/phy/phy.h>
#include "mt7988-pcie.dtsi"
/ {
..
And in mt7988-rfb.dts mt7988-sd-rfb.dts like this:
/dts-v1/;
#include "mt7988.dtsi"
#include "mt7988-pcie-enable.dtsi"
#include <dt-bindings/gpio/gpio.h>
I’m attaching the files in case somebody wants to try it.
mt7988-pcie.dtsi (4.8 KB)
mt7988-pcie-enable.dtsi (216 Bytes)
mt7988.dtsi.patch (337 Bytes)
mt7988-rfb.dts.patch (272 Bytes)
mt7988-sd-rfb.dts.patch (278 Bytes)
Files mt7988-pcie-enable.dtsi and mt7988-pcie.dtsi just copy them to arch/arm/dts.
The other 3 patches you can use them with patch -p1 -i ... I’ve preferred to keep them in a non-patch format to be able to modify fast.
Or you can use my U-boot 202501 fork in my git hub where I’ve applied these patches: GitHub - EasyNetDev/u-boot at mt7988
You could leave the status=disabled for the 3 pcie and drop the enable file…to make things easier.
Indeed I could do it. But I went on this approach to not enable globally the PCIe just for the specific .dtb
if you do it for you only, why not
currently there are only rfb and the bpi-r4…and imho all here having the r4 and no other mt7988 based board, right?
for mainlining of course the “disable first and enable in board” aproach is the way to go
Yes, was trying to leave close to mainline and in the same time to try to touch as little as possible the original dtsi and dts file.
In this way I can apply the 2 files easier to next U-boot release and make the changes than have a full patch over the 3 main dts/dtsi files.
Only wanted to show an easier way till it is not yet supported in mainline uboot (dts changes are stopped till OF_UPSTREAM changes for which i currently upstream linux dts and we need fdtoverlay command built in uboot and merging of dtso to basedt to have the dtbs for linking into uboot).
Thanks for the tip. In the end I was building a custom .dts for the board and I’m compiling it and used in U-boot.
Hello Gentlemen, kudos on your work so far!
I’ve been scouring the internet for anything similar, and came across this active forum post, so thought I’d throw myself out there early rather than regret it later.
I’m working on a product using a mt8395 (same as mt8195 from software perspective), for which we would like to boot into an OS on a NVMe drive. If it would take too many engineering hours, we may swap over to using a UFS drive, but the hardware engineer and I prefer NVMe. Right now I’m doing a little prototyping using the eval board, and at least confirmed the kernel level drivers work well with NVMe out-of-the-box. But, since I’m not seeing native U-boot support for NVMe (at least in MediaTek’s demo branch) I got curious about what’s already out there to address it.
Would you two (or any others) happen to have any suggestions, knowledge and/or advice? I’m a complete novice when it comes to U-boot (several coworkers historically take care of all the U-boot development, but I’ve been looking for a chance to get into it for a while), but I’m sure it’s not going to be simple. Still, I’m hoping I can port the existing upstream PCIe driver and/or your own work to the mt8195 fairly painlessly.
Are either of you aware of a better starting point and/or any major stumbling blocks I should consider before attempting to do so? Or conversely, if there is a better starting point/other project or resource out there which would be a better starting point or additional reference for my use case? Do you think it would be better to port the related files into MediaTek’s branch, or are the changes extensive enough that it would be better to just setup the eval board for upstream U-boot? I saw at least one of the Github links in this forum is dead, (perhaps because it was ported into mainline?), do you guys have all the source contained within one location? If you have any general guidance before I get too deep, please do share!
Thank you in advance and apologies if this is off topic! Good luck on your further developments!