[BPI-R4] Failed attempt: add NVMe support on uboot

@sam33 can help you with this issue

Sam is already in mail loop with daniel,john and steven. I only posted state and tried to figure out why clocks are not displayed in uboot. Thought you maybe have an idea about this.

hmm…Sam is fully in charge of the clk drivers. According to his description, the clk driver in u-boot is designed in a non-standard way that all clocks can’t be shown by the clk command.

Which description do you mean? I did not see this info in files or commits yet.

internal discussion :rofl:

Hoping @sam33 / @hackpascal have managed to shed some light into their unnatural implementation choices, @frank-w - nvme for the rootfs is the last item I’m personally hoping can be enabled for the bpi-r4; everything else seems to be working in my arrangement (with everything aside from the cpu fan fixes already having been mainlined).

In the mean time, @hackpascal / @sam33 , any details you can share here (beyond just giggling, @hackpascal ) would be helpful for others attempting to debug their way through this implementation process :pray:

@dangowrt , @frank-w , you guys are champions (@YuyuMarco too!)

1 Like

With big help of @sam33 i got it working when pcie2 is disabled. It seems that the not-detected slots breaking the pci core. So not yet final but working so far which is a huge step forward.

Merged it to my “main” tree,so the changes are also available by releases.

Last release also contains r3mini network fix (allow now multiple access).

1 Like

Does this mean when there is no other card used in the mini pcie slots? Or disabled via the device tree of the board?

Heck yes :+1: Awesome, @frank-w, I appreciate it :pray: I’ll pull/cherry-pick and give it a try - thanks a ton, @sam33 :man_superhero:

I’m assuming the menu interface for provisioning the nvme volume/configuring uboot env is still needing some implementation - once I’ve gotten a build that’s working + manually installed, I’ll take a crack at replicating these steps with the process that ends up working for me/adjusting the existing menu interface items to include nvme as an option: sdmmc, snand, emmc

Pcie2 is the keyb slot and empty in my case…have not tried without mpcie card but these are after the key-m so i guess this should not break

Have not done anything with menu for nvme yet,but if you use my precompiled you can add menuitems with uEnv.txt

btw. steps that have to be done (my nvme is currently empty, so i’m stuck here atm…need to create partitiontable first :slight_smile: ):

BPI-R4> if pci enum; then echo success;fi                                       
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card     
set trans table 0: 0x28200000 0x28200000, 0x7e00000                             
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card     
set trans table 0: 0x30200000 0x30200000, 0x7e00000                             
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card     
set trans table 0: 0x38200000 0x38200000, 0x7e00000                             
success                                                                         
BPI-R4> if nvme scan; then echo success;fi                                      
success                                                                         
BPI-R4> if nvme info; then echo success;fi                                      
Device 0: Vendor: 0x1cc1 Rev: VC0S036H Prod: 2M232LAJ8HNT                       
            Type: Hard Disk                                                     
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)                  
success                                                                         
BPI-R4> ls nvme 0:1                                                             
** No partition table - nvme 0 **                                               
Couldn't find partition nvme 0:1

after creating partitions i see my testfile :wink:

BPI-R4> if nvme info;then echo success;fi                                       
Device 0: Vendor: 0x1cc1 Rev: VC0S036H Prod: 2M232LAJ8HNT                       
            Type: Hard Disk                                                     
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)                  
success                                                                         
BPI-R4> ls nvme 0:1                                                             
        0   bpi-boot-vfat                                                       
                                                                                
1 file(s), 0 dir(s)                                                             
                                                                                
BPI-R4> ls nvme 0:2                                                             
<DIR>       4096 .                                                              
<DIR>       4096 ..                                                             
<DIR>      16384 lost+found                                                     
               0 bpi-root-ext                                                   
BPI-R4>
BPI-R4> if fatload nvme 0:1 $loadaddr bpi-boot-vfat;then echo load success;fi
0 bytes read in 0 ms
load success
BPI-R4>

if the loaded file (e.g. from environment - fit var) you can continue with the boot-chain (bootm $loadaddr#bootconf). maybe you can copy useusb as usenvme and change related parts (that would my way).

seems nvme info returns also “success” if no device is found…mhm, imho we need to access partition to check if nvme works via script

have it now defined like this:

usenvme=pci enum; nvme scan;nvme info; if ls nvme 0:1;then setenv device nvme;setenv partition 0:1;else echo nvme partition not found;fi

seems to work :slight_smile:

but it would be interesting, if nvme also works if there is nothing in mpcie slots

added nvme boot in my uboot 2024-04-bpi tree for r3 and r4 (currently tested only on r3mini, but should work across all devices)

1 Like

Cool, thanks for the update. Will try on my R4 the next days. Currently working on a windows build setup using Powershell and Docker to build things my way. Though I also got your build.sh working in WSL.

What are the chances of this patch making it to mainline U-Boot, since what I understood was, that this was more a workaround or hack than a final fix to the issue?

John will try to upstream nvme driver next week,not sure about mt7988 support as there is the problem with the key-b slot i currently have disabled. So from driver view it will be upstreamed.

The builtin environment is my specific way to use this driver and this is not upstreamable…maybe @dangowrt add nvme boot in a similar way. If you do not want the ask for filename you have to patch it out…at this point i want it configurable

If you have it working in docker/wsl please share way how to setup…maybe anybody else will use this too

1 Like

I’ve sent first version of uboot pcie driver to get first responses as rfc

https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/

4 Likes

Hi frank-w,

I was looking in U-boot for MT7988 NVME boot and I notice that is not working. I checked this thread and I’ve applied the patch to my U-boot and I got this:

MT7988> pci enum
detected a card at 0x11290000
no card detected at 0x11300000
no card detected at 0x11310000
MT7988> pci
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x14c3     0x7988     Bridge device           0x04
01.00.00   0x1ed0     0x2283     Mass storage controller 0x08
no card detected at 0x11300000
MT7988> nvme scan
MT7988> nvme info
Device 0: Vendor: 0x1ed0 Rev: APF1M3R1 Prod: 702172310010561     
            Type: Hard Disk
            Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)
MT7988> nvme detail
Blk device 0: Optional Admin Command Support:
        Namespace Management/Attachment: no
        Firmware Commit/Image download: yes
        Format NVM: yes
        Security Send/Receive: yes
Blk device 0: Optional NVM Command Support:
        Reservation: yes
        Save/Select field in the Set/Get features: no
        Write Zeroes: yes
        Dataset Management: yes
        Write Uncorrectable: no
Blk device 0: Format NVM Attributes:
        Support Cryptographic Erase: No
        Support erase a particular namespace: Yes
        Support format a particular namespace: No
Blk device 0: LBA Format Support:
        LBA Format 0 Support: (current)
                Metadata Size: 0
                LBA Data Size: 512
                Relative Performance: Better
Blk device 0: End-to-End DataProtect Capabilities:
        As last eight bytes: No
        As first eight bytes: No
        Support Type3: No
        Support Type2: No
        Support Type1: No
Blk device 0: Metadata capabilities:
        As part of a separate buffer: No
        As part of an extended data LBA: No
MT7988> nvme device 

IDE device 0: Vendor: 0x1ed0 Rev: APF1M3R1 Prod: 702172310010561     
            Type: Hard Disk
            Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)
MT7988> nvme part  

Partition Map for nvme device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name
        Attributes
        Type GUID
        Partition GUID
  1     0x00008000      0x1dcf2fff      ""
        attrs:  0x0000000000000000
        type:   0fc63daf-8483-4772-8e79-3d69d8477de4
                (linux)
        guid:   110b369b-ed3e-4244-9e89-d59fa1864e8b
MT7988> ls nvme 0:1 
<DIR>       4096 .
<DIR>       4096 ..
<DIR>      16384 lost+found

So I was able to see the FS on NVME in U-boot.

I’m using a BananaPI R4 brought 1-2 weeks ago.

U-boot version is:

commit fe2ce09a0753634543c32cafe85eb87a625f76ca (HEAD -> master, origin/master, origin/HEAD)
Merge: 16324b43db 1fd754cebd
Author: Tom Rini <[email protected]>
Date:   Tue Jun 18 08:34:56 2024 -0600

looks like you got it working, and yes, it is not yet merged as we have to do OF_UPSTREAM first which is very much work (requires complete clock driver change and some other driver patches to be compatible with upstream DTS)…

1 Like

I hope you will be able to push it to the mainline. Now I will be able to use NVME SSD to build my Booting and root FS on it and give it a try :smiley:

I was able to patch the latest version of U-boot:

commit fd46ea0e701920eb205c2bce9d527bf0dec10b59 (HEAD -> master, origin/master, origin/HEAD)
Merge: 18908395ed 7d22317a18
Author: Tom Rini <[email protected]>
Date:   Fri Jul 5 16:36:43 2024 -0600

And confirm that is still working.

Hi @frank-w

Seems that U-boot 202407 (not 202407-rc5) has already pushed PCIe 3.0 for Mediatek MT7988 driver in mainline. But still missing DTS patch for PCIe. I was able to build 202407 with partial patch from original patch, just to add missing devices in DTS and is working.

1 Like

Yes,dts patches are rejected because boards are not of_upstream yet,but good to know that dts patches still working…christian made a bit different handling of the optional phy…only tested it with of_upstream conversion

It is merged for 2024.10…not in 2024.07 release

1 Like