[BPI-R4] Failed attempt: add NVMe support on uboot

We are talking about getting nvme slot (pcie3) working in uboot…this is working in our linux kernel, so please stay on topic…nobody talkes about key-b slot (pcie2) or the 2 mpci slots (0+1)

Just wanted to check in again, has anyone got any news on this topic? @YuyuMarco maybe? I haven’t played with the source or the board for a while now, but got time again to test out new debug methods or new source code.

Looking at the other thread about the second pcie port ([BPI-R4] how to activate key B PCIe2 @ 11280000? [solved] - #53 by vexorg), it sounds like the clock configuration for the mediatek chip is a bit special sometimes.

This is only for p2 (the key-b slot)…but i try to get p3 (key m) working…but it seems either pinctrl or clock is missing something

i also tried adding fix for p2, but still same issue (as expected as i only enabled p3 on r4).

--- a/drivers/clk/mediatek/clk-mt7988.c
+++ b/drivers/clk/mediatek/clk-mt7988.c
@@ -802,7 +802,7 @@ static const struct mtk_gate infracfg_mtk_gates[] = {
        GATE_INFRA0(CK_INFRA_PCIE_PERI_26M_CK_P1,
                    "infra_pcie_peri_ck_26m_ck_p1", CK_INFRA_F26M_O0, 8),
        GATE_INFRA0(CK_INFRA_PCIE_PERI_26M_CK_P2,
-                   "infra_pcie_peri_ck_26m_ck_p2", CK_INFRA_F26M_O0, 9),
+                   "infra_pcie_peri_ck_26m_ck_p2", CK_INFRA_PCIE_PERI_26M_CK_P3, 9),
        GATE_INFRA0(CK_INFRA_PCIE_PERI_26M_CK_P3,
                    "infra_pcie_peri_ck_26m_ck_p3", CK_INFRA_F26M_O0, 10),
 };

compared clk with r3mini where pcie works, looks similar

BPI-R3M> pci enum
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[261] detected a card
set trans table 0: 0x20000000 0x20000000, 0x10000000
BPI-R3M> clk dump
 Rate               Usecnt      Name
------------------------------------------
 12000000             0        |-- dummy12m
BPI-R3M> nvme scan
BPI-R3M> nvme info                                                              
Device 0: Vendor: 0x1c5c Rev: 80002C00 Prod: ND94N163610404F0R                  
            Type: Hard Disk                                                     
            Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)                  
BPI-R3M>

it looks like clk command is not able to print enabled clocks somehow…

mtk-clock driver seems to use UCLASS_CLK , so it is strange that clocks are not listed

@hackpascal can you help here?

@sam33 can help you with this issue

Sam is already in mail loop with daniel,john and steven. I only posted state and tried to figure out why clocks are not displayed in uboot. Thought you maybe have an idea about this.

hmm…Sam is fully in charge of the clk drivers. According to his description, the clk driver in u-boot is designed in a non-standard way that all clocks can’t be shown by the clk command.

Which description do you mean? I did not see this info in files or commits yet.

internal discussion :rofl:

Hoping @sam33 / @hackpascal have managed to shed some light into their unnatural implementation choices, @frank-w - nvme for the rootfs is the last item I’m personally hoping can be enabled for the bpi-r4; everything else seems to be working in my arrangement (with everything aside from the cpu fan fixes already having been mainlined).

In the mean time, @hackpascal / @sam33 , any details you can share here (beyond just giggling, @hackpascal ) would be helpful for others attempting to debug their way through this implementation process :pray:

@dangowrt , @frank-w , you guys are champions (@YuyuMarco too!)

1 Like

With big help of @sam33 i got it working when pcie2 is disabled. It seems that the not-detected slots breaking the pci core. So not yet final but working so far which is a huge step forward.

Merged it to my “main” tree,so the changes are also available by releases.

Last release also contains r3mini network fix (allow now multiple access).

1 Like

Does this mean when there is no other card used in the mini pcie slots? Or disabled via the device tree of the board?

Heck yes :+1: Awesome, @frank-w, I appreciate it :pray: I’ll pull/cherry-pick and give it a try - thanks a ton, @sam33 :man_superhero:

I’m assuming the menu interface for provisioning the nvme volume/configuring uboot env is still needing some implementation - once I’ve gotten a build that’s working + manually installed, I’ll take a crack at replicating these steps with the process that ends up working for me/adjusting the existing menu interface items to include nvme as an option: sdmmc, snand, emmc

Pcie2 is the keyb slot and empty in my case…have not tried without mpcie card but these are after the key-m so i guess this should not break

Have not done anything with menu for nvme yet,but if you use my precompiled you can add menuitems with uEnv.txt

btw. steps that have to be done (my nvme is currently empty, so i’m stuck here atm…need to create partitiontable first :slight_smile: ):

BPI-R4> if pci enum; then echo success;fi                                       
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card     
set trans table 0: 0x28200000 0x28200000, 0x7e00000                             
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card     
set trans table 0: 0x30200000 0x30200000, 0x7e00000                             
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[262] detected a card     
set trans table 0: 0x38200000 0x38200000, 0x7e00000                             
success                                                                         
BPI-R4> if nvme scan; then echo success;fi                                      
success                                                                         
BPI-R4> if nvme info; then echo success;fi                                      
Device 0: Vendor: 0x1cc1 Rev: VC0S036H Prod: 2M232LAJ8HNT                       
            Type: Hard Disk                                                     
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)                  
success                                                                         
BPI-R4> ls nvme 0:1                                                             
** No partition table - nvme 0 **                                               
Couldn't find partition nvme 0:1

after creating partitions i see my testfile :wink:

BPI-R4> if nvme info;then echo success;fi                                       
Device 0: Vendor: 0x1cc1 Rev: VC0S036H Prod: 2M232LAJ8HNT                       
            Type: Hard Disk                                                     
            Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)                  
success                                                                         
BPI-R4> ls nvme 0:1                                                             
        0   bpi-boot-vfat                                                       
                                                                                
1 file(s), 0 dir(s)                                                             
                                                                                
BPI-R4> ls nvme 0:2                                                             
<DIR>       4096 .                                                              
<DIR>       4096 ..                                                             
<DIR>      16384 lost+found                                                     
               0 bpi-root-ext                                                   
BPI-R4>
BPI-R4> if fatload nvme 0:1 $loadaddr bpi-boot-vfat;then echo load success;fi
0 bytes read in 0 ms
load success
BPI-R4>

if the loaded file (e.g. from environment - fit var) you can continue with the boot-chain (bootm $loadaddr#bootconf). maybe you can copy useusb as usenvme and change related parts (that would my way).

seems nvme info returns also “success” if no device is found…mhm, imho we need to access partition to check if nvme works via script

have it now defined like this:

usenvme=pci enum; nvme scan;nvme info; if ls nvme 0:1;then setenv device nvme;setenv partition 0:1;else echo nvme partition not found;fi

seems to work :slight_smile:

but it would be interesting, if nvme also works if there is nothing in mpcie slots

added nvme boot in my uboot 2024-04-bpi tree for r3 and r4 (currently tested only on r3mini, but should work across all devices)

Cool, thanks for the update. Will try on my R4 the next days. Currently working on a windows build setup using Powershell and Docker to build things my way. Though I also got your build.sh working in WSL.

What are the chances of this patch making it to mainline U-Boot, since what I understood was, that this was more a workaround or hack than a final fix to the issue?

John will try to upstream nvme driver next week,not sure about mt7988 support as there is the problem with the key-b slot i currently have disabled. So from driver view it will be upstreamed.

The builtin environment is my specific way to use this driver and this is not upstreamable…maybe @dangowrt add nvme boot in a similar way. If you do not want the ask for filename you have to patch it out…at this point i want it configurable

If you have it working in docker/wsl please share way how to setup…maybe anybody else will use this too

1 Like

I’ve sent first version of uboot pcie driver to get first responses as rfc

https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/

4 Likes