BPI-R3 u-boot build help (with NVMe support, hopefully)

ericwoud · January 10, 2024, 8:10pm

Good news, at first glance, it does work! Still need a lot of testing, but looks promising.

totkeks · January 10, 2024, 8:36pm

OpenWrt just announced this for their birthday:

We will use M.2 with M-key for NVMe storage. There is a work-in-progress patch to make PCIe work inside the U-Boot bootloader. This will allow booting other Linux distributions such as Debian and Alpine directly from NVMe.

If I understand that correctly, that would benefit every board using U-Boot?

frank-w · January 10, 2024, 8:54pm

I know that there is some work on mt7986 pcie support,but currently there are problems on recognizing nvme. Each soc needs a matching driver for pcie so this is a board specific feature

ericwoud · January 11, 2024, 11:16am

@meehien

This will be the way to install my archlinuxarm image to have it chainloading. I still have to add the code to the initrd, but I need a few days to develop/test this part.

Setup booting from NVME on R3/R3mini/R4, using chain-loading kernel on emmc fip partition

ericwoud · January 11, 2024, 4:21pm

But anyway, good news that u-boot pcie support is coming for the R3 also.

meehien · January 16, 2024, 11:38am

Wow, these are some nice updates. Thanks @ericwoud for setting up the tutorial. I am also a big fan of Archlinux so this is quite fortuitous. Definitely saves me a lot of time migrating things.

@ericwoud , @frank-w. If/when the u-boot driver becomes available would you mind posting a message here? I have been checking the commit history @GitHub - frank-w/u-boot: U-Boot-Bootloader for BananaPI-R2/R64/R2Pro/R3 regularly (hoping for a stealthy pcie driver update), but some commits are quite large and difficult to follow.

ericwoud · January 16, 2024, 11:59am

So anyway, the kexec experiment stranded at pcie not initializing correctly. Although a have a custom ‘hack’ / feature in mind of arm trusted firmware, to do something similar as kexec. But more of a proof of concept and for my own use and anyone interested

Best way, indeed wait for pcie uboot support.

lu_zero · January 16, 2024, 12:17pm

In case, porting GitHub - open-power/petitboot would be very nice.

frank-w · January 17, 2024, 6:52am

got info from john, bug seems fixed and he will cleanup the pcie patch and send it soon to upstream uboot

ericwoud · January 26, 2024, 1:56pm

So I have the atf based ‘kexec’ functional. Can now boot with bootfiles on nvme, without using u-boot. But this is an experimental proof of concept.

Still easier to use u-boot with pcie support though…

meehien · February 2, 2024, 3:38pm

Hi @ericwoud. I had a look at your guide and I’d like to try the “2. ATF - KERNEL using boot partition.” boot method. However I do have three questions before I start and mess up my system.

All the code references in https://github.com/ericwoud/buildR64arch/blob/main/README.md#setup-booting-from-nvme-on-r3r3minir4-using-boot-partition-on-emmc refer to the r64. I have a bpi-r3, do I have to change anything, or the r64 and the r3 have the same codebase/configs?
I would like to attempt this first on a sd-card. I feel like that is easier to swap-in/out while I test out things. Do you think your guide will work with that? (I plan to run the commands manually first to figure out what happens, and can swap emmc references with the sd-card).
I am building my own kernel and would like to keep using that if possible. Do I need any specific config enabled in the kernel to support the ATF - KERNEL method or your method will work with any kernel itb image (assuming things are configured correctly)?

ericwoud · February 2, 2024, 4:16pm

Hi Meehien,

I will answer your question here:

meehien · February 4, 2024, 9:01am

buggz · February 4, 2024, 3:17pm

Would it be possible to write step by step instructions for us simple users? Please… I would like to try this. Thanks!

frank-w · February 4, 2024, 3:19pm

i’m about to add this patch and the config-options to my uboot-repo (2024-01-bpi) tree…already cleaned it in my r3mini test-branch.

edit: added…when pipeline finished you can update uboot (at least on my images) with the built one

meehien · February 4, 2024, 5:11pm

@frank-w you might also add something along the lines of the patch attached in the uEnv_r3. u3dev.patch (1.5 KB)

Also, going a bit off-topic here, but I think the NOR memory would be perfect for this. I have been trying to follow your tutorial at en:bpi-r3:uboot [FW-WEB Wiki] but can’t seem to get a bootable NOR setup. Do you think I need to change anything from the linked tutorial, or it should work as is?

frank-w · February 4, 2024, 6:17pm

If i not miss anything it should work as such,make sure you use the bl2.img to have the bootrom header. What happens when you try to boot nor? Make sure all bootswitches are right.

And you have to erase the parts written later.

Why have you changed useinitrd in your patch? It then misses the “root=/dev/ram rw”

meehien · February 4, 2024, 7:00pm

Why have you changed useinitrd in your patch? It then misses the “root=/dev/ram rw”

My initrd (Archlinux auto generated) was getting confused by that was halting the boot. Seemed easier to remove than figure out exactly what’s going on. This however was always the case for me and it’s not related to the NVME.

If i not miss anything it should work as such,make sure you use the bl2.img to have the bootrom header. What happens when you try to boot nor? Make sure all bootswitches are right.

Switches seem to be right. Triple checked. I am using your repo to generate the bl2 and fip. I suspect my atf build command might be invalid, see below, but I am not sure what I can use instead.

make -f ./Makefile PLAT="mt7986" BOOT_DEVICE="${device}" DRAM_USE_DDR4=1 DDR4_FREQ_3200=1 LOG_LEVEL=20 BL33="../u-boot/u-boot.bin" all fip

For ${device} I currently have sdmmc and emmc. Not sure what the correct device value is for NOR. I was assuming emmc.

frank-w · February 4, 2024, 7:05pm

github.com

frank-w/u-boot/blob/2023-10-bpi/.github/workflows/build.yml#L146


      
          - name: Build for BananaPi-R3 (NOR)
            run: |
              git checkout ${{ env.BRANCH }}
              echo -e "board=bpi-r3\ndevice=spi-nor" > build.conf
              bash build.sh importconfig
              bash build.sh build
              bash build.sh rename
              git checkout build.conf
          
              git checkout mtk-atf
              echo -e "board=bpi-r3\ndevice=nor" > build.conf
              bash build.sh build
              bash build.sh rename
              rm build.conf
          
          - name: Build for BananaPi-R4 (SD)
            run: |
              git checkout ${{ env.BRANCH }}
              echo -e "board=bpi-r4" > build.conf
              bash build.sh importconfig
              bash build.sh build

i only use emmc defconfig for nor/nand and drop the ENV_IS_IN_MMC from it…

Do you use the useinitrd? It is supposed to be called manually for testing…else root is set to the mmc-partition,i’m not sure if the initrd is used if the root is set to mmc

meehien · February 4, 2024, 8:42pm

Thanks, that helps. Hopefully I’ll track down whatever’s wrong.

Edited: Fixed. I was not using the compressed uboot for my bl33.

Do you use the useinitrd? It is supposed to be called manually for testing…else root is set to the mmc-partition,i’m not sure if the initrd is used if the root is set to mmc

Ah, that definitely explains the configs. I do actually use it as part of the normal boot. I find it useful for loading modules like rtc_ds1307, fonts, etc…