I assembled my BPi-R4 with the BE14 WiFi 7 card and a 128GB m.2. The goal is to boot Ubuntu 24.04 from the NVME drive.
I have been at this for almost two days now, and every time I think I’m getting somewhere I end up lost again
Does a simple guide or anything exist on how to set this up? All the pieces seem to exist, I just don’t know how to put them together. I can atleast say that the NVME is detected and works when I boot from an SD card.
As of right now my stock NAND won’t boot, the eMMC won’t boot and I have the official Banana Pi Ubuntu image flashed to an SD Card which will boot.
I have documented nearly everything along the way but I saw no point in posting a novel of information that doesn’t work. But if there’s any info needed I likely have it noted down somewhere.
I spend weeks getting nvme basicly to work and also at least a week to define a way for booting it
Firat you have to understand that you cannot boot nvme directly. You have to boot from mmc or spi till uboot.
The first step then would be partitioning the nmve to match bootloader config. When using my uboot it uses first partition as fat for kernel and second as rootfs (ext4). Then prepare a rootfs from not running system (e.g.compress a tar.gz from sdcrad rootfs with external pc) and put it e.g. on usb-drive. Then boot up your r4 and boot from sdcard till linux and unpack your tar.gz to the nvme root,change fstab to match your partition. You can also boot from sdcard and use initramfs (can be found on my gdrive) to copy 1:1 (including rights) from sdcard to nvme. That was the way i used.copy kernel from sdcard to root-partition and you are prepared to do a manual boot from nvme (after mmc/spi to uboot).
I tried multiple times to get it to boot from emmc and every time I just got “System Halted” on reboot. That was flashing this image to eMMC only.
Is that the right file? Booting to emmc or nand first makes sense, but I couldn’t figure out how to make it work. Am I using the wrong image? Or doing something in the wrong order?
My NVME drive is still empty for now because I was trying to get uboot working first. Then I was going to setup the nvme next, but I never got uboot working without the sd card.
Oh ok. So what would be the proper steps to get it going?
I re-flashed the latest official OpenWRT to nand and eMMC, just so I have a functioning environment to work in for now.
Im still not 100% sure Im following everything:
Format the NVME first while I still have a functional environment. Partition1 is fat32 (~200MB) and partition2 is the rest of the space as ext4.
partitioning the nmve to match bootloader config. When using my uboot it uses first partition as fat for kernel and second as rootfs (ext4).
The partitioning makes sense, But I don’t think Im fully understanding what data from the sd card needs to go to which partitions on the NVME drive.
unpack your tar.gz to the nvme root
I have a NVME to USB adapter. Could I insert my nvme drive into the usb adapter and flash bpi-r4_noble_6.12.6-main.img with etcher just like I would for an sdcard? Then I can resize the ext4 partition later, create swap space, etc… My Main PC is a Mac and cannot read ext4
Once I figure out step 2, I then modify the fstab file on the nvme drive to match, so something like:
And now at this point I should be able to switch over to eMMC booting and get to Uboot, and select the nvme boot option? Is that the correct steps? Or did I miss anything?
On nvme you do not need the partitions from sdcard as you only need p1 as boot and p2 as rootfs. Sd image contains more partitions like for bl2 and fip. So just create 2 partitions and copy files with permissions/owner information and change etc/fstab.
When emmc is bootable so far and nvme prepared,you can try booting from nvme from emmc uboot. If that works you can modify uenv.txt on emmc boot partition for automatic boot (bootmenu_default and pathing out the askkernel from nvme-boot).
I’m using bootloader on sdcard, because it’s for some reason hard to boot a custom one from nand, and emmc , easiest way is to grab sdcard image from @frank-w and use u-boot from this image
you can adjust uEnv.txt on /dev/mmcblk0p5, no need to copy it anywhere
there’s no need to mirror boot partitions from sd, you can create whatever you like , for example I have 1GB ext4 /boot partitions with multiple kernel variants selectable via boot menu
all you need to do is prepare boot vars , here is my setup for ubuntu
nvme_root_ubuntu=/dev/nvme0n1p3
ununtu_nvme_kernel=bpi-r4.itb
bootnvmeubuntu=run usenvme; setenv root $nvme_root_ubuntu; run buildargs; ext4load nvme 0:1 $kaddr $ununtu_nvme_kernel;bootm $kaddr
#bootmenu record, you should match your options count here
bootmenu_5=6. Boot ubuntu from nvme.=run bootnvmeubuntu
bootmenu_default=5
in this example - ext4load nvme 0:1 is location of boot partition , we using ext4load instead of fatload so we can use ext4 as boot partition filesystem
for kernel you can copy .itb file from sdcard , and it will work properly
one more small detail , if you plan to use non-root user , you should use kernel with
I’ve been at this all afternoon and still cant get it booting from eMMC. The nvme drive isn’t even coming into play yet, I cant even get to uboot. So the nvme is setup and should be ready to go, but I haven’t even made it far enough to try it yet.
The only way I have managed to get anywhere is by using an sd card.
My last attempt was to connect to the R4 via serial and booted into OpenWRT via eMMC. In there I got the nvme formatted and setup the partitions.
I then rebooted into OpenWRT via NAND to perform the flashing for the bl2 image and the mmc image. But I couldn’t, when booting using nand neither my flash drive or the nvme drive are visible so I had no way to get the image files to it.
I ended up rebooting OpenWRT via NAND again and did apk add sshfs so that I could use scp to flash the images.
I ssh into the R4 and ran echo 0 > /sys/block/mmcblk0boot0/force_ro
For right now the problem doesn’t involve my nvme drive yet, because I cant even get Ubuntu booting from the eMMC normally. So I haven’t had a chance to try setting up the NVME drive yet. Though it does work, I have it temporairly formatted and have been using it to store images and other files so I don’t have to keep using my flash drive.
Nothing works except booting from the SD Card. I was up until about 2am last night working on this. I feel like I’m on the right track, but there’s some piece of this process I’m just not understanding. To verify everything hardware wise is ok I even installed the latest snapshot of OpenWRT to the NAND and the eMMC, just to make sure it worked, and it boots just fine from both.
As an alternative, I’d be perfectly fine setting it up to boot from the NAND instead of the eMMC. In-fact it might even be better because then I can keep a backup OS on the eMMC incase anything ever goes wrong.
I also figured I’d mention that I have been doing all this using a serial connection. it’s a CH340G which no-one seems to like, but it’s been rock solid for me in MacOS.
Yeah, so far the only thing I can get to boot from either NAND or eMMC is OpenWRT.
I flashed the latest official snapshot just to make sure everything still worked and it wasn’t a hardware issue or me messing up something up big. OpenWRT can boot from NAND and eMMC perfectly every time. The issues don’t start until I try to setup Ubuntu.
