[BPI-R3] [BPi-R3-Mini] Imagebuilder R3 ArchlinuxArm, linux-rolling-stable

Banana Pi Router design BPI-R3/R3 Mini(MT7986)
63

  1. 256MiB is just an example. Place rootfs anywhere you like.

  2. itb is for u-boot. I have an example u-boot package here:

It is configured to use extlinux.conf. Have not tested it with itb

  1. The bpir-kexec script is only used for kernel chain loading. Having /boot/ on sdmmc/emmc and rootfs on nvme you do not need chain loading.
64

you miss this patch

https://patchwork.kernel.org/project/linux-mediatek/patch/[email protected]/

65

This is different from before. Now you want boot partition on nvme.

Then I suggest to use the patched u-boot with pcie support. Also possible to use bpir-kexec chain loading

66

you got nvme working on r3mini?

67

I have 2 boot scenarios working ok:

  1. /boot/ on emmc, Image on emmc, rootfs on nvme

  2. /boot/ on emmc, Image(1) on emmc, rootfs on nvme, image(1) chain loading image(2) on nvme rootfs/boot

68

/boot/bootcfg is examined by atf

69

forgot that you do not use uboot…you load linux initrd and then access nvme through linux driver

70

ATF is not able to access this boot partition on nvme. Only patched u-boot and kernel can access nvme.

Edit:

Easiest solution is to have the boot partition on emmc or sdmmc.

71

Not yet. When the u-boot pcie support is mainline, I will update my uboot package, so that this is also an option.

Having different options is ok here I think. I see there are many quirks around for supporting all the different nvme drives. Who knows which quirks are available in u-boot…

72

Then I guess this is coming to linux-rolling-stable too soon. It will be included automatically then in my bpir-rolling-stable branch.

73

Note that I do not have kexec really functional. It is rebooting into atf which is then booting pre-loaded images.

74

Thanks all. I now understand my confusion: I assumed that somehow ATF had support for the NVME. I was going for the ATF ‘kexec’ method mentioned here BPI-R3 u-boot build help (with NVMe support, hopefully) - #25 by ericwoud, but must have misunderstood how it actually works.

On the plus side I did manage to figure it out the scripts and give the method a try. Seems to work well on my bpi-r3. However, functionality-wise it is still similar to what I had before, ATF and u-boot on sdcard and rootfs on NVME (atf w/o u-boot is a little bit faster though).

@frank-w. I noticed you merged the pci: mediatek: add PCIe controller support for Filogic patch to your 2024-01-bpi-r3mini u-boot branch. Do you think this might work for the bpi-r3 too?

75

john says he tested it with r3, i have only tried with r3mini, but cannot get my nvme working with it

76

I’m tempted to give it a go. Is that the only patch I need to cherry pick, assuming I would use your 2024-01-bpi branch?

77

Yes and the patch for missing constant and of course enabling the options like i did for r3mini

Then you should be able to do this

OpenWrtOne> pci enum
OpenWrtOne> pci
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
00.00.00   0x14c3     0x1f32     Bridge device           0x04
01.00.00   0x144d     0xa80b     Mass storage controller 0x08
OpenWrtOne> nvme scan
OpenWrtOne> nvme info
Device 0: Vendor: 0x144d Rev: 7L1QHXC7 Prod: S67PNF0W854510
                Type: Hard Disk
                Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)

For debugging purposes you can load u-boot.bin from usb/tftp and use go command on the loadaddr. This way you don’t need built with atf everytime and flash it…but i guess your current uboot does not have go command yet,so you have to do it once…ah you using erics image and loading uboot from atf? Then you will not need atf compile around uboot i guess

78

Had to change the constant patch a little bit (see attached) but other than that it worked perfectly. I now have u-boot on my sdcard and everything else on my nvme. No more messy backup restores for me. Thanks for all the help.

Btw, if anyone is interested, my nvme is a Samsung 980 1TB.

9c6a1ff99845a5c0fcb60a4e25bbf55a20ee07ae.patch (15.6 KB)

1 Like
79

thank you very much for finding my mistake…with this fixed i can also see my nvme on r3mini

BPI-R3M> pci enum                                                               
drivers/pci/pcie_mediatek_gen3.c:mtk_pcie_startup_port[261] detected a card     
set trans table 0: 0x20000000 0x20000000, 0x10000000                            
BPI-R3M> pci                                                                    
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class                   
_____________________________________________________________                   
00.00.00   0x14c3     0x1f32     Bridge device           0x04                   
01.00.00   0x1c5c     0x1327     Mass storage controller 0x08                   
BPI-R3M> nvme scan                                                              
BPI-R3M>                                                                        
BPI-R3M> nvme info                                                              
Device 0: Vendor: 0x1c5c Rev: 80002C00 Prod: ND94N163610404F0R                  
            Type: Hard Disk                                                     
            Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)                  
BPI-R3M>
80

So we got PCIe support in U-Boot and we also got lots of on-board SPI-NOR flash and want to boot from NVMe. What I would do is what every PC or Apple mainboard is doing: store system firmware (“BIOS”) in the SPI-NOR and have that use some generic method to boot from the NVMe (e.g. UEFI or U-Boot’s distro standard boot).

In case of the R3 (or R3 mini or R4) this means that TF-A and U-Boot should be in SPI-NOR, and U-Boot can be configured to carry out UEFI Standard boot and consider both USB and NVMe according to EFI boot variables, just like on a PC. That would put an end to writing endless how-tos for each and every board, and would allow standard installers of most Linux distributions with sufficient kernel driver support to just work.

Maybe porting Tow-Boot for the R3 (and the R4 and esp. also the R2 and R2Pro!) would be a worthwhile idea for that use-case. I’m using that every day on the PineBook Pro… https://tow-boot.org/

For OpenWrt (and hence for me) this is not so attractive: OpenWrt is headless and fits in a couple of megabytes of flash; the userland is tightly tied to the kernel – no need for even a EFI boot filesystem to chose which kernel to load as kernel and userland are inseparable anyway, all in one uImage.FIT.

But I was kinda betting that all other (more storage intense) distros would jump on the Standard Boot train to rid themselves of per-board custom installer images and all the maintenance effort associated with that.

1 Like
81

Finally got to testing eMMC installation from Openwrt (in NAND). Downloaded pre-made AP image.gz from http://www.woudstra.mywire.org/images/bpir3-emmc-ap.img.gz and prepared to sda1 USB as instructed… Managed to install all necessary packages to openwrt and run the commands:

gunzip -c /mnt/sda1/bpir.img.gz | dd of=/dev/mmcblk0 bs=4M conv=fsync
mmc bootpart enable 7 1 /dev/mmcblk0

After few minutes all was done, but after switching DIPS to eMMC(LHHL) and PowerOn I can’t access AP on 192.168.1.33 and no WIFI24 on air. Seems something is not working as expected… Is the ’ bootpart enable 7 1 /dev/mmcblk0’ still valid for the latest image? image

Can I do something for diagnostics from openwrt?

82

You can connect debuguart before powering on and share log how far the system goes…maybe system boots completely and wifi firmware is missing or any setting is wrong which basicly prevents wifi to work