BPI-R3 u-boot build help (with NVMe support, hopefully)

Hi

I stumbled onto this website http://www.fw-web.de/dokuwiki/doku.php?id=en:bpi-r3:uboot which has very nice and detailed instructions about building u-boot for the BPI-R3. I want set up a custom U-BOOT with NVMe support on the eMMC or SD-card and then, hopefully get everything else (kernel, root, etc.) to load from the NVMe. Unfortunately I seem to be stuck at the first step, building u-boot.

The website explains a two step process where you first build bl1&2 images from one branch and then follow up with building bl33 on a different branch. The first step works fine, however when building from the second branch, r3-atf, i am getting a bunch of errors.

Has anyone tried building this version of u-boot and can share some insight? Errors seem to be related to some hardware registers and I’m not sure how to correct these. Any help would be appreciated. Also attaching the logs below.

uboot build log output
~/bpi-r3/u-boot (2022-10-r3) # ./build.sh importconfig
device: sd
import mt7986a_bpir3_sd_defconfig...
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  YACC    scripts/kconfig/zconf.tab.c
  LEX     scripts/kconfig/zconf.lex.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
#
# configuration written to .config
#

~/bpi-r3/u-boot (2022-10-r3) # ./build.sh
device: sd
device: sd
scripts/kconfig/conf  --syncconfig Kconfig
  CFG     u-boot.cfg
  GEN     include/autoconf.mk
  GEN     include/autoconf.mk.dep
  CFGCHK  u-boot.cfg
cat: ./Kconfiglib: Is a directory
  OFCHK   .config
  UPD     include/generated/timestamp_autogenerated.h
  ENVC    include/generated/env.txt
  ENVP    include/generated/env.in
  ENVT    include/generated/environment.h
  CC      lib/asm-offsets.s
  CC      arch/arm/lib/asm-offsets.s
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.h
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  YACC    scripts/dtc/dtc-parser.tab.c
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/dtc
  HOSTCC  tools/gen_eth_addr
  HOSTCC  tools/gen_ethaddr_crc.o
  HOSTCC  tools/lib/crc8.o
  HOSTLD  tools/gen_ethaddr_crc
  HOSTCC  tools/img2srec
  HOSTCC  tools/mkenvimage.o
  HOSTCC  tools/os_support.o
  HOSTCC  tools/lib/crc32.o
  HOSTLD  tools/mkenvimage
  HOSTCC  tools/aisimage.o
  HOSTCC  tools/atmelimage.o
  HOSTCC  tools/fit_common.o
  HOSTCC  tools/fit_image.o
  HOSTCC  tools/image-host.o
  HOSTCC  tools/boot/image-fit.o
  HOSTCC  tools/image-sig-host.o
  HOSTCC  tools/boot/image-fit-sig.o
  HOSTCC  tools/boot/image-cipher.o
  HOSTCC  tools/boot/fdt_region.o
  HOSTCC  tools/boot/bootm.o
  HOSTCC  tools/default_image.o
In file included from tools/imagetool.h:24,
                from tools/default_image.c:16:
In function ‘image_set_name’,
    inlined from ‘image_set_header’ at tools/default_image.c:134:2:
include/image.h:779:9: warning: ‘strncpy’ specified bound 32 equals destination size [-Wstringop-truncation]
  779 |         strncpy(image_get_name(hdr), name, IH_NMLEN);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  HOSTCC  tools/lib/fdtdec_common.o
  HOSTCC  tools/lib/fdtdec.o
  HOSTCC  tools/boot/image.o
  HOSTCC  tools/boot/image-host.o
  HOSTCC  tools/imagetool.o
  HOSTCC  tools/imximage.o
  HOSTCC  tools/imx8image.o
  HOSTCC  tools/imx8mimage.o
  HOSTCC  tools/kwbimage.o
  HOSTCC  tools/lib/md5.o
  HOSTCC  tools/lpc32xximage.o
  HOSTCC  tools/mxsimage.o
  HOSTCC  tools/omapimage.o
  HOSTCC  tools/pblimage.o
  HOSTCC  tools/pbl_crc32.o
  HOSTCC  tools/vybridimage.o
  HOSTCC  tools/stm32image.o
  HOSTCC  tools/lib/rc4.o
  HOSTCC  tools/rkcommon.o
  HOSTCC  tools/rkimage.o
  HOSTCC  tools/rksd.o
  HOSTCC  tools/rkspi.o
  HOSTCC  tools/socfpgaimage.o
  HOSTCC  tools/sunxi_egon.o
  HOSTCC  tools/lib/crc16-ccitt.o
  HOSTCC  tools/lib/hash-checksum.o
  HOSTCC  tools/lib/sha1.o
  HOSTCC  tools/lib/sha256.o
  HOSTCC  tools/lib/sha512.o
  HOSTCC  tools/common/hash.o
  HOSTCC  tools/ublimage.o
  HOSTCC  tools/zynqimage.o
  HOSTCC  tools/zynqmpimage.o
  HOSTCC  tools/zynqmpbif.o
  HOSTCC  tools/lib/fdt-libcrypto.o
  HOSTCC  tools/sunxi_toc0.o
  HOSTCC  tools/libfdt/fdt.o
  HOSTCC  tools/libfdt/fdt_ro.o
  HOSTCC  tools/libfdt/fdt_wip.o
  HOSTCC  tools/libfdt/fdt_sw.o
  HOSTCC  tools/libfdt/fdt_rw.o
  HOSTCC  tools/libfdt/fdt_strerror.o
  HOSTCC  tools/libfdt/fdt_empty_tree.o
  HOSTCC  tools/libfdt/fdt_addresses.o
  HOSTCC  tools/libfdt/fdt_overlay.o
  HOSTCC  tools/gpimage.o
  HOSTCC  tools/gpimage-common.o
  HOSTCC  tools/mtk_image.o
  HOSTCC  tools/mtk_nand_headers.o
  HOSTCC  tools/lib/ecdsa/ecdsa-libcrypto.o
  HOSTCC  tools/lib/rsa/rsa-sign.o
  HOSTCC  tools/lib/rsa/rsa-verify.o
In file included from tools/lib/rsa/rsa-verify.c:1:
./tools/../lib/rsa/rsa-verify.c: In function ‘padding_pss_verify’:
./tools/../lib/rsa/rsa-verify.c:275:11: warning: ‘*db’ may be used uninitialized [-Wmaybe-uninitialized]
  275 |         db[0] &= 0xff >> leftmost_bits;
      |         ~~^~~
./tools/../lib/rsa/rsa-verify.c:275:11: warning: ‘*db’ may be used uninitialized [-Wmaybe-uninitialized]
  HOSTCC  tools/lib/rsa/rsa-mod-exp.o
  HOSTCC  tools/lib/aes/aes-encrypt.o
  HOSTCC  tools/lib/aes/aes-decrypt.o
  HOSTCC  tools/dumpimage.o
  HOSTLD  tools/dumpimage
  HOSTCC  tools/mkimage.o
  HOSTLD  tools/mkimage
  HOSTCC  tools/fit_info.o
  HOSTLD  tools/fit_info
  HOSTCC  tools/fit_check_sign.o
  HOSTLD  tools/fit_check_sign
  HOSTCC  tools/proftool
  HOSTCC  tools/relocate-rela
  HOSTCC  tools/fdtgrep.o
  HOSTLD  tools/fdtgrep
  HOSTCC  tools/spl_size_limit
  CC      arch/arm/cpu/armv8/cpu.o
  CC      arch/arm/cpu/armv8/generic_timer.o
  CC      arch/arm/cpu/armv8/cache_v8.o
  AS      arch/arm/cpu/armv8/cache.o
  AS      arch/arm/cpu/armv8/exceptions.o
  CC      arch/arm/cpu/armv8/exception_level.o
  AS      arch/arm/cpu/armv8/tlb.o
  AS      arch/arm/cpu/armv8/transition.o
  CC      arch/arm/cpu/armv8/fwcall.o
  CC      arch/arm/cpu/armv8/cpu-dt.o
  AS      arch/arm/cpu/armv8/smccc-call.o
  AR      arch/arm/cpu/armv8/built-in.o
  AS      arch/arm/cpu/armv8/start.o
  AS      arch/arm/lib/crt0_64.o
  AS      arch/arm/lib/setjmp_aarch64.o
  AS      arch/arm/lib/relocate_64.o
  CC      arch/arm/lib/bootm-fdt.o
  CC      arch/arm/lib/bootm.o
  CC      arch/arm/lib/image.o
  CC      arch/arm/lib/bdinfo.o
  CC      arch/arm/lib/sections.o
  CC      arch/arm/lib/stack.o
  CC      arch/arm/lib/interrupts_64.o
  CC      arch/arm/lib/reset.o
  CC      arch/arm/lib/cache.o
  CC      arch/arm/lib/psci-dt.o
  AR      arch/arm/lib/built-in.o
  CC      arch/arm/mach-mediatek/cpu.o
  CC      arch/arm/mach-mediatek/mt7986/init.o
  AS      arch/arm/mach-mediatek/mt7986/lowlevel_init.o
  AR      arch/arm/mach-mediatek/mt7986/built-in.o
  AR      arch/arm/mach-mediatek/built-in.o
  CC      board/mediatek/mt7986/mt7986_rfb.o
  AR      board/mediatek/mt7986/built-in.o
  CC      boot/bootm.o
  CC      boot/bootm_os.o
  CC      boot/pxe_utils.o
  CC      boot/image.o
  CC      boot/image-board.o
  CC      boot/bootdev-uclass.o
  CC      boot/bootflow.o
  CC      boot/bootmeth-uclass.o
  CC      boot/bootstd-uclass.o
  CC      boot/bootmeth_distro.o
  CC      boot/image-fdt.o
  CC      boot/image-fit.o
  CC      boot/vbe.o
  CC      boot/vbe_simple.o
  AR      boot/built-in.o
  CC      cmd/boot.o
  CC      cmd/bootm.o
  CC      cmd/help.o
  CC      cmd/panic.o
  CC      cmd/version.o
  CC      cmd/blk_common.o
  CC      cmd/bootflow.o
  CC      cmd/source.o
  CC      cmd/bdinfo.o
  CC      cmd/blkcache.o
  CC      cmd/bootmenu.o
  CC      cmd/booti.o
  CC      cmd/console.o
  CC      cmd/echo.o
  CC      cmd/exit.o
  CC      cmd/fat.o
  CC      cmd/fdt.o
  CC      cmd/flash.o
  CC      cmd/fs.o
  CC      cmd/gpio.o
  CC      cmd/itest.o
  CC      cmd/load.o
  CC      cmd/lsblk.o
  CC      cmd/mem.o
  CC      cmd/sleep.o
  CC      cmd/mmc.o
  CC      cmd/mtd.o
  CC      cmd/legacy-mtd-utils.o
  CC      cmd/net.o
  CC      cmd/part.o
  CC      cmd/pinmux.o
  CC      cmd/read.o
  CC      cmd/sf.o
  CC      cmd/setexpr.o
  CC      cmd/strings.o
  CC      cmd/smccc.o
  CC      cmd/test.o
  CC      cmd/lzmadec.o
  CC      cmd/usb.o
  CC      cmd/disk.o
  CC      cmd/ximg.o
  CC      cmd/gpt.o
  CC      cmd/nvedit.o
  AR      cmd/built-in.o
  CC      common/init/board_init.o
  AR      common/init/built-in.o
  CC      common/main.o
  CC      common/exports.o
  CC      common/cli_hush.o
  CC      common/autoboot.o
  CC      common/board_f.o
  CC      common/board_r.o
  CC      common/board_info.o
  CC      common/fdt_support.o
  CC      common/miiphyutil.o
  CC      common/usb.o
  CC      common/usb_hub.o
  CC      common/usb_storage.o
  CC      common/splash.o
  CC      common/menu.o
  CC      common/cli_readline.o
  CC      common/cli_simple.o
  CC      common/console.o
  CC      common/dlmalloc.o
  CC      common/malloc_simple.o
  CC      common/hash.o
  CC      common/memsize.o
  CC      common/stdio.o
  CC      common/cli.o
  CC      common/command.o
  CC      common/log.o
  CC      common/log_console.o
  CC      common/s_record.o
  CC      common/xyzModem.o
  AR      common/built-in.o
  CC      disk/part.o
  CC      disk/disk-uclass.o
  CC      disk/part_dos.o
  CC      disk/part_efi.o
  AR      disk/built-in.o
  CC      drivers/block/blk-uclass.o
  CC      drivers/block/blkcache.o
  AR      drivers/block/built-in.o
  CC      drivers/clk/clk-uclass.o
  CC      drivers/clk/clk_fixed_rate.o
  CC      drivers/clk/clk_fixed_factor.o
  CC      drivers/clk/mediatek/clk-mtk.o
  CC      drivers/clk/mediatek/clk-mt7986.o
  AR      drivers/clk/mediatek/built-in.o
  AR      drivers/clk/built-in.o
  CC      drivers/core/device.o
  CC      drivers/core/fdtaddr.o
  CC      drivers/core/lists.o
  CC      drivers/core/root.o
  CC      drivers/core/uclass.o
  CC      drivers/core/util.o
  CC      drivers/core/tag.o
  CC      drivers/core/device-remove.o
  CC      drivers/core/simple-bus.o
  CC      drivers/core/dump.o
  CC      drivers/core/regmap.o
  CC      drivers/core/syscon-uclass.o
  CC      drivers/core/of_extra.o
  CC      drivers/core/ofnode.o
  CC      drivers/core/read_extra.o
  AR      drivers/core/built-in.o
  CC      drivers/cpu/cpu-uclass.o
  CC      drivers/cpu/mtk_cpu.o
  AR      drivers/cpu/built-in.o
  CC      drivers/crypto/fsl/sec.o
  AR      drivers/crypto/fsl/built-in.o
  AR      drivers/crypto/built-in.o
  CC      drivers/firmware/firmware-uclass.o
  CC      drivers/firmware/psci.o
  AR      drivers/firmware/built-in.o
  CC      drivers/gpio/gpio-uclass.o
  AR      drivers/gpio/built-in.o
  CC      drivers/input/key_matrix.o
  AR      drivers/input/built-in.o
  CC      drivers/mmc/mmc.o
  CC      drivers/mmc/mmc-uclass.o
  CC      drivers/mmc/mmc_bootdev.o
  CC      drivers/mmc/mmc_write.o
  CC      drivers/mmc/mtk-sd.o
  AR      drivers/mmc/built-in.o
  CC      drivers/mtd/mtdcore.o
  CC      drivers/mtd/mtd_uboot.o
  CC      drivers/mtd/mtd-uclass.o
  CC      drivers/mtd/mtdpart.o
  AR      drivers/mtd/mtd.o
  CC      drivers/mtd/nand/core.o
  CC      drivers/mtd/nand/bbt.o
  AR      drivers/mtd/nand/nandcore.o
  CC      drivers/mtd/nand/spi/core.o
  CC      drivers/mtd/nand/spi/gigadevice.o
  CC      drivers/mtd/nand/spi/macronix.o
  CC      drivers/mtd/nand/spi/micron.o
  CC      drivers/mtd/nand/spi/toshiba.o
  CC      drivers/mtd/nand/spi/winbond.o
  AR      drivers/mtd/nand/spi/spinand.o
  AR      drivers/mtd/nand/spi/built-in.o
  AR      drivers/mtd/nand/built-in.o
  CC      drivers/mtd/spi/sf-uclass.o
  CC      drivers/mtd/spi/sf_probe.o
  CC      drivers/mtd/spi/spi-nor-ids.o
  CC      drivers/mtd/spi/spi-nor-core.o
  AR      drivers/mtd/spi/spi-nor.o
  CC      drivers/mtd/spi/sf_mtd.o
  AR      drivers/mtd/spi/built-in.o
  CC      drivers/mtd/ubi/attach.o
  CC      drivers/mtd/ubi/build.o
  CC      drivers/mtd/ubi/vtbl.o
  CC      drivers/mtd/ubi/vmt.o
  CC      drivers/mtd/ubi/upd.o
  CC      drivers/mtd/ubi/kapi.o
  CC      drivers/mtd/ubi/eba.o
  CC      drivers/mtd/ubi/io.o
  CC      drivers/mtd/ubi/wl.o
  CC      drivers/mtd/ubi/crc32.o
  CC      drivers/mtd/ubi/misc.o
  CC      drivers/mtd/ubi/debug.o
  AR      drivers/mtd/ubi/built-in.o
  AR      drivers/mtd/built-in.o
  CC      drivers/net/mtk_eth.o
  CC      drivers/net/phy/phy.o
  CC      drivers/net/phy/fixed.o
  AR      drivers/net/phy/built-in.o
  AR      drivers/net/built-in.o
  CC      drivers/phy/phy-uclass.o
  CC      drivers/phy/phy-mtk-tphy.o
  AR      drivers/phy/built-in.o
  CC      drivers/pinctrl/pinctrl-uclass.o
  CC      drivers/pinctrl/pinctrl-generic.o
  CC      drivers/pinctrl/mediatek/pinctrl-mtk-common.o
  CC      drivers/pinctrl/mediatek/pinctrl-mt7986.o
  AR      drivers/pinctrl/mediatek/built-in.o
  AR      drivers/pinctrl/built-in.o
  CC      drivers/power/domain/power-domain-uclass.o
  CC      drivers/power/domain/mtk-power-domain.o
  AR      drivers/power/domain/built-in.o
  CC      drivers/power/regulator/regulator-uclass.o
  CC      drivers/power/regulator/regulator_common.o
  CC      drivers/power/regulator/fixed.o
  AR      drivers/power/regulator/built-in.o
  AR      drivers/power/built-in.o
  CC      drivers/reset/reset-uclass.o
  CC      drivers/reset/reset-mediatek.o
  AR      drivers/reset/built-in.o
  CC      drivers/serial/serial-uclass.o
  CC      drivers/serial/serial_mtk.o
  AR      drivers/serial/built-in.o
  CC      drivers/spi/spi-uclass.o
  CC      drivers/spi/spi-mem.o
  CC      drivers/spi/mtk_spim.o
  AR      drivers/spi/built-in.o
  AR      drivers/built-in.o
  CC      drivers/usb/common/common.o
  AR      drivers/usb/common/built-in.o
  CC      drivers/usb/host/usb-uclass.o
  CC      drivers/usb/host/usb_bootdev.o
  CC      drivers/usb/host/xhci.o
  CC      drivers/usb/host/xhci-mem.o
  CC      drivers/usb/host/xhci-ring.o
  CC      drivers/usb/host/xhci-mtk.o
  AR      drivers/usb/host/built-in.o
  CC      env/common.o
  CC      env/env.o
  CC      env/attr.o
  CC      env/flags.o
  CC      env/callback.o
  CC      env/mmc.o
  AR      env/built-in.o
  CC      fs/fs.o
  CC      fs/fat/fat_write.o
  AR      fs/fat/built-in.o
  CC      fs/fs_internal.o
  AR      fs/built-in.o
  CC      lib/libfdt/fdt.o
  CC      lib/libfdt/fdt_ro.o
  CC      lib/libfdt/fdt_wip.o
  CC      lib/libfdt/fdt_strerror.o
  CC      lib/libfdt/fdt_sw.o
  CC      lib/libfdt/fdt_rw.o
  CC      lib/libfdt/fdt_empty_tree.o
  CC      lib/libfdt/fdt_addresses.o
  CC      lib/libfdt/fdt_overlay.o
  AR      lib/libfdt/built-in.o
  CC      lib/lzma/LzmaDec.o
  CC      lib/lzma/LzmaTools.o
  AR      lib/lzma/built-in.o
  CC      lib/zlib/zlib.o
  AR      lib/zlib/built-in.o
  CC      lib/charset.o
  CC      lib/crc8.o
  CC      lib/crc16.o
  CC      lib/crc16-ccitt.o
  CC      lib/fdtdec_common.o
  CC      lib/ldiv.o
  CC      lib/net_utils.o
  CC      lib/rc4.o
  CC      lib/rbtree.o
  CC      lib/list_sort.o
  CC      lib/md5.o
  CC      lib/hash-checksum.o
  CC      lib/sha1.o
  CC      lib/sha256.o
  CC      lib/gunzip.o
  CC      lib/fdtdec.o
  CC      lib/qsort.o
  CC      lib/hashtable.o
  CC      lib/errno.o
  CC      lib/display_options.o
  CC      lib/crc32.o
  CC      lib/ctype.o
  CC      lib/div64.o
  CC      lib/hang.o
  CC      lib/linux_compat.o
  CC      lib/linux_string.o
  CC      lib/lmb.o
  CC      lib/membuff.o
  CC      lib/slre.o
  CC      lib/string.o
  CC      lib/tables_csum.o
  CC      lib/time.o
  CC      lib/hexdump.o
  CC      lib/uuid.o
  CC      lib/rand.o
  CC      lib/panic.o
  CC      lib/vsprintf.o
  CC      lib/strto.o
  CC      lib/abuf.o
  CC      lib/date.o
  CC      lib/rtc-lib.o
  AR      lib/built-in.o
  CC      net/arp.o
  CC      net/bootp.o
  CC      net/eth-uclass.o
  CC      net/eth_bootdev.o
  CC      net/eth_common.o
  CC      net/net.o
  CC      net/nfs.o
  CC      net/ping.o
  CC      net/tftp.o
  AR      net/built-in.o
  LDS     u-boot.lds
  LD      u-boot
aarch64-linux-gnu-ld.bfd: warning: u-boot has a LOAD segment with RWX permissions
  OBJCOPY u-boot.srec
  OBJCOPY u-boot-nodtb.bin
  RELOC   u-boot-nodtb.bin
  DTC     arch/arm/dts/mt7622-rfb.dtb
  DTC     arch/arm/dts/mt7623a-unielec-u7623-02-emmc.dtb
  DTC     arch/arm/dts/mt7622-bananapi-bpi-r64.dtb
  DTC     arch/arm/dts/mt7623n-bananapi-bpi-r2.dtb
  DTC     arch/arm/dts/mt7629-rfb.dtb
  DTC     arch/arm/dts/mt7981-rfb.dtb
  DTC     arch/arm/dts/mt7981-emmc-rfb.dtb
  DTC     arch/arm/dts/mt7981-sd-rfb.dtb
  DTC     arch/arm/dts/mt7986a-rfb.dtb
  DTC     arch/arm/dts/mt7986b-rfb.dtb
  DTC     arch/arm/dts/mt7986a-sd-rfb.dtb
  DTC     arch/arm/dts/mt7986b-sd-rfb.dtb
  DTC     arch/arm/dts/mt7986a-emmc-rfb.dtb
  DTC     arch/arm/dts/mt7986b-emmc-rfb.dtb
  DTC     arch/arm/dts/mt8183-pumpkin.dtb
  DTC     arch/arm/dts/mt8512-bm1-emmc.dtb
  DTC     arch/arm/dts/mt8516-pumpkin.dtb
  DTC     arch/arm/dts/mt8518-ap1-emmc.dtb
  SHIPPED dts/dt.dtb
  CAT     u-boot-dtb.bin
  COPY    u-boot.bin
  SYM     u-boot.sym
  COPY    u-boot.dtb
  MKIMAGE u-boot-mtk.bin

~/bpi-r3/u-boot (r3-atf) # ./build.sh importconfig
/usr/bin/python /root/bpi-r3/u-boot/Kconfiglib/defconfig.py /root/bpi-r3/u-boot/configs/mt7986_rfb_sdmmc_defconfig --kconfig /root/bpi-r3/u-boot/config.in
warning: style: quotes recommended around default value for string symbol MBEDTLS_DIR (defined at /root/bpi-r3/u-boot/config.in:255)
warning: style: quotes recommended around default value for string symbol CROSS_COMPILER (defined at /root/bpi-r3/u-boot/config.in:281)
warning: style: quotes recommended around default value for string symbol CROSS_COMPILER (defined at /root/bpi-r3/u-boot/config.in:281)
warning: style: quotes recommended around default value for string symbol CROSS_COMPILER (defined at /root/bpi-r3/u-boot/config.in:281)
Loaded configuration '/root/bpi-r3/u-boot/configs/mt7986_rfb_sdmmc_defconfig'
Configuration saved to '.config'
/usr/bin/python /root/bpi-r3/u-boot/Kconfiglib/oldconfig.py /root/bpi-r3/u-boot/config.in
warning: style: quotes recommended around default value for string symbol MBEDTLS_DIR (defined at /root/bpi-r3/u-boot/config.in:255)
warning: style: quotes recommended around default value for string symbol CROSS_COMPILER (defined at /root/bpi-r3/u-boot/config.in:281)
warning: style: quotes recommended around default value for string symbol CROSS_COMPILER (defined at /root/bpi-r3/u-boot/config.in:281)
warning: style: quotes recommended around default value for string symbol CROSS_COMPILER (defined at /root/bpi-r3/u-boot/config.in:281)
Loaded configuration '.config'
No change to configuration in '.config'

~/bpi-r3/u-boot (r3-atf) # ./build.sh
make -f /root/bpi-r3/u-boot/Makefile PLAT="mt7986" CROSS_COMPILE="/usr/bin/aarch64-linux-gnu-" BOOT_DEVICE="sdmmc" DRAM_USE_DDR4=1 DDR4_FREQ_3200=1 LOG_LEVEL=20 BL33="./u-boot.bin" clean
make[1]: Entering directory '/root/bpi-r3/u-boot'
  CLEAN
make[1]: Leaving directory '/root/bpi-r3/u-boot'
make -f /root/bpi-r3/u-boot/Makefile PLAT="mt7986" CROSS_COMPILE="/usr/bin/aarch64-linux-gnu-" BOOT_DEVICE="sdmmc" DRAM_USE_DDR4=1 DDR4_FREQ_3200=1 LOG_LEVEL=20 BL33="./u-boot.bin" all fip
make[1]: Entering directory '/root/bpi-r3/u-boot'
  CC      lib/libfdt/fdt.c
  CC      lib/libfdt/fdt_addresses.c
  CC      lib/libfdt/fdt_empty_tree.c
  CC      lib/libfdt/fdt_ro.c
  CC      lib/libfdt/fdt_rw.c
  CC      lib/libfdt/fdt_strerror.c
  CC      lib/libfdt/fdt_sw.c
  CC      lib/libfdt/fdt_wip.c
  AR      /root/bpi-r3/u-boot/build/mt7986/release/lib/libfdt.a
Building mt7986
  CC      lib/libc/abort.c
  CC      lib/libc/assert.c
  CC      lib/libc/exit.c
  CC      lib/libc/memchr.c
  CC      lib/libc/memcmp.c
  CC      lib/libc/memcpy.c
  CC      lib/libc/memmove.c
  CC      lib/libc/memrchr.c
  CC      lib/libc/memset.c
  CC      lib/libc/printf.c
  CC      lib/libc/putchar.c
  CC      lib/libc/puts.c
  CC      lib/libc/snprintf.c
  CC      lib/libc/strchr.c
  CC      lib/libc/strcmp.c
  CC      lib/libc/strlcat.c
  CC      lib/libc/strlcpy.c
  CC      lib/libc/strlen.c
  CC      lib/libc/strncmp.c
  CC      lib/libc/strnlen.c
  CC      lib/libc/strrchr.c
  CC      lib/libc/strtok.c
  CC      lib/libc/strtoul.c
  CC      lib/libc/strtoll.c
  CC      lib/libc/strtoull.c
  CC      lib/libc/strtol.c
  AS      lib/libc/aarch64/setjmp.S
  AR      /root/bpi-r3/u-boot/build/mt7986/release/lib/libc.a
  CC      bl2/aarch64/bl2_arch_setup.c
  CC      bl2/bl2_image_load_v2.c
  CC      bl2/bl2_main.c
  CC      common/desc_image_load.c
  CC      common/image_decompress.c
  CC      drivers/delay_timer/delay_timer.c
  CC      drivers/gpio/gpio.c
  CC      drivers/io/io_block.c
  CC      drivers/io/io_fip.c
  CC      drivers/io/io_storage.c
  CC      drivers/mmc/mmc.c
  CC      drivers/partition/gpt.c
  CC      drivers/partition/partition.c
  CC      lib/cpus/errata_report.c
  CC      lib/xz/tf_unxz.c
  CC      lib/xz/xz_crc32.c
  CC      lib/xz/xz_crc64.c
  CC      lib/xz/xz_dec_bcj.c
  CC      lib/xz/xz_dec_lzma2.c
  CC      lib/xz/xz_dec_stream.c
  CC      plat/mediatek/common/drivers/efuse/mtk_efuse.c
  CC      plat/mediatek/common/drivers/gpt/mt_gpt.c
  CC      plat/mediatek/common/drivers/mmc/mtk-sd.c
  CC      plat/mediatek/common/drivers/spi/mtk_spi.c
  CC      plat/mediatek/common/mtk_plat_common.c
  CC      plat/mediatek/mt7986/aarch64/platform_common.c
  CC      plat/mediatek/mt7986/bl2_boot_mmc.c
  CC      plat/mediatek/mt7986/bl2_plat_setup.c
  CC      plat/mediatek/mt7986/drivers/dram/emicfg.c
  CC      plat/mediatek/mt7986/drivers/gpio/mt7986_gpio.c
  CC      plat/mediatek/mt7986/drivers/pll/pll.c
  CC      plat/mediatek/mt7986/drivers/spi/boot_spi.c
  CC      plat/mediatek/mt7986/drivers/timer/timer.c
  CC      common/bl_common.c
  CC      common/tf_log.c
  CC      drivers/console/multi_console.c
  CC      plat/common/plat_bl_common.c
  CC      plat/common/plat_log_common.c
  CC      plat/common/aarch64/plat_common.c
  CC      lib/compiler-rt/builtins/popcountdi2.c
  CC      lib/compiler-rt/builtins/popcountsi2.c
  CC      lib/xlat_tables/xlat_tables_common.c
  CC      lib/xlat_tables/aarch64/xlat_tables.c
  AS      bl2/aarch64/bl2_el3_entrypoint.S
  AS      bl2/aarch64/bl2_el3_exceptions.S
  AS      bl2/aarch64/bl2_run_next_image.S
  AS      common/aarch64/early_exceptions.S
  AS      lib/cpus/aarch64/cortex_a53.S
  AS      lib/cpus/aarch64/cpu_helpers.S
  AS      lib/cpus/aarch64/dsu_helpers.S
  AS      lib/locks/exclusive/aarch64/spinlock.S
  AS      plat/common/aarch64/platform_up_stack.S
  AS      plat/mediatek/common/drivers/uart/aarch64/hsuart.S
  AS      plat/mediatek/mt7986/aarch64/plat_helpers.S
  CPP     fdts/mt7986.dts
  DTC     fdts/mt7986.dts
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.pre.dts:8.4-29: Warning (reg_format): /soc/spi@1100A000:reg: property has invalid length (8 bytes) (#address-cells == 2, #size-cells == 1)
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.pre.dts:29.4-30: Warning (reg_format): /soc/apmixedsys@1001e000:reg: property has invalid length (8 bytes) (#address-cells == 2, #size-cells == 1)
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.dtb: Warning (pci_device_reg): Failed prerequisite 'reg_format'
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.dtb: Warning (pci_device_bus_num): Failed prerequisite 'reg_format'
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.dtb: Warning (simple_bus_reg): Failed prerequisite 'reg_format'
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.dtb: Warning (i2c_bus_reg): Failed prerequisite 'reg_format'
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.dtb: Warning (spi_bus_reg): Failed prerequisite 'reg_format'
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.pre.dts:4.22-24.5: Warning (avoid_default_addr_size): /soc/spi@1100A000: Relying on default #address-cells value
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.pre.dts:4.22-24.5: Warning (avoid_default_addr_size): /soc/spi@1100A000: Relying on default #size-cells value
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.pre.dts:25.35-31.5: Warning (avoid_default_addr_size): /soc/apmixedsys@1001e000: Relying on default #address-cells value
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.pre.dts:25.35-31.5: Warning (avoid_default_addr_size): /soc/apmixedsys@1001e000: Relying on default #size-cells value
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.dtb: Warning (avoid_unnecessary_addr_size): Failed prerequisite 'avoid_default_addr_size'
/root/bpi-r3/u-boot/build/mt7986/release/fdts/mt7986.dtb: Warning (unique_unit_address): Failed prerequisite 'avoid_default_addr_size'
  AS      plat/mediatek/mt7986/dtb.S
  AS      common/aarch64/debug.S
  AS      lib/aarch64/cache_helpers.S
  AS      lib/aarch64/misc_helpers.S
  AS      plat/common/aarch64/platform_helpers.S
  PP      bl2/bl2_el3.ld.S
  LD      /root/bpi-r3/u-boot/build/mt7986/release/bl2/bl2.elf
/usr/bin/aarch64-linux-gnu-ld: warning: /root/bpi-r3/u-boot/build/mt7986/release/bl2/bl2.elf has a LOAD segment with RWX permissions
make[1]: *** [/root/bpi-r3/u-boot/Makefile:1305: /root/bpi-r3/u-boot/build/mt7986/release/bl2/bl2.elf] Error 1
make[1]: Leaving directory '/root/bpi-r3/u-boot'
make: *** [makefile:127: atf] Error 2

Nvme cannot be used in uboot due to missing pcie driver for r3

For the error you can try this [BPI-R3] ATF build error - Solved

Perfect. Thanks. Adding:

export LDFLAGS="--no-warn-rwx-segments"

to build.sh solved the error.

Also that’s unfortunate about the pcie driver. Any ideas if support is planned at some point in the future?

Mtk reported me that they try to support pcie for mt7986 after they fixed some other issues on another platform

Ah, that’s great. Fingers crossed it won’t take very long.

@hackpascal @sam33 is there any new state about pcie for r3 in uboot?

Thanks @frank-w for following up on this. I also like to know if there are any news about uboot nvme support. I feel like this is the only thing missing in my setup.

Do you have any particular reason you need to have u-boot nvme functionality?

You could setup the boot partition on the emmc or sd-card. From this partition u-boot can load the kernel. Then the kernel can mount the root partition on nvme.

Can’t answer for @meehien, but one reason is that you are only dealing with a single image. I would rather not deal with emmc and the other on board memory as I would rather keep that as a last resort backup. So that leaves sd, but sd is not what you would call robust after multiple writes. So, yup, I would rather go pure nmve myself.

A high endurance sd-card will be robust enough for accidental writing (kernel updates) for using it as boot partition. I have build a kernel on sd card rootfs for many many times, before it had finally worn out. For sure more than 100 kernel builds and continuous rootfs use more than a year.

But anyway, it is what one prefers…

“Pure” nvme is not possible as bl2+uboot needs to be loaded from sd/emmc/nor/nand because you cannot directly boot from nvme,but having only these 2 parts (which are mostly static) ther and then load all other from nvme would be great.

I guess mtk is busy for adding next soc support (mt7988) so chances are small to get pcie (and so nvme) support.

If pcie support is not coming and one must have the kernel on nvme, there could be another option.

In theory one could use kexec to chainload the kernel. So have one fixed kernel on emmc/sdmmc that loads the running kernel from nvme. But I have not tested kexec functionality on the bananapi’s.

(Then you could skip using u-boot all together, if one chooses to. I have a fork of ATF that can do this Then you are simply replacing u-boot with a kernel)

I do use this chainloading on an intel processor and there I successfully use one kernel as bootloader, to load and start the standard kernel of archlinux.

Anyway, still a lot of possible options…

Hi all,

My setup currently loads uboot and the kernel from the sd-card (or eMMC, usb) and the kernel modules and the rest of the rootfs from the nvme. I would prefer to have just uboot on the sd-card. For me this makes more sense because I like to keep my kernel/fs fairly updated whereas I don’t feel the same about u-boot.

A more important issue though is about backup/restore and failures where you have a clear separation between uboot and the rest, which is not true if the kernel is on one drive (sd) and the kernel modules (and rootfs) are in a different place (nvme).

I think having nvme support in uboot would definitely make the bpi-r3 platform more reliable and maintainable as it would be more modular, but I admit my issues are mostly quality-of-life related.

Thanks @ericwoud for your suggestion. I don’t immediately see the benefits of using kexec in my situation (because it introduces another layer of complexity and a secondary kernel to keep track of) but I’m sure people with different requirements will find it usefull (e.g. sd-card wear and tear related concerns)

So, I am assuming PCIE and NVME u-boot support is still not coming for the BPI-R3. A bit of a shame really…

@ericwoud, do you have a repo link, and/or can you provide some instructions on how to set up the kexec chainloading setup you mentioned? I would be willing to give that a try.

I am quite fed up with having my kernel and rootfs (i.e kernel modules) on separate storage devices. Makes restoring the system from backups a real pain.

I tried, but it did not work. I have not looked into why not…

Good news, at first glance, it does work! Still need a lot of testing, but looks promising.

OpenWrt just announced this for their birthday:

We will use M.2 with M-key for NVMe storage. There is a work-in-progress patch to make PCIe work inside the U-Boot bootloader. This will allow booting other Linux distributions such as Debian and Alpine directly from NVMe.

If I understand that correctly, that would benefit every board using U-Boot?

I know that there is some work on mt7986 pcie support,but currently there are problems on recognizing nvme. Each soc needs a matching driver for pcie so this is a board specific feature

@meehien

This will be the way to install my archlinuxarm image to have it chainloading. I still have to add the code to the initrd, but I need a few days to develop/test this part.

Setup booting from NVME on R3/R3mini/R4, using chain-loading kernel on emmc fip partition

But anyway, good news that u-boot pcie support is coming for the R3 also.