[BPI-R3] [BPI-R3-Mini] [BPI-R4] NAND distro-boot + linux-recovery

Banana Pi Router design
1

[ Alpha experimental phase ]

[ Just to be clear, the posts after this first post only explain how the internals work, for using the tool it is not necessary to apply all the posts below. ]

This section is about installing a nand image that contains:

U-Boot that scans emmc/nvme/nand for (/boot)/extlinux/extlinux.conf and boots it. The partition that contains this file needs to have the boot flag set.

If it fails on emmc and nvme, it boots the linux initramdisk on nand. It contains all basic utilities needed to setup an internet connection and install any distro.

There are some basic utilities on the initrd, but perhaps it is still missing some tools and/or kernel modules. It contains my bpi router scripts bpir-build bpir-toolbox, but also: debootstrap wget curl nano parted mkfs-btrfs tar xz gzip zstd, etc, etc.

Note: All images (sdmms/emmc/nvme/uart) all have the same initramdisk. You can interrupt normal boot by keeping ‘x’ pressed during early linux booting. You will then enter a bash shell from initramdisk.

Setup my archlinuxarm image on sd-card, via my script or prebuild image. Another possibility is to use my uartboot image.

https://github.com/ericwoud/buildR64arch

https://ftp.woudstra.mywire.org/images/

Recovery uart boot to a linux rescue image

After booting from uartboot, first make internet connection with:

bpir-dhcpc <interfacename>

bpir-toolbox needs to download some files needed to build the image.

When running archlinuxarm from sd-card or the initrd from uartboot on the BPI-R3/R3M/R4, you can:

bpir-toolbox --nand-format

This will format and install the image.

I need to add more documentation about ‘bpir-toolbox’, but you can look into the file to see which options to use.

‘bpir-build’ to install archlinuxarm (or experimental ubuntu) on nvme is added pretty recently also, so also needs documentation and testing. Basically the steps are:

bpir-build -F

and go through the menu.

But of course you can just manually use parted and debootstrap to install any other distro. Add extlinux.conf and set the bootflag. You will need to find a suitable linux kernel then also.

If you have used bpir-build to build the nvme/emmc rootfs, you can also use the same tool when running from the initramdisk to enter it via chroot. Just run the command without arguments.

This all needs more testing…

Note: Use archlinuxarm (not ubuntu) for now to build the initramdisk image. There is still a small issue in ubuntu, which is missing the bpi-r3m airoha firmware files in the standard linux-firmware package.

2

For those who are interested, this is the source that does the nand writing magic:



function mtdnr {
  cat /proc/mtd | grep '"'$1'"' | cut -d':' -f1 | tr -d [:alpha:]
}

function ubivol {
  for u in /sys/class/ubi/${ubidev/"/dev/"/}_*/name; do
    if [[ "$(cat $u)" == "$1" ]]; then
      echo $(basename $(dirname $u))
      return
    fi
  done
}

function ubiupdate {
  if ! diff -q "$1" "$2"; then
    echo "Updating "$(basename $1)" on NAND:"
    cp -vf "$1" "$2"
  else
    echo "Skipping "$(basename $1)" on NAND"
  fi
}

function get_ubidev {
  local ubidevice
  ubidevice="$(basename /sys/class/mtd/mtd${1}/ubi*)"
  [ ! -e "/sys/class/ubi/$ubidevice" ] && return 1
  echo "/dev/$ubidevice"
  return 0
}


###############
# Format NAND #
###############
if [[ $@ =~ "--nand-format" ]]; then
  [[ "$(tr -d '\0' 2>/dev/null </proc/device-tree/compatible)" != "bananapi,"* ]] && exit 1
  i=0
  ubinr=$(mtdnr ubi)
  ubidetach -p /dev/mtd${ubinr}
  ubiformat -y /dev/mtd${ubinr}
  ubiattach -p /dev/mtd${ubinr}
  ubidev=$(get_ubidev ${ubinr})
  [[ $? != 0 ]] && exit 1
  [[ "$ubidev" == "/dev/ubi_ctrl" ]] && exit 1
  ubimkvol ${ubidev} -n $i -N fip       -s 1MiB   -t static
  ((i++))
  ubimkvol ${ubidev} -n $i -N ubootenv  -s 128KiB
  ((i++))
  ubimkvol ${ubidev} -n $i -N ubootenv2 -s 128KiB
  ((i++))
  ubimkvol ${ubidev} -n $i -N rootfs    -m
  while [ ! -e "${ubidev}_$i" ]; do sleep 0.1; done
  mkfs.ubifs ${ubidev}_$i
fi


###############
# Update NAND #
###############
if [[ $@ =~ "--nand-update" ]] || [[ $@ =~ "--nand-format" ]]; then
  [[ "$(tr -d '\0' 2>/dev/null </proc/device-tree/compatible)" != "bananapi,"* ]] && exit 1
  ubinr=$(mtdnr ubi)
  bl2nr=$(mtdnr bl2)
  atfbin="/usr/share/bpir-atf/${target}-atf-spim-nand-atf.bin"
  dd if=/dev/mtdblock${bl2nr} of="$tmp/dump.bin" bs=$(du -b "$atfbin" | cut -f1) count=1 >/dev/null 2>&1
  if ! diff "$tmp/dump.bin" "$atfbin"; then
    echo "Updating bl2 on NAND:"
    dd if="$atfbin" of=/dev/mtdblock${bl2nr}
  else
    echo "Skipping bl2 on NAND"
  fi
  ubidev=$(get_ubidev ${ubinr})
  if [[ $? != 0 ]]; then
    ubiattach -p /dev/mtd${ubinr}
    ubidev=$(get_ubidev ${ubinr})
    [[ $? != 0 ]] && exit 1
  fi
  if [[ "$ubidev" == "/dev/ubi_ctrl" ]] || [[ "$ubidev" == "/dev/ubi*" ]]; then exit 1; fi
  ubifip=$(ubivol fip)
  fiptool --verbose create $tmp/fip.bin --nt-fw "/usr/share/bpir-uboot/u-boot-${target}-emmc.bin"
  fiptool info $tmp/fip.bin
  dd if="/dev/${ubifip}" of="$tmp/dump.bin"
  if ! diff "$tmp/dump.bin" "$tmp/fip.bin"; then
    echo "Updating fip on NAND"
    ubiupdatevol "/dev/${ubifip}" $tmp/fip.bin
  else
    echo "Skipping fip on NAND"
  fi
  ubirootfs=$(ubivol rootfs)
  mkdir -p $tmp/mnt
  mount -t ubifs ${ubirootfs} $tmp/mnt
  while ! mountpoint -q $tmp/mnt; do sleep 0.1; done
  mkdir -p $tmp/mnt/boot/extlinux $tmp/mnt/boot/dtbs
  [ -f "${linux}.gz" ] && ubootlinux="${linux}.gz" || ubootlinux="${linux}"
  write_dtb "$tmp/atf.dtb" \
            "root=" \
            "/boot/dtbs/${dtb}.dtb" \
	    /usr/share/buildR64arch/boot/${target^^}/*.dts \
	    /usr/share/buildR64arch/boot/${target^^}-EMMC/*.dts
  ubiupdate "$tmp/atf.dtb" "$tmp/mnt${atfdtb}"
  write_extlinux "$tmp/extlinux.conf" \
                 "${ubootlinux}" \
                 "${initrd}" \
                 "${atfdtb}"
  ubiupdate "$tmp/extlinux.conf" "$tmp/mnt/boot/extlinux/extlinux.conf"
  ubiupdate "$ubootlinux" "$tmp/mnt$ubootlinux"
  ubiupdate "$initrd" "$tmp/mnt$initrd"
  sync
  ls -lR $tmp/mnt
  sync
  while mountpoint -q $tmp/mnt; do umount $tmp/mnt; sleep 0.1; done
fi

Note that this code can still be changed. See:

https://github.com/ericwoud/buildR64arch/blob/main/rootfs/bin/bpir-toolbox

Note that everything in the update function first reads the data and only writes if it is changed. That is done for the mtd device (bl2), also the ubi volume (fip) and also for files on the ubifs (kernel), which is mounted on the rootfs ubi volume.

Small note: I use an ATF that has BL31 included in BL2 image, so you will not find it in the fip image.

1 Like
3

I wonder where you write the kernel (itb/Image.gz+dtb) and initrd.

I see you create these volumes

ubimkvol ${ubidev} -n $i -N fip       -s 1MiB   -t static
  ((i++))
  ubimkvol ${ubidev} -n $i -N ubootenv  -s 128KiB
  ((i++))
  ubimkvol ${ubidev} -n $i -N ubootenv2 -s 128KiB
  ((i++))
  ubimkvol ${ubidev} -n $i -N rootfs    -m

But ubiupdate (i guess this writes files to the partitions) use some vars i do not see their definition.

4

The script is a bit larger.

Files go like this (after the ubimkvol and mkfs.ubifs calls):

[root@bpir3 ~]# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 07a80000 00020000 "ubi"
mtd1: 00100000 00020000 "bl2"
[root@bpir3 ~]# ubiattach -p /dev/mtd0
UBI device number 0, total 980 LEBs (124436480 bytes, 118.6 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)
[root@bpir3 ~]# ubinfo /dev/ubi0 -N rootfs
Volume ID:   3 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        943 LEBs (119738368 bytes, 114.1 MiB)
State:       OK
Name:        rootfs
Character device major/minor: 243:4
[root@bpir3 ~]# mkdir /tmp/mnt
[root@bpir3 ~]# mount -t ubifs /dev/ubi0_3 /tmp/mnt
[root@bpir3 ~]# ls -lR /tmp/mnt
/tmp/mnt:
total 0
drwxr-xr-x 4 root root 448 Jun 22 14:42 boot

/tmp/mnt/boot:
total 22768
-rwx------ 1 root root  7758117 Jun 22 14:42 Image.gz
drwxr-xr-x 2 root root      248 Jun 22 14:42 dtbs
drwxr-xr-x 2 root root      232 Jun 22 14:42 extlinux
-rwx------ 1 root root 15549126 Jun 22 14:42 initramfs-bpir.img

/tmp/mnt/boot/dtbs:
total 24
-rw-r--r-- 1 root root 24055 Jun 22 14:42 mt7986a-bananapi-bpi-r3-atf.dtb

/tmp/mnt/boot/extlinux:
total 4
-rw-r--r-- 1 root root 251 Jun 22 14:42 extlinux.conf
[root@bpir3 ~]# cat /tmp/mnt/boot/extlinux/extlinux.conf 
DEFAULT linux-bpir-git
  MENU title U-Boot menu
  PROMPT 0
  TIMEOUT 50
LABEL linux-bpir-git
  MENU LABEL Archlinux AUR package BananaPi Routers
  LINUX /boot/Image.gz
  INITRD /boot/initramfs-bpir.img
  FDT /boot/dtbs/mt7986a-bananapi-bpi-r3-atf.dtb

And now you can copy/overwrite files to /tmp/mnt/ using cp or cat command.

So only Image.gz dtb initrd and extlinux.conf needed for distroboot, no itb. The extlinux.conf can also refer to dtb-overlays to apply, but I have it build to 1 dtb via my script.

BL2 is written directly to /dev/mtdblock1 with dd

UBOOT (+ BL31) is written to:

[root@bpir3 ~]# ubinfo /dev/ubi0 -N fip
Volume ID:   0 (on ubi0)
Type:        static
Alignment:   1
Size:        9 LEBs (1142784 bytes, 1.0 MiB)
Data bytes:  767336 bytes (749.3 KiB)
State:       OK
Name:        fip
Character device major/minor: 243:1

So to /dev/ubi0_0 with ubiupdatevol, but I think you can just dd it as well.

5

Ok,so you have kernel and initrd as files in the rootfs ubi Volume.

6

The ubi rootfs volume is formatted with:

mkfs.ubifs /dev/ubi0_3

Then mount ubifs and copy over the files to the ubifs.

U-Boot’s distro-boot will look for (/boot)/extlinux/extlinux.conf and do the rest. Can specify to U-Boot the name ubi and rootfs, which I did, because the original was UBI and I think it is case dependant.

1 Like
7

Can distroboot also do fit and overlays? Have only seen the Image.gz+dtb way

8

I do not think it does FIT, this is another method. extlinux.conf can do overlays, cmdline and has a menu to choose which kernel (or same kernel with different options) to boot. Distroboot also can use boot.scr instead of extlinux.conf.

This is done to make distroboot possible:

include/configs/mt7xxx.h


/* SPDX-License-Identifier: GPL-2.0 */
/*
 * Configuration for MediaTek MT7622/MT7986 SoC
 *
 */

#ifndef __MT7XXX_H
#define __MT7XXX_H

/* Uboot definition */
#define CFG_SYS_UBOOT_BASE		CONFIG_TEXT_BASE

/* SPL -> Uboot */
#define CFG_SYS_UBOOT_START		CONFIG_TEXT_BASE

/* DRAM */
#define CFG_SYS_SDRAM_BASE		0x40000000

/* This is needed for kernel booting */
#define FDT_HIGH			"0x4fd00000"

#define ENV_MEM_LAYOUT_SETTINGS				\
	"fdt_high=" FDT_HIGH "\0"			\
	"kernel_addr_r=0x44000000\0"			\
	"ramdisk_addr_r=0x48000000\0"			\
	"kernel_comp_addr_r=0x4c000000\0"		\
	"kernel_comp_size=0x03c00000\0"			\
	"scriptaddr=0x4fc00000\0"			\
	"fdt_addr_r=" FDT_HIGH "\0"			\
	"fdtfile=" CONFIG_DEFAULT_FDT_FILE "\0"

#ifdef CONFIG_DISTRO_DEFAULTS

#define BOOT_TARGET_DEVICES(func)	\
	func(MMC, mmc, 1)		\
	func(MMC, mmc, 0)		\
	func(NVME, nvme, 0)		\
	func(UBIFS, ubifs, 0, ubi, rootfs)

#include <config_distro_bootcmd.h>

/* Extra environment variables */
#define CFG_EXTRA_ENV_SETTINGS	\
	ENV_MEM_LAYOUT_SETTINGS		\
	BOOTENV

#endif /* ifdef CONFIG_DISTRO_DEFAULTS*/

#endif

diff --git a/include/configs/mt7622.h b/include/configs/mt7622.h
index 4a056954bf8..c61f7b00794 100644
--- a/include/configs/mt7622.h
+++ b/include/configs/mt7622.h
@@ -9,4 +9,6 @@
 #ifndef __MT7622_H
 #define __MT7622_H
 
+#include "mt7xxx.h"
+
 #endif
diff --git a/include/configs/mt7986.h b/include/configs/mt7986.h
index 090b4c6833e..e75674f69c1 100644
--- a/include/configs/mt7986.h
+++ b/include/configs/mt7986.h
@@ -9,4 +9,6 @@
 #ifndef __MT7986_H
 #define __MT7986_H
 
+#include "mt7xxx.h"
+
 #endif
diff --git a/include/configs/mt7987.h b/include/configs/mt7987.h
index 18ed3c7a55b..bd6f6d0d27a 100644
--- a/include/configs/mt7987.h
+++ b/include/configs/mt7987.h
@@ -11,4 +11,6 @@
 
 #define CFG_MAX_MEM_MAPPED		0xC0000000
 
+#include "mt7xxx.h"
+
 #endif
diff --git a/include/configs/mt7988.h b/include/configs/mt7988.h
index e63825a5a19..616f111aea1 100644
--- a/include/configs/mt7988.h
+++ b/include/configs/mt7988.h
@@ -11,4 +11,6 @@
 
 #define CFG_MAX_MEM_MAPPED		0xC0000000
 
+#include "mt7xxx.h"
+
 #endif
9

how does your dts/dtb look like? i’ve seen i upstreamed only the bl2 mtd partition to now break anything. But some of my trees defining the ubi-partition something like this:

            partition@200000 {
                    label = "ubi";
                    reg = <0x200000 0x7e00000>;
                    compatible = "linux,ubi";

                    volumes {
                            ubi-volume-ubootenv {
                                    volname = "ubootenv";
                                    nvmem-layout {
                                            compatible = "u-boot,env-redundant-bool-layout";
                                    };
                            };

                            ubi-volume-ubootenv2 {
                                    volname = "ubootenv2";
                                    nvmem-layout {
                                            compatible = "u-boot,env-redundant-bool-layout";
                                    };
                            };

                            ubi_rootfs: ubi-volume-fit {
                                    volname = "fit";
                            };
                    };
            };

i guess if i only define the ubi-partition i can freely define the partitions inside it.

do you have enabled additional drivers?

i guess this

uninr=$(cat /proc/mtd | grep '"'ubi'"' | cut -d':' -f1 | tr -d [:alpha:])
ubiformat -y /dev/mtd${ubinr}

formats the complete mtd device, not only the ubi-partition, right? and so erases bl2…how to format only the ubi partition?

or is mtd0=bl2 and mtd1=ubi?

seems like this because i see this:

# hexdump -C /dev/mtd0 | less                                                                                                                             
00000000  53 50 49 4e 41 4e 44 21  01 00 00 00 10 00 00 00  |SPINAND!........|
# hexdump -C /dev/mtd1 | less                                                                                                                             
00000000  55 42 49 23 01 00 00 00  00 00 00 00 00 00 00 00  |UBI#............|

now i hang here:

# ubiattach -p /dev/mtd1
[  847.732271] ubi0: attaching mtd1
[  848.096639] ubi0: scanning is finished
[  848.145172] ubi0: attached mtd1 (name "ubi", size 126 MiB)
[  848.150673] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[  848.157590] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[  848.164377] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[  848.171354] ubi0: good PEBs: 1008, bad PEBs: 0, corrupted PEBs: 0
[  848.177446] ubi0: user volume: 0, internal volumes: 1, max. volumes count: 128
[  848.184656] ubi0: max/mean erase counter: 0/0, WL threshold: 4096, image sequence number: 505724658
[  848.193692] ubi0: available PEBs: 984, total reserved PEBs: 24, PEBs reserved for bad PEB handling: 20
[  848.202996] ubi0: background thread "ubi_bgt0d" started, PID 3691
UBI device number 0, total 1008 LEBs (127991808 bytes, 122.0 MiB), available 984 LEBs (124944384 bytes, 119.1 MiB), LEB size 126976 bytes (124.0 KiB)

# ubidevice="$(basename /sys/class/mtd/mtd1/ubi*)"
root@bpi-r4-v11:~
# echo $ubidevice
ubi0
root@bpi-r4-v11:~
# i=0
# ubimkvol /dev/${ubidevice} -n $i -N fip       -s 1MiB   -t static                                                                                       
Volume ID 0, size 9 LEBs (1142784 bytes, 1.0 MiB), LEB size 126976 bytes (124.0 KiB), static, name "fip", alignment 1

now trying to flash the images, seems i cannot write directly to mtd0, but imho i set bl2 to readonly in dts

also cannot flash fip via dd to /dev/ubi0_0

root@bpi-r4-v11:~
# ubinfo /dev/ubi0 -N fip
Volume ID:   0 (on ubi0)
Type:        static
Alignment:   1
Size:        9 LEBs (1142784 bytes, 1.0 MiB)
Data bytes:  0 bytes
State:       OK
Name:        fip
Character device major/minor: 242:1
root@bpi-r4-v11:~
# mkdir /tmp/mnt
root@bpi-r4-v11:~
# dd if=/mnt/2025.04/bpi-r4_spim-nand_ubi_fip.bin of=/dev/ubi0_0                                                                                          
dd: writing to '/dev/ubi0_0': Operation not permitted
1+0 records in
0+0 records out
0 bytes copied, 0.00206046 s, 0.0 kB/s

seems like you do this with ubiupdatevol

root@bpi-r4-v11:~
# ubiupdatevol /dev/ubi0_0 /mnt/2025.04/bpi-r4_spim-nand_ubi_fip.bin

runs without error…

wrote bl2 from uboot:

fatload usb 0:1 $loadaddr 2025.04/bpi-r4_spim-nand_ubi_bl2.img
BPI-R4> mtd erase spi-nand0 0x0 0x100000                              
Erasing 0x00000000 ... 0x000fffff (8 eraseblock(s))
BPI-R4> mtd write spi-nand0 $loadaddr 0x0 0x100000
Writing 1048576 byte(s) (512 page(s)) at offset 0x00000000

looks good so far

F0: 102B 0000
FA: 1042 0000
FA: 1042 0000 [0200]ove, ENTER to select, ESC to quit
F9: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0600 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [1000]
MK: 0000 0000 [0000]
T0: 0000 01A8 [0101]
Jump to BL

NOTICE:  BL2: v2.12.0(release):d50bccad1-bpi-r4-spim-nand
NOTICE:  BL2: Built : 08:37:29, Jun 17 2025
NOTICE:  WDT: Cold boot
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7988
NOTICE:  EMI: Using DDR unknown settings
NOTICE:  EMI: Detected DRAM size: 4096 MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  LVTS: Enable thermal HW reset
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef
NOTICE:  Page size 2048, Block size 131072, size 134217728
NOTICE:  UBI: scanning [0x200000 - 0x8000000] ...
NOTICE:  UBI: scanning is finished
NOTICE:  UBI: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
NOTICE:  UBI: VID header offset: 2048 (aligned 2048), data offset: 4096
NOTICE:  UBI: Volume fip (Id #0) size is 309481 bytes
NOTICE:  BL2: Booting BL31
NOTICE:  BL31: v2.12.0(release):d50bccad1-bpi-r4-spim-nand
NOTICE:  BL31: Built : 08:37:32, Jun 17 2025


U-Boot 2025.04-bpi-g1e7804aaf7e0-dirty (Jun 17 2025 - 08:36:53 +0000)

CPU:   MediaTek MT7988
Model: mt7988-rfb
DRAM:  4 GiB
Core:  66 devices, 24 uclasses, devicetree: separate
MMC:   mmc@11230000: 0
Loading Environment from nowhere... OK
In:    serial@11000000
Out:   serial@11000000
Err:   serial@11000000
=> board_late_init...
bootmedia:spim-nand
Net:   MediaTek MT7988 built-in switch

Warning: ethernet@15110100 (eth0) using random MAC address - 1e:97:0f:f0:b0:bf
eth0: ethernet@15110100
Hit any key to stop autoboot:  0
10

I do not set any ubi volumes in my dtb. It is not needed. Only the mtd partitions in atf, uboot and linux-dtb need to match.

Please check /proc/mtd for some reason mine bl2 is mtd1, ubi is mtd0 somehow, would expect the opposite, but it could be the reverse…

So:

ubiformat -y /dev/mtd0

Formats the mtd-partition as ubi device

Then:

ubimkvol xxx

creates the ubi volumes on the ubi-device. Similar to using parted to create partitions on a gpt. The volume info is stored in the ubi table, so it would be double definition if it is also in the devicetree, so not needed.

Finally:

mkfs.ubifs /dev/ubi0_3

Formats the rootsf ubi-volume as ubifs. It can then be mounted and files copied over.

1 Like
11

I have bl2 on mtd0 and ubi on mtd1,see the hexdump above

So you have only the outer ubi-partition without the subpartitions?

Boot till fip works so far,will do the ubifs test next.

Do you know how i can manually load files from the ubifs in uboot?

Maybe there is a way to create a nand-image (to be directly written in uboot) with bl2 and ubi inside…do you know if this is possible (e.g. using loopdev)?

This way i can create the nand image in ci pipeline…

Edit: this could be a way: Memory Technology Device (MTD) Subsystem for Linux.

U-boot Commands are written here:

12

Yep, as in the R4 DTS, only MTD partitions bl2 and ubi

You can peek in the uboot <config_distro_bootcmd.h> file which commands it uses, I only did that once, from peeking there

I think you will always need at least 2 files.

  1. bl2.bin

  2. Ubiformat a file and execute all ubi commands on that file instead of the /dev/ubi0 device.

EDIT:

And do not forget to add MTD partitions to uboot dts. See: https://github.com/ericwoud/archlinuxarm-repo/blob/bpir-uboot-git/r4-nand.patch

EDIT2:

I’m not sure if all (R64/R3/R3m/R4) have the ubi start address set in arm-trusted-firmware the same as uboot and linux, so when building, i add

R64:   OVERRIDE_UBI_START_ADDR=0x280000
R3(M): OVERRIDE_UBI_START_ADDR=0x580000
R4:    OVERRIDE_UBI_START_ADDR=0x200000

Would be nice if they were all the same.

13

did this and added additional options,


--- a/arch/arm/dts/mt7988-rfb.dts
+++ b/arch/arm/dts/mt7988-rfb.dts
@@ -242,6 +242,23 @@
                spi-max-frequency = <52000000>;
                spi-rx-bus-width = <4>;
                spi-tx-bus-width = <4>;
+
+               partitions {
+                       compatible = "fixed-partitions";
+                       #address-cells = <1>;
+                       #size-cells = <1>;
+
+                       partition@0 {
+                               label = "bl2";
+                               reg = <0x0 0x200000>;
+                       };
+
+                       partition@200000 {
+                               label = "ubi";
+                               reg = <0x200000 0x7e00000>;
+                               compatible = "linux,ubi";
+                       };
+               };
        };
 };
diff --git a/configs/mt7988a_bpir4_emmc_defconfig b/configs/mt7988a_bpir4_emmc_defconfig
index b2ba75f6e4c7..5ca0ea5c09af 100644
--- a/configs/mt7988a_bpir4_emmc_defconfig
+++ b/configs/mt7988a_bpir4_emmc_defconfig
@@ -147,3 +147,8 @@ CONFIG_I2C_MUX_PCA954x=y
 CONFIG_MISC=y
 CONFIG_I2C_EEPROM=y
 CONFIG_CMD_EEPROM=y
+
+CONFIG_CMD_UBI=y
+CONFIG_CMD_UBIFS=y
+CONFIG_ENV_IS_IN_UBI=y
+CONFIG_ENV_UBI_VOLUME="ubootenv"

but seems ubi does not map mtd-partition…

BPI-R4> mtd list
spi-nand: spi_nand spi_nand@0: Winbond SPI NAND was found.
spi-nand: spi_nand spi_nand@0: 128 MiB, block size: 128 KiB, page size: 2048, OOB size: 64
jedec_spi_nor spi_nor@0: unrecognized JEDEC id bytes: ff, ff, ff
List of MTD devices:
* spi-nand0
  - device: spi_nand@0
  - parent: spi@1100a000
  - driver: spi_nand
  - path: /spi@1100a000/spi_nand@0
  - type: NAND flash
  - block size: 0x20000 bytes
  - min I/O: 0x800 bytes
  - OOB size: 64 bytes
  - OOB available: 24 bytes
  - 0x000000000000-0x000008000000 : "spi-nand0"
          - 0x000000000000-0x000000200000 : "bl2"
          - 0x000000200000-0x000008000000 : "ubi"
BPI-R4> ubi part
Error, no UBI device selected!

seems i have to pass the mtd partition name

BPI-R4> ubi part ubi
jedec_spi_nor spi_nor@0: unrecognized JEDEC id bytes: ff, ff, ff
ubi0: attaching mtd2
ubi0: scanning is finished
ubi0: attached mtd2 (name "ubi", size 126 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 1008, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 4, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 1/0, WL threshold: 4096, image sequence number: 505724658
ubi0: available PEBs: 0, total reserved PEBs: 1008, PEBs reserved for bad PEB handling: 20

BPI-R4> ubifsmount ubi0:rootfs
UBIFS (ubi0:3): UBIFS: mounted UBI device 0, volume 3, name "rootfs", R/O mode
UBIFS (ubi0:3): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
UBIFS (ubi0:3): FS size: 121896960 bytes (116 MiB, 960 LEBs), journal size 9023488 bytes (8 MiB, 72 LEBs)
UBIFS (ubi0:3): reserved for root: 0 bytes (0 KiB)
UBIFS (ubi0:3): media format: w4/r0 (latest is w4/r0), UUID 3029A899-CC2E-4178-B326-B693BD7502A0, small LPT model

BPI-R4> ubifsls
         16802850  Thu Mar 06 15:46:07 2025  rootfs_bpi-r3.cpio_lvm.zst
          9707432  Thu Mar 06 15:46:32 2025  bpi-r4-6.16-mt7988upstream.itb
BPI-R4> ubifsload ${loadaddr} bpi-r4-6.16-mt7988upstream.itb
Loading file 'bpi-r4-6.16-mt7988upstream.itb' to addr 0x46000000...
Done

seems generic load is also possible

BPI-R4> load ubi 0:3 ${loadaddr} bpi-r4-6.16-mt7988upstream.itb
9707432 bytes read in 1089 ms (8.5 MiB/s)

documented my manual steps here (i hope i catched all):

https://www.fw-web.de/dokuwiki/doku.php?id=en:bpi-r4:start#ubi

also updated u-boot-repo to include dts patch and additional config options…

updated my r4 builtin env to support useubi command

BPI-R4> run useubi
jedec_spi_nor spi_nor@0: unrecognized JEDEC id bytes: ff, ff, ff
         16802850  Thu Mar 06 15:46:07 2025  rootfs_bpi-r3.cpio_lvm.zst
          9707432  Thu Mar 06 15:46:32 2025  bpi-r4-6.16-mt7988upstream.itb
BPI-R4> setenv fit bpi-r4-6.16-mt7988upstream.itb
BPI-R4> setenv initrd rootfs_bpi-r3.cpio_lvm.zst
BPI-R4> run newboot
initrd=rootfs_bpi-r3.cpio_lvm.zst
16802850 bytes read in 1986 ms (8.1 MiB/s)
EXT_CSD[179], PARTITION_CONFIG:
BOOT_ACK: 0x1
BOOT_PARTITION_ENABLE: 0x1 (boot0)
PARTITION_ACCESS: 0x0 (user)
emmc available
bootconf=#conf-base#ov-emmc
fit=bpi-r4-6.16-mt7988upstream.itb
9707432 bytes read in 1144 ms (8.1 MiB/s)
Unmounting UBIFS volume rootfs!
## Loading kernel (any) from FIT Image at 46000000 ...

seems my buildroot initrd has some issues with the mtd/ubi

# ubiattach -p /dev/mtd1
libubi: error!: cannot stat "/dev/mtd1"
        error 2 (No such file or directory)
ubiattach: error!: cannot attach "/dev/mtd1"
           error 2 (No such file or directory)
# dmesg |grep mtd
[   12.440514] mtdblock: MTD device 'spi0.0' is NAND, please consider using UBI block devices instead.
14

Always get that to, I even patched kernel so it would only warn once (on R3/R3m). Still functional after this message

Does your initrd use the correct dtb(o) with the correct partition definitions?

What is the output of

cat /proc/mtd

When running initrd?

15

I used my own kernel (should be the same as before to write the ubifs) and standalone initrd.but it seems i put an older version of my kernel…updated it and now i see the ubi mtd partition

# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00200000 00020000 "bl2"
mtd1: 07e00000 00020000 "ubi"
# ubiattach -p /dev/mtd1
[   71.303083] ubi: mtd1 is already attached to ubi0
ubiattach: error!: cannot attach "/dev/mtd1"
           error 17 (File exists)
# mount -t ubifs /dev/ubi0_3 /mnt/
[  157.191121] UBIFS (ubi0:3): Mounting in unauthenticated mode
[  157.196897] UBIFS (ubi0:3): background thread "ubifs_bgt0_3" started, PID 1668
[  157.242670] UBIFS (ubi0:3): UBIFS: mounted UBI device 0, volume 3, name "rootfs"
[  157.250081] UBIFS (ubi0:3): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[  157.259988] UBIFS (ubi0:3): FS size: 121896960 bytes (116 MiB, 960 LEBs), max 971 LEBs, journal size 9023488 bytes (8 MiB, 72 LEBs)
[  157.271803] UBIFS (ubi0:3): reserved for root: 0 bytes (0 KiB)
[  157.277628] UBIFS (ubi0:3): media format: w4/r0 (latest is w5/r0), UUID 3029A899-CC2E-4178-B326-B693BD7502A0, small LPT model

works so far…