[BPI-R4] Boot from NAND fails

[chaotix]

Booting from NAND fails with the following message:

F0: 102B 0000
FA: 1042 0000
FA: 1042 0000 [0200]
F9: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0600 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [1000]
MK: 0000 0000 [0000]
T0: 0000 01A0 [0101]
Jump to BL

NOTICE:  BL2: v2.9(release):
NOTICE:  BL2: Built : 18:12:03, Oct 13 2023
NOTICE:  WDT: Cold boot
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7988
NOTICE:  EMI: Using DDR unknown settings
NOTICE:  EMI: Detected DRAM size: 4096 MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef
NOTICE:  Page size 2048, Block size 131072, size 134217728
ERROR:   BL2: Failed to load image id 3 (-2)

How can I recover from this? I have a bootable SD card with a Debian bookworm install from frank-w with a 6.8-dango kernel from GitHub - frank-w/BPI-Router-Linux: Linux kernel 4.14+ for BPI-R2, 5.4+ for R64, 6.1+ for R2Pro and R3.

I wanted to try writing a new NAND image as documented on Getting Started BPI-R4 | BananaPi Docs

mtd erase /dev/mtd0
dd if=mtk-bpi-r4-NAND-20231030.img of=/dev/mtdblock0

Unfortunately I have two problems with that:

1. I don’t have a ‘mtd’ executable and Debian -- Package Contents Search Results -- mtd does not show anything useful that I can install.
2. I cannot download mtk-bpi-r4-NAND-20231030.img because I cannot read Chinese (for Baidu Cloud) and the Google Drive link does not let me download because the file is too popular.

I’m also confused as to why this happened in the first place. I only touched /dev/mmcblk0 and the SD-card while trying to get the Debian image installed onto eMMC. I did not touch /dev/mtd* or /dev/ubi*.

Any help to recover from this is appreciated.

[frank-w]

You should be able to flash nand from uboot as openwrt and i did for r3

https://wiki.fw-web.de/doku.php?id=en:bpi-r3:uboot#nand_flash

Of course you have to use bpi-r4 binaries (bl2+fip from my uboot repo) or just use the image when you managed download. But i have not tested nand/nor/emmc on r4 yet.

On r3 i used nand/nor only for uboot where i can flash emmc or bootup kernel+initrd. So no full system on the spi storages.

[chaotix]

Thank you for the quick help.

Unfortunately this did not work. Now BL2 just hangs. I used the latest build (bpi-2024.01-2024-02-04_1535) from your Github U-Boot page. Should I try an older version? Or are the offsets different for BPI-R4? (I got one bad NAND block error, but it is at 0x04780000 so I don’t think it should interfere with the boot process)

I also want to use the NAND storage just as a rescue system and to try to get a Debian system to boot from eMMC.

Do you have a copy of the the OpenWRT images for SD and NAND so I can try flashing it from a boot from SD? If so I could send you an upload link.

Thanks again.

BPI-R4> ls mmc 0:5
    36711   bpi-r4.dtb
 25102912   uImage_nodt
  9119192   bpi-r4.itb
       15   uEnv.txt
   254821   bpi-r4_spim-nand_bl2.img
   278413   bpi-r4_spim-nand_fip.bin

6 file(s), 0 dir(s)

BPI-R4> mtd list
spi-nand: spi_nand spi_nand@0: Winbond SPI NAND was found.
spi-nand: spi_nand spi_nand@0: 128 MiB, block size: 128 KiB, page size: 2048, OOB size: 64
List of MTD devices:   
* spi-nand0
  - device: spi_nand@0
  - parent: spi@1100a000
  - driver: spi_nand
  - path: /spi@1100a000/spi_nand@0
  - type: NAND flash
  - block size: 0x20000 bytes
  - min I/O: 0x800 bytes
  - OOB size: 64 bytes
  - OOB available: 24 bytes
  - 0x000000000000-0x000008000000 : "spi-nand0"
BPI-R4> printenv loadaddr
loadaddr=0x46000000
BPI-R4> fatload mmc 0:5 $loadaddr bpi-r4_spim-nand_bl2.img
254821 bytes read in 19 ms (12.8 MiB/s)
BPI-R4> mtd erase spi-nand0
Erasing 0x00000000 ... 0x07ffffff (1024 eraseblock(s))
Skipping bad block at 0x04780000
BPI-R4> mtd write spi-nand0 $loadaddr 0x0 0x100000
Writing 1048576 byte(s) (512 page(s)) at offset 0x00000000
BPI-R4> fatload mmc 0:5 $loadaddr bpi-r4_spim-nand_fip.bin
278413 bytes read in 21 ms (12.6 MiB/s)
BPI-R4> mtd write spi-nand0 $loadaddr 0x380000 0x200000
Writing 2097152 byte(s) (1024 page(s)) at offset 0x00380000

And this is the boot log from NAND now:

F0: 102B 0000
FA: 1042 0000
FA: 1042 0000 [0200]
F9: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0600 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [1000]
MK: 0000 0000 [0000]
T0: 0000 01A0 [0101]
Jump to BL

NOTICE:  BL2: v2.9(release):5ea2352da spim-nand
NOTICE:  BL2: Built : 15:50:53, Feb  4 2024
NOTICE:  WDT: Cold boot
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7988
NOTICE:  EMI: Using DDR unknown settings
NOTICE:  EMI: Detected DRAM size: 4096 MB
NOTICE:  EMI: complex R/W mem test passed   
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef
NOTICE:  Page size 2048, Block size 131072, size 134217728

[frank-w]

@chaotix

BPI-R4> mtd erase spi-nand0
Erasing 0x00000000 ... 0x07ffffff (1024 eraseblock(s))
Skipping bad block at 0x04780000

This is strange…but as i said before,not yet tried nand on r4 yet

tried booting nand without flashing yet…got similar result, so it seems there is already a bl2 on nand

F0: 102B 0000
FA: 1042 0000
FA: 1042 0000 [0200]
F9: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0600 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [1000]
MK: 0000 0000 [0000]
T0: 0000 01A0 [0101]
Jump to BL

NOTICE:  BL2: v2.9(release):
NOTICE:  BL2: Built : 18:12:03, Oct 13 2023
NOTICE:  WDT: Cold boot
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7988
NOTICE:  EMI: Using DDR unknown settings
NOTICE:  EMI: Detected DRAM size: 4096 MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef                                              
NOTICE:  Page size 2048, Block size 131072, size 134217728                      
ERROR:   BL2: Failed to load image id 3 (-2)

while flashing i do not get any errors:

BPI-R4> version
U-Boot 2023.10-bpi-00053-g67b59b097347-dirty (Dec 10 2023 - 14:53:42 +0100)

aarch64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
GNU ld (GNU Binutils for Ubuntu) 2.38
BPI-R4> ls mmc 0:5
  9010412   6.7.0-rc3-dango.itb
      127   uEnv.txt
  9025092   6.6.0-rc2-dango.itb
 24735843   6.6.0-r4-dango
  8941016   6.6.0-r4-dango.itb
  9080636   6.8.0-rc3-dango.itb
 32890100   rootfs_arm64.cpio.gz
   254821   bpi-r4_spim-nand_bl2.img
  9031768   6.7.0-rc5-dango_2.itb
   278533   bpi-r4_spim-nand_fip.bin

10 file(s), 0 dir(s)

BPI-R4> mtd list
spi-nand: spi_nand spi_nand@0: Winbond SPI NAND was found.
spi-nand: spi_nand spi_nand@0: 128 MiB, block size: 128 KiB, page size: 2048, O4
List of MTD devices:
* spi-nand0
  - device: spi_nand@0
  - parent: spi@1100a000
  - driver: spi_nand
  - path: /spi@1100a000/spi_nand@0
  - type: NAND flash                                                            
  - block size: 0x20000 bytes                                                   
  - min I/O: 0x800 bytes                                                        
  - OOB size: 64 bytes                                                          
  - OOB available: 24 bytes                                                     
  - 0x000000000000-0x000008000000 : "spi-nand0"                                 
BPI-R4> mtd erase spi-nand0                                                     
Erasing 0x00000000 ... 0x07ffffff (1024 eraseblock(s))                          
BPI-R4> fatload mmc 0:5 $loadaddr bpi-r4_spim-nand_bl2.img                      
254821 bytes read in 18 ms (13.5 MiB/s)                                         
BPI-R4> mtd write spi-nand0 $loadaddr 0x0 0x100000                              
Writing 1048576 byte(s) (512 page(s)) at offset 0x00000000                      
BPI-R4> fatload mmc 0:5 $loadaddr bpi-r4_spim-nand_fip.bin
278533 bytes read in 20 ms (13.3 MiB/s)
BPI-R4> mtd write spi-nand0 $loadaddr 0x380000 0x200000
Writing 2097152 byte(s) (1024 page(s)) at offset 0x00380000
BPI-R4>

but i also get a hang

looked into atf-source (plat/mediatek/mt7988/bl2/bl2_dev_spi_nand.c) and bl2 uses offset 0x580000 for fip…so we flashed to wrong position

BPI-R4> ls mmc 0:5
  9010412   6.7.0-rc3-dango.itb
      127   uEnv.txt
  9025092   6.6.0-rc2-dango.itb
 24735843   6.6.0-r4-dango
  8941016   6.6.0-r4-dango.itb
  9080636   6.8.0-rc3-dango.itb
 32890100   rootfs_arm64.cpio.gz
   254821   bpi-r4_spim-nand_bl2.img
  9031768   6.7.0-rc5-dango_2.itb
   278533   bpi-r4_spim-nand_fip.bin

10 file(s), 0 dir(s)

BPI-R4> mtd erase spi-nand0
spi-nand: spi_nand spi_nand@0: Winbond SPI NAND was found.
spi-nand: spi_nand spi_nand@0: 128 MiB, block size: 128 KiB, page size: 2048, O4
Erasing 0x00000000 ... 0x07ffffff (1024 eraseblock(s))
BPI-R4> fatload mmc 0:5 $loadaddr bpi-r4_spim-nand_bl2.img
254821 bytes read in 19 ms (12.8 MiB/s)
BPI-R4> mtd write spi-nand0 $loadaddr 0x0 0x100000
Writing 1048576 byte(s) (512 page(s)) at offset 0x00000000
BPI-R4> fatload mmc 0:5 $loadaddr bpi-r4_spim-nand_fip.bin
278533 bytes read in 20 ms (13.3 MiB/s)                                         
BPI-R4> mtd write spi-nand0 $loadaddr 0x580000 0x200000                         
Writing 2097152 byte(s) (1024 page(s)) at offset 0x00580000                     
BPI-R4> reset                                                                   
resetting ...

and here we go

F0: 102B 0000
FA: 1042 0000
FA: 1042 0000 [0200]
F9: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0600 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [1000]
MK: 0000 0000 [0000]
T0: 0000 01A0 [0101]
Jump to BL

NOTICE:  BL2: v2.9(release):v2.9.0-356-g1743e388fb1f spim-nand
NOTICE:  BL2: Built : 19:08:00, Feb 13 2024
NOTICE:  WDT: [40000000] Software reset (reboot)
NOTICE:  CPU: MT7988
NOTICE:  EMI: Using DDR unknown settings
NOTICE:  EMI: Detected DRAM size: 4096 MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  SPI_NAND parses attributes from parameter page.                        
NOTICE:  SPI_NAND Detected ID 0xef                                              
NOTICE:  Page size 2048, Block size 131072, size 134217728                      
NOTICE:  BL2: Booting BL31                                                      
NOTICE:  BL31: v2.9(release):v2.9.0-356-g1743e388fb1f spim-nand                 
NOTICE:  BL31: Built : 19:08:04, Feb 13 2024                                    
                                                                                
                                                                                
U-Boot 2024.04-rc1-bpi-00020-g425db641eb1e-dirty (Feb 13 2024 - 19:06:30 +0100) 
                                                                                
CPU:   MediaTek MT7988                                                          
Model: mt7988-rfb                                                               
DRAM:  4 GiB                                                                    
Core:  53 devices, 19 uclasses, devicetree: separate                            
MMC:   mmc@11230000: 0                                                          
Loading Environment from nowhere... OK                                          
In:    serial@11000000                                                          
Out:   serial@11000000                                                          
Err:   serial@11000000                                                          
Net:                                                                            
Warning: ethernet@15100000 (eth0) using random MAC address - 5a:c2:ce:76:09:65  
eth0: ethernet@15100000                                                         
Hit any key to stop autoboot:  0                                                
BPI-R4>

also checked if i had changed fip offset on mt7986, but it was already in vendor-code on this value, idk why they used different value on mt7988. @moore can you answer this?

Openwrt seems to use a ubi with reg = <0x200000 0x7e00000>,so starting at 0x200000 and creates a partition fip inside…seems to be the first partition so also starts at same offset…i guess @dangowrt has patched above file from atf and changed the fip offset,right?

[chaotix]

Thanks a lot for your help.

Flashing to 0x580000 brought me to a U-Boot console. To my great suprise erasing spi-nand0 also erased the preinstalled OpenWrt there so there was nothing left to boot on NAND.

While trying to compile a new OpenWRT image I checked to Google drive links and was finally able to use the procedure from the “Getting started” page and got a bootable OpenWRT on NAND again.

Using the described procedure to flash OpenWRT to eMMC did not work as BL2 throws an “ERROR: BL2: Failed to load image id 3 (-2)”.

Maybe I let it be and get a NVMe SSD or use a UAS USB-SATA adapter to connect a SSD there.

Thanks again for all your help and the work you do here. Your posts have helped me a lot already.

[frank-w]

@dangowrt is about to prepare mainline openwrt for r4 if you want openwrt…here i cannot help much.

Basicly you can also flash debian to emmc,but i have not done yet on r4,but should be similar to r3

[dangowrt (Daniel Golle)]

I have abandoned using raw offset for fip in favor of using a static UBI volume now that @hackpascal made the code to support having fip stored on UBI which obviously has many advantages compared to having it stored on a raw offset.

See here for how it is done: [WIP/RFC] mediatek: switch to Linux 6.1 and add BananaPi BPi-R4 by dangowrt · Pull Request #14140 · openwrt/openwrt · GitHub

Looks like this then on the console

NOTICE:  BL2: v2.9.0(release):OpenWrt v2023-10-13-0ea67d76-1 (mt7988-spim-nand-ubi-comb)
NOTICE:  BL2: Built : 00:34:15, Jan 24 2024
NOTICE:  WDT: Cold boot
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7988
NOTICE:  EMI: Using DDR unknown settings
NOTICE:  EMI: Detected DRAM size: 4096 MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef
NOTICE:  Page size 2048, Block size 131072, size 134217728
NOTICE:  UBI: scanning [0x200000 - 0x8000000] ...
NOTICE:  UBI: scanning is finished
NOTICE:  UBI: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
NOTICE:  UBI: VID header offset: 2048 (aligned 2048), data offset: 4096
NOTICE:  UBI: Volume fip (Id #0) size is 2097152 bytes
NOTICE:  BL2: Booting BL31
NOTICE:  BL31: v2.9.0(release):OpenWrt v2023-10-13-0ea67d76-1 (mt7988-spim-nand-ubi-comb)
NOTICE:  BL31: Built : 00:34:15, Jan 24 2024


U-Boot 2024.01-OpenWrt-r0+24924-902566ddd6 (Jan 24 2024 - 00:34:15 +0000)

CPU:   MediaTek MT7988
Model: Bananapi BPI-R4
DRAM:  4 GiB
...

[chaotix]

@frank-w

@dangowrt is about to prepare mainline openwrt for r4 if you want openwrt…here i cannot help much.

My goal is a Debian system on eMMC and maybe later on NVMe if 8GB becomes a limitation. OpenWRT on NAND as a rescue system is fine.

Basicly you can also flash debian to emmc,but i have not done yet on r4,but should be similar to r3

I did not manage to do that. I can’t get BL2 to call U-boot. That’s why I tried the OpenWRT eMMC image from the wiki and as it does not do that either, I figured booting from eMMC is just not ready yet.

I dd’ed bpi-r4_emmc.img from your U-Boot repository to /dev/mmcblk0 and then dd’ed bpi-r4_emmc_bl2.img and bpi-r4_emmc_fip.bin to the respective partitions. BL2 started but failed to start U-boot. And then I did not know what else to do.

[dangowrt (Daniel Golle)]

@chaotix

I figured booting from eMMC is just not ready yet.

I’m booting this board from eMMC every day, it definitely works fine for me – which doesn’t mean that you are wrong, but rather it can be that something (such as setting the active hardware boot partition) is not done in factory and has to still be done as part of the installation.

(I’m using work-in-progress OpenWrt build from [WIP/RFC] mediatek: switch to Linux 6.1 and add BananaPi BPi-R4 by dangowrt · Pull Request #14140 · openwrt/openwrt · GitHub)

[chaotix]

@frank-w I noticed a pattern with the NAND boot problem.

I get the following NAND boot failure after starting the bookworm installation from SD:

NOTICE:  BL2: v2.9(release):
NOTICE:  BL2: Built : 18:12:03, Oct 13 2023
NOTICE:  WDT: Cold boot
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7988
NOTICE:  EMI: Using DDR unknown settings
NOTICE:  EMI: Detected DRAM size: 4096 MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef
NOTICE:  Page size 2048, Block size 131072, size 134217728
ERROR:   BL2: Failed to load image id 3 (-2)

Steps to reproduce:

1. Boot mtk-bpi-r4-SD-20231030-New.img from SD.
2. Flash mtk-bpi-r4-NAND-20231030.img to NAND according to the instuctions on the wiki.
3. You can boot the OpenWRT image from SD or NAND as often as you like.
4. Boot the bookworm install from SD.
5. Switch back to NAND and I get the boot failure.

Weird…

I’m using the 6.8.0-rc3-next-20240207-bpi-r4-dango kernel (with some config changes) in case that makes any difference.

[frank-w]

Booting my bookworm image from sd does not write to nand…so it cannot be the cause of nand issue. Which install do you mean?

Edit: ah do you mean the commands in builtin environment in uboot? Maybe you still have old fip offset (0x380000) in your flashed uboot

frank-w/u-boot/blob/2024-04-bpi/uEnv_r4.txt#L56

[chaotix]

@dangowrt

I’m booting this board from eMMC every day, it definitely works fine for me – which doesn’t mean that you are wrong, but rather it can be that something (such as setting the active hardware boot partition) is not done in factory and has to still be done as part of the installation.

I used the following instructions:

echo 0 > /sys/block/mmcblk0boot0/force_ro
dd if=bl2_emmc-r4.img of=/dev/mmcblk0boot0
dd if=mtk-bpi-r4-EMMC-20231030.img of=/dev/mmcblk0
mmc bootpart enable 1 1 /dev/mmcblk0

I thought the last command would set the active boot partition.

(At the moment I’m using a 2.5" SSD attached to an UASP adapter which works fine. A simple dd benchmark showed about 350MB/s from the SSD as well as from the internal eMMC. So it’s not high on my todo list to get this to work.)

[dangowrt (Daniel Golle)]

Ah ok, I have never tried SinoVoip’s eMMC images but only installed what I’m building from source and with more recent kernel, and that works with all 3 boot devices (uSD, SPI-NAND, eMMC)

@dangowrt

(I’m using work-in-progress OpenWrt build from [WIP/RFC] mediatek: switch to Linux 6.1 and add BananaPi BPi-R4 by dangowrt · Pull Request #14140 · openwrt/openwrt · GitHub)

[chaotix]

Ran a couple of tests:

U-boot is not causing the problem. I can enter the U-boot console and reset and NAND still boots fine. I do had the “wrspimnand” variable set, but deleting it with uEnv.txt did not change anything.

Maybe the kernel itself is to blame. This is the log from the first Debian boot:

[    0.754965] spi-nand spi0.0: Winbond SPI NAND was found.
[    0.760303] spi-nand spi0.0: 128 MiB, block size: 128 KiB, page size: 2048, OOB size: 64
[    0.768501] 2 fixed-partitions partitions found on MTD device spi0.0
[    0.774865] OF: Bad cell count for /soc/spi@11007000/spi_nand@0/partitions
[    0.781745] OF: Bad cell count for /soc/spi@11007000/spi_nand@0/partitions
[    0.788911] Creating 2 MTD partitions on "spi0.0":
[    0.793697] 0x000000000000-0x000000200000 : "bl2"
[    0.800283] 0x000000200000-0x000008000000 : "ubi"
[    0.880974] ubi0: attaching mtd1
[    1.499194] ubi0: scanning is finished
[    1.529628] ubi0: volume 2 ("rootfs_data") re-sized from 9 to 498 LEBs
[    1.536560] ubi0: attached mtd1 (name "ubi", size 126 MiB)
[    1.542047] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    1.548910] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    1.555683] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    1.562631] ubi0: good PEBs: 1007, bad PEBs: 1, corrupted PEBs: 0
[    1.568712] ubi0: user volume: 3, internal volumes: 1, max. volumes count: 128
[    1.575919] ubi0: max/mean erase counter: 0/0, WL threshold: 4096, image sequence number: 1697192243
[    1.585036] ubi0: available PEBs: 0, total reserved PEBs: 1007, PEBs reserved for bad PEB handling: 19
[    1.594332] ubi0: background thread "ubi_bgt0d" started, PID 855

So it’s doing something to the NAND.

And this is the log after a reboot. Clearly some corruption has happened between the first boot and the second boot. If U-boot was to blame the errors should already appear in the first boot.

[    0.751984] spi-nand spi0.0: Winbond SPI NAND was found.
[    0.757296] spi-nand spi0.0: 128 MiB, block size: 128 KiB, page size: 2048, OOB size: 64
[    0.765642] 2 fixed-partitions partitions found on MTD device spi0.0
[    0.772025] OF: Bad cell count for /soc/spi@11007000/spi_nand@0/partitions
[    0.778900] OF: Bad cell count for /soc/spi@11007000/spi_nand@0/partitions
[    0.786052] Creating 2 MTD partitions on "spi0.0":
[    0.790833] 0x000000000000-0x000000200000 : "bl2"
[    0.797484] 0x000000200000-0x000008000000 : "ubi"
[    0.880410] ubi0: attaching mtd1
[    1.320050] ubi0: fixable bit-flip detected at PEB 533
[    1.350570] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 533:4096, read only 126976 bytes, retry
[    1.388231] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 533:4096, read only 126976 bytes, retry
[    1.425889] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 533:4096, read only 126976 bytes, retry
[    1.463452] ubi0 error: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 533:4096, read 126976 bytes
[    1.474488] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.8.0-rc3-next-20240207-bpi-r4-dango #1
[    1.482999] Hardware name: Banana Pi BPI-R4 (DT)
[    1.487602] Call trace:
[    1.490036]  dump_backtrace+0x98/0x118
[    1.493777]  show_stack+0x18/0x24
[    1.497079]  dump_stack_lvl+0x48/0x60
[    1.500732]  dump_stack+0x18/0x24
[    1.504035]  ubi_io_read+0x2c4/0x3e8
[    1.507598]  ubi_attach+0xb28/0x13a0
[    1.511163]  ubi_attach_mtd_dev+0x564/0xd84
[    1.515333]  ubi_notify_add+0x80/0xb4
[    1.518984]  add_mtd_device+0x4a0/0x670
[    1.522811]  add_mtd_partitions+0xfc/0x254
[    1.526895]  parse_mtd_partitions+0x2d0/0x590
[    1.531238]  mtd_device_parse_register+0x16c/0x374
[    1.536017]  spinand_probe+0x5ec/0x674
[    1.539755]  spi_mem_probe+0x6c/0xbc
[    1.543320]  spi_probe+0x84/0xe4
[    1.546538]  really_probe+0x184/0x3c8
[    1.550190]  __driver_probe_device+0x7c/0x16c
[    1.554535]  driver_probe_device+0x3c/0x110
[    1.558707]  __device_attach_driver+0xbc/0x158
[    1.563138]  bus_for_each_drv+0x80/0xdc
[    1.566962]  __device_attach+0x9c/0x1ac
[    1.570786]  device_initial_probe+0x14/0x20
[    1.574957]  bus_probe_device+0xac/0xb0
[    1.578781]  device_add+0x5d0/0x7c0
[    1.582257]  __spi_add_device+0x1a4/0x248
[    1.586254]  spi_add_device+0x4c/0x6c
[    1.589904]  of_register_spi_device+0x468/0x734
[    1.594422]  spi_register_controller+0x4f4/0x744
[    1.599026]  devm_spi_register_controller+0x4c/0xa4
[    1.603892]  mtk_spi_probe+0x508/0x698
[    1.607630]  platform_probe+0x68/0xc4
[    1.611281]  really_probe+0x184/0x3c8
[    1.614932]  __driver_probe_device+0x7c/0x16c
[    1.619278]  driver_probe_device+0x3c/0x110
[    1.623448]  __driver_attach+0xf4/0x1fc
[    1.627271]  bus_for_each_dev+0x74/0xd0
[    1.631094]  driver_attach+0x24/0x30
[    1.634658]  bus_add_driver+0x110/0x214
[    1.638482]  driver_register+0x60/0x128
[    1.642305]  __platform_driver_register+0x28/0x34
[    1.646998]  mtk_spi_driver_init+0x1c/0x28
[    1.651085]  do_one_initcall+0x44/0x260
[    1.654909]  kernel_init_freeable+0x200/0x3e4
[    1.659255]  kernel_init+0x20/0x1d4
[    1.662733]  ret_from_fork+0x10/0x20
[    1.667142] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 64 bytes from PEB 534:2048, read only 64 bytes, retry
[    1.679118] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 64 bytes from PEB 534:2048, read only 64 bytes, retry
[    1.691090] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 64 bytes from PEB 534:2048, read only 64 bytes, retry
[    1.703059] ubi0 error: ubi_io_read: error -74 (ECC error) while reading 64 bytes from PEB 534:2048, read 64 bytes
[    1.713399] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.8.0-rc3-next-20240207-bpi-r4-dango #1
[    1.721908] Hardware name: Banana Pi BPI-R4 (DT)
[    1.726510] Call trace:
[    1.728943]  dump_backtrace+0x98/0x118
[    1.732680]  show_stack+0x18/0x24
[    1.735982]  dump_stack_lvl+0x48/0x60
[    1.739631]  dump_stack+0x18/0x24
[    1.742934]  ubi_io_read+0x2c4/0x3e8
[    1.746497]  ubi_io_read_vid_hdr+0x58/0x310
[    1.750667]  ubi_attach+0x4a8/0x13a0
[    1.754232]  ubi_attach_mtd_dev+0x564/0xd84
[    1.758402]  ubi_notify_add+0x80/0xb4
[    1.762052]  add_mtd_device+0x4a0/0x670
[    1.765876]  add_mtd_partitions+0xfc/0x254
[    1.769959]  parse_mtd_partitions+0x2d0/0x590
[    1.774302]  mtd_device_parse_register+0x16c/0x374
[    1.779082]  spinand_probe+0x5ec/0x674
[    1.782818]  spi_mem_probe+0x6c/0xbc
[    1.786382]  spi_probe+0x84/0xe4
[    1.789599]  really_probe+0x184/0x3c8
[    1.793249]  __driver_probe_device+0x7c/0x16c
[    1.797594]  driver_probe_device+0x3c/0x110
[    1.801765]  __device_attach_driver+0xbc/0x158
[    1.806196]  bus_for_each_drv+0x80/0xdc
[    1.810019]  __device_attach+0x9c/0x1ac
[    1.813842]  device_initial_probe+0x14/0x20
[    1.818013]  bus_probe_device+0xac/0xb0
[    1.821837]  device_add+0x5d0/0x7c0
[    1.825312]  __spi_add_device+0x1a4/0x248
[    1.829309]  spi_add_device+0x4c/0x6c
[    1.832958]  of_register_spi_device+0x468/0x734
[    1.837477]  spi_register_controller+0x4f4/0x744
[    1.842082]  devm_spi_register_controller+0x4c/0xa4
[    1.846947]  mtk_spi_probe+0x508/0x698
[    1.850684]  platform_probe+0x68/0xc4
[    1.854334]  really_probe+0x184/0x3c8
[    1.857985]  __driver_probe_device+0x7c/0x16c
[    1.862329]  driver_probe_device+0x3c/0x110
[    1.866500]  __driver_attach+0xf4/0x1fc
[    1.870323]  bus_for_each_dev+0x74/0xd0
[    1.874147]  driver_attach+0x24/0x30
[    1.877710]  bus_add_driver+0x110/0x214
[    1.881534]  driver_register+0x60/0x128
[    1.885357]  __platform_driver_register+0x28/0x34
[    1.890050]  mtk_spi_driver_init+0x1c/0x28
[    1.894136]  do_one_initcall+0x44/0x260
[    1.897959]  kernel_init_freeable+0x200/0x3e4
[    1.902303]  kernel_init+0x20/0x1d4
[    1.905781]  ret_from_fork+0x10/0x20
[    2.295314] ubi0: scanning is finished
[    2.303606] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 22528 bytes from PEB 534:4096, read only 22528 bytes, retry
[    2.320203] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 22528 bytes from PEB 534:4096, read only 22528 bytes, retry
[    2.336812] ubi0 warning: ubi_io_read: error -74 (ECC error) while reading 22528 bytes from PEB 534:4096, read only 22528 bytes, retry
[    2.353400] ubi0 error: ubi_io_read: error -74 (ECC error) while reading 22528 bytes from PEB 534:4096, read 22528 bytes
[    2.364260] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.8.0-rc3-next-20240207-bpi-r4-dango #1
[    2.372769] Hardware name: Banana Pi BPI-R4 (DT)
[    2.377372] Call trace:
[    2.379805]  dump_backtrace+0x98/0x118
[    2.383543]  show_stack+0x18/0x24
[    2.386846]  dump_stack_lvl+0x48/0x60
[    2.390496]  dump_stack+0x18/0x24
[    2.393799]  ubi_io_read+0x2c4/0x3e8
[    2.397363]  ubi_read_volume_table+0x104/0xaa0
[    2.401794]  ubi_attach+0x390/0x13a0
[    2.405358]  ubi_attach_mtd_dev+0x564/0xd84
[    2.409527]  ubi_notify_add+0x80/0xb4
[    2.413176]  add_mtd_device+0x4a0/0x670
[    2.417001]  add_mtd_partitions+0xfc/0x254
[    2.421085]  parse_mtd_partitions+0x2d0/0x590
[    2.425428]  mtd_device_parse_register+0x16c/0x374
[    2.430207]  spinand_probe+0x5ec/0x674
[    2.433944]  spi_mem_probe+0x6c/0xbc
[    2.437509]  spi_probe+0x84/0xe4
[    2.440726]  really_probe+0x184/0x3c8
[    2.444376]  __driver_probe_device+0x7c/0x16c
[    2.448721]  driver_probe_device+0x3c/0x110
[    2.452892]  __device_attach_driver+0xbc/0x158
[    2.457323]  bus_for_each_drv+0x80/0xdc
[    2.461147]  __device_attach+0x9c/0x1ac
[    2.464970]  device_initial_probe+0x14/0x20
[    2.469141]  bus_probe_device+0xac/0xb0
[    2.472965]  device_add+0x5d0/0x7c0
[    2.476440]  __spi_add_device+0x1a4/0x248
[    2.480437]  spi_add_device+0x4c/0x6c
[    2.484087]  of_register_spi_device+0x468/0x734
[    2.488604]  spi_register_controller+0x4f4/0x744
[    2.493209]  devm_spi_register_controller+0x4c/0xa4
[    2.498073]  mtk_spi_probe+0x508/0x698
[    2.501810]  platform_probe+0x68/0xc4
[    2.505461]  really_probe+0x184/0x3c8
[    2.509111]  __driver_probe_device+0x7c/0x16c
[    2.513455]  driver_probe_device+0x3c/0x110
[    2.517625]  __driver_attach+0xf4/0x1fc
[    2.521449]  bus_for_each_dev+0x74/0xd0
[    2.525272]  driver_attach+0x24/0x30
[    2.528834]  bus_add_driver+0x110/0x214
[    2.532658]  driver_register+0x60/0x128
[    2.536481]  __platform_driver_register+0x28/0x34
[    2.541174]  mtk_spi_driver_init+0x1c/0x28
[    2.545259]  do_one_initcall+0x44/0x260
[    2.549082]  kernel_init_freeable+0x200/0x3e4
[    2.553427]  kernel_init+0x20/0x1d4
[    2.556904]  ret_from_fork+0x10/0x20
[    2.560483] ubi0 error: vtbl_check: bad CRC at record 1: 0x3d80bad4, not 0xe1f998b4
[    2.568138] Volume table record 1 dump:
[    2.571965] reserved_pebs   453
[    2.575179] alignment       1
[    2.578219] data_pad        0
[    2.581263] vol_type        1
[    2.584304] upd_marker      0
[    2.587344] name_len        6
[    2.590384] name            rootfs
[    2.593862] crc             0xe1f998b4
[    2.597684] ubi0 error: ubi_read_volume_table: both volume tables are corrupted
[    2.605074] ubi0 error: ubi_attach_mtd_dev: failed to attach mtd1, error -22

@frank-w I used you bookworm image “bpi-r4_bookworm_nokernel.img” as a base with a self-compiled 6.8-dango kernel from your Github-page.

[frank-w]

Kernel does read partitions from devicetree and creates mtd devices for partitions,imho nothing else. But imho it looks like your nand is broken you had also write errors on write above.

Wrspinand function is not called automaticly…i thought you have called it manually.

[chaotix]

Booting OpenWRT from SD does not cause the issue, so it has to be something in the kernel or Debian does something. I reproduced this behaviour several times now without fail. So it’s not some weird hardware glitch. Although it is of course possible that the bad NAND block is triggering this.

I’m compiling a new kernel without UBI support now and will check if this makes a difference. If it does not I will remove MTD support completely from the kernel and test that.

(I already contacted the seller because of the bad NAND block but as I could not buy from Sinovoip (because they don’t ship to Germany) I don’t think that I will get a replacement.)

[frank-w]

using my defconfig (as you see i also have mtd and some ubi-options) i have on 6.8-dango tree, my nand stays intact

root@bpi-r4-v11:~# uname -a
Linux bpi-r4-v11 6.8.0-rc3-next-20240207-bpi-r4-dango #2 SMP Sun Feb 11 13:58:48 CET 2024 aarch64 GNU/Linux
root@bpi-r4-v11:~# dmesg | grep -i 'ubi\|mtd'
[    0.763966] 2 fixed-partitions partitions found on MTD device spi0.0
[    0.784295] Creating 2 MTD partitions on "spi0.0":
[    0.795642] 0x000000200000-0x000008000000 : "ubi"
[    0.879204] ubi0: attaching mtd1
[    1.297783] ubi0: scanning is finished
[    1.301537] ubi0 error: ubi_read_volume_table: the layout volume was not found
[    1.308831] ubi0 error: ubi_attach_mtd_dev: failed to attach mtd1, error -22
[   11.067120] mtdblock: MTD device 'bl2' is NAND, please consider using UBI block devices instead.
[   11.068154] mtdblock: MTD device 'ubi' is NAND, please consider using UBI block devices instead.
root@bpi-r4-v11:~# zgrep -i 'ubi\|mtd' /proc/config.gz 
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
CONFIG_MTD=y
# OpenWrt specific MTD options
CONFIG_MTD_ROOTFS_ROOT_DEV=y
CONFIG_MTD_SPLIT_FIRMWARE=y
CONFIG_MTD_SPLIT_FIRMWARE_NAME="firmware"
CONFIG_MTD_SPLIT=y
CONFIG_MTD_SPLIT_SUPPORT=y
CONFIG_MTD_SPLIT_SQUASHFS_ROOT=y                                                                                  
# CONFIG_MTD_SPLIT_BCM63XX_FW is not set                                                                          
# CONFIG_MTD_SPLIT_BCM_WFI_FW is not set                                                                          
# CONFIG_MTD_SPLIT_SEAMA_FW is not set                                                                            
# CONFIG_MTD_SPLIT_WRGG_FW is not set                                                                             
# CONFIG_MTD_SPLIT_UIMAGE_FW is not set                                                                           
CONFIG_MTD_SPLIT_FIT_FW=y                                                                                         
# CONFIG_MTD_SPLIT_LZMA_FW is not set                                                                             
# CONFIG_MTD_SPLIT_TPLINK_FW is not set                                                                           
# CONFIG_MTD_SPLIT_TRX_FW is not set                                                                              
# CONFIG_MTD_SPLIT_BRNIMAGE_FW is not set                                                                         
# CONFIG_MTD_SPLIT_EVA_FW is not set                                                                              
# CONFIG_MTD_SPLIT_MINOR_FW is not set                                                                            
# CONFIG_MTD_SPLIT_JIMAGE_FW is not set                                                                           
# CONFIG_MTD_SPLIT_ELF_FW is not set                                                                              
# CONFIG_MTD_SPLIT_H3C_VFS is not set                                                                             
# end of OpenWrt specific MTD options                                                                             
# CONFIG_MTD_TESTS is not set                                                                                     
# CONFIG_MTD_CMDLINE_PARTS is not set                                                                             
CONFIG_MTD_OF_PARTS=y                                                                                             
# CONFIG_MTD_AFS_PARTS is not set                                                                                 
CONFIG_MTD_PARSER_TRX=y                                                                                           
# CONFIG_MTD_REDBOOT_PARTS is not set                                                                             
CONFIG_MTD_BLKDEVS=y                                                                                              
CONFIG_MTD_BLOCK=y                                                                                                
# Note that in some cases UBI block is preferred. See MTD_UBI_BLOCK.                                              
# CONFIG_MTD_OOPS is not set                                                                                      
# CONFIG_MTD_SWAP is not set                                                                                      
# CONFIG_MTD_PARTITIONED_MASTER is not set                                                                        
CONFIG_MTD_CFI=y                                                                                                  
# CONFIG_MTD_JEDECPROBE is not set                                                                                
CONFIG_MTD_GEN_PROBE=y                                                                                            
# CONFIG_MTD_CFI_ADV_OPTIONS is not set                                                                           
CONFIG_MTD_MAP_BANK_WIDTH_1=y                                                                                     
CONFIG_MTD_MAP_BANK_WIDTH_2=y                                                                                     
CONFIG_MTD_MAP_BANK_WIDTH_4=y                                                                                     
CONFIG_MTD_CFI_I1=y                                                                                               
CONFIG_MTD_CFI_I2=y                                                                                               
CONFIG_MTD_CFI_INTELEXT=y                                                                                         
CONFIG_MTD_CFI_AMDSTD=y                                                                                           
# CONFIG_MTD_CFI_STAA is not set                                                                                  
CONFIG_MTD_CFI_UTIL=y                                                                                             
# CONFIG_MTD_RAM is not set                                                                                       
# CONFIG_MTD_ROM is not set                                                                                       
# CONFIG_MTD_ABSENT is not set                                                                                    
CONFIG_MTD_COMPLEX_MAPPINGS=y                                                                                     
# CONFIG_MTD_PHYSMAP is not set                                                                                   
# CONFIG_MTD_PCI is not set                                                                                       
# CONFIG_MTD_INTEL_VR_NOR is not set                                                                              
# CONFIG_MTD_PLATRAM is not set                                                                                   
# Self-contained MTD device drivers                                                                               
# CONFIG_MTD_PMC551 is not set                                                                                    
# CONFIG_MTD_DATAFLASH is not set                                                                                 
# CONFIG_MTD_MCHP23K256 is not set                                                                                
# CONFIG_MTD_MCHP48L640 is not set                                                                                
# CONFIG_MTD_SST25L is not set                                                                                    
# CONFIG_MTD_SLRAM is not set                                                                                     
# CONFIG_MTD_PHRAM is not set                                                                                     
# CONFIG_MTD_MTDRAM is not set                                                                                    
# CONFIG_MTD_BLOCK2MTD is not set                                                                                 
# CONFIG_MTD_DOCG3 is not set                                                                                     
# end of Self-contained MTD device drivers                                                                        
CONFIG_MTD_NAND_CORE=y                                                                                            
# CONFIG_MTD_ONENAND is not set                                                                                   
CONFIG_MTD_RAW_NAND=y                                                                                             
# CONFIG_MTD_NAND_DENALI_PCI is not set                                                                           
# CONFIG_MTD_NAND_DENALI_DT is not set                                                                            
# CONFIG_MTD_NAND_CAFE is not set                                                                                 
# CONFIG_MTD_NAND_BRCMNAND is not set                                                                             
CONFIG_MTD_NAND_MTK=y                                                                                             
# CONFIG_MTD_NAND_MXIC is not set                                                                                 
# CONFIG_MTD_NAND_GPIO is not set                                                                                 
# CONFIG_MTD_NAND_PLATFORM is not set                                                                             
# CONFIG_MTD_NAND_CADENCE is not set                                                                              
# CONFIG_MTD_NAND_ARASAN is not set                                                                               
# CONFIG_MTD_NAND_INTEL_LGM is not set                                                                            
# CONFIG_MTD_NAND_NANDSIM is not set                                                                              
# CONFIG_MTD_NAND_RICOH is not set                                                                                
# CONFIG_MTD_NAND_DISKONCHIP is not set                                                                           
CONFIG_MTD_SPI_NAND=y                                                                                             
CONFIG_MTD_NAND_ECC=y                                                                                             
CONFIG_MTD_NAND_ECC_SW_HAMMING=y                                                                                  
# CONFIG_MTD_NAND_ECC_SW_HAMMING_SMC is not set                                                                   
# CONFIG_MTD_NAND_ECC_SW_BCH is not set                                                                           
# CONFIG_MTD_NAND_ECC_MXIC is not set                                                                             
CONFIG_MTD_NAND_ECC_MEDIATEK=y                                                                                    
CONFIG_MTD_NAND_MTK_BMT=y                                                                                         
# CONFIG_MTD_LPDDR is not set                                                                                     
CONFIG_MTD_SPI_NOR=y                                                                                              
CONFIG_MTD_SPI_NOR_USE_4K_SECTORS=y                                                                               
# CONFIG_MTD_SPI_NOR_SWP_DISABLE is not set                                                                       
CONFIG_MTD_SPI_NOR_SWP_DISABLE_ON_VOLATILE=y                                                                      
# CONFIG_MTD_SPI_NOR_SWP_KEEP is not set                                                                          
CONFIG_MTD_UBI=y                                                                                                  
CONFIG_MTD_UBI_WL_THRESHOLD=4096                                                                                  
CONFIG_MTD_UBI_BEB_LIMIT=20                                                                                       
# CONFIG_MTD_UBI_FASTMAP is not set                                                                               
# CONFIG_MTD_UBI_GLUEBI is not set                                                                                
CONFIG_MTD_UBI_BLOCK=y                                                                                            
# CONFIG_MTD_UBI_NVMEM is not set                                                                                 
# CONFIG_MTD_HYPERBUS is not set                                                                                  
# CONFIG_LEDS_TRIGGER_MTD is not set                                                                              
# CONFIG_JFFS2_RUBIN is not set                                                                                   
CONFIG_UBIFS_FS=y                                                                                                 
# CONFIG_UBIFS_FS_ADVANCED_COMPR is not set                                                                       
CONFIG_UBIFS_FS_LZO=y                                                                                             
CONFIG_UBIFS_FS_ZLIB=y                                                                                            
CONFIG_UBIFS_FS_ZSTD=y                                                                                            
# CONFIG_UBIFS_ATIME_SUPPORT is not set                                                                           
CONFIG_UBIFS_FS_XATTR=y                                                                                           
CONFIG_UBIFS_FS_SECURITY=y                                                                                        
# CONFIG_UBIFS_FS_AUTHENTICATION is not set                                                                       
root@bpi-r4-v11:~#

powered off changed first bootswitch to up position and booted again (now nand)

F0: 102B 0000                                                                                                     
FA: 1042 0000                                                                                                     
FA: 1042 0000 [0200]                                                                                              
F9: 0000 0000                                                                                                     
V0: 0000 0000 [0001]                                                                                              
00: 0000 0000                                                                                                     
BP: 0600 0041 [0000]                                                                                              
G0: 1190 0000                                                                                                     
EC: 0000 0000 [1000]                                                                                              
MK: 0000 0000 [0000]                                                                                              
T0: 0000 01A0 [0101]                                                                                              
Jump to BL                                                                                                        
                                                                                                                  
NOTICE:  BL2: v2.9(release):v2.9.0-356-g1743e388fb1f spim-nand                                                    
NOTICE:  BL2: Built : 19:08:00, Feb 13 2024                                                                       
NOTICE:  WDT: Cold boot                                                                                           
NOTICE:  WDT: disabled                                                                                            
NOTICE:  CPU: MT7988                                                                                              
NOTICE:  EMI: Using DDR unknown settings                                                                          
NOTICE:  EMI: Detected DRAM size: 4096 MB                                                                         
NOTICE:  EMI: complex R/W mem test passed                                                                         
NOTICE:  SPI_NAND parses attributes from parameter page.                                                          
NOTICE:  SPI_NAND Detected ID 0xef                                                                                
NOTICE:  Page size 2048, Block size 131072, size 134217728                                                        
NOTICE:  BL2: Booting BL31                                                                                        
NOTICE:  BL31: v2.9(release):v2.9.0-356-g1743e388fb1f spim-nand                                                   
NOTICE:  BL31: Built : 19:08:04, Feb 13 2024                                                                      
                                                                                                                  
                                                                                                                  
U-Boot 2024.04-rc1-bpi-00020-g425db641eb1e-dirty (Feb 13 2024 - 19:06:30 +0100)                                   
                                                                                                                  
CPU:   MediaTek MT7988                                                                                            
Model: mt7988-rfb                                                                                                 
DRAM:  4 GiB                                                                                                      
Core:  53 devices, 19 uclasses, devicetree: separate                                                              
MMC:   mmc@11230000: 0                                                                                            
Loading Environment from nowhere... OK                                                                            
In:    serial@11000000                                                                                            
Out:   serial@11000000                                                                                            
Err:   serial@11000000                                                                                            
Net:                                                                                                              
Warning: ethernet@15100000 (eth0) using random MAC address - 5e:46:9c:8f:31:68                                    
eth0: ethernet@15100000                                                                                           
Hit any key to stop autoboot:  0                                                                                  
BPI-R4>

[chaotix]

Then it seems to be the kernel that is causing the problems. Maybe it tries to fix something and messes it up. (Although I thinks this would be a weird and unexpected behavior.) I had the same UBI kernel settings enabled, disabled them and now NAND stays intact after a boot. (It would be still possible (but unlikely) that something from userland messes with the UBI partitions and now can’t because the drivers are not compiled in.)

I checked the OpenWRT SD-version and it did not have UBI compiled in as well. The NAND version does, of course, but it looks to me that UBI is maybe initialized by U-boot. At least there are UBI messages before the first kernel messages. So maybe the kernel knows that it is in use already and does not try to fix anything. (Pure speculation on my part.)

ubi0: attaching mtd6
UBI: EOF marker found, PEBs from 488 will be erased
ubi0: scanning is finished
ubi0: volume 2 ("rootfs_data") re-sized from 9 to 391 LEBs
ubi0: attached mtd6 (name "ubi", size 112 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 900, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 3, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 1/0, WL threshold: 4096, image sequence number: 1697192243
ubi0: available PEBs: 0, total reserved PEBs: 900, PEBs reserved for bad PEB handling: 19
No size specified -> Using max size (4190208)
Read 4190208 bytes from volume kernel to 0000000046000000
## Loading kernel from FIT Image at 46000000 ...
   Using 'config-1' configuration
   Trying 'kernel-1' kernel subimage
     Description:  ARM64 OpenWrt Linux-5.4.246
     Type:         Kernel Image
     Compression:  lzma compressed
     Data Start:   0x460000e8
     Data Size:    4060807 Bytes = 3.9 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x48080000
     Entry Point:  0x48080000
     Hash algo:    crc32
     Hash value:   cacd2182
     Hash algo:    sha1
     Hash value:   8cb587f5cb585cabfca1142602360b63aff9440f
   Verifying Hash Integrity ... crc32+ sha1+ OK
## Loading fdt from FIT Image at 46000000 ...
   Using 'config-1' configuration
   Trying 'fdt-1' fdt subimage
     Description:  ARM64 OpenWrt BPI-R4-NAND device tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x463df8ac
     Data Size:    34547 Bytes = 33.7 KiB
     Architecture: AArch64
     Hash algo:    crc32
     Hash value:   dab10d5e
     Hash algo:    sha1
     Hash value:   ac75d9bde8766c8cfbecee360a488f7bd89b5108
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Booting using the fdt blob at 0x463df8ac
Working FDT set to 463df8ac
   Uncompressing Kernel Image
   Loading Device Tree to 00000000ff7ec000, end 00000000ff7f76f2 ... OK
Working FDT set to ff7ec000
Add 'ramoops@42ff0000' node failed: FDT_ERR_EXISTS

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd090]
[    0.000000] Linux version 5.4.246 ([email protected]) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 unknown)) #0 SMP Fri Oct 13 10:17:23 2023
[    0.000000] Machine model: Bananapi BPI-R4

One more thing that I find strange is that the size of the UBI volume is detected differently between U-boot(?) and 6.8-dango:

OpenWRT (U-boot?):

ubi0: volume 2 ("rootfs_data") re-sized from 9 to 391 LEBs
ubi0: attached mtd6 (name "ubi", size 112 MiB)

Linux 6.8-dango:

[    1.529628] ubi0: volume 2 ("rootfs_data") re-sized from 9 to 498 LEBs
[    1.536560] ubi0: attached mtd1 (name "ubi", size 126 MiB)

[kevok]

Hi all, apologies if the two are not related but I am seeing the same thing after flashing the new openwrt mainline to eMMC on a r3-mini; as with chaotix I only touched /dev/mmcblk0, yet neither the NAND nor the eMMC partition are booting now, both stuck at the same BL2: Failed to load image id 3. Since I’m on the mini, is there any hope for recovery?

F0: 102B 0000
FA: 1040 0000
FA: 1040 0000 [0200]
F9: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 2400 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [1000]
T0: 0000 021E [010F]
Jump to BL

NOTICE:  BL2: v2.6(release):
NOTICE:  BL2: Built : 20:18:14, May  7 2023
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7986 (2000MHz)
NOTICE:  EMI: Using DDR4 settings
NOTICE:  EMI: Detected DRAM size: 2048MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef
NOTICE:  Page size 2048, Block size 131072, size 134217728
NOTICE:  Initializing NMBM ...
NOTICE:  Signature found at block 1023 [0x07fe0000]
NOTICE:  First info table with writecount 0 found in block 960
NOTICE:  Second info table with writecount 0 found in block 963
NOTICE:  NMBM has been successfully attached in read-only mode
ERROR:   BL2: Failed to load image id 3 (-2)

[dangowrt (Daniel Golle)]

@kevok

NOTICE:  BL2: v2.6(release):
NOTICE:  BL2: Built : 20:18:14, May  7 2023
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7986 (2000MHz)
NOTICE:  EMI: Using DDR4 settings
NOTICE:  EMI: Detected DRAM size: 2048MB
NOTICE:  EMI: complex R/W mem test passed
NOTICE:  SPI_NAND parses attributes from parameter page.
NOTICE:  SPI_NAND Detected ID 0xef
NOTICE:  Page size 2048, Block size 131072, size 134217728
NOTICE:  Initializing NMBM ...

This is the SinoVoip-built stock firmware booting, not OpenWrt. Please do not hijack the thread for the R4 for that, but rather open a new thread discussing any difficulties you may have with the R3 mini with stock firmware, OpenWrt, ImmortalWrt or whatever you are using.