[BPI-R64] Uboot error when rootfs size > 40MB

Banana Pi BPI-R4/BPI-R3 Mini/BPI-R3/BPI-R64/BPI-R2
1

I need docker service. After applying docker service in openwrt master branch. Rootfs and recovery will be larger. Now openwrt MediaTek mt7622 bananapi_ Bpi-r64-sdcard.img.gz is about 117mb

I modified target / Linux / MediaTek / image / mt7622.mk as follows

diff --git a/target/linux/mediatek/image/mt7622.mk b/target/linux/mediatek/image/mt7622.mk
index 6077e623ff..e0f37f9f49 100644
--- a/target/linux/mediatek/image/mt7622.mk
+++ b/target/linux/mediatek/image/mt7622.mk
@@ -45,13 +45,13 @@ define Build/mt7622-gpt
                ) \
                        -t 0xef -N fip          -r      -p 2M@2M \
                        -t 0x83 -N ubootenv     -r      -p 1M@4M \
-                               -N recovery     -r      -p 32M@6M \
+                               -N recovery     -r      -p 64M@6M \
                $(if $(findstring sdmmc,$1), \
-                               -N install      -r      -p 7M@38M \
-                       -t 0x2e -N production           -p 211M@45M \
+                               -N install      -r      -p 7M@70M \
+                       -t 0x2e -N production           -p 422M@77M \
                ) \
                $(if $(findstring emmc,$1), \
-                       -t 0x2e -N production           -p 980M@40M \
+                       -t 0x2e -N production           -p 980M@72M \
                )
        cat [email protected] >> $@
        rm [email protected]
@@ -87,12 +87,12 @@ define Device/bananapi_bpi-r64
                                   pad-to 512k | bl2 sdmmc-2ddr |\
                                   pad-to 2048k | bl31-uboot bananapi_bpi-r64-sdmmc |\
                                   pad-to 6144k | append-image-stage initramfs-recovery.itb |\
-                                  pad-to 38912k | mt7622-gpt emmc |\
-                                  pad-to 39424k | bl2 emmc-2ddr |\
-                                  pad-to 40960k | bl31-uboot bananapi_bpi-r64-emmc |\
-                                  pad-to 43008k | bl2 snand-2ddr |\
-                                  pad-to 43520k | bl31-uboot bananapi_bpi-r64-snand |\
-                                  pad-to 46080k | append-image squashfs-sysupgrade.itb | gzip
+                                  pad-to 71680k | mt7622-gpt emmc |\
+                                  pad-to 72192k | bl2 emmc-2ddr |\
+                                  pad-to 72704k | bl31-uboot bananapi_bpi-r64-emmc |\
+                                  pad-to 75264k | bl2 snand-2ddr |\
+                                  pad-to 75776k | bl31-uboot bananapi_bpi-r64-snand |\
+                                  pad-to 78848k | append-image squashfs-sysupgrade.itb | gzip
   KERNEL                       := kernel-bin | gzip
   KERNEL_INITRAMFS             := kernel-bin | lzma | fit lzma $$(DTS_DIR)/$$(DEVICE_DTS).dtb with-initrd | pad-to 128k
   IMAGE/sysupgrade.itb         := append-kernel | fit gzip $$(DTS_DIR)/$$(DEVICE_DTS).dtb external-static-with-rootfs | append-metadata

CRC 32 error after booting rootfs

switch to partitions #0, OK
mmc1 is current device

MMC read: dev # 1, block # 157696, count 8 ... 8 blocks read: OK

MMC read: dev # 1, block # 157696, count 135624 ... 135624 blocks read: OK
## Loading kernel from FIT Image at 4007ff28 ...
   Using 'config-mt7622-bananapi-bpi-r64-pcie1' configuration
   Trying 'kernel-1' kernel subimage
     Description:  ARM64 OpenWrt Linux-5.10.35
     Type:         Kernel Image
     Compression:  gzip compressed
     Data Start:   0x40080f28
     Data Size:    5288381 Bytes = 5 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x44000000
     Entry Point:  0x44000000
     Hash algo:    crc32
     Hash value:   eed74fd6
     Hash algo:    sha1
     Hash value:   ada2c8f3727ee90fed9cb7e6a960b43159c9ccd7
   Verifying Hash Integrity ... crc32+ sha1+ OK
## Loading fdt from FIT Image at 4007ff28 ...
   Using 'config-mt7622-bananapi-bpi-r64-pcie1' configuration
   Trying 'fdt-1' fdt subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 device tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x4058cf28
     Data Size:    31796 Bytes = 31.1 KiB
     Architecture: AArch64
     Load Address: 0x43ff83cc
     Hash algo:    crc32
     Hash value:   5f04dd9c
     Hash algo:    sha1
     Hash value:   b6b5aa7f9adb7cdfb82e98a1366b88e5e6a182c0
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x4058cf28 to 0x43ff83cc
## Loading fdt from FIT Image at 4007ff28 ...
   Trying 'fdt-mt7622-bananapi-bpi-r64-pcie1' fdt subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 device tree overlay mt7622-bananapi-bpi-r64-pcie1
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x40594f28
     Data Size:    408 Bytes = 408 Bytes
     Architecture: AArch64
     Load Address: 0x43ff8234
     Hash algo:    crc32
     Hash value:   7a017eca
     Hash algo:    sha1
     Hash value:   1c9ab4777c11f5fedb5674c69e7e4c456960fe8a
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x40594f28 to 0x43ff8234
   Booting using the fdt blob at 0x43ff83cc
## Loading loadables from FIT Image at 4007ff28 ...
   Trying 'rootfs-1' loadables subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 rootfs
     Type:         Filesystem Image
     Compression:  uncompressed
     Data Start:   0x40596f28
     Data Size:    64102400 Bytes = 61.1 MiB
     Hash algo:    crc32
     Hash value:   84e7f94d
     Hash algo:    sha1
     Hash value:   ea2152c599600dd392ec96af9140463c94c48fc1
   Verifying Hash Integrity ... crc32 error!
Bad hash value for 'hash@1' hash node in 'rootfs-1' image node
Bad Data Hash
Loadable(s) is corrupt or invalid
switch to partitions #0, OK
mmc1 is current device

MMC read: dev # 1, block # 12288, count 8 ... 8 blocks read: OK

MMC read: dev # 1, block # 12288, count 93044 ... 93044 blocks read: OK
## Loading kernel from FIT Image at 4007ff28 ...
   Using 'config-mt7622-bananapi-bpi-r64-pcie1' configuration
   Trying 'kernel-1' kernel subimage

There are also “reserved-memory/FDT_ERR_EXISTS” errors

switch to partitions #0, OK
mmc1 is current device

MMC read: dev # 1, block # 157696, count 8 ... 8 blocks read: OK

MMC read: dev # 1, block # 157696, count 122864 ... 122864 blocks read: OK
## Loading kernel from FIT Image at 4007ff28 ...
   Using 'config-mt7622-bananapi-bpi-r64-pcie1' configuration
   Trying 'kernel-1' kernel subimage
     Description:  ARM64 OpenWrt Linux-5.10.35
     Type:         Kernel Image
     Compression:  gzip compressed
     Data Start:   0x40080f28
     Data Size:    5288263 Bytes = 5 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x44000000
     Entry Point:  0x44000000
     Hash algo:    crc32
     Hash value:   8ae05828
     Hash algo:    sha1
     Hash value:   754874c0096326cb2a5b21bdce7c062390e4251f
   Verifying Hash Integrity ... crc32+ sha1+ OK
## Loading fdt from FIT Image at 4007ff28 ...
   Using 'config-mt7622-bananapi-bpi-r64-pcie1' configuration
   Trying 'fdt-1' fdt subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 device tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x4058cf28
     Data Size:    31796 Bytes = 31.1 KiB
     Architecture: AArch64
     Load Address: 0x43ff83cc
     Hash algo:    crc32
     Hash value:   5f04dd9c
     Hash algo:    sha1
     Hash value:   b6b5aa7f9adb7cdfb82e98a1366b88e5e6a182c0
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x4058cf28 to 0x43ff83cc
## Loading fdt from FIT Image at 4007ff28 ...
   Trying 'fdt-mt7622-bananapi-bpi-r64-pcie1' fdt subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 device tree overlay mt7622-bananapi-bpi-r64-pcie1
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x40594f28
     Data Size:    408 Bytes = 408 Bytes
     Architecture: AArch64
     Load Address: 0x43ff8234
     Hash algo:    crc32
     Hash value:   7a017eca
     Hash algo:    sha1
     Hash value:   1c9ab4777c11f5fedb5674c69e7e4c456960fe8a
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x40594f28 to 0x43ff8234
   Booting using the fdt blob at 0x43ff83cc
## Loading loadables from FIT Image at 4007ff28 ...
   Trying 'rootfs-1' loadables subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 rootfs
     Type:         Filesystem Image
     Compression:  uncompressed
     Data Start:   0x40596f28
     Data Size:    57569280 Bytes = 54.9 MiB
     Hash algo:    crc32
     Hash value:   cc899a00
     Hash algo:    sha1
     Hash value:   0c9f4e6c926aa7cad91b829fc843ee7e090f8d93
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Uncompressing Kernel Image
   Loading Device Tree to 000000007f7f0000, end 000000007f7fac2f ... OK
Add 'reserved-memory' node failed: FDT_ERR_EXISTS

Starting kernel ...

how to solve this problem… thk!

2

i guess it is a kind of overflow…the target-position ( 0x40596f28) imho after the 1GB of RAM (1029MB)

i’m not sure how this position is calculated/set, but maybe someone knows this…

3

Simply speaking: Don’t put docker into the recovery image. The best way to achieve this is by using the OpenWrt ImageBuilder.

Step-by-step (assuming SD Card is accessible via /dev/mmcblk0, change to /dev/sdX or whatever it ends up to be on your build host):

wget -O- https://downloads.openwrt.org/snapshots/targets/mediatek/mt7622/openwrt-imagebuilder-mediatek-mt7622.Linux-x86_64.tar.xz | tar xJ
cd openwrt-imagebuilder-mediatek-mt7622.Linux-x86_64
make image PROFILE=bananapi_bpi-r64 PACKAGES="docker kmod-veth"
gzip -cd bin/target/mediatek/mt7622/openwrt-mediatek-mt7622-bananapi_bpi-r64-sdcard.img.gz | dd of=/dev/mmcblk0

You can, of course, add more additional packages or use podman instead of docker if you like :wink:

I must admit I never ended up with squashfs that huge, the above example generating an image with docker ends up to be around 12MiB rootfs.

4

thank you.

I’ll try it.

I don’t know if I can configure the package in the recovery image

I think we can add a makefile configuration variable to exclude the integration of some packages in the recovery image.

In addition, can you provide a compilation configuration scheme for ext4? I choose ext4 to compile and report errors.

5

sorry. I don’t quite understand how to adjust the load variable. Can you give us more references? thank you.

6

The SD card image generated by podman can boot normally.

But there is an error writing EMMC.

boot log on emmc. ps:After modifying the recovery to 32MB (I changed it to 64MB), the boot is normal.

MMC erase: dev # 0, block # 147456, count 2007040 ... 2007040 blocks erased: OK

MMC write: dev # 0, block # 147456, count 71952 ... 71952 blocks written: OK
Saving Environment to MMC... Writing to redundant MMC(1)... OK
Saving Environment to MMC... Writing to MMC(1)... OK
Press ENTER to return to menu 


      ( ( ( OpenWrt ) ) )  [SD card]       U-Boot 2021.04 (May 11 2021 - 00:05:38 +0000)

     1. Run default boot command.
     2. Boot system via TFTP.
     3. Boot production system from SD card.
     4. Boot recovery system from SD card.
     5. Load production system via TFTP then write to SD card.
     6. Load recovery system via TFTP then write to SD card.
     7. Install bootloader, recovery and production to eMMC.
     8. Install bootloader, recovery and production to NAND.
     9. Reboot.
     a. Reset all settings to factory defaults.
     0. U-Boot console

  Hit any key to stop autoboot: 58 
F0: 102B 0000WN to move, ENTER to select, ESC/CTRL+C to quit
F5: 480A 0031
F5: 480A 0031
F3: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0400 0041 [0000]
G0: 1190 0000
T0: 0000 02D0 [000F]
Jump to BL

NOTICE:  BL2: v2.4(release):OpenWrt v2021-05-08-d2c75b21-1 (mt7622-emmc-2ddr)
NOTICE:  BL2: Built : 00:05:38, May 11 2021
ERROR:   BL2: Failed to load image id 3 (-2)

F0: 102B 0000
F5: 480A 0031
F5: 480A 0031
F3: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0400 0041 [0000]
G0: 1190 0000
T0: 0000 02D8 [000F]
Jump to BL

NOTICE:  BL2: v2.4(release):OpenWrt v2021-05-08-d2c75b21-1 (mt7622-emmc-2ddr)
NOTICE:  BL2: Built : 00:05:38, May 11 2021
ERROR:   BL2: Failed to load image id 3 (-2)

boot on sdcard. is right now…

switch to partitions #0, OK
mmc1 is current device

MMC read: dev # 1, block # 157696, count 8 ... 8 blocks read: OK

MMC read: dev # 1, block # 157696, count 71952 ... 71952 blocks read: OK
## Loading kernel from FIT Image at 4007ff28 ...
   Using 'config-mt7622-bananapi-bpi-r64-pcie1' configuration
   Trying 'kernel-1' kernel subimage
     Description:  ARM64 OpenWrt Linux-5.10.35
     Type:         Kernel Image
     Compression:  gzip compressed
     Data Start:   0x40080f28
     Data Size:    5053714 Bytes = 4.8 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x44000000
     Entry Point:  0x44000000
     Hash algo:    crc32
     Hash value:   c9c94ea9
     Hash algo:    sha1
     Hash value:   84b26e0bd5d03cd5a43a0973153e4b6b102dcf4e
   Verifying Hash Integrity ... crc32+ sha1+ OK
## Loading fdt from FIT Image at 4007ff28 ...
   Using 'config-mt7622-bananapi-bpi-r64-pcie1' configuration
   Trying 'fdt-1' fdt subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 device tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x40552f28
     Data Size:    31796 Bytes = 31.1 KiB
     Architecture: AArch64
     Load Address: 0x43ff83cc
     Hash algo:    crc32
     Hash value:   5f04dd9c
     Hash algo:    sha1
     Hash value:   b6b5aa7f9adb7cdfb82e98a1366b88e5e6a182c0
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x40552f28 to 0x43ff83cc
## Loading fdt from FIT Image at 4007ff28 ...
   Trying 'fdt-mt7622-bananapi-bpi-r64-pcie1' fdt subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 device tree overlay mt7622-bananapi-bpi-r64-pcie1
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x4055af28
     Data Size:    408 Bytes = 408 Bytes
     Architecture: AArch64
     Load Address: 0x43ff8234
     Hash algo:    crc32
     Hash value:   7a017eca
     Hash algo:    sha1
     Hash value:   1c9ab4777c11f5fedb5674c69e7e4c456960fe8a
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x4055af28 to 0x43ff8234
   Booting using the fdt blob at 0x43ff83cc
## Loading loadables from FIT Image at 4007ff28 ...
   Trying 'rootfs-1' loadables subimage
     Description:  ARM64 OpenWrt bananapi_bpi-r64 rootfs
     Type:         Filesystem Image
     Compression:  uncompressed
     Data Start:   0x4055cf28
     Data Size:    31739904 Bytes = 30.3 MiB
     Hash algo:    crc32
     Hash value:   14198681
     Hash algo:    sha1
     Hash value:   0cb01faf2fad95c8e2f06a01d2f68be09bc0dc30
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Uncompressing Kernel Image
   Loading Device Tree to 000000007f7f0000, end 000000007f7fac2f ... OK
Add 'reserved-memory' node failed: FDT_ERR_EXISTS

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 5.10.35 (darcy@Darcy-NH50-70RA) (aarch64-openwrt-linux-musl-gcc (OpenWrt GCC 8.4.0 r16708-e7249669d2) 8.4.0, GNU ld (GNU Binutils) 2.34) #0 SMP Tue May 11 00:05:38 2021
[    0.000000] Machine model: Bananapi BPI-R64
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000040000000-0x000000007fffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x0000000042ffffff]
[    0.000000]   node   0: [mem 0x0000000043000000-0x000000004302ffff]
[    0.000000]   node   0: [mem 0x0000000043030000-0x000000007fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000007fffffff]
[    0.000000] On node 0 totalpages: 262144
[    0.000000]   DMA zone: 4096 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 262144 pages, LIFO batch:63
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 20 pages/cpu s43544 r8192 d30184 u81920
[    0.000000] pcpu-alloc: s43544 r8192 d30184 u81920 alloc=20*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1
7

Just to make sure I understood correctly: You wrote everything to eMMC by selecting the option

     7. Install bootloader, recovery and production to eMMC.

in the bootloader menu of the SD Card generated by the OpenWrt ImageBuilder you have downloaded from downloads.openwrt.org, right?

8

Yes, I chose to burn from SD card to EMMC.

9

Sorry, it’s still not clear enough for me to understand what happened. The image was generated using the ImageBuilder from downloads.openwrt.org ? And you have used the bootloader menu to write it to eMMC? (just copying the SD Card to eMMC using dd and such will not work, different bootloader is required to work on eMMC!)

I’m asking because I’m using this myself and it boots fine, so in that case we have to investigate further.