[BPi-R3-Mini] Review and mainline ImmortalWrt firmware support

hi, I flashed emmc on immortalwrt 23.05.1 and now I’m trying to flash nand immortalwrt, I’m cleaning and updating bl2 and fip and when I boot from nand I get an error like in the screenshot. Help, how to fix booting from nand?

Is this change still relevant? Do I have to adjust the .dts too? I finally got my antenna but they’ve sent me the RM500U-CNV instead of the RM500U-CN

I compiled the immortalwrt source and installed it on the emmc. This is my kernel log:

cat /sys/kernel/debug/usb/devices 
us=02 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  2 Spd=5000 MxCh= 0
D:  Ver= 3.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 9 #Cfgs=  1
P:  Vendor=2c7c ProdID=0900 Rev= 4.04
S:  Manufacturer=Quectel
S:  Product=RM500U-CNV
S:  SerialNumber=0123456789ABCDEF
C:* #Ifs= 7 Cfg#= 1 Atr=c0 MxPwr=504mA
A:  FirstIf#= 0 IfCount= 2 Cls=02(comm.) Sub=0d Prot=00
I:* If#= 0 Alt= 0 #EPs= 1 Cls=02(comm.) Sub=0d Prot=00 Driver=cdc_ncm
E:  Ad=82(I) Atr=03(Int.) MxPS=  16 Ivl=32ms
I:  If#= 1 Alt= 0 #EPs= 0 Cls=0a(data ) Sub=00 Prot=01 Driver=cdc_ncm
I:* If#= 1 Alt= 1 #EPs= 2 Cls=0a(data ) Sub=00 Prot=01 Driver=cdc_ncm
E:  Ad=81(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms
E:  Ad=01(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms
I:* If#= 2 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=00 Prot=00 Driver=option
E:  Ad=83(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms
E:  Ad=02(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms
I:* If#= 3 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=00 Prot=00 Driver=option
E:  Ad=84(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms
E:  Ad=03(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms
I:* If#= 4 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=00 Prot=00 Driver=option
E:  Ad=85(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms
E:  Ad=04(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms
I:* If#= 5 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=00 Prot=00 Driver=option
E:  Ad=86(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms
E:  Ad=05(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms
I:* If#= 6 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=00 Prot=00 Driver=option
E:  Ad=87(I) Atr=02(Bulk) MxPS=1024 Ivl=0ms
E:  Ad=06(O) Atr=02(Bulk) MxPS=1024 Ivl=0ms

So it seems the 5G Module is found. But when I add an interface with the QMI Protocol I get:

Protocol: QMI Cellular
RX: 0 B (0 Pkts.)
TX: 0 B (0 Pkts.)
Error: Network device is not present

and this is the system log:

Sat Jan 20 05:55:28 2024 daemon.notice netifd: WWAN (2517): ls: /net: No such file or directory
Sat Jan 20 05:55:28 2024 daemon.notice netifd: WWAN (2517): The interface could not be found.
Sat Jan 20 05:55:28 2024 daemon.notice netifd: wan (2583): udhcpc: started, v1.36.1
Sat Jan 20 05:55:28 2024 daemon.err odhcp6c[2595]: Failed to send RS (Address not available)
Sat Jan 20 05:55:28 2024 daemon.notice netifd: WWAN (2596): Stopping network WWAN
Sat Jan 20 05:55:28 2024 daemon.notice netifd: WWAN (2596): Command failed: ubus call network.interface notify_proto { "action": 0, "link-up": false, "keep": false, "interface": "WWAN" } (Permission denied)
Sat Jan 20 05:55:28 2024 daemon.notice netifd: Interface 'WWAN' is now down

Any idea what this means? I haven’t inserted a SIM Card yet. Is it because of that? Or do I need to send an AT Command?

Also: I configured tftp-server on my fedora39 laptop and set a static ip (192.168.1.254) in the NetworkManager. But when booting u-boot on the banana pi both ethernet ports wouldn’t work. What I did is to dd the immortalwrt-mediatek-filogic-bananapi_bpi-r3-mini-squashfs-sysupgrade.itb with:

dd if=immortalwrt-mediatek-filogic-bananapi_bpi-r3-mini-squashfs-sysupgrade.itb of=/dev/mmcblk0p5 bs=512 conv=fsync

after doing this immortalwrt booted but when switching back to NAND i now get a boot loop.

EMMC with immortalWRT boots fine. but the NAND just boot loops. I will flash the original image on the NAND and see if I then can boot from EMMC and NAND again

From the logs, your card is booting using ECM mode. To use QMI or MBIM, you have to send AT commands. install picocom tool to send this command to switch QMI mode AT+QCFG=“usbnet”,0

Thank you! I will try this after i made sure the original image on the nand works (don’t want to end up with a bricked device) So the changes you made in the .dts should be obsolete? I already compiled a new images with your changes to the dts but I keep them around

Question about flashing the NAND. The official documentation says this:

 4. Execute following command to erase the whole Nand flash and copy image to nand device:
    * mtd erase /dev/mtd0
    * dd if=mtk-bpi-r3mini-NAND-20230719-single-image.bin of=/dev/mtdblock0

mtd erase /dev/mtd0 outputs: Could not open mtd device: mtd0

I couldn’t find good answers via google

You have to run the following commands prior to start erasing and writing the nand

opkg install kmod-mtd-rw

insmod mtd-rw i_want_a_brick=y

Based on past experience, following the official instruction was not okay for me. From EMMC boot-loader console I had to erase the nand and write bl2 and bl31 through tftpboot command

Also, I still keep dts changes to automatically detect the 5g card.

Thanks again for the fast reply! I’m really greatful for your help. My biggest issue is that with the immortalWRT u-boot I have no ethernet link and I can’t serve files using tftp :frowning:

I will have to use dd to get the initramfs image to the nand. Do you know by any chance to which mtdblock 0 - 4 i have to dd the initramfs image?

I just found this thread about erasing and flashing the nand: BPI-R3 Mini NAND flash problem

I think I misunderstood something. Can I only run either immortalWRT or the official image on EMMC and NAND? I can’t have the official on NAND and immortalWRT on emmc?

I had tested before, you can have two different images on nand and emmc. For your case, I think the referenced URL in ur post has the solution and you can follow the @windows10 instructions.

Thanks! I succeded in booting from NAND again. Funny enough, now the EMMC boot loops (more down below). Nonenthless let me share the steps I did:

Steps:

  1. Flip the Switch to boot from EMMC
  2. Don’t boot into openwrt/immortalwrt but start the U-BOOT console
  3. Type “usb start” to load the USB Controller (didn’t work for me otherwise)
  4. Type “usb tree” to find attached USB Devices
  5. Type “usb storage”, verify that the USB Drive is the same as in tree and remember the the Device Number. Mine was Device 0:
  6. Type “ls usb 0:X” ( X is the number you had to remember) to list the files on the USB Drive and verify your image is there
  7. Type “fatload usb 0:X $loadaddr firmware.bin” to load firmware (again type the device number here for X). firmware.bin is the image you want to put on the NAND
  8. Type “nand info” to get information about the NAND
  9. Erase the ubi block with “nand erase ubi” (Disclaimer: frank-w in is tutorial deleted the whole spi-nand - but deleting only ubi worked for me)
  10. Write the image you loaded before with “nand write $loadaddr 0 0x8000000”
  11. Power off the R3-Mini
  12. Flip the Switch to NAND and pray it worked

Unfortunately this will bring the stock image in NAND back and it will boot but for me, once i switched to EMMC, immortalWRT started boot looping. Maybe my custom compiled image is at fault but at least now i can boot from nand and try another immortalWRT build to see if this helps.

I have modified your steps to help you boot immortalWRT image from the nand.

Here you go:

1- Put bl2, bl31 and initramfs images into USB flash.

2- Form EMMC uboot console, fatload usb 0:X $loadaddr bl2_nand.bin

3- Erase the bl2 block with nand erase bl2

4- nand write $loadaddr 0x00000 0x100000

5- fatload usb 0:X $loadaddr bl3_nand.fip

6- Erase the bl3 block with nand erase fip

7- nand write $loadaddr 0x380000 0x200000

8- Flip the Switch to boot from nand

9- Form nand uboot console, fatload usb 0:X $loadaddr initramfs.itb

10- bootm ${loadaddr}

You boot till uboot here and load kernel+rootfs from usb. Is there a way to flash immortalwrt to ubi-partition?

@petraant basicly it should be all erased what is written afterwards. So if you only erase ubi,the bl2+fip may stay old or cause strange issues like the bootloop

Is there a specific nand image or do you flash the emmc image to nand? Normally bootrom needs specific header for each bootdevice

Thank you very much! Where did you get the bl2 from? I can’t see it on EDIT: nevermind the preloader is the BL2

https://firmware-selector.immortalwrt.org/?version=23.05.1&target=mediatek/filogic&id=bananapi_bpi-r3-mini

and when I compile the image myself, for NAND I only get those two files:

...snand-bl31-uboot.fip
...snand-preloader.bin

I forgot to mention that the steps above I did were with the stock image and I only deleted the ubi because the wiki only provides the image .bin (Banana Pi BPI-R3 Mini - Banana Pi Wiki)


@frank-w

You boot till uboot here and load kernel+rootfs from usb. Is there a way to flash immortalwrt to ubi-partition?

I think if you do steps 9. and 10. from @nezar_taima post you can then either use luci to sysupgrade or scp the squashfs to /tmp and start the sysupgrade using the cli. Or was your question to do all the steps directly in U-BOOT without ever booting into openwrt/immortalwrt?

basicly it should be all erased what is written afterwards. So if you only erase ubi,the bl2+fip may stay old or cause strange issues like the bootloop

That makes sense! I was wary of deleting the rest because I flashed the official image and they only provide the image .bin. My goal is to have immortalWRT on EMMC and the “official” image on NAND in case something goes wrong on the EMMC.

Do I have to compile the image myself from SinoVoips github in order to get the bl31 and preloader for NAND?

Is there a specific nand image or do you flash the emmc image to nand? Normally bootrom needs specific header for each bootdevice

What I did until I noticed that my NAND image bootloops is:

  1. I compiled immortalWRT.
  2. Followed the steps above from @1715173329 original post to flash the EMMC up until the TFTP part. Booting immortalWRT U-BOOT doesn’t activate my PHY Links. So I had to find another way without TFTP
  3. To get the sysupgrade image onto the EMMC I booted back into NAND and dd the sysupgrade like mentioned in this thread:
dd if=immortalwrt-mediatek-filogic-bananapi_bpi-r3-mini-squashfs-sysupgrade.itb of=/dev/mmcblk0p5 bs=512 conv=fsync

  1. Switched back to EMMC and immortalWRT was working. I only noticed NAND is bootlooping when I wanted to update my newly compiled immortalWRT image to EMMC.

I didn’t touch the NAND since I wanted to keep it stock. That’s why I was a bit confused how this happened.

Sidenote: Right now, when I “repair” the NAND (using the nand image of course) it boots but the EMMC bootloops. When I run the recovery image on EMMC, the EMMC works but the NAND starts bootlooping

I’ve just followed the immortalWRT official instructions by run the following once initramfs has been booted:

umount /mnt/ubi0*
ubidetach -p /dev/mtd4
ubiformat -y /dev/mtd4
ubiattach -p /dev/mtd4

ubimkvol /dev/ubi0 -n 0 -N ubootenv -s 128KiB
ubimkvol /dev/ubi0 -n 1 -N ubootenv2 -s 128KiB
ubimkvol /dev/ubi0 -n 2 -N recovery -s 20MiB
ubimkvol /dev/ubi0 -n 3 -N fit -s 20MiB

ubiupdatevol /dev/ubi0_2 initramfs.itb
ubiupdatevol /dev/ubi0_3 sysupgrade.itb

Ok,was not aware of these additional steps

My plan is to replace official image on nand with the immortalwrt,because the official image has very limited uboot and takes very long to bootup linux (with very much debugs). On emmc i want to boot a debian system like on my other boards (stil struggeling with network-support). I have the phy driver from @ericwoud and dts i’ve prepared with him,phy is detected,but cannot get link on 1g speed

Running debian sounds awesome. Is there a repo I can take a look at?

My plan is to replace official image on nand with the immortalwrt,because the official image has very limited uboot and takes very long to bootup linux (with very much debugs).

I think the steps @nezar_taima described + after booting the initramfs doing the sysupgrade should be the correct way.

not for r3mini yet as i have to get kernel fixed for it first

but for R3 and some other boards: GitHub - frank-w/BPI-Router-Images

basic steps are same, but uboot needs to be changed for r3 mini, as full r3 needs probing for sd/emmc and nand/nor where on r3mini bootconf is fixed (config inside fit image) and uboot needs some change to support network-support (have not yet got phy driver working for the airoha phys - using dts and driver from immortalwrt initial commit - maybe it is fixed afterwards)

your claim that immortalwrt uboot has no network support amaze me. I use the same uboot on EMMC and NAND, and I still can boot a recovery image from tftp. Strange behavior. are u using 1 gig or 2.5 gig nic? if its a 1 gig nic, can you try to test use one of USB 2.5 gbps adapters

I have not said immortalwrt has no network support…my uboot/kernel does not have it and i take patch from immortalwrt:) i guess i miss anything…

this is my uboot tree for r3mini: Commits · frank-w/u-boot · GitHub

I use 1g as link,but yes i can try 2g5 nic on the other side

Oh sorry, I had it wrong. pls ignore my reply

that might be why I can’t get a link in U-BOOT. I’m connecting to a 1gbe nic and nothing happens until i boot into openwrt

wan-port (right one)

BPI-R3M> setenv ipaddr 192.168.90.2
BPI-R3M> ping 192.168.90.1
ethernet@15100000 Waiting for PHY auto negotiation to complete......... TIMEOUT!
ret -110!
ping failed; host 192.168.90.1 is not alive
BPI-R3M>

lan (left port including bootup):

U-Boot 2024.01-bpi-r3mini-00008-g9448f7a5c567-dirty (Jan 17 2024 - 22:57:09 +01)

CPU:   MediaTek MT7986
Model: Bananapi BPi-R3 Mini
DRAM:  2 GiB
Core:  44 devices, 19 uclasses, devicetree: separate
MMC:   mmc@11230000: 0
Loading Environment from MMC... *** Warning - bad CRC, using default environment

In:    serial@11002000
Out:   serial@11002000
Err:   serial@11002000
Net:   PHY = 3a2 - a411
EN8811H PHY ready!
EN8811H Mode 1 !
Tx, Rx Polarity(0xca0f8): 01a01501
MD32 FW Version(0x3b3c) : 23051209
LED initialize OK !
EN8811H initialize OK ! (v1.0.4)

Warning: ethernet@15100000 (eth0) using random MAC address - 22:f9:43:af:15:7c
eth0: ethernet@15100000
Hit any key to stop autoboot:  0 
BPI-       
BPI-R3M> setenv ipaddr 192.168.90.2
BPI-R3M> ping 192.168.90.1         
Using ethernet@15100000 device

ARP Retry count exceeded; starting again
ping failed; host 192.168.90.1 is not alive
BPI-R3M>

link on other side is up (with 2500/fdx) when using lan-port…not when using wan-port, netmask=255.255.255.0, and the other side has also /24 set. so ip configuration is right.