Extroot configuration on BPI-R3 using nvme

Banana Pi Router design BPI-R3/R3 Mini(MT7986)
1

Is it possible? I have been trying following this extroot configuration but obviously changing /dev/sda to /dev/nvme0n1, I’m struggling to get the desired result.

block info; uci show fstab; logread | sed -n -e “/- preinit -/,/- init -/p”

/dev/ubiblock0_2p1: UUID="5d683641-46dff0f5-0f062a7d-fb985e0f" VERSION="4.0" TYPE="squashfs"
/dev/ubi0_5: UUID="fbfe7fef-9e67-458d-8c11-aad4a82c3deb" VERSION="w5r0" TYPE="ubifs"
/dev/mmcblk0p65: UUID="75735bd9-90f51e04-7a481520-54d3e852" VERSION="4.0" MOUNT="/rom" TYPE="squashfs"
/dev/mmcblk0p66: UUID="77ed7ce2-89d4-4be4-961a-5bf9c6f8139a" LABEL="rootfs_data" VERSION="1.16" MOUNT="/overlay" TYPE="f2fs"
/dev/nvme0n1: UUID="6bdb3bd7-2788-4614-b389-a4378f63cfc3" LABEL="extroot" VERSION="1.0" TYPE="ext4"
fstab.@global[0]=global
fstab.@global[0].anon_swap='0'
fstab.@global[0].anon_mount='0'
fstab.@global[0].auto_swap='1'
fstab.@global[0].auto_mount='1'
fstab.@global[0].delay_root='5'
fstab.@global[0].check_fs='0'
fstab.@mount[0]=mount
fstab.@mount[0].target='/rom'
fstab.@mount[0].uuid='5d683641-46dff0f5-0f062a7d-fb985e0f'
fstab.@mount[0].enabled='0'
fstab.@mount[1]=mount
fstab.@mount[1].target='/overlay'
fstab.@mount[1].uuid='1a0e17ae-1dd2-11b2-994b-4ef40b1c07bc'
fstab.@mount[1].enabled='0'
fstab.rwm=mount
fstab.rwm.device='/dev/mmcblk0p66'
fstab.rwm.target='/rwm'
fstab.rwm.enabled='0'
fstab.overlay=mount
fstab.overlay.uuid='6bdb3bd7-2788-4614-b389-a4378f63cfc3'
fstab.overlay.target='/overlay'
fstab.overlay.enabled='1'
Sat May 27 08:04:49 2023 user.info kernel: [    6.749146] init: - preinit -
Sat May 27 08:04:49 2023 kern.info kernel: [    6.920539] mtk_soc_eth 15100000.ethernet eth0: configuring for fixed/2500base-x link mode
Sat May 27 08:04:49 2023 kern.info kernel: [    6.928981] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 2.5Gbps/Full - flow control rx/tx
Sat May 27 08:04:49 2023 kern.info kernel: [    6.935658] mt7530-mdio mdio-bus:1f lan1: configuring for phy/gmii link mode
Sat May 27 08:04:49 2023 kern.info kernel: [    6.949350] mt7530-mdio mdio-bus:1f lan1: Link is Up - 1Gbps/Full - flow control rx/tx
Sat May 27 08:04:49 2023 kern.notice kernel: [    6.950206] random: procd: uninitialized urandom read (4 bytes read)
Sat May 27 08:04:49 2023 kern.info kernel: [    6.957347] IPv6: ADDRCONF(NETDEV_CHANGE): lan1: link becomes ready
Sat May 27 08:04:49 2023 kern.info kernel: [    7.170296] usb 1-1: new high-speed USB device number 2 using xhci-mtk
Sat May 27 08:04:49 2023 kern.info kernel: [    7.350940] hub 1-1:1.0: USB hub found
Sat May 27 08:04:49 2023 kern.info kernel: [    7.354754] hub 1-1:1.0: 4 ports detected
Sat May 27 08:04:49 2023 user.info kernel: [    8.990646] mount_root: loading kmods from internal overlay
Sat May 27 08:04:49 2023 user.info kernel: [    9.000042] kmodloader: loading kernel modules from //etc/modules-boot.d/*
Sat May 27 08:04:49 2023 user.info kernel: [    9.007369] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
Sat May 27 08:04:49 2023 kern.notice kernel: [    9.780304] random: crng init done
Sat May 27 08:04:49 2023 kern.notice kernel: [   10.383475] UBIFS (ubi0:5): Mounting in unauthenticated mode
Sat May 27 08:04:49 2023 kern.notice kernel: [   10.389200] UBIFS (ubi0:5): background thread "ubifs_bgt0_5" started, PID 686
Sat May 27 08:04:49 2023 kern.notice kernel: [   10.886868] UBIFS (ubi0:5): UBIFS: mounted UBI device 0, volume 5, name "rootfs_data"
Sat May 27 08:04:49 2023 kern.notice kernel: [   10.894694] UBIFS (ubi0:5): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
Sat May 27 08:04:49 2023 kern.notice kernel: [   10.904593] UBIFS (ubi0:5): FS size: 90914816 bytes (86 MiB, 716 LEBs), max 727 LEBs, journal size 4571136 bytes (4 MiB, 36 LEBs)
Sat May 27 08:04:49 2023 kern.notice kernel: [   10.916220] UBIFS (ubi0:5): reserved for root: 4294130 bytes (4193 KiB)
Sat May 27 08:04:49 2023 kern.notice kernel: [   10.922817] UBIFS (ubi0:5): media format: w5/r0 (latest is w5/r0), UUID FBFE7FEF-9E67-458D-8C11-AAD4A82C3DEB, small LPT model
Sat May 27 08:04:49 2023 user.info kernel: [   10.941673] block: attempting to load /tmp/ubifs_cfg/upper/etc/config/fstab
Sat May 27 08:04:49 2023 user.err kernel: [   10.967094] block: unable to load configuration (fstab: Entry not found)
Sat May 27 08:04:49 2023 user.info kernel: [   10.973826] block: attempting to load /tmp/ubifs_cfg/etc/config/fstab
Sat May 27 08:04:49 2023 user.err kernel: [   10.980288] block: unable to load configuration (fstab: Entry not found)
Sat May 27 08:04:49 2023 user.info kernel: [   10.986993] block: attempting to load /etc/config/fstab
Sat May 27 08:04:49 2023 user.err kernel: [   10.993596] block: unable to load configuration (fstab: Entry not found)
Sat May 27 08:04:49 2023 user.err kernel: [   11.000316] block: no usable configuration
Sat May 27 08:04:49 2023 kern.notice kernel: [   11.015204] UBIFS (ubi0:5): un-mount UBI device 0
Sat May 27 08:04:49 2023 kern.notice kernel: [   11.019905] UBIFS (ubi0:5): background thread "ubifs_bgt0_5" stops
Sat May 27 08:04:49 2023 kern.notice kernel: [   11.074119] F2FS-fs (mmcblk0p66): Disable nat_bits due to incorrect cp_ver (9304940510518559981, 2219382037156578531)
Sat May 27 08:04:49 2023 kern.notice kernel: [   11.087358] F2FS-fs (mmcblk0p66): Mounted with checkpoint version = 3999c4ed
Sat May 27 08:04:49 2023 user.info kernel: [   11.095718] mount_root: loading kmods from internal overlay
Sat May 27 08:04:49 2023 user.info kernel: [   11.107486] kmodloader: loading kernel modules from /tmp/overlay/upper/etc/modules-boot.d/*
Sat May 27 08:04:49 2023 user.info kernel: [   11.117662] kmodloader: done loading kernel modules from /tmp/overlay/upper/etc/modules-boot.d/*
Sat May 27 08:04:49 2023 kern.notice kernel: [   12.450316] UBIFS (ubi0:5): Mounting in unauthenticated mode
Sat May 27 08:04:49 2023 kern.notice kernel: [   12.456036] UBIFS (ubi0:5): background thread "ubifs_bgt0_5" started, PID 695
Sat May 27 08:04:49 2023 kern.notice kernel: [   12.953500] UBIFS (ubi0:5): UBIFS: mounted UBI device 0, volume 5, name "rootfs_data"
Sat May 27 08:04:49 2023 kern.notice kernel: [   12.961326] UBIFS (ubi0:5): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
Sat May 27 08:04:49 2023 kern.notice kernel: [   12.971217] UBIFS (ubi0:5): FS size: 90914816 bytes (86 MiB, 716 LEBs), max 727 LEBs, journal size 4571136 bytes (4 MiB, 36 LEBs)
Sat May 27 08:04:49 2023 kern.notice kernel: [   12.982841] UBIFS (ubi0:5): reserved for root: 4294130 bytes (4193 KiB)
Sat May 27 08:04:49 2023 kern.notice kernel: [   12.989433] UBIFS (ubi0:5): media format: w5/r0 (latest is w5/r0), UUID FBFE7FEF-9E67-458D-8C11-AAD4A82C3DEB, small LPT model
Sat May 27 08:04:49 2023 user.info kernel: [   13.008280] block: attempting to load /tmp/ubifs_cfg/upper/etc/config/fstab
Sat May 27 08:04:49 2023 user.err kernel: [   13.033769] block: unable to load configuration (fstab: Entry not found)
Sat May 27 08:04:49 2023 user.info kernel: [   13.040496] block: attempting to load /tmp/ubifs_cfg/etc/config/fstab
Sat May 27 08:04:49 2023 user.err kernel: [   13.046954] block: unable to load configuration (fstab: Entry not found)
Sat May 27 08:04:49 2023 user.info kernel: [   13.053664] block: attempting to load /etc/config/fstab
Sat May 27 08:04:49 2023 user.err kernel: [   13.058902] block: unable to load configuration (fstab: Entry not found)
Sat May 27 08:04:49 2023 user.err kernel: [   13.065607] block: no usable configuration
Sat May 27 08:04:49 2023 kern.notice kernel: [   13.080444] UBIFS (ubi0:5): un-mount UBI device 0
Sat May 27 08:04:49 2023 kern.notice kernel: [   13.085147] UBIFS (ubi0:5): background thread "ubifs_bgt0_5" stops
Sat May 27 08:04:49 2023 user.info kernel: [   13.097263] mount_root: switching to f2fs overlay
Sat May 27 08:04:49 2023 kern.warn kernel: [   13.103615] overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.
Sat May 27 08:04:49 2023 user.debug kernel: [   13.115392] urandom-seed: Seeding with /etc/urandom.seed
Sat May 27 08:04:49 2023 kern.info kernel: [   13.136413] mt7530-mdio mdio-bus:1f lan1: Link is Down
Sat May 27 08:04:49 2023 user.info kernel: [   13.145147] procd: - early -
Sat May 27 08:04:49 2023 user.info kernel: [   13.148068] procd: - watchdog -
Sat May 27 08:04:49 2023 user.info kernel: [   13.673615] procd: - watchdog -
Sat May 27 08:04:49 2023 user.info kernel: [   13.677795] procd: - ubus -
Sat May 27 08:04:49 2023 user.info kernel: [   13.732598] procd: - init -
2

It’s been a while since I’ve used extroot - primarily because BPI devices don’t really need it. TBH, I don’t think OpenWrt is looking at its fstab for your rootfs_data - I think it’s the device tree.

When it switches to f2fs overlay, what device ends up being your rootfs and overlay?

3

Hi. Did you manage to find any way to increase the overlay with a nvme disk? I’m in a similar situation. I’m running OpenWrt in a BPI R3 with one docker container, and I can see how the emmc storage is becaming smaller and in a short future I’ll need to increase the storage. I’ve made a test with a BPI M2 + and I get the same resaults than you.