[BPI-R3] Change or add partion to /overlay

[dangowrt (Daniel Golle)]

LUCI only supports backups of /etc/config and a few other files in /etc. Having a larger rootfs has no impact on the amount of data someone stores in /etc/config. For the vast majority of users there would be no impact to they way they do business having a large rootfs.

Preservation is configurable and many package automatically add files to be preserved, it’s not just /etc/config…

For the others, it’s not an unreasonable expectation that those who are performing manual sysupgrades who want to preserve everything that they have other means, as I do.

What do you mean by “manual”? Any kind of sysupgrade (which is also the only way to update the Linux kernel and modules) always wipes rootfs_data. Expecting that users will backup their data (or whatever it is they want to store there) onto some external media before performing sysupgrade is more complicated than asking them to installing uvol and autopart which allows to easily manage additional filesystem volumes, e.g. for container images or any kind of data, which will survive sysupgrade. Also note that many application hungry for disk-space don’t like running on top of overlayfs, e.g. Docker won’t work in that way.

Not everyone, as this thread shows, is technically adept enough to work out how to do this nor should they have to be . How many computers come with a terrabyte hard drive, and then only provide the end user with small fraction of that for C:? And then just expect people that if they want more will mount it in, or expand C:?

Well, OpenWrt is not a general purpose OS like, let’s say Debian/GNU Linux or Microsoft Windows. It’s a firmware meant for special purpose embedded devices and has a quite different idea about e.g. software lifecycle. Inviting users to fill up the rootfs overlay is not a good idea, as that will either make them loose everything every time they update the firmware or they just won’t update. Both is a bad result.

For end-users who need space for additional data, e.g. to be served via HTTP or adguard lists or whatever, I’d recommend:

opkg update
opkg install uvol autopart
uvol create userdata $(uvol free) rw

This will provide a read-write filesystem volume using all remaining space which is mounted to /tmp/run/uvol/userdata by default. The mountpoint can be adjusted, if needed.

[Haldi]

So what would be the best option?

Increasing the 104MB mmcblk0p5 is not possible. Adding another 100-500mb partition for packages.

Then adding the rest via uvol for Userdata from docker and other stuff?

[Haldi]

how exactly did you manage to get the mounted partition to show up in software? Did pretty much the same as you. Can see the “mounted” partition in the faile systems menu. But Software is still not any bigger

[Boysa22]

Hi Haldi,

Increasing mmcblk0p5 is posible. I actually booted from NAND and then used the “Install to EMMC option”. This creates the partitions on EMMC. Then I used parted to resize mmcblk05 before booting from EMMC. Then set the switches to boot from MMC. Check the post dragowrt here:

Cannot resize my root partition on BPI R3

On the R3 the /dev/mmcblk0p66 device is automatically mapped as the remaining space on /dev/mmcblk0p5 after parsing the uImage.FIT for kernel, device tree and read-only squashfs rootfs (squashfs filesystem is mapped as /dev/mmcblk0p65). So those p65 and p66 partitions are not actual GPT partitions but rather p65 is the filesystem sub-image of the uImage.FIT stored in p5 and p66 is the remaining space on p5 after the uImage.FIT. Hence you only need to edit the size of /dev/mmcblk0p5, then reboot…

[Haldi]

does that not cause issues with Updates?

[dangowrt (Daniel Golle)]

It will not prevent you from updating, just be aware that any data stored on the rootfs will be wiped, apart from configuration files and what ever else you put into /etc/sysupgrade.conf (and all that together needs to fit into /tmp which is tmpfs, ie. RAM-backed during the upgrade process, so cannot be very huge).

Hence, for any kind of persistent data other than software packages and their configuration, it is advisable to resort to additional storage volumes instead of using the root filesystem. Esp. in case you are planning to use Docker or podman, be aware that these do not work on overlayfs, and OpenWrt’s rootfs is overlayfs – so for those space-hungry things like container images you will anyway always need an additional storage volume as the rootfs cannot be used for that.

[Haldi]

Okay… So increasing mmcblk0p5 should work and persist in updates? Only when you install a new image from MicroSD would that be overwritten.

Currently Tempfs is around 1gb (as shown in Mounts) but we do have 2gb RAM. So isn’t that too small? Or does the router need more ram during the update and 50% of RAM is the optimal size?

Going by that we could increase the size of mmcblk0p5 untill all together reaches 1gb aka Tempfs size. I’ll do the math when I’m back home… And try it out.

Then the rest of the 8gb eMMC can be mounted separately.

[dangowrt (Daniel Golle)]

So increasing mmcblk0p5 should work and persist in updates?

The size itself will persist, ie. you will not have to make the change of partition size again after upgrade. But the content will be wiped, unless you register files to be kept across updates in /etc/sysupgrade.conf – and there you are limited to the size of tmpfs in RAM.

Only when you install a new image from MicroSD would that be overwritten.

Exactly. The partition table is created only when installing from eMMC.

Going by that we could increase the size of mmcblk0p5 untill all together reaches 1gb aka Tempfs size. I’ll do the math when I’m back home… And try it out.

That can work, but affect both, the partition table on the microSD and eMMC. However, most microSD cards as of today are also 1GiB or larger. When adding support for eMMC on MT7622 and MT7986 targets I’ve basically just copied the existing default in OpenWrt, but maybe this should be re-considered and instead of 104MiB default we can increase it globally to 768MiB (still leaving enough space for kernel and bootloader on a 1GiB medium).

[FerrumMaster (Zigmars Henzels)]

Well it would be great that the partition would be at least 256MB, we have a really powerful device here.

Python3 takes a lot of space, and I would like to run DIYHUE emulator on it for example.

[Haldi]

the 104MiB default come from the NAND version? As there is only 128mb Nand and 32mb NOR. or is that separate from the emmC image/config?

I think 768MiB would help a lot as current Snapshot builds only have around 30mb Free space. Especially since most OPKG packages tell WAY to low values for Storage they need.

[sulaco]

I haven’t seen any mention of it, but is it possible to boot from m.2/NVME or can it only be used for additional storage?

[frank-w]

You cannot directly boot from it,but you can have rootfs from it. Uboot does not yet support pcie controller. If this is supported,kernel can be loaded from there too

[lu_zero (Luca Barbato)]

Is the nvme support in uboot working better now? I saw https://u-boot.readthedocs.io/en/latest/develop/driver-model/nvme.html

[frank-w]

Uboot still misses pcie driver for mt798x as i’ve wrote above

[yohru]

Extend software free space, about 100MB to eMMC 8GB.

Completed environment:
・Boot from SPI-Nand
・eMMC 8GB or nvme for “/overlay” area.
・If expand using this method, there is no problem even if restart the BPI-R3.

Extroot configuration => https://openwrt.org/docs/guide-user/additional-software/extroot_configuration

Note: Change the target device, in to the eMMC.

It now allocate eMMC(8GB) to the “/overlay” area.
DEVICE="/dev/mmcblk0"

Can also allocate nvme instead of eMMC.
DEVICE="/dev/nvme0n1"

[C_Soellinger]

This sounds very interesting!

Do you made any performance tests? I’m interested if this make a difference than using eMMC as main storage and extending it’s partition

cheers

[yohru]

There is a benchmark for nvme with “hdparm”, but no benchmark for SPI-Nand.
nvme benchmark [KP230 Gen3.0 × 4]

---

root@BPI-R3:~# hdparm -t --direct /dev/nvme0n1

/dev/nvme0n1: Timing O_DIRECT disk reads: 1908 MB in 3.00 seconds = 635.47 MB/sec

---

Can get a benchmark by booting with an SD card and running “hdparm” on SPI-Nand, but since are already using to, it may be a little impossible.

But, I’ve used it with both SPI-Nand and eMMC and can’t feel any difference.
I don’t think there will be any problems in practice.
However, SPI-Nand is probably slower than eMMC in terms of Bus speed specifications.

[buggz]

Hello,

Does this work for BPI-R3 and an external USB drive?

Extroot configuration

Doesn’t seem to work for me… I currently boot from eMMC. Wanting to expand /overlay for more packages, etc.

[C_Soellinger]

With this method described by yohru you have to boot from SPI-Nand and then follow the extroot configuration but with eMMC as device.

@yohru thanks for informations, i also think in real world the performance difference is not necessary. i think i’ll try that too

[yohru]

I have only tested it with eMMC and mvne, but it should work.

Is a ext4 formatted USB memory are visible with “lsusb -t”?
Is it recognized by “block info” and “blcok detect”?
Are the following packages installed?

---

Required

opkg install kmod-usb-storage
opkg install kmod-usb-storage-uas
opkg install kmod-fs-ext4
opkg install usbutils
opkg install block-mount
opkg install e2fsprogs

It might be convenient

opkg install fdisk cfdisk gdisk

---

Reference => https://openwrt.org/docs/guide-user/storage/usb-drives