Running OpenWRT latest build (6/10) on a 128GB SD card. I’d like to be able to install software without worrying about overrunning the 100MB root partition. Extroot instructions don’t work and instructions for messing with overlay in order to do it seem to make incorrect assumptions about what paths are already in /overlay (i.e. ignore “upper” and “work.”). I tried resizing the production partition in gparted but it won’t do it because all the filesystems are read as “unknown” even after installing f2fs-tools and squashfs-tools-ng. What is the right way to do this? Thanks.
I think I have narrowed down the issue to this:
[ 11.019758] UBIFS (ubi0:5): Mounting in unauthenticated mode
[ 11.025492] UBIFS (ubi0:5): background thread "ubifs_bgt0_5" started, PID 697
[ 11.517591] UBIFS (ubi0:5): UBIFS: mounted UBI device 0, volume 5, name "rootfs_data"
[ 11.525414] UBIFS (ubi0:5): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[ 11.535306] UBIFS (ubi0:5): FS size: 91168768 bytes (86 MiB, 718 LEBs), max 729 LEBs, journal size 4571136 bytes (4 MiB, 36 LEBs)
[ 11.546930] UBIFS (ubi0:5): reserved for root: 4306125 bytes (4205 KiB)
[ 11.553526] UBIFS (ubi0:5): media format: w5/r0 (latest is w5/r0), UUID 864E1E15-9008-4117-9DE5-7AE7C7182ED7, small LPT model
[ 11.572347] block: attempting to load /tmp/ubifs_cfg/upper/etc/config/fstab
[ 11.595934] block: unable to load configuration (fstab: Entry not found)
[ 11.602667] block: attempting to load /tmp/ubifs_cfg/etc/config/fstab
[ 11.609124] block: unable to load configuration (fstab: Entry not found)
[ 11.615836] block: attempting to load /etc/config/fstab
[ 11.621105] block: unable to load configuration (fstab: Entry not found)
[ 11.627805] block: no usable configuration
But I have no idea how to fix it.
This is an ongoing problem, and an argument I have had with the OpenWrt devs. Unfortunately they have not seen fit to change the default partition size to something reasonable. This should be part of a pinned FAQ here because every new user runs into this issue.
You can’t use parted or gparted to do the resizing, because they change other things in the partition table and it won’t boot right. You have to use plain fdisk and do it manually.
Resizing the partition is tricky - it’s doable, and I’ll explain how, but it’s tricky. The better solution, if you have the skill, is to build your own image. You can select the size of the root partition when you build it yourself. Once you create your custom config file with the new partition size, you can save it and re-use it on subsequeny builds.
To do the actual resize, you’ll need either:
- A second sdcard, along with a USB sdcard reader
- To set up your device to boot from NAND flash
This is because you need to use the device itself to resize the partition, and the partition can’t be active when you do. So you need to boot off NAND, or boot off one sdcard while you resize the other. If you have the equipment, a second sdcard is easier. The instructions are going to assume that - adjust as required if you’re going the NAND route:
- Boot your R3 using a temp sd card
- Install the needed packages
opkg install f2fs-tools f2fsck mkf2fs fdisk kmod-usb3 kmod-usb-storage kmod-usb-storage-uas parted
We’re not actually using parted, but we need it because it haspartprobewhich we will use. - Plug the sdcard you are going to use into your reader and connect it to the R3’s USB port. I’m going to assume it’s /dev/sda - adjust as required below if it’s not.
- Copy the sdcard OpenWrt image to the sdcard
- Reload the card’s partition table
partprobe /dev/sda fdisk /dev/sda- Print the partition table with the “p” command. Then switch to extended mode “x” and print the partition table in extended mode “p” before returning to normal “r”
- Copy and paste both partition table printouts to a text file.
- Delete partition 7 “d”
- Create a new partition 7 “n”, start it on the same sector as the old one (likely 131072), but have the end point further down. I use a 2GiB sized partition for mine, so if 131072 was your start the end would then be 4325375
- Set the type of partition 7 to what it was - use the “t” command, and set it to the type UUID it was before, you’ll see it in the expert printout you made - likely it’s “CAE9BE83-B15F-49CC-863F-081B744A2D93”
- Switch to expert mode “x”
- Set the UUID of partition 7 to what it was bedore “u” - likely “5452574F-2211-4433-5566-778899AABB07”
- Set the name of partition 7 to what it was before “n” - this will be “production”
- Print out the partition “p” in while still in expert mode and check to make sure it matches the old partition 7 in every way except the ending sector
- Return to normal mode “r”, then write the partition table “w”
- At this point I usually
syncand remove the USB sdcard reader and then plug it back in because sometimes when the partition table is soft-read back after fdisk writes it the FIT table doesn’t read back correctly. - Now you need to resize the virtual overlay partition, which should be
/dev/sda66:
resizef2fs /dev/sda66 - Sometimes the above doesn’t work unless you first mount and dismount the overlay:
mkdir /mnt/tmp
mount /dev/sda66 /mnt/tmp -t f2fs
umount /mnt/tmp -
sync, remove the sdcard reader, halt the device, power down, and swap cards. Hopefully it is now large enough to be useful.
EDIT: I haven’t actually done this in some time, so this was from memory, so apologies in advance if I’ve missed anything
1 Like
Agreed on all points. I switched from OpenWRT to real Linux in large part because of the OpenWRT team’s insistence on assuming every device has less storage than my first PC 30 years ago. You will be better off taking the wifi configs generated by OpenWRT (they’re at /tmp/run/hostapd-*.conf) and installing hostapd on a real Linux distribution. It may seem daunting to set everything up yourself, but I promise it is not that hard, and there are so. many. benefits. for doing so, far beyond being able to use all the space on that big SD card.
for me the main draw of OpenWRT is LuCi. installing the Ubuntu image is tempting but I think not having the ease of management would get frustrating after a while.
I personally don’t go this far - I have found ways to work around OpenWrt’s self-imposed limitations OpenWrt puts in. OpenWrt’s other benefits outweigh even dumb decisions like this. Plus if users like us leave, they loose voices for change. But that being said, I don’t blame people for switching.
I realized I posted about resizing the partition without actually speaking to the title of the thread. You want to know how best to use the entire card.
You are using a 128GB card - I use 64GB cards myself, but the process I use will work for you too. In my post I suggested a 2GB OpenWrt overlay partition. I suggest an OpenWrt overlay partition this size because 1) it’s enough for any number of packages, and 2) so you can use the bulk of the space on the card for a persistent partition that stays the same even after you reimage with new OpenWrt releases. In general terms, here is my card’s layout:
- Space for the OpenWrt image
- Buffer space in case #1 changes in size
- Persistent “workbench” partition
- Swap space
- More buffer space in case I want to move to a different manufacturer’s SD card (because the exact size of two SD cards with the same capacity is often different)
Here is the actual partition layout of my SD card:
Disk /dev/mmcblk0: 58.27 GiB, 62562238464 bytes, 122191872 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5452574F-2211-4433-5566-778899AABB00
Device Start End Sectors Size Type
/dev/mmcblk0p1 34 8191 8158 4M Linux filesystem
/dev/mmcblk0p2 8192 9215 1024 512K Linux filesystem
/dev/mmcblk0p3 9216 13311 4096 2M Linux filesystem
/dev/mmcblk0p4 13312 21503 8192 4M EFI System
/dev/mmcblk0p5 24576 90111 65536 32M EFI System
/dev/mmcblk0p6 90112 131071 40960 20M EFI System
/dev/mmcblk0p7 131072 4325375 4194304 2G unknown
/dev/mmcblk0p8 4325376 8388607 4063232 1.9G Linux reserved
/dev/mmcblk0p9 8388608 109051903 100663296 48G Linux filesystem
/dev/mmcblk0p10 109051904 117440511 8388608 4G Linux swap
/dev/mmcblk0p11 117440512 122189823 4749312 2.3G Linux reserved
Here is the same device shown in fdisk’s expert mode:
Disk /dev/mmcblk0: 58.27 GiB, 62562238464 bytes, 122191872 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5452574F-2211-4433-5566-778899AABB00
First usable LBA: 34
Last usable LBA: 122191838
Alternative LBA: 122191871
Partition entries starting LBA: 2
Allocated partition entries: 128
Partition entries ending LBA: 33
Device Start End Sectors Type-UUID UUID Name Attrs
/dev/mmcblk0p1 34 8191 8158 0FC63DAF-8483-4772-8E79-3D69D8477DE4 5452574F-2211-4433-5566-778899AABB01 bl2 RequiredPartition LegacyBIOSBootable
/dev/mmcblk0p2 8192 9215 1024 0FC63DAF-8483-4772-8E79-3D69D8477DE4 5452574F-2211-4433-5566-778899AABB02 ubootenv RequiredPartition
/dev/mmcblk0p3 9216 13311 4096 0FC63DAF-8483-4772-8E79-3D69D8477DE4 5452574F-2211-4433-5566-778899AABB03 factory RequiredPartition
/dev/mmcblk0p4 13312 21503 8192 C12A7328-F81F-11D2-BA4B-00A0C93EC93B 5452574F-2211-4433-5566-778899AABB04 fip RequiredPartition
/dev/mmcblk0p5 24576 90111 65536 C12A7328-F81F-11D2-BA4B-00A0C93EC93B 5452574F-2211-4433-5566-778899AABB05 recovery RequiredPartition
/dev/mmcblk0p6 90112 131071 40960 C12A7328-F81F-11D2-BA4B-00A0C93EC93B 5452574F-2211-4433-5566-778899AABB06 install RequiredPartition
/dev/mmcblk0p7 131072 4325375 4194304 CAE9BE83-B15F-49CC-863F-081B744A2D93 5452574F-2211-4433-5566-778899AABB07 production
/dev/mmcblk0p8 4325376 8388607 4063232 8DA63339-0007-60C0-C436-083AC8230908 5452574F-2211-4433-5566-778899AABB08 dmz
/dev/mmcblk0p9 8388608 109051903 100663296 0FC63DAF-8483-4772-8E79-3D69D8477DE4 5452574F-2211-4433-5566-778899AABB09 workbench
/dev/mmcblk0p10 109051904 117440511 8388608 0657FD6D-A4AB-43C4-84E5-0933C84B4F4F 5452574F-2211-4433-5566-778899AABB0A sdswap
/dev/mmcblk0p11 117440512 122189823 4749312 8DA63339-0007-60C0-C436-083AC8230908 5452574F-2211-4433-5566-778899AABB0B dmz
Partitions 1-7 come from the OpenWrt sdcard image. Partitions 8-11 I added myself. 8 is the buffer betweeen the OpenWrt partitions and my persistent workbench partition (I labelled it “dmz”), 9 is the workbench itself, 10 is swap space, and 11 is the end buffer.
This way when a new OpenWrt image comes out, I can copy it directly over my existing card, and know that even if it grows a little it won’t overwrite my persistent partition. This will hammer the partition table, but that’s ok, since I know where to rebuild it (I have a setup file with the partition layouts). So I just recreate partition 8 to start wherever partition 7 ends and then 8 will always end in the same spot. I can then remake partition 9 to start and end in exactly the same spot, and voila, all my persistent data is back again.
In my configuration process for a new build, I replace /root with a softlink to /mnt/workbench/root, which gives me a persistent home directory. Once I install the same packages I had before, recreate that /root link, and restore the configuration, in 5 minutes I have a new build/release that is functionally identical.
I think in the original version of your post you linked to image build instructions - do you still have those somewhere?
I didn’t, but here they are:
- The official instructions are here: https://openwrt.org/docs/guide-developer/toolchain/use-buildsystem
- This is also very useful information: https://hamy.io/post/0015/how-to-compile-openwrt-and-still-use-the-official-repository/
- If you’re brave some day and want to build OpenWrt right on your R3, here’s a howto I wrote: https://openwrt.org/docs/guide-developer/toolchain/building_openwrt_on_openwrt
Here are the actual steps I use to build master into a snapshot:
git clone https://git.openwrt.org/openwrt/openwrt.git
cd openwrt
git pull
git checkout master
./scripts/feeds update -a
./scripts/feeds install -a
#wget https://downloads.openwrt.org/snapshots/targets/mediatek/filogic/config.buildinfo -O .config
#cp -a ~/devel/wrt/diffconfig_mt7986_alluserspace_v3 .config
cp -a ~/devel/wrt/diffconfig_mt7986 .config
wget https://va1der.ca/~public/openwrt/llvm/llvm-bpf-15.0.7.Linux-x86_64.tar.xz
tar -xaf llvm-bpf-15.0.7.Linux-x86_64.tar.xz
make defconfig
# To test the kernel fingerprint do the following (the first one will fail because the toolchain isn't built yet, this is expected)
#make target/linux/compile
#find build_dir/ -name .vermagic -exec cat {} \;
make -j $(nproc) toolchain/install
make -j $(nproc) download
# mt76 eeprom.c patch to allow driver to load the WiFi eeprom from a file - totally optional but useful
mkdir package/kernel/mt76/patches
wget https://va1der.ca/~public/openwrt/patches/mt76/999-mt76_eeprom_file_220302.diff -O package/kernel/mt76/patches/999-mt76_eeprom_file_220302.diff
make -j $(nproc) world
Here are the configuration files referenced above that I use for the R3:
- diffconfig_mt7986 (2.7 KB)
- diffconfig_mt7986_alluserspace_v3 (2.7 KB)
The first one builds just a standard image. It is based on the stock snapshot configuration with the following changes:
- Adds mkfs.f2fs, f2fs, ntfs3, and ntfs3.mount to the image
- Changes LLVM setting to use the pre-supplied one rather than building it from source, saves about half the compile time
- Makes the default wpad one that uses WolfSSL and Wolf’s hardware encryption acceleration library
- Changes stripping to one that is friendly to libraries
- Adds LuCI
- Changes root partition size to 2048MiB
The second one is basically just the first one but also set to build ALL userspace packages (in case you want to self host all your packages) with a couple esoteric packages that cause problems deselected. This will take a long time to build, and there are often one or two packages with build problems at any particular time.
2 Likes
this rules, thank you. I really appreciate your going to the effort of documenting this so thoroughly. I hope someone at BPi is paying attention, because all this should be added to the wiki!
OpenWRT’s parted worked for me. Below are my steps. It’s a shame that p65 and p66 are only accessible from inside OpenWRT. The devs should make them regular partitions.
Thanks to VA1DER for all the important bits! I agree that his work should be on the OpenWRT wiki or at least link to these threads.
parted -s /dev/mmcblk0 resizepart 7 100%
# reboot into NAND (up, down, up, up)
# necessary if you get this error:
# [f2fs_do_mount:3651] Mount unclean image to replay log first
mkdir -p /mnt/tmp && mount /dev/mmcblk0p66 /mnt/tmp -t f2fs && umount /mnt/tmp
resize.f2fs /dev/mmcblk0p66
# reboot into SD card (up, up, up, up)
1 Like
Hi,
I am new to OpenWRT and I found this topic very useful. So I decided to share my experimental setup for automation of the above-described steps. It should support both SDCard and EMMC repartitioning. I don’t know if this approach is correct, but after a couple of re-flashes of my BananaPi Bpi-R3, it seems to be consistent and working.
-
I’ve based my work on branch/tag v23.05.0. I do manual build of OpenWRT, using diffconfig_mt7986. I use 8GB SDCard, to be close to the onboard 8GB EMMC.
-
Add CONFIG_PACKAGE_block-mount=y to diffconfig_mt7986
-
Open file /target/linux/mediatek/image/filogic.mk and find “define Build/mt798x-gpt”. This function seems to be responsible for the partitioning of the storage (both SDCard and EMMC). Update it in the following manner:
ptgen -g -o [email protected] -a 1 -l 1024 \
$(if $(findstring sdmmc,$1), \
-N install -r -p 20M@44M \
-t 0x2e -N production -p $(CONFIG_TARGET_ROOTFS_PARTSIZE)M@64M \
+ -t 0x2e -N expand_production -p 256M@2120M \
+ -t 0x83 -N data -p 4096M@2377M \
+ -t 0x82 -N swap -p 512M@6474M \
+ -t 0x2e -N rest -p 384M@6987M \
) \
$(if $(findstring emmc,$1), \
-t 0x2e -N production -p $(CONFIG_TARGET_ROOTFS_PARTSIZE)M@64M \
+ -t 0x2e -N expand_production -p 256M@2120M \
+ -t 0x83 -N data -p 4096M@2377M \
+ -t 0x82 -N swap -p 512M@6474M \
+ -t 0x2e -N rest -p 384M@6987M \
)
cat [email protected] >> $@
rm [email protected]
The parameters of the “ptgen” tool are the key point here:
/build_dir/host/firmware-utils-2023-06-29-9e2de851/ptgen -h Usage: ./host/firmware-utils-2023-06-29-9e2de851/ptgen [-v] [-n] [-g] -h -s -o [-a 0…4] [-l ] [-G ] [[-t | -T ] [-r] [-N ] -p [@]…]
Note: the values given (custom partitions, partition types and sizes) are purely experimental and should be adapted according to your needs. I guess, with some decent coding, one can implement automatic partitioning.
- Create uci-default file in /target/linux/mediatek/filogic/base-files/etc/uci-defaults/. This is my file: 50_auto-gen-fstab (3.1 KB). Here I create the filesystems for the partitions and mount them. Then create equivalent “uci” description. The file will be executed only once, after re-flashing, and will be deleted (this is the expected behavior of ‘uci-defaults’). From then on “uci” is responsible for the mounting.
Note: files 50_auto-gen-fstab and filogic.mk go hand by hand. They use same names of the partitions. Also 50_auto-gen-fstab uses hardcoded /dev/mmcblk* strings, which most-likely should be updated per device.
I hope someone finds this interesting. Any comments are more then welcome.
Greetings.
1 Like