It took me some time to figure this out, so I post it here that it might help somebody. This is most probably not the most elegant way to do it, but it works for me.
(After I had a working system, I installed frank-w’s Linux repo on the BPI-R4 itself and compiled the kernel there. As frank-w himself points out, there is no …/build directory when crosscompiling. So if you do crosscompile, you have to figure out where to get the necessary files I reference in the next section yourself.)
Compile the kernel with all the modules you want. You can convert most of the compiled-in configuration options to modules. Copy bpi-r4.dtb, bpi-r4.itb and uImage_nodt to /boot as well as the modules to /lib/modules as usual.
Now the things that need to be done for the initramfs (I am assuming a 6.8.0-rc3-next-20240207-bpi-r4-dango kernel version and a BTRFS root fs on /dev/md0. Change if necessary):
(The last command will overwrite your /boot/uEnv.txt of course. So be sure to adapt it to you needs or just use an editor to make the necessary changes by hand.)
(Sidenote: If you want to use BTRFS the kernel option CONFIG_XARRAY_MULTI has to be selected which you cannot select manually and which is not done automatically when enabling BTRFS support. I guess this is a kernel bug. I enabled Memory Management options->Transparent Hugepage Support (CONFIG_TRANSPARENT_HUGEPAGE) which auto-selects CONFIG_XARRAY_MULTI. There are a couple of different options which select the option. Just search for XARRAY_MULTI in the Kernel configuration menu and it will list them. If you don’t do this you get a kernel panic when mounting a BTRFS fs.)
You are absolutely right. Sorry for the confusion. I changed the post above to not confuse anyone else.
As stated I justed copied “bpi-r4.dtb, bpi-r4.itb and uImage_nodt to /boot as well as the modules to /lib/modules as usual.” For bpi-r4.dtb, bpi-r4.itb and uImage_nodt I used the files in …/SD/BPI-BOOT and the modules from …/SD/BPI-ROOT of course.
I don’t think so. But I have the kernel source on the BPI-R4 itself, as I am compiling it there.
Args. You are right again. I just used an editor to make the necessary changes. I just wanted a solution that is simple to copy. To test that it works, I tried it with uEnv.txt in /tmp and copied and pasted that without further thinking about it. It needs to be /boot/uEnv.txt of course. Fixed the post regarding this as well.
The changes in uEnv.txt are for your U-boot environment. When initrd is set and loading the file succeeds checkrd calls useinitrd. But this one sets the root to a hard-coded /dev/ram0. That’s why useinitrd has to be changed from your defaults to just adjust the bootopts variable and leave “root” alone.
I could try cross-compiling again and see if I can document it that way as well. (I switched to compiling on the BPI-R4 itself because I got an error when loading the cross-compiled nf_tables.ko module. I can see if the problem has resolved itself.)
I hope my post has not created more confusion than it tried to solve.
as far as i see, you only need .config from source in the running system right? or does mkinitramfs need also the full source? btw. the …/build directory is only used (configurable) if out-of-tree build is used (i use it to build in ram for less write Ops )
btw. boot on r64,r3 and r4 only needs the itb…the uImage_nodt is currently not used…i only build and pack it as someone wants to use separate dtb (to modify) without fit image.
mhm, someone else reported that the hardcoded root for initrd is not working for him…maybe best way would be change this in the builtin environment, that will make it also easier for you, right?
Yes. I believe you are right that update-initramfs only uses the config-kernel version from /boot and does not depend on the kernel sources.
I figured that’s why you used ramfs for the build-directory and think this is a very good idea when compiling from SD-card (or maybe even on eMMC). As I’m using an UAS SSD I disabled “ramdisksize” from boot.conf so the compiles don’t have to start from scratch after every reboot. (I was very pleased to see that you made it so easy to disable this feature.)
Good to know that uImage-nodt is not needed. I just have experience on regular PCs so I’m not really familiar with U-boot and booting on ARM in general.
It’s not much work to adjust the useinitrd variable in uEnv.txt. Although I don’t see the reason for setting root to a hard-coded /dev/ram0 if an initrd is used. If you are using an initrd you will probably adjust root anyway and can set it to /dev/ram0 if you want to.
Having just said that I have no clue about booting on ARM, I think the initramfs-way is much more flexible than a kernel with everything you might ever need to boot compiled in. :-). You can just compile everything as a module and the kernel will free the RAM of everything it does not need after the boot has completed. And some things are even impossible without an initrd like rootfs on LVM (I think) or an encrypted rootfs.
As the BPI-R4 is really powerful I think people will want to do interesting stuff with it and will appreciate the flexibility that an initrd brings to the table.
I just did a crosscompile with ./build.sh build and ./build.sh pack and have the exact same directory structure with ../build and ../SD/BPI/[BR]OOT/. So it doing this should be more or less identical to a compile on the BPI-R4 except that you need to copy ../build/.config, the kernel and the modules to the BPI-R4 instead of between directories on the BPI-R4.
(If anyone is interested: The compile took about 5min on a Ryzen 5600 with 32GB RAM and about 50min on the BPI-R4… quite the difference… :-))
At the moment my BPI-R4 is not powered on as I did not have the time to replace my current router yet as I had so much other stuff to do, but the instructions above worked for me and I did not have any problems with this setup.
Maybe you also need to set “root=…” in your uEnv.txt but this should be obvious.
(If somebody wants to follow these steps: You need to use the same kernel version when executing these steps or you replace $(uname -r) with the kernel version you want to use for your initrd build.)
for r3 i have dropped the override of root…works on r3mini added now to r4 too so last step is not needed in future
uname -r returns the current kernel version string and the /proc/config.gz contains the current kernel config so both will match…of couse if kernel changes this maybe needs to be done again, but i guess the update-initramfs only looks whats supported by the kernel with config to choose what to include in the initrd (e.g. lvm-utils - i hope)
)
added now to r4 too so last step is not needed in future