NixOS on Banana Pi BPI-R3

at the moment I only have this snippet to patch the kernel, but I haven’t tested it

  boot.kernelPatches = [{
    name = "patches_for_Banana_r3";
    patch = pkgs.fetchpatch {
      url = "https://github.com/frank-w/BPI-Router-Linux/compare/fbe58b8..379c024.patch";
      hash = "sha256-+dkorhIe3PlhK+xUycoAsssu7BywiaWyZfecXq5p2P0=";
    };
  }];

nixos.iso contains these dtbs. As I understand they are built as described in the kernel. Are they correct if booting via uboot?

name_snrl in ../media/name_snrl/NIXOS_SD/boot/nixos/h46bpi1q2bs90yj08isdxnjcwk4ykxdg-linux-6.3.1-dtbs [RO]
> ls -la mediatek/mt7986*
.r-xr-xr-x 477 root  1 Jan  1970 mediatek/mt7986a-bananapi-bpi-r3-emmc.dtbo
.r-xr-xr-x 913 root  1 Jan  1970 mediatek/mt7986a-bananapi-bpi-r3-nand.dtbo
.r-xr-xr-x 956 root  1 Jan  1970 mediatek/mt7986a-bananapi-bpi-r3-nor.dtbo
.r-xr-xr-x 327 root  1 Jan  1970 mediatek/mt7986a-bananapi-bpi-r3-sd.dtbo
.r-xr-xr-x 20k root  1 Jan  1970 mediatek/mt7986a-bananapi-bpi-r3.dtb
.r-xr-xr-x 18k root  1 Jan  1970 mediatek/mt7986a-rfb.dtb
.r-xr-xr-x 15k root  1 Jan  1970 mediatek/mt7986b-rfb.dtb

so the extlinux.conf file should be on the boot partition, right?

What should be there? For now I only have this one which was generated for nixos:

# Generated file, all changes will be lost on nixos-rebuild!

# Change this to e.g. nixos-42 to temporarily boot to an older configuration.
DEFAULT nixos-default

MENU TITLE ------------------------------------------------------------
TIMEOUT 50

LABEL nixos-default
  MENU LABEL NixOS - Default
  LINUX ../nixos/r80r91zqfg25516i7wz49vd5paiwd1g5-linux-5.15.110-Image
  INITRD ../nixos/gcvd9iaqkzlq20hv76shf15marqkr2sd-initrd-linux-5.15.110-initrd
  APPEND init=/nix/store/369walk9f0qm2h66r3zq71524wys0iik-nixos-system-nixos-22.11.4087.7629f9b0680/init console=ttyS0,115200n8 console=ttyAMA0,115200n8 console=tty0 nohibernate loglevel=7
  FDTDIR ../nixos/r80r91zqfg25516i7wz49vd5paiwd1g5-linux-5.15.110-dtbs

Will it just work if I put it into the boot partition?

So I tried also removing u-boot.bin to force direct initialization atf->kernel but then I got:

NOTICE:  BL2: v2.8(release):v2.8-538-g2f158d215-dirty
NOTICE:  BL2: Built : 11:26:21, May  9 2023
INFO:    BL2: Doing platform setup
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7986 (1998MHz)
NOTICE:  EMI: Using DDR4 settings
NOTICE:  EMI: Detected DRAM size: 2048MB
NOTICE:  EMI: complex R/W mem test passed
INFO:    MediaTek MMC/SD Card Controller ver 20200520, eco 0
INFO:    Located GPT partition 'fip/fat' at 0x100000, size 0x7f00000
INFO:    BL2: Loading image id 3
INFO:    Reading filename from bootcfg/bl31
WARNING: fat_file_open: failed opening bootcfg/bl31
WARNING: Failed to access image id=3 (-2)
INFO:    Image id=3 copied: 0x43001000 - 0x43009061
INFO:    BL2: Loading image id 5
INFO:    Reading filename from bootcfg/linux
INFO:    Opening (/boot/Image)
WARNING: fat_file_open: failed opening /boot/Image
WARNING: Failed to access image id=5 (-2)
ERROR:   BL2: Failed to load image id 5 (-2)

Though the file is there and its checksum is correct.

It cannot find the file Image inside the root of the boot partition.

You have not specified any dtb in the extlinux.conf

Use the dtb I supplied. If it works then experiment with others. Otherwise I can’t say why it doesn’t work.

I also think that linux image won’t work…

You could just use my script and build an archlinux image. There you can see exactly what you need and how it looks like. Then read [BPI-R3] Imagebuilder R3 Archlinux

I think you can use the example and apply it to nixos

It cannot find the file Image inside the root of the boot partition.

Indeed, I got confused and put both the Image and dtbs folder under /boot on the boot partition while they should be on root. After moving them there, it tries to load kernel but it fails with:

[    1.405024] FAT-fs (mmcblk0p3): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[    1.414989] VFS: Mounted root (vfat filesystem) on device 179:3.
[    1.421052] devtmpfs: error mounting -2
[    1.426198] Freeing unused kernel memory: 3264K
[    1.488942] Run /sbin/init as init process
[    1.493032]   with arguments:
[    1.495988]     /sbin/init
[    1.498681]   with environment:
[    1.501819]     HOME=/
[    1.504167]     TERM=linux
[    1.507094] Run /etc/init as init process
[    1.511095]   with arguments:
[    1.514049]     /etc/init
[    1.516656]   with environment:
[    1.519794]     HOME=/
[    1.522141]     TERM=linux
[    1.528479] Run /bin/init as init process
[    1.532491]   with arguments:
[    1.535445]     /bin/init
[    1.538052]   with environment:
[    1.541186]     HOME=/
[    1.543533]     TERM=linux
[    1.546322] Run /bin/sh as init process
[    1.550151]   with arguments:
[    1.553104]     /bin/sh
[    1.555537]   with environment:
[    1.558663]     HOME=/
[    1.561015]     TERM=linux
[    1.563760] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation.

I know, I am still using FAT, but is it because of that?

Full stack trace
F0: 102B 0000
FA: 1040 0000
FA: 1040 0000 [0200]
F9: 103F 0000
F3: 1001 0000 [0200]
F3: 1001 0000
F6: 300C 0028
F5: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 2400 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [3000]
T0: 0000 01E1 [010F]
Jump to BL

NOTICE:  BL2: v2.8(release):v2.8-538-g2f158d215-dirty
NOTICE:  BL2: Built : 11:26:21, May  9 2023
INFO:    BL2: Doing platform setup
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7986 (2000MHz)
NOTICE:  EMI: Using DDR4 settings
NOTICE:  EMI: Detected DRAM size: 2048MB
NOTICE:  EMI: complex R/W mem test passed
INFO:    MediaTek MMC/SD Card Controller ver 20200520, eco 0
INFO:    Located GPT partition 'fip/fat' at 0x100000, size 0x7f00000
INFO:    BL2: Loading image id 3
INFO:    Reading filename from bootcfg/bl31
WARNING: fat_file_open: failed opening bootcfg/bl31
WARNING: Failed to access image id=3 (-2)
INFO:    Image id=3 copied: 0x43001000 - 0x43009061
INFO:    BL2: Loading image id 5
INFO:    Reading filename from bootcfg/linux
INFO:    Opening (/boot/Image)
INFO:    Loading image id=5 at address 0x44000000
INFO:    Image id=5 loaded: 0x44000000 - 0x4525fa00
INFO:    BL2: Loading image id 22
INFO:    Loaded BL33 image is linux kernel image, loading DTB/INITRD
INFO:    Reading filename from bootcfg/initrd
INFO:    Opening (/boot/initramfs-linux-bpir64-git.img)
INFO:    Loading image id=22 at address 0x48000000
INFO:    Image id=22 loaded: 0x48000000 - 0x48000008
INFO:    BL2: Loading image id 27
INFO:    Loaded BL33 image is linux kernel image, loading DTB/INITRD
INFO:    Reading filename from bootcfg/atfdtb
INFO:    Opening (/boot/dtbs/mt7986a-bananapi-bpi-r3-atf.dtb)
INFO:    Loading image id=27 at address 0x4fd00000
INFO:    Image id=27 loaded: 0x4fd00000 - 0x4fd05f59
NOTICE:  BL2: Booting BL31
INFO:    Entry point address = 0x43001000
INFO:    SPSR = 0x3cd
INFO:    Total CPU count: 4
INFO:    MCUSYS: Disable 512KB L2C shared SRAM
INFO:    EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE KERNELLLLLLLLLLLLLL
INFO:    GICv3 without legacy support detected.
INFO:    ARM GICv3 driver initialized in EL3
INFO:    Maximum SPI INTID supported: 671
INFO:    SPMC: Changed to SPMC mode
NOTICE:  BL31: v2.8(release):v2.8-538-g2f158d215-dirty
NOTICE:  BL31: Built : 11:26:21, May  9 2023
INFO:    [MPU](Region0)sa:0x0300, ea:0x0302
INFO:    [MPU](Region0)apc0:0x80b6db69, apc1:0x00b6db6d
INFO:    [MPU](Region1)sa:0x0000, ea:0x0000
INFO:    [MPU](Region1)apc0:0x00000000, apc1:0x00000000
INFO:    [MPU](Region2)sa:0x0000, ea:0x0000
INFO:    [MPU](Region2)apc0:0x00000000, apc1:0x00000000
INFO:    [MPU](Region3)sa:0x0000, ea:0x0000
INFO:    [MPU](Region3)apc0:0x00000000, apc1:0x00000000
INFO:    [DEVAPC] devapc_init done
INFO:    BL31: Initializing runtime services
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x44000000
INFO:    SPSR = 0x3c9
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 6.2.15-bpi (nathalieeneric@rk3588) (gcc (GCC) 12.1.0, GNU ld (GNU Binutils) 2.38) #13 SMP PREEMPT3
[    0.000000] Machine model: Bananapi BPI-R3
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000011002000 (options '')
[    0.000000] printk: bootconsole [uart8250] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] OF: fdt: Reserved memory: failed to reserve memory for node 'wo-emi@4fd00000': base 0x000000004fd00000, size 0 B
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x0000000042ffffff]
[    0.000000]   node   0: [mem 0x0000000043000000-0x000000004302ffff]
[    0.000000]   node   0: [mem 0x0000000043030000-0x000000004fbfffff]
[    0.000000]   node   0: [mem 0x000000004fc00000-0x000000004fcfffff]
[    0.000000]   node   0: [mem 0x000000004fd00000-0x000000004fd3ffff]
[    0.000000]   node   0: [mem 0x000000004fd40000-0x000000004ffbffff]
[    0.000000]   node   0: [mem 0x000000004ffc0000-0x00000000bfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 27 pages/cpu s72616 r8192 d29784 u110592
[    0.000000] pcpu-alloc: s72616 r8192 d29784 u110592 alloc=27*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: ARM erratum 845719
[    0.000000] alternatives: applying boot alternatives
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 516096
[    0.000000] Kernel command line: root=PARTLABEL=bpir3-sdmmc-root earlycon=uart8250,mmio32,0x11002000 console=ttyS0,115200 d0
[    0.000000] audit: disabled (until reboot)
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 2025352K/2097152K available (9792K kernel code, 1982K rwdata, 3668K rodata, 3264K init, 419K bss, 71800)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu:     RCU event tracing is enabled.
[    0.000000]  Trampoline variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 640 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: GICv3 features: 16 PPIs
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x000000000c080000
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] arch_timer: cp15 timer(s) running at 13.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x2ff89eacb, max_idle_ns: 440795202429 ns
[    0.000000] sched_clock: 56 bits at 13MHz, resolution 76ns, wraps every 4398046511101ns
[    0.008333] Console: colour dummy device 80x25
[    0.012824] Calibrating delay loop (skipped), value calculated using timer frequency.. 26.00 BogoMIPS (lpj=130000)
[    0.023239] pid_max: default: 32768 minimum: 301
[    0.028004] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.035453] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.044080] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    0.050731] cblist_init_generic: Setting adjustable number of callback queues.
[    0.057993] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.064186] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.070450] rcu: Hierarchical SRCU implementation.
[    0.075274] rcu:     Max phase no-delay instances is 1000.
[    0.081044] EFI services will not be available.
[    0.085823] smp: Bringing up secondary CPUs ...
[    0.090819] Detected VIPT I-cache on CPU1
[    0.090891] cacheinfo: Unable to detect cache hierarchy for CPU 1
[    0.090901] GICv3: CPU1: found redistributor 1 region 0:0x000000000c0a0000
[    0.090938] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[    0.091412] Detected VIPT I-cache on CPU2
[    0.091466] cacheinfo: Unable to detect cache hierarchy for CPU 2
[    0.091472] GICv3: CPU2: found redistributor 2 region 0:0x000000000c0c0000
[    0.091489] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[    0.091912] Detected VIPT I-cache on CPU3
[    0.091964] cacheinfo: Unable to detect cache hierarchy for CPU 3
[    0.091970] GICv3: CPU3: found redistributor 3 region 0:0x000000000c0e0000
[    0.091985] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[    0.092041] smp: Brought up 1 node, 4 CPUs
[    0.167013] SMP: Total of 4 processors activated.
[    0.171736] CPU features: detected: 32-bit EL0 Support
[    0.176896] CPU features: detected: CRC32 instructions
[    0.182109] CPU features: emulated: Privileged Access Never (PAN) using TTBR0_EL1 switching
[    0.190504] CPU: All CPU(s) started at EL2
[    0.194637] alternatives: applying system-wide alternatives
[    0.201161] devtmpfs: initialized
[    0.207940] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.217864] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.224882] pinctrl core: initialized pinctrl subsystem
[    0.230446] DMI not present or invalid.
[    0.234755] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.241162] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations
[    0.248274] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.256098] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.264476] thermal_sys: Registered thermal governor 'fair_share'
[    0.264480] thermal_sys: Registered thermal governor 'bang_bang'
[    0.270618] thermal_sys: Registered thermal governor 'step_wise'
[    0.276651] thermal_sys: Registered thermal governor 'user_space'
[    0.282813] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.295827] ASID allocator initialised with 65536 entries
[    0.312845] cryptd: max_cpu_qlen set to 1000
[    0.318273] iommu: Default domain type: Translated
[    0.323194] iommu: DMA domain TLB invalidation policy: strict mode
[    0.329733] SCSI subsystem initialized
[    0.333679] libata version 3.00 loaded.
[    0.337647] usbcore: registered new interface driver usbfs
[    0.343179] usbcore: registered new interface driver hub
[    0.348548] usbcore: registered new device driver usb
[    0.353966] i2c-gpio i2c-gpio-0: using lines 528 (SDA) and 529 (SCL)
[    0.360511] i2c-gpio i2c-gpio-1: using lines 530 (SDA) and 531 (SCL)
[    0.366936] mc: Linux media interface: v0.10
[    0.371250] videodev: Linux video capture interface: v2.00
[    0.376792] pps_core: LinuxPPS API ver. 1 registered
[    0.381782] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    0.390971] PTP clock support registered
[    0.395865] vgaarb: loaded
[    0.398883] clocksource: Switched to clocksource arch_sys_counter
[    0.411503] NET: Registered PF_INET protocol family
[    0.416659] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.425652] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
[    0.434341] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.442158] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.450194] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    0.457883] TCP: Hash tables configured (established 16384 bind 16384)
[    0.464577] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.471346] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.478711] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.484434] PCI: CLS 0 bytes, default 64
[    0.488626] Unpacking initramfs...
[    0.489733] Initialise system trusted keyrings
[    0.492079] Initramfs unpacking failed: invalid magic at start of compressed archive
[    0.496614] workingset: timestamp_bits=46 max_order=19 bucket_order=0
[    0.507773] Freeing initrd memory: 8232K
[    0.511135] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.520616] jffs2: version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
[    0.549953] Key type asymmetric registered
[    0.554077] Asymmetric key parser 'x509' registered
[    0.559036] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    0.566471] io scheduler mq-deadline registered
[    0.571027] io scheduler kyber registered
[    0.586416] Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
[    0.594512] printk: console [ttyS0] disabled
[    0.618973] 11002000.serial: ttyS0 at MMIO 0x11002000 (irq = 116, base_baud = 2500000) is a ST16650V2
[    0.628298] printk: console [ttyS0] enabled
[    0.628298] printk: console [ttyS0] enabled
[    0.636663] printk: bootconsole [uart8250] disabled
[    0.636663] printk: bootconsole [uart8250] disabled
[    0.669931] 11003000.serial: ttyS1 at MMIO 0x11003000 (irq = 117, base_baud = 1625000) is a ST16650V2
[    0.700680] 11004000.serial: ttyS2 at MMIO 0x11004000 (irq = 118, base_baud = 1625000) is a ST16650V2
[    0.710876] mtk_rng 1020f000.rng: registered RNG driver
[    0.711153] random: crng init done
[    0.716451] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    0.728988] loop: module loaded
[    0.735359] sfp sfp-1: Host maximum power 1.0W
[    0.740515] sfp sfp-2: Host maximum power 1.0W
[    0.761188] mtk_soc_eth 15100000.ethernet: generated random MAC address 3a:d5:e1:db:85:dc
[    0.770679] mtk_soc_eth 15100000.ethernet eth0: mediatek frame engine at 0xffffffc009c80000, irq 123
[    0.779881] mtk_soc_eth 15100000.ethernet: generated random MAC address 36:e3:98:b2:72:22
[    0.789330] mtk_soc_eth 15100000.ethernet eth1: mediatek frame engine at 0xffffffc009c80000, irq 123
[    0.798862] usbcore: registered new interface driver rtl8150
[    0.804779] usbcore: registered new interface driver usb-storage
[    0.811131] i2c_dev: i2c /dev entries driver
[    0.817209] mtk-wdt 1001c000.watchdog: Watchdog enabled (timeout=31 sec, nowayout=0)
[    0.827439] ledtrig-cpu: registered to indicate activity on CPUs
[    0.833687] SMCCC: SOC_ID: ID = jep106:0426:7986 Revision = 0x00000000
[    0.840331] usbcore: registered new interface driver usbhid
[    0.845890] usbhid: USB HID core driver
[    0.850269] NET: Registered PF_INET6 protocol family
[    0.855934] Segment Routing with IPv6
[    0.859623] In-situ OAM (IOAM) with IPv6
[    0.863567] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.869974] NET: Registered PF_PACKET protocol family
[    0.875138] 8021q: 802.1Q VLAN Support v1.8
[    0.886467] mmc0: host does not support reading read-only switch, assuming write-enable
[    0.893092] Loading compiled-in X.509 certificates
[    0.899703] mmc0: new high speed SDXC card at address e624
[    0.906188] mmcblk0: mmc0:e624 SE128 119 GiB
[    0.907904] mtk-pcie-gen3 11280000.pcie: host bridge /soc/pcie@11280000 ranges:
[    0.917928] mtk-pcie-gen3 11280000.pcie:      MEM 0x0020000000..0x002fffffff -> 0x0020000000
[    0.919069]  mmcblk0: p1 p2 p3
[    1.138917] mtk-pcie-gen3 11280000.pcie: PCIe link down, current LTSSM state: detect.active (0x1000001)
[    1.148337] mtk-pcie-gen3: probe of 11280000.pcie failed with error -110
[    1.178846] mt7530 mdio-bus:1f: configuring for fixed/2500base-x link mode
[    1.186018] mt7530 mdio-bus:1f: Link is Up - 2.5Gbps/Full - flow control rx/tx
[    1.195219] mt7530 mdio-bus:1f wan (uninitialized): PHY [mt7530-0:00] driver [MediaTek MT7531 PHY] (irq=129)
[    1.215308] mt7530 mdio-bus:1f lan0 (uninitialized): PHY [mt7530-0:01] driver [MediaTek MT7531 PHY] (irq=130)
[    1.235181] mt7530 mdio-bus:1f lan1 (uninitialized): PHY [mt7530-0:02] driver [MediaTek MT7531 PHY] (irq=131)
[    1.255040] mt7530 mdio-bus:1f lan2 (uninitialized): PHY [mt7530-0:03] driver [MediaTek MT7531 PHY] (irq=132)
[    1.274898] mt7530 mdio-bus:1f lan3 (uninitialized): PHY [mt7530-0:04] driver [MediaTek MT7531 PHY] (irq=133)
[    1.286223] device eth0 entered promiscuous mode
[    1.290897] DSA: tree 0 setup
[    1.294390] xhci-mtk 11200000.usb: supply vbus not found, using dummy regulator
[    1.301790] xhci-mtk 11200000.usb: supply vusb33 not found, using dummy regulator
[    1.310281] xhci-mtk 11200000.usb: xHCI Host Controller
[    1.315516] xhci-mtk 11200000.usb: new USB bus registered, assigned bus number 1
[    1.325985] xhci-mtk 11200000.usb: hcc params 0x01403f99 hci version 0x110 quirks 0x0000000000210010
[    1.335156] xhci-mtk 11200000.usb: irq 134, io mem 0x11200000
[    1.340901] xhci-mtk 11200000.usb: xHCI Host Controller
[    1.346115] xhci-mtk 11200000.usb: new USB bus registered, assigned bus number 2
[    1.353504] xhci-mtk 11200000.usb: Host supports USB 3.2 Enhanced SuperSpeed
[    1.361099] hub 1-0:1.0: USB hub found
[    1.364866] hub 1-0:1.0: 2 ports detected
[    1.369280] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    1.377820] hub 2-0:1.0: USB hub found
[    1.381608] hub 2-0:1.0: 1 port detected
[    1.386561] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    1.393675] of_cfs_init
[    1.396193] of_cfs_init: OK
[    1.405024] FAT-fs (mmcblk0p3): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[    1.414989] VFS: Mounted root (vfat filesystem) on device 179:3.
[    1.421052] devtmpfs: error mounting -2
[    1.426198] Freeing unused kernel memory: 3264K
[    1.488942] Run /sbin/init as init process
[    1.493032]   with arguments:
[    1.495988]     /sbin/init
[    1.498681]   with environment:
[    1.501819]     HOME=/
[    1.504167]     TERM=linux
[    1.507094] Run /etc/init as init process
[    1.511095]   with arguments:
[    1.514049]     /etc/init
[    1.516656]   with environment:
[    1.519794]     HOME=/
[    1.522141]     TERM=linux
[    1.528479] Run /bin/init as init process
[    1.532491]   with arguments:
[    1.535445]     /bin/init
[    1.538052]   with environment:
[    1.541186]     HOME=/
[    1.543533]     TERM=linux
[    1.546322] Run /bin/sh as init process
[    1.550151]   with arguments:
[    1.553104]     /bin/sh
[    1.555537]   with environment:
[    1.558663]     HOME=/
[    1.561015]     TERM=linux
[    1.563760] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation.
[    1.577897] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 6.2.15-bpi #13
[    1.584234] Hardware name: Bananapi BPI-R3 (DT)
[    1.588750] Call trace:
[    1.591184]  dump_backtrace+0xe0/0x124
[    1.594928]  show_stack+0x18/0x30
[    1.598231]  dump_stack_lvl+0x64/0x80
[    1.601882]  dump_stack+0x18/0x34
[    1.605185]  panic+0x130/0x300
[    1.608229]  kernel_init+0x110/0x124
[    1.611793]  ret_from_fork+0x10/0x20
[    1.615359] SMP: stopping secondary CPUs
[    1.619270] Kernel Offset: disabled
[    1.622743] CPU features: 0x00000,01000108,0000420b
[    1.627604] Memory Limit: none
[    1.630647] Rebooting in 1 seconds..

Looks good so far. Fat could be the issue, your rootfs is not as it should be. But now this will be nixos specific …

Right, I can add there something like:

fdt /path/to/dtb_file.dtb

but what bothers me the most is that my nixos kernel and the rest of the stuff that extlinux.conf refers to is on a different partition (extlinux.conf is on boot while rest is on rootfs)

When I tried to boot it the process failed with:

Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
------------------------------------------------------------
1:      NixOS - Default
Enter choice: 1:        NixOS - Default
Retrieving file: /boot/extlinux/../nixos/r80r91zqfg25516i7wz49vd5paiwd1g5-linux-5.15.110-Image
Skipping nixos-default for failure retrieving kernel
EXTLINUX FAILED: continuing...

Which is somehow not a surprise. Could you tell me how can I refer to the rootfs partition in that file? Or maybe I could merge both boot and rootfs partitions and then I wouldn’t have that problem at all?

The entire purpose of the boot partition in fat is that the bootloader can access it, and the rootfs partition can be anything, not readable by bootloader

However, uboot can also read ext4, but I’m not sure if distro-boot reads from ext4.

You need to parted set boot on then also on the rootfs partition, formatted ext4. It will then hopefully search it also. I think it could find the /extlinux/extlinux.conf file on that partition too. But first remove it from boot partition before trying, as this will be searched earlier

Or you make nixos copy the files to the boot partition mounted at /boot

Finally some success :slight_smile:

CPU:   MediaTek MT7986
Model: mt7986-rfb
DRAM:  2 GiB
Core:  38 devices, 14 uclasses, devicetree: separate
MMC:   mmc@11230000: 0
Loading Environment from nowhere... OK
In:    serial@11002000
Out:   serial@11002000
Err:   serial@11002000
Net:
Warning: ethernet@15100000 (eth0) using random MAC address - 3e:5a:18:68:97:ce
eth0: ethernet@15100000
Hit any key to stop autoboot:  0
MMC Device 1 not found
no mmc device at slot 1
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:2...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
------------------------------------------------------------
1:      NixOS - Default
Enter choice: 1:        NixOS - Default
Retrieving file: /boot/extlinux/../nixos/wbv7qr9q5bqlr4dn5srjqmhxppp2sxgn-linux-6.3.1-Image
Retrieving file: /boot/extlinux/../nixos/slnriwqj7rw29dqp1fh04736j7vwfqn9-initrd-linux-6.3.1-initrd
append: init=/nix/store/9ifmb0qa9jm6bl95walaf0gn1w7nsfn7-nixos-system-nixos-23.05pre483514.790a208ab94/init console=ttyS0,11527
Retrieving file: /boot/extlinux/../nixos/wbv7qr9q5bqlr4dn5srjqmhxppp2sxgn-linux-6.3.1-dtbs/mt7986a-sd-rfb
## Flattened Device Tree blob at bf7fdcd0
   Booting using the fdt blob at 0xbf7fdcd0
Working FDT set to bf7fdcd0
   Loading Ramdisk to bec3b000, end bf7f79c7 ... OK
   Loading Device Tree to 000000004fcfa000, end 000000004fcff0df ... OK
Working FDT set to 4fcfa000

Starting kernel ...

However it hangs here. The problem might be the one described here: https://nixos.wiki/wiki/NixOS_on_ARM#Enable_UART

If you try to use UART to log on NixOS, you might hang on the line “Starting kernel …”. To enable UART, you will need to add at the end of the line that contains loglevel4 in the file /extlinux/extlinux.conf the text:

Breeze-text-x-plain.png

/extlinux/extlinux.conf

console=ttyAMA0,115200n8

Breeze-text-x-plain.png

/extlinux/extlinux.conf

console=ttyS0,115200n8

However I already have in my extlinux.conf this:

LABEL nixos-default
  MENU LABEL NixOS - Default
  LINUX ../nixos/wbv7qr9q5bqlr4dn5srjqmhxppp2sxgn-linux-6.3.1-Image
  INITRD ../nixos/slnriwqj7rw29dqp1fh04736j7vwfqn9-initrd-linux-6.3.1-initrd
  APPEND init=/nix/store/9ifmb0qa9jm6bl95walaf0gn1w7nsfn7-nixos-system-nixos-23.05pre483514.790a208ab94/init console=ttyS0,115200n8 console=ttyAMA0,115200n8 console=tty0 loglevel=7
  FDTDIR ../nixos/wbv7qr9q5bqlr4dn5srjqmhxppp2sxgn-linux-6.3.1-dtbs

Can I bruteforce it and add S1,S2 and S3?

Perhapse someone at nixos can help you with the kernel.

Is it configured for the appropriate mediatek devices?

Edit: i found out that I cannot use the standard archlinuxarm linux kernel, because many mtk config options are not enabled. Mediatek boards are not included. It is why I need to use my own linux package.

It doesn’t apply any patching to code, only some changes to devicetree. Then I added Dynamic overlay support, which is an addition to the code. I use Dynamic overlay to access Emmc at hs200 when booting from sdmmc

Is it configured for the appropriate mediatek devices?

Not sure what do you mean by “configured”. I used latest version of unstable channel that contains the latest kernel and I can see dtb files for mediatek devices in the nixos image:

❯ ls mediatek                                                                                                         
mt2712-evb.dtb                     mt8173-elm-hana.dtb                     mt8183-kukui-kakadu.dtb
mt6755-evb.dtb                     mt8173-elm.dtb                          mt8183-kukui-kodama-sku16.dtb
mt6779-evb.dtb                     mt8173-evb.dtb                          mt8183-kukui-kodama-sku32.dtb
mt6795-evb.dtb                     mt8183-evb.dtb                          mt8183-kukui-kodama-sku272.dtb
mt6795-sony-xperia-m5.dtb          mt8183-kukui-jacuzzi-burnet.dtb         mt8183-kukui-kodama-sku288.dtb
mt6797-evb.dtb                     mt8183-kukui-jacuzzi-cozmo.dtb          mt8183-kukui-krane-sku0.dtb
mt6797-x20-dev.dtb                 mt8183-kukui-jacuzzi-damu.dtb           mt8183-kukui-krane-sku176.dtb
mt7622-bananapi-bpi-r64.dtb        mt8183-kukui-jacuzzi-fennel-sku1.dtb    mt8183-pumpkin.dtb
mt7622-rfb1.dtb                    mt8183-kukui-jacuzzi-fennel-sku6.dtb    mt8186-evb.dtb
mt7986a-bananapi-bpi-r3-emmc.dtbo  mt8183-kukui-jacuzzi-fennel-sku7.dtb    mt8192-asurada-hayato-r1.dtb
mt7986a-bananapi-bpi-r3-nand.dtbo  mt8183-kukui-jacuzzi-fennel14-sku2.dtb  mt8192-asurada-spherion-r0.dtb
mt7986a-bananapi-bpi-r3-nor.dtbo   mt8183-kukui-jacuzzi-fennel14.dtb       mt8192-evb.dtb
mt7986a-bananapi-bpi-r3-sd.dtbo    mt8183-kukui-jacuzzi-juniper-sku16.dtb  mt8195-cherry-tomato-r1.dtb
mt7986a-bananapi-bpi-r3.dtb        mt8183-kukui-jacuzzi-kappa.dtb          mt8195-cherry-tomato-r2.dtb
mt7986a-rfb.dtb                    mt8183-kukui-jacuzzi-kenzo.dtb          mt8195-cherry-tomato-r3.dtb
mt7986b-rfb.dtb                    mt8183-kukui-jacuzzi-willow-sku0.dtb    mt8195-demo.dtb
mt8167-pumpkin.dtb                 mt8183-kukui-jacuzzi-willow-sku1.dtb    mt8195-evb.dtb
mt8173-elm-hana-rev7.dtb           mt8183-kukui-kakadu-sku22.dtb           mt8516-pumpkin.dtb

i found out that I cannot use the standard archlinuxarm linux kernel, because many mtk config options are not enabled. Mediatek boards are not included. It is why I need to use my own linux package.

Ah that might be it. What do you mean not enabled? How can I enable it? Also what do you mean “mediatek boards are not included”?

It doesn’t apply any patching to code, only some changes to devicetree. Then I added Dynamic overlay support, which is an addition to the code. I use Dynamic overlay to access Emmc at hs200 when booting from sdmmc

You mean that even though mediatek devices are in the devicetree, there are no patches applied to kernel? Sorry, I am not sure if I understand.

Anyway, I decided to try out the dtb file that you sent earlier. I modified the extlinux.conf and changed

  FDTDIR ../nixos/wbv7qr9q5bqlr4dn5srjqmhxppp2sxgn-linux-6.3.1-dtbs

to

FDT ../dtbs/mt7986a-bananapi-bpi-r3-atf.dtb

And it worked (sort of):

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 6.3.1 (nixbld@localhost) (gcc (GCC) 12.2.0, GNU ld (GNU Binutils) 2.40) #1-NixOS SMP Sun Apr 30 23
[    0.000000] Machine model: Bananapi BPI-R3
[    0.000000] efi: UEFI not found.
[    0.000000] OF: reserved mem: 0x0000000015194000..0x0000000015194fff (4 KiB) nomap non-reusable wo-boot@15194000
[    0.000000] OF: reserved mem: 0x00000000151e0000..0x00000000151e7fff (32 KiB) nomap non-reusable wo-ilm@151e0000
[    0.000000] OF: reserved mem: 0x00000000151e8000..0x00000000151e9fff (8 KiB) nomap non-reusable wo-dlm@151e8000
[    0.000000] OF: reserved mem: 0x00000000151f0000..0x00000000151f7fff (32 KiB) nomap non-reusable wo-ilm@151f0000
[    0.000000] OF: reserved mem: 0x00000000151f8000..0x00000000151f9fff (8 KiB) nomap non-reusable wo-dlm@151f8000
[    0.000000] OF: reserved mem: 0x0000000043000000..0x000000004302ffff (192 KiB) nomap non-reusable secmon@43000000
[    0.000000] OF: reserved mem: 0x000000004fc00000..0x000000004fcfffff (1024 KiB) nomap non-reusable wmcpu-reserved@4fc00000
[    0.000000] OF: reserved mem: 0x000000004fd00000..0x000000004fd3ffff (256 KiB) nomap non-reusable wo-emi@4fd00000
[    0.000000] OF: reserved mem: 0x000000004fd40000..0x000000004fd7ffff (256 KiB) nomap non-reusable wo-emi@4fd40000
[    0.000000] OF: reserved mem: 0x000000004fd80000..0x000000004ffbffff (2304 KiB) nomap non-reusable wo-data@4fd80000
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xbfbe88c0-0xbfbebfff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   empty
[    0.000000]   Device   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x0000000042ffffff]
[    0.000000]   node   0: [mem 0x0000000043000000-0x000000004302ffff]
[    0.000000]   node   0: [mem 0x0000000043030000-0x000000004fbfffff]
[    0.000000]   node   0: [mem 0x000000004fc00000-0x000000004ffbffff]
[    0.000000]   node   0: [mem 0x000000004ffc0000-0x00000000bfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000] cma: Reserved 32 MiB at 0x00000000bac00000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 31 pages/cpu s89320 r8192 d29464 u126976
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: ARM erratum 845719
[    0.000000] alternatives: applying boot alternatives
[    0.000000] Fallback order for Node 0: 0
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 516096
[    0.000000] Policy zone: DMA
[    0.000000] Kernel command line: init=/nix/store/9ifmb0qa9jm6bl95walaf0gn1w7nsfn7-nixos-system-nixos-23.05pre483514.790a2088
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
[    0.000000] Memory: 1947424K/2097152K available (21376K kernel code, 5812K rwdata, 19504K rodata, 12736K init, 740K bss, 11)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] ftrace: allocating 67185 entries in 263 pages
[    0.000000] ftrace: allocated 263 pages with 4 groups
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU event tracing is enabled.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.
[    0.000000]  Rude variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 640 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: GICv3 features: 16 PPIs
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x000000000c080000
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] arch_timer: cp15 timer(s) running at 13.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x2ff89eacb, max_idle_ns: 440795202429 ns
[    0.000000] sched_clock: 56 bits at 13MHz, resolution 76ns, wraps every 4398046511101ns
[    0.000454] Console: colour dummy device 80x25
[    0.000461] printk: console [tty0] enabled
[    0.000808] Calibrating delay loop (skipped), value calculated using timer frequency.. 26.00 BogoMIPS (lpj=52000)
[    0.000822] pid_max: default: 32768 minimum: 301
[    0.000997] LSM: initializing lsm=capability,landlock,yama,integrity,bpf
[    0.001047] landlock: Up and running.
[    0.001053] Yama: becoming mindful.
[    0.001092] LSM support for eBPF active
[    0.001215] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.001233] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.002671] cblist_init_generic: Setting adjustable number of callback queues.
[    0.002696] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.002753] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.002902] rcu: Hierarchical SRCU implementation.
[    0.002909] rcu:     Max phase no-delay instances is 1000.
[    0.004863] EFI services will not be available.
[    0.005189] smp: Bringing up secondary CPUs ...
[    0.005614] Detected VIPT I-cache on CPU1
[    0.005658] GICv3: CPU1: found redistributor 1 region 0:0x000000000c0a0000
[    0.005693] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[    0.006130] Detected VIPT I-cache on CPU2
[    0.006159] GICv3: CPU2: found redistributor 2 region 0:0x000000000c0c0000
[    0.006175] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[    0.006554] Detected VIPT I-cache on CPU3
[    0.006578] GICv3: CPU3: found redistributor 3 region 0:0x000000000c0e0000
[    0.006591] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[    0.006636] smp: Brought up 1 node, 4 CPUs
[    0.006682] SMP: Total of 4 processors activated.
[    0.006689] CPU features: detected: 32-bit EL0 Support
[    0.006694] CPU features: detected: 32-bit EL1 Support
[    0.006700] CPU features: detected: CRC32 instructions
[    0.006751] CPU: All CPU(s) started at EL2
[    0.006769] alternatives: applying system-wide alternatives
[    0.008707] devtmpfs: initialized
[    0.012373] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.012411] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.013292] pinctrl core: initialized pinctrl subsystem
[    0.014426] DMI not present or invalid.
[    0.014993] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.015907] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations
[    0.015995] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.016137] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.016217] audit: initializing netlink subsys (disabled)
[    0.016345] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1
[    0.017189] thermal_sys: Registered thermal governor 'step_wise'
[    0.017194] thermal_sys: Registered thermal governor 'power_allocator'
[    0.017261] cpuidle: using governor ladder
[    0.017284] cpuidle: using governor menu
[    0.017518] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.017598] ASID allocator initialised with 65536 entries
[    0.018864] Serial: AMBA PL011 UART driver
[    0.030299] KASLR disabled due to lack of seed
[    0.054645] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[    0.054673] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page
[    0.054680] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
[    0.054686] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page
[    0.054693] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[    0.054699] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page
[    0.054705] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
[    0.054711] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
[    0.498535] fbcon: Taking over console
[    0.498586] ACPI: Interpreter disabled.
[    0.500397] iommu: Default domain type: Translated
[    0.500420] iommu: DMA domain TLB invalidation policy: strict mode
[    0.500726] SCSI subsystem initialized
[    0.501017] usbcore: registered new interface driver usbfs
[    0.501039] usbcore: registered new interface driver hub
[    0.501062] usbcore: registered new device driver usb
[    0.501571] pps_core: LinuxPPS API ver. 1 registered
[    0.501578] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    0.501600] PTP clock support registered
[    0.501678] EDAC MC: Ver: 3.0.0
[    0.502082] scmi_core: SCMI protocol bus registered
[    0.502519] FPGA manager framework
[    0.502613] Advanced Linux Sound Architecture Driver Initialized.
[    0.503384] vgaarb: loaded
[    0.512627] clocksource: Switched to clocksource arch_sys_counter
[    0.512975] VFS: Disk quotas dquot_6.6.0
[    0.513027] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.513231] pnp: PnP ACPI: disabled
[    0.517978] NET: Registered PF_INET protocol family
[    0.518195] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.519845] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
[    0.519902] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.519917] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.519993] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    0.520378] TCP: Hash tables configured (established 16384 bind 16384)
[    0.520725] MPTCP token hash table entries: 2048 (order: 3, 49152 bytes, linear)
[    0.520835] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.520869] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.521025] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.521509] RPC: Registered named UNIX socket transport module.
[    0.521526] RPC: Registered udp transport module.
[    0.521532] RPC: Registered tcp transport module.
[    0.521537] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.521551] NET: Registered PF_XDP protocol family
[    0.521573] PCI: CLS 0 bytes, default 64
[    0.521848] Trying to unpack rootfs image as initramfs...
[    0.529333] kvm [1]: IPA Size Limit: 40 bits
[    0.531837] kvm [1]: vgic-v2@c420000
[    0.531884] kvm [1]: GIC system register CPU interface enabled
[    0.531930] kvm [1]: vgic interrupt IRQ9
[    0.531956] kvm [1]: Hyp mode initialized successfully
[    0.533422] Initialise system trusted keyrings
[    0.533795] workingset: timestamp_bits=42 max_order=19 bucket_order=0
[    0.533886] zbud: loaded
[    0.534595] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.535050] NFS: Registering the id_resolver key type
[    0.535102] Key type id_resolver registered
[    0.535109] Key type id_legacy registered
[    0.535142] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.535151] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    0.535330] 9p: Installing v9fs 9p2000 file system support
[    0.557740] Key type asymmetric registered
[    0.557818] Asymmetric key parser 'x509' registered
[    0.557927] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.558193] io scheduler mq-deadline registered
[    0.558211] io scheduler kyber registered
[    0.566109] EINJ: ACPI disabled.
[    0.582601] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.584995] printk: console [ttyS0] disabled
[    0.605193] 11002000.serial: ttyS0 at MMIO 0x11002000 (irq = 116, base_baud = 2500000) is a ST16650V2
[    0.605284] printk: console [ttyS0] enabled
[    0.856144] Freeing initrd memory: 12016K
[    0.882126] 11003000.serial: ttyS1 at MMIO 0x11003000 (irq = 117, base_baud = 1625000) is a ST16650V2
[    1.783840] 11004000.serial: ttyS2 at MMIO 0x11004000 (irq = 118, base_baud = 1625000) is a ST16650V2
[    1.793875] SuperH (H)SCI(F) driver initialized
[    1.798721] msm_serial: driver initialized
[    1.808158] loop: module loaded
[    1.812099] megasas: 07.725.01.00-rc1
[    1.820983] tun: Universal TUN/TAP device driver, 1.6
[    1.826601] thunder_xcv, ver 1.0
[    1.829862] thunder_bgx, ver 1.0
[    1.833100] nicpf, ver 1.0
[    1.836492] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.843707] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.849036] hclge is initializing
[    1.852355] e1000: Intel(R) PRO/1000 Network Driver
[    1.857223] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.862970] e1000e: Intel(R) PRO/1000 Network Driver
[    1.867923] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.873853] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.879240] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.884817] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.891072] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.897205] sky2: driver version 1.30
[    1.902853] usbcore: registered new interface driver usb-storage
[    1.910364] i2c_dev: i2c /dev entries driver
[    1.918240] mtk-wdt 1001c000.watchdog: Watchdog enabled (timeout=31 sec, nowayout=0)
[    1.926401] ghes_edac: GHES probing device list is empty
[    1.927665] sdhci: Secure Digital Host Controller Interface driver
[    1.939176] sdhci: Copyright(c) Pierre Ossman
[    1.944004] Synopsys Designware Multimedia Card Interface Driver
[    1.950841] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.958178] ledtrig-cpu: registered to indicate activity on CPUs
[    1.965472] SMCCC: SOC_ID: ID = jep106:0426:7986 Revision = 0x00000000
[    1.972378] hid: raw HID events driver (C) Jiri Kosina
[    1.977799] usbcore: registered new interface driver usbhid
[    1.983364] usbhid: USB HID core driver
[    1.991374] drop_monitor: Initializing network drop monitor service
[    2.011035] mmc0: host does not support reading read-only switch, assuming write-enable
[    2.024406] mmc0: new high speed SDXC card at address e624
[    2.031185] mmcblk0: mmc0:e624 SE128 119 GiB
[    2.045824]  mmcblk0: p1 p2
[    2.054396] NET: Registered PF_INET6 protocol family
[    2.061363] Segment Routing with IPv6
[    2.065102] In-situ OAM (IOAM) with IPv6
[    2.069084] NET: Registered PF_PACKET protocol family
[    2.074298] 9pnet: Installing 9P2000 support
[    2.078629] Key type dns_resolver registered
[    2.089447] registered taskstats version 1
[    2.093841] Loading compiled-in X.509 certificates
[    2.098846] zswap: loaded using pool lzo/zbud
[    2.109093] Key type .fscrypt registered
[    2.113038] Key type fscrypt-provisioning registered
[    2.126598] xhci-mtk 11200000.usb: supply vbus not found, using dummy regulator
[    2.134123] xhci-mtk 11200000.usb: supply vusb33 not found, using dummy regulator
[    2.142515] xhci-mtk 11200000.usb: xHCI Host Controller
[    2.147748] xhci-mtk 11200000.usb: new USB bus registered, assigned bus number 1
[    2.158152] xhci-mtk 11200000.usb: hcc params 0x01403f99 hci version 0x110 quirks 0x0000000000210010
[    2.167326] xhci-mtk 11200000.usb: irq 124, io mem 0x11200000
[    2.173183] xhci-mtk 11200000.usb: xHCI Host Controller
[    2.178404] xhci-mtk 11200000.usb: new USB bus registered, assigned bus number 2
[    2.185792] xhci-mtk 11200000.usb: Host supports USB 3.2 Enhanced SuperSpeed
[    2.193272] hub 1-0:1.0: USB hub found
[    2.197040] hub 1-0:1.0: 2 ports detected
[    2.201307] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    2.209689] hub 2-0:1.0: USB hub found
[    2.213453] hub 2-0:1.0: 1 port detected
[    2.218234] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    2.225776] ALSA device list:
[    2.228763]   No soundcards found.
[    2.241244] Freeing unused kernel memory: 12736K
[    2.264663] Run /init as init process
[    2.388830] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: [email protected]
[    2.612802] usb 1-1: new high-speed USB device number 2 using xhci-mtk
[    2.765579] hub 1-1:1.0: USB hub found
[    2.769531] hub 1-1:1.0: 4 ports detected
[    9.464635] random: crng init done
[   33.673466] /dev/disk/by-label/NIXOS_SD: Can't open blockdev

The error obviously stems from the fact that I am not using partition layout from the original nixos image but rather my modification using atf etc. But that seems to be mostly concern of nixos configuration.

However, I wonder why it worked after switching to that file ? Could you explain what kind of changes have you made there comparing to the original file?

edit:

Have not tested this u-boot for applying overlays, so use this .dtb for now (sdmmc and compatible with direct atf->kernel boot): mt7986a-bananapi-bpi-r3-atf.dtb (23.8 KB)

Maybe that was the reason :slight_smile: wdyt?

See def config of @frank-w

https://github.com/frank-w/BPI-Router-Linux/blob/6.3-main/arch/arm64/configs/mt7986a_bpi-r3_defconfig

And/or mine:

https://aur.archlinux.org/cgit/aur.git/tree/defconfig?h=linux-bpir64-git

It selects which drivers are build into the kernel.

With a glance at your log looks like at least some mtk support build in.

The dtb I shared has overlays applied, so you can try that to start with, not worrying about overlays.

Just set label on your rootfs partition :slight_smile:

@frank-w if that was only that easy :slight_smile: tried that (keep in mind that I merged boot and rootfs partitions for the sake of testing):

NOTICE:  BL2: v2.8(release):v2.8-538-g2f158d215-dirty
NOTICE:  BL2: Built : 11:26:21, May  9 2023
INFO:    BL2: Doing platform setup
NOTICE:  WDT: disabled
NOTICE:  CPU: MT7986 (2000MHz)
NOTICE:  EMI: Using DDR4 settings
NOTICE:  EMI: Detected DRAM size: 2048MB
NOTICE:  EMI: complex R/W mem test passed
INFO:    MediaTek MMC/SD Card Controller ver 20200520, eco 0
ERROR:   Partition 'fip/boot' not found
PANIC at PC : 0x0000000000204bf8

I guess that @ericwoud’s atf requires u-boot partition to be called exactly bpir3-sdmmc-boot. I will try to un-merge boot and rootfs partitions now.

edit: Although it also makes me wonder because the nixos installation disk does not have any labels assigned:

sudo parted /dev/loop1                                                                                               19:02:09
GNU Parted 3.5
Using /dev/loop1
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: Loopback device (loopback)
Disk /dev/loop1: 7340MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      8389kB  39.8MB  31.5MB  primary  fat16
 2      39.8MB  2597MB  2557MB  primary  ext4         boot

(parted)

How could that be?

@ericwoud

The dtb I shared has overlays applied, so you can try that to start with, not worrying about overlays.

That was the result of trying it. The regular one doesn’t work which I think indicates that the u-boot I got from ftp doesn’t support overlays.

I think overlay would work as i have not changed config for it in u-boot. But it is more difficult to get it right and be able to boot. And I have not used it. Worry about it when you have booting ok.

The partlabel indeed needs to be exactly bpir3-sdmmc-boot. You can leave u-boot on boot partition, save all kernel files on rootfs partition. So keep 2 partitions separate. ATF cannot read ext4.

You can checkout partlabels with

lsblk -o name,partlabel

Edit:

But any normal os will have the ability to save extlinux.conf, initrd and Image Image to the boot partition mounted at /boot

Worry about it when you have booting ok.

good idea :slight_smile:

You can checkout partlabels with

Yes, lsblk was the answer. parted only shows labels for gpt partition table while that was msdos partition table, then I had to use dosfslabel to change the label name. After that the system booted without any errors up to the point:

[    2.179931] xhci-mtk 11200000.usb: Host supports USB 3.2 Enhanced SuperSpeed
[    2.187419] hub 1-0:1.0: USB hub found
[    2.191187] hub 1-0:1.0: 2 ports detected
[    2.195463] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    2.203838] hub 2-0:1.0: USB hub found
[    2.207602] hub 2-0:1.0: 1 port detected
[    2.212389] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    2.219940] ALSA device list:
[    2.222925]   No soundcards found.
[    2.235408] Freeing unused kernel memory: 12736K
[    2.256557] Run /init as init process
[    2.381067] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: [email protected]
[    2.616592] usb 1-1: new high-speed USB device number 2 using xhci-mtk
[    2.769458] hub 1-1:1.0: USB hub found
[    2.773457] hub 1-1:1.0: 4 ports detected
[    9.636530] random: crng init done

where it hung. I briefly looked looked that last line over the internet and it might indicate some problem with mounting rootfs. Maybe it tries to mount the same disk for the second time and fail?

Another reason to split them into boot and rootfs :slight_smile: I might also try to increase log level in extlinux.conf

But any normal os will have the ability to save extlinux.conf, initrd and Image Image to the boot partition mounted at /boot

I actually thought about putting all of them in the rootfs. Is there any preference for one over the other?

Whatever works best for you.

Why do you choose for nixos? It is definitely more complicated in some aspects. I’ve chosen Archlinux because of the Rolling updates and the pkgbuild system

Afaik bl2 only looks for gpt partition fip to load uboot from and rootfs should be an ext2/3/4 or similar handling linux filesystem permissions…so not fat