BPI-R2 Pro Ubuntu 22.04 Kernel 5.18 -> 6.1; DSA Ports MAC Blocking

Figuring out steps, please skip unless need back tracing. Used cheap SD card, 32 GB, on an Ubuntu 20.04 laptop, downloaded @frank-w r2-pro-ubuntu2204.tar.gz on Google Drive (May 24, 2022), untarred, then

sudo dd if=...img of=sd card block=4M conv=fsync status=progress

Unplug SD card and insert into slot powered off. Hold Maskrom and apply power to boot into SD. I plan to press the Maskrom every time, so… This image is wonderful, no heavy desktop overhead, the UART serial console is interactive, and the HDMI is presumably outputting another tty. No overzealous NetworkManager trying to claim every port it could find and do configs trying to show it knows better than the user.

I am using an M1 Mac Pro to access serial console. On /dev/tty.usbserial-1130, tried cu, screen, and minicom and finally settled on picocom

picocom -b 1500000 /dev/tty.usbserial-1130

Booted up the board and logged in.

Now my problems and solutions come

  1. try configuring an interface
ip link set dev lan0 up
ip addr add 192.168.88.140/24 dev lan0 (I am configuring Mikrotik too, so hence the 88 default choice)
ip route (shows the /24 subnet entry, OK)
ping 192.168.88.141 (MacOS, no response...)

Wondering what’s wrong, used tcpdump

tcpdump -i lan0 (the ping immediately responds?!)
Ctrl+C stops tcpdump capture and blocks the ping!
dmesg (says lan0 enters promiscuous state..., so need to enable eth1 promiscuous?)

Could this be a MAC address block that the eth1 port is rejecting frames that doesn’t match its MAC address? This behavior is not observed in the official Ubuntu 22.04 (provided by BPi, heavy version), but all interfaces have the same MAC. Try enable promisc on eth1

ip link set dev eth1 down (this brings lan0 down together)
ip link set dev eth1 promisc on
ip link set dev eth1 up (the route and addr are preserved)
ping 192.168.88.141 (now responds!)
  1. Try speed test a DSA port
iperf3 -s -p 5002 (on MacOS)
iperf3 -c 192.168.88.141 -p 5002 -t 300 (this image has iperf3 shipped with, continuous performance at 940 Mbps)
iperf3 -c 192.168.88.141 -p 5002 -t 300 -R (bad performance at 93.9 Mbps, driver problem)

This prompts the kernel upgrade exercise/figuring out stuff.

  1. Try use secure shell, side tracked by service --status-all shows sshd is running
ssh [email protected] (MacOS, yes to recognize fingerprint, default password kept rejecting)
cd /etc/ssh
vi sshd_config
PermitRootLogin yes (this is a bad practice, but in an isolated LAN, whatever, :wq, of course)

Now I could login via ssh with the same serial console username and password.

  1. Upgrade the kernel from 5.18 to 6.1 from Frank’s Github release (CI build 6.1 main 6.1.22, 04-02-2023). My first time dealing with Linux kernel upgrade. Before did some apt guided distro upgrade but probably doesn’t count. Downloaded linux-image-6.1.22…deb, linux-libc-dev and linux-headers 6.1.22…deb, Frank advised only linux-image is necessary, oh well.
scp *.deb [email protected]:/root/ (MacOS, again, might not want to write to the root home folder in a production device)
dpkg -i *.deb (unpacking... takes a long time, maybe I am inpatient, but should complete without errors and nothing seemed to happen after dpkg)
~~ apt -f install ~~ (useless command #1)

Now needs to modify the bootloader (?, excuse me on the terminology)

df (where is the /boot mount point?)
cd /boot (this just magically takes you to the /boot and now /boot could be found in df)
cd extlinux
ls (see the new 6.1.22 Image....gz and new 6.1.22 r2pro.dtb)
vi extlinux.conf
linux <replace with new 6.1.22 Image name>
fdt <replace with new 6.1.22 .dtb>
:wq
reboot (reboot seems to just turn off, to boot up again, you need to press reset while pressing maskrom because I have something in the eMMC I don't want to boot)
uname -r (I got 6.1.22)

Retry 2.

[usual promisc config exercise]
ip addr add 192.168.88.140/24 dev lan0
ip addr add 192.168.1.39/24 dev eth0 (another LAN segment)
iperf3 -s -p 5002 &
iperf3 -s -p 5003 &
iperf3 -c 192.168.88.140 -p 5002 -t 300 (MacOS, continuous performance at 940 Mbps)
iperf3 -c 192.168.1.39 -p 5003 -t 300 (from a second computer, Windows continuous performance at 890 Mbps+, not bad)

I also have an Intel AX210 and ip rule, ip xfrm, and strongswan to get working before I will achieve what I have planned to achieve with this $107 board.

R2pro required changes in mainline dsa driver to work so far (to use p5 as cpu port -it was hardcoded to p6 before). It is possible that some cornercases not yet working.

There are some patches from arinc that extend this behaviour,but you have to apply them by yourself,compile and test with them on 6.4-rc1/net-next

This is his overview:

Patches can be found in mediatek patchwork…there are several series, so i cannot say you the exact one…maybe contact him directly.

Hth frank

Hello Frank, I will look into the MT3751 patching. I haven’t done any source code contribution, the closest one was trying to get a DisplayLink USB 2.0 screen to work on Raspberry Pi 4B+.

Could you advise on how to insert the kernel modules for ip advanced router, policy routing and ip transform?

UPDATE Bit the bullet and compiled the kernel myself.

vi build.conf (board=r2-pro)
./build.sh importconfig
./build.sh config
* IP: advanced router
     * IP: policy routing
* IP: AH transformation
* IP: ESP transformation
* IP: IPComp transformation
* IPv6: AH transformation
* IPv6: ESP transformation
* IPv6: IPComp transformation
* IPv6: Multiple Routing Tables
* Network packet filtering framework
    * Advanced netfilter configuration
         * Netfilter Xtables support
               * IPsec policy match support
* 802.1Q/802.1ad VLAN Support
* Device support
    * network device support
         * Ethernet driver support
               * Realtek devices
                    * Realtek 8169/8125 ethernet support
         * USB Network Adapters
                    * Realtek RTL8152/53
                    * Multipurpose USB Networking framework
                            * ASIX 88xxxx
                            * ASIX AX88179 USB 3.0 to Gigabit
                            * CDC NCM support
         * Wireless LAN
                  * Intel devices
                            * Intel Wireless WiFi Next Gen AGN
                                   * Intel Wireless WIFI MVM Firmware support

Successfully crosscompiled, used option 3 to generate .deb.

dpkg -i .deb

Find the uncompressed image and .dtb in /boot/bananpi/bpi-r2pro/linux, modified extlinux.conf accordingly and got a Pi stucking at Loading Kernel Image

switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:2...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
1:	linux
Retrieving file: /extlinux/../bananapi/bpi-r2pro/linux/uImage_6.1.22_nodt
append: earlycon=uart8250,mmio32,0xfe660000 console=ttyS2,1500000n8 root=/dev/mmcblk0p3 rootwait rw
Retrieving file: /extlinux/../bananapi/bpi-r2pro/linux/dtb/bpi-r2pro-6.1.22.dtb
## Booting kernel from Legacy Image at 02080000 ...
   Image Name:   Linux Kernel 6.1.22
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    30046720 Bytes = 28.7 MiB
   Load Address: 7f000001
   Entry Point:  7f000001
   Verifying Checksum ... OK
## Flattened Device Tree blob at 0a100000
   Booting using the fdt blob at 0xa100000
   Loading Kernel Image

Please advise, thanks.

r2pro don’t use the uImage_nodt…it uses the image.gz (arch/arm64/boot/)…you can use “./build.sh install” to install to sdcard…but this (and compilation process) requires to have board-var correctly set

after changing build.conf, make “./build.sh importconfig” to load the defconfig, then “./build.sh config” to load the menuconfig to activate your options (maybe ./build.sh defconfig to change the defconfig for you, but you have to make sure adding options in right order and add depencies before). if you done with it run build.sh without argument to build…after build is finished, script asks you for install to sd…this calls the install-function which should copy Image.gz and the right dtb to your extlinux folder…then you have to only change the conf

Ok. I find the Image.gz in build/arch/arm64/boot/ and manually scp’ed that to the SD card for the BPI. Modified the extlinux.conf accordingly and booted the machine up again. I could see ip rule and ip xfrm is alive and well. Also, added bunch of device drivers, will test them one by one and report back. None of the Intel wireless drivers loaded successfully for some reason.

If I choose install to sdcard, do I need to compile and run build.sh on the BPI or another computer with better computing power?

What does defconfig stand for? default config? If I have loaded in the defconfig then do config to add in my favorite modules, then save it to .config, is the .config automatically loaded the next time I run ./build.sh config. If I wish to load back the default starting config, I would do ./build.sh defconfig again?

Sorry, my ignorance probably annoys you.

Also, Frank, how did you make that green LED to light up next to the red LED? I believe there is also a blue LED left to the green LED?

Build.sh is mainly for crosscompile on x86. Have not tested build on arm itself for some time. Install is also thought to be run on x86. On r2pro you maybe need mount before install (my ubuntu does not automount boot partition because of bootable flag).

Intel drivers will require firmware to be installed in userspace (like other wifi chips).

defconfig is default config yes…it will open this file for editing. Then run importconfig.

Afair this can be done via sysfs. See my wiki (gpio page for any of my boards) for command

Intel drivers will require firmware to be installed in userspace (like other wifi chips).

Where would I put those .ucode from the the Linux firmware gigantic repository. I only downloaded the iwlwifi-ty-a0-gf-a0-72.ucode and iwlwifi-ty-a0-gf-a0.pnvm and scp’ed them into /lib/firmware.

But still, iwlwifi couldn’t find the downloaded firmware

[    2.504884] Intel(R) Wireless WiFi driver for Linux
[    2.505442] iwlwifi 0001:01:00.0: vgaarb: pci_notify
[    2.506196] iwlwifi 0001:01:00.0: assign IRQ: got 40
[    2.506709] iwlwifi 0001:01:00.0: enabling device (0000 -> 0002)
[    2.507597] iwlwifi 0001:01:00.0: enabling bus mastering
[    2.511136] iwlwifi 0001:01:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-72.ucode failed with error -2

But if I do a manual iwlwifi driver binding after the boot echo pcie-addr > /sys/bus/pci/drivers/iwlwifi/bind, I could get the driver successfully bind to the pcie AX210 card

[   62.506815] iwlwifi 0001:01:00.0: vgaarb: pci_notify
[   62.506992] iwlwifi 0001:01:00.0: assign IRQ: got 40
[   62.507215] iwlwifi 0001:01:00.0: enabling bus mastering
[   62.510886] iwlwifi 0001:01:00.0: vgaarb: pci_notify
[   62.581494] iwlwifi 0001:01:00.0: api flags index 2 larger than supported by driver
[   62.581842] iwlwifi 0001:01:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 0.0.2.36
[   62.586197] iwlwifi 0001:01:00.0: loaded firmware version 72.a764baac.0 ty-a0-gf-a0-72.ucode op_mode iwlmvm
[   62.586398] iwlwifi 0001:01:00.0: Detected Intel(R) Wi-Fi 6 AX210 160MHz, REV=0x420
[   62.587216] thermal thermal_zone2: failed to read out thermal zone (-61)
[   62.788144] iwlwifi 0001:01:00.0: loaded PNVM version e4a49534
[   62.803948] iwlwifi 0001:01:00.0: Detected RF GF, rfid=0x10d000
[   62.872953] iwlwifi 0001:01:00.0: base HW address: 3c:e9:f7:78:23:34
[   62.921592] iwlwifi 0001:01:00.0 wlP1p1s0: renamed from wlan0

And then install wpa_supplicant, write a simple or rather convoluted .conf to connect to a 4 x 4 WIFI 6E on WPA3(!) and I got 904 down 890 up (!!) for 5 minutes and the card was burningly hot.

I discovered /lib/firmware is mirrored with /usr/lib/firmware? How do you achieve this or by design? Again, excuse my ignorance.

I do not have such card,so cannot give exact advice here…maybe it is onown issue…just google for it.

Maybe it runs hot because thermal cannot be read…i would focus on this error

Firmware can be installed in ubuntu via apt,if the right firmware is already packaged…else manual install to /lib/firmware should be right.

I investigated the Direct firmware load -2 error for a bit and found that the firmware blob needs to be included in the kernel when compiling iwlwifi module as included. This will be another round of re-compiling and setting the iwlwifi to <M> instead.