It uses 4.14
@frank-w Frank, trying to add the rgmii patch to compile from master on Openwrt I get a Hunk error:
Applying /tmp/openwrt/target/linux/mediatek/patches-5.4/0402-net-ethernet-mtk_eth_soc-Always-call-mtk_gmac0_rgmii.patch using plaintext: patching file drivers/net/ethernet/mediatek/mtk_eth_soc.c Hunk #1 FAILED at 171. Hunk #2 FAILED at 262. 2 out of 2 hunks FAILED ā saving rejects to file drivers/net/ethernet/mediatek/mtk_eth_soc.c.rej
5.4.58 already contains trgmii patch
good catch @frank-w, the Hunk errors make perfect sense thenā¦ lol.
So, bad news: the rgmii patch does not solve speed issue on Iperf using BPI-R2 on Openwrt Master Kernel 5.4.58. Getting around 400-450 Mbps tops. Activating Software offloading does not improve speed, although it improves cpu usage, which was already not bad.
Seeing a shitton of retransmissions too. Donāt see those with the NanoPi R2Sā¦ maybe BPI-R2 is sending TCP packets out of order?
Do you have direct connection or via switch? Maybe itās an flow control issueā¦
With my 5.4 and debian i got 940mbit/s without any offloadingā¦and default settings in iperf. One āproblemā in openwrt is upporting net patches,so it is more comparable to 5.7/5.8
Maybe check reverse direction with -R
Ok, answers first: I do have a switch from PFsense to BPI-R2. Although there is no issue with the NanoPI R2s in the same setup.
Issue looks real weird, so I decided to try a few more tests, with some interesting results. Still donāt know what is causing the issue, though.
Original test was: PFSense WAN (Iperf Client) -> BPI-R2 -> Win10 Desktop (Iperf Server)
- Results are the same using Reverse Direction (around 400-450Mbps).
New test - Simplified setup:
- PFSense WAN (Iperf Server) -> BPI-R2 (Iperf Client)
Normal Direction (BPI-R2 to PFSense) Result: Same Issue
New test - Simplified setup - Reverse Order
- PFSense WAN (Iperf Server) -> BPI-R2 (Iperf Client)
Reverse Direction (PFSense to BPI-R2) Result: No Issues, Speed as expected
In the first setup you use routing or switching between wan and lanX? Here you can look at cpu utilization and interrupts (but i donāt know where here is critical marker).all ports are connected to gmac0 which is finally trgmii and should have 2,5gbit/s throughput.so it should be enough for 2x1gbit (iperf works unidirectional so traffic is basicly is 1gb bidirectionalā¦wan in - lan out,and reversed)
For the second test it looks like a flowcontrol issue (r2 is sending) where your switch or pfsense do not send (or block towards r2) flowcontrol packets so that in this direction packets got lost and need to be retransmitted. Make sure your switchports are configured both with flowcontrol enabled and also your pfsense box.
You can look in both ends with ethtool -S interface if there are dropped packets and or flowcontrol packets
Nice @frank-w, thanks for the attention! This is not my main device either at work or home, so Iām just trying to figure it out and maybe help others who face the same issue.
I agree with you, gmac0 should be able to forward this traffic without issues, and it looks to me as if it does not have a performance issue. So, hereās what wireshark at the Windows desktop told me.
- Apparently the packets are NOT arriving out of order
- Every once in a while (about 10 times per second) a random packet gets lost (wasnāt dropped in windows) so TCP will start doing some SACKs and after the 3rd time Activate fast retransmission.
OK, so I decided to investigate even further, as it looked like the packets were dropping before arriving at the test desktop.
Got TCPDUMP on Openwrt to see what was happening and made a double wireshark: One running on BPI-R2 LAN bridge (Openwrt br-lan) and another one on Win10 Desktop, during the same iperf test.
Hereās how that went: BPI-R2 TcpDump - All packets right and in order
Win10 Wireshark - Random packets donāt arrive
As you can see on both images, the Segment with ACK 261341:
- arrived OK from PFsense on the WAN port;
- Was forwarded to LAN Bridge (I was capturing br-lan on BPI-R2);
- Got captured being sent normally to Desktop (192.168.1.206)
- Never arrived to Desktop, prompting a series of 3-Selective ACKS and a fast retransmission a few packets later.
- Disabled the br-Lan Bridge and connected directly to eth0 LAN, same results;
- Cable problem is discarded (NanoPi R2S with the setup, cable and connected to the same desktop works fine)
Still no luck trying to figure it out.
Do you use routing? Can you make sure wan and lans having different mac address? They inherite from gmac0 so they are same at boottime. It should not disturb because wan and lan are different segments and routing is layer3,but just to test thisā¦how about cpu-load/interrupts while routing from wan to lan?
hey @frank-w! Yes, I do use simple routing and per ifconfig WAN and LAN have different MAC Addressesā¦
Continuing troubleshooting I seem to find something interesting (finally!): Everytime I run Iperf and it gets retransmissions (every scenario, except BPI-R2 to PFsense, using reverse) two counters on ethtool will increase on eth0 interface:
- p06_RxCrcErr
- p06_RxIngressDrop
The same errors DO NOT appear on mdio bus ports (wan, lan0, etc), only on eth0 (frame engine)
The Crc error usually indicates a Layer1 problem, so probably Cable, NIC or Driver? But the cable is the same I use with NanoPI R2s with no errors.
Creation of correct sd-card is tricky. I wasted some hours today for this job. You have to chain preloader, u-boot, kernel (so far simple) and rootfs together in the image. I failed at calculation of offset for the rootfs. At least I could change a lexa2 sdcard image to emmc and successfully write to mmc.
After I found mmc related options in make menuconfig, I could successfully write to mmcblk0. So far so good, but Openwrt 19.07.3 cannot boot because I donĀ“t know how to chain the imageā¦
hey @gotthard.anger the support on 19.07.3 is indeed very tricky.
I suggest you build from Openwrt master or use the snapshot images - just note that they donāt have Web-GUI Luci by default , as master has some improvements were made by a lot of good people like @frank-w and @dwmw2 and a few others.
A small guide on Installing Snapshot Master Openwrt to BPI-R2 EMMC would be:
- Go to https://downloads.openwrt.org/snapshots/targets/mediatek/mt7623/
- Download 3 files: a) bpi_bananapi-r2-scatter.txt , b) bpi_bananapi-r2-squashfs-img.gz and c) bpi_bananapi-r2-preloader.bin
- Follow the instructions on scatter.txt (1st link above), GUnzip the IMG file to burn it to e-MMC. Youāre going to need a usb OTG cable and the SP Flash Tool.
If you did everything correctly it will boot from e-MMC with Openwrt defaults. HDMI and Wifi donāt work yet, but the rest works fine, except for the weird speed issues Iām seeing (see posts above).
Regards,
Rod
Where does otg work? I tried getting otg working with additional drivers in linux 5.4,but canāt get it working
Flashing does also work if booted from sdcard by using dd in running system (writing the image not copy running system). For emmc hw-partitions have to be set (mmc partconf in uboot or with mmc-utils from linux) and preloader needs to be flashed to boot0 block
OTG does work for flashing OpenWRT+Preloader, @frank-w. I did it on my bpi-R2.
This link has the instructions: https://downloads.openwrt.org/snapshots/targets/mediatek/mt7623/openwrt-mediatek-mt7623-bpi_bananapi-r2-scatter.txt
Where does otg work? Uboot or linux?
Can you or @dwmw2 point me to the otg settings (dts,driver used)? I see no otg patch, and did not found out where uboot is built
It uses the MediaTek SP Flash Tool.
See the instructions I put in the top of the scatter file:
# OpenWrt eMMC scatter file for Bpi Banana Pi R2
# For use with SP Flash Tool: https://spflashtool.com/download/
#
# Unzip the file system file openwrt-mediatek-mt7623-bpi_bananapi-r2-squashfs-img.gz before flashing.
# Connect the device's USB port, although it won't appear to the host yet.
# Load this scatter file into the SP Flash Tool, select 'Format All + Download'
# After pressing the 'Download' button, power on the board.
# The /dev/ttyACM0 device should appear on USB and the tool should find it.If you boot OpenWrt from SD card, thereās a U-Boot menu option which does all that for you.
Hi, i read the scatter file,but it does not explain in which bootstate (uboot,ā¦) r2 is while using spitool over otg. R2 needs to configure otg port for spitool to connect,right? 
If it is uboot i need position in the sources where otg is defined/initialized have not found where in openwrt uboot is built (and from which sources/patches).
This is done by the MT7623 boot ROM; you can flash it with the SP Flash Tool when it is entirely bricked and there is nothing on the eMMC (or SD).
As you power it up, it appears to the host as a serial port briefly, until it actually finds something to start the preloader from and disappears again.
OpenWrtās U-Boot is in package/boot/uboot-mediatek
1 Like