@moore can you please confirm that you also have no traffic on cpu_port0 and working on a fix?
Hi Frank
I’m not sure why the cpu_port0(eth0) doesn’t work, but I think we can check the statisitics of switch port to make sure if traffic isn’t sent out over lan0. If the traffic is not sent out from lan0, that means the problem is caused by mtk eth driver.
And I think we have to ensure the switch tag is enabled on switch side, and each package made by cpu must have the field of switch tag.
Also Ryder said the patch of hardware NAT is ready for reviewing, so can you please wait that version release.
I have currently the gmac-kernel not runnimg because i’m testing some other things in main.
I also assume that the switch-tag is not set for eth0/cpu-port0. How can i see this in code (wgere should it be set)?
for the most people 2nd cpu-port and hdmi is more important than hnat
Ryder said that patches ae for 4.16…i had tried to port wifi-driver to 4.16, but hang at timer-functions where api has changed
Hi Frank
I just guess this problem is related to switch-tag, but we have to check the statistics of all switch ports, and debug the code to see what problem happens.
ok, i try to get the statistics for lan0 on my second R2 with “ethertool -i lan0” while ping my router over it (must revert my routing back to cpu_port0 first)
Linux bpi-r2 4.14.19-bpi-r2-gmac #132 SMP Mon Feb 26 16:23:26 CET 2018 armv7l
[16:42] root@bpi-r2:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defau0
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qle0
link/ether c6:86:c6:05:22:e7 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qle0
link/ether ba:6b:99:cd:5e:20 brd ff:ff:ff:ff:ff:ff
4: wan@eth1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group 0
link/ether ba:6b:99:cd:5e:20 brd ff:ff:ff:ff:ff:ff
5: lan0@eth0: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group0
link/ether c6:86:c6:05:22:e7 brd ff:ff:ff:ff:ff:ff
6: lan1@eth0: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group0
link/ether c6:86:c6:05:22:e7 brd ff:ff:ff:ff:ff:ff
7: lan2@eth0: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group0
link/ether c6:86:c6:05:22:e7 brd ff:ff:ff:ff:ff:ff
8: lan3@eth0: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group0
link/ether c6:86:c6:05:22:e7 brd ff:ff:ff:ff:ff:ff
[16:42] root@bpi-r2:~# ethtool -i lan0
driver: dsa
version:
firmware-version: N/A
bus-info: platform
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
[16:42] root@bpi-r2:~# ethtool -i wan
driver: dsa
version:
firmware-version: N/A
bus-info: platform
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
statistics:
[16:43] root@bpi-r2:~# ethtool -S eth0
NIC statistics:
tx_bytes: 0
tx_packets: 0
tx_skip: 0
tx_collisions: 0
rx_bytes: 0
rx_packets: 0
rx_overflow: 0
rx_fcs_errors: 0
rx_short_errors: 0
rx_long_errors: 0
rx_checksum_errors: 0
rx_flow_control_packets: 0
[16:44] root@bpi-r2:~# ethtool -S eth1
NIC statistics:
tx_bytes: 0
tx_packets: 0
tx_skip: 0
tx_collisions: 0
rx_bytes: 0
rx_packets: 0
rx_overflow: 0
rx_fcs_errors: 0
rx_short_errors: 0
rx_long_errors: 0
rx_checksum_errors: 0
rx_flow_control_packets: 0
p05_TxDrop: 0
p05_TxCrcErr: 0
p05_TxUnicast: 0
p05_TxMulticast: 0
p05_TxBroadcast: 0
p05_TxCollision: 0
p05_TxSingleCollision: 0
p05_TxMultipleCollision: 0
p05_TxDeferred: 0
p05_TxLateCollision: 0
p05_TxExcessiveCollistion: 0
p05_TxPause: 0
p05_TxPktSz64: 0
p05_TxPktSz65To127: 0
p05_TxPktSz128To255: 0
p05_TxPktSz256To511: 0
p05_TxPktSz512To1023: 0
p05_Tx1024ToMax: 0
p05_TxBytes: 0
p05_RxDrop: 0
p05_RxFiltering: 0
p05_RxMulticast: 0
p05_RxBroadcast: 0
p05_RxAlignErr: 0
p05_RxCrcErr: 0
p05_RxUnderSizeErr: 0
p05_RxFragErr: 0
p05_RxOverSzErr: 0
p05_RxJabberErr: 0
p05_RxPause: 0
p05_RxPktSz64: 0
p05_RxPktSz65To127: 0
p05_RxPktSz128To255: 0
p05_RxPktSz256To511: 0
p05_RxPktSz512To1023: 0
p05_RxPktSz1024ToMax: 0
p05_RxBytes: 0
p05_RxCtrlDrop: 0
p05_RxIngressDrop: 0
p05_RxArlDrop: 0
as you see statistics for eth1 (cpu-port1) is more detailed than eth0 (cpu-port0). in eth0 there are p05-sets (port 5 of mt7530??), these entries are missing on eth0 (maybe port 6)
now to lan0:
[16:50] root@bpi-r2:~# ifup lan0
[16:50] root@bpi-r2:~# ifconfig lan0
lan0 Link encap:Ethernet HWaddr 08:00:00:00:00:00
inet addr:192.168.0.10 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::c486:c6ff:fe05:22e7/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:276 errors:0 dropped:4 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:45834 (44.7 KiB) TX bytes:826 (826.0 B)
[16:50] root@bpi-r2:~# ping 192.168.0.5
PING 192.168.0.5 (192.168.0.5) 56(84) bytes of data.
^C
--- 192.168.0.5 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6235ms
[16:50] root@bpi-r2:~# ethtool -S lan0
NIC statistics:
tx_packets: 36
tx_bytes: 3216
rx_packets: 356
rx_bytes: 54746
TxDrop: 0
TxCrcErr: 0
TxUnicast: 28
TxMulticast: 14
TxBroadcast: 1
TxCollision: 0
TxSingleCollision: 0
TxMultipleCollision: 0
TxDeferred: 0
TxLateCollision: 0
TxExcessiveCollistion: 0
TxPause: 0
TxPktSz64: 2
TxPktSz65To127: 37
TxPktSz128To255: 4
TxPktSz256To511: 0
TxPktSz512To1023: 0
Tx1024ToMax: 0
TxBytes: 3957
RxDrop: 0
RxFiltering: 0
RxMulticast: 229
RxBroadcast: 125
RxAlignErr: 0
RxCrcErr: 0
RxUnderSizeErr: 0
RxFragErr: 0
RxOverSzErr: 0
RxJabberErr: 0
RxPause: 0
RxPktSz64: 174
RxPktSz65To127: 38
RxPktSz128To255: 72
RxPktSz256To511: 45
RxPktSz512To1023: 27
RxPktSz1024ToMax: 0
RxBytes: 61154
RxCtrlDrop: 0
RxIngressDrop: 0
RxArlDrop: 0
as i see there are RX and TX bytes…so lan-port itself works, but the internal routing (is assume mt7530 => SOC) does not work
after the pings eth0 has traffic in booth directions but not the same amount
[17:09] root@bpi-r2:~# ethtool -S eth0
NIC statistics:
tx_bytes: 9258
tx_packets: 98
tx_skip: 0
tx_collisions: 0
rx_bytes: 307700
rx_packets: 2036
rx_overflow: 0
rx_fcs_errors: 0
rx_short_errors: 0
rx_long_errors: 0
rx_checksum_errors: 0
rx_flow_control_packets: 0
where does ethtool measure that traffic on eth0 (on soc-side or switch-side)?
some thinkings: soc->switch works and the backway not (packets from switch are not accepted by soc)…ok…did some extra-tests
ping again my router (192.168.0.5):
[root@bananapi ~]# tcpdump -nni eth0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:13:34.104680 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 1, length 64
17:13:35.150095 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 2, length 64
17:13:36.190053 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 3, length 64
17:13:37.230044 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 4, length 64
17:13:38.270038 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 5, length 64
17:13:39.310056 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 6, length 64
17:13:40.350044 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 7, length 64
17:13:41.390038 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 8, length 64
17:13:42.430047 IP truncated-ip - 6 bytes missing! 192.168.0.10 > 192.168.0.5: ICMP echo request, id 815, seq 9, length 64
packet goes out, as assumed, and of course will be answered by my (old) router. for same on r2 i need to install tcpdump (have to temporary boot with working kernel, install and again with gmac-kernel, have done this with ethtool before).
strange is the “truncated-ip”-information…
because i cannot use ssh (no connection to lan) i tcpdumped to file
[17:29] root@bpi-r2:~# tcpdump -nni lan0 icmp > lan0ping.log &
[17:30] root@bpi-r2:~# tcpdump -nni eth0 icmp > eth0ping.log &
pinged again, see incoming pings on my (old) router, but booth files on R2 stay empty 
after killing tcpdump the lan0-log contains the outgoing pings:
[17:34] root@bpi-r2:~# tailf lan0ping.log
17:31:34.745963 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 4,
length 64
17:31:35.785962 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 5,
length 64
17:31:36.825984 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 6,
length 64
17:31:37.865970 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 7,
length 64
17:31:38.905964 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 8,
length 64
17:31:39.945963 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 9,
length 64
17:31:40.985961 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 10
, length 64
17:31:42.025960 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 11
, length 64
17:31:43.065960 IP 192.168.0.10 > 192.168.0.5: ICMP echo request, id 791, seq 12
, length 64Hardware NAT should be still working on current DSA configuration with single GMAC.
I think Software HNAT can work. and also Hardware NAT has to work!
You could add HWNAT on the top of current DSA configuration at first if you just want to have a verification for Hardware NAT.
Otherwise, you will meet tons of frustration during you’re integrating second GMAC into existing DSA framework.
This is still hard work for general people in specifically you don’t have deep thoughts on the DSA core.
Please don’t mix hnat and second gmac…this thread is for second gmac. hnat have separate thread where no traffic on wan is reported if using patched mtk_eth_soc.c
Yes, I knew and agreed it’s good to split these topics into two threads. So I just replied my thoughts to above message.
Actual i had only applied gmac-patches from lede made for 4.14 without modify anything,just testing. I had thought that they should work,but they did not so i do some debug see here:
@garywang is this information enough or do you need additional infos?
tested tcpdump on my router again with a working ping:
[root@bananapi ~]# tcpdump -nni eth0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:02:16.781003 IP 192.168.0.21 > 192.168.0.5: ICMP echo request, id 20647, seq 1, length 64
18:02:16.781234 IP 192.168.0.5 > 192.168.0.21: ICMP echo reply, id 20647, seq 1, length 64
18:02:17.781727 IP 192.168.0.21 > 192.168.0.5: ICMP echo request, id 20647, seq 2, length 64
18:02:17.781913 IP 192.168.0.5 > 192.168.0.21: ICMP echo reply, id 20647, seq 2, length 64
18:02:18.781675 IP 192.168.0.21 > 192.168.0.5: ICMP echo request, id 20647, seq 3, length 64
18:02:18.781869 IP 192.168.0.5 > 192.168.0.21: ICMP echo reply, id 20647, seq 3, length 64
18:02:19.781757 IP 192.168.0.21 > 192.168.0.5: ICMP echo request, id 20647, seq 4, length 64
18:02:19.781967 IP 192.168.0.5 > 192.168.0.21: ICMP echo reply, id 20647, seq 4, length 64
18:02:20.781712 IP 192.168.0.21 > 192.168.0.5: ICMP echo request, id 20647, seq 5, length 64
18:02:20.781912 IP 192.168.0.5 > 192.168.0.21: ICMP echo reply, id 20647, seq 5, length 64
as you can see with same command now there are outgoing icmp echo reply’s…so i assume that the problem is that the outgoing packets from gmac-kernel are getting damaged (truncated-ip - 6 bytes missing!) so my current router does not send a reply…
contacted John crispin (author from the patches on lede), hoping for a reply
added task to lede-project: https://bugs.lede-project.org/index.php?do=details&task_id=1403
No answer till now…has anybody contact to lede-team or john crispin and give him a hint for that?
i have contacted john via this email…also the lede-bug-report is not confirmed yet
Thanks for the patch. I am curious: why add second gmac? If MT7632 connect switch with 2 RGMII, the bandwidth will be doubled? How to test this kind of performance boost?
i will use that for LAN/WAN-separation…if wan is connected to one and LAN to the other you can test it with traffic wan-lan 
To confirm my understand on kernel 4.14 to enable native/hardware switch/bridge we need 2gmac as available in hardware. From what i can see in the forum it is not functional yet. How can i test it?
if the lan-port-separation is disabled anyhow you need a lan/wan-separation, so you need the 2nd gmac between switch (mt7530) and cpu (mt7623)…
the gmacs are recognized in debian based systems as ethx…currently you have only eth0…with the lede-patches you’ll see also eth1, but here eth0 will not work because outgoing frames are damaged
if the second gmac works it may be possible to bridge the lan-ports in the switch, not via software in CPU.
currently you can bridge the ports with bridge-utils, but imho this is done in CPU. so if Traffic goes from LAN0 to LAN1 (assuming ports are bridged via br0) this way:
LAN0 => SWITCH => ETH0 => CPU (br0) => ETH0 => SWITCH => LAN1
this means, if you you have full bi-directional traffic on these ports you will get only the half speed
@Ryder.Lee @garywang @moore @linkerosa please correct me if i’m wrong…and can you please contact john to look at the lede-4.14-patches for 2nd gmac
Thanks for the clarification, therefor i should not use the DSA code and ensure the 2Gmac to get full speed.
Regarding speed, it should be noted that GMAC1 supports ‘Turbo-RGMII’ mode (trgmii), running at 2.5 Gbps instead of the usual 1Gbps. In theory this should leave enough bandwidth for two bidirectional gig-e links running simultaneously over a single GMAC.