BPI-R3 isolation wan and lan eth

Hy

News about my problem ^^

About the SFP rj45, the mine doesn’t work with Bananapi r3.

It doesn’t matter, i will take another.

it’s the only one sfp which has an error message. I tested with sfp gpon onu, and 4 sfp lc. They return no error message.

For my sfp gpon onu. In the sfp wan, the network doesn’t work. Even if it is recognized ( see with ethtool and ethtool -m)

But when i used it on the sfp lan(sfp cage close to rj45),it works if :

i create a bridge and put the sfp into. so i just put the sfp into. But the big problem is the debit.

I have a connexion of 2gpbs down and 800mb up. My sfp is a 2gbps, tested in antoher sbc.

I done a speedtest and i have :

 Server: ORANGE FRANCE - Marseille (id = 29545)
    ISP: Orange
Latency:     4.43 ms   (0.11 ms jitter)
   Download:    63.72 Mbps (data used: 122.5 MB)
 Upload:     0.66 Mbps (data used: 880.4 kB)
Packet Loss:     3.3%

Here my interfaces config :

iface eth0 inet static
        address 192.168.1.1
        netmask 255.255.255.0

iface br0 inet static
  address 192.168.1.2
  netmask 255.255.255.0

iface lan4 inet manual
  post-up /sbin/ip l s up dev $IFACE


iface orange inet manual
  #Creer le vlan 832 et les priorités
  pre-up /sbin/ip l a link lan4 name $IFACE type vlan id 832 egress-qos-map 0:6

If i don’t give an address to eth0, nothing work.

I tried to create a bridge wan with sfp wan and eth wan. But same result. Have you 2 bridges : 1 for lan and 1 for wan?

My trees had some wrong gpio definitions (properties need to be plural - so “gpios” and mod-def0 needs additional dash) have already fixed on 6.1 tree. My sfp worked with these wrong properties (so i had not noticed it),but maybe other sfp do not.

I currently have no bridge…woeking directly on interfaces,but can only test 1g (iperf3.to my laptop or my main router which looked good for both sfp and rj45 ports before and after russels patch).

I think i have found the problem, not sur ^^

In my last SBC, i use iptables for qos

# Generated by iptables-save v1.4.21 on Wed Feb 20 23:25:13 2019
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o orange -j MASQUERADE
COMMIT
# Completed on Wed Feb 20 23:25:13 2019
# Generated by iptables-save v1.4.21 on Wed Feb 20 23:25:13 2019
*mangle
:PREROUTING ACCEPT [604:477856]
:INPUT ACCEPT [589:476608]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [487:83466]
:POSTROUTING ACCEPT [487:83466]
-A POSTROUTING -o orange -j CLASSIFY --set-class 0000:0001
-A POSTROUTING -o orange -p igmp -j CLASSIFY --set-class 0000:0006
-A POSTROUTING -o orange -p icmp -j CLASSIFY --set-class 0000:0006
-A POSTROUTING -o orange -m dscp --dscp 0x2e -j CLASSIFY --set-class 0000:0005
-A POSTROUTING -o orange -p udp --dport 67 -j CLASSIFY --set-class 0000:0006

But when i apply this rules i have an error :

iptables v1.8.5 (legacy): unknown option "--set-class"
Try `iptables -h' or 'iptables --help' for more information.

Found :slight_smile:

I have forgotten some options in my Kernel for the netfilter part.

with the good iptables rules :

   Speedtest by Ookla

     Server: ORANGE FRANCE - Marseille (id = 29545)
        ISP: Orange
    Latency:     3.92 ms   (0.15 ms jitter)
   Download:  2202.73 Mbps (data used: 1.1 GB)
     Upload:   808.81 Mbps (data used: 394.7 MB)
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/3a29041b-5fb1-4e64-a6aa-8959e12a1cdd
#

stay only one little problem :
the sfp wan cage, why the network doesn’t pass.

If above config is still active you have still ip set on eth0…

Wan-sfp is eth1 and needs different ip subnet than your lan-bridge

I’ve seen this error with FINISAR 1000Base-T SFP module. Turns out it needs Marvell PHY driver to function, which I have now packaged in OpenWrt:

Try enabling kmod-phy-marvell in your build. If that still doesn’t help, please install mdio-tools and show the output of mdio *sfp2 (assuming the SFP is inserted to sfp2 cage).

If you are using vanilla Linux (e.g. build e.g. from Frank’s tree) make sure you enable CONFIG_MARVELL_PHY in your build.

1 Like

Hy, that the solution for my sfp rj45. Thank you a lot

Hy :slight_smile:

So i tried like you, no bridge for wan.

Set configuration into interfaces files Example for interface wan in dhcp mode :slight_smile:

iface wan inet dhcp

Results:

For ethernet wan : OK

For sfp lan (next to rj45) : OK tested with sfp rj45 (dhcp) and sfp gpon onu

For sfp wan:

# ifup eth1
[  141.296441] mtk_soc_eth 15100000.ethernet eth1: configuring for inband/2500base-x link mode
udhcpc: started, v1.32.0
[  141.365490] mtk_soc_eth 15100000.ethernet eth1: switched to inband/sgmii link mode
udhcpc: sending discover
[  141.542412] mtk_soc_eth 15100000.ethernet eth1: PHY [i2c:sfp-1:16] driver [Marvell 88E1111] (irq=POLL)
udhcpc: sending discover
udhcpc: sending discover
udhcpc: no lease, failing
# ifup eth1
udhcpc: started, v1.32.0
udhcpc: sending discover
udhcpc: sending discover
udhcpc: sending discover
udhcpc: no lease, failing

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1504 qdisc fq_codel state UP group default qlen 1000
    link/ether 8a:36:4f:f9:8c:b0 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 scope global eth0
       valid_lft forever preferred_lft forever
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 00:0a:35:00:00:01 brd ff:ff:ff:ff:ff:ff
4: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
5: wan@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
    link/ether 00:0a:35:00:00:01 brd ff:ff:ff:ff:ff:ff

Interface ifles :

iface eth1 inet dhcp

Hy :slight_smile: I do the command ethtool for the two sfp and i have a difference in the result :

Current message level: 0x000000ff (255)
                       drv probe link timer ifdown ifup rx_err tx_err

it appears for sfp wan ( which doesn’t work). Is it an error?

ethtool eth1 
        Supported ports: [ TP    MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
                                1000baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
                                1000baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Half 1000baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Auto-negotiation: on
        Port: Twisted Pair
        PHYAD: 22
        Transceiver: external
        MDI-X: Unknown
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: no



ethtool lan4
       Supported ports: [ TP    MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                1000baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                1000baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 10Mb/s
        Duplex: Half
        Auto-negotiation: off
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        MDI-X: Unknown
        Supports Wake-on: d
        Wake-on: d
        Link detected: yes


ethtool -i lan4
driver: dsa
version: 6.1.0-rc1-diagbox
firmware-version: N/A
expansion-rom-version:
bus-info: platform
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

# ethtool -i eth1
driver: mtk_soc_eth
version: 6.1.0-rc1-diagbox
firmware-version:
expansion-rom-version:
bus-info: 15100000.ethernet
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

Have you applied the fix for sfp1 properties?

Your autoneg state,speed,duplex for lan-sfp is wrong…

Hy,

Yes i have the patch applied.

For the lsfp lan output, i token the wroong one, without sfp into.

Here the good one, with sfp rj45 :slight_smile:

Settings for lan4:
        Supported ports: [ TP    MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                1000baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                1000baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Half 1000baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Auto-negotiation: on
        Port: Twisted Pair
        PHYAD: 22
        Transceiver: external
        MDI-X: Unknown
        Supports Wake-on: d
        Wake-on: d
        Link detected: yes

I i have good understood, every itnerfaces are conencted to mt7531ae except the sfp wan which is directly to mt7986a,isn’t it?

mt7986 have 2 gmac (one to switch-chip, one to wan-sfp)

https://www.fw-web.de/dokuwiki/doku.php?id=en:bpi-r3:start#network_connections

I tried to put a static address to sfp wan

Here the interface config files :

iface wan inet static <== ETH WAN address 192.168.1.100 netmask 255.255.255.0 gateway 192.168.1.1

iface eth1 inet static <== SFP WAN address 192.168.1.101 netmask 255.255.255.0 gateway 192.168.1.1

iface lan4 inet static <== SFP LAN address 192.168.1.102 netmask 255.255.255.0 gateway 192.168.1.1

For the sfp lan and eth wan, it works. I can ping the gateway and the web (8.8.8.8)

But when i ifup the sfp wan (eth1), i have an error :

ifup eth1

Error: either “local” is duplicate, or “/24” is a garbage. [ 1155.033048] mtk_soc_eth 15100000.ethernet eth1: configuring for inband/sgmii link mode Error: Nexthop has invalid gateway.

[ 1155.272390] mtk_soc_eth 15100000.ethernet eth1: PHY [i2c:sfp-1:16] driver [Marvell 88E1111] (irq=POLL)

Yes because your device does not know where to send out the paket (eth1 or your lan)…every interface needs an own subnet configured…if multiple interfaces need same subnet you have to use a bridge. Everything else is not clean setup. And your current setup is a security mess…lan and wan in same subnet

I forgotten to Say that i ifup one interface at a Time, ro test. Lan will be in a bridge(4 RJ45,sfp lan and 2 wlan).

then you have to set the bridge down (where you have set the ip address), not the lan-interfaces.

how about you ethernet-wan-port which is also same subnet?

and you have an syntax-error somewhere for the eth1 port…

Error: either “local” is duplicate, or “/24” is a garbage.

so please post full config

Hy :slight_smile:

At the boot, i have a script which create the bridge and up all the lan.

brctl addbr br0
brctl addif br0 lan0
brctl addif br0 lan1
brctl addif br0 lan2
brctl addif br0 lan3
brctl addif br0 lan3

ifup lan0
ifup lan1
ifup lan2
ifup lan3
ifup lan4
ifup br0
ifup eth0

Here my full interface conf :

> auto lo
> iface lo inet loopback
> 
> #Lan
> iface br0 inet static
>   address 10.0.0.2
>   netmask 255.255.255.0
> 
> iface eth0 inet manual
>   hwaddress ether 00:00:22:33:44:50
>   
> iface lan0 inet manual
>   hwaddress ether 00:00:22:33:44:50
>   
> iface lan1 inet manual
>   hwaddress ether 00:00:22:33:44:50
>   
> iface lan2 inet manual
>   hwaddress ether 00:00:22:33:44:50
>   
> iface lan3 inet manual
>   hwaddress ether 00:00:22:33:44:50
>   
> iface lan4 inet manual
>   hwaddress ether 00:00:22:33:44:50
> 
>   #Wan
> iface wan inet manual
> 
> iface eth1 inet static
>   address 192.168.1.101
>   netmask 255.255.255.0
>   gateway 192.168.1.1

And i wake up the wan manually :

ifup eth1

lan4 = sfp lan eth1 = sfp wan wan = eth wan

interface file looks right, but somewhere you have “local” and “/24” which causes error above when eth1 is set to up (if-up-script?)

I have deleted and re created my interfaces conf.

Error doesn’t appear :slight_smile:

when i ifup wan i have :

ifup wan

[ 60.167307] mtk_soc_eth 15100000.ethernet eth0: configuring for fixed/2500base-x link mode [ 60.175676] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 2.5Gbps/Full - flow control rx/tx [ 60.176391] mt7530 mdio-bus:1f wan: configuring for phy/gmii link mode [ 60.195650] mt7530 mdio-bus:1f wan: Link is Up - 1Gbps/Full - flow control off

And when i ifup sfp wan

ifup eth1

[ 99.766208] mtk_soc_eth 15100000.ethernet eth1: configuring for inband/2500base-x link mode udhcpc: started, v1.32.0 [ 99.835614] mtk_soc_eth 15100000.ethernet eth1: switched to inband/sgmii link mode udhcpc: sending discover [ 100.022419] mtk_soc_eth 15100000.ethernet eth1: PHY [i2c:sfp-1:16] driver [Marvell 88E1111] (irq=POLL)

driver parsed is okay?

One seperate question :

Can we use the ethernet wan with 2.5gbps, even if we must damn others rj45? Or it is blocked to 1gpbs