SFP Direct Attach

Is there any chance of an SFP direct attach working on the Omnia? My CRS305 has the option to advertise only 2.5g, but neither the CRS nor the Omnia acknowledges a link with any combination of advertisement speeds or disabling auto-negotiation. I’m guessing 2.5g/5g over SFP direct attach just isn’t particularly widely supported?

I did some research on this topic many months ago to find out there is no DAC available on the market. Tested several SFP+ -cables without any luck.
The TO SFP-port needs modules with exactly 2,5 GBE, so the firmware of the module needs to have that enabled.
Only relevant use cases for the port are either using the Turris SFP+ -module (SFP+ to 2,5 GBE) or a 1 GBE fibre-internet :man_shrugging: I do the former with 1 MOX and 2 TOs.

Are you sure that the autonegotiation speeds in the Ethernet tab are related to SFP?

My understanding is, that these autonegotiation settings are applied to a transceiver inserted into the SFP+ port. So if you insert 2.5/5 gbit capable transceiver, it will communicate with switch at 10 gbit, but over the metallic link it will advertise whatever you’ve set up there.

SFP/SFP+ themselves do not have autonegotiation. When I needed to connect SFP to SFP+ via DAC, I had to turn off the autonegotiation and set the link speed manually.

(I have a computer connected to CRS305 via DAC. The auto negotiation is on and advertisment is set to 10/100/1000 mbps, and it is running at 10 gbit anyway).

That’s the part I’m not sure about. Some sources seem to imply that the ASIC on the switch/NIC itself needs support for 2.5/5, but I suspect that may be related to flow control so you don’t overload a 2.5gb line with 10gb worth of traffic.

Also, what’s strange is that ethtool at least sort of recognizes that something has been plugged in, and the port LED on the CRS briefly flashes on.

Before:

    Supported ports: [ MII ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Supported pause frame use: Symmetric
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Speed: 10Mb/s
    Duplex: Half
    Port: MII
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    Supports Wake-on: d
    Wake-on: d
    Link detected: no

After:

    Supported ports: [ ]
    Supported link modes:   2500baseX/Full
                            1000baseX/Full
    Supported pause frame use: Symmetric
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  2500baseX/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Speed: 2500Mb/s
    Duplex: Full
    Port: Direct Attach Copper
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    Supports Wake-on: d
    Wake-on: d
    Link detected: no

In dmesg:

[ 3218.085910] sfp sfp: module OEM              SFP-H10GB-CU3M   rev R    sn CSC211004230130  dc 211014
[ 3218.095285] mvneta f1034000.ethernet eth2: interfaces=[mac=4,8-11,16,18-19 sfp=18-19,22] selected 19 (2500base-x)
[ 3218.105583] mvneta f1034000.ethernet eth2: switched to inband/2500base-x link mode

If I force both sides to 1gb (autoneg off), it actually does work. But…I already had a 1gb link, so not much use there.

I got it working!

First off, use RouterOS, not SwitchOS. Upgrade to at least 7.3.1, which added the option to force a 2.5Gbps SFP.

On the Omnia side (not persistent - put it in an init script):

ethtool -s eth2 speed 2500 autoneg off

On the Mikrotik side:

Interfaces → whichever SFP+ port you’re going to use → Ethernet → Auto Negotiation off, Speed 2.5Gbps, Full Duplex

(Unsure if flow control should be on or off)

That should be it.

# iperf3 -c 192.168.1.1
Connecting to host 192.168.1.1, port 5201
[  5] local 192.168.1.4 port 43936 connected to 192.168.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   282 MBytes  2.36 Gbits/sec    0    814 KBytes
[  5]   1.00-2.00   sec   279 MBytes  2.33 Gbits/sec    0    947 KBytes
[  5]   2.00-3.00   sec   279 MBytes  2.35 Gbits/sec    0    947 KBytes
[  5]   3.00-4.00   sec   278 MBytes  2.33 Gbits/sec    0   1.06 MBytes
[  5]   4.00-5.00   sec   278 MBytes  2.33 Gbits/sec    0   1.11 MBytes
[  5]   5.00-6.00   sec   280 MBytes  2.35 Gbits/sec    0   1.11 MBytes
[  5]   6.00-7.00   sec   275 MBytes  2.31 Gbits/sec    0   1.22 MBytes
[  5]   7.00-8.00   sec   276 MBytes  2.32 Gbits/sec    0   1.30 MBytes
[  5]   8.00-9.00   sec   280 MBytes  2.35 Gbits/sec    0   1.30 MBytes
[  5]   9.00-10.00  sec   275 MBytes  2.31 Gbits/sec    0   1.30 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.72 GBytes  2.33 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec                  receiver

iperf Done.
4 Likes

Good work & great news! I have Mikrotik CRS309 & CRS305 along with a QNAP QSW-2104-2S and I’m looking to integrate my Omnia as an AX 4x4 access point. What Turris OS version are you running mattventura (in case it may make a difference)?

SFP DA cables are great when viable options with low cost & low power draw/heat generation! Downsides are cable rigidity and length of run limits to about 10 meters without going active. Supposedly there are optical DACs for longer runs but I know nothing about them.

I’m currently using the Mikrotiks in SwitchOS mode since I have an ISP required router in play. But I’d assume I’ll be able to run RouterOS with minimal actual routing being done so they don’t conflict. I’ll be taking my time with this though!

Thanks for your report on this!

I’m running 6.0 on HBL branch.

Great, Thanks! Me too!