Integrating Hardware support into LEDE/OpenWrt by the community

JamesT42 · May 10, 2017, 3:55pm

Dear Turris Omnia community!

As the devs are only interested in their fork “TurrisOS” it seems to be left to the community to get basic LEDE/OpenWrt support going. These are the previous attempts at this that I’m aware off right now.

If there are any other works into this, please come forward.

I’m definitely willing to test anything, but my programming skills are not much beyond “Hello world”. My ultimate dream is to just download the next LEDE release, put it on a USB stick, put it into my Omnia and go from there. No DNS betatesting, no autoupdates destroying my config etc.

Cheers,
James

P.S.: As LEDE and OpenWrt are remerging somehow in the future, we also have to see how to manage that.

danrl · May 10, 2017, 4:04pm

I am a LEDE contributor and can help when it comes to packaging and testing or adopting LuCI integrations.

neheb · May 13, 2017, 2:08am

They’re planning on re-merging. Newer OpenWRT tree will just use LEDE as a base and deprecate the OpenWRT git.

No idea why Turris Omnia support has not been integrated to LEDE yet.

iddqd · May 13, 2017, 9:02am

I think it has to do with Project Turris. If you don’t have control over the underlying base, changes to said base could negative negatively influence it, especially the distributed adaptive firewall. Also, I doubt that either the LEDE or the OpenWRT will welcome Project Turris in their work.

andrewsh · May 14, 2017, 5:15pm

Also, I doubt that either the LEDE or the OpenWRT will welcome Project Turris in their work.

What makes you think so? I’m certain OpenWRT/LEDE will be happy to support another piece of hardware.

rotanid · May 14, 2017, 8:31pm

i doubt he meant the hardware, but the project with the distributed firewall etc.

dlang · May 17, 2017, 11:05am

If we can get the hardware support in, we can worry about the fancy firewall features later.

It will make it easier for the Turris folks to ship a newer system if they don’t have to do all the work of porting the hardware support, so it will even help those who do want the fancy firewall features.

I think the first step is to make a list of what’s different. What drivers are there that are in Turris and not LEDE, make a branch of LEDE for each of the drivers that only includes the code needed for that driver.

Then we can work with the upsteam folks (LEDE or linux-kernel) to get the driver into shape to be acceptable for upstream.

We may find that in some cases, there are upstream changes that support the hardware already and we can drop the turris specific version of the code with minimal difficulty.

But the first step is to get our hands around the problem and find out what the differences are.

dlang · May 17, 2017, 11:12am

It looks like heinzek is still working on this. I’ve pinged him on his thread to find out what the current status is (his last post was from Feb

ryan · June 1, 2017, 2:52pm

As there has not been any further movement on this, I am giving it a go. I have started by re-basing Xiche’s commits on LEDE master (4.9 kernel), then removing some of the bits to build a btrfs image and instead just creating a medkit image as per heinzek’s fork. I haven’t properly reviewed the code so far, but I am able to build a LEDE medkit image that boots on my Omnia. In fact, I am currently accessing this forum via my Omnia running LEDE

My branch is here: https://github.com/rmounce/lede-source/tree/omnia

I like the approach of only distributing the medkit image, leveraging the Omnia’s built-in recovery to handle btrfs formatting. After a quick look, it should also be possible to update or write another wrapper for sysupgrade so that a single image can be used for both the initial flash from USB and in-place upgrades. btrfs snapshots should also be able to facilitate failsafe mode/factory reset as they do on Turris OS, but this may be a bit more work to adapt on LEDE.

My immediate goal is to distill everything down to a minimal device support patch that I can submit to LEDE for review. Fancier stuff can come later, by that point others should be able to easily install a mostly working LEDE image on their Omnia and help fill in the gaps.

This will be the third time I have seen abandoned yet almost complete patches for a device on my shelf, brought them up on my device, and submitted them to LEDE. They got merged the first two times, so hopefully I can help here.

turrios · June 8, 2017, 4:41pm

Hi Ryan,

Can you provide some documentation for it? How did you install it on omnia?
Is all hardware supported? (wifi, GPIOs)

Thanks!

ryan · June 9, 2017, 1:02am

It is still in progress, and not currently in a state I would feel comfortable distributing.

I have been unable to get the 88E6176 switch working properly. LEDE supposedly already supports this switch and swconfig is able to read data from it, but applying any configuration renders the switch inoperable until a reboot. Turris OS and all of the other Omnia ports I’ve seen are based on kernel 4.4, so possibly there has been a regression with 4.9. When I get time I am going to re-base my patches on LEDE 17.01 rather than master to see whether this is the case.

WiFi works, the RJ45 WAN port works. SFP will not work, and I would not be able to test anyway as my SFP-to-RJ45 modules don’t work even on Turris OS. I will check GPIOs, they should be fine as I am using an existing and well specified device tree file from the mainline kernel.

Configuration of LEDs is not supported, but they do retain their default function. It should be possible to add support for these, but this will be in a separate patch. To be honest I don’t care so much about the LEDs personally, so if I don’t get it working after a bit of time I will just give up.

Xiche · June 11, 2017, 11:43pm

I’m glad that someone was able to make use of my initial porting effort. To be completely honest, I got fed up with having to scavenge and claw through various other projects using similar SoCs (with and without the Marvell SDK) for clues about how to get things working correctly (especially with the 88E61676 switch) and sort of lost interest. Last I remember, the switch was in fairly good working order, so it must be something in 4.9. It seemed like progress was being made toward upstreaming support in the Linux kernel, but going in a different direction (DSA rather than swconfig), so I started to feel like it was going to be throwaway work. Also, it didn’t look like the phylink patchset was getting any traction upstream, even though it seems perfect/necessary for the WAN SFP/fixed interface support. My Omnia is currently sitting unused in a drawer; this reminds me I should make some time to play with it again.

leif.liddy · June 12, 2017, 1:40am

The guy who wrote the DTS file --stated that the 88E6176 Ethernet switch is an unsupported peripheral.

https://patchwork.kernel.org/patch/9442351/

I’m assuming he means it’s not supported by the linux kernel. If that’s the case, I’m wondering why they haven’t up-streamed it.

ryan · June 12, 2017, 4:05am

The 88E6176 was added to the DTS for the Omnia in a later patch, and the 88E6176 has been supported in Linux for a while (for DSA).

github.com

torvalds/linux/blob/master/arch/arm/boot/dts/armada-385-turris-omnia.dts

/*
 * Device Tree file for the Turris Omnia
 *
 * Copyright (C) 2016 Uwe Kleine-König <uwe@kleine-koenig.org>
 * Copyright (C) 2016 Tomas Hlavacek <tmshlvkc@gmail.com>
 *
 * This file is dual-licensed: you can use it either under the terms
 * of the GPL or the X11 license, at your option. Note that this dual
 * licensing only applies to this file, and not this project as a
 * whole.
 *
 *  a) This file is licensed under the terms of the GNU General Public
 *     License version 2.  This program is licensed "as is" without
 *     any warranty of any kind, whether express or implied.
 *
 * Or, alternatively,
 *
 *  b) Permission is hereby granted, free of charge, to any person
 *     obtaining a copy of this software and associated documentation
 *     files (the "Software"), to deal in the Software without

This file has been truncated. show original

LEDE uses their own swconfig rather than DSA so this doesn’t really help. Turris OS also uses swconfig so at least there is something to refer to, but it is more of a puzzle than it should be.

88E6176 is also used by the Linksys WRT1900ACS (+family) and Clearfog Pro, both supported by LEDE. The Clearfog Pro’s is also indirectly connected like the Omnia, I think this is where the issue is.

It appears I have no choice but to dig into mvsw61xx, based on the following Turris OS patches I would prefer not to.

github.com

CZ-NIC/turris-os/blob/test/target/linux/generic/patches-4.4/734-phy_mvsw61xx-88e6176.patch

--- a/drivers/net/phy/mvsw61xx.c	2016-01-17 22:19:39.644231765 +0100
+++ b/drivers/net/phy/mvsw61xx.c	2016-01-26 13:54:28.225141054 +0100
@@ -566,6 +566,104 @@
 	return mvsw61xx_update_state(dev);
 }
 
+static int smi_wait_mask_raw(struct switch_dev *dev, int addr,
+		int reg, u16 mask, u16 val)
+{
+	int i = 100;
+	u16 r;
+
+	do {
+		r = sr16(dev, addr, reg);
+		if ((r & mask) == val)
+			return 0;
+	} while (--i > 0);
+
+	return -ETIMEDOUT;
+}

This file has been truncated. show original

github.com

CZ-NIC/turris-os/blob/test/target/linux/mvebu/patches-4.4/905-omnia_switch.patch

Index: linux-4.4.39/drivers/net/phy/mvsw61xx.c
===================================================================
--- linux-4.4.39.orig/drivers/net/phy/mvsw61xx.c
+++ linux-4.4.39/drivers/net/phy/mvsw61xx.c
@@ -640,8 +640,25 @@ static void mvsw6176_quirks(struct switc
 		smisw16(dev, PHY_ADDR(i, PHY_SPECCONTROL_REG), copperspecreg);
 
 		copperreg=smisr16(dev, PHY_ADDR(i, PHY_CONTROL_REG));
-		copperreg&=~PHY_CONTROL_PWR;
+		/* Reset copper PHY. Added to overcome problems with Broadcom and Netgear
+		 * EEE-enabled switches that didn't set the link up after software reset
+		 * of the MVSW.
+		 */
+		smisw16(dev, PHY_ADDR(i, PHY_CONTROL_REG), (copperreg | MV_CONTROL_RESET));
+
+		/* Cleanup after PHY reset: Set autonegotiation, set mode advertisement. */
+		copperreg &= ~PHY_CONTROL_PWR;
+		copperreg |= (MV_CONTROL_ANEG | MV_CONTROL_ANEG_RESTART);
 		smisw16(dev, PHY_ADDR(i, PHY_CONTROL_REG), copperreg);
+

This file has been truncated. show original

My favourite is

+	/* Hack Hack Hack */
+	mvsw6176_quirks(dev);

Edit:
‘Good’ news, the 88E6176 doesn’t work properly with 4.4 either. swconfig isn’t included in Xiche nor heinzek’s images so it doesn’t have an opportunity to configure VLANs and bork the switch on boot. Xiche’s patch to set RGMII-ID is inside mvsw61xx itself and does work.

With no visible correlation between 4.4 and 4.9 and switch behaviour, I’m now more confident that I can clean up CZ-NIC’s mvsw61xx patches and and move some of the fixes upstream.

Others don’t seem to be so concerned about configuring VLANs on the switch (I need this for my use case), so tonight I am going to have a look at the sysupgrade situation with medkit images and see if I can put together a PR for LEDE without VLAN support.

Update 2017/7/3: I haven’t forgotten, I have just been busy with other things the past few weeks. I may have some more time the next few days.

JamesT42 · June 20, 2017, 11:46am

I definitely need VLAN. Maybe you can uplaod a finished medkit image we can test?

LukasTribus · June 20, 2017, 2:18pm

I wish the Turris guys would have spent a moment upstreaming the HW support, given that they benefited so much from the OpenWRT/LEDE project (and they would continue to benefit from this as they would have less work forward-porting and merging upstream changes).

But I guess that is not how things are done in the embedded world …

ryan · July 8, 2017, 4:44pm

Good news, I have revisited this with a clear mind and the switch is all working (VLANs etc) in LEDE master/kernel 4.9. I am too embarrassed to admit what I was missing…

I am now quite sure that it will be possible to have a unified medkit/sysupgrade image without having to do anything special on either side. Right now sysupgrade accepts the ‘hybrid’ image but I am yet to write the script to handle upgrades, so it’s still necessary to restore via recovery and wipe every time.

Once I write the upgrade script I will upload my image here for people to test, and submit the PR to LEDE as a RFC.

Edit:
It may be some time before I return to working on this and think of a sysupgrade approach that I think will satisfy both myself and LEDE, so I am going to release what I’ve got so far.

#YOLO
LEDE MEDKIT IMAGE FOR TURRIS OMNIA
#NOSYSUPGRADE
#NORGBLEDS
#NOWARRANTY
vvvv

^^^^

opotonil · July 10, 2017, 7:16pm

Thanks you very much @ryan.

It’s working well on my Turris Omnia. I don’t know if all targets share the same “/etc/config/network” if not, I would change:

config interface 'lan'
        option ifname 'eth1.1'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 4 5t'

by next, as default config:

config interface 'lan'
        option ifname 'eth1 eth0'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 5'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '4 6'

Or if you prefer:

config interface 'lan'
        option ifname 'eth1.1 eth0.2'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 5t'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '4 6t'

And a question, can I get built packages for the kernel version you are using?

# opkg install luci-app-sqm
Installing luci-app-sqm (1.1.3-1) to root...
Downloading http://downloads.lede-project.org/snapshots/packages/arm_cortex-a9_vfpv3/packages/luci-app-sqm_1.1.3-1_all.ipk
Collected errors:
 * satisfy_dependencies_for: Cannot satisfy the following dependencies for luci-app-sqm:
 *      kernel (= 4.9.34-1-6119e44c82015f82d23f03a5b8b48628) *  kernel (= 4.9.34-1-6119e44c82015f82d23f03a5b8b48628) *  kernel (= 4.9.34-1-6119e44c82015f82d23f03a5b8b48628) *  kernel (= 4.9.34-1-6119e44c82015f82d23f03a5b8b48628) *  kernel (= 4.9.34-1-6119e44c82015f82d23f03a5b8b48628) * 
 * opkg_install_cmd: Cannot install package luci-app-sqm.

Pepe · July 10, 2017, 7:28pm

and here we can check status of pull request at LEDE’s github from @ryan

ryan · July 11, 2017, 1:41am

Glad to hear that it’s working.

The default switch configuration is defined here: https://github.com/rmounce/lede-source/blob/omnia/target/linux/mvebu/base-files/etc/board.d/02_network

I think ucidef_add_switch automatically assigns the first VLAN (1) to the first CPU port it sees (eth1), so it is not possible to change this default so long as switch ports are correctly labeled. My preference was to have almost the same as the default configuration in the image using eth0.1 (0 1 2 3 4 6t), but this conflicts with the above behaviour.

I am curious about the reasoning behind your configuration. The Omnia only has 1Gbps WAN, so using a single interface as uplink for all 5 LAN ports is not a bottleneck in most cases. By breaking out one port from the switch to a separate CPU port, it is now necessary to bridge any LAN traffic between this and other ports through the CPU increasing the load on it. The only benefit I could think is if the device on this last switch port is sending lots of data to WiFi devices or using the Omnia itself as a NAS, while other LAN ports are sending lots of data to WAN. In the multiple VLAN case this is different, because inter-VLAN traffic must be routed via CPU anyway so splitting traffic onto the second port is always beneficial.

Regardless, you are free to change your switch configuration to the one you have described. You may also consider using one of the Linux bonding modes that does not require switch support like balance-rr or balance-alb so that the bandwidth to all LAN ports is doubled, and traffic between LAN ports does not go via the CPU.

I only intended for this image to be used for preliminary testing, I would have to figure out how to do a full build and then also host packages. I would suggest either to wait until support is merged into LEDE, or to download my tree and build your own image with all desired packages.