This all seems very anaemic


#1

So I bought a Turris Omnia about eight months ago because I’d had enough of running my own firewall from the toolchain up and felt like having someone else take a little of the maintenance burden away.

Unfortunately, the selection of packages in the Omnia – and, for that matter, frequently in upstream OpenWRT – appears to be so anaemic, missing huge numbers of packages I’d consider fundamental, that I suspect it would be easier to wipe the thing and keep just building my own images from scratch, without using any OpenWRT machinery (let alone the ludicrously inappropriate-for-an-8GiB-box squashfs flashing and overlayfs, which I wouldn’t expect to be necessary on any machine with over a gig of flash).

I mean, so far my list of apparently unavailable packages goes from stuff obviously crucial for authentication among anyone with a fairly widespread hardware token (need to package libykclient, libyubikey, yubikey-personalization, yubikey-pam), through stuff hilariously obsolete until four days ago and obviously crucial for any network router (Apache, which was at the unmaintained-for-years 2.2 until the upgrade four days ago, which took it to… 2.4.37, which was already insecure and obsolete at the time of release)…

… and when I discovered that sendmail wasn’t packaged, nor was procmail, and that the firewall is intrinsically bound to particular network interfaces so is probably completely broken if you are using a line-bonded configuration to your ISP (so I’ll have to figure out how to turn it off and use raw iptables and no doubt fight constantly with other packages assuming that I’m using the luci-based configuration, and all the data collection etc will no doubt be totally broken), I started to wonder what the hell the selling point of this firewall actually is.

Are you expected to have to redo your entire network configuration from mail reception up simply because you replaced your firewall? Right now the probability of mail-losing breakage when I install this thing appears so high that there’s no way I can use it without figuring out how to wipe it clean and replace the entire OS with Debian or something actually maintained in a meaningful fashion – which is difficult because there is no easily accessible serial console without taking the bloody thing to bits. (And oh yes I could switch to Postfix if I learned how to use it and rewrote several hundred lines of sendmail.mc accordingly, but even that package is four years obsolete!)

And these are the problems I’ve spotted before I even turned it on! At this point I’m wondering if I’ll get the thing working before the warranty expires.

I am… not happy. I suspect I wasted several hundred quid, and I still have an existing firewall that is giving up the ghost and needs replacement :frowning: but I’m not sure I can replace it with this.


#2

Again, packaging a dozen fundamental-seeming packages is not a problem – though it’s very surprising that it’s necessary – but I do continue to have a feeling that I’ll be fighting the firewall and LuCI non-stop because I’m line-bonding while the Turris has only one port labelled ‘WAN’ (I’ll have to use a LAN port for one of them, and obviously I’m going to rename all the interfaces using OpenWRT’s weird replacement for udev or a straight ‘ip link set name’ to try to keep other packages’ hands off it).

[… elided rant about insecurity of a glibc in the turris-os tree that the Omnia doesn’t use, I should pay more attention]


#3

Of course this thing is musl-based: I knew but forgot. Phew, that looks a lot less terrifying: it’s being kept up to date and I don’t think it has any unfixed CVEs in sight. (Previous rant elided.)

Maybe this is usable after all, if I can avoid fighting the firewall config the entire time. Getting the line-bonding working is my biggest single worry: the rest is mostly a simple matter of packaging, even if packaging horror shows like procmail. (I already have a Perl script that monitors the remote end and brings the links up and down as needed, if I can figure out where to run it from.)


#4

Not sure whether an email server is really a router feature but in any case such might be easier to achieve via alxc guest. The irony though is that the lxc packages itself are badly outdated in the TOS repo and TOS is also lacking unprivileged containers whilst lxc upstream on OpenWRT features unprivileged containers but is currently outdated there as well and not maintained.

The network manager package netifd is also badly outdated in the current TOS3.x branch and could potentially impact that line bonding you are trying. Alternatively you could see whether the TOS4.hbd (here be dragons) development branch suits better, which is based on OpenWRT’s master and thus highly experimental.

Well the firewall is what it is (or what not) and it might be better to setup your own. I prefer nftables over the ancient iptables but then again nftables is not in the focus of OpenWRT development.
Just make sure that prior turning off the fw3 shipping in TOS/OpenWRT to have your own rules in place (at boot time) or you else you will be cut off from accessing the router. For that purpose schnapps comes in handy, if you are already familiar with its rollback potential.


#5

The question is whether the development branch boots and works, really. If it doesn’t even boot there’s not much point my trying, but if the failures are more along the lines of “the watchdog timer isn’t working yet”, it might be worth a try. :slight_smile:

(And surely mail relaying, at least, counts. At least it used to before everyone assumed all their email would be done by gmail.)


#6

TOS4.hbd is booting but

  • second core is not working
  • switch is not working
  • this far I could not get PPPoE connectivity on the WAN

#7

OK, that definitely seems worth trying! I’m replacing a one-core firewall with 1/16th the memory, after all :slight_smile: and switch functionality, well, as long as IP forwarding is working I’ll be happy.

Gotta bisect this unrelated pair of kernel bugs first, I thought this would be a weekend


#8

Another idea occurs to me – I can get several of these packages by adding pieces of upstream OpenWRT packages.git as extra feeds. (Not all, but some of the nastiest ones.)

The question is whether anything has changed in the OpenWRT build system in the interim that would break things… I guess I’ll find out.


#9

@n8v8r wtf? Second core and switch were working day one of hbd. I can’t confirm problems with pppoe but the problem might be there, only that n oone else reported it and we don’t have logs for it.

The problems with 4.0 on Omnia are much more subtle. I am running it on my routers and only things that do not work are leds configuration, second ethernet between cpu and switch and sfp. Everything else is working, well at least in kernel it self. There is going to be alpha2 just in few days that fixes many software problems and atsha204. That is if you don’t want to run dragons or kittens.

@NullNix have you even looked in to the device in depth? Turris OS is not a joke and addresses a lot of things you mentioned. We don’t have squashfs, we have btrfs. We provide automatic updates, not manual reflash. I acknowledge that some of the packages are outdated in OpenWRT and even in upstream but the point of OpenWRT is network oriented devices not servers and for that is its software is tailed to. It is like ranting on fish that it does not run on shore. You can always run debian in lxc container if you need a server OS. If you don’t like firewall setup or have some specific configuration then don’t automatically assume that it is not possible to configure it or that it is just garbage. You know that Luci interfaces do not map one to one to netlink interfaces? I am not sure if you are able to setup bonding from luci but you should be able from cli and then put luci wan interface on bonding netlink interface and that way you have everything setup correctly. Also if you need debian then just search, one of our kernel developers is running debian on it and it is documented somewhere on debian wiki I think.

@NullNix let me tell you something. I was so close to marking your post as inappropriate and hiding it. It contains so much wrong things and it is such clear that you wrote it even before you thought about it or did a research. We acknowledge problems and do not censor them but we don’t like runts here on forum. Those are someone else’s personal busyness and there are blogs for it. May I suggest you to instead describe your problem and ask if there is a solution in future instead of this? And this is not applicable only to this forum but to all forums. This sort of posts are counter productive and do not solve anything. Thank you.


#10

That would break big time most probably…


#11

How many users you got with PPPoE and trying out TOS4.x? Perhaps only one…


#12

We have one person in the office I know about without problem and you. No one else reported anything about pppoe on TOS4.x.


Tos4.hbd experience on TO
#13

[reshuffled to put more-rant-like things at the end, though they’re not very rant-like: I’m starting to think that the only thing wrong here is documentation that makes it hard to tell what things the Omnia can do that differ from stock OpenWRT, and the chaotic organization of OpenWRT’s own documentation. OpenWRT really really needs to steal FreeBSD’s documentation team. :slight_smile: ]

@cynerd:

only things that do not work are leds configuration, second ethernet between cpu and switch and sfp

… hmm, 4.0 is seeming more and more desirable! I might well upgrade before even setting it up once I get the serial console working so I can revert if it goes wrong: there’s no point spending ages setting everything up if I have to rethink it all after a massive upgrade. The leds don’t matter to me since it’s going to be under a wooden cowling anyway (assuming the 5GHz wifi can get through it!) and I’m not using SFP (yet) so the (wonderful) second internal Ethernet interface isn’t a problem either. One presumes all the physical RJ45/Ethernet network ports work, since I have planned uses for all of them :slight_smile:

I am not sure if you are able to setup bonding from luci but you should be able from cli

Oh no the firewall I’m replacing is a special snowflake (A&A is an unusual ISP with some strange features and I sometimes think I’m using all of them) and its handling of line bonding is both wonderful and loony. The ADSL lines are bonded, but it’s done by the ISP so the bonding need not be reflected on the Linux side at all! You plug two ADSL routers into two network interfaces with different IP addresses, throw packets out of both of them using multihop (so any given flow emerges from only one), and all incoming packets addressed to anything in my assigned IP ranges come in on both (one packet on A, the next on B, the first on A, etc), and get agglomerated automatically because they are part of the same flows. Linux can do this with a multihop outgoing route and nothing else, as long as the rp_filter is turned off (a good idea for multihomed devices anyway). All you need on top of that is a little 200-line daemon to watch the incoming network traffic on both interfaces, ping out of them individually if there is none in some suitable interval to see if either interface is down, and tear it out of the multihop route if it is (re-pinging at intervals to see if it can be brought up again).

I looked at the bonding driver in 2013 when I set this stuff up and found it wasn’t then up to the job. It wasn’t possible to tear interfaces out of the routing when the ping stopped working: the bonding driver expects to be able to do all of this itself by watching the Ethernet link status, but unfortunately my ISP’s router does not take its Ethernet connection down when the ISP link dies, so the bonding driver would not notice and I’d end up routing packets out of a link to nowhere, and there was no way to declare an interface with still-live Ethernet dead and rip it out of the bond. I just checked again, and since I investigated this last, bonding 3.0.0 has come out, which should let me detect dead links myself and tear them down on demand via e.g.

echo -eth0 > /sys/class/net/bond0/bonding/slaves

etc, and bring them up again later as well. That seems very nice, and makes using bonding rather than raw multihop practical, at least if I can still direct pings out of the component interfaces, and I don’t see why I can’t. And all of a sudden I can make multiple interfaces look like one to LuCI, I think! cheer

Time to test this on my existing Soekris firewall…

We don’t have squashfs, we have btrfs. We provide automatic updates, not manual reflash.

As for squashfs: I think I was led astray by a hyperlink pointing into the OpenWRT documentation, and read a bunch of it thinking that it would apply to the Turris (which is, after all, running OpenWRT), and failed to realise how much of what I’d read was non-Turris-specific documentation – either that, or the documentation for the Turris has changed a lot since November of last year when I read most of this. I knew it was using btrfs, but had no idea it was using it instead of the squashfs mess: I thought it was simply overlaying a btrfs fs on top of the usual OpenWRT squashfs thing, and nothing I read disabused me of this. I’m extremely happy to hear it’s using btrfs and rolling updates instead; I thought at the time it was very strange that a box with so much flash would be using squashfs and intermingling rolling updates with reflashes like a 64MiB micro-router would. IMHO rolling updates are the only way to run any system you actually want to rely on for anything: no big flag days, ever! I am much, much less worried now: it’s much easier to recover from disasters on rolling-update systems. (Because of course I expect there will be disasters. That’s how you prevent them being too disastrous.)

You know that Luci interfaces do not map one to one to netlink interfaces?

Well, no, because everything I’d read, including OpenWRT’s own documentation, said quite clearly that they did. (That’s my root problem here, which is nothing you can do anything about: OpenWRT’s problem for a new user like myself is that its documentation, while copious, is so poorly organized that it took me ages to figure out even that much, and I ended up with a whole bunch of wrong impressions).

What other interpretation could I take from this (on the old blog so more likely to apply, I suspect, to Turris OS): “UCI Firewall maps two or more Interfaces together into Zones that are used to describe default rules for a given interface”… “Zones must always be mapped onto one or more Interfaces which ultimately map onto physical devices”. At no point is it even implied that the “interfaces” mentioned are not, well, the things that Linux has always called “interfaces”: they even have the same “eventually map onto physical devices” semantics.

Was I seriously supposed to think “oh yes, these things they call interfaces are different things from Linux network interfaces and can have a many:1 mapping with them, even though this mapping is nowhere described and there is no suggestion anywhere in the documentation how one might map multiple physical interfaces onto this LuCI concept”? It would frankly have been crazy for me to think any such thing: far more likely that two things with the same name, serving the same purpose, are the same thing. Apparently they’re not: good – though I still have no idea how one maps more than one physical interface to a Luci interface, maybe that will be clearer once I start the thing up. I suppose it is vaguely implied by https://oldwiki.archive.openwrt.org/doc/uci/network, but frankly even reading that I would have no inkling that you could attach more than one network device to one of these LuCI interfaces: it says “list of interfaces if type bridge is set”, but again that is no different from Linux network interfaces. There is no mention of attaching multiple physical interfaces to one LuCI interface without bridging them. There is no mention of how to do anything to single Linux network interfaces other than associate them with single LuCI interfaces or bridges. So yes, I assumed they were the same thing because they looked exactly like the same thing. They still do.

I have spent probably upwards of two weeks all told planning and reading all the documentation I can get my hands on on OpenWRT and the Turris both, including significant chunks of this forum. If I still have the wrong impression of things as fundamental as what filesystem the thing is using and how network interfaces work, so be it: since you point directly at the OpenWRT documentation I did assume that it documented this system too, and that if some of it did not apply there might at least be some indication as to which bits those were. Apparently not :confused:

The reason why I was trying to plan everything first, particularly stuff to do with actual connectivity, is that I wanted to be sure I understood it properly first before turning it on even if that took months, since things of this nature can in my experience fairly easily FUBAR your entire network if you start them up in an unexpected fashion (in my case, as a test installation indirectly connected to my existing firewall on a network that already has most of the services running that will later be taken over by the Omnia, so if I mess up I don’t end up severed from the Internet and unable to do anything). In the past I’ve had things as simple as wifi routers try to mess up my local net by doing DHCP and IPv6 RAs of the default route out through them without being asked, so something this complex seemed much riskier.

If my Internet connection is severed and I can’t fix it, I lose my job, so you’ll pardon me if I don’t just turn the thing on and hope, but try to do some research first! I know I’ll have to invest considerable time redoing my firewall rules if (as originally planned) I use LuCI for them for better integration with the rest of the system, so I wanted to make sure that the rather special snowflake of my existing firewall was replicable, let alone the very special snowflake of my outbound routing. It was disconcerting to find out just how many packages I had to rebuild to reproduce what I was already doing. I knew I’d have to rebuild a few (there’s no way you’d have something like Simtec’s ekeyd packaged, for instance), but finding that procmail wasn’t packaged was a disturbing shock. I had this terrible sinking feeling that I’d be fighting the thing non-stop, but I’m starting to think that perhaps I was wrong. I hope so.

the point of OpenWRT is network oriented devices not servers and for that is its software is tailed to. It is like ranting on fish that it does not run on shore.

Um… if you want the thing to only be used as a router, and complain when people turn up wanting it to do not terribly obscure things like mail delivery, you might also want to change your own description of the Omnia as ‘more than just a router’. If it is in fact meant to be just a router and not ‘the open-source center of your home’, I clearly bought the wrong device, but thankfully it seems to me perfectly possible to do mail delivery etc on it, which seems like a perfectly normal thing to want to do on a network-oriented device: it’s just that the packages to do so have to be snarfed from upstream OpenWRT and/or built by hand (e.g. procmail). As far as I know. Still replanning, not tested yet, have to bisect and fix a bug in the USB-serial code in kernel 4.20 first so I can actually connect to the serial console rather than just getting a spurious I/O error when I try. (Planning for disaster, again.)

The fact that the build system is OpenWRT’s is very nice, because that’s a nice build system as cross-compiling buildroots go. (FYI, should you ever get sick of getting packages working with cross-compilers, another approach that could be used for a subset of annoying packages might be to run a static qemu-user ARM instance via binfmt_misc, and “natively” compile on a faster x86 box. I’ve built huge monsters never intended to be cross-compiled like all of GNOME that way, and it works fine.)


#14

TOS features schnapps for easy rollback/recovery.

And UART serial is connecting just fine in the current TOS3.x branch but have not tried it with TOS4.x.

Kernel 4.20 is not available in OpenWRT master either.


#15

I have to contradict you on this one. I am a packager for a Linux distribution, which has a considerable better tooling than OpenWRT, and yet packaging is a problem. It’s almost never just a matter of “doing it” save for the most trivial of things.


#16

Which one?
(spaces don’t counts into at least 20 characters of reply:-(


#17

@einar, oh are you ever right in the general case: I look at my coworkers being driven mad by packaging and thank my lucky stars I only work on easy stuff like dynamic decompilation and realtime debugging. :slight_smile: but for a single-purpose user it’s not that tricky, nor for the sorts of packages one is likely to package here. I mean I’m not planning to package the OpenJDK or LibreOffice or something really awful like Samba 4. (And, obviously, any packaging I do will be offered upstream. Why should people duplicate effort? Not sure I’ll bother with ekeyd though, it’s not like any Entropy Keys have been made for something like five years.)


#18

Yeah, I’m worried about 4.20 because all my other machines run it, i.e. the machines that will have to connect to said serial console if things go wrong. If the pl2303 driver in the Linux kernel is broken that kind of puts the kibosh on things. (It seems to be something more fundamental, though – USB as a whole stops negotiating new connections of any kind after about four days. This is going to be fun to debug.)


#19

OpenWRT has its fair share of quirks and although not too hard it took me a while to figure how to get Wireguard to compile, for example. And that’s just a very simple kernel module with < 4k LoC. The main problem is that you have to build everything together.

PS @jada4p: it’s openSUSE, in particular the KDE software stack.


#20

To get a rough idea, it’s sufficient to have a look at the number of (packaging) contributors that popular distributions have. The order of magnitude is somewhere else, probably even compared to OpenWRT. I’ve been active for years in yet another distribution and I must agree there’s really lots of work to keep it running if you want lots of packages to work well, unless you e.g. take Debian and just change some minor things. Routers also complicate things by less usual HW, cross-compilation, etc.