Developer update #4

devupdate

#1

It was about two months since last update. Today, I will describe our work in the last three months of the year 2018.

TL;DR

To sum it up, we released 3.11 minor release with two patch releases and also 4.0-alpha for Turris MOX. We have finished manufacturing process for MOXes and we have sent first boards. Next batches are prepared and we will send them soon. We continue our work on new features we have promised and we need to reliably provide Turris OS updates and associated services such as data collecting.

Developer update

Michal Hrušecký

Michal (@miska) spent most of his time with Turris OS 4.0 development. This work includes kernel and U-boot finalization (together with Marek) and also the preparation of MOX network boot from Omnia. Michal managed to finish TOS 3.11 into production (3.11.2 at the time of writing) and prepares NOR-update for Turris 1.0 and 1.1 hardware.

Michal has also represented Turris team on several conferences such as OpenAlt, IT18 and Junior hackaton.

Pepe

Pepe (@Pepe) tested with Michal the NOR-update very intensively as it has a significant impact on the rescue system. They didn’t want to underestimate anything and tried to cover all possible issues, which could appear. Our users help us to test this crucial update and we have prepared Turris OS 3.10.9 release with this update for Turris 1.x routers.

Pepe started to update packages from upstream to TOS 3.x regularly (such as tor, wireguard, haveged, vim, and so on) and added recent Ubuntu among LXC containers. He also started to develop TOS 4.0 on MOX prototype and prepared several pull request to OpenWRT upstream packages (such as bind, Netdata, findutils, youtube-dl, …). Most of them were already merged.

Martin Matějek

Martin works on replacement of notification system with a new one written from scratch. He made good progress and all important pieces are in place, however, there are still parts that need attention and polishing before release. Also, we haven’t tested integration with other parts of Turris OS yet, so it will take more time before it is ready.

Štěpán

Štěpán (@shenek) works intensively on Foris interface. You have probably noticed a lot of changes in TOS 3.11 like DHCP clients list, separate data collect module, DNS forwarding options, NTP configuration, etc. He modifies most of the Foris modules to be more compliant with PEP508.

Štěpán put all his effort during October to make MOX devices be easily configurable. MOX can run with a only single Ethernet interface – such device can’t act as a router anymore. This (and similar device setup) requires to add a new workflow into the Foris guide and a new mode for LAN tab.

On the end of December, he started the replacement of original Foris message bus ubus with mqtt message broker Mosquitto which will allow to configure the device using mqtt API over the network. This API would be used in new prepared mobile application.

Martin Petráček

Martin (@mpetracek) spent a lot of time on finishing TurrisHW, a library for detecting available interfaces on Turris MOX. In response to some bug reports, he reworked an archive feature of PaKon (our local network monitoring tool) which should solve the problems with huge databases. He also ported other parts of our data collection system to Sentinel – hopefully replacing uCollect in the following months.

Karel

Karel (@cynerd) worked on programmer software that supports MOX manufacturing. This part of the manufacturing process brings the dead silicon alive: it programs SPI flash memory and then it tests board. It also takes care about secret keys generation for the boards and stores public keys in our database.

As to the Updater, he reworked our repository layout for Turris OS 4.0 – you might have noticed new directories on our repo.

The second great feature in Updater is called relative URI which will allow us to link repositories or another script relative to the introduced script. This feature is needed with new network boot for MOX.

Karel also spent some time with refactoring and cleaning up Updater code so it would be more clear and maintainable in the future.

Vojta

I have spent most of the time last two months preparing factory environment for MOX production. This environment consists of several “programmer” computers, database servers and a router – of course, Turris Omnia. Omnia takes care – besides the next – of persistent (Open)VPN connection to our server infrastructure, the database is replicated across three dbservers (one in our server infrastructure) because it holds crucial data from the production, and everything is Ansible-automated. So it is not big deal for us to tune PostgreSQL features or to deploy another programmer computer in the future.

Me, Štěpán and Robin have worked together to revise the device registration process as we want to drop some obsolete code and old Turris infrastructure components.

Michal Mládek

Michal worked on implementing this new registration process for Turris users and devices and the whole system we call My:Turris. Due to comfort user experience and security of user data, we will use Single-Sign-On design pattern for this web application. It will allow you to use all our services with only one account including login into (so-called) third-party services such as HaaS or our Cloud Backups.

Martin Prudek

Martin’s (@mprudek) work was divided into two main parts: Sentinel and Netmetr. Sentinel is our new data-collecting system that replaces uCollect. And Netmetr is a tool to measure the quality of Internet connection which you can simply install on your Turris device as well.

Martin worked on Sentinel:Checker component (which checks authenticity of Turris devices) and Sentinel:Dumper component (which stores Sentinel data into a database for later processing). Remarkable work (in cooperation with Robin) was also given to Sentinel:Monitoring: the concept of monitoring and debugging system for Sentinel network pipeline components.

Jan Pavlinec

Honza (@paja) updated security issues in packages like apache, git, openssh, nginx, curl, samba4, mosquitto, ntpd and so on. Some of these security fixes were sent to OpenWrt upstream repositories together with Unbound’s DNS64 module). Thanks to @Ondrej_Caletka for bug report! Honza also did some user-list reduction and cleanup in our feeds for upcoming Turris OS 4.0.


I would like to say goodbye to our developers @mpetracek and @iiic who left our team by the end of the year. Good luck with your new job opportunities.


#2

Good reading, albeit little is said for the T & O user base and rather focused on the M.

Whilst certainly understanding the work on the M will benefit the T & O the anticipation for TOS4.x is growing considering that:

  • the current kernel in TOS3.x is growing a bit long in the tooth and missing certain feature sets compared to more contemporary versions (naturally)
  • some core packages for TOS3.x branch are outdated, e.g. netifd, and causing issues that are fixed at upstream but likely not being backported. Thence leaving a vacuum of an undetermined time frame till TOS4.x will be ready for production, or at least some stable beta - concerning T & O.
  • developers of complementary userland apps at upstream already stated that they will not backport for TOS3.x usage and thus spreading the gap for an unforeseeable time even further. With the rather imminent release of upstream’s next branch such gap might widen then again.

Is the departure of those developers going to hurt the development pace?


#3

Hi never. You are right, we all want to have TOS 4.0 on all devices as soon as possible.


#4

Thanks very much to all Turris team members. :handshake:


#5

What is the point when:

  • 2 developers left and there is no mentioning of replacement. Since it is not clear what their duties been it is thus not clear whether or to which extent their departure slows the development of TOS4.x

  • Maintenance of TOS3.x is scraping along to the best abilities of the TO team (diminishing manpower and added M development) but the gap with upstream keeps widening to no end, as pointed out earlier

  • development resources are focused on the M, which has a way smaller user base compared to T & O. The particular requirements for the T & O have not even started development which becomes apparent when testing 4.HBD.

Mind you of TO team statements from the 1st half of last year (not that 2018 could be explicitly inferred as meaning of year, could be any year in the future indeed and 2018 has passed by now)

compared to (machine translation). At least it infers 2019 as year…

We expect it this year, but we do not have a more accurate estimate for the time being, so far we are still very unknown and we are still solving Mox.


Requesting new packages / new versions
#6

Looks like OpenWRT is facing its own struggle to get the next branch 19.x off for launch, thus far it does not appear to have been branched from master as it was planned last year.

Instead mainline 18.x being incremented recently https://openwrt.org/releases/18.06/notes-18.06.2 and even shows another one planned soon

https://openwrt.org/releases/18.06/start#roadmap