Discussions about state of optional migration from Turris 3.x for advanced users

Software
1

@Pepe Honestly? This is not a place but you asked for it.

First off all, the documentation is a tragedy. I read the article about the upgrade. None of the situations which I faced was there. The troubleshooting is scattered all over the forum. How about use some tag upgradeRelated for issues and solutions?

List of issues:

  • A partial installation of reForis after upgrade (runtime error due to some missing packages). Problem was reported two months ago, still there.
  • Broken opkg out of the box. Same like before.
  • Obsolete packages are still present after installation (hard to say what could removed, I haven’t found any documentation about that :thinking:).
  • Missing packages for NetMetr, OpenVPN, LXC, Samba.
  • DLNA had to be reconfigured.
  • AdBlock had be reconfigured.
  • Transmission had to reconfigured.
  • Service startups are messed up.
  • LXC containers missing (containers are there, LuCI not showing anything despite the fact that I was able to install a new container => LXC service was not running even after manual installation)
  • WiFi issues (non related to upgrade but I have problems with connecting 5 m avail from the antenna after one wall but I can see a WiFi of my neighbour 25 m avail from be behind three solid walls)

Yeah, it is an experimental process but for how long? Six months? One year? A smooth, automatic update will be… Ehm probably never.

Don’t take it personally, but I’m not here to motivate the team of CZ.NIC. I’m here to fix my problems. Yes, I didn’t need to re-flash the system and yes, I’m able to connect to the Internet but state of the systems as it is after upgrade is completely undistinguishable from re-flash state to me. I’m not a power-user, I’m not a BFU also. Right now, I don’t see any value in the upgrade. I believe that there is a value but I simply can’t see it.

Yeah, people do mistakes. That is the reason why Test-Driven-Development and pair-programming was invented. Try to employ those.

Sorry, I do PRs for living, I don’t have a time or energy to fix/fill in the blanks in the documentation here. Maybe someday, sometime. If you would be smart, you would use gamification principles for PRs, documentation, etc.

BTW: My friend worked for CZ.NIC, so I’m quite familiar with the situation over there.

3 Likes
2

By looking at your posts, I see that you are trying to discourage anyone to use optional migration. We understand that in your case, it was not as smooth as you expected. We didn’t say that we will migrate each setup or configuration. This is not possible and we are saying it multiple times.

If you are interested in more details you might want to take look at this issue wget missing after TOS 3->5 upgrade (#309) ¡ Issues ¡ Turris / updater / updater ¡ GitLab and maybe you would be surprised that it is hard to fix bugs, which we are not able to reproduce.

You were using some packages or if you prefer services, which are not covered by automatic migration like DLNA, Transmission or Adblock. These packages can not be installed via Foris.

Also, you seem like a power user and I think we can agree that a forum is not meant to be for bug reporting and tracking all bugs. For that purpose, there is GitLab and if you want any help or report some issues if you are not familiar with GitLab, feel free to do it via our technical support deparment.

1 Like
3

@Pepe Yeah, yeah, we can argue about the update whole day. I gave you mine point and that is the end for me on that one. I could help to track (mine) problems with migrations but I don’t see any specific action which I could do to help. The problems which I had are either reported or unsupported (a.k.a community problem). I’m not sure why e.g. DLNA is consider as community but :woman_shrugging:t6:

Anyway, where I can find a list of packages which shouldn’t be installed after migration to TO 5.1.4. I think that I don’t need 758 packages to be installed on my TO.

4

My opinion: the CZ.NIC team is doing a fine job. TO is a great piece of hardware, since 2016 supported with constant updates. Despite of the fact that it was introduced a few years ago it is still my choice today. I believe the CZ.NIC team is rather small and therefore needs to focus.

What I don’t understand is as follows: why is the CZ.NIC team investing so much resources into the “migration tool”? All TO3.X users where capable to set-up and configure there TO in the first place. Why is it not just acceptable that these users go through the effort to set it up once more when they transition to TO5.X? I’m not an advanced user and I was worried about the transtion to TO5.X myself. But for me it was always clear that I rather go through the re-learning experience by a clean TO5.X install vs. using this “migration tool”.

Also, would it not make sense to set a timeline after which only TO5.X gets further improvements? How many operational TO’s are still on TO3.X anyhow?

2 Likes
5

As @Pepe already mentioned migration is highly experimental …
… so even I am not so confident to do it smoothly on first try, I am actually pretty scared of it as my TOS 3.x is quite tailored (ehm sorry, broken to my needs by me) so i really do not know how it will went. Personally i do not understand why users with working 3.x version are migrating actually :slight_smile:

just notes

And in fact TOS 3.x is (from my point of view) most stable branche/version. I just like it. So i am not migrating just because new sotftware. You know migration is always tricky no matter what OS it is. Devs can’t cover all scenarios and configuration(misconfiguration). So blaming them is … … ,… they are trying to do the best (look around the gitlab/github, repository .)…with such team that quite huge amount of work. Last note, TOS is more-like for nerds with time to play with it. When i got mine from initial campain i was so frustrated with it , but after some time i kind of blend into it.

I know that not always there is bugless update, but that’s why we (users) are here, to find , to suggest fix, to even do the change (those granted ones :slight_smile: ) and push it to upstream(s)., if not it is up to devs (and that takes time …:slight_smile:

6

Why to upgrade? IMHO

  1. Security.
  2. Nextcloud.
  3. Some problem with DNS resolver which should be resolved in 4.0 and newer. I don’t really remember what was it but I do remember that it was annoying.

Like I wrote - for how long it is highly experimental? Turris OS 4.0 was released more than 12 months ago. The version 5.0 almost 6 months ago and it is still a highly experimental process. Not experimental, not not recommended, it is highly experimental. Come on, after more than 12 months? BTW: TOS3 Migration does not mention anything experimental.

I won’t adore TO team just because. I am a demanding customer. That is the reason why I paid a small fortune for the retail version of TO. A software means a soft product. A changeable product. If you can really change it, it isn’t a software or a product.

I think it is obvious that TO is not a one-year hardware. You have to keep this in mind. You have to build a technological stack to able fulfil this fact. You have keep in mind that you are going to change a lot of things. These changes should be smooth (:x:) and easy (:heavy_check_mark:).

A fear point with the fact that TO is a geeky device but… Please, do not interchange correlation and causality.
It would be a really beneficial if this paradigm would be widely deprecated here. I don’t want to spent hours by reading Bash documentation or dozens of posts on www.forum.turris.cz and I think I am not alone.

To be constructive, I recommand to

  1. Clean up the discussion about update and real solutions for issues in Optional migration from Turris OS 3.x for advanced users. 108 posts is just to much to read when you upgrade your TO and nothing is really working.
  2. Keep this section up-to-date. Maybe some dynamic list to GitLab issues related to upgrade and if there is a straightforward solution or fix for that, keep the links to the solution close to each other.
  3. Use a potential of this community. Award active community users by some real reward like yubikey fob or Turris MOX. @Maxmilian_Picmaus wrote an awesome notes about Samba use. Users like him deservers more than just likes on the post.
1 Like
7

Well, i’m a nOOb, and went from 3 to 5. Ended up in a fac reset on 3, and moved over to 5. Re configged 5, and up and running.

i do agree in the ‘’ stick to the plan " part. Concentrate on what is actually running. And not on the next ‘’ one more thing" ?
I like to see the whole dynamic firewall part more on the to-do list.

best.

8

@KUTlime thank you for suggestions. I am cleaning up that topic regularly but that topic is for discussions not to serve as a way to resolve issues. That is the docs link you posted as well. I think that you are missing the complexity of the migration. Yes I agree that for users it is easier to just reflash router and setup all from scratch. I do not live in illusion that migration can be hassle free for everybody but not doing it is also not an option (specially for routers that are left almost unmaintained). It is experimental because it is not ready as you experienced on your own. I am regularly fixing know issues with migration. If I can’t fix something I note it in the list of known problems. Today alone I think I finally nailed down some issues causing various problems such as missing links for busybox applets. It was simple fix but it was really hard to found it and I have to thank to David Hopfmueller who pointed me in the right direction. Thanks to him and others we are moving towards more stable migration but unless there are more people like him testing migration and meaningfully reporting issues with it rather than just complaining we might never have it stable enough. Here is issue with just subset of issues and of work that was done to make migration possible. You can see there also what was tested and what wasn’t. In most cases the tested setup in there is also the minimal setup (that is in case of LXC it is for example simple container with basic setup as it looks after bootstrap from Luci, nothing more). Wouldn’t it be better to instead of discussing here what should be done to rather do something about it? I mean how about reporting to me steps you took to make your setup working again with backup or even better exported snapshot before and after migration. I really can’t parse anything from “services do not run”. Thank you.

10

What exactly do you mean? I am really now sure. This is reaction to my request to provide me snapshot before and after migration? If so then it is just ultimate way of something that is written there. I quote: Create backup of your settings before you start not only to potentially recover it but also to provide it to support so Turris team can troubleshoot possible problems.
I have no idea what you mean with experimental status, really.

That documentation is for less technical savvy people. I can put some reference there but that issue might be parseble for you, because you are programmer, but it is for sure confusing to anyone else. That documentation tries to be end user friendly. I can rather add it to post here on forum and I am going to do that.

1 Like
11

The terminology is important. I thought that a snapshot is something else than a backup. I literally thought that a snapshot is a binary footprint of my system.

I can give you a snapshot after migration if you are interested. I don’t have any backup before migration.

About the link. Do whatever you think is the best. :+1:t6: A term user friendly documentation really made me laught. We have a totally different idea of what an user friendly documentation is. :wink:

About an experimental status. This was a term which was used by others users in my other posts when they were describing the current migration process/tool status.

12

Can anybody from ops tell what is so offensive here that this post was hidden?

14

Snapshot is not backup, you are right. Snapshot is full system image while backup contains just configuration. Documentation asks for backups because those can be received from web interface. I asked if you have backup or snapshot while I prefer snapshot as that can recreate your environment fully and allow be to better debug your issues.

I would appreciate if you can send snapshot before migration to me (please encrypt it before you upload it somewhere and send me just key to decrypt it by different channel such as PM here on forum).

Because it was reported as inappropriate by some users. We prefer to keep this forum managed by community not by CZ.NIC employs because of that I won’t judge if that was or wasn’t offensive.

15

Back to the original topic. Could someone from the team please update on the current migration status? Is it still disabled (by technical means), discouraged, or re-enabled again (albeit with the ‘advanced users only’) clause?

And, in order to save time of the team members: is there some authoritative place where to look for information on migration status changes?

TIA

16
17

What about watching issue https://gitlab.nic.cz/turris/turris-os-packages/-/issues/344 mentioned by cynerd?

18

It have been disabled for Turris 1.x only. There are still huge warnings for Turris 1.x. For Omnia it was never disabled.

It is still experimental but encouraged… well you can expect in some cases issues but I would appreciate any unknown ones to be reported to us (as the is the point of experimental migration).

Latest documentation can be found in official documentation. Of course feel free to investigate issue @peci1 linked as well as it might give you more in depth view on state of migration.

19

Sorry, my fault. I forgot to mention that I did indeed mean migration of 1.x, not Omnia (in this particular case).

21

Manual migration from Turris OS 3.last to 5.last

After waiting for optional migration tool for some time (and doing some unsuccesful attempts to migrate) I decided that my configuration is reasonably simple (TO from Indiegogo campain, 2GB, 2x WiFi, HaaS, Ludus, Ripe Atlas, lxc, mail, …), so I could try to migrate manually… reflashing it using last medkit. Unfortunately, I used - as I learned later - corrupted USB stick (even though formating and checking under Win10 didn’t find any problems), thus first 3 attempts of reflashing TO by medkit were in wain (LOL if you want). Only after using another sound USB stick it was successful, so TO was on 5.1.last level.

Switching branch to HBK brought TO to 5.2.0 RC3. SFSG.

Initial guided tour brought internet (WAN) connection, I was able to set some innitial options to be able to use TO router.

In TO 3.x I created profile for SSH sessions to display some basic information, e.g. CPU temperature etc… and to mount two USB sticks for logs and containers etc… Unfortunately, there are a lot of changes: some functions like thermometer, /etc/openwrt_version, branch info and mounting USB does not work any more :frowning: I found that thermometer could be substituted by sensor command, but openwrt_version displays not openwrt version but most probably its hash (which is of no use), branch info could be picked another way - but mounting USB via /etc/config/fstab and/or /etc/fstab does not work. ReForris Storage section offers “Format and set” option for USB, as far as there are usefull data on them it is no go for me.

Fortunately, adding two lines to /etc/rc.local did help (I changed /srv directory to something more meaningful)
mount /dev/sda1 /usbLogRB
mount /dev/sdb1 /usbKont

Next steps are to set up WiFi, redirect logs, recreate containers, set up Haas, Ludus, Ripe Atlas etc… BTW, isn’t it overkill to have both Haas and Ludus?

Trying to rebuild crontab, I got an error:

root@TOjp:~# crontab -e
no crontab for root - using an empty one
/bin/sh: /usr/bin/vi: not found
crontab: “/usr/bin/vi” exited with status 127

What is with vi editor? Let us see:

root@TOjp:~# find / -name vi
…
/bin/vi

Thus we need to create new symlink for it:

root@TOjp:~# ln -s /bin/vi /usr/bin/vi
root@TOjp:~# ls -l /bin/vi /usr/bin/vi
lrwxrwxrwx 1 root root 7 May 12 02:00 /bin/vi → busybox
lrwxrwxrwx 1 root root 7 May 13 11:24 /usr/bin/vi → /bin/vi

and at last I was able to edit crontab…

After it I set up WiFi, which is fortunately working. Then I started to redirect logs, setting symlinks for them in /etc/rc.local… it seems to work. SFSG.

TBC

NB is there any problem with Ludus? Trying to install it I got errors
root@TOjp:~# opkg update
root@TOjp:~# opkg install ludus-gui
Unknown package ‘ludus-gui’.
Collected errors:

  • opkg_install_cmd: Cannot install package ludus-gui.
    root@TOjp:~# opkg install ludus
    Unknown package ‘ludus’.
    Collected errors:
  • opkg_install_cmd: Cannot install package ludus.
22

(Un)fortunately, package thermometer fits into this category:

Be prepared that some packages and features you are used to have might no longer be there.

We had some packages that were not maintained anymore, and since Turris OS 4.x, they are not present, so less our maintained packages, then better overall maintenance? :slight_smile: It should also result in faster builds, though.

As said here:

There are two better replacements: sensors/collectd with collectd-mod-sensor.

Can you provide us more details? Mounting should work, because by default, there are included USB drivers, which is not so common in OpenWrt for some routers, but we need it to use Storage default by default without checking NAS package list. Anyway, it also depends on the used file system on a USB flash drive, and in that case, NAS package can indeed help.

No, these are two individual independent services and they are also provided by someone else.

Package Ludus is present only in Turris OS 3.x.

23

Mounting USB sticks by /etc/rc.local:

Summary

root@TOjp:~# cat /etc/rc.local

Put your custom commands here that should be executed once

the system init finished. By default this file does nothing.

mount /dev/sda1 /usbKont
mount /dev/sdb1 /usbLogRB
…
exit 0

Using username “root”.
…
BusyBox v1.30.1 () built-in shell (ash)
______ _ ____ _____
/_ / () / __ / /
/ / / / / / / / / / / / / /_
/ / / // / / / / / ( ) / // // /
// _,// // /// _//___/

TurrisOS 5.2.0, Turris Omnia

Sun May 16 11:10:55 CEST 2021
uptime: 11:10:55 up 2 days, 2:16, load average: 0.01, 0.02, 0.00
turris-version: 5.2.0
branch: hbk
…
Filesystem 1K-blocks Used Available Use% Mounted on
…
/dev/sda1 14747216 2914368 11060668 21% /usbKont
/dev/sdb1 7304224 55256 6854888 1% /usbLogRB
root@TOjp:~# blkid
…
/dev/sda1: LABEL=“KontejnerUb” UUID=“7301662b-48cd-4396-8767-371bd10c682a” TYPE=“ext4” PARTUUID=“150b4329-01”
/dev/sdb1: LABEL=“TOjpLogs8G” UUID=“90da6f93-5978-442d-9933-c1e564d56345” TYPE=“ext4” PARTUUID=“40f86163-01”

Mounting USB sticks with help of /etc/config/fstab (mount commands in /etc/rc.local changed to comments) after reboot:

Summary

root@TOjp:~# cat /etc/config/fstab
config ‘global’
option anon_swap ‘0’
option anon_mount ‘0’
option auto_swap ‘0’
option auto_mount ‘0’
option delay_root ‘15’
option check_fs ‘0’
config ‘mount’
option target ‘/usbKont’
option uuid ‘7301662b-48cd-4396-8767-371bd10c682a’
option enabled ‘1’
config ‘mount’
option target ‘/usbLogRB’
option uuid ‘90da6f93-5978-442d-9933-c1e564d56345’
option enabled ‘1’

Using username “root”.
BusyBox v1.30.1 () built-in shell (ash)
______ _ ____ _____
/_ / () / __ / /
/ / / / / / / / / / / / / /_
/ / / // / / / / / ( ) / // // /
// _,// // /// _//___/

TurrisOS 5.2.0, Turris Omnia

Sun May 16 11:34:10 CEST 2021
uptime: 11:34:10 up 2 min, load average: 0.38, 0.31, 0.12
turris-version: 5.2.0
branch: hbk
…
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mmcblk0p1 7633920 420812 7224376 6% /
devtmpfs 512 0 512 0% /dev
tmpfs 512 0 512 0% /sys/fs/cgroup
tmpfs 1033996 20724 1013272 2% /tmp
tmpfs 512 0 512 0% /dev

  • NOTE: no USB dev were shown by “df” command in indirectly called in root .profile *

root@TOjp:~# blkid
…
/dev/sda1: LABEL=“KontejnerUb” UUID=“7301662b-48cd-4396-8767-371bd10c682a” TYPE=“ext4” PARTUUID=“150b4329-01”
/dev/sdb1: LABEL=“TOjpLogs8G” UUID=“90da6f93-5978-442d-9933-c1e564d56345” TYPE=“ext4” PARTUUID=“40f86163-01”

As to sensors/collectd, is there any more thorough explanation/guide/etc?

As to Ludus, would it be re-introduced into OS v5?

As to HaaS, I don’t see any data even though I allowed it in reForis and logged of HaaS web page and later removed old instance of my TO from it and created new one.

How can I reenable my RIPE-Atlas probe? There is message : * No data available for Probe # 1000261 for this time period".

Trying to create Ubuntu container, I got error message : Please note: For LXC Containers you need a custom OpenWrt image.
The image should include at least support for “kernel cgroups’, ‘kernel namespaces’ and ‘miscellaneous LXC related options’ plus ‘kmod-veth’ for optional network support.”