My Turris Omnia (OS 3.10.7) is querying the domain project.turris.cz. Looking at the website attached to this domain (https://project.turris.cz/en/) it’s the endpoint for the Data Collection project which I am not part of (nor have ever been).
A quick grep across my system indicates it’s probably /usr/bin/nuci-helper-autocollect which is being run on an hourly basis by something and also run daily by /etc/cron.d/autocollect. Assuming the underlying nuci-helper-autocollect script is only cause, the reason is the script is phoning home before checking if I have agreed to data collection:
[...]
# Get today's registration code
CODE=$(curl -k -m $TIMEOUT "$CHALLENGE_URL" | atsha204cmd challenge-response | head -c 16)
# Ask for the status of the contract
RESULT=$(curl -s -S -L -H "Accept: plain/text" --cacert "$CA_FILE" --cert-status -m "$TIMEOUT" "$CONTRACT_URL$CODE" | sed -ne 's/^result: *\(..*\)/\1/p')
if [ "$RESULT" = "valid" ] ; then
if uci -d'
' get updater.pkglists.lists | grep -q -F i_agree_datacollect ; then
[...]
So, the script is seemingly curl'ing out in two different ways (api.turris.cz and project.turris.cz) before it checks if I’ve agreed to data collection. This script should check if the user has opted in first before phoning home, otherwise the point of opt-in is somewhat moot (even if the data is non-identifiable).
How can I report a bug about this issue? I tried finding where this file exists in GitLab but the search is somewhat ineffective and you can’t seemingly search source code across repos (example). There’s a secondary issue here as well where request to https://api.turris.cz/is made insecurely (eg this domain has a self-signed HTTPS cert), but I’ll report that as well.
In time meantime, whilst this is getting addressed, what is running this script every hour? I’ve obviously removed the cron.d entry and removed the script itself, but should I be expecting they’ll come back with a future update? Beyond black-holing the domains (which I probably can’t for api.turris.cz), can I prevent the router making these requests?
Fwiw, I see there’s similar topics of this nature (eg Why does router phone home?) but they don’t mention specifically mention project.turris.cz. However, this domain does resolve to the same IP as repo.turris.cz, so the behaviour the OP is noticing could be to proxy.turris.cz either instead of or as well as repo.turris.cz.
Aha, thanks. I did find that file but had skipped over it because it was named differently.
I’m completely new to what that script does and how, but on balance, the code is relying on the remote response to toggle whether to forcibly add data collection into the router’s configuration or not, rather querying a local setting first.
Fixing this would require a new uci setting (or a detailed reworking of current options) so this goes beyond an easy fix for someone in my position. At any rate, it turns out I don’t have the ability to fork/merge request in that GitLab (a permissions or limits issue?). I’ve open issues in that case:
Outstanding questions if anyone can answer them:
What is running the nuci-helper-autocollect script every hour?
Can I do something to prevent the cron.d entry and nuci-helper-autocollect script being restored after an update?
This is unintuitive, and a problem. The router does data collection based on the response from the server, regardless of the setting in the GUI.
I discovered it because one of my routers is in a site that had so much traffic that the ucollect was filling the 2GB of RAM and triggering the oom-killer. But usually the DNS server would die first, which was very confusing. I disabled data collection in the GUI, and ucollect still filled the memory and killed the DNS server. I had to log into my Turris account and remove the router there.
No it does not do data collection when not enabled. At least not for you. We have routers that are under contract and should collect data. Instead of publicaly advertising serial numbers of routers under contract we have routers to check it on their own. This is now almost obsolete concept and in 3.11 is going to be dropped, atleast in most of the cases (nuci is going to be dropped but might be still installed in some cases).
My Omnia is now on 3.11 but the autocollect cron configuration and nuci-helper-autocollect script (which I’d manually removed) have been reinstalled. I’m guessing the cleanup on this front (as per https://gitlab.labs.nic.cz/turris/nuci/issues/22#note_88788) is still continuing and hasn’t been finalised by 3.11?
In the meatime, aside from rm'ing the scripts each time an update comes along, is there any better solution for removing this obsolete software? The comment on issue #22 indicates that stuff might depend on nuci. From a package perspective:
root@turris:~# opkg whatdepends nuci
Root set:
nuci
What depends on root set
nuci-nethist 161-1 depends on nuci
So if I opkg remove these packages, is anything going to break / will they be reinstalled?
It won’t work if you just remove it with opkg. It should no longer be used but because we might have missed something we decided to not drop it from default installation unless we are sure (just to prevent unnecessary breakages). You can uninstall it with adding file to directory /etc/updater/conf.d/ with following content:
Uninstall("nuci", "nuci-nethist", {priority=60})
Running pkupdate (updater) after that should remove it.
I don’t guaranty that nothing is going to break and that it will work but if you want this is more permanent solution than just removing those files every time. Up to my knowledge only thing that is affected is mobile app, everything else should just work.
So upon doing so, I’m prompted to remove a bunch of dependencies from nuci. @cynerd Happen to know if any of this still in use elswhere in the software? I’d imagine the only issue this might break is if a pkg has an undeclared dependency on something:
INFO:Queue removal of nuci-nethist
INFO:Queue removal of nuci
INFO:Queue removal of libnetconf
INFO:Queue removal of libxslt
INFO:Queue removal of coreutils-base64
INFO:Queue removal of nethist
INFO:Queue removal of thermometer
INFO:Queue removal of lm-sensors
INFO:Queue removal of libsensors
INFO:Queue removal of sysfsutils
INFO:Queue removal of libsysfs
INFO:Queue removal of ntpdate
INFO:Queue removal of socat
As for the uninstaller, do I need to leave that file in /etc/updater/conf.d/ permanently or is it just a one-off pkgupdate and then I can delete the file?
You have to have that file there permanently. Well at least until we drop nuci our self. Updater is basically reading that configuration which gives it requirements for system state. Such state is then compared to current state and changes are proposed. That is what you are approving. The configuration you just added to updater is saying updater that you don’t want to have nuci installed with priority 60. All our packages are requested to be installed with priority 50 or 40 (or critical) so when there is a request to both not to have nuci and to have nuci then high priority wins. It this case that is not suppose to be installed even if some other package requires it. I know it is complicated. Just left that file there until we drop nuci completely.
Looking quickly over those packages you might want to have thermometer and ntpdate. Thermometer is used to get board temperature. ntpdate is used to update system time using NTP. Neither one of them is essential but they are handy. There are others such as sysfsutils or socat but those are really required only if you are going to use them from cli.
Super, thanks for the info, that’s really helpful.
I added a further Install(...) line to request that thermometer be installed (eg remain installed) but the rest of packages look unnecessary. The NTP functionality in Foris is unaffected by the removal of ntpdate and everything else that I’ve checked so far continues to work so that’s great.
As for ‘official’ dropping of nuci, I’ll just keep an eye on release notes for future releases and then take my config away at that point.
Ah, okay, so here we go again. Updated to 3.11 and the router is now trying to access the domains project.turris.cz and api.turris.cz again for checking “contract status” but under a different set of scripts.
/etc/cron.d/server-uplink.cron and associated scripts in /usr/share/server-uplink both phone home every hour and are now being provided by the server-uplink package. I can’t use the above-mentioned override to uninstall this package as well because it’s a dependency of foris-controller-app, so removing server-uplink kills Foris.
I’ve put a post-update hook in place to manually remove these scripts and cronjob by writing a script into /etc/updater/hook_postupdate/ – any better suggestions than that, @cynerd?
So, now will this server-uplink package be removed or be made optional in some later OS? I note the scripts now have been rewritten and the package being a dependency of foris makes it seem like it’s going the other way.