Data collection isn't collecting data

although ucollect is running, the project.turris.cz site still shows much less network activity than I’d expect. I have a support ticket open with @Pepe will advise what is found.

It looks like something goes wrong when ucollect is started at the router’s boot time. ucollect doesn’t start up properly when the router is booted, as a result it never collects any data to send. After the router has booted, if I stop and restart ucollect, it starts sending data properly. I saw that @miska had rewritten the init scripts for ucollect recently, could a bug there be related? I do have working IPv6, it looks like ucollect/socat are used over IPv6 per the logs.

Here’s the log messages for ucollect when it first starts up at router boot time. the socat error connecting to api.turris.cz look concerning.

2018-06-29 23:40:11 err ucollect[4295]: Error from socat: 2018/06/29 23:40:11 socat[4841] E connect(5, AF=10 [2001:1488:ac15:ff80:0000:0000:0000:0101]:5679, 28): Permission denied
2018-06-29 23:40:11 warning ucollect[4295]: Remote closed the uplink api.turris.cz:5679, reconnecting
2018-06-29 23:40:11 warning ucollect[4295]: epoll_wait on 4 interrupted, retry
2018-06-29 23:40:11 info ucollect[4295]: Reconnecting to api.turris.cz:5679 now
2018-06-29 23:40:11 info ucollect[4295]: Socat started
2018-06-29 23:41:11 warning ucollect[4295]: Remote closed the uplink api.turris.cz:5679, reconnecting
2018-06-29 23:41:11 warning ucollect[4295]: epoll_wait on 4 interrupted, retry
2018-06-29 23:41:11 info ucollect[4295]: Reconnecting to api.turris.cz:5679 now
2018-06-29 23:41:11 info ucollect[4295]: Socat started
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Count
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Sniff
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Flow
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Refused
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Bandwidth
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Fake
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Fwup
2018-06-29 23:41:13 info ucollect[4295]: Activating plugin Spoof
2018-06-29 23:41:13 info ucollect[4295]: Received configuration 49828441 (max. 5000 flows, 1800000 ms timeout)
2018-06-29 23:41:13 info ucollect[4295]: Received Refused config version 3
2018-06-29 23:41:13 info ucollect[4295]: Fake configuration version 1
2018-06-29 23:41:13 info ucollect[4295]: FWUp config 48834208 with 2 sets
2018-06-29 23:41:13 warning ucollect[4295]: epoll_wait on 4 interrupted, retry
2018-06-29 23:41:15 info ucollect[4295]: Updating filter addresses from version 0 to version 51011347 (epoch 4)
2018-06-29 23:41:18 info ucollect[4295]: Updating ipset turris_100FA4E0_lb_a_4_X from version 0 to version 51011347 (epoch 4)
2018-06-29 23:41:18 warning ucollect[4295]: epoll_wait on 4 interrupted, retry
2018-06-29 23:41:28 warning ucollect[4092]: Last message 'epoll_wait on 4 inte' repeated 1 times, suppressed by syslog-ng on turris
2018-06-29 23:46:52 info ucollect[4295]: Started task Ping as PID 10231 and fd 19
2018-06-29 23:46:56 info ucollect[4295]: Finished task Ping, success 1
2018-06-29 23:46:56 warning ucollect[4295]: epoll_wait on 4 interrupted, retry
2018-06-29 23:52:18 info ucollect[4295]: Updating filter addresses from version 51011347 to version 51011383 (epoch 4)
2018-06-29 23:52:20 info ucollect[4295]: Updating ipset turris_100FA4E0_lb_a_4_X from version 51011347 to version 51011383 (epoch 4)
2018-06-29 23:52:25 warning ucollect[4295]: epoll_wait on 4 interrupted, retry
2018-06-30 00:11:13 info ucollect[4295]: Sending 0 flows

Confirmed that ucollect startup at boot is broken somehow. If ucollect is restarted manually after boot it works. This worked properly before 3.10.3.

Hello,

I can confirm the issue, which you experience that some graphs, which can be found on https://project.turris.cz don’t have data. We’re looking into this issue.

Thank you for reporting.

@Pepe any chance of this getting fixed in 3.10.4? I’ve been working around it but would like to know if I can remove those workarounds.

We’re still looking into this issue. We’d like to have it fixed in Turris OS 3.10.4.

@Pepe any further updates on this one?

Not yet. I’ll tell you.

Discussion continues here: Data collection broken at project.turris.cz?