HaaS does not log [WORKAROUND]

Hi!

Since I get Dynamic Public IP and I am running a web server I supposed that there will be some activity in HaaS log. But it claims that there is 0 sessions so far. It’s 3rd month I am open to the internet and nothing. Was I so lucky or something is not working?

Screens from HaaS:

and my /etc/config/haas:
config haas ‘settings’
option local_port ‘2525’
option setup_fw ‘1’
option log ‘syslog’
option token ‘526XXXXXXXXXXXXXXXXXXXXXXXXXX569’

My token in the setting matches the one in HaaS web interface but there is nothing logged.
I did restart the device since I renew the token.

Any ideas how to debug some more? THX


TOS 3.10.1 brought also some changes to the firewall, if I am not mistaken, perhaps causing this. Apparently the ipt rules set by HaaS are not in play. Haven’t found a workaround yet.

Well, after a restart of the haas-proxy service is see following appear in the log:

2018-06-11 17:00:02 err server_uplink[]: Failed to download contract status
2018-06-11 17:00:34 err server_uplink[]: Failed to get registration code

2018-06-11 17:02:09 err turris-firewall-rules[]: (v63) Failed to download https://api.turris.cz/firewall/turris-ipsets.gz.sign

Maybe, this gives someone a clue.

Greetings.

Seems like there is bigger problem:

It must have break with the updates. I did not change configs since I switched to rc.
I do get rqandom connectivity loss due to weak signal. But even now when there is internet acces and I hit refresh there is an error.

Will try to search in logs for related lines and update.

EDIT:


Now firewall works but not uCollect (its enabled and started in services)

I did also get:
2018-06-11 17:52:53 info kresd[10461]: [ ta ] next refresh for . in 24 hours
2018-06-11 17:52:53 err kresd[10461]: [priming] cannot resolve address ‘a.root-servers.net.’, type: 1
2018-06-11 17:52:53 err kresd[10461]: [priming] cannot resolve address ‘a.root-servers.net.’, type: 28
2018-06-11 17:52:53 err kresd[10461]: [priming] cannot resolve address ‘b.root-servers.net.’, type: 1
2018-06-11 17:52:53 err kresd[10461]: [priming] cannot resolve address ‘b.root-servers.net.’, type: 28
[…] ascending letters untill m[…]
2018-06-11 17:52:53 err kresd[10461]: [priming] cannot resolve address ‘m.root-servers.net.’, type: 1
2018-06-11 17:52:53 err kresd[10461]: [priming] cannot resolve address ‘m.root-servers.net.’, type: 28
2018-06-11 17:52:53 err kresd[10461]: [priming] cannot resolve any root server address, next priming query in 10 seconds

but when I try it manually, it works:
root@turris:~# nslookup a.root-servers.net
nslookup: can't resolve '(null)': Name does not resolve
Name: a.root-servers.net
Address 1: 198.41.0.4 a.root-servers.net
Address 2: 2001:503:ba3e::2:30 a.root-servers.net

EDIT2:


After reboot it’s the other way around. Could it be related to connection quality?

EDIT3:


Now everything back online.

Yeah, like I thought so. There are separate issues I have now.

Not to mention DNS resolving self test… I did started here: DNS resolving self-test doesn't work [Partially solved]

Oww Omnia omnia so much fun with ya!

Right now I’m thinking about three issues, which can cause it.

  1. You don’t have public IPv4 address
  2. Your ISP is blocking port 22.
  3. You’re behind NAT.

In 1st issue it’s should be easy to test it. You said that you have web server, if it is running on the Omnia and it works from the outside then you can try to access the SSH remotely. Probably in your case it won’t work. So the issue could be that your ISP is blocking some ports or you’re behind NAT.

The issue apparead with TOS 3.10.1 and going to be fixed in the next release, no?

  1. I do have a public dynamic IP address. But it doesn’t not show on the interfaces page (see screen):

Overview page:

  1. ISP is not blocking that port I asked them to disable all filtering and I can ssh to the debian server in lxc container from outside (via VPN to avoid nat loopback). I forwarded ports in firewall settings to be able to do so and it works.

  2. I am not behind NAT. I used to be but I have solved it to be able to host web server. I do have IP directly on the wan interface and it’s public ip.

So none of the three ;(

EDIT:
After some try outs (Thanks! @Pepe) I think the issue it that my ssh server on the router is "Refusing Connection"s on port 22 instead of silently routing it to the honeypot on port 2525.

So what I’ve tried today:

  1. Unckecked “SSH Honeypot” in Foris, waited untill it sends a message that packages were uninstalled.
  2. Checked “SSH Honeypot” in Foris, waited untill it sends a message that packages were installed.
  3. root@turris:~# /etc/init.d/haas-proxy restart

and after a while it send back to console this:
/etc/rc.common: .: line 97: check_fw: not found
// line 97 in /etc/rc.common is . "$initscript"

somebody know which package provide check_fw?

EDIT:
I also get:
err nikola[]: (v42) turris firewall rules might not be active in syslog. Related?

Also when I nmap my public IP I do not get port 22 open.

The bug is still not fixed in TOS 3.10.2 Turris OS 3.10.2 released

I can confirm that I am running TOS 3.10.2 and HaaS still doesn’t work for me.

I’ve got the same errors in my logs. Additional pretty frequent entry is:

err nikola[]: (v42) turris firewall rules might not be active

The log entries are really annoying as I do not know if there is any problem or not. However HaaS seems to work - tested port 22.

Can I ask if someone has an option for ssh in foris? I mean here on data collection page:


EDIT:
Ok according to that page https://doc.turris.cz/doc/en/howto/ssh_honeypot I should have that option here but I don’t.

I think that honepot for telnet is working. I can see port 23 open by nmap. Will see tomorrow if it logs credentials in HaaS.

Haas is seperate from Data Collection and has not section in Foris

Yup you are right. I looked at /etc/init.d/haas-proxy and then I tried to force the firewall rules by setting:
uci -q set haas.settings.setup_fw=0
uci commit haas.settings.setup_fw
uci commit firewall
/etc/init.d/haas-proxy check_fw

but it doesn’t work as well. I have no more ideas what might be broken.

root@turris:~# cat /var/log/haas.log
2018-06-28 19:19:25+0200 [-] Log opened.
2018-06-28 19:19:25+0200 [-] twistd 16.0.0 (/usr/bin/python2.7 2.7.12) starting up.
2018-06-28 19:19:25+0200 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
2018-06-28 19:19:25+0200 [-] ProxySSHFactory starting on 2525
2018-06-28 19:19:25+0200 [-] Starting factory <haas_proxy.proxy.ProxySSHFactory instance at 0x1674460>

and I can see with netstat that haas-proxy is listening on port 2525. Seems like ssh server is not working for connections from outside or turris firewall rules are not applied to forward to honeypot or the problem in honeypot itself. I am not familiar with the implementation of that mechanism.

tcp port 2525 is the correct port for the HaaS socket to listen on. Traffic from tcp port 22 on your WAN ip is being forwarded to tcp port 2525 on your WAN ip.

The thing is that I do not have port 22 open to the public:
PORT STATE SERVICE
23/tcp open telnet
53/tcp open domain
80/tcp open http
443/tcp open https
587/tcp open submission
993/tcp open imaps
995/tcp open pop3s
2323/tcp open 3d-nfsd
3128/tcp open squid-http
8080/tcp open http-proxy

That is the point of HaaS, tcp traffic on port 22 is just being forwarded (as in redirected) to port 2525. At latter HaaS is then ingressing the traffic into the honeypot and letting the agents doing their probing to a certain extent and collecting the data you then can peruse on the HaaS website.

OK I found a reason and temporary solution. So in short why my haas didn’t work:
In /etc/init.d/haas-proxy in check_fw function there is a variable
WAN_IP="ubus call network.interface.wan status | sed -n 's|.*address":[[:blank:]]*"\([0-9.]*\)".*|\1|p'"
and it returns no value for me. Thats why firewall rules are not applied.

I know its not good solution but I found a workaround to set WAN_IP to:
WAN_IP="curl myip.dnsomatic.com" in my temporary /etc/init.d/haas-proxy_TEST.

after restart of that temporary test service I get port 22 open when I scan my IP with nmap and I could login to honeypot like that:
areyouloco@frank:~ $ ssh root@mydomain.tld
root@mydomain.tld's password:
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
root@svr04:~# su
root@svr04:~# cat /etc/*version
Noneroot@svr04:~# timed out waiting for input: auto-logout
Connection to haas-app.nic.cz closed.
Connection to mydomain.tld closed

So the main reason is that I do not get an ip address in statistics of the wan interface I think it’s because I am using not so common protocol to connect to the internet (described here: [Solved] Huawei E3372s-153 in NCM mode not working).

I think it might be the main reason for my other issues. I am sending my wishes to the TurrisTeam to help me find the exact issue. Willing to post logs, debug, break stuff. Thx!

EDIT:
Will post some results after few login attempts from the https://haas.nic.cz/ after it (hopefully!?) updates logged credentials.

YEAH! I got my first 3 sessions logged! So I know the workaround is working. So @Pepe it was reason no.1 but kinda hidden. i do get public IPv4 but it’s not shown in interface statistics.