What is happening with HaaS?

[jada4p]

As I stated on Jan 26, 2022,

Summary

([quote=“jada4p, post:14, topic:16573, full:true”]
MOX classic 0.5 GB, 2x WiFi, HBS branch, Sentinel, Haas, simple config. Reboot problem persisting, HaaS not working. Other OK.
TO 2GB, 2x WiFi, HBK branch, Sentinel, HaaS, RIPE Atlas, simple config. HaaS not working. Other OK.
[/quote])

HaaS seems not to work (at least for me)… Latest entry were on 2021-12-30 for my TO (on HBK branch, i.e. up to date on daily changes), or on 2022-01-19 for my MOX (on HBS branch, i.e. on stable version of TOS this ntime).

Even though I changed HaaS token for both devices (more times, to be sure), nothing has changed, i.e. no new data.

What is happening with HaaS?

[DIKKEHENK]

Working here, but HaaS sometimes randomly stops, mostly after phyton errors.
It also has trouble here to start after a reboot, both on MOX and TO2016 ( latest OS, simple setup and config )

What i did was to add this cron job in Luci> System> scheduled tasks :
0 */6 * * * /etc/init.d/haas-proxy reload

so, if it for whatever reason stops running, this will reload/start it every 6 hours ( again)

[jada4p]

Tried restarting haas-proxy more times, as well as changing HaaS token, nothing helps, HaaS is not working.

[hagrid]

Hm, can you try to uninstall it and install it again?

[jada4p]

Unfortunately it didn’t help. HaaS still not working, last entry: 2021-12-30.
(TO 2GB, 2x WiFi, HBK branch - i.e. now at ver. 5.3.6, Sentinel, HaaS, RIPE Atlas, simple config)
Do you need any more info?

[DIKKEHENK]

Does the process run? ( luci>status>processes )

[jada4p]

Yes, it does. . .

[Pepe]

Mine HaaS on router works as it should.

You can also verify if you can connect to the honeypot locally.

[jada4p]

I can ping honeypot:

ping

root@TOjp:~# ping haas.nic.cz
PING frontend.labs.nic.cz (217.31.192.130) 56(84) bytes of data.
64 bytes from frontend.labs.nic.cz (217.31.192.130): icmp_req=1 ttl=55 time=15.5 ms
64 bytes from frontend.labs.nic.cz (217.31.192.130): icmp_req=2 ttl=55 time=11.7 ms
64 bytes from frontend.labs.nic.cz (217.31.192.130): icmp_req=3 ttl=55 time=10.2 ms
64 bytes from frontend.labs.nic.cz (217.31.192.130): icmp_req=4 ttl=55 time=9.74 ms
64 bytes from frontend.labs.nic.cz (217.31.192.130): icmp_req=5 ttl=55 time=10.2 ms
64 bytes from frontend.labs.nic.cz (217.31.192.130): icmp_req=6 ttl=55 time=10.9 ms
64 bytes from frontend.labs.nic.cz (217.31.192.130): icmp_req=7 ttl=55 time=10.2 ms
^C
— frontend.labs.nic.cz ping statistics —
7 packets transmitted, 7 received, 0% packet loss, time 6006ms
rtt min/avg/max/mdev = 9.748/11.261/15.568/1.861 ms

but on HaaS results page there isn’t anything new:

[Pepe]

I was not talking about pinging the honeypot, but try to connect to your’s honeypot locally and form outside of your network. Ref:

Turris Omnia - HaaS has not been working for me for a long time now

For me, I’d say it is not clear if you have a public IPv4 address or not. You can try connecting to running honeypot on your router using your local network and from outside your local network. Steps: Honeypot as a Service is running on your router on port 2525, so you can try to connect to it to verify to ensure that it is indeed working. (ssh test@ipaddress -p 2525) Try to connect from any public network to the SSH honeypot on your router by using the assigned IPv4 address to your route…

[Skippi]

I had similar problem - “haas” process was running, but no new entries on https://haas.nic.cz/device.
And connection to port 22 was not working.
I had to restart “haas” manually:

/etc/init.d/haas-proxy stop
/etc/init.d/haas-proxy start

Everything is OK now.

[jada4p]

I’am not able to imagine where the problem could be… today’s situation is like following

Summary

root@TOjp:~# ssh test@62.245.112.12 -p 2525
ssh: connect to host 62.245.112.12 port 2525: Connection refused
root@TOjp:~# /etc/init.d/haas-proxy stop
root@TOjp:~# /etc/init.d/haas-proxy disable
root@TOjp:~# /etc/init.d/haas-proxy enable
root@TOjp:~# /etc/init.d/haas-proxy start
root@TOjp:~# ssh test@62.245.112.12 -p 2525
ssh: connect to host 62.245.112.12 port 2525: Connection refused

[Nones]

@Skippi

I had to restart “haas” manually:
/etc/init.d/haas-proxy stop /etc/init.d/haas-proxy start

“/etc/init.d/haas-proxy restart” works fine too

[Skippi]

Can you check iptables rules?

root@turris:~# iptables -t nat -S|grep 2525
-A zone_wan_prerouting -p tcp -m tcp --dport 22 -m comment --comment “!sentinel: HaaS proxy port redirect” -j REDIRECT --to-ports 2525


And has your Turris really public IPv4 address?

[jada4p]

AFAIK all is OK:

I’ll try one more thing - to remove device TOjp from HaaS and create it again…

Edit: unfortunately neither this change didn’t help: “No sessions in specified interval.” (which was empty).

I wonder whether there is anything more what I could change/test…

[Skippi]

Is “haas” process (visiable as “python3”) listening on port 2525 and match “device token” configuration on https://haas.nic.cz/device/?

root@turris:~# netstat -nlp|grep :2525
tcp 0 0 0.0.0.0:2525 0.0.0.0:* LISTEN 32186/python3
root@turris:~# ps -ef|grep 32186
root 9365 9083 0 22:08 pts/0 00:00:00 grep 32186
root 32186 1 0 Feb19 ? 00:00:41 python3 -m haas_proxy --pidfile=/var/run/haas-proxy.pid --nodaemon haas_proxy --log-level=warning –device-token=67… --port=2525


[jada4p]

Thanks to your hints I found that there is no HaaS process running. Only after this I searched system log for any message regarding HaaS… Sorry it didn’t come to my mind sooner: I found source of my trouble with HaaS.

It’s, as in most cases, user fault - my fault: instead of specifying log file in HaaS config, I errorneously set log directory thus HaaS started but crashed on this error.

After correcting HaaS config and restarting haas-proxy there was no error message and haas process is running… and there are new sessions displayed on HaaS My Honeypot device page for my Omnia.

Mea culpa

[WayOutWest]

I have the same thing occur randomly. After a time, it just starts logging again. I just noticed that the last entries on my account are from September 2021.

[jada4p]

As far as HaaS could sometimes stop working, I created small script which checks whether HaaS (haas-proxy) is working, and in case not, restart it. This script could be run by cron (with option -s and maybe -l)… I’d like to publish it for anyone use… Feel free to use/modify it as you like

checkHaaS.sh

# checkHaaS.sh        (c) jada4p                  v2 20220222

# check whether HaaS is running, restart it when not

SILENT=NO
LOG=NO

LOGFILE=`awk -v FS="\'" '/log / { print $2 }' /etc/config/haas`
#LEVEL=`awk -v FS="\'" '/log_level / { print $2 }' /etc/config/haas`
#TOKEN=`awk -v FS="\'" '/token / { print $2 }' /etc/config/haas`
#echo LOGFILE=$LOGFILE
#echo LEVEL=$LEVEL
#echo TOKEN=$TOKEN

for i
do
case $i in
-i) # -----------------------------------help
    echo "checkHaaS.sh check whether HaaS is running, restart it when not"
    echo "checkHaaS.sh -s silent - no runtime comments"
    echo "checkHaaS.sh -l log actions"
    exit
    ;;
-s) # -----------------------------------silent
    SILENT=YES
    ;;
-l) # -----------------------------------log
    LOG=YES
    if [ "$LOGFILE" = "" ]
      then LOGFILE="/var/log/messages"
    fi
    ;;
*)  echo "invalid parameter(s), aborted"
    /root/checkHaaS.sh -i
    exit
    ;;
esac
done

# ---------------------------------------------------------check proxy pid
if [ "$SILENT" = "NO" ]
  then echo "Checking if exist haas-proxy PID"
fi
if [ ! -s /var/run/haas-proxy.pid ]
  then if [ "$SILENT" = "NO" ]
         then echo "HaaS is not running, no haas-proxy PID, starting it"
       fi
       /etc/init.d/haas-proxy reload
       /etc/init.d/haas-proxy start
       if [ "$LOG" = "YES" ]
         then DATE=`date +"%Y-%m-%d %T %Z"`
              MSG="checkHaaS: no PID found, HaaS restarted"
              echo "$DATE $MSG" >> $LOGFILE
       fi
  else if [ "$SILENT" = "NO" ]
         then echo "HaaS is running, haas-proxy PID found"
       fi
fi
# ---------------------------------------------------------check process
if [ "$SILENT" = "NO" ]
  then echo "Checking if there is HaaS process"
fi
sleep 10         # for some reason PID file is not found when testing
                 # immediately after restart
if [ $(ps `cat /var/run/haas-proxy.pid` | grep token | wc -l) -ne 1 ]
  then if [ "$SILENT" = "NO" ]
         then echo "HaaS is not running, starting it"
       fi
       /etc/init.d/haas-proxy reload
       /etc/init.d/haas-proxy start
       if [ "$LOG" = "YES" ]
         then DATE=`date +"%Y-%m-%d %T %Z"`
              MSG="checkHaaS: no process found, HaaS restarted"
              echo "$DATE $MSG" >> $LOGFILE
       fi
  else if [ "$SILENT" = "NO" ]
         then echo "HaaS is running"
       fi
fi

Note there are some commented lines on beginning of script, they were used for debugging and left for possible future use. The script is supposed to reside in /root directory.

[gsamu]

I too have experienced problems with nothing being logged to the HaaS site for my Turris Omnia. Some details:

Device Turris Omnia
reForis version 1.1.2
Turris OS version 5.3.5
Turris OS branch HBS
Kernel version 4.14.264

I have generated a token on haaz.nic.cz and specified the token in reForis → Sentinel → HaaS. I’ve tried multiple times generating a new token and setting that. But to no avail, as I continue to get the message “No sessions in specified interval.” on haas.nic.cz.

Some more data:

1. The haas-proxy is definitely running on my Omnia:

   root@turris:/etc/config# ps -ef |grep haas
   root 18297 1 0 08:17 ? 00:00:01 python3 -m haas_proxy --pidfile=/var/run/haas-proxy.pid --nodaemon haas_proxy --log-level=debug --device-token=3bf… --port=2525

2. The haas-proxy is listening on port 2525

   root@turris:/etc/config# netstat -nlp |grep :2525
   tcp 0 0 0.0.0.0:2525 0.0.0.0:* LISTEN 18297/python3

3. The firewall rule appears to be set correctly.

   root@turris:/etc/config# iptables -t nat -S|grep 2525
   -A zone_wan_prerouting -p tcp -m tcp --dport 22 -m comment --comment “!sentinel: HaaS proxy port redirect” -j REDIRECT --to-ports 2525

4. With the log class set to “debug” in the script /etc/init.d/haas-proxy I see the following in the system logs:

   root@turris:/tmp/log# grep haas *
   messages:Mar 3 13:17:19 turris haas-proxy-start[18297]: 2022-03-03T08:17:19 INFO twisted twistd 19.10.0 (/usr/bin/python3 3.7.12
   ) starting up.
   messages:Mar 3 13:17:19 turris haas-proxy-start[18297]: 2022-03-03T08:17:19 INFO twisted reactor class: twisted.internet.epollre
   actor.EPollReactor.
   messages:Mar 3 13:17:19 turris haas-proxy-start[18297]: 2022-03-03T08:17:19 INFO twisted ProxySSHFactory starting on 2525
   messages:Mar 3 13:17:19 turris haas-proxy-start[18297]: 2022-03-03T08:17:19 INFO twisted Starting factory <haas_proxy.proxy.Prox
   ySSHFactory object at 0x24…>

I’m definitely not ruling out any user error at this stage. I once upon a time had the HaaS working and could see many attempts being made. Is there any other information that I can provide here to help debug this?