Constant WAN activity with strange IP addresses

Software SW help
1

Since a few weeks I’m running a raspberry-pi with domoticz to monitor my electricity usage at home.
Previous weekend I was at the OGP at the Nürburgring and staying with the oldtimer club at our usual hotel. Since a few years the hotel hosts free wifi from Freifunk. I wanted to monitor my home electrical usage by remote connection. But the problem is that Freifunk apparently does not allow you to use a VPN connection. When I try to make a VPN connection (I use NordVPN), it just won’t establish a connection. And when it does after a long wait, there is no connection to the internet. Only without VPN you have full access. So, I made a connection with my Pi at home a few times without VPN.
When I got home again I started looking at the firewall of my Omnia again to see if it could be improved. (only adding and removing port forwards)
I had full internet and router access at that time. But after a short time I lost all connection to the internet and router. No websites and no Foris or LuCI. The router was unreachable.
Even other computers that haven’t ran in weeks didn’t have a connection any more. Wifi or wired.

Now I connected a computer directly to the ISP modem/router and downloaded a medkit image from repo.turris.cz and reflashed my Omnia with the four led reflash option.
Now I had access again and got through the setup page again.
I have my Omnia running again. Only my SSD’s I can’t get to work again. Still working on that.

But I don’t know if I had this already before, but my WAN led keeps blinking with several seconds intervals.
So, I monitored the eth1 traffic with tcpdump and noticed a lot of traffic with a lot of strange IP-addresses from all over the world.
Here is a screen capture of some of the traffic:
https://www.walagata.com/w/nightwalker/Vreemde_IP_adressen.jpg
Some of the IP-addresses I looked up:
222.175.44.66 Qingdao, China
223.97.191.233 Qingdao, China
5.188.87.6 Moscow, Russia
217.31.192.69 Prague, Czechia
34.226.254.109 Ashburn, Washington, USA
185.209.0.10 Malpils, Latvia
77.72.82.23 Stoke-on-trent, UK
193.29.13.157 Romania
89.101.251.228 Dublin, Ireland
88.221.254.202 Switzerland
Why is my Omnia making connections with IP-addresses all over the world?
Does this mean my Omnia is hacked, even after a fresh software flash?

My Domoticz Pi does not seem to make WAN connections. I ssh into the pi and ran tcpdump but only get high ssh traffic between the pi and the pc I’m using. No other IP’s come up.
My laptop also gives no strange IP’s except for 217.31.192.69 at this moment of typing this post. But this IP is from Prague, Czechia, from Turris I presume. This forum.

Also the ‘history’ command doesn’t give any strange output on the pi or the laptop I’m using. (the laptop I also used at the hotel)
Except that every time I log out (ssh) of my Omnia the history of the Omnia gets deleted. When I log in and run some commands, they show up with the history command. But only as long as I stay logged in. After an exit it is all gone. Is this normal?

A while back I installed a 30GB SSD to store the persistent data, as advised, instead of using the 8GB internal flash.
Before that in LuCI the memory usage looked like:
https://www.walagata.com/w/nightwalker/Veel_geheugen.jpg
After installing the SSD it looked like:
https://www.walagata.com/w/nightwalker/Veel_geheugen02.jpg
But at the moment it looks like this:
https://www.walagata.com/w/nightwalker/Veel_geheugen03.jpg
I have no extra processes running and no LXC-containers (yet). Why is the memory usage at about 50%

I don’t know how to make a log file of all current processes, but here is a copy/paste txt-file of all the processes running at the moment.
https://www.walagata.com/w/nightwalker/Processes.txt
Does anyone see anything out of the ordinary?

2

Do you have data collection a/o haas proxy enabled? One of the ports from tcpdump is showing 23 (telnet) and that would be worrysome unless you have enabled it or it being utilized by data collection.

The screenshot of the tcpsump is not that helpful. Instead you should create a dumpfile and analyze the captured data with the tool like wireshark which then should provide more inisight. You might want to capture udp traffic as well.

3

Hello, thanks for the replay.

After the reflash I havn’t installed any software.
In LuCI at System -> Startup
Telnet is disabled
Collectd is diabled (I presume this means Collect Data)
And I can’t find anything of Haas. Or where should I look for it?

I had a few attempts at running tcpdump making a dumpfile but was not that successful. How do I make a dumpfile (with both tcp, udp) and where/how do I store it.

4

Had a second look at that tcmdump screenshot - there is quite some length 0 traffic showing, that incl. the chinese ip 222.175.44.66 on port 23 (telnet).
Thus likely it is just unsolicited traffic attempts (port scans etc) being blocked by the router firewall and with latter in the default mode reject this may account for the traffic captured in the screenshot.

As for the usage of tcpdump (options) I would suggest a web search, in needful perhaps in your native tongue.

5

Telnet is disabled at startup, so, port 23 cannot be used?

I managed to capture a dumpfile but I can’t find anywhere a description of capturing tcp and udp with one command.
I loaded the file into wireshark, but because I’m new to the program I don’t really know what to look for.
Maybe you can see something out of the ordinary in the dump file.
https://www.walagata.com/w/nightwalker/0002.pcap

6

suppose 192.168.178.1 is the ISP modem/router and 192.168.178.11 the TO router?

The dump does not show the ip of the TO router’s lan clients which makes it difficult to assert which traffic belongs solely to the TO router and which to its clients.

Having filtered out

  • 192.168.178.1 traffic
  • dns traffic
  • dhcp traffic
  • ntp traffic
  • arp traffic

looking then on the remaider of traffic and this looks legit

  • tcp (tls) traffic exchange with 52.85.255.173 | 34.226.254.109 (amazon)
  • tcp (tls) traffic exchange with 172.217.19.202 | 172.217.20.106 (google)
  • tcp (tls) traffic outbound to czech ip 217.31.192.69 (suppose that is statistic submission to Turris)

this looks questionable perhaps

  • tcp traffic exchange on port 5555 with ip 180.40.224.150 which would not be uncommon for softether VPN, or else https://www.speedguide.net/port.php?port=5555
  • tcp (plain) outbound to ip 185.209.0.10 (seems a bit of strange traffic being outbound only)

this looks suspicious rather

7

@Nightwalker : which portforwardings have you had active?

8

@anon50890781:
Yes, 192.168.178.1 is the ISP modem/router and 192.168.178.11 is the TO router.
The amazon and google ip’s/traffic look suspicious to me to. I don’t have accounts at neither of these companies. Amazon I rarely visit. Only if someone on a forum provides a link to a product on amazon, I only have a look at it.
From google I sometimes use translate, maps and youtube. But I almost never use the search engine google. I use startpage.com as primary search engine.
Also I use Firefox with the Ghostery plug-in. And except for a few sites I visit on a daily basis, like several forums and others, I use the normal Firefox browser to use cookies to remember login details. For all other sites I use Firefox private browsing.
So, why would there be traffic from amazon and google?

The dumpfile I made while doing nothing on the internet to get only the suspicious traffic.
The only LAN hosts connected at that time where my laptop (Xubuntu) and the Pi running Domoticz.
But I figured out a way to monitor a single LAN IP with tcpdump and it showed no traffic with my Pi. Only when I refreshed the Domoticz web interface on my laptop, that traffic showed up. Only the IP-addresses of the Pi and my laptop.

Because I flashed my Omnia with a new software image, there should be nothing left of anything that was on the old software. Is it possible that this traffic is generated by the devices from the outside IP’s still trying to connect?

@hadc:
I have the Domoticz Pi running now for about one and a half week.
To have access to the Domoticz web interface from WAN I saw in a description on a website I had to open port 8080. So, I opened that port only to the IP of the Pi and configured Domoticz to ask for a username and password.
Several days later I found a better description which pointed out that port 8080 is not a secure connection and for WAN connections you should use port 443 and open the web interface with https.
I removed the 8080 port forward and replaced it with the 443 port forward. And also only to the IP of the Pi.
This is the only port forward I have at this time.

9

google and amazon traffic can originate from everything nowadays (nearly all websites use resources from at least one).

you should not expose webinterfaces from local appliances to the internet, ever!

not unlikely that your device(s) joined a botnet.
use a vpn if you need to access your home network from the internet.

2 Likes
10

Google/Amazon is probably CDN, e.g. jquery

That is not a good plugin at all, similar to the WOT disaster or the more recent Mozilla bans 23 snooping Firefox extensions - gHacks Tech News

Nope, the dump is very clear that the traffic to those dubious ips is outbound generated from your LAN.

I would concur with @hadc’s assessment that one or more of your IoT devices has been enslaved by an agent with malicious intent.
There are even seach enginges such as https://www.shodan.io/ looking for IoT devices exposed to the web. Lot of those IoT devices are made/designed cheap with large profit margins in mind but more than often lacking proper security measures.

Whether you use http or https does not matter to the underlying securiy of Domotizc, TLS (port 443) just encrypts the traffic and thus is a (basic) barrier against eavesdropping (Man in the Middle attack) only.
I am not familiar however with the overall security concept of Domotizc.

For further investigation this forum is not likely to be your best best in regard to cleaning up and thus I would suggest to seek counsel from an IT pro in your local domain rather.

That is basically correct unless the malicious code took (permanent) residence in a place (e.g epom of the processor) that is not being touched by the flash of the router, though rather unlikely. More likely though is that one or more of the IoT devices are compromised and/or the Domotizc server app.

12

Than I’ll have to look at the Pi again. That’s the only computer that is running and connected to my Omnia 24/7.
I don’t have any IoT devices. I don’t even want then.
I even have WAN traffic when every pc is shut down. Only the Pi is running.

1 Like
13

If you got no IoT devices then why are you running a Domotizc server, which is apparently for IoT like the electricity meter you mentioned initially.

From the symptoms you mentioned it seems that Domotizc server got compromised but that is a rather vague diagnosis from this remote perspective.

Other options could be that the D server app is designed that way to collect telemetry data about their users and phoning home to those dubious ips. But in that context the outbound connection to a telnet (port 23) of an Egyptian mobile network provider does not make sense.

Or the source code of the D server is compromised without the knowledge of the developers. But then perhaps other users of that may have noticed and reported it already. Maybe worth checking in their forum nonetheless.

From what you wrote initially username and password for the D server could have been harvested when you were at that hotel and could not establish a vpn connection. Perhaps the D server at that time still running on port 8080 and thus traffic in plain text…

Perhaps destroy the current D server installation and even setup the system it is running on (pi ?) fresh just to make sure and then reinstall the D server and see if it sorts the issue.

And perhaps run the D server on port 443 only and not expose it to the WAN but LAN only.

14

The electricity meter in my house is no an IoT device. It is, what we call a “smart meter” because the power company is able to read it out by GSM signal. The only other way to connect to an other device is a RJ11 port on which the meter spits out the meter readings every 10 seconds. You can not send signals back. With a special USB decoder cable and Domoticz you can read out the meter and keep track of your electrical and gas usage. The gas meter is also an electronic one connected to the electricity meter. These meters are property of the power distribution company and are sealed. We, the users, are only able to read the meter readings of a display and ‘listen’ at the RJ11 port.

I had tcpdump make another dumpfile of all the traffic of the IP of my Pi.
Got a bit surprised. In a short time, 5 strange IP’s showed up.
Here’s the dumpfile:
https://www.walagata.com/w/nightwalker/0003.pcap
With at least these IP’s:
185.101.33.2 Oslo, Norway
154.155.3.21 Kenya
77.72.85.17 Sofia, Bulgaria
124.13.85.215 Petaling Jaya, Malaysia
193.29.13.25 Romania

After this I deleted the 443 port forward to my Pi and closed the bridge connection of my ISP modem/router.

Than I ran another tcpdump check. (I used a very old SD-card to get the dumpfiles off my Omnia, but it got corrupted and does not work any more.)
And over a longer period this was the output.
Only connecting to 224.0.0.251 once every ~5min. This IP doesn’t show up at an IP-checker. But this does not look like a normal IP-address.
https://www.walagata.com/w/nightwalker/tcpdump155.jpg
After a search on the internet I found out that this address is of the Avahi daemon which is installed by default.

I guess I indeed have to clean out the SD of the Pi and re-install the software.

Edit: I fixed the links. Made some typo’s

1 Like
15

The D server is turning the pi into an actual IoT device, not the meter. But you may interpret IoT differently and it is anyway beside the point of the thread,

From my understanding of Freifunk it is a public mesh of community wlan nodes acting as gateway to the public internet. Thus either the FF node been compromised or someone on the node might have been snooping traffic, or at some other point along the route.
It seems that the FF node blocked either the port NORD VPN utilises or the ip range of NVPN, though probably the former or for some other reason the NVPN connection did not materialise.

Suppose a lesson learned and hopefully you can remedy the situation with a fresh installation of the pi and subsequent installation of the D server.
Perhaps you report back the situation then.

If you still want to access the D server whilst roaming away from home you could install a VPN node on the TO router and on the roaming device and then from the roaming device connect securely to the TO router and thus the LAN.

16

I already looked up the way of operating of Freifunk and figured out that FF uses a VPN from the access point (at the hotel in my case) to their server(s) before going to the internet. The weak spot is the connection to the access point. If someone else acts like a FF access point, intercepts your data and then re-routing it to the real FF access point, the other person is able to read your data.
This is multiple times shown in actual practice in news programs here on Dutch television.
That’s why I wanted to use a VPN connection to have a secure connection with the access point. But it just wouldn’t work.
But because I already know this hotel for over 15 years now, and also the type of guests they house, I just risked it.
The hotel was also pretty empty at the time.
but yes, lesson learned.

I already tried to configure a private VPN to have a secure connection to my home network from the outside. But I haven’t succeeded yet. In a magazine I have about Pi’s there is an article how to put up such a VPN connection with a Pi. I tried it with a LXC container. That’s where they’re for, right? That way I don’t have to connect an extra Pi to the router. And I wanted to play with the LXC containers anyway.

17

The plain text traffic could have been intercepted also after having left the FF mesh. Even TLS traffic can be intercepted/read without the user noticing but that will hopefully soon change with the implementation of the new TLS v 1.3 standard.

Purpose of containers is basically to isolate/segregate content a/o services.

If you run a VPN node on the pi only then it would require some routing/forwarding/firewall configuration to access the LAN other than the pi only.

Else you could install a VPN node on the TO router.

18

Could that be done with the OpenVPN option in Foris?

19

Yep, that is one way to go about it

20

Oké, I’ll have a look into it soon.

I also had a second look at the Pi and how the store, where I bought this complete Pi-kit with Domoticz and the USB-cable to connect to the electricity meter, installed Domoticz on the included pre-installed SD-card.
With the ‘history’ command I can see all commands used at the CLI since first boot-up.
Compared to the install instructions on the Domoticz site it is somewhat different.
So, I have to look at this before formatting this SD-card and try to install it myself.

21

I don’t have much time to work at this since my vacation is over, but I’m cleaning out my computers of personal files before re-installing them.

I did some research again regarding this infection.
I took an old laptop and re-fitted it with an old HDD with XP which I didn’t used in several years.
I disconnected the Omnia and put my old Sitecom DC-202 router (not used in over a decade) behind my ISP modem/router. So, everything is isolated from the (infected) Omnia and the RPi with Domoticz. Downloaded new images of raspbian en installed one new uSD-card and booted an other RPi with it. Performed the update and upgrade.
On this website there are several commands to see if a system is infected.
https://www.welivesecurity.com/2014/02/21/an-in-depth-analysis-of-linuxebury/
On the new installed raspbian it says “infected” even when it has not come in contact with the infected machines.
https://www.walagata.com/w/nightwalker/Omnia/nieuwe_install.jpg
And an old installation in an other RPi which has been in contact with the Omnia still gives “clean”. Even after an update and a ,very long taking, upgrade.
Because the check commands are all described on websites with an submit date of several years old, I thought there would be some changes to the files which would give “false positives” by now. But the other RPi is still “clean”
https://www.walagata.com/w/nightwalker/Omnia/niet_geinfecteerd.jpg

Is this still a good way to determine if a system is infected or does it give “false positives” to quickly?
And if not, what is a good way to spot an infection?