Space usage on the root fs

Software SW help
#1

It looks that after upgrading to TOS 5 I do miss free space on my sd card used as root FS.

Space usage:

# df -Ph
Filesystem                Size      Used Available Capacity Mounted on
/dev/mmcblk0p1            7.3G      6.5G    768.0M  90% /
devtmpfs                512.0K         0    512.0K   0% /dev
tmpfs                   512.0K         0    512.0K   0% /sys/fs/cgroup
tmpfs                  1009.8M     67.0M    942.8M   7% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda1                 9.0T      4.3T      4.2T  51% /mnt/nas
/dev/sdb1                 9.0T      5.5T      3.1T  64% /mnt/backup
/dev/sdc1                28.1G    595.1M     26.0G   2% /mnt/sdc1

Snapshots:

# schnapps list
    # | Type      | Size        | Date                      | Description
------+-----------+-------------+---------------------------+------------------------------------
  398 | post      |    13.53MiB | 2021-09-22 17:14:22 +0200 | Automatic post-update snapshot
  407 | time      |    13.48MiB | 2021-11-21 01:05:03 +0100 | Snapshot created by cron
  408 | time      |    13.48MiB | 2021-11-28 01:05:03 +0100 | Snapshot created by cron
  409 | pre       |    13.48MiB | 2021-12-01 17:56:45 +0100 | Automatic pre-update snapshot
  410 | post      |    13.49MiB | 2021-12-01 17:56:54 +0100 | Automatic post-update snapshot
  411 | time      |    13.49MiB | 2021-12-05 01:05:02 +0100 | Snapshot created by cron
  412 | pre       |    13.49MiB | 2021-12-08 17:31:00 +0100 | Automatic pre-update snapshot
  414 | pre       |    13.69MiB | 2021-12-08 20:17:24 +0100 | Automatic pre-update snapshot
  415 | post      |    16.88MiB | 2021-12-08 20:43:19 +0100 | Automatic post-update snapshot (TurrisOS 5.3.2)
  416 | pre       |    16.81MiB | 2021-12-09 00:22:50 +0100 | Automatic pre-update snapshot (TurrisOS 5.3.2)
  417 | post      |    16.77MiB | 2021-12-09 00:22:56 +0100 | Automatic post-update snapshot (TurrisOS 5.3.2)
  418 | time      |    16.88MiB | 2021-12-12 01:05:02 +0100 | Snapshot created by cron
  419 | pre       |    17.18MiB | 2021-12-15 13:40:10 +0100 | Automatic pre-update snapshot (TurrisOS 5.3.2)
  420 | post      |    16.89MiB | 2021-12-15 13:40:33 +0100 | Automatic post-update snapshot (TurrisOS 5.3.2)
  421 | time      |   748.33MiB | 2021-12-19 01:05:02 +0100 | Snapshot created by cron

And directories size in /

root:~# cd /
root:/# ls / | grep -v mnt | xargs du -sh | sort -h
0	dev
0	overlay
0	proc
0	sys
0	t-output
4.0K	boot.scr
4.0K	rom
4.0K	tisk
4.0K	var
8.0K	run
20.0K	srv
48.0K	root
1.3M	www
1.9M	bin
3.3M	boot
3.6M	sbin
5.4M	etc
11.5M	lib
67.0M	tmp
307.6M	usr

It looks most of the SD card size is used, but I have no clue what causes the usage. Or is that expected and I shall migrate to a larger sd card?
Thanks, ales

#2

try something like:

du -kx / |sort -nr|less

#3

Here is the output:

root@atom:~# du -kx / |sort -nr| head -30
344040	/
315008	/usr
240656	/usr/lib
80072	/usr/lib/python3.7
68800	/usr/lib/python3.7/site-packages
31196	/usr/bin
29236	/usr/lib/python3.7/site-packages/babel
28628	/usr/lib/python3.7/site-packages/babel/locale-data
25916	/usr/share
24036	/usr/lib/python2.7
20812	/usr/lib/opkg
20620	/usr/lib/opkg/info
16380	/usr/sbin
15972	/usr/lib/samba
13828	/usr/lib/python2.7/site-packages
11772	/lib
10992	/usr/lib/perl5/5.28
10992	/usr/lib/perl5
8936	/usr/share/nmap
8844	/lib/modules/4.14.254
8844	/lib/modules
8732	/usr/lib/python3.7/site-packages/reforis_static
7776	/usr/lib/python3.6
6492	/usr/lib/python2.7/site-packages/mx
5980	/usr/lib/python3.7/site-packages/twisted
5604	/usr/lib/python3.7/site-packages/reforis_static/reforis
5504	/etc
5340	/usr/lib/python3.7/site-packages/reforis_static/reforis/js
4804	/usr/share/postgresql
4568	/usr/lib/python3.6/site-packages

root@atom:~# df -Ph
Filesystem                Size      Used Available Capacity Mounted on
/dev/mmcblk0p1            7.3G      6.6G    758.5M  90% /
devtmpfs                512.0K         0    512.0K   0% /dev
tmpfs                   512.0K         0    512.0K   0% /sys/fs/cgroup
tmpfs                  1009.8M     72.1M    937.6M   7% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda1                 9.0T      4.3T      4.2T  51% /mnt/nas
/dev/sdb1                 9.0T      5.5T      3.1T  64% /mnt/backup
/dev/sdc1                28.1G    595.2M     26.0G   2% /mnt/sdc1
#4

this looks like a process has file opened on your root filesystem

lsof -n |grep DELETED

reboot should clear that up but I personally would like to know what process uses such file.

you can also remove some snapshots, I’d start with older time snapshots

1 Like
#5

Alternativ you can run

lsof -nP +L1

That will list processes with open files with linkcount less than 1 link
I.e. open a deleted file

But as mentioned - a reboot should clear that out

#6

Thanks:

root:~# lsof -n |grep DELETED
root:~# lsof -nP +L1
COMMAND  PID USER  FD   TYPE DEVICE SIZE/OFF NLINK NODE NAME
8       4176 root txt    REG    0,5    45283     0  961 /memfd:liblxc (deleted)

root:~# df -Ph
Filesystem                Size      Used Available Capacity Mounted on
/dev/mmcblk0p1            7.3G      6.6G    758.5M  90% /
devtmpfs                512.0K         0    512.0K   0% /dev
tmpfs                   512.0K         0    512.0K   0% /sys/fs/cgroup
tmpfs                  1009.8M     72.2M    937.5M   7% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda1                 9.0T      4.3T      4.2T  51% /mnt/nas
/dev/sdb1                 9.0T      5.5T      3.1T  64% /mnt/backup
/dev/sdc1                28.1G    595.2M     26.0G   2% /mnt/sdc1

reboot

root:~# df -Ph
Filesystem                Size      Used Available Capacity Mounted on
/dev/mmcblk0p1            7.3G      6.6G    758.4M  90% /
devtmpfs                512.0K         0    512.0K   0% /dev
tmpfs                   512.0K         0    512.0K   0% /sys/fs/cgroup
tmpfs                  1009.8M     65.1M    944.6M   6% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sdc1                28.1G    598.9M     26.0G   2% /mnt/sdc1
/dev/sdb1                 9.0T      5.5T      3.1T  64% /mnt/backup
/dev/sda1                 9.0T      4.3T      4.2T  51% /mnt/nas

I had to made a mistake while looking for the space usage, but can’t spot the fault :frowning:

#7

Aaaa, you probably had some glitch some time on the external drives.
That way you have ended up written data to the mountpoints(then pointing at the sd card)

Try to unmount all drives and run du again(add h flagg for human readable)