Normally there’s nothing in /srv.
root@turris:/# btrfs filesystem du -s /
Total Exclusive Set shared Filename17.62GiB 17.28GiB 349.79MiB /
root@turris:/# df -h
Filesystem Size Used Available Use% Mounted on
/dev/mmcblk0p1 7.3G 6.7G 731.9M 90% /
tmpfs 1010.8M 5.4M 1005.3M 1% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
/dev/sda 931.5G 17.3G 913.2G 2% /srv
root@turris:/# btrfs filesystem df /
System, single: total=4.00MiB, used=4.00KiB
Data+Metadata, single: total=7.28GiB, used=6.56GiB
GlobalReserve, single: total=136.00MiB, used=0.00B
I don’t understand the different size of “/”
What about subvolid=257 ?
/dev/sda is mounted at subvolid=257, but btrfs subvolume list / don’t show this subvolid.
root@turris:/# mount -l
/dev/mmcblk0p1 on / type btrfs (rw,noatime,ssd,space_cache,commit=5,subvolid=420,subvol=/@)
proc on /proc type proc (rw,noatime)
sysfs on /sys type sysfs (rw,noatime)
none on /sys/fs/cgroup type cgroup (rw,relatime,cpuset,cpu,cpuacct,blkio,memory,devices,freezer,net_cls,pids)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
tmpfs on /dev type tmpfs (rw,relatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=000)
debugfs on /sys/kernel/debug type debugfs (rw,noatime)
/dev/sda on /srv type btrfs (rw,noatime,nodatasum,nodatacow,ssd,space_cache,subvolid=257,subvol=/@) [srv]
mountd(pid2579) on /tmp/run/mountd type autofs (rw,relatime,fd=5,pgrp=2577,timeout=60,minproto=5,maxproto=5,indirect)
root@turris:/# btrfs subvolume list /
ID 258 gen 1186 top level 5 path @factory
ID 260 gen 31956 top level 5 path certbackup
ID 280 gen 31983 top level 5 path @33
ID 413 gen 32352 top level 5 path @145
ID 420 gen 33123 top level 5 path @
ID 422 gen 32803 top level 5 path @154
ID 423 gen 33063 top level 5 path @155
/dev/sda is a different physical drive and thus different btrfs instance than the one for /.
Thanks, I thought subvolid257 must be shown, because it was mounted under /.
Thats OK:
root@turris:/mnt/TO-Backup# btrfs subvolume list /srv
ID 257 gen 13103 top level 5 path @
The reason I mentioned unmounting it above is because I thought there may be a bug that caused it to take into account the size of sda/srv as well as internal storage. But that doesn’t seem to be the case.
Honestly I’m not sure what’s going on, it doesn’t seem like the snapshots are to blame. Maybe try deleting all of the older snapshots, just to see what happens?
Bad blocks would cause the overall storage size to decrease, right?
schnapps list
# | Type | Size | Date | Description------±----------±------------±----------------------------±-----------------------------------
ls: /mnt/.snapshots/*.info: No such file or directory
root@turris:/# btrfs subvolume list /
ID 258 gen 1186 top level 5 path @factory
ID 260 gen 33134 top level 5 path certbackup
ID 424 gen 33408 top level 5 path @
root@turris:/# df -h
Filesystem Size Used Available Use% Mounted on
> /dev/mmcblk0p1 7.3G 6.6G 817.0M 89% /
tmpfs 1010.8M 6.6M 1004.2M 1% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
/dev/sdb1 58.6G 5.4G 50.1G 10% /mnt/TO-Backup
/dev/sda 931.5G 17.3G 913.2G 2% /srv
/dev/sdb1 58.6G 5.4G 50.1G 10% /tmp/run/mountd/sdb1
How large is the free memory space on “/” at factory release ?
Bad Blocks ?
Should I find them there: ?
root@turris:/# btrfs filesystem df /
System, single: total=4.00MiB, used=4.00KiB
Data+Metadata, single: total=7.28GiB, used=6.48GiB
GlobalReserve, single: total=136.00MiB, used=0.00B
The @factory subvolume contains the factory files, so as you can see above it’s around 103MB.
Could you run:
btrfs fi usage /
root@turris:/# btrfs fi usage /
Overall:
Device size: 7.28GiB
Device allocated: 7.28GiB
Device unallocated: 0.00B
Device missing: 0.00B
Used: 6.48GiB
Free (estimated): 681.02MiB (min: 681.02MiB)
Data ratio: 1.00
Metadata ratio: 1.00
Global reserve: 136.00MiB (used: 0.00B)Data+Metadata,single: Size:7.28GiB, Used:6.48GiB
/dev/mmcblk0p1 7.28GiBSystem,single: Size:4.00MiB, Used:4.00KiB
/dev/mmcblk0p1 4.00MiBUnallocated:
/dev/mmcblk0p1 0.00B
From this post :
Explanation :
https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-balance#ENOSPC
try btrfs balance start -dusage=0 -musage=0 /
btrfs balance start -dusage=0 -musage=0 /
Done, had to relocate 0 out of 12 chunks
root@turris:/# df -h
Filesystem Size Used Available Use% Mounted on
/dev/mmcblk0p1 7.3G 6.6G 804.8M 89% /
tmpfs 1010.8M 7.7M 1003.0M 1% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
/dev/sdb1 58.6G 5.5G 50.1G 10% /mnt/TO-Backup
/dev/sda 931.5G 17.3G 913.2G 2% /srv
/dev/sdb1 58.6G 5.5G 50.1G 10% /tmp/run/mountd/sdb1
Nothing has changed.
You don’t happen to have any hidden files/directories in root, do you? That’s the only thing left that I can think of. Try:
btrfs fi du -s /.*
root@turris:/# btrfs fi du -s /.*
Total Exclusive Set shared Filename17.62GiB 17.28GiB 349.79MiB /.
17.62GiB 17.28GiB 349.79MiB /…
0.00B 0.00B 0.00B /.rnd
Try running a full re-balance - this will most likely take a long time, but may be the only way to “reclaim” the lost blocks:
btrfs balance start /
No space left on device 
root@turris:~# df -h
Filesystem Size Used Available Use% Mounted on
/dev/mmcblk0p1 7.3G 6.6G 804.8M 89% /
tmpfs 1010.8M 10.5M 1000.3M 1% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
/dev/sdb1 58.6G 5.5G 50.0G 10% /mnt/TO-Backup
/dev/sda 931.5G 17.3G 913.2G 2% /srv
/dev/sdb1 58.6G 5.5G 50.0G 10% /tmp/run/mountd/sdb1
root@turris:~# btrfs balance start /
WARNING:
Full balance without filters requested. This operation is very
intense and takes potentially very long. It is recommended to
use the balance filters to narrow down the scope of balance.
Use ‘btrfs balance start --full-balance’ option to skip this
warning. The operation will start in 10 seconds.
Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting balance without any filters.
ERROR: error during balancing ‘/’: No space left on device
There may be more info in syslog - try dmesg | tail
root@turris:~# dmesg | tail
[245573.150192] BTRFS info (device mmcblk0p1): quota is enabled
[288770.557706] BTRFS info (device mmcblk0p1): quota is enabled
[305869.215475] BTRFS info (device mmcblk0p1): quota is enabled
[331967.771943] BTRFS info (device mmcblk0p1): quota is enabled
[351858.150668] nf_conntrack: automatic helper assignment is deprecated and it will be removed soon. Use the iptables CT target to attach helpers instead.
[375164.948395] BTRFS info (device mmcblk0p1): quota is enabled
[392263.369690] BTRFS info (device mmcblk0p1): quota is enabled
[418361.586187] BTRFS info (device mmcblk0p1): quota is enabled
[420566.975051] BTRFS info (device mmcblk0p1): relocating block group 7109345280 flags 5
[420567.078898] BTRFS info (device mmcblk0p1): relocating block group 7109345280 flags 5
This pretty much result you need. It clearly tells you that single snapshot takes over 6G of storage. That means that you have some data stored there that account to that size.
You might not discover it if you mounted over them (/srv is hot candidate here).
I would suggest you to mount current root to some directory (mount -o subvol=@ /dev/mmcblk0p1 /mnt) and repeat search for big files. First path I would look in to is /mnt/srv. You can remove those big files and after that also snapshots that contain those big files.
2 Likes
mount -o subvol=@ /dev/mmcblk0p1 /mnt
Filesystem Size Used Available Use% Mounted on
/dev/mmcblk0p1 7.3G 6.6G 804.8M 89% /
tmpfs 1010.8M 10.6M 1000.2M 1% /tmp
tmpfs 512.0K 0 512.0K 0% /dev
df: /mnt/TO-Backup: No such file or directory
/dev/sda 931.5G 17.3G 913.2G 2% /srv
/dev/sdb1 58.6G 5.5G 50.0G 10% /tmp/run/mountd/sdb1
/dev/mmcblk0p1 7.3G 6.6G 804.8M 89% /mnt
root@turris:/# umount /dev/sda
umount: /srv: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
root@turris:/# umount /srv
umount: /srv: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).
I don’t know, whats going on.
You say, I can delete now this directorys ?
root@turris:/mnt/srv# ls -l
drwxr-xr-x 1 root root 20 Feb 27 2019 lxc
drwxr-xr-x 1 root root 12 Mar 14 2019 storage