Updater failed: /usr/share/updater/download/' failed: Read-only file system

Software SW bugs discussion
1

After automatic update to Turris OS 6.0.1 updater is failing.
Any hits? Thank you

Updater execution failed:
e[34mINFOe[0m:Target Turris OS: 6.0.1
e[34mINFOe[0m:Queue upgrade of libpython3/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-base/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-light/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-email/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-distutils/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-logging/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of pkglists/turrispackages/1.8.1-43[1.8.0-42]
e[34mINFOe[0m:Queue upgrade of base-files/core/1458-6.0.1-r16691+128-060aa0021f[1458-6.0-r16691+127-060aa0021f]
e[34mINFOe[0m:Queue upgrade of python3-multiprocessing/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of fix-updater-v65.0-alternatives-update/turrispackages/1-59[1-54]
e[34mINFOe[0m:Queue upgrade of turris-version/base/6.0.1[6.0]
e[34mINFOe[0m:Queue install of libpam/packages/1.5.1-1
e[34mINFOe[0m:Queue install of lighttpd-mod-auth/turrispackages/1.4.67-3
e[34mINFOe[0m:Queue install of lighttpd-mod-authn_pam/turrispackages/1.4.67-3
e[34mINFOe[0m:Queue install of turris-snapshots-web/turrispackages/16.0.0-4
e[34mINFOe[0m:Queue upgrade of python3-urllib/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-openssl/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-xml/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-unittest/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-sqlite3/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-readline/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-pydoc/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-ncurses/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-lzma/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-gdbm/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-decimal/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-dbm/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-ctypes/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-codecs/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-cgi/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-cgitb/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3-asyncio/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of python3/packages/3.9.15-3.9-1[3.9.14-3.9-1]
e[34mINFOe[0m:Queue upgrade of omnia-rainbow-backend/turrispackages/0.1.1-3[0.1.1-2]
e[34mINFOe[0m:Queue install of turris-diagnostics-web/turrispackages/16.0.0-4
e[34mINFOe[0m:Queue upgrade of foris-controller-storage-module/turrispackages/0.10.1-3.9-1[0.10.0-3.9-5]
e[34mINFOe[0m:Queue removal of fix-firewall-zone-limit
e[34mINFOe[0m:Queue removal of fix-network-devices
e[34mINFOe[0m:Queue removal of fix-remote-access-ca-permissions
e[34mINFOe[0m:Queue removal of fix-lighttpd-sync-with-upstream
e[34mINFOe[0m:Queue removal of fix-omnia-leds-migrate
e[34mINFOe[0m:Downloading packages
line not found
line not found
line not found
e[31;1mDIEe[0m:
[string "utils"]:175: [string "utils"]:169: kdimr '/usr/share/updater/download/' failed: Read-only file system
2

Another errors found regarding this issue:

[    4.012608] F2FS-fs (mmcblk0p1): Magic Mismatch, valid(0xf2f52010) - read(0x0)
[    4.019864] F2FS-fs (mmcblk0p1): Can't find valid F2FS filesystem in 1th superblock
[    4.027723] F2FS-fs (mmcblk0p1): Magic Mismatch, valid(0xf2f52010) - read(0x0)
[    4.034965] F2FS-fs (mmcblk0p1): Can't find valid F2FS filesystem in 2th superblock
[    4.042906] BTRFS: device fsid 1e54c5b8-60b4-4c1b-b06e-be445dd9afa4 devid 1 transid 39375 /dev/root scanned by swapper/0 (1)
[    4.054804] BTRFS info (device mmcblk0p1): disk space caching is enabled
[    4.061538] BTRFS info (device mmcblk0p1): has skinny extents
[    4.073385] BTRFS info (device mmcblk0p1): enabling ssd optimizations
[    4.092157] BTRFS warning (device mmcblk0p1): block group 12582912 has wrong amount of free space
[    4.101078] BTRFS warning (device mmcblk0p1): failed to load free space cache for block group 12582912, rebuilding it now
[    4.113814] BTRFS critical (device mmcblk0p1): corrupt leaf: block=203407360 slot=3 extent bytenr=19922944 len=4096 previous extent [19877888 168 360448] overlaps current extent [19922944 169 0]
[    4.131177] BTRFS error (device mmcblk0p1): block=203407360 read time tree block corruption detected
[    4.226915] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.244280] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.253465] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.286047] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.303409] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.312587] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.333793] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.351149] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.360323] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.394280] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.411640] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.420812] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.442938] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.460294] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.469468] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.482041] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.499389] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.508562] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.521106] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.538455] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.547627] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.560800] BTRFS critical (device mmcblk0p1): corrupt leaf: block=326955008 slot=2 extent bytenr=27795456 len=4096 previous extent [27631616 168 229376] overlaps current extent [27795456 169 2]
[    4.578148] BTRFS error (device mmcblk0p1): block=326955008 read time tree block corruption detected
[    4.587320] BTRFS warning (device mmcblk0p1): error accounting new delayed refs extent (err code: -5), quota inconsistent
[    4.598408] VFS: Mounted root (btrfs filesystem) on device 0:14.

[  203.118562] BTRFS info (device mmcblk0p1): disk space caching is enabled
[  203.118580] BTRFS error (device mmcblk0p1): Remounting read-write after error is not allowed
[  251.390429] BTRFS info (device mmcblk0p1): disk space caching is enabled
[  251.390442] BTRFS error (device mmcblk0p1): Remounting read-write after error is not allowed
[43447.447369] BTRFS info (device mmcblk0p1): disk space caching is enabled
[43447.447383] BTRFS error (device mmcblk0p1): Remounting read-write after error is not allowed

[string "utils"]:175: [string "utils"]:169: mkdir '/usr/share/updater/download/' failed: Read-only file system
Oct 27 02:24:13 SkyNET updater-supervisor: pkgupdate exited with: -6
Oct 27 02:24:13 SkyNET updater-supervisor: Traceback (most recent call last):
   File "/usr/bin/updater-supervisor", line 33, in <module>
    sys.exit(load_entry_point('svupdater==1.5.3', 'console_scripts', 'updater-supervisor')())
   File "/usr/lib/python3.9/site-packages/svupdater/__main__.py", line 104, in main
   File "/usr/lib/python3.9/site-packages/svupdater/_supervisor.py", line 143, in run
   File "/usr/lib/python3.9/site-packages/svupdater/notify.py", line 150, in changes
   File "/usr/lib/python3.9/pathlib.py", line 1252, in open
   File "/usr/lib/python3.9/pathlib.py", line 1120, in _opener
 OSError: [Errno 30] Read-only file system: '/usr/share/updater/.changelog.lastreport'
Oct 27 02:24:54 SkyNET netdata[5250]: BTRFS: failed to read '/sys/fs/btrfs/1e54c5b8-60b4-4c1b-b06e-be445dd9afa4/allocation/data/total_bytes'
Oct 27 02:25:01 SkyNET crond[6570]: (root) CMD (/usr/bin/notifier)
Oct 27 02:25:01 SkyNET crond[6569]: (root) CMDOUT (Working on message: 1666837453-6416)
Oct 27 02:25:01 SkyNET crond[6569]: (root) CMDOUT (There is no message to send.)
Oct 27 02:25:01 SkyNET crond[6569]: (root) CMDEND (/usr/bin/notifier)
Oct 27 02:25:56 SkyNET netdata[5250]: BTRFS: failed to read '/sys/fs/btrfs/1e54c5b8-60b4-4c1b-b06e-be445dd9afa4/allocation/data/total_bytes'
Oct 27 02:26:20 SkyNET odhcpd[2785]: A default route is present but there is no public prefix on lan thus we don't announce a default route!
Oct 27 02:26:58 SkyNET netdata[5250]: BTRFS: failed to read '/sys/fs/btrfs/1e54c5b8-60b4-4c1b-b06e-be445dd9afa4/allocation/data/total_bytes'
Oct 27 02:27:58 SkyNET netdata[5250]: BTRFS: failed to read '/sys/fs/btrfs/1e54c5b8-60b4-4c1b-b06e-be445dd9afa4/allocation/data/total_bytes'
Oct 27 02:29:00 SkyNET netdata[5250]: BTRFS: failed to read '/sys/fs/btrfs/1e54c5b8-60b4-4c1b-b06e-be445dd9afa4/allocation/data/total_bytes'
Oct 27 02:30:00 SkyNET netdata[5250]: BTRFS: failed to read '/sys/fs/btrfs/1e54c5b8-60b4-4c1b-b06e-be445dd9afa4/allocation/data/total_bytes'
Oct 27 02:30:01 SkyNET crond[7141]: (root) CMD (/usr/bin/notifier)
Oct 27 02:30:01 SkyNET crond[7140]: (root) CMDOUT (There is no message to send.)
Oct 27 02:30:01 SkyNET crond[7140]: (root) CMDEND (/usr/bin/notifier)
3

Workaround manually rollback to 5.4.4 via hardware button. Turn off automatic updates.

Since this is not permanent solution I send diagnostic logs to support. Didnt received any answer yet.

Any hints and tips are still wellcomed.

Thank you

4

Did your run btrfs scrub on the file system? Can be run in read-only mode with -r flag.

5 
btrfs scrub status /
UUID:             1e54c5b8-60b4-4c1b-b06e-be445dd9afa4
Scrub started:    Thu Nov  3 11:30:15 2022
Status:           finished
Duration:         0:00:27
Total to scrub:   1.10GiB
Rate:             41.79MiB/s
Error summary:    csum=51
  Corrected:      0
  Uncorrectable:  0
  Unverified:     0

Found also this in dmesg

[104266.111602] BTRFS warning (device mmcblk0p1): checksum error at logical 19922944 on dev /dev/mmcblk0p1, sector 38912: metadata leaf (level 0) in tree 648
[104266.111608] BTRFS warning (device mmcblk0p1): checksum error at logical 19922944 on dev /dev/mmcblk0p1, sector 38912: metadata leaf (level 0) in tree 648
[104266.111614] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 1, gen 0
[104266.111774] BTRFS warning (device mmcblk0p1): checksum error at logical 27267072 on dev /dev/mmcblk0p1, sector 53256: metadata leaf (level 0) in tree 720
[104266.121502] BTRFS warning (device mmcblk0p1): checksum error at logical 27267072 on dev /dev/mmcblk0p1, sector 53256: metadata leaf (level 0) in tree 720
[104266.121508] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
[104266.122965] BTRFS warning (device mmcblk0p1): checksum error at logical 27430912 on dev /dev/mmcblk0p1, sector 53576: metadata leaf (level 0) in tree 710
[104266.131391] BTRFS warning (device mmcblk0p1): checksum error at logical 27430912 on dev /dev/mmcblk0p1, sector 53576: metadata leaf (level 0) in tree 710
[104266.131398] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 3, gen 0
[104266.141341] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
[104266.151798] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 5, gen 0
[104266.162136] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 6, gen 0
[104266.162309] BTRFS warning (device mmcblk0p1): checksum error at logical 27529216 on dev /dev/mmcblk0p1, sector 53768: metadata leaf (level 0) in tree 643
[104266.172028] BTRFS warning (device mmcblk0p1): checksum error at logical 27529216 on dev /dev/mmcblk0p1, sector 53768: metadata leaf (level 0) in tree 643
[104266.172034] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 7, gen 0
[104266.182756] BTRFS warning (device mmcblk0p1): checksum error at logical 27348992 on dev /dev/mmcblk0p1, sector 53416: metadata leaf (level 0) in tree 720
[104266.182762] BTRFS warning (device mmcblk0p1): checksum error at logical 27348992 on dev /dev/mmcblk0p1, sector 53416: metadata leaf (level 0) in tree 720
[104266.182767] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 8, gen 0
[104266.193168] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 9, gen 0
[104266.195164] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 10, gen 0
[104266.225777] BTRFS warning (device mmcblk0p1): checksum error at logical 27541504 on dev /dev/mmcblk0p1, sector 53792, root 258, inode 708, offset 0, length 4096, links 1 (path: usr/sbin/sfdisk)
[104271.423713] scrub_handle_errored_block: 40 callbacks suppressed
[104271.423729] btrfs_dev_stat_print_on_error: 40 callbacks suppressed
[104271.423735] BTRFS error (device mmcblk0p1): bdev /dev/mmcblk0p1 errs: wr 0, rd 0, flush 0, corrupt 51, gen 0
root@SkyNET:~# btrfs scrub status
btrfs scrub status: exactly 1 argument expected, 0 given

That means 51 erorrs or so? Thanks

6

Solved by reflashing router with medkit.

7

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.