Read only filesystem

Dobry den,

mam Omniu a z casu na cas, niekedy raz za tyzden, niekedy castejsie sa file system prepne do read-only rezimu. Je zaujimave, ze nikdy nepomoze prvy restart ale az druhy, niekedy treti.
Vzdy po restarte je /var/log/messages prazdny, takze neviem ani pozriet. Teraz som si spravil cron job, ktory bude kazdy pol hodinu posielat na ssd disk, ktory mam pripojeny tieto subory.

Kazdopadne tusi niekto, co by to mohlo byt? Je to strasne otravne, pretoze vzdy ked sa to stane, tak prestane fungovat internet. VPN pripojenie funguje.

dakujem

Takto se mi projevovala umírající eMMC. :frowning:

To neznie dobre, mam na routeri LXC kontajner, ale kvoli tomu som instaloval do routera SSD disk:

Rad by som sa este opytal ohladne casteho zapisu na eMMC a suboru /var/log/messages, do ktoreho sa zapisuje relativne casto. Neda sa tento subor presmerovat na SSD?

podla toho co som cital od @Pepe, tak eMMC by malo byt moznost vymenit. Otazka je, ze ci je to teda mozne a kolko by vymena stala:

From both outputs of the serial console, I can see both eMMC are dead, which means all of your data is gone, and we can’t recover them. We have soldered eMMC on board, and it’s not easy to replace. It is almost impossible to swap it without expensive equipment. In our case, hopefully, the repair will be done by the 3rd company, and it will be paid repair because this is not what we can cover by warranty as it’s not manufacturers fault.

Toto som nasiel v logu turrisu. Mohol by mi prosim niekto z timu odpovedat? Dakujem

2020-01-17 00:00:02 err server_uplink[]: Failed to download contract status
2020-01-17 00:00:02 info updater-supervisor[]: Suspending updater start for 4182 seconds
2020-01-17 00:00:03 info nikola[]: (v43.1) recognized WAN interfaces: eth1, lo
2020-01-17 00:00:31 info nikola[]: (v43.1) Establishing connection took 0.690866 seconds
2020-01-17 00:00:31 info nikola[]: (v43.1) Logrotate took 0.005656 seconds
2020-01-17 00:00:31 info nikola[]: (v43.1) Syslog parsing took 0.055268 seconds
2020-01-17 00:00:32 info nikola[]: (v43.1) Session init took 1.053973 seconds
2020-01-17 00:00:32 info nikola[]: (v43.1) Records parsed: 59
2020-01-17 00:00:32 info nikola[]: (v43.1) Records after filtering: 59
2020-01-17 00:00:32 info nikola[]: (v43.1) Records filtering took 0.011330 seconds
2020-01-17 00:00:32 info nikola[]: (v43.1) {'msg': 'Data were inserted into work queue.'}
2020-01-17 00:00:32 info nikola[]: (v43.1) Sending records took 0.376873 seconds
2020-01-17 00:01:01 info /usr/sbin/cron[21764]: (root) CMD ((cat /tmp/nethist.stats | sort ; echo "uptime = $(cat /proc/uptime | cut -d. -f1)" ) | logger -p info -t nethist ; rm /tmp/nethist.stats)
2020-01-17 00:01:01 info /usr/sbin/cron[21761]: (root) CMD (/root/bin/guest-wifi-led.sh)
2020-01-17 00:01:01 info /usr/sbin/cron[21766]: (root) CMD (/usr/bin/rainbow_button_sync.sh)
2020-01-17 00:01:01 info /usr/sbin/cron[21763]: (root) CMD (/root/bin/openvpn-led.sh)
2020-01-17 00:01:01 info nethist[]: fs_avg = 87952.109034268
2020-01-17 00:01:01 info nethist[]: fs_max = 112827868
2020-01-17 00:01:01 info nethist[]: fs_min = 80
2020-01-17 00:01:01 info nethist[]: fs_samples = 1284
2020-01-17 00:01:01 info nethist[]: fs_sum = 112930508
2020-01-17 00:01:01 info nethist[]: last_timestamp = 1579215597
2020-01-17 00:01:01 info nethist[]: load_avg = 0.16180685358255
2020-01-17 00:01:01 info nethist[]: load_max = 1.36
2020-01-17 00:01:01 info nethist[]: load_min = 0
2020-01-17 00:01:01 info nethist[]: load_samples = 1284
2020-01-17 00:01:01 info nethist[]: load_sum = 207.76
2020-01-17 00:01:01 info nethist[]: mem_buffers_avg = 1709023.834891
2020-01-17 00:01:01 info nethist[]: mem_buffers_max = 1785352
2020-01-17 00:01:01 info nethist[]: mem_buffers_min = 1686164
2020-01-17 00:01:01 info nethist[]: mem_buffers_samples = 1284
2020-01-17 00:01:01 info nethist[]: mem_buffers_sum = 2194386604
2020-01-17 00:01:01 info nethist[]: mem_cached_avg = 2484
2020-01-17 00:01:01 info nethist[]: mem_cached_max = 2484
2020-01-17 00:01:01 info nethist[]: mem_cached_min = 2484
2020-01-17 00:01:01 info nethist[]: mem_cached_samples = 1284
2020-01-17 00:01:01 info nethist[]: mem_cached_sum = 3189456
2020-01-17 00:01:01 info nethist[]: mem_free_avg = 1490684.4080997
2020-01-17 00:01:01 info nethist[]: mem_free_max = 1701788
2020-01-17 00:01:01 info nethist[]: mem_free_min = 1427748
2020-01-17 00:01:01 info nethist[]: mem_free_samples = 1284
2020-01-17 00:01:01 info nethist[]: mem_free_sum = 1914038780
2020-01-17 00:01:01 info nethist[]: mem_total_avg = 2070032
2020-01-17 00:01:01 info nethist[]: mem_total_max = 2070032
2020-01-17 00:01:01 info nethist[]: mem_total_min = 2070032
2020-01-17 00:01:01 info nethist[]: mem_total_samples = 1284
2020-01-17 00:01:01 info nethist[]: mem_total_sum = 2657921088
2020-01-17 00:01:01 info nethist[]: temp_board_avg = nan
2020-01-17 00:01:01 info nethist[]: temp_board_max = 0
2020-01-17 00:01:01 info nethist[]: temp_board_samples = 0
2020-01-17 00:01:01 info nethist[]: temp_board_sum = 0
2020-01-17 00:01:01 info nethist[]: temp_cpu_avg = nan
2020-01-17 00:01:01 info nethist[]: temp_cpu_max = 0
2020-01-17 00:01:01 info nethist[]: temp_cpu_samples = 0
2020-01-17 00:01:01 info nethist[]: temp_cpu_sum = 0
2020-01-17 00:01:01 info nethist[]: uptime = 25656
2020-01-17 00:01:47 info dnsmasq-dhcp[8037]: DHCPREQUEST(br-lan) 192.168.10.174 08:c5:e1:e8:77:fb 
2020-01-17 00:01:47 info dnsmasq-dhcp[8037]: DHCPACK(br-lan) 192.168.10.174 08:c5:e1:e8:77:fb Galaxy-S9
2020-01-17 00:02:01 info /usr/sbin/cron[21842]: (root) CMD (/root/bin/guest-wifi-led.sh)
2020-01-17 00:02:01 info /usr/sbin/cron[21844]: (root) CMD (/usr/bin/rainbow_button_sync.sh)
2020-01-17 00:02:01 info /usr/sbin/cron[21846]: (root) CMD (/root/bin/openvpn-led.sh)
2020-01-17 00:02:01 info /usr/sbin/cron[21845]: (root) CMD (nethist_stats.lua)
2020-01-17 00:02:02 err server_uplink[]: Failed to get registration code
2020-01-17 00:02:09 err turris-firewall-rules[]: (v63) Failed to download https://api.turris.cz/firewall/turris-ipsets.gz.sign
2020-01-17 00:03:01 info /usr/sbin/cron[21921]: (root) CMD (/usr/bin/rainbow_button_sync.sh)
2020-01-17 00:03:01 info /usr/sbin/cron[21922]: (root) CMD (/root/bin/openvpn-led.sh)
2020-01-17 00:03:01 info /usr/sbin/cron[21920]: (root) CMD (/root/bin/guest-wifi-led.sh)
2020-01-17 00:03:17 info ucollect[13351]: Started task Ping as PID 21966 and fd 26
2020-01-17 00:03:37 info ucollect[13351]: Finished task Ping, success 1
2020-01-17 00:03:37 warning ucollect[13351]: epoll_wait on 4 interrupted, retry
2020-01-17 00:03:57 info kernel[]: [25830.262113] Core dump to |/usr/lib/systemd/systemd-coredump 20219 0 0 11 1579215837 0 turris pipe failed
2020-01-17 00:04:01 info /usr/sbin/cron[22056]: (root) CMD (/root/bin/guest-wifi-led.sh)
2020-01-17 00:04:01 info /usr/sbin/cron[22060]: (root) CMD (nethist_stats.lua)
2020-01-17 00:04:01 info /usr/sbin/cron[22061]: (root) CMD (/usr/bin/rainbow_button_sync.sh)
2020-01-17 00:04:01 info /usr/sbin/cron[22059]: (root) CMD (/root/bin/openvpn-led.sh)
2020-01-17 00:04:14 info hostapd[]: wlan1: STA 60:ab:67:e9:a4:47 WPA: group key handshake completed (RSN)
2020-01-17 00:04:14 info hostapd[]: wlan1: STA 10:5b:ad:c6:47:1d WPA: group key handshake completed (RSN)
2020-01-17 00:05:01 info /usr/sbin/cron[22200]: (root) CMD (/root/bin/guest-wifi-led.sh)
2020-01-17 00:05:01 info /usr/sbin/cron[22205]: (root) CMD (/usr/bin/rainbow_button_sync.sh)
2020-01-17 00:05:01 info /usr/sbin/cron[22206]: (root) CMD (/usr/share/firewall/turris)
2020-01-17 00:05:01 info /usr/sbin/cron[22204]: (root) CMD ( schnapps cleanup)
2020-01-17 00:05:01 info /usr/sbin/cron[22208]: (root) CMD (/usr/bin/notifier)
2020-01-17 00:05:01 info /usr/sbin/cron[22207]: (root) CMD (/root/bin/openvpn-led.sh)
2020-01-17 00:05:01 info turris-firewall-rules[]: (v63) IPv4 WAN interface used - 'eth1'
2020-01-17 00:05:01 info turris-firewall-rules[]: (v63) IPv6 WAN interface used - 'lo'
2020-01-17 00:05:02 info turris-firewall-rules[]: (v63) 3524 ipv4 address(es) and 3 ipv6 address(es) were loaded (c83b72d5b03454292e5000b3967bc350), 0 rule(s) overriden, 0 rule(s) skipped
2020-01-17 00:05:03 info kernel[]: [25896.031431] BTRFS info (device mmcblk0p1): quota is enabled
2020-01-17 00:05:47 warning kernel[]: [25939.964661] BTRFS warning (device mmcblk0p1): mmcblk0p1 checksum verify failed on 86552576 wanted 83559308 found B5CC31E2 level 1
2020-01-17 00:05:47 warning kernel[]: [25939.964683] BTRFS warning (device mmcblk0p1): Skipping commit of aborted transaction.
2020-01-17 00:05:47 warning kernel[]: [25939.964688] ------------[ cut here ]------------
2020-01-17 00:05:47 warning kernel[]: [25939.964699] WARNING: CPU: 0 PID: 682 at fs/btrfs/transaction.c:1840 btrfs_commit_transaction+0xaa4/0xc0c()
2020-01-17 00:05:47 debug kernel[]: [25939.964703] BTRFS: Transaction aborted (error -5)
2020-01-17 00:05:47 warning kernel[]: [25939.964706] Modules linked in: qcserial option iptable_nat ip6table_nat ath9k usb_wwan rndis_host qmi_wwan pppoe nf_nat_pptp nf_nat_ipv6 nf_nat_ipv4 nf_nat_amanda nf_conntrack_pptp nf_conntrack_ipv6 nf_conntrack_ipv4 nf_conntrack_amanda ipt_REJECT ipt_MASQUERADE ebtable_nat ebtable_filter ebtable_broute cdc_ether ath9k_common armada_thermal xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_socket xt_recent xt_nfacct xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_id xt_hl xt_helper xt_geoip xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TPROXY xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_DSCP xt_CT xt_CLASSIFY usbserial usbnet ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda ts_kmp ts_fsm ts_bm thermal_sys pppox ppp_mppe ppp_async nfnetlink_acct nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_redirect nf_nat_proto_gre nf_nat_masquerade_ipv4 nf_nat_irc nf_nat_h323 nf_nat_ftp nf_log_ipv4 nf
2020-01-17 00:05:47 notice kernel[]: nf_defrag_ipv4 nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_rtcache nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast mvsdio iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables hwmon ebtables ebt_vlan ebt_stp ebt_redirect ebt_pkttype ebt_mark_m ebt_mark ebt_limit ebt_among ebt_802_3 crc_ccitt cdc_wdm ath9k_hw fuse sch_teql sch_tbf sch_sfq sch_red sch_prio sch_pie sch_netem sch_htb sch_gred sch_fq sch_dsmark sch_codel em_text em_nbyte em_meta em_cmp cls_basic act_vlan act_police act_pedit act_nat act_ipt act_gact act_csum act_bpf act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_ingress ath10k_pci ath10k_core ath mac80211 cfg80211 compat ledtrig_usbdev ledtrig_oneshot xt_LED ledtrig_morse ledtrig_heartbeat ledtrig_gpio cryptodev xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_ma
2020-01-17 00:05:47 notice kernel[]: ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set ip6t_NPT ip6t_MASQUERADE nf_nat_masquerade_ipv6 nf_nat nf_conntrack ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_raw ip6table_mangle ip6table_filter ip6_tables pppoatm ppp_generic slhc nfsd nfsv3 msdos ip_gre gre ifb sit ip6_tunnel tunnel6 tunnel4 ip_tunnel veth tun snd_compress snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd_rawmidi snd_seq_device snd_hwdep snd input_core soundcore rxkad vfat fat udf crc_itu_t ntfs nfs_layout_nfsv41_files nfsv4 nfs auth_rpcgss oid_registry lockd sunrpc grace minix isofs hfsplus hfs cramfs configfs cifs autofs4 kafs af_rxrpc dns_resolver dm_crypt dm_mirror dm_region_hash dm_log dm_mod br2684 atm multipath fscache raid456 async_raid6_recov async_pq async_xor async_memcpy async_tx raid10 raid1 raid0 linear md_mod xt_NFLOG x_tables nfnetlink_log nfnetlink nls_utf8 nls_koi8_r nls_cp1255 nls_iso8859_6 nls_
2020-01-17 00:05:47 notice kernel[]: nls_iso8859_15 nls_iso8859_13 nls_iso8859_1 nls_cp932 nls_cp866 nls_cp864 nls_cp862 nls_cp852 nls_cp850 nls_cp775 nls_cp437 nls_cp1251 nls_cp1250 xts algif_skcipher algif_hash af_alg sha512_generic sha256_generic sha1_generic seqiv jitterentropy_rng drbg pcbc md5 md4 marvell_cesa hmac gf128mul fcrypt ecb des_generic ctr cmac ccm cbc authenc xhci_plat_hcd xhci_pci xhci_hcd uhci_hcd ledtrig_transient ahci_platform uas xfs libcrc32c jfs f2fs exfat mii aead crypto_null
2020-01-17 00:05:47 warning kernel[]: [25939.965101] CPU: 0 PID: 682 Comm: btrfs-cleaner Not tainted 4.4.199-a890a5a94ebb621f8f1720c24d12fef1-0 #1
2020-01-17 00:05:47 warning kernel[]: [25939.965105] Hardware name: Marvell Armada 380/385 (Device Tree)
2020-01-17 00:05:47 warning kernel[]: [25939.965108] Backtrace: 
2020-01-17 00:05:47 warning kernel[]: [25939.965117] [<c001c654>] (dump_backtrace) from [<c001c894>] (show_stack+0x18/0x1c)
2020-01-17 00:05:47 warning kernel[]: [25939.965120]  r6:00000000 r5:60000013 r4:c077aea8 r3:00000000
2020-01-17 00:05:47 warning kernel[]: [25939.965131] [<c001c87c>] (show_stack) from [<c0304164>] (dump_stack+0x98/0xac)
2020-01-17 00:05:47 warning kernel[]: [25939.965138] [<c03040cc>] (dump_stack) from [<c00293c0>] (warn_slowpath_common+0x8c/0xbc)
2020-01-17 00:05:47 warning kernel[]: [25939.965141]  r6:00000730 r5:c023c71c r4:ee179dc8 r3:00000006
2020-01-17 00:05:47 warning kernel[]: [25939.965151] [<c0029334>] (warn_slowpath_common) from [<c0029428>] (warn_slowpath_fmt+0x38/0x40)
2020-01-17 00:05:47 warning kernel[]: [25939.965153]  r8:eea075f4 r7:eea67000 r6:ee41b0d0 r5:ee46c148 r4:ee765330
2020-01-17 00:05:47 warning kernel[]: [25939.965165] [<c00293f4>] (warn_slowpath_fmt) from [<c023c71c>] (btrfs_commit_transaction+0xaa4/0xc0c)
2020-01-17 00:05:47 warning kernel[]: [25939.965168]  r3:fffffffb r2:c067a7c8
2020-01-17 00:05:47 warning kernel[]: [25939.965175] [<c023bc78>] (btrfs_commit_transaction) from [<c023d074>] (__btrfs_end_transaction+0x2fc/0x328)
2020-01-17 00:05:47 warning kernel[]: [25939.965178]  r10:00000201 r9:00000001 r8:00000020 r7:eea07000 r6:eea67000 r5:ee765330
2020-01-17 00:05:47 warning kernel[]: [25939.965187]  r4:ee41b0d0
2020-01-17 00:05:47 warning kernel[]: [25939.965193] [<c023cd78>] (__btrfs_end_transaction) from [<c023d394>] (btrfs_end_transaction_throttle+0x14/0x18)
2020-01-17 00:05:47 warning kernel[]: [25939.965196]  r10:ee42d690 r9:edabfc00 r8:eea67000 r7:ec191480 r6:edabfdcb r5:ee41b0d0
2020-01-17 00:05:47 warning kernel[]: [25939.965204]  r4:00000008
2020-01-17 00:05:47 warning kernel[]: [25939.965212] [<c023d380>] (btrfs_end_transaction_throttle) from [<c0224304>] (btrfs_drop_snapshot+0x404/0x7d8)
2020-01-17 00:05:47 warning kernel[]: [25939.965219] [<c0223f00>] (btrfs_drop_snapshot) from [<c023bbd8>] (btrfs_clean_one_deleted_snapshot+0xa0/0xd0)
2020-01-17 00:05:47 warning kernel[]: [25939.965222]  r10:00000000 r9:00000000 r8:00000000 r7:00000001 r6:edabfc00 r5:eea07000
2020-01-17 00:05:47 warning kernel[]: [25939.965230]  r4:edabff20
2020-01-17 00:05:47 warning kernel[]: [25939.965239] [<c023bb38>] (btrfs_clean_one_deleted_snapshot) from [<c02330ec>] (cleaner_kthread+0x15c/0x1c8)
2020-01-17 00:05:47 warning kernel[]: [25939.965242]  r6:c07a19e0 r5:ee178000 r4:eea67000 r3:00000001
2020-01-17 00:05:47 warning kernel[]: [25939.965254] [<c0232f90>] (cleaner_kthread) from [<c0044698>] (kthread+0x110/0x124)
2020-01-17 00:05:47 warning kernel[]: [25939.965257]  r9:00000000 r8:00000000 r7:c0232f90 r6:eea67000 r5:00000000 r4:eea0ff00
2020-01-17 00:05:47 warning kernel[]: [25939.965269] [<c0044588>] (kthread) from [<c0009dd0>] (ret_from_fork+0x14/0x24)
2020-01-17 00:05:47 warning kernel[]: [25939.965271]  r7:00000000 r6:00000000 r5:c0044588 r4:eea0ff00
2020-01-17 00:05:47 warning kernel[]: [25939.965301] ---[ end trace debf1fea40f48a03 ]---
2020-01-17 00:05:47 crit kernel[]: [25939.965308] BTRFS: error (device mmcblk0p1) in cleanup_transaction:1840: errno=-5 IO failure
2020-01-17 00:05:47 info kernel[]: [25939.973796] BTRFS info (device mmcblk0p1): forced readonly
2020-01-17 00:05:47 info kernel[]: [25939.973802] BTRFS info (device mmcblk0p1): delayed_refs has NO entry
2020-01-17 00:06:01 info /usr/sbin/cron[22836]: (root) CMD (/root/bin/guest-wifi-led.sh)
2020-01-17 00:06:01 info /usr/sbin/cron[22840]: (root) CMD (/root/bin/openvpn-led.sh)
2020-01-17 00:06:01 info /usr/sbin/cron[22841]: (root) CMD (/usr/bin/rainbow_button_sync.sh)
2020-01-17 00:06:01 info /usr/sbin/cron[22839]: (root) CMD (nethist_stats.lua)
2020-01-17 00:06:48 info ucollect[13351]: Updating filter addresses from version 52640490 to version 52640531 (epoch 4)
2020-01-17 00:06:49 info ucollect[13351]: Updating ipset turris_100FA4E0_lb_a_4_X from version 52640490 to version 52640531 (epoch 4)
2020-01-17 00:06:54 warning ucollect[13351]: epoll_wait on 4 interrupted, retry
2020-01-17 00:07:01 info /usr/sbin/cron[22912]: (root) CMD (/usr/bin/rainbow_button_sync.sh)
2020-01-17 00:07:01 info /usr/sbin/cron[22913]: (root) CMD (/root/bin/openvpn-led.sh)
2020-01-17 00:07:01 info /usr/sbin/cron[22911]: (root) CMD (/root/bin/guest-wifi-led.sh)
2020-01-17 00:08:01 info /usr/sbin/cron[22979]: (root) CMD (/root/bin/guest-wifi-led.sh)