Hello guys,
I’m not aware of any eMMC failures, which wasn’t caused by LXC containers and I’d like to tell you, what and why it happened. If you had or have any LXC containers, I’d need to explain to you, why it is a really bad idea.
Common GNU/Linux distributions in LXC containers don’t count with running them the router and their logs or potentially databases writing into storage with very high frequency. That’s why we have system logs in RAM. This can be seen in the articles in our documentation. First see Error/bug reporting and then LXC containers.
Keep in mind, LXC containers are not enabled by default. They require to have at least some knowledge, how you can install one of those available images of Linux distributions as LXC containers and how you can use them.
Why it happened?
Internal storage in the router is eMMC, which the flash memory and it is used in micro SD cards, USB flash drives and so on. My point is all of them has a lifespan of writes and they don’t count with excessive amounts of writes, which can wear it and it is just a matter of time when you’ll wear it. The advantage of what I think of eMMC is that it is more reliable and faster than those devices, which I mention.
From both outputs of the serial console, I can see both eMMC are dead, which means all of your data is gone, and we can’t recover them. We have soldered eMMC on board, and it’s not easy to replace. It is almost impossible to swap it without expensive equipment. In our case, hopefully, the repair will be done by the 3rd company, and it will be paid repair because this is not what we can cover by warranty as it’s not manufacturers fault.
When you’d like to have LXC containers on your router, which is completely fine, but you’ll need to have external removable storage. In that case, even USB drive would be OK for that because they’re cheap and very easily replaceable just plug and play. Since Turris OS 3.10 you’re able to use Storage plugin, which is Foris to avoid any misconfigurations.
If you’d like to have to work your router ASAP, you can boot from mSATA SSD. The mSATA SSD should be inserted to the rightest slot near to heatsink. For more details I recommend you to see our documentation, where you can find more details about how you can boot from mSATA SSD. For future, we’d like to have the option to boot from USB stick.
The most recent warning what I can think for now was introduced in release notes for Turris OS 3.10 that this situation can happen. Release notes are available to be seen in our documentation, where you can also find Errata, which is the list of known bugs.
Once you had created LXC containers in CLI or GUI, you received notification in Foris, if you have configured sending notifications from Foris to your email address, you can find them in email as well. This warning is there for a long time, and it is described in the documentation for LXC containers. For the next version of Turris OS 3.10.6, which we’d like to release soon, we implemented more intensive warnings, which will be shown, when it detects that you’re running LXC containers on eMMC, it will tell you to use e.g. Storage plugin.