Unfortunately, rescueboot will still work only with eMMC. I haven’t looked into that yet, but looking at it’s boot variable suggests that this behaviour might be hardcoded into the FW of some internal chip.
All this change does is shifting all recovery modes upwards by 1 (which @adminX already mentioned here), so that we could boot from mmc0 on LED 2:
1 LED: (Re)boot to SSD
2 LEDs: (Re)boot to eMMC
3 LEDs: Rollback to latest snapshot on eMMC
etc.
I quess it might be more readable what happens in that modified bootcmd if we format it:
bootcmd=i2c dev 1; #select the button device
i2c read 0x2a 0x9 1 0x00FFFFF0; #read number of LEDs from the button memory (this number is actually shifted, so 1 == 2 LEDs are shining)
setexpr.b rescue *0x00FFFFF0; #append number of LEDs to the rescue variable
if test $rescue -ge 1; then #if rescue is equal or greater than 1, then...
setexpr rescue $rescue - 1; #subtract 1 from rescue
if test $rescue -ge 1; then #if rescue is equal or greater than 1, then run rescueboot
echo BOOT RESCUE;
run rescueboot;
else echo BOOT eMMC FS; #otherwise boot from eMMC
run mmcboot;
fi;
else echo BOOT SSD FS; # otherwise boot from SSD
run ssdboot;
fi
To be honest, I can’t remember where I made that assumption. Looking back at the schnapps script, it appears to be still used
You actually could install multiple internal drives by using additional SATA controller in one of the available mPCI slots, which is what I’m planning to do, therefore using UUID seemed like a good idea
I’ve set it that way so that if I break something, I could easily reboot to a functional OS from which I will be able to repair the SSD. Since the rescue modes apply only to the internal eMMC memory, I can’t go back one snapshot using the back button anymore.
The current rescue version can do this if root=/dev/sda1. This may be why the official docs add scsiboot (called ssdboot in mine) and replace root=b301 (meaning /dev/mmcblk0p1) with root=/dev/sda1 in bootargs.
You will lose the MMC based fallback but all rescue modes work on the SSD the same way the work with MMC.
The controller is not detected until after TurrisOS is booted. You can’t boot from them so using UUID is not a great idea as you lose most rescue functionality.
You can use the rescue modes with SSD if root=/dev/sda1. Rescue modes then apply to the SSD and not the MMC.
possible rescue usage with different configurations
Rescue Mode 4 (5 LEDS, Rescue Shell) always works.
Rescue Mode 6 (7 LEDs, Rescue Shell on serial) also works independent of the root setting.
All other Modes depend on root=/dev/sda1 or use MMC.
Replace TurrisOS with another OS on SSD
My goal was to replace TurrisOS(OpenWRT) with a full fledged Linux distribution running on SSD. My ssdboot command even sets different bootargs for the SSD based boot.
This does not support schnapps and required a rescue environment. The main OS on MMC is the rescue system in this case. All rescue modes should work with this “rescue” installtion so if i killed the TurrisOS in MMC i could use medkit to restore it without touching the SSD. This even happened once.
Replace MMC with TurrisOS on mSATA SSD
In this case the rescue modes should not access the MMC but the SSD.
The rescue system handles the special case for root=/dev/sda1 only. In every other case it assumes MMC.
Using root=/dev/sda1 is safe for this configuration because this connects to the platform AHCI controller which is running even before the USB controllers are detected. USB drives do not matter in this case.
So root=/dev/sda1 should be prefered as it allows using all rescue modes the same way they would work on MMC but on SSD.
Replace MMC with TurrisOS on USB drive
If there are multiple USB drives connected you will have to use UUID and lose the rescue modes.
If there is a mSATA SSD you will have to use UUID and lose the rescue modes.
If there is a single USB drive only then root=/dev/sda1 works and allows using all rescue modes for USB.
Other configurations
If you replaced u-boot to enable other hardware as a boot devices you should know what you are doing.
Stock Omnia u-boot can only boot from MMC, mSATA, USB, SPI-Flash (rescue) and maybe network. Network boot does not need a rescue function as it is one.
Nope, verified it doesn’t work. I have SSD boot configured per Turris Documentation, with root=/dev/sda1, but rollback (2 LED) doesn’t work. It doesn’t do anything but reboot the router twice (probably once into the rescue boot menu, and then normal reboot). I have the oldest Omnia from Indiegogo. Isn’t what you say true only for the newer Omnias?
I assume the following is correct and checked as best as possible with my configuration but was not executed as i don’t want to reboot my router now.
If you have a pre-2019 Omnia this needs the updated old rescue image. It basically is:
cd /tmp
wget https://repo.turris.cz/omnia-stable/nor_fw/omnia-initramfs-zimage
mtd -e /dev/mtd1 omnia-initramfs-zimage /dev/mtd1
For some background on flashing and recovering the older images:
Newer Omnias have multiple differences in hardware, u-boot and rescue images. More information about the changes and upcoming changes for older Omnias are in the thread below:
I thought it might be something like that… I’ll rather wait until the rescue image is updated officialy - don’t wanna break the only thing that lets me fix my hiccups =)
As long as you do not update u-boot there is no big risk.
You can check the rescue system by creating a snapshot and roll back using rescue afterwards. If it worked your rescue is working. If it does not then there is something wrong.
Backup /dev/mtd1
Create snapshot with schnapps create -t time or so
Update mtd1 with new rescue system
Reboot to rescue mode 1 (2 LEDs)
Check if schnapps list shows the rollback
If it shows the rollback you got it working.
My current omnia has U-Boot 2015.10-rc2 (Aug 18 2016 - 20:43:35 +0200), Build: jenkins-omnia-master-23 which is probably the original one on the campaign omnias.
The 2019 and newer ones seem to be out of luck this time. As i can’t get a hold of the real rescue image (dump of mtd1) burned into these devices it is a bit of reading code and guesswork. There are some conflicts with your u-boot variables and the ones that should be there according to the u-boot source and packages.
Possible rescue limitations:
schnapps always works with MMC in rescue (rollback and factory reset)
flashing a medkit uses the partition uuid from u-boot variable root_uuid
there may be some problems with selecting a rescue mode (wrong kernel command line)
1 LED: Standard (re)boot
2 LEDs: Rollback to latest snapshot
3 LEDs: Rollback to factory reset
4 LEDs: Re-flash router from flash drive
5 LEDs: Unsecure SSH on 192.168.1.1 (Omnia 2019 and newer)
6 LEDs: Flash from the internet (Omnia 2019 and newer)
7 LEDs: Rescue shell
Because of the explicit hint to the Omnia 2019 and newer versions, I think, that this doc is up to date for my router. And because of that and my minimal changes (I’ve only added a third boot prefix and have changed the boot order between scsi0 and mmc0) and the fact, that schnaps saves the new / directory mounted on the SSD, I think also, that the rollback modes should work with my changes.
Adding the third boot prefix (/@/boot/) is not necessary as there should be a symlink installed but this way is faster. /@/ would have worked too as the symlink in the subvolume-root should be there by default.
schnapps in the installed system works fine but there are a few bugs:
schnapps in the rescue system always uses /dev/mmcblk0p1 because this gets set before the rescue system checks if there is another partition set using root_uuid.
And there is absolutly no access to the SATA drive in the rescue system.
Apart from this the new rescue system takes ages to load because u-boot’s lzmadec is slow…
Not optimal, but I think, that I can live with that, because the eMMC is still there and if a rollback has installed the last snapshot from /dev/sda1 in the eMMC I should be able to reinstall it after that by hand again on /dev/sda1. Maybe with the help of the serial console.
Interesting, but I’m pretty sure, that it didn’t work on my router. At first I tried it only with the change of the boot order and I then found out, that the boot script was not found.
I think, that you can add another bug to your list, which is relevant, if you boot from a SSD. Look at this post.
The snapshots are on the same device that has the current root. If /dev/sda1 is your root then your snapshots are on /dev/sda1. You can even browse these snapshots if you mount the device without subvol=@. schnapps is a simple frontend for btrfs snapshots and subvolumes.
The link is created with ln -s @/boot/boot.scr boot.scr in the real root of the filesystem.
I don’t think so. First they are trying something schnapps was probably never designed to do. Second there are some assumptions about the used root subvolume that is named @. schnapps is a simple frontend to btrfs snapshots and does its job great but was never meant as a generic snapshot manager.
History and background as seen by me
root on SSD was not designed to be used as root device. SSD was meant to be used as /srv for LXC and other data. This is a much simpler use case and works like a charm. Minimized writes on MMC and you got all your data with speed. This is the way the system is designed and works.
There was no real need for root on SSD back then apart from running debian or other Linux distributions. My motivation for root on SSD was to allow me to run ArchLinux on my Omnia.
The original work for TurrisOS was done later. These threads are a bit about the motivation for others.
It was about destroying data or having enough space for opkg install '*'…
So me was a bit shocked to see /dev/sda1 got supported in the rescue image. I didn’t even notice the new image and this feature until now. Maybe it was a solution for some devices with broken MMC.
Back to 2020.
There are a least 3 hardware revisions of the omnia. These have at least 2 different u-boot and rescue images running.
The pre-2019 ones have a rescue that does not support SATA but there is a update with SATA support available. It has to be installed manually. Everything except cert-backup works with SATA the same way it works with MMC then.
Then we got Mox and it needed a more capable bootloader and rescue system that only has MMC. The micro SD card is accessed using mmcblk0 internally like on the Raspberry PI.
The rescue system for Mox got support for Omnia too. So there are 2 rescue images built from basically the same source.
Now comes the catch with it: Mox got the Turris Shield version and Omnia got contract versions. These probably run basically the same TurrisOS with some UI changes to tailor them to the target audience, e.g. simplified entering of user credentials for internet access on the omnia or something like that.
Up to here we got a bunch of devices which all run from MMC and got a quite capable rescue system. It even is able to reset the u-boot environment variables if you use rescue mode 3 (4LEDs).
There is no real need to run root on SSD apart from broken MMC. This should not happen unless you fill your MMC and write tons of data. Modern eMMC devices are quite durable and work like SSDs. Mobile phones have a ton of writes to eMMC and still work for years. OpenWRT and TurrisOS write seldom compared to Android. Compiling software is something different and can be really a flash killer.
I just got a working rescue image done and it supports SATA and MMC but required some hacks and a small patch. I may put them untested in the bug tracker but i am not sure. I am just in a really really bad mood and totally upset by OpenWRT.
Yesterday I recognized, that a command like schnapps -d /srv/lcx create is working if (and only if) there is no line like the following in /etc/schnapps/config: ROOT_DEV="/dev/sda1"
In my opinion a script or program option should always overwrite any settings of its configuration files. But the line is no longer necessary and so I think, one can live with it.
schnapps is a really nice script, thank you for working on it.
Believe me, I’ve not used the eMMC in my old Omnia router very extensive. I’ve installed a few programs at the beginning, played a little bit with it and after that I used it only as a router. But after two years the eMMC was damaged.
The destruction of the eMMC didn’t mean the router stopped working at all. Rather, it had to be reset more and more frequently in order to work properly again for a few hours or even days. It takes month to find out the reason with the help of the support. I’ve not heard from other users, that they had similar problems, so I think that I had bad luck.
But however, I really, really never want to see that again. Therefore I’m happy, that I’ve now a running system, which is using only the SSD - and by the way it is more than twice as fast as the eMMS. I’ve just installed a SmartHome system in lxc containers and everything is fine.
Then I am even more grateful to you for taking the time to shed more light on the whole thing. I hope this thread is also useful for other users, so I posted my instructions above.
It’s already here. I’ve marked it as solution.
Note: Each step is collapsed so you need to expand it by clicking the triangles. I have also included the later suggestions from other posts.
Please let me know if you find any bugs. I wrote down the instructions during the installation and I think I haven’t missed anything - but who knows
I think it should work as long as Turris creates a device named /dev/sda from your SSD device attached to USB. With my router, this is the case when no internal SSD and only a USB stick is connected.
However, pay attention to the U-Boot setting setenv boot_targets scsi0 mmc0 usb0 pxe dhcp
It defines the starting target order. scsi0 stands for /dev/sda1, mmc0 stands for the internal eMMC, pxe and dhcp stand for network-based boot protocols. I don’t know which devices “usb0” is for. On all of my Linux systems, a /dev/sd? Is created when an USB drive is attached.
So you should first check if there is a /dev/sda on your system when your SSD is connected and after creating a primary partition on it you need to have a /dev/sda1 as well. With that everything should work as described.
Anybody can register and login to community documentation and improve/add new articles and it could be a better place, where newcomers can see it instead of reading thread, which has almost 40 posts. I added a warning box that the article, which was there for booting from external storage does not apply to Turris Omnia 2019/2020.
My reason for my post is that we noticed a post from @drhm. We started to wonder if we ever decided that we officially support anything else than eMMC. Even though some members in our team including me are booting from mSATA SSD from returned units, which have worn out eMMC. So using booting from external device is the way how you can rescue/revive the device.
We see that there is enough demand for it and before we can support this method, we need to look at some things like rescue, etc. But the good news is that we will slowly start supporting this.
That sounds good. It wasn’t clear to me that anyone could change or add to the documentation. That was the reason why I sent a link to this thread to Marcela.
If I should post my documentation as community documentation anyway, I am happy to do so. I think I’ll find some free time in the next few weeks to add my instructions to this place. So please let me know, what I should do.
I can confirm that there is a pending pull request - Boot from SSD on Omnia (!159) · Merge requests · Turris / user-docs · GitLab which will add it to the user-docs repository I mean as a article to docs.turris.cz, which is our official documentation. In this documentation anyone can report issues to us through several ways - support/GitLab or even send is pull/merge request.
