Omnia router with u-boot updated stuck with reflash/boot

Hi,
I run an older omnia device (black) and ran into troubles yesterday. it suddenly was stuck in a boot loop. until now I am still not successful to run it properly.
I have the system on an internal ssd and tried to upgrade the u-boot environment using a serial console. I think that was successful so far. but on rebooting the subvolume on the ssd is not found and ends up in a kernel panic.
my latest idea was to use the medkit and reflash but this leads to the same result. any ideas for me? hopefully someone can help please! and basically I would like to keep my data and configs on the ssd…
I really like the solution and I see some more years for the router…
thanks in advance!!!
Thomas

Could you share some details on your current u-boot configuration (version, environment)?

Hi, thanks, sure.
u-boot version is u-boot 2022.10-rc4-openwrt* something (currently I have no access)
printenv:
printenv
arch=arm
baudrate=115200
board=turris_omnia
board_name=turris_omnia
boot_a_script=load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; source ${scriptaddr}
boot_efi_binary=load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootarm.efi; if fdt addr -q ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r};else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi
boot_efi_bootmgr=if fdt addr -q ${fdt_addr_r}; then bootefi bootmgr ${fdt_addr_r};else bootefi bootmgr;fi
boot_extlinux=sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf}
boot_net_usb_start=usb start
boot_pci_enum=pci enum
boot_prefixes=/ /boot/ /@/boot/
boot_script_dhcp=boot.scr.uimg
boot_scripts=boot.scr.uimg boot.scr
boot_syslinux_conf=extlinux/extlinux.conf
boot_targets=scsi0 mmc0 usb0 pxe dhcp
bootcmd=run distro_bootcmd
bootcmd_dhcp=devtype=dhcp; run boot_net_usb_start; run boot_pci_enum; if dhcp ${scriptaddr} ${boot_script_dhcp}; then source ${scriptaddr}; fi;setenv efi_fdtfile ${fdtfile}; if test -z “${fdtfile}” -a -n “${soc}”; then setenv efi_fdtfile ${soc}-${board}${boardver}.dtb; fi; setenv efi_old_vci ${bootp_vci};setenv efi_old_arch ${bootp_arch};setenv bootp_vci PXEClient:Arch:00010:UNDI:003000;setenv bootp_arch 0xa;if dhcp ${kernel_addr_r}; then tftpboot ${fdt_addr_r} dtb/${efi_fdtfile};if fdt addr -q ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r}; else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi;fi;setenv bootp_vci ${efi_old_vci};setenv bootp_arch ${efi_old_arch};setenv efi_fdtfile;setenv efi_old_arch;setenv efi_old_vci;
bootcmd_mmc0=devnum=0; run mmc_boot
bootcmd_nvme0=devnum=0; run nvme_boot
bootcmd_pxe=run boot_net_usb_start; run boot_pci_enum; dhcp; if pxe get; then pxe boot; fi
bootcmd_rescue=i2c dev 2; i2c mw 0x2a.1 0x3 0x1c 1; i2c mw 0x2a.1 0x4 0x1c 1; mw.l 0x01000000 0x00ff000c; i2c write 0x01000000 0x2a.1 0x5 4 -s; setenv bootargs “earlyprintk console=ttyS0,115200 omniarescue=$omnia_reset rescue_mode=$omnia_reset”; sf probe; sf read 0x1000000 0x100000 0x700000; lzmadec 0x1000000 0x1700000; if gpio input gpio@71_4; then bootm 0x1700000#sfp; else bootm 0x1700000; fi; bootz 0x1000000
bootcmd_scsi0=devnum=0; run scsi_boot
bootcmd_usb0=devnum=0; run usb_boot
bootdelay=3
bootfstype=btrfs
console=ttyS0,115200
cpu=armv7
distro_bootcmd=scsi_need_init=; setenv nvme_need_init; for target in ${boot_targets}; do run bootcmd_${target}; done
efi_dtb_prefixes=/ /dtb/ /dtb/current/
eth1addr=d8:58:d7:00:3d:9b
eth2addr=d8:58:d7:00:3d:9c
ethact=ethernet@70000
ethaddr=d8:58:d7:00:3d:9d
ethprime=eth2
fdt_addr_r=0x2000000
fdt_high=0x10000000
fdtcontroladdr=7fb19b30
fdtfile=armada-385-turris-omnia.dtb
initrd_high=0x10000000
kernel_addr_r=0x1000000
load_efi_dtb=load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile}
loadaddr=0x800000
mmc_boot=if mmc dev ${devnum}; then devtype=mmc; run scan_dev_for_boot_part; fi
nvme_boot=run boot_pci_enum; run nvme_init; if nvme dev ${devnum}; then devtype=nvme; run scan_dev_for_boot_part; fi
nvme_init=if ${nvme_need_init}; then setenv nvme_need_init false; nvme scan; fi
omnia_reset=0
pxefile_addr_r=0x1900000
ramdisk_addr_r=0x2200000
regdomain=**
sata_boot=if sata dev ${devnum}; then devtype=sata; run scan_dev_for_boot_part; fi
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${distro_bootpart}…; for prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done;run scan_dev_for_efi;
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then run scan_dev_for_boot; fi; done; setenv devplist
scan_dev_for_efi=setenv efi_fdtfile ${fdtfile}; if test -z “${fdtfile}” -a -n “${soc}”; then setenv efi_fdtfile ${soc}-${board}${boardver}.dtb; fi; for prefix in ${efi_dtb_prefixes}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then run load_efi_dtb; fi;done;run boot_efi_bootmgr;if test -e ${devtype} ${devnum}:${distro_bootpart} efi/boot/bootarm.efi; then echo Found EFI removable media binary efi/boot/bootarm.efi; run boot_efi_binary; echo EFI LOAD FAILED: continuing…; fi; setenv efi_fdtfile
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf}; then echo Found ${prefix}${boot_syslinux_conf}; run boot_extlinux; echo SCRIPT FAILED: continuing…; fi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILED: continuing…; fi; done
scriptaddr=0x1800000
scsi_boot=run boot_pci_enum; run scsi_init; if scsi dev ${devnum}; then devtype=scsi; run scan_dev_for_boot_part; fi
scsi_init=if ${scsi_need_init}; then scsi_need_init=false; scsi scan; fi
serial#=0000000B00008085
soc=mvebu
usb_boot=usb start; if usb dev ${devnum}; then devtype=usb; run scan_dev_for_boot_part; fi
vendor=CZ.NIC

Environment size: 5102/65532 bytes

And also about the procedure.

what do you mean exactly? I am connected via serial console and try to configure the box properly with printenv and setenv to be able to boot without errors.

And which commands you used?

first I tried to update u-boot which currently shows “U-Boot 2022.10-rc4-OpenWrt-r16653+119-44ce70f0e2” as version.
then I connect via serial and reach the u-boot prompt in console. I already tried almost all of the recovery methods including factory reset, medkit from usb, go back to last snapshot …
I think the current main problem is because of a current u-boot version with some “historic” settings. I also think that I am only missing small parts in the u-boot environment to successfully boot the installed system from the “old” msata ssd disc. for some reason during boot the process can not use the btrfs subvolume. maybe that leads to some helpful direction?

What’s the SSD’S subvolume and directory which you’re trying to boot from?

it is /dev/sda1/@ and the full path (as I found in some docu as boot_prefixes=/ /boot/ /@/boot/
I can see the device with the partitions and also the subvolume in the rescue shell.
before I have to run “scsi scan”.

  • Was this setup working before (boot from SSD) and which parts did you change since then?
  • Did you try to revert to a supported u-boot version?

yes, before it was working properly. two days ago it started with a boot loop and that’s why I started to update u-boot.
I was hoping that the latest release would also fully support my device. but if you recommend I will try with an older version.

I don’t. I was just trying to understand the context.

If I understand correctly, you didn’t make any changes to either the SSD or U-Boot and the problems (boot loop) started out of the blue, correct?

yes, more or less this was the starting point. in the meantime- two minutes ago- I have downgraded u-boot and have my box up again! so I switched back to some previous snapshot using schnapps.
currently I am checking the system.
maybe I should add that I started with two additional lxc containers some weeks ago but I thought that lxc on msata ssd does not harm the system. is that true?
anyway- thanks for your attention and support so far!

Hi again,
my router is alive but I do still have some issues. I am missing network devices and this and that…
I consider a full factory reset but I do not know how this works with an msata ssd as external drive. during my tries the last days the medkit was not successful- at least if I started with the 4LED reset. any help appreciatetd!
thanks,
Thomas

Good to hear that you were able to repair the U-Boot portion! I think it makes sense to summarize the current state of U-Boot (in a separate thread) so people know where to start if they want to boot off an SSD.

If You have working system - start with upgrading u-boot via nor-update:

Then, You need change one variable (boot_targets) responsible for booting ssd

Then You have to update @factory and rollback

for me that did not work because it looks my router is from before 2019… if you update to a current u-boot it did not work for me at all. I only was successful to go back to a really old version of u-boot. in that env there is no boot_targets attribute.
I am still asking for the proper way to factory reset such old router…

Mine is indiegogo-edition and works:)
Did you reset entire envoronment after update?

what do you mean by “reset entire environment”? I would like to factory reset the box although there is a lot to configure again… but I was not successful so far. and I wanted to avoid a complete fdisk/format of the current msata ssd

It was something like “env default -a”.
You need to reset all old variables, new ones are different.