I have a Turris Omnia router from the crowdfunding campaign (revision CZ11NIC13, according to the markings on PCB), the version with 2 GB of RAM.
Board firmware type is ‘stm32-rev23-user-regulator’ (according to omnia-mcutool output).
The U-Boot version is still ‘2015.10-rc2 (Aug 18 2016 - 20:43:35 +0200)’.
The router was unused for a long period and few days ago I decided to re-flash it from a USB flash drive using the 4 LEDs reset mode. The process went smoothly and in the end it was running Turris OS version 7.0.1 (from HBS branch, using Kernel version 5.15.148), configured into default router mode.
Although the plan was to also update the firmware, since I knew from another topic that I might run into DDR3 training issue I decided to do it step-by-step.
I accessed the /reforis/package-management/packages page and check-marked the ‘Latest firmware (Experimental)’ checkbox as well as the ‘Factory image’ sub-checkbox and pressed ‘Save’. As a result firmware-updater 1.1-5 was installed.
After few minutes I’ve also enabled the MCU firmware updates by ticking the corresponding ‘MCU’ checkbox and saving. As a result omnia-mcu-firmware 3.4-1 and omnia-mcutool 0.2-1 were installed.
After a short time I noticed that all the LEDs were blinking green.
Waited few minutes and decided to manually restart from reForis (and later found out, from the documentation, that this was the actual meaning of the signaling).
And then my Omnia failed to boot; the serial console output was:
U-Boot 2015.10-rc2 (Aug 18 2016 - 20:43:35 +0200), Build: jenkins-omnia-master-23
SoC: MV88F6820-A0
Watchdog enabled
I2C: ready
SPI: ready
DRAM: 2 GiB (ECC not enabled)
Enabling Armada 385 watchdog.
Disabling MCU startup watchdog.
Regdomain set to **
MMC: mv_sdh: 0
SF: Detected S25FL164K with page size 256 Bytes, erase size 64 KiB, total 8 MiB
*** Warning - bad CRC, using default environment
PCI:
00:01.0 - 168c:002e - Network controller
PCI:
01:00.0 - 11ab:6820 - Memory controller
01:01.0 - 168c:003c - Network controller
Model: Marvell Armada 385 GP
Board: Turris Omnia SN: 0000000B00008BF3
Regdomain set to **
SCSI: MVEBU SATA INIT
SATA link 0 timeout.
SATA link 1 timeout.
AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq led only pmp fbss pio slum part sxs
Net: neta2
Hit any key to stop autoboot: 0
Setting bus to 1
BOOT eMMC FS
btrfs probe failed
** Unrecognized filesystem type **
btrfs probe failed
** Unrecognized filesystem type **
Kernel image @ 0x1000000 [ 0x000000 - 0x415ae8 ]
## Flattened Device Tree blob at 02000000
Booting using the fdt blob at 0x2000000
Loading Device Tree to 0fff7000, end 0ffff3b9 ... OK
fdt_find_or_add_subnode: chosen: FDT_ERR_BADSTRUCTURE
ERROR: /chosen node create failed
- must RESET the board to recover.
FDT creation failed! hanging...### ERROR ### Please RESET the board ###
Also, the router was self rebooting about every minute and failing to boot because of the same error.
Next I tried 2 and 3 LEDs reset modes (rollback to latest snapshot / factory reset) but the same error occurred.
After a new re-flash from USB the FDT error was gone but there were other issues (timeouts, errors).
...
Searching for /srcmnt/omnia-medkit*.tar.gz on sda1
Found medkit file /srcmnt/omnia-medkit-latest.tar.gz on device sda1
[ 17.970590] mmc0: Timeout waiting for hardware interrupt.
[ 27.990590] mmc0: Timeout waiting for hardware interrupt.
[ 27.996041] mmcblk0: error -110 sending stop command, original cmd response 0x900, card status 0xe00
[ 28.058647] random: nonblocking pool is initialized
[ 49.470855] mmc0: Got data interrupt 0x00100000 even though no data operation was in progress.
… // 27 times the same 'Got data interrupt 0x00100000...' message
[ 607.816605] mmc0: Got data interrupt 0x00100000 even though no data operation was in progress.
[ 628.010599] mmc0: Card stuck in programming state! mmcblk0 card_busy_detect
[ 628.017589] blk_update_request: I/O error, dev mmcblk0, sector 0
[ 628.023629] Buffer I/O error on dev mmcblk0, logical block 0, lost async page write
1+0 records in
1+0 records out
512 bytes (512B[ 628.044487] mmc0: Controller never released inhibit bit(s).
) copied, 620.07[ 628.050147] mmcblk0: unknown error -5 sending read/write command, card status 0xe00
2380 seconds, 0B[ 628.059186] blk_update_request: I/O error, dev mmcblk0, sector 0
/s
Welco[ 628.066593] blk_update_request: I/O error, dev mmcblk0, sector 8
me to fdisk (uti[ 628.074003] blk_update_request: I/O error, dev mmcblk0, sector 16
l-linux 2.25.2).[ 628.081500] blk_update_request: I/O error, dev mmcblk0, sector 24
Changes wi[ 628.099008] mmc0: Controller never released inhibit bit(s).
ll remain in mem[ 628.104650] mmcblk0: unknown error -5 sending read/write command, card status 0xe00
ory only, until [ 628.113691] blk_update_request: I/O error, dev mmcblk0, sector 0
you decide to wr[ 628.121100] Buffer I/O error on dev mmcblk0, logical block 0, async page read
ite them.
Be careful before usi[ 628.141028] mmc0: Controller never released inhibit bit(s).
ng the write com[ 628.146675] mmcblk0: unknown error -5 sending read/write command, card status 0xe00
mand.
fdisk: [ 628.155720] blk_update_request: I/O error, dev mmcblk0, sector 0
cannot open /dev[ 628.163128] Buffer I/O error on dev mmcblk0, logical block 0, async page read
/mmcblk0: I/O er[ 628.181690] mmc0: Controller never released inhibit bit(s).
ror
[ 628.187323] mmcblk0: unknown error -5 sending read/write command, card status 0xe00
[ 628.195412] blk_update_request: I/O error, dev mmcblk0, sector 0
[ 628.201447] Buffer I/O error on dev mmcblk0, logical block 0, async page read
[ 628.208603] mmcblk0: unable to read partition table
blockdev: /dev/mmcblk0: I/O error
Error: Partition /dev/mmcblk0p1 missing. Exit.
umount_fs /srcmnt
[ 629.270630] [EXFAT] trying to unmount...
[ 629.274576] [EXFAT] unmounted successfully
BusyBox v1.23.2 (2016-08-18 20:38:45 CEST) built-in shell (ash)
/ # [ 649.662145] mmc0: Got data interrupt 0x00100000 even though no data operation was in progress.
… // 101 times the same 'Got data interrupt 0x00100000...' message
[ 671.136982] mmc0: Got data interrupt 0x00100000 even though no data operation was in progress.
…
In the end I unplugged the power cable and let it cool down for several minutes. After plugging back in the power cable my Omnia successfully booted and after all LEDs turned off I noticed that LAN was up (unfortunately I wasn’t able to re-establish the serial connection therefore I don’t have the logs for this boot).
Anyway, seeing that everything seems to work I reinstalled omnia-mcutool (this time via SSH) in order to check the output:
root@turris:~# omnia-mcutool --firmware-version
Bootloader version: b5a8a24e007eb72be16aeb3fff6f03ec647023e4
Application version: 67eddc9540526d0a9d9660f7a7867af9a28a68d6
MCU type: STM32
Board firmware type: stm32-rev23-user-regulator
Features: 0x1f6e
EXT_CMDS
WDT_PING
LED_STATE_EXT
LED_GAMMA_CORRECTION
NEW_INT_API
FLASHING
NEW_MESSAGE_API
BRIGHTNESS_INT
POWEROFF_WAKEUP
CAN_OLD_MESSAGE_API
Application firmware length: 21400 Bytes
Application firmware checksum: 0xfeeb0f31
According to the above my MCU firmware now has the new functionalities (e.g. poweroff/wakeup, gamma correction) so I guess the update was finalized (as the LEDs blinking green signalized) but somehow the partition table / filesystem got messed up.
I have written this post just in case someone else wants to update their Omnia’s MCU firmware and might run into similar issues or in case someone from development team might find these logs useful.