MOX APT resistance

CZ.NIC seems to take security much more seriously than typical router vendors and I believe using Turris devices is a net security win for a very large majority of their owners/renters. Though I don’t own an Omnia, so I don’t have practical experience.

That being said, have you made effort to resist backdoors and advanced persistent threats? Once an attacker has (potentially) penetrated MOX system security, can one assuredly clean the system, or at least verify, whether it is free from infection? There are aspects of my 10-year old router, that I would not trust, but I believe it only has a single storage chip, which I can read/write via ISP, so I can verify/clean it, unlike most newer systems.

MOX is listed as using Marvell Armada 3720 Cortex A-53 CPU. It includes TrustZone, which creates a privilege level below kernel, probably unused on linux and a prime target for malware. Can TrustZone perhaps be permanently disabled?

Assuming CPU has nonvolatile storage (e.g. fuse, flash), can its state be reliably verified?
Assuming MOX boots from flash (before reading card/network). Can it be accessed/written externally (ISP or similar)? Does it have any other storage?

Will JTAG be accessible? Can it nonspoofably access TrustZone realm?

1 Like

@paja @cynerd
Any answers to that question?

When attacker penetrates Turris in general if you want to be sure then there is no other way then to wipe all internal storages.

To check if your system is not potentially penetrated you can verify checksums of files provided by packages against packages on our server (at the moment there is no tool for it but it is planned but no promises). On top of that you want to check what applications are running and what is automatically started (/etc/rc.d and /lib/preinit).

Both Omnia and MOX have on top of basic system storage also on board mmc with u-boot (bootloader) and recovery system. If Turris is attacked directly then that can be place where malware can preserve it self between factory resets. That memory is also directly accessible from running system. That is disadvantage and advantage at the same time. Disadvantage is that it can be attacked directly but advantage is that such attack can be detected and both u-boot and recovery system updated/reflashed. Although at the moment we don’t have any documentation on how to do so.

Both MOX and Omnia have also flash for storing switch configuration. Those are not used at the moment and system is not reading nor using them. But it’s potentially possible to write to them from CPU.

Although you have asked for MOX I will also note that there is secondary microcontroller on Turris Omnia that has internal storage. Although improbable it is flash memory that can be attacked too but it can’t be programmed from CPU it self.

Considering TrustZone. Yes it is not used in Turris OS but you are probably thinking that it is something like Intel ME and it is not. It is more like additional CPU mode. It divides memory and other CPU resources to trusted and untrusted. By design boot and OS have to run in trusted mode. For example in case of memory CPU it self limits access from untrusted mode to memory sections marked as trusted. If TrustZone is not used then everything is run as trusted. It is not additional privileged mode that undetected virus could be running in. So no ARM TrustZone is not creating some privileged level that is not accessible by kernel. Instead it creates level that can be used to separate trusted and untrusted applications while kernel it self has to be trusted.

Yes there are fuses but according to my knowledge there is no way to change them without JTAG (the same is for verification). There is additional storage in CPU that can be used to store keys. That is just for storing keys and I doubt that someone will store malware there specially when there is no way how to run it and most of that memory is write only (it is a key store after all).

You can access mmc on both MOX and Omnia externally but only trough test points and that is probably not something you can consider as a access (of course you can use programming clip directly on chip). And also using JTAG if you know what you are doing as it’s accessible from CPU.

Our target is to ensure that there is no breach at all but if it happens then there are ways to wipe router clean. Factory reset is enough for malware not targeting embedded devices, full reflash is desirable for some malware that is aware of embedded hardware. That is possible with Turris although at the moment not documented. As last I want to state that we don’t know about any recorded breach on our routers that would be embedded aware.

1 Like

MOX uses SD cards for OS and if I am not mistaken, it is the only mmc.

MMC and SD cards are similar. U-boot calls the card mmc, and linux /dev/mmcblk. MOX sdcard is removable, which is good. Unfortunately SD cards are also definitely vulnerable to all kinds false storage/APT attacks (they are controller + flash and the controller can be owned [1][2]). SD card APT vector could be mitigated with verified secure boot (signature in flash would suffice), though it is not trivial to setup.

MOX boots u-boot from 8 MiB SPI flash, accessible on expansion header, which is nice. It is a less common 1.8V type, so finding a compatible programmer is a bit harder, but shoudn’t be a big deal.

As an update, here is the list of MOX persistent storage I know of:

  1. SD card for OS / rootfs storage
  2. 8 MiB SPI flash for u-boot
  3. Marvell Armada 3720 CPU persistent storage consisting of
    a. Integrated BootROM
    b. Enhanced Secure-Boot flow using integrated One Time Programmable (OTP) memory
    c. probably some fuses in ARM CPU
    • 69 bits in North Bridge for manufacturing purposes [3]
    • 97 bits in South Bridge for manufacturing purposes [3]

I still have a couple of questions:

  • @cynerd mentions flash for potentially storing switch configuration. Is it the (2) SPI flash?
  • Regarding the CPU, how much can it be reprogrammed/modified?
  • Can the Integrated BootROM be modified via software? Perhaps only changing bits in a single direction?
  • How does Secure Boot work? Can it somehow be re-programmed/modified in the CPU? Does it only store root-of-trust keys/hashes?

The PCIe wifi card (WLE900VX) and the SDIO wifi card (Marwell W8997-M1216) both load their firmware from the OS, so probably no interesting storage there. Even if they have some tiny amount of OTP storage, they are somewhat separated from the main CPU and the OS and wouldn’t really expect any exploits to persist there.

I haven’t checked all chips on the boards, but I notice any interesting persistence points. Lots of shift registers, things I assume to be power regulators and passives. Some bridges and ethernet controllers, which I wouldn’t expect to persist their configuration.

[1] https://www.techwalla.com/articles/how-to-program-sd-cards
[2] https://www.bunniestudios.com/blog/?page_id=3592
[3] https://www.marvell.com/documents/qc8hltbjybmpjhx36ckw/