Why Trusted Execution Environment and Security by Separation on IoT Edge Devices Are Important?

Carlos Moratelli, Ramão Tiago Tiburski, Sergio Johann, Everton de Matos, and Fabiano Hessel
September 16, 2019



The Internet is changing quickly into a model where billions of everyday objects will be interconnected, which we call the Internet of Things (IoT). Traditionally, IoT devices communicate directly with the Cloud, but that is changing to a layered architecture. The direct IoT-Cloud model works poorly for a fair share of all applications.

For example, the amount of data generated by sensors will be prohibitive in some instances, as seen on connected cars, which can create tens of megabytes each second [1]. Some long-range, low bandwidth radio technologies, like that provided by SigFox or NB-IoT, are charged by communication amount, making it desirable to minimize the data exchange. Some applications require fast response, as voice recognition. In this context, Fog and Edge layers were added to the IoT architecture [1] as an alternative to diminish the cloud communication and perform faster response. Therefore, sensors will communicate with nearby devices (edge) interconnected by medium-range networks (fog). Data processing and local decisions will be performed at the Fog/Edge layers avoiding additional communication with the Cloud. [2].

Behind the IoT well-known benefits, it is hidden an obscure treat: the digital security risks. Security emerged as one of the most critical concerns for the broad IoT adoption. Wonder the Internet flooded by millions of potentially vulnerable edge devices with significant processing power. Such devices will bring their vulnerabilities to private networks, turning the Internet into a fertile environment for hackers willing to steal sensitive information or to perform denial-of-service and denial-of-sleep attacks. Improvements in the security of all devices connected to the Internet are a vital concern for the future of the IoT [3].

In the run for safer devices, a realm of technologies can be applied. In this article, we discuss how two fundamental security trends can be put together to build the foundation of IoT edge security. First, the use of the Trusted Execution Environment (TEE) is essential to guarantee software and data integrity. A TEE requires separation to allow the concurrent execution of multiple isolated flows, so security by separation is also addressed.

Figure 1: Establishing a CoT from hardware to higher software layers.

Figure 1: Establishing a CoT from hardware to higher software layers.


Trusted Execution Environment (TEE)

As many edge devices are placed in public environments with easy access by non-authorized personnel, it is necessary to guarantee that the running software was not modified or changed maliciously. Even devices without a physical interface may be attacked by having its code or data changed remotely. A TEE allows for the detection of unintentionally software substitution, consisting of a protected machine’s memory area. In this environment, application code and data are verified for confidentiality and integrity using cryptography before execution [4]. There are two elements to build a TEE: the Root of Trust (RoT) and the secure-boot process. Both parts result in the called Chain of Trust (CoT).

The RoT is a trusted element that cannot be changed and constitutes the foundations for the device’s software integrity [4]. A typical implementation approach consists of hardware capable of performing software verification based on a cryptography key stored in a write-once memory. The chip’s manufacturer is responsible for providing support for verification and storage memory. In this scheme, developers are responsible for the software stack, so updates are possible even on devices already in the field.

The RoT allows for the secure boot process, where only verified software can be executed on the device’s startup. Therefore, this mechanism involves a set of verification at all layers of the system’s software until the level of the application, implementing end-to-end security, and defining a Chain of Trust (CoT). Figure 1 describes this scheme in a multi-layered environment, which includes an embedded virtualization layer. First, the hardware authenticates the bootloader. If successful, it is considered a trusted element and is allowed to verify the next boot stage. In this case, the next boot stage to be trusted is the hypervisor. Once verified, the hypervisor boots up and check its domains before their boot. Note that, non-trusted domains can coexist along with trusted areas in the same device which will be better addressed in the following sections.

Security by Separation

Current edge devices require significant processing power in order to handle sensor data, decision making and to communicate over the Fog layer. As a consequence, software complexity increases, and multiple separated execution flows are required. Enforcing isolation between the flows in a lightweight way, while still maintaining the TEE, can be challenging. In a compromised system, an attacker may try to spread its attack to other subsystems, taking control of all possible functionalities: this is called lateral movement and is seen as a widely used tactic. Separation can be used to avoid lateral movement, thus, helping to keep the TEE integrity. One way to achieve separation is by using virtualization, which is capable of creating logical isolation and allowing multiple applications to share the underlying hardware, unaware of other instances.

Although virtualization is a well-established technology in the Cloud, IoT virtualization is still in development. The requirements for embedded systems virtualization differs from enterprise systems, as restrictions about response time, processing power, memory size, and battery life are the primary concerns. The natural starting point for embedded virtualization was to adapt hypervisors widely used in server virtualization to embedded systems. However, their size and complexity proved to be unacceptable for small embedded devices, which motivated the appearance of hypervisors specially designed for embedded virtualization, as seen in Tiburski et al. [5]. Among the goals for the development of embedded hypervisors, two of them are frequently addressed: to keep low memory requirements and some level of support for real-time applications.

A strategy to make hypervisors lightweight is to simplify or even cut-off subsystems that are not necessary for embedded systems. Although memory management is essential for virtualization, since it provides the basis for separation, it must be adapted for IoT. For example, the swapping subsystem is unnecessary, and the paging implementation can be radically reduced, while a strong separation between domains is still enforced. The memory management can still be simpler if the processor implements hardware support for virtualization. If carefully designed, virtualization can provide security by separation on devices even smaller than that reached by containerization. Although containerization is known as lightweight virtualization, it still requires an underline operating system (OS), like Linux. Hypervisors for IoT are implemented as bare-metal, also known as type-1, controlling the hardware directly and dismissing an underline OS.


The combination of TEE and virtualization can be used to provide integrity checks over multiple domains. The hypervisor guarantees that, once an area is compromised, the attack will not spread over other domains, hence, allowing the coexistence of trusted and non-trusted environments. Additionally, different vendors can deliver their own with custom application-defined signatures. Therefore, it is possible to verify an application for non-repudiation, avoiding vendors to deny his responsibility or role. Finally, virtualization can go still more deeply than containerization on embedded systems, allowing cheaper devices to be used on the edge.


  1. M. Chiang and T. Zhang. Fog and IoT: An Overview of Research Opportunities. IEEE Internet of Things Journal, 3(6):854–864, Dec 2016
  2. OpenFog Consortium. OpenFog Reference Architecture for Fog Computing. Technical report, 02 2017.
  3. PeiYun Zhang, Mengchu Zhou, and Giancarlo Fortino. Security and trust issues in Fog computing: A survey. Future Generation Computer Systems, 88, 05 2018.
  4. M. Sabt, M. Achemlal, and A. Bouabdallah. Trusted Execution Environment: What It is, and What It is Not. In IEEE Trustcom, volume 1, pages 57–64, Aug 2015.
  5. R. T. Tiburski, C. R. Moratelli, S. F. Johann, M. V. Neves, E. d. Matos, L. A. Amaral, and F. Hessel. Lightweight security architecture based on embedded virtualization and trust mechanisms for iot edge devices. IEEE Communications Magazine, 57(2):67–73, February 2019.



Carlos Roberto MoratelliCarlos Roberto Moratelli received his Ph.D. in computer science from PUCRS. He is an adjunct professor at UFSC. He worked ten years in the telecommunication industry, acting on software engineering related to embedded systems. His research interests are embedded real-time systems, Linux Embedded, and virtualization for embedded systems.


Ramao Tiago TiburskiRamão Tiago Tiburski received his M.S. degree in computer science from PUCRS. He is a Ph.D. student of computer science at PUCRS and a professor at Federal Institute of Santa Catarina (IFSC). His research interests are IoT, fog and edge computing, and security for IoT resource-constrained devices.


Sergio F JohannSérgio F. Johann (sergio.filho@pucrs.br) received his Ph.D. degree in computer science from PUCRS. He is an adjunct professor at PUCRS, Brazil. He has experience in computer architecture design and organization, operating systems, embedded systems (design and integration), embedded software support, real-time systems, and control systems.


Everton de MatosEverton de Matos received his M.S. degree in computer science from PUCRS. He is an adjunct professor at Meridional Faculty (IMED). He is a Ph.D. student of computer science at PUCRS. His research interests are IoT, middleware, fog and edge computing, context-awareness, and context sharing.


Fabiano HesselFabiano Hessel (IEEE Member) is Full Professor of Computer Science at PUCRS. He received his Ph.D. in computer science from UJF, France (2000). He has experience as a General and Program Chair in several committees of prestigious conferences and journals. His research interests are embedded real-time systems, RTOS and MPSoC systems applied to IoT/SmartCities.