Why Trusted Execution Environment and Security by Separation on IoT Edge Devices Are Important?

Carlos Moratelli, Ramão Tiago Tiburski, Sergio Johann, Everton de Matos, and Fabiano Hessel
September 16, 2019



The Internet is changing quickly into a model where billions of everyday objects will be interconnected, which we call the Internet of Things (IoT). Traditionally, IoT devices communicate directly with the Cloud, but that is changing to a layered architecture. The direct IoT-Cloud model works poorly for a fair share of all applications.

For example, the amount of data generated by sensors will be prohibitive in some instances, as seen on connected cars, which can create tens of megabytes each second [1]. Some long-range, low bandwidth radio technologies, like that provided by SigFox or NB-IoT, are charged by communication amount, making it desirable to minimize the data exchange. Some applications require fast response, as voice recognition. In this context, Fog and Edge layers were added to the IoT architecture [1] as an alternative to diminish the cloud communication and perform faster response. Therefore, sensors will communicate with nearby devices (edge) interconnected by medium-range networks (fog). Data processing and local decisions will be performed at the Fog/Edge layers avoiding additional communication with the Cloud. [2].

Behind the IoT well-known benefits, it is hidden an obscure treat: the digital security risks. Security emerged as one of the most critical concerns for the broad IoT adoption. Wonder the Internet flooded by millions of potentially vulnerable edge devices with significant processing power. Such devices will bring their vulnerabilities to private networks, turning the Internet into a fertile environment for hackers willing to steal sensitive information or to perform denial-of-service and denial-of-sleep attacks. Improvements in the security of all devices connected to the Internet are a vital concern for the future of the IoT [3].

In the run for safer devices, a realm of technologies can be applied. In this article, we discuss how two fundamental security trends can be put together to build the foundation of IoT edge security. First, the use of the Trusted Execution Environment (TEE) is essential to guarantee software and data integrity. A TEE requires separation to allow the concurrent execution of multiple isolated flows, so security by separation is also addressed.

Figure 1: Establishing a CoT from hardware to higher software layers.

Figure 1: Establishing a CoT from hardware to higher software layers.


Trusted Execution Environment (TEE)

As many edge devices are placed in public environments with easy access by non-authorized personnel, it is necessary to guarantee that the running software was not modified or changed maliciously. Even devices without a physical interface may be attacked by having its code or data changed remotely. A TEE allows for the detection of unintentionally software substitution, consisting of a protected machine’s memory area. In this environment, application code and data are verified for confidentiality and integrity using cryptography before execution [4]. There are two elements to build a TEE: the Root of Trust (RoT) and the secure-boot process. Both parts result in the called Chain of Trust (CoT).

The RoT is a trusted element that cannot be changed and constitutes the foundations for the device’s software integrity [4]. A typical implementation approach consists of hardware capable of performing software verification based on a cryptography key stored in a write-once memory. The chip’s manufacturer is responsible for providing support for verification and storage memory. In this scheme, developers are responsible for the software stack, so updates are possible even on devices already in the field.

The RoT allows for the secure boot process, where only verified software can be executed on the device’s startup. Therefore, this mechanism involves a set of verification at all layers of the system’s software until the level of the application, implementing end-to-end security, and defining a Chain of Trust (CoT). Figure 1 describes this scheme in a multi-layered environment, which includes an embedded virtualization layer. First, the hardware authenticates the bootloader. If successful, it is considered a trusted element and is allowed to verify the next boot stage. In this case, the next boot stage to be trusted is the hypervisor. Once verified, the hypervisor boots up and check its domains before their boot. Note that, non-trusted domains can coexist along with trusted areas in the same device which will be better addressed in the following sections.

Security by Separation

Current edge devices require significant processing power in order to handle sensor data, decision making and to communicate over the Fog layer. As a consequence, software complexity increases, and multiple separated execution flows are required. Enforcing isolation between the flows in a lightweight way, while still maintaining the TEE, can be challenging. In a compromised system, an attacker may try to spread its attack to other subsystems, taking control of all possible functionalities: this is called lateral movement and is seen as a widely used tactic. Separation can be used to avoid lateral movement, thus, helping to keep the TEE integrity. One way to achieve separation is by using virtualization, which is capable of creating logical isolation and allowing multiple applications to share the underlying hardware, unaware of other instances.

Although virtualization is a well-established technology in the Cloud, IoT virtualization is still in development. The requirements for embedded systems virtualization differs from enterprise systems, as restrictions about response time, processing power, memory size, and battery life are the primary concerns. The natural starting point for embedded virtualization was to adapt hypervisors widely used in server virtualization to embedded systems. However, their size and complexity proved to be unacceptable for small embedded devices, which motivated the appearance of hypervisors specially designed for embedded virtualization, as seen in Tiburski et al. [5]. Among the goals for the development of embedded hypervisors, two of them are frequently addressed: to keep low memory requirements and some level of support for real-time applications.

A strategy to make hypervisors lightweight is to simplify or even cut-off subsystems that are not necessary for embedded systems. Although memory management is essential for virtualization, since it provides the basis for separation, it must be adapted for IoT. For example, the swapping subsystem is unnecessary, and the paging implementation can be radically reduced, while a strong separation between domains is still enforced. The memory management can still be simpler if the processor implements hardware support for virtualization. If carefully designed, virtualization can provide security by separation on devices even smaller than that reached by containerization. Although containerization is known as lightweight virtualization, it still requires an underline operating system (OS), like Linux. Hypervisors for IoT are implemented as bare-metal, also known as type-1, controlling the hardware directly and dismissing an underline OS.


The combination of TEE and virtualization can be used to provide integrity checks over multiple domains. The hypervisor guarantees that, once an area is compromised, the attack will not spread over other domains, hence, allowing the coexistence of trusted and non-trusted environments. Additionally, different vendors can deliver their own with custom application-defined signatures. Therefore, it is possible to verify an application for non-repudiation, avoiding vendors to deny his responsibility or role. Finally, virtualization can go still more deeply than containerization on embedded systems, allowing cheaper devices to be used on the edge.


  1. M. Chiang and T. Zhang. Fog and IoT: An Overview of Research Opportunities. IEEE Internet of Things Journal, 3(6):854–864, Dec 2016
  2. OpenFog Consortium. OpenFog Reference Architecture for Fog Computing. Technical report, 02 2017.
  3. PeiYun Zhang, Mengchu Zhou, and Giancarlo Fortino. Security and trust issues in Fog computing: A survey. Future Generation Computer Systems, 88, 05 2018.
  4. M. Sabt, M. Achemlal, and A. Bouabdallah. Trusted Execution Environment: What It is, and What It is Not. In IEEE Trustcom, volume 1, pages 57–64, Aug 2015.
  5. R. T. Tiburski, C. R. Moratelli, S. F. Johann, M. V. Neves, E. d. Matos, L. A. Amaral, and F. Hessel. Lightweight security architecture based on embedded virtualization and trust mechanisms for iot edge devices. IEEE Communications Magazine, 57(2):67–73, February 2019.



Carlos Roberto MoratelliCarlos Roberto Moratelli received his Ph.D. in computer science from PUCRS. He is an adjunct professor at UFSC. He worked ten years in the telecommunication industry, acting on software engineering related to embedded systems. His research interests are embedded real-time systems, Linux Embedded, and virtualization for embedded systems.


Ramao Tiago TiburskiRamão Tiago Tiburski received his M.S. degree in computer science from PUCRS. He is a Ph.D. student of computer science at PUCRS and a professor at Federal Institute of Santa Catarina (IFSC). His research interests are IoT, fog and edge computing, and security for IoT resource-constrained devices.


Sergio F JohannSérgio F. Johann (sergio.filho@pucrs.br) received his Ph.D. degree in computer science from PUCRS. He is an adjunct professor at PUCRS, Brazil. He has experience in computer architecture design and organization, operating systems, embedded systems (design and integration), embedded software support, real-time systems, and control systems.


Everton de MatosEverton de Matos received his M.S. degree in computer science from PUCRS. He is an adjunct professor at Meridional Faculty (IMED). He is a Ph.D. student of computer science at PUCRS. His research interests are IoT, middleware, fog and edge computing, context-awareness, and context sharing.


Fabiano HesselFabiano Hessel (IEEE Member) is Full Professor of Computer Science at PUCRS. He received his Ph.D. in computer science from UJF, France (2000). He has experience as a General and Program Chair in several committees of prestigious conferences and journals. His research interests are embedded real-time systems, RTOS and MPSoC systems applied to IoT/SmartCities.




Subscribe to the Newsletter

Join our free IoT Technical Community and receive our Newsletter.

Subscribe Now

Calendar of Events

IEEE International Conference on Omni-layer Intelligent Systems (COINS 2020)
27-29 July 2020
Barcelona, Spain

IEEE WF-IoT 2020 – IoT Tutorial Week Virtual Event
3-7 August 2020

IEEE WF-IoT 2020 – IoT Standards Week Virtual Event
17-21 August 2020

IEEE WF-IoT 2020 – IoT Vertical and Topical Week Virtual Event
14-18 September 2020

IEEE IoT Smart Cities Summit - Rabat, Morocco
16-17 September 2020

See More Events

Call for Papers

IEEE Internet of Things Journal

Special Issue on Enabling Massive IoT with 6G: Applications, Architectures, Challenges and Research Directions
Submission Deadline: 1 October 2020
Special Issue on Space-Air-Ground Integrated Networks for Internet of Vehicles
Submission Deadline: 15 September 2020
Special Issue on Internet of Things for Smart Health and Emotion Care
Submission Deadline: 1 September 2020
Special Issue on Intelligent Blockchain for Internet of Things
Submission Deadline: 15 August 2020
Special Issue on 6G-Enabled Internet of Things
Submission Deadline: 1 August 2020

See More IoTJ Call for Papers

Past Issues
May 2020
March 2020
January 2020
November 2019
September 2019
July 2019
May 2019
March 2019
January 2019
November 2018
September 2018
July 2018
May 2018
March 2018
January 2018
November 2017
September 2017
July 2017
May 2017
March 2017
January 2017
November 2016
September 2016
July 2016
May 2016
March 2016
January 2016
November 2015
September 2015
July 2015
May 2015
March 2015
January 2015
November 2014
September 2014