Intelligence Orchestration for Future IoT Platforms

Edgar Ramos, Senthamiz Selvi, Roberto Morabito, Aitor Hernandez, Valentin Tudor, and Jan Höller
November 16, 2021



The expected increase in IoT investments is mainly driven by the adoption of Digital Twins and Artificial Intelligence (AI) enhanced IoT applications. Therefore, accelerating the building, deployment, and management of these IoT applications is crucial and can be enabled by orchestrating IoT components, devices, functions, and systems. However, IoT Intelligence orchestration raises several challenges that need to be addressed to allow a broad spectrum of domain-specific use cases and applications to follow and support business logic.

In the context of IoT systems, the orchestration of processes and intelligence goes beyond the idea of cloud-based orchestration. In cloud systems, an orchestration framework mainly deals with the deployment of microservices and the interactions between them with their platforms and hosting environments. Differently, an intelligence orchestration for evolving future IoT platforms demands additional components that allow the management of the entire lifecycle of intelligent-based IoT services. This also includes the fulfillment of performance requirements that depend on optimizing the AI algorithms to be executed and the near real-time delivery capabilities of IoT mobile networks. To the same extent, such emerging platforms need to embed the flexibility given by conventional orchestration frameworks on managing microservices, the capability of consolidating the scattered world of AI-based systems and effectively handling mobile connectivity among the different computing entities involved. Thus, as schematically depicted in Figure 1, there is richness in needed system capabilities related to how IoT applications become intelligent that needs to be addressed at a high level by future IoT platforms.

Figure 1: High-level challenges and characteristics of the intelligence orchestration services for future IoT platforms.

Figure 1: High-level challenges and characteristics of the intelligence orchestration services for future IoT platforms.

Heterogeneity plagues IoT in many aspects, from the devices' nature and their platforms, the services they provide, the use cases they serve, the stakeholders and actors who interact with them, the applications they serve, and their communication needs and patterns, and several other aspects.

Suppose we assume that the highest-level goal of an intelligent system is to fulfill a use case. In that case, each use case will be characterized by a set of procedures, goals, and interacting entities that need to be considered when being implemented. A first step towards facilitating such an intelligent system is to model the use cases based on services or application workflow. The decomposition of services and applications through chained functions is also referred to as workflows in this text.

Challenges and Characteristics of IoT Intelligence Orchestration

The intelligence services supporting applications rely on functions executing machine learning or AI models and assisting data processing functions. These services should be decoupled from the application but placed in a context, customized with input data, and their output directed where deemed useful (an application or another function or process). This requires that such functions be exposed to be manipulated or "orchestrated" together with other functions that may reside in the same or different systems or physical devices. In some cases, functions are being used by an application residing in a cloud, a control system, or even an industrial machine. To make the functions addressable and customizable, it is required that they are made available to other entities different than their platform or processing units (incl. devices) through what is known as service exposure.

Figure 2: IoT Intelligence orchestration structures and processes.

Figure 2: IoT Intelligence orchestration structures and processes.

An orchestrator employing the services would need to find where and what services are available. Service discovery becomes a key requirement to navigate the complexity of distributed service registrars that hold peer relationships with the service holders. This implies utilizing proper querying semantics to match meaningful services with interoperable retrieval mechanisms and localization of such services and their exposure in a secure way. The registration of such services to a register in such a heterogeneous environment to be later exposed is also a challenge. It may involve federation and peering between multiple registration instances to enable the discovery, taking into account nomadisms and the activity cycle of devices. In addition, new services may be on-boarded or old services updated which implies changes that need to be propagated to the exposure visible to an orchestrator. Also, an orchestrator may envision the on-boarding of certain services in one particular IoT device, tapping into the life cycle management of the functions and their relationship to the software updates on the device. In this aspect, the availability of the device's capabilities (ex. constraints or AI services acceleration) requires special attention.

The workflows are implemented through chained functions where the output from one function is used as input for another, in the fashion of intelligence pipelines. Defining these intelligence pipelines also implies their activation triggers and their connection to the events related to business logic, operations, and applications.

During the execution of the workflows, it is essential to have access to observability and monitoring in terms of resource utilization and execution performance to provide optimizations and auditing of the systems. Another type of necessary monitoring is about the progress of the functions or the workflow progress. In the case of machine learning, the explainability of the information to document the inference process.

When extending the concept of workflows to cover not only the view of one party, from where the processes have been organized but also include multiple stakeholders to the orchestration processes. This also brings the issue of resource administration, particularly when such resources are shared, including the security and policies considerations of when, by who, and for what a resource can be used or not, what are the priorities between the different tenants, and the need for isolation between their "execution spaces". Workflow Security and Privacy must address both the component orchestration and the data processed by these components. Given the heterogeneity of the IoT components, the trustworthiness of the whole workflow is dictated by proper authentication and authorization of its components, the secure connection and data communication between pipelines, and their underlying hardware and software security characteristics. Therefore, security and privacy must be intrinsic to the workflow components and integrated as early as possible in the design phase. In addition to this, to ensure interoperability, these solutions need to be standardized. They should be expressed via policy and intent languages that can be interpreted and/or translated by the various components.

Application workflow modeling and instantiation are needed, including a proper integration with the orchestration functionality to realize the desired functionality and match the underline deployments. Efforts in this direction have already been taken, e.g., based on BPMN. All these aspects should be possible to be described in some detail to make such a workflow specification portable. The portability may be addressed by a device itself or by a supporting middleware engine that makes adaptation to a specific IoT device possible. Either way, it would require some degree of coordination to manage the connections and relationships between instances belonging to the same workflow and ensure that their instantiations are successful and functional.

Traditionally, a human domain expert has been in charge of composing the needed processes and mapping them to the automation functions in a tailor-made integration. However, the enablement of the intelligence orchestration also implies that a reasoning agent, implementing AI cognitive technology, may be used to compose the workflows either from experience models (i.e., templates or blueprints) or from intents or goal (objective) oriented behavior models. This kind of AI would close the loop of automation and provide even higher optimizations that are not evident to human abilities.

Figure 2 illustrates the different structures and functions described in this section and how they relate from the device and infrastructure perspectives to the management and orchestration services.

Use Case Example: Smart Logistics

To contextualize the concept of an intelligence orchestrator, let us consider a use case that is rich in AI features, namely Smart Logistics. A logistics application is the movement of items from one point to another and involves several stakeholders and transportation modalities. Transporters carry items in aggregations, like items in a crate placed on a pallet that goes into a container carried by trucks, rail, etc. During the end-to-end logistics process, items are loaded and unloaded at logistics hubs and generally change transporters and aggregations. A suitable illustration is the food supply chain where different food types have different handling requirements, like temperature (e.g., frozen vs. refrigerated), humidity (vegetables vs. groceries), orientation (herbs in pots). It may even be alive (crabs, oysters). As can be easily understood, monitoring various sensing modalities during the end-to-end process is vital by embedding IoT devices across the aggregation hierarchies. If the condition during the transportation varies, it can have different effects on the items that can require necessary contingency actions, like the need to speed up delivery for some food types by re-loading on another transporter, changing best-before dates, or even discarding them before reaching the endpoint. There are many reasons for unexpected events, like traffic delays, poor goods handling, and unexpected environmental conditions. Hence, the entire process itself needs to be monitored so that any planned initial logistics flow can be constantly re-planned. Also, gathering information on the quality of different distributors, handlers, etc., gives insights about reliability that can be factored into the logistics planning process. As can be understood, AI capabilities are central to achieving all objectives on a reliable and efficient end-to-end logistics process. AI is, in this example, the "logistics conductor," which will be highly distributed from constrained devices to distributed across domains, with a large variety in insights and continuous decision making to achieve the different intents and objectives. Key features include planning and adaptive re-planning, monitoring, and fulfillment like predictive analytics, anomaly detection, actuation, and post-process analysis of observed behaviors and actions. There is also a requirement on full reconfigurability and programmability at all tiers with individual and aggregate level analytics and facilitating closed loops of work order creation, planning, execution, and delivery.

As understood by this example, an AI orchestrator must provision and re-provision used devices with the suitable algorithms and manage the employed models' lifecycle management up to their deployment depending on the availability and needs of the use case at hand. How to effectively model such a use case is not a straightforward task. Details such as data pipelining, configurability, and data processing on the device or cloud need to be managed. From the given intent or objective, the modeling of the solution, deployment, and execution must be met by the intelligence orchestrator. Some of the use cases may require processing at the edge, while sometimes it needs to be offloaded to the cloud due to certain requirements or constraints. The orchestrator must manage all the needs of such use cases and facilitate ease of use. The orchestrator is responsible for maintaining the inventory of available devices, services, goals and must provide an optimal configuration for the deployment, application parameters, and maintenance. It must facilitate service discovery of available cloud capabilities and match device capabilities. Also, some policy requirements can be applied on use cases, such as the required quality of service (QoS) to be maintained up to a certain degree, latency parameters since these applications are mostly real-time or part of critical services. The orchestrator thus needs to cover all the requirements and mitigate plans to model and deploy them on a suitable execution environment. Orchestrating intelligence is hence a key feature of any IoT platform to enable future complex applications.


Future empowering IoT platforms will need to handle heterogeneous use-cases and scenarios with complex interactions between different actors, components, and functions. Decoupling the intelligence components from the IoT applications is one step towards accelerating the creation and deployment of new IoT applications which can be provisioned by the same intelligence components. The highlighted intelligence orchestration challenges and characteristics presented here need to be addressed together in/by the IoT ecosystem to capture heterogeneous properties and enable, enhance, and simplify the interoperability between the various components.

As a disclaimer, this article does not do an extensive survey of all the requirements relevant to this topic. Still, it attempts to call attention to some of the most interesting topics in the area and generate awareness on the need to understand them and research them further.



Aitor HernandezAitor Hernandez is a master researcher at Ericsson. His research interests include cloud technologies and network evolution, focusing on the orchestration of distributed computing and intelligence in the context of the Internet of Things (IoT). He holds a double degree in Telecommunications Engineering and Electronics Engineering from Barcelona School of Telecommunications Engineering (UPC), Spain.


Senthamiz SelviSenthamiz Selvi is a Senior Researcher at Ericsson Research, currently focusing on the Internet of Things and Artificial Intelligence. In addition, she is an HFI-Certified Usability Analyst. Selvi joined Ericsson in 2009, working on the Business Support System portfolio products as a Software Architect before moving to the corporate research unit. She received her bachelor's degree in Computer Science and Technology from Usha Mittal Institute of Technology, Mumbai. Her current research interests include IoT, Machine Learning, Distributed Ledger Technologies and Business Process Modelling.


Valentin TudorValentin Tudor is a Senior Researcher in the IoT Technologies and Cyber-Physical Systems team at Ericsson Research. His research interests include security and privacy issues in IoT, focusing on cyber-physical systems and industrial applications. He holds a Ph.D. degree from Chalmers University of Technology, Sweden.


Roberto MorabitoRoberto Morabito is a Senior Researcher in the IoT Technologies and Cyber-Physical Systems team at Ericsson Research. His research interests lie in the intersection between IoT, Edge Computing, and Distributed Artificial Intelligence. He holds a Ph.D. in Networking Technology awarded by Aalto University in 2019. In addition, he has been awarded the Marie Curie Fellowship as part of the EU-funded project METRICS. Between 2019 and 2021, he has also been a research associate with Princeton University.


Edgar RamosEdgar Ramos' research has been focused on the AI and intelligence enabling for IoT and far edge. He has a long experience in wireless systems protocols and architecture from 3G to early 5G design. He has been involved in concept development for heterogeneous networks and contributed to the standardization of various features. He received his M.Sc. degree in Computer Sciences from Helsinki University of Technology in 2007 and a Computer system engineering degree from the Universidad Tecnológica de Panamá in 2000 and is currently a Ph.D. Tech. Candidate in Aalto University, Finland. He has been part of Ericsson since 2001 and is presently a Master Researcher in the IoT technologies and Cyber-Physical Systems team at Ericsson Research.


Jan HollerJan Höller is a Research Fellow at Ericsson, driving technology and research strategies and related activities around IoT and Cyber-Physical Systems. He has, over the years, also contributed to corporate and product division strategies in the field. He established Ericsson's research activities on IoT in 2006 and has been active in several research collaborations with academia and industrials. Jan is a co-author of the book "Internet of Things: Technologies and Applications for a New Age of Intelligence," recently released in its 2nd edition by Elsevier. He has held various positions in strategic product management and technology management and has, since he joined Ericsson Research in 1999, led different research activities and research groups. Jan currently serves on the Board of Directors of the Open Mobile Alliance and is a co-chair of the Networking Task Group in the Industrial Internet Consortium.