Madhurima Pore Vinaya Chakati Ayan Banerjee and Sandeep K. S. Gupta
Edge computing and fog computing have combined in a way to facilitate a wide variety of applications that involve human interactions, which are geographically distributed and have stringent real‐time performance requirements. The Internet of Things (IoT) or Internet of Everything (IoT) has introduced edge devices that now obtain information from user environment and need to respond intelligently to changes in real time. The scale of an application has increased from mere single mobile device to a large number of edge devices that are geographically distributed and change locations dynamically. Even though cloud support can be used in processing the data generated by the edge devices, the delay incurred in communication to cloud devices is excessively more than the real‐time constraints of some of the latency sensitive applications. With such a large scale of data being generated in geographically distributed locations, sending the code toward the data is in some cases more efficient than processing in the cloud. Fog computing introduced computation solution in the form of fog devices, cloudlets, and mobile edge computing (MEC), which provide computation services in the network edge. It can also meet the real‐time requirements of such applications.
Apart from the components that pertain to application logic, there are large design components that perform the underlying task of managing the network, computation, and resources of fog and edge architecture (FEA). Due to the dynamically changing context in the edge devices, underlying algorithms for managing the execution and processing data become complex. In addition to the processing and data communication of the distributed application, the control data and algorithm decisions incur excess resources when they are executed on the edge devices. In this chapter, we discuss different aspects of design of middleware for FEA.
Fog and edge computing are gaining acceptance due to high availability, low latency and low cost. Domains such as smart cities, virtual reality and entertainment, vehicular systems use edge processing for real‐time operations [1, 2]. The efficient design of middleware enables the realization of full potential of fog and edge infrastructures. Middleware handles different tasks such as communication management, network management, task scheduling, mobility management, and security management, thereby reducing the complexity of the distributed mobile application design.
Middleware design of fog/edge infrastructure is challenging because of the stringent application requirements such as (i) availability of context on the sensing devices; (ii) cost of data transfer and processing in different tiers of FEA; (iii) limitations on number of edge devices present and dynamic changes in context and mobility of the devices; and (iv) strict latency constraints. Including the dynamically changing context of a user and capturing the user interactions patterns can essentially enable intelligent and informed execution of the applications.
In this chapter, we present state‐of‐the‐art middleware for fog and edge infrastructures and propose an architecture of middleware that supports distributed mobile applications with specific requirements of applications. Proposed middleware primarily focuses on application‐aware task scheduling and data acquisition.
A varied class of mobile applications can utilize FEA middleware. Requirements for emerging applications can be summarized as following:
Even though the FEA can support different types of applications, common functionalities that are required in such applications can be provided by the middleware. Following are design goals of FEA middleware.
Data sources in fog/edge may belong to a wide category of devices ranging from IoT sensors, mobile devices to fixed sensors. The data are processed locally or sent to fog/edge devices for further processing. A channel of communication needs to be set up between the requesting devices and ad‐hoc discovered devices that perform the application task of acquisition and processing. Once a communication channel is set up, it allows dynamically changing set of devices to join and participate. Given the dynamic nature of participating edge devices that acquire and process the data, the device discovery allows setup of a communication layer to enable further communication between devices.
The middleware provides a platform that executes the application task remotely on the edge devices. Functionality includes code download, remote execution in the edge devices, and delivery of results such that they are available to the requesting device.
Task disruption during execution affects the reliability of execution of FEA task. Often it results in reinitialization of the task or undesirable/unavailable results. Device usage patterns, mobility, and network disconnections may cause unexpected changes in the context of the device. This may render the device inappropriate for continuing the execution of sensing or computation task. Anticipatory techniques can be used to minimize the interruption in tasks, thereby promoting intelligent scheduling decisions.
Establishing communication between ad‐hoc edge devices, selection of candidate edge devices, distribution of FEA tasks between multiple edge devices, and managing remote execution in a sequence of FEA tasks incurs additional usage of bandwidth and energy consumption on the edge devices. As these resources are expensive, minimizing these operational parameters is an important aspect of middleware operations. Additionally, several devices may enforce usage limit on their resources that are available for sharing.
Innovative contexts such as the mental state of [3–5] and user activity [6] are now used in mobile applications for sensing useful data. For successful execution in FEA, dynamic changes in the context of the devices as well its environment require the middleware to adapt to these changes. Self‐adaptive services can enhance its operations and improve the FEA quality of service.
Quality of service (QoS) of an architecture is highly dependent on the application. Many edge/fog applications use multidimensional data for achieving specific goals. Acquiring and processing such huge sensor data within real‐time constraints is a requirement for these applications. Real‐time response is an important QoS measure. Other application specific QoS parameters can be the relevance of the acquired data, its correctness, and uninterrupted data acquisition.
Applications in fog and edge computing are discussed in some of the recent works. Real‐time data streaming applications include traffic monitoring Waze [7, 8], smart traffic light systems [9], real‐time replay in the stadium [10], and video analytics [12, 13]. Real‐time applications that process the requests for emergency rescue in disaster and searching for missing persons [10, 11]. Application of geographically distributed systems such as wind farms [9] and smart vehicle‐to‐vehicle systems [14] are becoming popular in fog and edge computing. Common requirements of these applications present a need for middleware to support easy design and development of such applications.
Middleware features such as security, mobility, context awareness and data analytics addressed in recent research are shown in Table 6.1. Popular IoT platforms such as GoogleFit [15, 16] have a cloud‐based IoT middleware for smartphones. Provisioning of sensing services on the mobile devices is discussed in M‐Sense [17]. Service oriented middleware like GSN [18–20] are proposed for processing data in the distributed environment. Further, Carrega et al. propose microservices‐based middleware using a user interface [20]. In FemtoCloud system, the mobile devices in the edge can be configured to provide the services to requesting devices [21]. “Process on Our Own (PO3)” concept where a data stream generated on each device is processed on itself is proposed by Nakamura et al. [22]. CoTWare middleware proposed by Jaroodi et al. suggests a novel way of integrating things, fog devices, and cloud by using cloud‐hosted services to manage processing of IoT data in fog resources [23]. MobiPADs [24] and MobiCon [25] are context‐aware middleware solutions that enable adaptive design in mobile applications by reconfiguring the services with respect to dynamic changes in context for edge devices. Recently proposed CloudAware [26] is an example of adaptive middleware for constantly changing context such as connectivity for cloudlet.
Table 6.1 Middleware features in fog and edge architectures.
Devices | Security | Mobility support | Context awareness | Data analytics | Optimized selection of devices | |
FemtoCloud [21] | Mobile | N | N | Y | Y | Y |
Nakamura et al. [22] | Mobile and sensor | N | N | N | Y | N |
Aazam et al. [27] | Fog, MEC, Cloud | Y | N | N | Y | Y |
Bonomi et al. [9] | Fog, Cloud | N | Y | N | Y | Y |
Verbelen et al. [28] | Mobile Cloudlet | N | N | N | N | Y |
Cloudaware [26] | Cloudlet | N | Y | Y | N | Y |
Hyrax [10] | Cloud | N | Y | N | N | Y |
Grewe et al. [14] | MEC | Y | Y | N | Y | Y |
Carrega et al. [20] | MEC, Fog | Y | Y | N | Y | Y |
Piro et al. [16] | Cloud | Y | Y | Y | Y | Y |
FEA includes devices that can be broadly classified into five types, as shown in Figure 6.1. Mobile devices connected with sensors and actuators are the nearest devices to the users. As the processing devices move away from the edge, the latency of the communication increases. Also, the availability of resources for processing and data storage increases toward the cloud. The components of FEA are discussed in detail in the following sections.
Embedded sensors and actuators are installed in physical structures or deployed on a human body. The sensors are responsible for obtaining the environmental or physiological signals that are processed by the system, while the actuators execute the actions initiated by the system. Built‐in networking capabilities allow these devices to communicate to the nearby devices. They may also have a limited computing capability.
Personal devices, or smartphones, inherently demonstrate mobility as they are owned by human users. These devices connect with the embedded sensors and actuators. They often act as an intermediate data hub or computation platform, and/or provide a communication link to servers. A part of their resources may be shared to execute the fog and edge distributed applications.
Fog servers are computationally more powerful than the personal mobile devices.
As these devices are closer to the edge, they provide a cheaper option for offload with respect to communication costs. These nodes exist between the edge devices and cloud. They can be used to process data and also act as intermediary storage. Further, communication to other edge devices can be achieved through peer‐to‐peer (P2P) or device‐to‐device (D2D) techniques such as WiFi, Bluetooth, and WiFi Direct.
Cloudlets were proposed by Satyanarayan et al. [29] as a small‐scale dedicated set of servers that have high bandwidth Internet connectivity but are situated close to the edge. They are also known as a data center in a box. Another implementation of edge computation is offered by telecom companies that bring the compute resources in the base station of mobile towers. They are known as mobile edge computing (MEC) servers.
Cloud servers have the most computational, communication, and storage capability in the hierarchy. The cloud servers are usually associated with a pay‐as‐you‐go model. They can easily scale the number of VMs according to the request.
Fog and edge computing applications include the following: (i) batch processing that needs large‐scale data acquisition and distributed processing; (ii) quick‐response application that needs a response in real time; and (iii) stream applications that require processing of a continuous data stream in real time [11, 12].
Such applications exist in different domains such as healthcare, emergency rescue and response systems, traffic management, vehicle‐to‐vehicle systems, and environment monitoring. Due to huge processing requirements, such applications need a large distributed architecture that processes data in multiple tiers. Lower tier near the edge perform filtering, preprocessing, and extraction of useful information while the edge and fog servers are used for processing and analytics. FEA (Figure 6.2) mainly consists of middleware services that are common to FEA applications, as discussed below.
Services common to fog and edge applications can be designed as an API. The API is then integrated into an app, enabling the design of different functionalities in the middleware with simple and easy to use functions.
Security in edge devices is essential to ensure access to authorized users and establish a secure communication channel for communication of user data.
Data ownership and protecting access to the private information is very important to the edge participants. Many “pay‐as‐you‐go” services necessitate authentication to prevent any unwanted access. Public key infrastructure‐based systems have been proposed for user and device authentications for distributed key management [30]. Authentication of VM instances and migration of VMs that has been used in cloud [31] can be adopted for VMs in MEC and fog servers. Authentication as a service is proposed by Mukherjee et al. to enable authentication in the participating fog nodes [32]. Ibrahim proposed a lightweight mutual authentication scheme for roaming fog nodes [33] using one long‐lived master secret key. This algorithm is efficient and lightweight for limited resource devices such as sensors and actuator and doesn't require the devices to re-register.
Data privacy is very important with respect to handling data from user devices. The main challenge in FEA is to ensure the privacy of devices that exhibit mobility in the edge. Though the sensors and edge devices have limited resources, the fog nodes can provide the necessary encryption capabilities for edge processing [34]. Existing works propose anonymization techniques or pseudo anonymization through which user identity is protected. A lightweight privacy preserving scheme using Chinese remainder theorem is proposed by Lu et al. [35]. Laplacian mechanism query model is proposed by Wang et al. [36] that deals with privacy preservation of location‐aware services. Policy‐based access control mechanisms for fog computing is proposed by Dsouza et al. in [37].
Many existing studies propose the use of data encryption [34]. Recent work proposes crypto processors and encryption functions [38]. However, using encryption increases the computation, energy usage, and time incurred.
Device discovery allows the new devices to participate and leave the network as they become available in the network. Many researchers use MQTT as the standard publish‐subscribe message API [39], which is a lightweight messaging protocol, designed for constrained devices and low‐bandwidth, high‐latency, or unreliable networks. Fog and edge‐distributed middleware can also use pub‐sub as a service from a third party such as Nearby Message [40], or PubNub [41] that may be integrated into the middleware. Additionally, these services may provide security, scalability, and reliability for message exchange.
Components of middleware commonly used in fog and edge applications are discussed in the following subsections.
FEA can adapt to dynamic changes in the user environment using the context‐aware design of middleware. This may involve continuous monitoring of relevant context and adaptive actions that are based on changes in the context. Also, recent research shows that several human‐dependent contexts have patterns. These patterns can be learned to intelligently manage the operations between multiple devices. Several techniques such as time series, stochastic, or machine learning can be used to model and predict the human‐mobile contextual changes [42, 43].
FEA employs devices from the environment that can sense and/or process the data acquired in the FEA applications. Selection of the surrogate device can be based on different policies designed in the middleware. Research shows several policies such as fairness‐based selection, game theoretic [8], context optimization [44], and resource optimization approaches that are used in surrogate selection. Participating users are selected based on different criterion ranging from simple user context such as the location of the device to selection based on the reputation of user task completion history [45]. Following are different surrogate selection techniques.
Remaining battery is critical to every mobile user and it determines the amount of resources that the device owner may share. Selection of surrogates is a trade‐off between the quality of information gathered and the remaining battery on the device with an incentive budget [46].
Real‐time applications and streaming data application require the processing to be completed in a given time constraint [12]. Performance‐based selection of surrogates is proposed in incentivized schemes by Petri et al. [47].
Context‐aware functionality is used in many mobile applications. Applications are designed to adapt themselves based on changes in context on the mobile device or that of the user. Recently proposed context‐aware recruitment scheme focused on improving the mobile selection based on context requirements of the application [44]. In applications like crowdsensing apart from individual context prediction, large‐scale activity prediction such as proposed in [48] is now becoming useful. Change in location of mobile users can be modeled using different techniques such as random waypoint model and statistical models such as Markov [49, 50]. Spatiotemporal model of user location is proposed by Wang et al. [51].
FEA introduces the idea of processing near the edge. Extensive analytics may be involved in an application that is processing across different layers in the architecture. Some of the analytics tasks can also be used to extract the essential information from the raw data obtained on the user devices. This not only reduces the processing requirements centrally but also reduces the communication costs. Data analytics module on the user device can be used to send essential data towards a central server [52]. Cloud server/data center may be used to aggregate information and process high‐level data analytics tasks. Bonomi et al. [9] discuss processing data analytics tasks in multiple use cases in a fog/edge environment.
This engine works continuously to monitor the incoming tasks and their assignment using the surrogate selection policy. It monitors the availability of resources in different layers such as the availability of new, incoming user devices as well as tenant resources such as VMs that process data in fog devices and the cloud.
FEA uses the multitier network to distribute the fog and edge applications. It may use software‐defined networking or virtual network topology in multitenant resources in fog and cloud devices. User devices are usually connected using point‐to‐point network topologies that may either use TCP socket – WiFi connection, WiFi direct, or Bluetooth communication. This module is also responsible for monitoring connection and triggering the connection resume procedures for a lost connection.
This module facilitates the application specific code functionality to execute on the edge and fog nodes. Existing work in fog computing proposed the use of a virtual environment [28] or use of private OS stack provided by CISCO iox [53]. Virtualization with migration support on mobile devices is proposed by Bellavista et al. [54]. In some research, the code offload techniques such as DEX compositions in android [55] or .NET may be used. Other works propose plug‐in based designs that are downloaded and integrated into the app in runtime [56].
MEC supports mobile edge devices that are constantly on the move. In such cases, the data and the middleware services follow the devices. The idea is commonly known as Follow me Cloud (FMC) [57] and uses Locator/ID separation (LISP) protocol.
The sensors handle the important task of obtaining real‐time data from the environment and user's surrounding. The information obtained through sensors is used in several forms. Sensor data may be acquired in the FEA application itself. It can also be used to evaluate and extract context information of the device user. In more complex applications, the closed‐loop information is acquired and analyzed and further used for taking real‐time actions using the actuators.
This section describes an example of a perpetrator tracking application that can be designed through middleware in Section 6.5. This is a mobile application that performs real‐time tracking of perpetrators through video surveillance using surrogate mobile phones available in the vicinity:
Different aspects of middleware can be explored in future research in order to improve the performance of mobile distributed applications.
The FEA may involve context‐based decisions in several aspects of middleware, such as choice of participating devices, activation triggers of distributed applications, and anticipatory scheduling based on historical context data. Increasingly, more context‐aware control and management of fog and edge devices will be used to intelligently schedule the distributed applications. The existing works primarily focus on location tracking of the users [51] and several other useful contexts [58] for executing an application. Several other contexts such as user activity patterns, prediction of user environments, and device usage patterns may be explored to improve the execution of the distributed application. Anticipatory context‐aware computing is studied for mobile domain [59], but its adoption in a collaborative edge environment is yet to be seen.
Mobile edge nodes need to provide services as requested by the application as they move from one location to another. The standard methods of network virtualization and VM migration exist. Research involves managing costs in VM migration, consideration of the mobility changes in edge devices, intermittent connectivity, and task partitioning within time constraints. In a mobile environment like vehicle‐to‐vehicle systems focuses on the prefetching, data caching and migration of services [14]. Future work in MEC must address how to guarantee service continuity in highly dynamic scenarios.
Participating nodes can include many private devices, as well as resources with ownership of telecom companies or companies that provide cloud computing services. With a wide variety of devices involved, establishing and maintaining a secured channel of communication that is lightweight enough to execute on the personal mobile devices is a future research area. Methods such as encryption of data and key‐based authentication can incur excessive energy and computation on edge devices that have limited resources. Another area of possible research is the secure offload of application tasks to other edge devices for outsourcing.
Traditionally, VM management includes migration and replication that easily allows the transfer of the soft state of the application from one node to another. Such techniques are currently being proposed in fog resources [60]. However, heterogeneity in networks and devices in FEA is a barrier for migration. A dynamically changing set of suitable devices to execute application requires seamless handover of tasks. Methods, checkpoints, and offload mechanisms need to be explored to make this handover seamless as well as time and resource efficient. The devices must be able to make real‐time decisions regarding whether to execute the task in the cloud or in the edge. Also, they need to account for the overhead of management and other security features such as encryption for optimal task scheduling.
Modular software components should exist vertically in different layers FEA. Different platforms such as network virtualization and software‐defined networking (SDN) are being explored [61] for orchestrating distributed execution in edge resources. Standard protocols like OpenFlow are promising options for virtual network design across cloud, fog, and edge devices. Recent works propose the use of dockers with migration support for edge processing [11].
Existing research demonstrates SLA for VM in edge devices [62] for MEC architectures or MCC [63], which are static resources. However, in the case of edge devices associated guarantees with the payment model is not studied. As such, the fog nodes and other edge participants incur energy and bandwidth as well as provide access to several resources and private data while executing distributed applications. Designing a payment model for such fog and edge node services is yet to be explored.
Some of the existing works propose service‐oriented middleware approach for scalability of edge devices [20]. For data acquisition and processing from a large number of edge devices that might be geographically distributed, the middleware design can be distributed with decentralized processing and decision making [64]. Hierarchical clustering may be an approach for designing these systems to attain the specific goal of edge applications such as real‐time responsiveness.
In this chapter, we discussed the changes introduced by edge and fog devices in the distributed computing. Fog and edge architecture can now support mobile sensing applications that are real time, latency sensitive, and geographically distributed. The dynamically changing set of fog and edge participating devices also need more design support to execute such distributed applications. We discussed middleware architecture and existing works that deal with different aspects of fog and edge computing, such as context awareness, mobility support, selection of edge participants apart from the network, and computation management. Broadly speaking, these architecture aspects can be adapted to MEC, fog, and cloudlet implementation of FEA. We also highlighted some of the newer areas of research that will improve the design of FEA in the future.