Edge Clouds – Pushing the Boundary of Mobile Clouds
The Edge... there is no honest way to explain it because the only people who really know where it is are the ones who have gone over.
Hunter S. Thompson
Abstract
Edge cloud is pushing the frontier of cloud's services and resource management, where applications, data, and services are usually located away from centralized datacenter computing and storage nodes to the logical extremes of a network. Edge cloud is different from Internet cloud in two main ways: resource organization and service location are different; and data value and processing model are different. Edge computing is a great complement to Internet clouds targeting mainly mobile applications. Recent advances of microservices have several unique features such as monolithic style, being independently deployable, and possessing decentralized governance, which make it a perfect fit to support edge cloud and IoT application scenarios. Emerging microservices application scenarios such as personal health management, smart building, and autonomous vehicles are also illustrated in this chapter.
Keywords
Edge cloud; Microservices; Service-oriented Architecture (SoA); Internet cloud; Autonomous vehicle; Smart building; Personal health management
In cloud and mobile computing communities, there is no consensus on the taxonomic relations between edge and mobile clouds. If the definition of “mobiles” is extended to various embedded devices, sensors, and we relax the computing bounding between Internet clouds and mobile devices, then edge cloud can be considered as a special case of mobile cloud. The offloading targets in mobile cloud computing can be extended to mobile devices, sensors, and many nontraditional computers. As shown in Fig. 6.1, if the code transfer procedure is not required in the offloading models, then we can use composing approach to build a more flexible mobile cloud application scenario to maximally reuse the mobile edge devices' functions.
Edge cloud is pushing the frontier of computing applications, data, and services away from centralized nodes to the logical extremes of a network. It enables analytics and knowledge generation to occur at the source of the data, especially mobile devices. This approach requires leveraging resources that may not be continuously connected to a network such as laptops, smartphones, tablets, and sensors. Edge computing covers a wide range of technologies including wireless sensor networks, mobile data acquisition, mobile signature analysis, cooperative distributed peer-to-peer ad hoc networking and processing also classifiable as local cloud/fog computing and grid/mesh computing, dew computing, mobile edge computing, cloudlet, distributed data storage and retrieval, autonomic self-healing networks, remote cloud services, augmented reality, and more.
This chapter describes edge cloud concept in Section 6.1, followed by the comparisons of edge cloud vs Internet cloud in Section 6.1.1. The edge cloud examples and applications are presented in Section 6.1.2. Finally, the microservices and several case studies are presented.
6.1 Edge Cloud
Security cameras, phones, machine sensors, thermostats, cars, and televisions are just a few of the items in daily use that create data which can be mined and analyzed. Add to it the data created at retail stores, manufacturing plants, financial institutions, oil and gas drilling platforms, pipelines and processing plants, and it is not hard to understand that the deluge of streaming and IoT sensor data can — and will — very quickly overwhelm today's traditional data analytics tools.
Organizations are beginning to look at edge cloud computing (or simply edge computing) as the answer. Edge computing consists of putting micro data centers or even small, purpose-built high-performance data analytics machines in remote offices and locations in order to gain real-time insights from the data collected, or to promote data thinning at the edge, by dramatically reducing the amount of data that needs to be transmitted to a central data center. Without having to move unnecessary data to a central data center, analytics at the edge can simplify and drastically speed analysis while also cutting costs.
Edge computing pushes applications, data, and computing power (services) away from centralized points to the logical extremes of a network. Edge computing replicates fragments of information across distributed networks of web servers, which may be vast. As a topological paradigm, edge computing is also referred to as mesh computing, peer-to-peer computing, autonomic (self-healing) computing, grid computing, and using other names implying noncentralized, nodeless availability.
To ensure acceptable performance of widely dispersed distributed services, large organizations typically implement edge computing by deploying Web server farms with clustering. Previously available only to very large corporate and government organizations, technology advancement and cost reduction for large-scale implementations have made the technology available to small and medium-sized businesses. The target end-user is any Internet client making use of commercial Internet application services. Edge computing imposes certain limitations on the choices of technology platforms, applications or services, all of which need to be specifically developed or configured for edge computing. We summarize the main advantages of edge computing as follows:
1. Edge application services significantly decrease the data volume that must be moved, the consequent traffic, and the distance the data must go, thereby reducing transmission costs, shrinking latency, and improving quality of service (QoS).
2. Edge computing eliminates, or at least deemphasizes, the core computing environment, limiting or removing a major bottleneck and a potential point of failure.
3. Security is also improved as encrypted data moves further in, toward the network core. As it approaches the enterprise, the data is checked as it passes through protected firewalls and other security points, where viruses, compromised data, and active hackers can be caught early on.
4. Finally, the ability to “virtualize” (i.e., logically group CPU capabilities on an as-needed, real-time basis) extends scalability. The edge computing market is generally based on a “charge for network services” model, and it could be argued that typical customers for edge services are organizations desiring linear scale of business application performance to the growth of, e.g., a subscriber base.
Edge cloud is fundamentally different from Internet clouds. First, resource organization and location are different. Edge computing refers to data processing power at the edge of a network instead of holding that processing power in a cloud or a central data warehouse. There are several examples where it's advantageous to do so. For example, in industrial IoT applications such as power production, smart traffic lights, or manufacturing, the edge devices capture streaming data that can be used to prevent a part from failing, reroute traffic, optimize production, and prevent product defects. Edge computing relates to mobile networks application and data stream acceleration through caching and/or compressing of relevant (mainly localized) data at the edge of the mobile network, as near as possible to the end user location. In this sense edge computing is a method of accelerating and improving the performance of cloud computing for mobile users. An edge device could also be an ATM (the bank wants to stop fraudulent financial transactions); a retail store that is using a beacon to push in-store incentives to a mobile application; a smartphone; a gateway device that collects data from other endpoints before sending it to the cloud, etc. Second, data value and processing model are different. Much like the time value of money, the time value of data means that the data you have in this second will not mean as much a week, day or even hour from now. This coupled with the proliferation of IoT sensor, social, and other streaming data is driving organizations to use edge computing to provide the real-time analytics that impact the bottom line, or in some cases, stop a disaster from happening before it starts. Organizations are currently reliant on large and complex clusters for data analytics, and these clusters are rife with bottlenecks including data transport, indexing and extract, as well as transform and load processes. While centralized infrastructures work for analyses that rely on static or historical data, it is critical for many of today's organizations to have fast and actionable insight by correlating newly obtained information with legacy information in order to gain and maintain a strong competitive advantage. An increasing amount of data is priceless in the seconds after it's collected – consider the instance of a fraudster or hacker accessing accounts – but it loses all value during the time it takes to move it to the centralized data center infrastructure or upload it to the cloud. Losing value from that data due to slow decisions is not acceptable, especially when an edge-computing platform that eliminates moving data provides the near-instant intelligence needed. Organizations cannot afford to wait days, weeks or even months for insights from data. With data analytics at the edge, they do not have to. When data analysis is done at the edge of a network, that's known as “edge analytics.” For example, recently Facebook announced Caffe2 [12] to help developers and researchers train large machine learning models and deliver AI on mobile devices. This release provides access to many of the same tools, allowing you to run large-scale distributed training scenarios and build machine learning applications for mobile. Caffe2 comes with native Python and C++ APIs that work interchangeably so a developer can prototype quickly now, easily optimize later. Caffe2 also integrates with Android Studio, Microsoft Visual Studio, or Apple XCode for mobile development. Finally, edge computing does not replace cloud computing. In reality, an analytic model or rules might be created in a cloud then pushed out to edge devices. Some edge devices are also incapable of doing analysis. Edge computing is also closely related to “fog computing”, which also entails data processing from the edge to the cloud. In this section, we provide an overview of two edge-cloud computing platforms, fog computing and Nebula. Fog computing or fog networking, also known as fogging, is an architecture that uses one or more collaborative multitude of end-user clients or near-user edge devices to carry out a substantial amount of storage (rather than stored primarily in cloud data centers), communication (rather than routed over the Internet backbone), control, configuration, measurement, and management (rather than controlled primarily by network gateways such as those in the LTE core network). On November 19, 2015, Cisco Systems, ARM Holdings, Dell, Intel, Microsoft, and Princeton University founded the OpenFog Consortium [25] to promote interests and development in fog computing. Fog computing can be perceived both in large cloud systems and big data structures, making reference to the growing difficulties in accessing information objectively. This results in a lack of quality of the obtained content. The effects of fog computing on cloud computing and big data systems may vary; yet, a common aspect that can be extracted is a limitation in accurate content distribution, an issue that has been tackled with the creation of metrics that attempt to improve accuracy. Fog networking consists of a control plane and a data plane. For example, on the data plane, fog computing enables computing services to reside at the edge of the network as opposed to servers in a data-center. Compared to cloud computing, fog computing emphasizes proximity to end-users and client objectives, dense geographical distribution and local resource pooling, latency reduction for QoS and edge analytics/stream mining, resulting in superior user-experience and redundancy in case of failure. Fog networking supports the Internet of Everything (IoE) concept, in which most of the devices used by humans on a daily basis will be connected to each other. Examples include phones, wearable health monitoring devices, connected vehicle and augmented reality using devices such as the Google Glass. ISO/IEC 20248 [18] is an ISO standard describing automatic identification and data capture techniques – data structures – digital signature meta structure. It is an international standard specification provides a method whereby the data of objects identified by edge computing using Automated Identification Data Carriers (AIDC), a bar-code and/or RFID tag, can be read, interpreted, verified and made available into the “Fog” and on the “Edge” even when the AIDC tag has moved on. Nebula [234] is a dispersed cloud infrastructure that uses voluntary edge resources for both computation and data storage, which is developed by University of Minnesota. The design goals of Nebula include the following features: • Support for distributed data-intensive computing. Nebula is designed to support data-intensive applications that require efficient movement and availability of large quantities of data to compute resources. As a result, in addition to an efficient computational platform, Nebula must also support a scalable data storage platform. Further, Nebula is designed to support applications where data may originate in a geographically distributed manner, and is not necessarily preloaded to a central location. • Location-aware resource management. To enable efficient execution of distributed data-intensive applications, Nebula considers network bandwidth along with computation capabilities of resources in the volunteer platform. As a result, resource management decisions must optimize on computation time as well as data movement costs. In particular, compute resources may be selected based on their locality and proximity to their input data, while data may be staged closer to efficient computational resources. • Sandboxed execution environment. To ensure that volunteer nodes are completely safe and isolated from malicious code that might be executed as part of a Nebula-based application, volunteer nodes must be able to execute all user-injected application code within a protected sandbox. • Fault tolerance. Nebula ensures fault tolerant execution of applications in the presence of node churn and transient network failures that are typical in a volunteer environment.
Nebula consists of volunteer nodes that donate their computation and storage resources, along with a set of global and application-specific services that are hosted on dedicated, stable nodes. These resources and services together constitute four major components in Nebula: Nebula Central, DataStore, ComputePool, and Nebula Monitor. These components work with each other to enable the execution of data-intensive applications on the Nebula platform. Each volunteer node can choose to participate as a data node or a compute node depending on whether it donates its storage, compute resources, or both. The volunteer nodes are multiplexed among different applications, so each compute (or data) node can be part of multiple ComputePools (or DataStores). The centralized components only provide control and monitoring information, and all data flow and work flow happens directly between the volunteer nodes in the system. As a result, these components do not become bottlenecks in the execution path. A collection of colocated devices can be orchestrated to provide a cloud service at the edge. Scenarios with colocated devices include, but are not limited to, passengers with mobile devices using public transit services, students in classrooms and groups of people sitting in a coffee shop. The FemtoCloud system [124] which provides a dynamic, self-configuring and multidevice mobile cloud out of a cluster of mobile devices. The FemtoCloud system architecture designed to enable multiple mobile devices to be configured into a coordinated cloud computing service despite churn in mobile device participation. The FemtoCloud computing service executes a variety of tasks that arrive at the control device. The FemtoCloud client service, running on the mobile devices, estimates the computational capability of the mobile device, and uses it along with user input to determine the computational capacity available for sharing. This client leverages device sensors, user input, and utilization history, to build and maintain a user profile. Afterwards, the service shares the available information with the control device, which is then responsible for estimating the user presence time and configuring the participating mobile devices as a cloud offering compute as a service.6.1.1 Edge Cloud vs Internet Cloud
6.1.2 Edge Cloud Platforms
Fog Computing
Nebula
FemtoClouds
6.2 Microservices for Mobile Cloud Computing
6.2.1 Microservices
“Microservices” became a hot term in 2014 [179], attracting lots of attention as a new way to think about structuring applications. In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
6.2.1.1 Microservices vs. SOA
Some advocates believe that the microservice style is very similar to that of Service Oriented Architecture (SOA) [264]. However, SOA usually means too many different things, and the differences are usually due to the SOA's focus on Enterprise Service Buses (ESBs)1 used to integrate monolithic applications.
Microservice architecture is an agile software architecture. While microservices and SOA are different on many levels – architectural style, implementation examples, associated technologies – they have played strikingly similar roles in the technology landscape. Both promised to be transformational, and they succeeded in attracting adherents who attracted still more adherents. In short, while both microservices and SOA began as architectures, they ultimately became movements.
The distributed nature of cloud infrastructure challenged the placement of the centralized ESB topology. Organizationally and culturally, the agile movement was driving decentralization and team autonomy. The combination of factors among others took the SOA movement out of the mainstream. By 2009, people were not merely questioning the SOA approach but marking its death. RESTful Web APIs – a style of interconnecting applications that had evolved organically on the Web – arose as a lighter-weight alternative to SOAP services.
Matt McLarty provided a detailed discussion [198] on explaining the history of SOA, contrasting SOA and Microservices as movements rather than technology, and pointing out the lessons the microservices movement needs to learn from the fate of SOA. In short, the design of a successful service model should be rooted in the business need and not in a conceptual design.
6.2.1.2 Microservices Architectural Style
The microservices relations are highlighted in Fig. 6.2. The architectural styles of microservices are described below.
Monolithic style – a monolithic application built as a single unit. For example, enterprise applications are often built in three main parts: a client-side user interface (consisting of HTML pages and JavaScript running in a browser on the user's machine), a database (consisting of many tables inserted into a common, and usually relational, database management system), and a server-side application. The server-side application will handle HTTP requests, execute domain logic, retrieve and update data from the database, and select and populate HTML views to be sent to the browser. This server-side application is a monolith – a single logical executable. Any changes to the system involve building and deploying a new version of the server-side application.
Many of web-based applications use JSON via REST endpoints applied a monolithic architecture. The application is deployed as a single file (e.g., Java) or a collection of files rooted at the same directory (e.g., Rails [31]). All the application code runs in the same process. Scaling requires deploying multiple copies of the exact same application code to multiple servers. As shown in Fig. 6.2, the services defined in the monolithic application architecture are decomposed into individual services, and deployed separately from one another on separate hosts.
Each microservice is aligned with a specific business function, and only defines the operations necessary to that business function. This may sound exactly like SOA, and indeed, microservices architecture and SOA share some common characteristics. Both architectures organize code into services, and both define clear boundaries representing the points at which a service should be decoupled from another. However, SOA arose from the need to integrate monolithic applications that exposed an API (usually SOAP-based2) with one another. In SOA, integration relies heavily on middleware, in particular Enterprise Service Bus (ESB). Microservices architecture may often make use of a message bus, but there is no logic in the messaging layer whatsoever – it is purely used as a transport for messages from one service to another. This differs dramatically from ESB, which contains substantial logic for message routing, schema validation, message translation, and business rules. As a result, microservices architectures are substantially less cumbersome than traditional SOA, and do not require the same level of governance and canonical data modeling to define the interface between services. With microservices, development is rapid and services evolve alongside the needs of the business.
Monolithic applications can be successful, but increasingly people are feeling frustrations with them – especially as more applications are being deployed to the cloud. Change cycles are tied together – a change made to a small part of the application requires the entire monolith to be rebuilt and deployed. Over time it is often hard to keep a good modular structure, making it harder to keep changes that ought to only affect one module within that module. Scaling requires scaling of the entire application rather than parts of it that require greater resource. This leads to the second architectural style.
Independently deployable services are independently deployable and scalable, each service also provides a firm module boundary, even allowing for different services to be written in different programming languages. They can also be managed by different teams. This is a key advantage of the microservices architecture, in which a service can be individually scaled based on its resource requirements. Rather than having to run large servers with lots of CPU and RAM, microservices can be deployed on smaller hosts containing only those resources required by that service. In addition, each service can be implemented in the language most suitable for the operations that service performs. An image processing service can be implemented using a high-performance language like C++. A service performing math or statistical operations may be implemented in Python. Services performing basic CRUD operations3 for resources might be best implemented in Ruby. The microservices architecture does not require the “one size fits all” model of the monolithic architecture, which will generally use a single application framework and a single programming language.
Decentralized Governance – one of the consequences of centralized governance is the tendency to standardize on single technology platforms, and make it incompatible with other platforms' components.
The above three described architectural styles fit perfectly into mobile cloud application scenarios, where mobile devices may request services from other nodes. Since microservices inherit some features of SOA, the POEM approach presented in Chapter 4 and Appendix A can be used as a foundation to implement the microservices architectural styles, and this will lead to a future development.
6.2.1.3 Using Container for Microservices
But there are some disadvantages to microservices as well. Because services will be spread across multiple hosts, it can be difficult to keep track of which hosts are running certain services. Also, even though each host may not be as powerful as the host running the monolithic application, as the microservices architecture scales out, the number of hosts will grow faster than it will with a monolithic architecture. In the AWS environment, there may even be microservices that do not require all the resources of even the smallest EC2 instance type. This results in over-provisioning and increased costs. If services are implemented in different programming languages, this means the deployment of each service will require a completely different set of libraries and frameworks, making deployment to a server complex.
Linux containers can help mitigate many of these challenges with the microservices architecture as shown in Fig. 6.2. Linux containers make use of kernel interfaces such as cnames and namespaces, which allow multiple containers to share the same kernel while running in complete isolation from one another. The Docker execution environment uses a module called libcontainer, which standardizes these interfaces. Docker also provided a GitHub-like repository for container images called DockerHub, making it easy to share and distribute containers. It is this isolation between containers running on the same host that makes deploying microservice code developed using different languages and frameworks very easy. Using Docker, we could create a DockerFile describing all the language, framework, and library dependencies for that service.
The container execution environment isolates each container running on the host from one another, so there is no risk that the language, library, or framework dependencies used by one container will collide with that of another. The portability of containers also makes deployment of microservices easily. To push out a new version of a service running on a given host, the running container can simply be stopped and a new container started that is based on a Docker image using the latest version of the service code. All the other containers running on the host will be unaffected by this change.
Containers also help with the efficient utilization of resources on a host. If a given service is not using all the resources on an Amazon EC2 instance, additional services can be launched in containers on that instance that make use of the idle resources. Of course, deploying services in containers, managing which services are running on which hosts, and tracking the capacity utilization of all hosts that are running containers will quickly become unmanageable if done manually.
6.3 Microservices Patterns for IoT
Microservices patterns, like all patterns, are meant not to define a single solution, API, standard, or framework, but rather to identify common patterns found in systems that successfully manage important forces [179]. It is important to keep in mind, therefore, that a given IoT system might simultaneously involve multiple technical implementations of key elements like containers and event notification protocols – some of these legacies and difficult to change – but these particulars should not stop us from recognizing the pattern in use.
We wish to work towards a vision future in which we are able to efficiently develop IoT systems that are extremely interoperable, flexible, and secure. The structure of the vision consists of a high-level, abstract model of general microservices-based IoT systems, together with patterns of solutions that one might expect to see employed in the design and deployment of any given IoT system following the vision. We must note that some of the key requirements and forces within IoT systems are exhibited differently than within typical cloud-only, web-centric microservices solutions. Therefore, one should reasonably expect that the sorts of patterns that one might find in use in successful IoT systems will contain common elements to cloud systems, but nevertheless can be instantiated and combined in different ways.
Though undoubtedly many patterns will be used in IoT systems in the future, we focus our vision on the combination of six patterns that we expect to combine constructively to build solutions well. These are listed in Table 6.1 which, for each pattern, provides a name to the type of the pattern, what kinds of IoT problems it may be applied to, and some of the desirable system qualities that may be enabled more easily through using the pattern.
Table 6.1
Key patterns in the vision of microservices-focused IoT solutions
| Name | Type | Solution of | Qualities |
| Microservices | Structural | Decomposition | Team structure alignment (Conway's law); independent development and testing; enabling microservices marketplaces |
| API Gateway | Structural | Aggregation and distribution | Service API aggregation; event distribution; policy enforcement; network bridging |
| Distribution | Structural | Locality decomposition | Performance; resilience; scaling |
| Service discovery | Action | System coupling | Dynamic reconfiguration; extensibility |
| Containers | Structural | Deployment | Deployment simplicity; security; scaling |
| Access Control | Structural | Data sharing | Interoperability; scaling; flexibility; security |
6.3.1 IoT-Based Microservices
We take “microservices”-based IoT systems to be composed of one or more individual self-contained and independent deployable software components, i.e., microservices interacting with each other by messages abstracted as method calls (e.g., web service call, or message RPC), or through publishing–subscribing events.
For example, consider a hypothetical body weight management IoT system that provides a service to help a user manage his/her body weight. The system consists of 3 components: a wristband with sensors that measures how many calories the user is burning, an application on his/her smartphone that asks to input what he/she eats and calculates the calorie intake, and a cloud-based software component that offers recommendations for meals and sport activities. Following a microservices pattern, the overall functionality could be separated into a collection of independent microservices, which communicate with each other through well-defined message interface. For example, the wristband may connect to the smartphone through Bluetooth and send body activity intensity information to the app, and the application can connect to the recommendation service through platform-level inter-application messaging, or perhaps through an REST API in the case of a recommender service in the cloud. It is completely possible these three microservices are developed by different teams, or even different companies, as long as their interfaces are well maintained.
From the service consumer's point of view, a microservice is nothing but a collection of APIs. To make it easier to discuss the organization of these APIs, we propose an abstract model of the internal structure of a microservice, which can be mapped onto various protocols such as MQTT, CoAP, AMQP, or REST API. In our model, a microservice contains one or more virtual objects, which we take to be akin to objects in object oriented programming (OOP) languages. A virtual object can have two types of service APIs, methods and event channels. Methods resemble the methods in OOP languages or services in REST APIs: they provide a remote procedure call (RPC). Event channels implements a publisher–subscriber model such that anyone that subscribes to the event channel will be notified when an event is published by an event producer.
Sensors and devices with sensing capability are examples of what might typically be an event producer. The events they publish might usually be intermittent, and their traffic is normally a push style. Actuators and services in the cloud are examples of event consumers and method providers (callee). They are typically continuously online, and their traffic style is typically that of request-response. It may be that one event or method call might generate a cascade of additional events or method calls. A method that does not return anything can be used as an event consumer as long as the method and the event channel are compatible in data format as well as semantics. Virtual objects, event channels and methods can also be dynamically created by a microservice. Fig. 6.3 shows the relationship of virtual object, method, event channel, and the binding with service endpoints.
A common claim of microservice patterns is that they can ease development in several ways [179]. For example, for the developers, it is easy to create simulators and sandbox environments for a real world IoT systems since devices can be abstracted to microservices with well-specified message interface. Additionally, sensor events can be logged and replayed in the sandbox environment to help debugging, testing, and troubleshooting of the microservices under development. Since a self-contained microservice usually does not have external dependencies, the development of a large IoT system can be broken into many small, semiindependent and focused teams.
For the system administrator, each microservice can be independently deployed, and if the method and event channel interfaces are well-defined and common then the connections may be as simple as plugging in light bulbs into standard receptacles. These facts of deployment mean that new components can be incrementally introduced to the system and deployed side-by-side with existing ones. If some microservices are broken or obsolete, they can be simply unplugged and switched out. We envision that there will be a “supermarket” of microservices as well as devices for IoT systems, where the “goods” are containerized ready-to-deploy microservices, such that a system administrator merely needs to “shop” online, like shopping the App Store for a smartphone. For the operator, it is possible to create an IoT operations center that collects the status of all microservices and presents the information on a dashboard. The IoT operations center functionality itself can be implemented as a microservice with required privileges so that it can access the entire system service registry, call any methods, and listen to any events. In this way, the operator is similar to a commander in the NASA mission control center, sitting in her office and manipulating smart devices and microservices as required.
API Gateway
An API gateway is an interface that relays calls or requests between two or more microservices. API gateways can aggregate the APIs for multiple microservices into one client interface, and can also distribute or route the calls or requests from one entry point to multiple target microservices. Rather than allowing direct communication among microservices, therefore, an API gateway can be placed in-between them, which can have the effect of adjusting interconnectivity through the aggregation and distribution function of the gateway. Microservice connection is then through registration; when a microservice registers itself to an API gateway. It creates one or more endpoints with unique identity, which can be further bound to an event channel or method through static configuration or dynamic negotiation with the API gateway. Besides the microservices, any user or machine client can also be required to present its identity as a name of identifier to interact with the API gateway to access its associated microservices. The identity management and authentication service might leverage existing systems that managing accounts, such as single-sign-on (SSO) or OAuth [24].
From the API gateway's point of view, each endpoint is uniquely identifiable, where the identity may be represented as a unique name or just an identifier. Such endpoint name or identifier only exist in the model, and the API gateway implementation needs to map it into actual protocol, such as an URL in REST API or CoAP, a queue or exchange in AMQP, or a topic in MQTT. In a cloud-supported body weight management IoT system as described in the example above, the weight management plan virtual object can be mapped to REST resource URL on the cloud server, which runs the weight management and recommendation microservices, and the method as well as parameter format on this virtual object can be mapped to the HTTP methods. In typical uses of this approach, the method name is encoded in the JSON payload to augment the limited types of HTTP methods. Such mappings impose additional constraints on our model, such as requiring that consecutive method calls in a session must be “nonsticky.”
For addressing across heterogeneous IoT devices and networks, virtual objects, methods, and event channels have site-scope unique hierarchical names like URNs. In some cases it may be feasible to create unique IDs using URN style approaches that encodes the device unique identifier (e.g., snac.asu.edu/device/edb15825-92e2-e7b1-7c97-ddd102258a70 which uses the Bluetooth MAC address for uniqueness). Taking our body weight management IoT system as an example, imagine the API gateway is in the cloud with domain name snac.asu.edu; then there can be a virtual object snac.asu.edu/WeightManagementPlan with createPlan method. After a plan with user profile and weight management goal is created, a virtual object snac.asu.edu/WeightManagementPlan/plan1 is created with checkGoal and getRecommendation methods.
Our model is compatible with many message queue protocols as well as REST or CoAP API used in real world IoT systems. For example, the role of our API gateway is similar as a load balancing proxy for web services, or a message broker of MQTT protocol. The virtual object can be mapped to the resource using an RESTful URL, and the method can be mapped to the combination of HTTP methods on that URL and JSON payload. This allows us to apply a microservices pattern on a broader scope than just web based applications. However, some protocols may not be able to provide both method type and event channel type of service APIs. For example, MQTT can only provide publish–subscribe interface and, while CoAP can provide service call natively, publish–subscribe through CoAP will probably be difficult and unnatural to implement. In such scenarios, it is important for the API gateway to support multiple protocols to connect those “IoT islands” to the rest of the system.
In many cases microservices on the devices do not possess publicly reachable IP address, a trait which prevents them being accessed over the Internet outside the domain intranet. Multiprotocol API gateways can bridge the gap. In our model, we call the subsystem made of an API gateway and its associated microservices an IoT site (like website in traditional Internet). The API gateway works as a hub that translating messages from one protocol to another, making microservices in different domain accessible to each other. Meanwhile, the gateway also accepts API calls from user or machine clients outside the site and translates them to corresponding messages to intranet API calls. Sometimes an API gateway is able to deliver messages or initiate service call to another API gateway to fulfill complex services spanning over multiple sites.
The existence of API gateway as well as the binding of endpoint to method or event channel makes a natural place to enforce policies. We borrowed the approach of policy enforcement from SDN and, in our model, there are three types of policies: rules to allow or deny the creation or destruction of a certain named service APIs, rules to allow or deny a binding of service API to an endpoint, and constraints on the binding. The first two resemble file system permissions in operating system, and the last one resembles flow policies in SDN, in which the flow is a binding. We explain three examples of policy based on the usage, which are name resolution policy and QoS policy (in this subsection), and access control policy (in a later subsection).
A name resolution policy is a static binding between a service API and an endpoint. This type of policy is useful for integrating microservices of different versions, or for load balancing. Consider again our body weight management IoT system: the developers may construct a new version of the recommendation microservice and deploy it to the cloud. In response, the system administrator needs only change a few policies to bind the methods on the API gateway to the endpoints of new version of the microservice. The replacement is easy if the new version is backward-compatible in syntax and semantics. If it is incompatible, the system administrator can instead create new virtual objects and methods, bind them to the microservice of the new version, and run both versions in parallel until all clients that use the old microservice shift to the new version. For load balancing, the administrator has the option of running multiple instances of the same microservice and to bind them to the same set of virtual objects and methods. Similarly, balance can also be facilitated by binding multiple event consumers to the same event channel so that events are dispatched among them instead of duplicated to all of them. Name resolution policies and API gateways can combine to decouple the event producer from the consumer (or caller from callee). In an IoT system, microservices might not have fixed location on the network. For example, services on mobile IoT devices might need to attach to different API gateways at different times. Name resolution policies might be used also to redirect requests to other sites if they can work together with service registry mechanism.
A QoS policy is primarily concerned with traffic restriction on a binding (if we regard a binding as a traffic flow). Restrictions could include limiting the number of messages per second or traffic amount in bytes per second. Such policies can help protect the microservice from DoS attacks, or prevent launching DoS from some device or external client. Additionally it provides more flexibility needed to fulfill service level agreements and accounting processes. As with access control policies, QoS policies can also be dynamically generated by a policy controller. Another usage of a QoS policy is to implement so-called “circuit breaker” [207] in which, if the running status of the microservice shows it is exhausting its capability, a QoS policy can be dynamically generated to restrict further requests to prevent cascading failures.
Distribution
Traditionally, microservices are all located in the cloud. In IoT systems, we expect that microservices can additionally be associated with a device, edge cloud or gateway component, or deployed in the larger Internet cloud. Because of the distribution, API gateways also may be distributed on the edge, including within a personal area network or on a smartphone. These gateways may have various forms, which can be an independent physical machine, or a software component deployed on a router or server.
One of the advantages of distributing microservices and gateways among the cloud is better performance, due to the locality and the data stream management. In many IoT scenarios, the cost of carrying all the device generated data to the centralized remote cloud for processing and decision making outweigh the benefit. For example, some IoT applications may be latency sensitive, or the sheer amount of data generated by the device exceeds available network bandwidth, which requires us to run microservices close to the devices. An API gateway within a smartphone can aggregate data from a connected wearable that provides event data at high data rates, but it might relay only summarized, aggregated, or otherwise “rolled-up” data back to a cloud microservice. Edge cloud and fog computing extends the traditional cloud computing paradigm to the rim of IoT, which enables new types of applications [67,78]. In our architecture, edge cloud or fog is a virtualized platform serving as an on-demand execution environment of microservices close to the devices or the data source, which is different from running the microservices on the device itself. Providing an edge cloud or fog platform can also provide better architectural flexibility. For example, microservices can be off-loaded or migrated to the edge cloud or fog from devices to meet changing requirements of service performance.
Service Description and Discovery
In an IoT system, integrating microservices requires description of the service APIs in both syntactic and semantic level. Such description will also serve as an important way to search existing services and compose new services by the machine. We propose using ontology for service description and discovery. Our ontology is made of three parts: the IoT system domain ontology, the service profile, and ontology of microservices in the IoT system, shown in Fig. 6.4. The first part is the domain ontology, which contains the domain knowledge of entities and relations. For example, in a smart building IoT system domain ontology may describe the structure of the building, real world entities such gate, room, and the IoT devices attached to the entity. This part of ontology should follow IoT-Lite ontology [17], or SSN ontology [33], or other domain standards. The second part is service profile, which contains the description of service interface of a domain, including the category, process, parameter, result, condition of a method or an event channel. Additionally, service level agreement (SLA) should also be specified as part of the profile. This part of ontology should partially follow WSMO [37], or OWL-S [32], and the service grounding should support commonly used protocols in IoT system other than only web service. The third part is the ontology describing the microservice structure in the IoT system, including the virtual objects, methods, event channels, and their enclosing microservice. This part may also include deployment related entities, such where is the microservice deployed, on a device or in the cloud, in the fog/edge cloud or remote cloud. The three parts of ontology can be managed by an individual ontology database microservice associated with the API gateway. When new services are registered, corresponding ontology segments are added to this ontology database. Other microservice may query the ontology or run reasoning procedure through this ontology management microservice.
Besides ontology, each microservice should provide a human readable service manifest document for developers. We recommend that each microservice should provide a virtual object of the microservice's name, and the virtual object should have a method called “introspect”, which returns a manifest detailing all the virtual objects and service interfaces (with the mapping to various protocols) of the microservice. Such manifest may just contain a plaintext document for human developers, or an annotated XML/HTML to be rendered on browser, or even an annotated document in OWL/RDF to be readable for both human developers and machine programs. In addition, since microservices can be dynamically added to an IoT system, ontology is subject to change. It is helpful to provide a global service registry microservice (possibly colocated with the API gateway) which manages the ontology and manifest to allow service registration and discovery among multiple sites. Each individual microservice can talk to the registry in order to publish its structures and interfaces so that other microservices can make SPARQL query to locate it.
Containers
Containers are used frequently in cloud computing for ease of deploy and to provide secure containment between different tenants. We treat containers generally here as an infrastructure that provides a local context for execution that can be instantiated multiply and separately, and hold something that can be recognized as a microservice. By this we mean to include not only OS-level container solutions like Docker [201], but also typical application-level or platform-level “sandbox” containers like Chromium containers [229] or Android for Work [121].
Containers in IoT systems can serve functions similar in cloud computing in terms of ease of deployment, replication, and scaling. They permit convenient and efficient deployment of a microservice by packing the microservice along with its dependencies into a single image deployed to the container. From the system administrator's point of view, the deployed container with the microservice is a black box with a few interfaces that can be easily “plugged into” the API gateway. The containerized microservice maintains its own states and stores its data, so that some of the complexity is encapsulated inside the black box. In addition, virtualized execution environments provide isolation and control of the microservices, such as by forcing all traffic to follow the policies enforced on the API gateway. This can ease troubleshooting, online A/B tests, and roll back, and it can simplify overall management. However, when the containers are used on smartphones or other smart devices, the security and isolation of each microservice is the more compelling need.
The use of containers for security, as a pattern, can succeed because of the way that essential security functions of isolation and encryption can be performed comprehensively according to enforced policy rather than requiring the microservice developers to implement them – and relying on them to unfailingly and correctly implement them too. For example, some containers can force all traffic out of the container to be passed through a VPN all data stored locally be encrypted with a container-specific key stored securely in trusted memory. As such, they can establish a level of security for the data-in-transit and data-at-rest. Wrapping microservices and API gateways in containers with known, tested levels of security can greatly simplify the problem of certifying or verifying some facts of the system's overall security. As the security of containers improve, the security of the IoT solutions can improve without changing the microservices, and the container solutions are patched to close holes from newly-discovered vulnerabilities, every part of the IoT system can rise with the tide without any changes to the code whatsoever. While this containment may not help stop device manufacturers from shipping devices with other weaknesses (e.g., default passwords in the manual), using containers as a pattern for pervasive security meshes well with a comprehensive microservices approach to service decomposition.
6.3.2 Case Study: Personal Health Management
We show an example of the aforementioned weight management IoT system with microservice architecture using simulated wearable sensors and cloud resources. In this system, there are 5 microservices (Fig. 6.5). First, an activity intensity tracking microservice on a wristband reports calorie consumption of the user (simulated using recorded heart rate data and an algorithm calculating metabolism rate and further calorie consumption based on heart rate). Second, a smartphone application allows the user to input what he/she eats to calculate calorie intake (simulated using a dialog panel on a PC). Third, a program running in the cloud obtains online hourly weather forecast and translates it to our defined record format. Fourth, a program running in the cloud retrieves the user's Google calendar and translate his/her daily schedule to our defined record format.
All these 4 microservices are considered as sensors bonded to their own event channels, and they connect to the API gateway (in the cloud) through a custom message protocol. Microservice connection is then through registration; when a microservice registers itself to an API gateway, it creates one or more endpoints with unique identity, which can be further bound to an event channel or method through static configuration or dynamic negotiation with the API gateway. Besides the microservices, any user or machine client can also be required to present its identity as a name of identifier to interact with the API gateway to access its associated microservices. The identity management and authentication service might leverage existing systems that managing accounts, such as Single-Sign-On (SSO)4 or OAuth [24].
The last part is a microservice that makes exercise recommendations according to the user's weight management goal, and schedules a time slot for each of the recommended exercise instances in some type of sport with a certain duration. Then the user can be notified and decide which is preferred and add it to his/her calendar. The exercise recommendation microservice will consider both weather and user schedule constraints.
For example, if it is raining, we should not recommend outdoor sports; it will not recommend any activity after a meal or before bedtime. The weight management goal is set by the user, in a format of expected body weight loss or gain within a certain period of time, e.g., losing 10 kg in 6 months. Together with a user profile and preference of the intensity of sports, the recommendation microservice decides which sport type and for how long the exercise activity should be recommended. The wearable sensors and smartphone App track daily calorie input and consumption so as to tell the user whether the weight management plan is effectively executed. Optionally, a smart body weight scale can be included in this system to further improve the accuracy of the weight management process.
The API gateway delivers sensor events to the exercise recommendation microservice through HTTP, and they are stored in a Resource Description Framework (RDF)5 database (Apache Jena Fuseki [10]) managed by the microservice itself. We choose to use RDF database instead of SQL database because the core part of the recommendation microservice is written in a reasoning language (Potassco [28]), so it is easy to translate RDF triples to predicates of parameters. A sensor data record can also be easily converted to RDF triples by creating a record object and attaching each field with a data relation (through Apache Jena RDF API [10]), e.g., hasType(record1, BodyWeight) and hasValue(record1, 85), which means a body weight measurement record of 85 kg. The exercise recommendation microservice exposes a few REST API to allow for user input and interaction with the other parts of the system.
The loosely coupled microservice architecture shows its benefits in our prototype system, especially, enhanced interoperability among different software components implemented with different technology. If we want add a smart body scale (e.g., Withings Body [41]) to track weight besides the wearable sensors and smartphone Apps that only track calories, we just need to write an adapter program to retrieve weight measurement through their API (e.g., Withings API [40]) and feed to our system.
6.3.3 Case Study: Smart Building
In this section, we show an example of a smart building system with microservice architecture. A smart building may be equipped with many networked devices as well as software services which are gradually installed with a period of many years or even decades. Microservice architecture provides a graceful solution to cope with the diversity and evolution of the system, since each individual microservice can be independently developed and deployed.
The presented example consists of 3 microservices that monitor the status of a few public desktop computers, printers, smart light bulbs, and a server room with physical server machines that support many virtual machines. An imagined view of the IoT operation center of the smart building system will be like that in Fig. 6.6. Each physical server machine in the server room is equipped with a software sensor that can monitor a CPU temperature and chassis fan speed in real-time (using Linux lm-sensors [22]). Sensor data is streamed through Advanced Message Queue Protocol (AMQP, we use RabbitMQ implementation [30]). The whole server status monitoring software is a self-contained microservice. We developed a microservice to detect an anomalous server status, using a machine learning algorithm (we trained a decision tree using WEKA library [38] with collected data from normal and abnormal server running conditions, as well as simulated data when the server machines are injected with an error, like CPU fan power loss).
The anomaly detection microservice listens to server status sensor, and sends out server status data. Then, a decision making microservice listens to such an event, and if an unhealthy physical server is detected, it quickly migrates all virtual machines on that server to other physical servers in a way that the load is evenly distributed. In real applications, the decision making microservice may have many available actions, and different actions can be taken for a different server status event. All these microservices are independently deployed (anomaly detection and decision making microservices are deployed in their own virtual machines), and they are connected to the API gateway (in the example, it is the RabbitMQ broker) through a messaging protocol. If new machines are added, we just need to deploy status monitoring microservices on them. If a new version of the decision making microservice is available, we can simply deploy it in a newly created virtual machine and turn off the old version, as long as they follow the same sensor data format. This example demonstrates the flexibility of the microservice architecture in an IoT system.
6.3.4 Case Study: Autonomous Vehicle
We built a prototype demonstration system to explore the use of microservice architectures in the context of exploring so-called “platooning” algorithms for autonomous vehicles. Longitudinal platooning means that several vehicles form a physically-local chain that maintains close proximity while traveling down a road [241]. The system employed six VC-trucks within our VC-mobile test bed [190]. Each mobile robot vehicle is an independent IoT subsystem both offering event services to other vehicles and connecting to other services. The event services include both data and control APIs. For example, the vehicles have an event stream reportState() to provide vehicle state, and may provide join() and leave() so that a lead vehicle can check safety constraints to decide whether to approve vehicles to join.
Platooning requires continuous intercommunication of vehicle motion and location data, including speed and acceleration, and frequent adjustment of motion as needed to maintain a tight platoon. Motion data comes from the cruise control, and location data are collected from ultrasonic range finders. The data are exchanged through vehicle-to-vehicle (V2V) communication over WiFi. Our platooning algorithm uses a platoon leader vehicle, which broadcasts its motion data to the entire cluster, while each different leader vehicle in the chain interchanges motion information with its neighbors behind. Platooning calculation uses a PID controller to calculate desired speeds based on predecessor states, as well as the separation distance to the predecessor, and also uses the leader's states to make adjustments in order to maintain string stability of the whole platoon, and thereby prevent speed oscillation. The platooning gathers sensor readings and also adjusts target driving speed 10 times per second. At a lower level, the vehicle motion controller drives the cruise control to adjust or maintain the vehicle's speed. In our demo, we operate 6 VC-trucks to form a platoon. Starting from zero speed, when the leader starts to move, the platoon successfully achieves a stable state of constant velocity (0.2 m/s) and a preset separation distance (0.7 m), as illustrated in Fig. 6.7.
Microservices architecture. We built the system according to a distributed microservices architecture, as illustrated in Fig. 6.7. The services offered by the vehicles are decomposed into microservices that collect coherent but loosely coupled parts of the service in the form of separate platoon and status reporting microservices. In real-world contexts, future automated vehicles might have a variety of other microservices offered by third parties in addition to these platooning services offered by the manufacturer, and each can be deployed in a different microservice.
API gateways. Each vehicle has its own API gateway that aggregates the services making it simpler to deploy the microservices differently or manage independent protocol evolution for the microservices. In addition, the API gateway can provide a flexible and fine-grained location for configuring and enforcing access control policies. Since the platooning microservice can control critical safety features like vehicle speed, and since the appropriateness of access may depend upon context not “owned” by the platooning microservice itself, the gateway can be the most suitable location to implement the enforcement.
Containers. Robot vehicles are outfitted with computers running a Hypervisor (KVM [164]), which lets us deploy microservices flexibly. It also provides a security boundary for isolation of the microservices, which is important perhaps particularly for the speed-controlling platooning microservice.
From this example, we showed that microservice architecture is also suitable for mobile IoT system. In this example, each vehicle is an independent IoT subsystem with its own API gateway and microservices, and other vehicles can connect to this vehicle and call its service. For example, vehicle A may provide an event channel called “vehicleA.asu.edu/sensors/LIDAR/scan” to expose its LIDAR point cloud frame so that other vehicles can share its vision. In this scenario, access control on the service APIs are crucial to the safety of the vehicle. For example, trusted applications like the platooning microservice is allowed to control the motion of the vehicle, while other applications or external requests are denied. This can be achieved through the access control policy enforced by the API gateway on each vehicle, and such policy can be generated when the platooning microservice is deployed.