2.1 Introduction

Industrial applications have been going through significant changes in recent years. Indeed, the trend toward globalization has changed the game significantly. While it offers huge opportunities, it also comes with severe challenges (cf. Mařík and McFarlane, 2005; Mařík et al., 2007). Global business means global competition, which requires shorter product life cycles. Particularly in consumer-oriented businesses, this leads to a trend toward highly customized and individualized products, especially when produced in high-wage countries. For companies, this may mean they need to join forces in order to develop and market trendy or niche products. As a consequence, the trend toward virtual enterprises (see, e.g., Camarinha and Afsarmanesh (1999) for a nice overview) and short-term collaborations will continue to grow. This imposes a number of profound and far-reaching demands on modern industrial manufacturing systems. The former principle goal of the manufacturing industry, namely optimization of the scheduling algorithm, has to take a back seat for the time being and has been replaced by several new goals such as adaptability, agility, responsiveness, robustness, flexibility/reconfigurability, dynamic optimization, openness to new innovations, and in some environments, to continuously varying collaborations. Such goals can only be achieved if massively software-controlled integrated industrial manufacturing tools, machines, and environments become the default (cf., e.g., Pěchouček and Marík, 2008; Mendes et al., 2009). In many cases, typical standalone, compartmentalized operations need to move toward decentralized, distributed, and networked manufacturing system architectures with intensive communication and collaboration, especially over long distances. For such complex systems to work efficiently, a high level of understanding is necessary, which translates into a reasonable understanding of domain-specific semantics (cf., e.g., Vittikh et al., 2013; Leitão et al., 2013a,b).

This chapter only concentrates on industrial application systems that rely on decentralized decision making and control. This is what is usually meant from here on when the term industrial application is used. Against this background, multi-agent-based industrial application systems seem to be a promising and natural realization choice. Multi-agent systems (MASs) provide, among other things, decentralized architecture and decision making, modularity, robustness, flexibility, and adaptability to changes (cf., e.g., Zimmermann and Mönch, 2007; Leitão, 2009; Bratukhin et al., 2011; Sayda, 2011; Vrba et al., 2011). This chapter is meant to provide an introduction to industrial agent technology for industrial application systems that rely on decentralized decision making and control. Thus, the remainder of this chapter is organized as follows. Section 2.2 concentrates on industrial applications, its evolution, the consequences of this evolution on modern industrial application systems, and the specific aspects and requirements on (multi-)agent-based industrial application systems. It also especially discusses the holonic paradigm. Finally, Section 2.8 summarizes the conclusions from this chapter.

2.2 Modern Industrial Manufacturing Systems and Their Requirements

The last decade has seen a massive trend toward the computerization of nearly everything we have to deal with in our life. This, especially, also applies to machines and tools in industry. Computerization here means that hardware is equipped with some kind of software-controlled intelligence. It permits getting feedback and improving the flexibility, adaptability, and robustness of the hardware and the system as a whole. Additionally, more automatic machine-related communication can take place. Recent technology inventions such as RFID, smart cards, embedded systems, Wi-Fi, and Bluetooth communication have accelerated this process because they extend the communication possibilities significantly and may even allow products to become active decisional entities that react in real time to the actual state of the production system (cf. Zbib et al., 2012). In parallel to the digitalization of our world, a globalization of production, as well as competition, has taken place. With it comes the demand for shorter life cycles and individualized products. For industry, this implies a shift from static optimization for long production cycles to dynamic optimization for short product cycles (cf. Trentesaux, 2009). However, dynamic optimization has a long way to go before it will be as efficient and effective as static optimization. These market-driven requirements are often referred to as agility requirements. Gunasekaran (1999) defines agility “as the capability of surviving and prospering in a competitive environment of continuous and unpredictable change by reacting quickly and effectively to changing markets, driven by customer-designed products and services.” Trentesaux (2009) differentiates between business and technical agility. The first means aligning the production process toward continuously evolving economic as well as financial objectives and concentrates as such on the extra-production issues. In contrast, technical agility concentrates on intra-production issues (i.e., on its efficiency and effectiveness; cf. Bousbia et al., 2005). Efficiency stands for maximal exploitation of resources and hardware. Effectiveness translates to the capability of achieving the expected goal as well as possible, especially in situations of disturbances (e.g., order changes, machine breakdowns, production problems). It is widely studied in the industrial as well as the scientific communities (cf., e.g., Leitão and Restivo, 2008). Efficiency and effectiveness are related, but there are also differences. A system will be working with high efficiency if it exploits a given input of resources and machine availability and capability as much as possible regardless of whether the output is completely needed at the time. From the effectiveness point of view, the goal is achieved if production stops as soon as the target is reached, instead of producing any (unwanted) surplus.

A very good example of these partially drastic changes in industry is the smart grid, also called the future energy grid (FEG) (cf. Trentesaux, 2009; Ramchurn et al., 2012). In a FEG, communication needs to switch from an order-based, one-way hierarchical flow to cooperative decision making based on two-way communication. Households that own renewable energy generators will become so-called prosumers because they produce as well as consume electricity. The term prosumer was coined around 1970 by Toffler in his book Future Shock. It describes actors in the marketplace who not only consume but also actively participate in the production of customized goods. In a FEG, prosumers may feed into the electricity grid or take from it, depending on their current requirements. While this leads to a two-way flow of electricity (from and into the grid), it also requires an intense two-way flow of information between power-generation facilities and the appliances/consumers, and all relevant units in between, in order to coordinate and manage this huge conglomerate of producers, consumers, prosumers, and service providers. The general assumption is that future consumers/prosumers will act intelligently and interactively and will rely in their behavior upon smart meters, smart devices, and smart business models. Several conclusions can be drawn from this scenario:

1. Collaborative style of work/decentralized decision making: The fact that the communication is two-way first of all means that more intelligent decisions can be made. However, in order to not overload the system, decisions need to be made as close as possible to the source. This implies that the former mainly centralized system needs to evolve toward a distributed system with mainly decentralized decision making, at least as long as local decisions with limited global influence must be dealt with.

2. Autonomy and decentralized control: Decentralization implies a move toward more autonomy for all entities within a complex decentralized system. On the other hand, complete autonomy seems to be too much to control such complex systems and environments, even if they are not nearly as complex as the FEG. Thus, a smart mix between the acceptance and application of overall general rules, on the one hand, and far-reaching local autonomy on all levels of a hierarchy, on the other hand, is inevitable.

3. Real-time requirements: In traditional systems with centralized control, decision making is mostly “hard-wired.” This means, on the positive side, that the system can react extremely fast, and on the negative side, that flexibility cannot be supported. In an open, collaborative environment, flexibility can be provided, but it comes at the price of additional overhead. Intense communication and cooperation is necessary and some intelligence is to be applied. This takes time. Thus, special care has to be taken if real-time requirements come into the picture. Guaranteeing them in a distributed environment is much more subtle, complex, and demanding.

4. Traceability/confidence degree: In a centralized approach, decisions can, comparatively, be easily traced and understood because there is only one single decision point. In a distributed, collaborative environment with decentralized decision making, the traceability issue is more difficult, not to say nearly impossible to achieve. The huge amount of communication and decentralized decision making needs to be understood and arranged into a global fully integrated picture in order to offer traceability. Closely related to traceability is confidence and trust. If expensive machines and goods are to be dealt with, it is of paramount importance that the decisions felt are logical, comprehensible, and as close to optimal as possible. Another eminently important factor is determinism. It can be comparatively easily achieved in a centralized system, but is nearly impossible to achieve in a decentralized system with autonomous components that have to negotiate with each other in order to come to a decision.

5. Self-/autonomic features: The complexity and decentralization of control requires the realization of properties such as self-healing, self-optimization, self-monitoring, and self-diagnostics.

Table 2.1 summarizes important requirements of modern decentralized industrial application systems.

Because this chapter is about industrial agents and MASs for industrial applications, it will discuss in some detail in what way agent-based systems can satisfy the requirements listed in Table 2.1. In this sense, an industrial MAS is an application system that fulfills those requirements from Table 2.1 that are relevant for the underlying industrial application. In consideration of the requirements of industrial application systems, an industrial agent is defined as follows:

The next section discusses in what way industrial applications have evolved in reaction to (some of) the requirements listed in Table 2.1.

2.3 Architectural Types of Industrial Manufacturing Systems

In general, complex manufacturing systems need to be controlled by operators, as well as automatically by the systems themselves. For this, the system usually relies on sensors to understand the relevant part of its environment, and depend on actuators to act proactively to anticipated changes, or just react to non-anticipated changes, in its environment whenever the information provided by the sensors requires it. A closed loop describes a system that consists of two parts. The first part is the system as such, and the second is the system unit that controls the system. Following Trentesaux (2009), every entity that is capable of making decisions will be called a decisional entity from now on. Three principle types of control architectures can be identified: centralized, (modified) hierarchical, and (semi-)heterarchical—the latter two have two instances each. The first two control architectures only permit vertical decision movements and communication (i.e., decisions can only flow strictly downward). However, information (requests), respectively feedback, may move upward as well. This means that if a leaf node detects a problem, it needs to send the relevant information upward till it reaches the first decisional entity on its path. If it is able to make a decision, it will do so and send it back downward. Otherwise, the problem will continue to move upward.

A centralized architecture consists of one decisional entity, the root of the system, and an arbitrary number of pure decision executors (see Figure 2.1). The root controls all planning and processing issues (cf. Leitão, 2009). All other entities are passive nodes that neither decide nor act on their own. This approach works well in smaller systems with short and efficient paths down to the basic leaf components. Due to the centralization, optimization efforts can be highly effective and efficient as long as the underlying tasks remain relatively static. With the growing complexity and size of the system, its reaction time may go down drastically. Moreover, the root is critical for the functioning of the system. If it fails, the complete system will be out of order.

With the introduction of the computer-integrated manufacturing (CIM) paradigm in the early 1970s, the first significant move toward distributed decision making was made. Originally, decision making was organized in a fully hierarchical manner, which means that centralized decision making was replaced by a hierarchically organized decision tree, allowing the distribution of decision making along these hierarchical levels (see Figure 2.2). Higher levels concentrate on strategically oriented decisions, while lower levels are restricted to comparatively simple local decisions. Such an architecture delivers (close to) optimal solutions, better robustness, predictability, and efficiency in a comparatively static environment with small product diversity, rare production changes, and very few system component failures (cf. Mařík and Lazansky, 2007; Borangiu, 2009; Trentesaux, 2009; Sallez et al., 2010). While the hierarchical structure indicates a hierarchy of control, it does not necessarily imply a distributed system in the sense of a distribution of resources, tools, or machines. However, usually the underlying system architecture is a distributed one as well. The modified hierarchical architecture adds to the purely hierarchical, respectively vertical one, offering the chance for horizontal communication (not decision making) between decisional entities on the same level (see Figure 2.2). This permits faster reactions to disturbances because a direct communication is possible, in contrast to the indirect first-up-and-then-down hierarchy communication.

The main difference between a hierarchical and a heterarchical decision and control structure¹ is that the latter relies on collaboration instead of strict decision dependence between hierarchically organized decisional entities. This is achieved by allowing decisional entities to communicate and cooperate on decision making in an arbitrary yet still predefined way (see Figure 2.3). In general, the vertical axes may no longer reflect a strict hierarchical decision path but may define a vertical cooperation between decisional entities with equal authority. According to Trentesaux (2009), a heterarchy can be described using a directed graph comprised of decisional entities made up of nodes and master-slave relationships formed between them as arcs. Thus, a hierarchy can be seen as one extreme of a heterarchy. The other extreme is a so-called full heterarchy, where each node plays the role of both master and slave. The hierarchy disappears completely, and all decisional entities are on the same level. This is equivalent to the default architecture of a MAS. Thus, it does not come as a surprise that MAS became popular in some industrial areas. However, while such an environment is highly flexible and adaptable, Trentesaux (2009) states that “long-term optimization is hard to obtain and to verify due to the difficulty of proving that a sufficient level of performance can be attained, while short-term optimization is easy to achieve.” This realization led to the invention of semi-heterarchical architectures. Coming back to the smart grid, it is obvious that it needs to have a decentralized control architecture in which the lowest levels (e.g., appliances in households) only reveal limited intelligence, while the higher the level, the more strategically and abstract the decision needs to be. If this is translated into the terminology of agents, it can be said that the lower levels are realized by simple, reactive agents, while higher levels tend to become more and more deliberative. This also captures the idea behind semi-heterarchical systems. While higher levels deal more with medium- to long-term objectives, which may evolve but do not change drastically from one second to the next, the lowest levels have to react extremely fast to disturbances and unforeseen relevant changes in their environment. Fast reaction translates to no or little communication (overhead) and predefined reaction strategies such as plans (i.e., follows the reactive paradigm). On those lower levels, a more hierarchically organized control structure, maybe with horizontal communication so as to come to quick decisions, seems to be appropriate, while higher levels lean more toward a heterarchical control structure. Due to the integration of both concepts in one system, it is called semi-heterarchical. For the time being, this architecture seems to be the best choice for many manufacturing systems because it provides robustness and maybe even real-time behavior on lower levels, while the higher levels provide intelligent decision making. According to Bousbia and Trentesaux (2002), semi-heterarchical control architectures mainly rely on three kinds of modeling approaches: the bionic and bio-inspired one, the MASs one, and the holonic one. The first option mainly relies on swarm intelligence approaches as the underlying concept, often implemented by using a MAS with simple agents. The holonic approach is discussed next.

2.4 The Holonic Paradigm and MAS-Based Holonic Systems

A holonic manufacturing system (HMS) provides a highly flexible and robust manufacturing environment. Originally introduced by Koestler (1967) in a different context, holonic systems combine bio-inspired approaches with conclusions drawn from social organizations in order to solve complex problems through the combined effort of partners, called holons. Holons, as an underlying view specific atomic concept, are organized in a hierarchy, called a holarchy. View specific atomic concept means it is observed from the current level of abstraction: On a given level, a holon is seen as being an atomic unit; however, it can be decomposed into a set of subordinate holons when analyzed on the next lower level of abstraction. In order to distinguish between those two levels when necessary, the atomic unit will be written in italics, while its subholons will be named holons^S. Holons are self-contained, self-regulating, and semi-autonomous entities. They appear to be autonomous wholes for the lower level, while at the same time are a (conditionally) dependent part of the upper (control) level. Usually, emerging problems are solved internally within the concerned holon and its subordinate holons as long as this is possible, especially if it does not violate the overall rules and norms imposed on the holon by higher-level holons in the holarchy. If tasks within a holarchy are to be done regularly, the holarchy may exist for a longer period. However, it nevertheless may constantly adapt and optimize its structure. In the case of short-term, respectively individual, tasks, the holarchy may be formed in an ad-hoc way, and may only exist temporarily. On the one hand, this ensures that a holarchy exhibits robust and stable behavior that allows it to automatically and efficiently deal with many kinds of disturbances and unforeseen events (cf. Leitão et al., 2013a, b). On the other hand, it allows it to automatically adapt itself to changing environments and requirements from the outside.

Holons, in order to cooperate, need a communication and coordination strategy. Shen and Norrie (1998) identified three possible approaches: facilitator, broker, and mediator. All three kinds rely on facilitation, which reduces overhead, ensures stability, and provides scalability. This chapter will only concentrate on the most common one, the mediator approach (see Figure 2.4). The mediator holon is a specific and unique holon^S. On the one hand, it represents the common interface to the outside (outside duties), while on the other hand it coordinates and manages the inside (inside duties) for the holon it represents. In every non-leaf holon, exactly one holon^S acts as a mediator, representing the unique interface between the holons^S inside and those outside of the holon. The inbound duties of the mediator can range from pure administrative tasks to the authority to issue directives to other holons^S. It may also broker and/or supervise the interactions between holons^S. The outbound duties comprise the necessary interaction tasks with the outside based on the common plans and goals of the holon. In a way, a holarchy is a combination of a hierarchy and a heterarchy. Within a holon, all holons^S can be organized as a heterarchy, while the overall holarchy is organized hierarchically.

For the implementation of holonic systems, a MAS seems to be a perfect candidate because it shares many principle concepts with the holonic paradigm. In a multi-agent-based holonic factory (MAS-HF), respectively a manufacturing system (MAS-HMS), holons are realized by autonomous agents with holonic properties that group together to recursively form higher-level holons until, finally, the factory is represented by a holarchy (cf. Fischer, 1999). In principle, each holon represents a logical unit of the factory, while an agent represents its actual implementation. However, from now on both terms will be used interchangeably because the functionality they represent is the same. A logical unit of a MAS-HF can represent a physical device on all levels of abstraction, such as a warehouse, shelve units, shelves, shelf boards, machines, robots, tools, conveyor belts, raw materials, products, and so on, as well as non-physical entities such as customer orders, production plans, and production schedules. Such logical, respectively physical entities usually already exist, which means that the agent wraps it in the sense that it, on the one hand, provides through its wrapper part the unique logical interface, inclusively its actual functionality, to the environment, and on the other hand, implements in its body part the decision making, the knowledge management, and the execution concepts/plans for its functionality (see Figure 2.5, adapted from Mařík and McFarlane, 2005: Figure 2.4). Such a wrapper-based agentification process of a holonic system provides an elegant, even though time-consuming mechanism for system integration. Agentification allows an arbitrary entity to be integrated into the manufacturing system because its static and often proprietary interface can be wrapped by the agent, and thus be replaced by an interface understandable by the HMS. Of course, standards are a better and less time-consuming approach to integration, but reality usually proves that their comprehensive realization is just lip service and not wholeheartedly supported by the industry. In a fully standardized environment, the wrapper part of an agent will be trivial (i.e., its overhead will disappear).

The agentification process of an HMS may strongly influence its original architecture. For example, a leaf holon often represents a physical device such as a machine or tool. In the agentified holarchy, the functionality and capability of such a leaf holon can be subdivided and split between several agents, each of which represents a different duty or task of the device, such as loading and unloading of components, the negotiation of the machine utilization, maintenance planning, supervision of the machine behavior, and so on. This mutation from a leaf-holon to a non-leaf-holon permits a better modularization of different tasks and a better adaptation of the architecture of the different agents to their specific duties, and thus also guarantees the leanness of the MAS-HMS.

The holonic paradigm stands for a clear, well-defined, and robust level-based control structure. In accordance to Mařík and McFarlane (2005), Figure 2.3, a MAS-HF may consist of three general levels, which, in reality, may consist of many more levels (see Figure 2.6). The highest level deals with the general management, coordination, and high-level goals of the MAS-HF. It corresponds to the duties of the management board and the CEO of a company. The medium level is the one where the production process is planned and scheduled. The lowest level deals with the actual execution and supervision of the production processes. As a rule of thumb, it can be said that the necessity for real-time decisions and operations increases from top to bottom, while the level of intelligence needed for the decisions decreases. Because the highest level deals primarily with long-term strategic and economic decisions, it needs a lot of computing capacity, knowledge, information, and intelligence to deal with the usually exponentially growing solution space for the multi-objective decision making. This can only be realized by complex, deliberative agents that cooperate on a semantically high level. In any case, it will reach even in simpler cases the limits of the capacity and capabilities of highly intelligent deliberative agents and their semantically meaningful interplay. The medium level is concerned with the actual planning and scheduling of the production process, which can be seen as a very relaxed version of real-time requirements. The lowest level supervises the actual execution of the production process and is highly real-time dependent. Whenever a disturbance occurs (machine fault, missing raw material, locally restricted re-planning—perhaps due to a delay in a production process), this level has to react in real time. The lowest level is usually realized by reactive pattern-based agents. For the time being, only these kinds of agents may guarantee real-time behavior. Because real-time behavior needs a fast communication path, the underlying devices may exchange information directly (see Figure 2.7, adapted from Mařík and McFarlane, 2005: Figure 2.4). Only if the agent needs to know about such information is it communicated from the device to the agent body. The decision capability is in any case only with the agent body. If an unforeseen situation occurs, it tries to match it with its pattern. If that leads to a solution, it is directly realized. If not, the problem is forwarded to the next higher level. In general, this means that higher levels may move from a reactive architecture via a hybrid architecture to a deliberative one. In case of locally controllable failures, they may be solved on the lowest level. However, if more serious failures occur, more intelligence might be necessary, which conflicts with the real-time requirement. Thus, a hybrid agent may, on the one hand, maintain flexible higher-level patterns for fast real-time reactions. On the other hand, its deliberative part may monitor and constantly analyze the production process, as well as possible feedback in order to not only anticipate possible disturbances but to develop in advance useful reaction patterns to them.

Another option for faster decision processes is to supplement the vertical communication and decision process with a strictly regulated horizontal one. The normal communication between neighboring holons is done via the mediator. This may be too slow for real-time requirements, especially if a (sub)holon in a neighboring holon is to be consulted. The mediator approach requires first a walk up and then down the hierarchy, which, in urgent cases, will cost too much time. Therefore, only the official and semantically meaningful communication, and especially the decision making, is realized via the mediator agent, while the exchange of relevant information is done via fast direct communication regardless of whether the agents belong to the same holon or not (see Figure 2.8). In extremely urgent situations, even horizontal decision making may be supported. However, the fact that the vertical communication is now complemented by a horizontal one requires that both forms are clearly semantically separated from each other in order to exclude negative influences on higher-level holons due to missing or outdated information. Such a separation may be given if lower-level decisions can only influence the local environment without any relevant consequences for other levels, or if the lower-level holons decide on the basis of prefabricated decisions (e.g., pattern matching) and only need to collect necessary input—such as facts—from neighboring holons. Such information may then either be communicated vertically as well, or is not relevant on higher levels. The higher that one gets in the holarchy, the more long term the decisions are that need to be made. At high levels, time may still be an important factor. Nevertheless, the reliability and soundness of the proposed solutions are more important. Thus, these levels are usually represented by more intelligent (deliberative) agents that obey the predefined communication and decision paths.

Currently, this lowest real-time sensitive control level is often realized by industrial programmable logic controllers running in a classical scan-based manner (cf., e.g., Hegny et al., 2008; Zoitl and Prähofer, 2013). They ensure real-time responsiveness from the control system and provide natural I/O interconnectivity to the real manufacturing process. The common IEC 61131-3 compatible standard programming languages are more and more being replaced by the IEC 61499 function blocks standard, which builds on top of the IEC 61131-3 definitions and function block diagrams. It pursues a platform-independent, component-based approach to the modeling of control structures via instantiating and interconnecting function blocks (FBsFBs are similar to event-action rules. An FB remains passive until triggered by the occurrence of an event. Then, it executes its underlying algorithm, which may result in output data and new events. The latter may trigger the execution of other FBs. This allows the programmer to implicitly specify a well-defined execution sequence of FBs. IEC 61499, in particular, suits automation systems in which the control and decision logic is decentralized and distributed across several (semi)autonomous hardware and software devices (cf., e.g., Zoitl et al., 2007; Leitão, 2009; Zoitl and Vyatkin, 2009). It also supports the specification of real-time applications.

In the meantime, MAS-HMSs have found their place in industry (see Mařík and McFarlane, 2005; Babiceanu and Chen, 2006; Mařík et al., 2007; Pěchouček and Marík, 2008; Bratukhin et al., 2011; Leitão et al., 2013a,b).

2.5 Development Tools for Industrial MASs

Agent-based concepts, systems, and technologies have already gone a long way to satisfy the requirements of modern industrial application systems. A lot of innovative concepts and systems have been developed. So, while for many problems good solutions are already available, the overall picture is not yet finalized, which means that a MAS that integrates all relevant concepts in one system in a mature way is still not visible at the horizon. This is, unfortunately, also true for development tools for agent-based industrial applications. According to our knowledge and Trentesaux (2009), up to now no global design methodology exists, although there are already a few encouraging first steps. A necessary prerequisite for the appearance of a global design methodology and its supporting tools might be the development of a solid foundation for emergence engineering, as well as engineering for decentralized systems. Both are related but comparatively young disciplines, where little knowledge and experience is yet available, especially among engineers (cf. Leitão and Vrba, 2011). To really understand how decentralized control behaves and what the relevant screws for fine-tuning are in order to control the emergent behavior of a system in such a way that it evolves in the desired direction, it will take more time and will need some enthusiasm and pioneering spirit.

2.6 How MASs Can Nourish Other Industrial Approaches

Because MASs have their strength in distributed environments with decentralized control, where robustness, flexibility/reconfigurability, and adaptability are required, all those approaches can profit because they essentially rely on parts of those features, such as virtual enterprises, swarm intelligence systems, autonomic, respectively organic systems, self-organizing, respectively emergent systems, holonic systems, or service-oriented computing (SOC). Some of those symbioses were already discussed. Here, some more brief examples will be given.

According to Camarinha and Afsarmanesh (1999), a virtual enterprise (VE) is a temporary alliance of enterprises that cooperatively work together to share skills or core competencies and resources in order to better respond to business opportunities, and whose cooperation relies on computer networks and a cooperative, yet distributed information systems architecture. The tasks of a VE are inherently distributed. Usually, involved enterprises are only willing to share the necessary information. Moreover, they have their own (business) goals besides the common overall goals of the VE. A MAS is an excellent implementation alternative for a VE. Each enterprise can be represented by an (interface) agent or even a holon, and respectively a holarchy that provides the necessary functions, information, and knowledge. VE creation is analogous to coalition formation (cf. e.g., Klusch and Gerber, 2002) in a MAS environment. The overall rules, constraints, and goals of the VE can be realized by agent norms, respectively policies, and accompany the individual goals of all relevant agents. Due to the unique common interface, the privacy issues of each involved enterprise can be obtained. This general communication path may be supplemented by clearly specified and controlled direct communication paths between relevant agents/holons of the involved enterprises in order to speed up their collaboration and quickly deal with possible failure situations. The relevant brokering and negotiation capabilities of a VE are already provided by a MAS, such as all kinds of auctions, the contract net protocol, blackboards, and brokering services such as white and yellow pages, and so on.

Emergent VE (cf. Kirn et al., 1994; Abbas et al., 1996) or holonic enterprises (cf. Ulieru, 2002; Ulieru and Unland, 2004) go one step further by constantly and automatically monitoring and analyzing their performance, their business processes, and the market in order to detect and eliminate bottlenecks and weak points and/or check whether there are (more) suitable possible partners available that may either replace existing ones or add to the overall business objectives of the virtual enterprise in a positive way. This may result in ongoing organizational changes, especially in the underlying information system (architecture). Such self-reflecting and self-regulating tasks can be performed by highly specialized deliberative agents, sometimes called meta-agents (cf., e.g., Pěchouček et al., 2003).

The objective of SOC is to construct complex software applications from pre-developed basic services existing within the underlying network, respectively the Web, and execute them wherever and whenever necessary. Such an objective can only be achieved if efficient and reliable techniques for the discovery, integration, and safe execution of such services are provided, especially in due consideration of desired quality of service (QoS) metrics and service-level agreements (SLAs). This is a shift from specifying exactly what service is to be used, to an (abstract) description of what the functionality of the service must look like, including the desired QoS metrics. In principle, four general tasks are to be covered: service description, discovery, orchestration (respectively, choreography in the following neutrally called composition), and execution. Next, we will discuss how agents can contribute to those tasks (cf. Payne, 2008; Unland, 2012):

Description: If an organization’s success depends on services provided by third parties, it has to trust that the services will perform as promised. This requires detailed descriptions of the behavior of a service, not only its functionality, in order to ensure that its runtime interactions are predictable, observable, controllable, and influenceable. Agent technology can help little on the level of a pure (even semantically enriched) description of a service, because the underlying techniques, such as semantic annotations or ontologies in the first instance, have little to do with core agent technology. However, depending on the standpoint, it can be argued that agents especially can make better use of these techniques, and can also wrap proprietary service descriptions. The situation is definitely different when negotiation is necessary (e.g., to negotiate whether service requests and service offers fit with each other, or to negotiate functional (QoS), as well as non-functional, service requirements (SLAs). Negotiation is one of the real strengths of agents.

Discovery: Here, agent technology offers many possibilities, which together with the intelligence of agents provides for a nearly optimal service discovery even if the services are described in a proprietary way.

Composition: In principle, services can be discovered and integrated during runtime or beforehand in a separate step—either on an abstract level (describing just the requirements expected from the concrete service) or a concrete level (integration of concrete services). To speed up the selection of a concrete service at runtime, services could be clustered and ranked according to their functionality, behavior, and quality metrics. These clusters could then be used to decide on possible services at runtime and in cases where originally chosen services need to be replaced (e.g., due to malfunctioning). All those tasks could be perfectly performed by agents. SOC has already achieved quite a lot when it comes to flexible compositions of services and/or abstract process models. Agents can improve these efforts by adding goal-driven negotiation processes, especially in cases where fuzzy nonfunctional properties are to be negotiated between the involved parties.

Execution: While more advanced approaches to SOC already offer some flexibility and fault-tolerance during service execution, this area is the one where agent technology can help the most. Agents permit specifying an extreme robust and flexible execution environment that can deal with all kinds of failures (functional and nonfunctional) in a flexible, smart, and individual way. Their ability to negotiate and reconcile all kinds of problems enables them to provide more stable and reliable execution environments. Finally, agents can exploit the redundancy provided by multiple alternative services by dynamically replacing nonconforming services during runtime. However, some issues are not yet solved satisfactorily, such as the problems of transparency/explainability, autonomy, and deterministic behavior. The latter two may lead to unpredictable behavior.

Governance: Service governance deals from the business point of view with all aspects of the people, processes, and technologies involved in the entire SOC life cycle and from the IT point of view with connectivity, configuration, execution, supervision, quality assurance (QoS, SLAs), and reuse during the whole lifetime of a service. This adds to the previously discussed technical aspects of SOC, a social and cultural level for specifying the actors and institutions involved, their objectives, requirements, dependency relations, and mutual agreements and contracts, and the organizational, business, and cultural rules that govern their interaction and the general composition and execution of services (cf. Brazier et al., 2012). Norms, rules, and agreements in institution specifications need to abstract from the concrete events and situations that they are meant to cover. They are intentionally specified on a higher level of abstraction in order to cover as many situations as possible and to keep maintenance needs over time as low as possible. On the one hand, this abstraction creates stability over time and the flexibility of application for the norms. On the other hand, the abstract concepts need to be related to the concrete actions that are to be performed during service execution to ensure compliance with the norms. Agent technology provides mature concepts for constructing formal machine-processable policies that capture high-level organizational, cultural, and governmental roles, rules, expectations, agreements, and norms, whose adherence can be guaranteed on each level (cf., e.g., Dignum et al., 2009; Aldewereld et al., 2010; Vazquez-Salceda et al., 2010).

In the recent past, cloud computing became popular. It relies on the sharing of resources to achieve coherence as well as economies of scale (cf., e.g., Mell and Grance, 2011). Like with SOA, cloud computing can also be about shared services, but in a converged infrastructure and on different service levels. Services may not only be provided on the software level but also in the infrastructure, platform, and communication levels. Thus, cloud computing can offer a reliable and highly predictable, yet extremely scalable computing infrastructure for the execution of agent-based, service-oriented industrial systems. Like the Internet, a cloud may offer huge quantities of services, but in a more controlled way. A MAS can, for example, comb through the utility market of a cloud in order to find the resources and services configuration that satisfies the requirements of the application best.

In industrial applications, the lowest (real-time) control level is often realized by industrial programmable logic. Here, the IEC 6113 and, especially in the recent past, its successor, the IEC 61499, are very popular. As discussed earlier, the lowest control level often only has to react in real-time. Thus, IEC 61499-based distributed low-level control may be supplemented on top by a MAS that provides the high-level control. Such an architecture can, for example, provide much better agility for the industrial application, because adaptivity to new demands may be achieved by the infusion of relevant information and knowledge to the MAS. Agents that understand the structure of the manufacturing system and the produced goods can then reconfigure the underlying system in an intelligent way. Thus, agents and the low-level IEC 61499-based control form a powerful and agile integrated system (cf., e.g., Hegny et al., 2008).

Large-scale industrial application systems—for example, for forecasting, prediction, and exhaustive simulation studies—can be highly demanding computationally and may require deploying super computers and parallel processing techniques, something often described as high-performance computing (HPC). If such applications rely on a MAS, the latter needs to use the HPC resources as efficiently as possible. This, especially, means that the distribution of the execution of a complex task on a huge number of nodes or processors needs to be organized in an efficient and balanced way. Also, communication and data transfer needs to be performed in an intelligent way in order to avoid an overflow of messages and to ensure the correct and efficient synchronization of tasks. Parallelism as supported by HPC is different than parallel work or cooperation within a MAS. Thus, in order to combine both technologies, some middleware may be necessary, such as that provided by Repast HPC (Leitão et al., 2013a, b).

A final example is the FEG. The integration of renewable energy production facilities and the move from a few large-scale producers to a plethora of small and medium-sized producers will also mean that the FEG cannot be run as a mainly centralized system any longer. A massive move toward decentralized control, management, and decision making is inevitable. Here, MASs seem to be a very promising realization alternative (cf. Ramchurn et al., 2012; Strasser et al., 2013). The FEG needs to integrate a large number of legacy systems and applications with new inventions such as renewable energy production facilities or smart appliances. Unfortunately, this means that a huge number of incompatible entities need to be integrated. However, the absence of necessary unifications and standards blocks further developments that would enable the creation of novel, market-driven, and hybrid control solutions for various types of technical systems. To overcome these problems, the notion and definition of a unified autonomous software entity, called energy agent, was introduced. Based on the energy conservation law and a generalized energy option model, an energy agent has the capabilities to enable cross-domain interactions between different types of energy systems and networks by wrapping the proprietary entities in an intelligent way, allowing them to cooperate and function in a FEG no matter what the actual underlying entity looks like.

To summarize, in the meantime, agent technology has matured enough to be on the edge of being accepted as an experimental platform for constructing fully integrated systems from (proprietary) third-party components, especially due to the automated communication, negotiation, bargaining and argumentation techniques, distributed resource allocation and aggregation techniques, and the development of formal concepts for organizational modeling.

2.7 Industrial MASs: Challenges and Research Areas

Agent-based solutions, especially for industrial application, bear a huge potential. In order to fully exploit it, several conceptual and technical challenges need to be addressed (cf. e.g., Bratukhin et al., 2011; Leitão and Vrba, 2011):

Autonomy: A MAS with its autonomy-based decentralized control relying on collaboration and communication seems to be an ideal candidate for the realization of a full heterarchy, especially because the agent paradigm stipulates autonomy for agents. Thus, a pure MAS cannot be organized in a hierarchical way but needs to exhibit a flat structure in which decisions are taken through a collaborative approach. However, in reality, many industrial applications reveal a semi-heterarchical or even hierarchical control architecture. This implies that the autonomy criterion has to be compromised. The holonic paradigm, extended by agent norms and policies, offers a starting point for MAS-based systems that also support hierarchical (control) structures and decision making.

Unpredictable emergent behavior: The combination of decentralized control, autonomy, and intense collaboration and communication usually leads to nondeterministic and nontraceable behavior. In such an environment with decentralized decisions and real-time requirements, a decision ideally is to be made as close as possible to the source of the triggering event. However, a locally acceptable solution may only guarantee a local optimum, not a global one. This phenomenon is sometimes called myopic behavior in literature because the way the control structures are implemented allows a good reaction to actual events and disturbances but does not support long-term or even medium-term objectives very well (cf. Leitão, 2009; Adam et al., 2011; Rey et al., 2012). Myopic behavior makes it difficult for human beings to accept the overall behavior of the system, respectively to build up substantial trust in the system, especially if the performance of a system is analyzed and judged only afterward when all parameters and behaviors are known. A possible solution is to introduce a set of global rules, norms, and behavior patterns that have to be obeyed by all agents. This approach will not solve the problem of global optimization, but it can ease it substantially. The most promising approach, however, is the application of extensive simulations.

Education: In terms of conceptual challenges, the agent-based paradigm is a new way of thinking that comes with a fundamental paradigm shift from centralized to decentralized control and decision making. For the time being, the organization of industrial manufacturing systems and the thinking of the relevant people in the manufacturing industry business are still heavily influenced by the idea and operating principles of centralized control systems. As soon as engineers and managers fully understand the new technology and its potential, their trust in its capabilities and robustness will increase substantially.

Semantics: Interoperability, loose coupling, SLAs, and reactions to runtime problems during the execution of complex tasks require more research on ontologies and semantic web domains and can profit a lot from the research done in SOC if its relevant features, concepts, and solutions are integrated into agent technology (cf. Leitão, 2009; Borangiu, 2009; Obitko et al., 2010).

Real-time capabilities: MAS and real-time capabilities are not love at first sight. Agents have at least some autonomy and, furthermore, collaborate in order to come to decisions. The latter means that some time needs to be spent on communication and computation. However, this necessary amount of time is not predictable, nor can a time limit be guaranteed. Moreover, communication between agents takes more time than the simple message exchange in object-oriented systems. This is caused by the additional overhead that the semantically higher level of agent communication produces, and sometimes by the less mature implementation of the communication process in agent frameworks and environments. In particular, reactive agents often cannot make use of this semantically higher level of communication. Therefore, the literature proposes more efficient communication mechanisms. Pokahr and Braubach (2013) propose active components. These are autonomous and possibly hierarchically organized software entities capable of interacting with each other in different modes, including message passing and method calls. Especially when deliberative agents are involved, the predictability of reaction times becomes difficult because such agents usually need (too) much computing power. For the time being, only a few MAS-based industrial applications deal with real-time requirements. Usually, those systems rely on bio-inspired, swarm intelligence approaches. Here, the agents are usually extremely simple, so computing power is not the problem, even if these systems consist of a huge (more than 5 digits) number of agents. Skobelev (2011) presents some good examples. Finally, the discussed combination of reactive and proactive agents in a MAS may relax this problem as well. Nevertheless, this area would profit from more profound research.

Scalability: When it comes to scalability, the majority of laboratorial prototypes deals with less than 1000 agents. Complex industrial applications may require much larger systems. Many platforms cannot yet handle big numbers with the robustness and efficiency required by industry (cf. Mařík and McFarlane, 2005; Pěchouček and Marík, 2008). Although the capabilities of agent-based systems have increased substantially in the recent past, more experiences with big numbers under industrial conditions are required.

Standards: Standards play a key role in the creation of all-comprehensive industrial systems spanning multiple software and hardware components. In particular, industry names the standardization issue as the major challenge for the industrial acceptance of the agent technology. On the machine level, a set of standards is already positively influencing the development of industrial agent-based applications. Besides the already mentioned standards, such as IEC 61499, and IEC 61131-3, the following standards are expected to increase in importance: ISA 88 & 95, IEC 61850, or OPC UA (cf. Leitão et al., 2013a, b). In the agent community, the IEEE Foundation for Intelligent Physical Agents (FIPA) (2014) is the most important standardization body, especially from an industrial point of view.

Mature development and simulation tools: Formal, structured, and integrated development engineering frameworks can improve the specification, design, verification, and implementation of agent-based industrial applications by engineers substantially. Ideally, the actual complexity of the agent-based solution remains internal to the system. Instead, the developer only needs to deal with a high-level interface with easily understandable configuration parameters and tools. In order to proof the maturity and applicability in industrial scenarios, comprehensive and trustworthy verification and testing tools are required that permit the execution of realistic tests. Some benchmarking issues are still open, namely the selection of proper performance indicators, especially those that permit the evaluation of qualitative indicators, the definition of evaluation criteria, and the storage and maintenance of best practices (including easy access to this service). As discussed already, simulations play an indispensable role in the understanding, verification, and trust-building of agent-based industrial system behavior before its real deployment. Due to the paradigm shift from centralized to decentralized decision making, such simulations will help to understand the functioning and behavior of such a system much better. Moreover, agent-based simulation tools can be used especially for applications that require a smooth transformation from the agent-based simulation to the agent-based system. Sometimes it may even be possible to use the agents from the simulation environment directly in the actual application.

If all these challenges are resolved, agent-based industrial applications can be expected to be more of a revolutionary than an evolutionary character.

2.8 Conclusions

The use of agent technology in industrial applications, especially when decentralized decision making and control is required, comes with several important strengths, namely in terms of modularity, adaptability, flexibility, robustness, reusability, and scalability. According to Leitão and Vrba (2011) and Leitão et al. (2013a,b), promising application areas are production planning, supply chain, logistics, traffic control, smart grids, building and home automation, military defense, humanitarian relief applications, network security, and unmanned aerial vehicles.

The robustness of such a system mainly stems from the fact that a MAS-based approach relies on decentralized decision making, which means that the loss of a single decisional entity may cause some local challenges but will not endanger the functioning of the overall system. For example, if production is to be restructured (e.g., due to the occurrence of a disturbance), the negotiation process will not change. However, different actors may now be deployed, making the system robust to changes.

Agent-based systems are pluggable systems that allow changes to be made in the production facilities, such as the addition, removal, or modification of hardware equipment, as well as software modules, without needing to stop, reprogram, and reinitialize the system. This feature is crucial to support the requirements imposed by customized processing, allowing dynamic system reconfigurability in order to face the variability of the demand. The migration to, or update of, old technologies or systems by new ones can also be performed in a smooth way without the need to shut down the system for some time (cf. Leitão, 2009).

The challenges identified in the last section constitute research opportunities from which the following are especially excellent candidates: verification and testing, interoperability, development of engineering frameworks and methodologies (especially ones that are directly targeting large distributed systems with decentralized control), simulation tools for large agent applications, and the integration of semantic technologies and concepts from SOC.

While agent research has in the meantime proposed good solutions for nearly all relevant topics, their full integration in a homogenous and all-encompassing development methodology and platform for full-fledged industrial applications is not fully achieved. There are many reasons for that. A very relevant one is standards. When it comes to industrial applications in highly complex and heterogeneous environments, they are an absolute must. However, as said already, producers of products are not very keen to (develop and) use them because that may open the door for too many competitors. The following chapters will shed more light on the strengths and weaknesses of agent-based solutions. However, altogether the author agrees with the statement of Leitão et al. (2013a,b), page 2370: “We believe that the basic concepts of MASs combined with the modern software technologies like service oriented architectures and semantic web will address this challenge and will help to make this idea come through.”