14.1 Introduction/Motivation

Facing the impact of global climate change, as well as the economic risks that come with a scarcity of carbon-based natural resources such as coal, oil, or natural gas, an ever-growing number of states and companies are rethinking their processes and energy usage patterns (Organization for Economic Co-operation and Development, 2013; United Nations Environment Programme, 2013). Apart from the provision of electrical energy by means of renewable energy sources, such as wind, sun, and water, other forms of energy can be provided by nature as well. Most prominently used for several centuries are different forms of solar water heating or the ever-growing utilization of volcanic heat for heat and electricity generation.

In modern infrastructures, the energy demand for room heating and cooling still surpasses all other energy needs such as fresh water heating, cooking, and lighting applications (UK Department of Energy and Climate Change, 2013a). For illustrative purposes, Figure 14.1 shows the energy demand of an average European household and a modern office building. This includes the electrical energy demand, as well as all caloric energy needs and infrastructures that its inhabitants may have. The resulting energy use over time can be synthesized by means of aggregation to so-called standard load profiles. Some distinctive peaks are observable throughout the day. They are related to the generation of thermal energy when showering in the morning, and cooking at noon and in the evening.

Because thermal loads—especially in the provisioning of hot water for all kinds of grooming activities—may be operated in a certain temperature range, it is possible to shift the generation of heat or cold on the timeline according to some very limited external factors. Furthermore, the energy source is not predefined. This means that it is irrelevant whether the water is heated up by solar energy, gas, or electricity. These facts result in two degrees of freedom, allowing systems to save scarce energy sources by shifting to another form of energy on the one hand and using temporal adaption to a fluctuating energy provision pattern from renewable energy sources on the other. By coupling different energy networks, such as electricity, gas, or heat grids, the peak demand in single energy domains may thus be clipped by using storage effects offered in the interconnected infrastructures.

Electrical energy generation is shifting away from large-scale centralized coal and nuclear power plants toward small and medium-sized distributed power generators, often based on fluctuating energy sources such as wind and sun. Due to the rise in complexity accompanying this development, the controllability and manageability of the electricity grid becomes a real issue. Situations in which the stability of the grid may be endangered will either lead to a short-time reduction in decentralized power generation or make it necessary to store the superfluous energy. Different national and international initiatives such as the e-energy program of the German Federal Ministry of Economics and Technology or the European Union’s Strategic Energy Technology Plan have addressed this point with a strong focus on electrical energy (Commission, 2010; VDE, 2013). Several mid-size to large-scale research projects, including real-life implementations, were funded in this context. The Model Region Harz featuring a virtual power plant made up of different distributed renewable energy plants (Speckmann et al., 2011) or the E-DeMa project enabling a market-based demand-side management (Belitz et al., 2012) shall be mentioned in this context.

As throttling renewable energy production is not in accordance with today’s legislative mandates and may be considered a waste of generation capacity, storage facilities should be installed. Other positive aspects of this approach may be found in the reduced need for physical grid extensions and the flattening of demand profiles when installing energy storage on a local level. For this, a number of viable options are technically feasible today. For example, flywheel storage facilities are already in use for quick response corrective actions. Furthermore mid-sized battery systems have been installed to some minor extent. The well-known pumped storage hydroelectric power stations complete the storage capabilities on the upper end. Other techniques such as compressed air energy storage or large-scale electricity storage based on other chemical processes are still subject to further technical and economic research (Lund and Salgi, 2009; Ribeiro et al., 2001).

This underlines the necessity for alternative storage concepts, which may be implemented in the near future. Using already available resources, energy networks, for example, can be regarded as one promising approach to unlocking the potential of storage capacities. Fluctuations in different energy domains, such as electricity, gas, or heat networks, may be compensated by interconnecting these supply infrastructures. The required energy converters, such as gas-fired power stations or power-to-gas converters, are already at hand and are expected to be working economically soon. As this linkage of different energy domains implicates a further rise of the overall system complexity, a shift from monolithic control structures to more flexible paradigms seems to be unavoidable.

The energy agent described in this chapter is such a flexible approach, supporting enhanced control scalability and multi-energy domain capabilities, based on multi-agent systems (MASs). As the basic ideas of MAS architectures fit perfectly with the challenges found in many visions of future energy grids, many labs and universities are actively doing research in this field. Besides general agent-based grid management systems (e.g., Kamphuis et al., 2010; Wedde et al., 2006; Platt, 2007; Ramchurn et al., 2011), approaches for grid restoration incorporating distributed generation capacities (e.g., Li et al., 2012), specialized agent-based micro-grid control systems (Roche et al. (2010) provides a good overview), and a multitude of energy-centered building automation solutions (e.g., Zhao et al., 2010) do exist today. The major drawbacks of such systems are stakeholder concerns in regard to stability, safety, and security related to unfavorable inter-agent communication and autonomous agent behavior.

Various solutions mentioned earlier do apply proprietary data exchange formats and architectures to avoid such issues. Even though some of them may rest upon the Foundation for Intelligent Physical Agent’s (FIPA’s) Agent Communication Language (ACL), non-uniform message body contents may lead to compatibility and comparability issues. A multitude of available standards such as IEC61850, the Common Interface Model (CIM), and other proprietary solutions for the different aspects of data exchange and control needs in the energy domain are currently available. Nevertheless, they are mainly designed for centralized and hierarchical decision-making systems (Benze et al., 2010).

Regarding the rise in system complexity, coming along with the interconnection of different energy domains, and the need for a unified communication architecture interconnected with a common knowledge base, the “energy agent” presented in this chapter may support the further development of future energy grids. It features a multi-energy domain environment model, ensuring an economically sound and energy-efficient way of coupling different energy infrastructures such as gas, electricity, and heat distribution grids for the purpose of interdomain energy exchange and intra-domain load and cost optimization. In this context, the energy agent’s primary objective is the unified provision and exchange of information supporting the decentralized decision-making process in the design, implementation, and utilization stage of the project. The optimizers used are not the center of attention and will therefore be described very briefly. There are—to the best knowledge of the authors—only a few analogous projects. One of them is looking for integration strategies to place heat pumps into smart electric grids (Heat Pumps Consortium, 2013). Besides heat pump hardware–related works, a control platform for air/water heat pumps is supposed to be developed, allowing the integration of other renewable sources as well. Due to the project being in the early stages, no detailed technical descriptions or objectives are available yet.

The “optimizing hybrid energy grids for smart cities” project, which aims to support the interactions of coexisting energy grids in smart cities (ORPHEUS, 2014), features the most comparable characteristics, with a focus on control strategies and design. Because the project was founded in September 2013, no substantial decisions on technical backgrounds have been published yet. Thus, the energy agent can be seen as a first-time attempt to merge different energy domains by means of decentralized decision making on the same semantic base. Consequently, the implementation of a unified energy agent or a similar approach in connection with a systematically defined development process is indispensable for further developments and the ongoing transformation process of the energy supply.

In this chapter, the concept of the energy agent and its required environment is introduced. Furthermore, an application scenario featuring a hybrid testbed connecting simulation and real-world devices consuming and producing different forms of energy is presented. Finally, the benefits of this approach will be pointed out and a brief overview of the potentials and possible drawbacks of the approach will be given.

14.2 Application Overview

As suggested in Derksen et al. (2013), an energy agent may be defined as follows:

The goal of this definition is to describe a generalized prototype of an energy agent, which may be used in different energy domains such as electricity, gas, heat, and water. This energy agent shall be applicable in different types of equipment and on different levels of the global control system. It shall not only enhance the level of sophistication in regard to dynamic resource allocation but integrate all energy systems that are subjected to the fundamental first law of thermodynamics, stating that energy can neither be created nor destroyed. This affects and encompasses all systems that are capable of storing energy or converting it from one form to another. Regarding the energy flows into and out of a defined single system, the sum must thus be always equal to zero:

$\Sigma {\dot{E}}_i=0$

An electricity generator, where mechanical energy is converted to electrical energy, or a heating system, where chemical energy is converted to thermal energy, may be seen as good examples of systems where energy conversions from form i to form p take place. Taking this principle as the basis for all further considerations, energy agents, and the technical systems they control, may be regarded as individual entities that are nevertheless connected by several, hybrid energy transport, as well as information, networks. Thus, an energy agent must be capable of describing and handling the amount of energy its underlying system may emit (produce) or obtain (consume) over time. Out of this core skill and these assumptions, the need for a well-defined and suitable data or option model arises, enabling the energy agents to exploit and communicate the obtained information and imparted degrees of freedom of the underlying system. Furthermore, a consistent time model is required for modeling temporal effects. This model has to consider the fact that the temporal behavior and controllability of single or clustered energy conversion and storage systems can range between unpredictable and barely controllable production and usage patterns on one side to deterministic and flexible systems that are easily led on the other side. For compatibility reasons, with the current implementations of data acquisition systems and for safety reasons, time has not yet been discretized in equidistant steps and monitored by the individual energy agent. Concerning data acquisition, solution finding, and negotiations, the allowable step sizes will depend on the dynamics of the underlying system. The energy or equipment type, the throughput of the system in regard to information or energy, and other factors do affect this important definition. Including this fact in the agent’s deliberations, coordination, and planning processes may be considered for time-critical processes.

Regarding the varying complexity of the different systems, located in dissimilar energy domains controlled by the energy agent, it becomes obvious that individual systems require distinct capabilities. These may be classified in levels of sophistication according to the degrees of freedom a technical system may offer to an agent. In the following, the local capabilities of the onsite system are described to reflect the fact that current energy management systems are in general not capable of complex interactions with their environment, as it is commonly envisioned in the current scientific discussion.

Table 14.1 illustrates the different levels of sophistication—so-called integration levels—using an electricity meter. The functionalities described therein may be transferred to other systems and different energy domains as required.

As energy agents require some basic means of communication, an implementation in very simple systems corresponding to Integration Level (IL) 0 is of limited use. If additional data acquisition or control processes are required on a local level, as depicted in IL 1 and higher, energy agents may form a useful supplement. As the degree of decentralization rises within the higher integration levels, the implementation of more and more sophisticated energy agents becomes necessary. On the highest level, IL 5 represents a fully autonomous and decentralized energy supply system, based on the most elaborate energy agent. It can be assumed that a multitude of energy supply systems with a varying degree of decentralization in regard to their physical characteristics, as well as their control design, will be developed in parallel. This will lead to a concurrent presence of differently sophisticated energy agents, introducing another degree of complexity. Economic considerations prohibit the use of advanced full-scale control systems on IL 1 or IL 2. Nevertheless, the need for communication between systems of different complexities arises, making an abstract means of information exchange necessary. When sharing the system’s energy-related degrees of freedom and negotiating for consumption or production patterns with their cooperation partners, a common denominator has to be defined. Especially in cross-domain scenarios where intermediate technologies between different energy supply networks exist, a unified data model is required, enabling the communication of the system’s energy-related abilities over time and system borders.

On top of that, the data model described represents the internal processes of the energy agent. As the individual entity needs to have knowledge about its technical capabilities, boundaries, and degrees of freedom, a scalable model appears to be essential. The information stored may be very simple for basic consumers such as light bulbs, as there are nearly no constraints in the operation mode, but should be more enhanced for complex systems such as cogeneration plants, which may require characteristic maps to record all operation points, predefined patterns for transitive states such as ramp-up, shutdown, or maintenance periods, and time restraints for certain modes.

One of the major challenges found in this context was the design and merging process of systems featuring variable complexities. Even though the architecture presented allows for scalable data handling and processing, the decision on whether a certain information or dataset was required or not would still depend on an individual design decision.

In order to support this decision-making process, a fundamental two-dimensional subdivision is proposed for the creation of energetic option models. Applying this system allows us to describe the typical characteristics of an energy system in the first dimension, while the second dimension depicts the temporal behavior of the individual system. For this purpose, the energy systems’ temporal consumption, production, and storage behavior will be classified, as shown in Table 14.2

The collection of models and constraints given earlier results in the systemic picture of an energy agent, as depicted in Figure 14.2. All information gathered will be aggregated in the agent by means of the trinity domain specific model (e.g., gas, electricity, heat, etc.), time model (time step size and temporal behavior of system or energy carrier), and the option and cost model (energy-related degrees of freedom and related costs). The constraints arising from the agent’s energy domain, as well as the option and cost model based on the underlying technical system define the operational boundaries to remain within. All message parsing and handling for inter–energy agent information exchange will be in accordance with the derived option and cost model. Communications with other technical systems and the underlying infrastructure may be established via a flexible plug-in architecture–based link, allowing the utilization of different protocols and standards easily. This permits the easy integration of the agent in available or additional hardware or simulation environments, connecting with the technical system and gaining partial or full control and monitoring capabilities. Figure 14.3 depicts the internal agent structure of an energy agent controlling a tumble dryer. Surrounding the central TumbleDryerAgent class, classes containing the specific agent’s behaviors representing the optimizers on the bottom, a class holding the agents data model on the left, as well as some classes managing the input and output of information for a simulated and real-world environment on the right hand side, can be found.

Thus, the agent enables a decentralized energy conversion and storage facility integration in scalable energy management systems for multiple types of energy by mapping energy-related processes to basic thermodynamic principles. In this context, it should be pointed out that the energy agent provides a set of interfaces for communication, data handling, and decision making, allowing for the interconnection of different devices in different energy domains. Because the energy agent allows for an easy integration of different optimizers, the decision-making process, as such, is not part of the research done in this project.

14.3 Application Details

In order to demonstrate the feasibility of the approach, a first showcase was implemented using a hybrid testbed made up of simulated and real-world hardware components.

The Java Agent DEvelopment Framework (JADE)¹-based Agent.GUI² simulation framework and end-user toolkit, developed at the University of Duisburg-Essen, was used to facilitate the implementation. As one main feature, Agent.GUI allows the modeling of different networks based on generic NetworkComponents. These can be defined by an identifier, a network type affiliation (e.g., electricity, natural gas, communications, and others), a graph prototype, representing a single technical system in a setup, and a specialized agent that will represent and control the technical system in the context of the scenario (Derksen and Unland, 2012).

In the use case described, the energy agents were implemented on the basis of the JADE framework, due to JADE’s high usability. It features a FIPA ACL message transport system, and white and yellow page services, as well as some auxiliary tools facilitating the analysis and debugging of the MAS. However, it should be pointed out that the energy agent concept is not merely limited to JADE-based application scenarios. Rather, it is supposed to describe a more general perception of an agent layout.

In the specific implementation as presented in this article, the following design decisions were made:

To ensure the portability of the energy agents as defined in Derksen et al. (2013), the equipment models are kept separate from the control elements in the simulation environment. For this, an approach similar to that used when developing the DEcentralized MArket based POwer control System (DEMAPOS) at Helmut Schmidt University in Hamburg (Linnenberg et al., 2011) was applied. Communication between control elements and equipment models was established via web services, featuring bidirectional information exchange. This allows running the control application, as well as the simulated equipment models, and real-world hardware devices in one runtime environment, while separating the individual entities from each other. As pointed out in Linnenberg et al. (2011), it is necessary to merely provide a network connection featuring an http-capable transport layer such as TCP/IP via Ethernet or the Wireless Application Protocol for this purpose.

The protocol used is based on Simple Object Access Protocol (SOAP),³ which is the most established protocol in inter-web-service communication. SOAP rests upon the eXtensible Markup Language (XML)⁴ that facilitates the message translation and communication handling.

To provide the required information and maintain a high flexibility, a modular simulation concept, as shown in Figure 14.4 was chosen. Commands from the control are received through an Apache Tomcat Webserver,⁵ from which they are transferred to MATLAB through the Modelit Webserver Toolbox.⁶ MATLAB is the core of the simulation, linking different sources of information with MATLAB Simulink modules, representing the devices connected to the grid. The characteristics and diurnal variations of those devices are based on real-world datasets stored in separate data files.

In the development and testing process, this choice permits exchanging the simulated entities partially or as a whole using real-world hardware components. This practice is supported by the fact that many well-known hardware manufacturers offer http web-service-based communication solutions for their products.

In this context, it has to be pointed out that the interfaces of Object Linking and Embedding for Process Control eXtensible Markup Language (OPC XML), as well as OPC Unified Architecture⁷ (OPC UA), allow for an easy data exchange via SOAP.

Other appliances such as refrigerators, washing machines, or heating systems that may not support an http interface yet may be easily upgraded by means of micro–web servers based on microcontrollers. This versatile expandability, as well as the usability of different transport channels designates web services as the preferred communication interface. Nevertheless, the integration of other communication protocols is ensured by applying the aforementioned plug-in architecture.

Internal data storage and inter-energy agent communication were established based on the option and cost model presented earlier. Because the energy agent’s footprint is supposed to be as small as possible, a very lightweight ontology based on Web Ontology Language⁸ (OWL) was designed and implemented, meeting the requirements of this specific-use case. It defines the required states and variables used in the context of this application. Following basic principles as described in Onto-ENERGY (Linnenberg et al., 2013) and the NASA SWEET ontologies,⁹ different energy-related domains were divided for better visualization:

The energy flow concept introduced allows for a differentiation of real- (measured-) and simulated (calculated) energy flows. Furthermore, the temporal behavior of such energy flows can be divided into dynamic or fluctuating behavior and constant flows. Different energy carriers—in this case, electricity, heat, and gas—are defined in order to describe the specific domain properties in regard to energy characteristics, as well as its particular temporal behavior. In order to assure an efficient and safe operation, the concept of plant properties provides plant-inherent boundaries to the energy agent. Further distinct actions and values such as the state transition and state concept, as well as the connected concepts of costs and energy amounts were defined to facilitate the later usage of state-machines in the process of system design and analysis. The basis for the resulting structure can be found in Figure 14.5.

The implemented testbed spans the three different energy domains: electricity, gas, and heat. Every domain features unique characteristics, resulting in specific-use cases and advantageous application scenarios:

– Electrical energy is made up of 100% exergy—this means that it can be converted into other forms of energy easily. Electricity has to be gained by converting other forms of energy such as mechanical, chemical, or thermal energy. Examples for energy conversion plants are generators (mechanical energy), batteries (chemical energy), or peltier-elements (thermal energy). Storage of electrical energy may be directly effectuated in electrical fields (e.g., capacitors). All long-term storage possibilities are based on other forms of energy such as mechanical (potential) energy in pumped storage hydropower stations or chemical energy in batteries. These conversions are all subject to the second law of thermodynamics and are, therefore, lossy. Due to the simplicity of its transport, its ease of controllability and its high-power density, electrical energy remains, nevertheless, the most common form of energy used by most technical systems.

– Gas features a high amount of exergy as well. However, the conversion in other forms of energy is very lossy due to its exothermic manner. Because the gas supply infrastructure offers vast storage capacities, it is seen as a potential balancing element for fluctuating energy sources such as wind or solar power. Due to their small turbine size, gas-fired power plants and heating systems need comparatively short ramp-up times and are, thus, ideally suited for flexible on-demand operation.

– Heat is a type of thermal energy, which is very lossy. Thus, it features only a small amount of exergy. Heat may be transported thermally across system borders. As in most thermal applications such as heating, ventilation, and air conditioning systems the operation point is not a fix but rather a target corridor in which the control may operate. Thus, these systems offer a certain flexibility in operations. In this way, thermal systems are comparable to gas applications—they offer temporal and energy-related degrees of freedom in positive and negative directions.

To allow energy exchange between these three domains, as well as reactions on fluctuating demand and provision patterns in particular domains, the following eight energy conversion systems were identified:

– Solar thermal plant—The solar thermal plant converts solar energy to thermal energy and may be ranged in the class of medium-temperature collectors. Due to its dependence on availability and the strength of solar radiation, the missing heat-storage capacities and the noncontrollability of production, it is classified as a fluctuant source of thermal energy. The implemented plant model is based on physical characteristics and 24-hour production profiles measured within a real-life plant. In Figure 14.6 the system is symbolized by a sun with a flame next to it. See Table 14.3 for more details.

Table 14.3

Implemented energy conversion systems

Icon	Description	Icon	Description
	Solar thermal plant		Fridge
	Photovoltaic plant		Radiator
	Gas heating system		Constant consumer
	Combined heat and power plant		Electric heating

– Photovoltaic plant—The photovoltaic plant converts solar energy to electricity. The system properties are comparable to those of the solar thermal plant described earlier. It can thus be seen as a fluctuant source of energy allowing for binary control only. The system model features physical characteristics and production patterns recorded over 24 hours on a real-life plant as well. The electricity provision thus follows the same scheme as the heat production in the solar thermal plant. The plant is depicted as a sun with a lightning bolt beside it in Figure 14.6.

– Gas heating system—The gas heating system features flexible gas-to-heat conversion capabilities in the form of a central heating system. In the absence of sunlight, the central heating system can provide sufficient heat to cover all heat demands in the simulated infrastructure. It may be adjusted to any given operating point in the offered temperature spectrum. The effectuation will be subject to a short delay, introduced by a PT1-term in the simulation. Due to thermal losses, the gas-to-heat conversion is nonlinear and is described by an exponential function relative to the gas input. The heating system is symbolized in Figure 14.6 by a gas tower next to a flame.

– Combined heat and power plant—The combined heat and power plant (CHP) features a simple physical model of a real-life, mid-sized CHP. The model’s power output is limited by an upper and lower operational boundary defined by the manufacturer. Within these limits, the control can choose any operation point, which will be approached considering the underlying PT1-term. This approach was revealed to be the best trade-off between simulation complexity and the quality of results achieved. Within the featured system, it may be power- or heat-operated. Thus, it is the most flexible electrical power generator and allows for control in a wide electrical power and heat range. When power-operated, it will generate thermal energy as a by-product, and vice versa. The gas-to-power-and-heat ratio can apply to the characteristic map of a small-sized plant. In Figure 14.6, the CHP is represented by a box with a heat exchanger and central lighting.

– Fridge—The fridge used is a real device furnished with a Raspberry Pi microcomputer¹⁰ to provide a JADE-agent runtime environment. Hardware control is realized via an Arduino¹¹ microcontroller board interface featuring binary compressor control and internal, as well as ambient, temperature acquisition. These values are combined to calculate the energy-related needs of the device. It is capable of forecasting energy demands and adapting its usage pattern according to energy availability by working in a given temperature range of between two and eight degrees Celsius. Thus, it provides valuable degrees of freedom for energy usage optimization. The implementation of the device serves for the introduction of error sources such as arbitrary delays due to communication, real-world physical situations, and user interaction.

– Radiator—The radiator features a thermal energy sink. It may be operated within a certain room temperature range. The temperature is calculated on the basis of a physical model, considering measured temperature patterns of a real-life infrastructure, and thermal transfer and compensatory processes are calculated there from. The radiator symbol in Figure 14.6 stands for this particular system.

– Constant consumer—The constant consumer represents all electric needs a household has apart from the fridge and electric heating. It is based on a standard consumption profile used for grid calculations. The consumer cannot be switched off. Therefore, a constant electricity supply in the required extend must be ensured. A fluorescent energy saving bulb represents the constant consumer in Figure 14.6.

– Electric heating—The electric heating system converts electrical energy into heat and may be used to eliminate surplus amounts of electrical energy influencing grid stability. Different forms of electric heating exist, varying from simple resistor-based appliances, such as radiative heaters, to electric motor–based systems driving a refrigeration cycle—better known as heat pumps. In the scenario simulated, a heat coil in the closed heating circuit was chosen for reasons of simplicity. It is represented in Figure 14.6 by a flash accompanied by a flame.

Table 14.3 gives a short overview of energy converters used in the scenario presented next. They are sorted by sources of different energy types on the left and their adversary sinks on the right.

The unity of the hybrid testbed is thus made up of three different energy domains in which seven different energy conversion systems are located. These are predominantly set in the field of electrical and thermal energy. A gas heating system and the combined heat and power plant bridge the domains of gas and heat. The latter also interconnects thermal energy with gas and electricity. While heat generation may be provided by a solar thermal plant from solar energy, an electric heating system, or alternatively a gas heating system or even a CHP, can do the same. Thermal energy will be consumed by the radiator model. In contrast to this, electricity generation will be ensured by a photovoltaic plant or the CHP. A real-life fridge, a constant consumer model, and the aforementioned electric heating system serve as sinks (see Figure 14.6 for a general overview).

Following the experiences made when developing DEMAPOS in 2011, a holonic control system architecture was implemented. Compliant with this hierarchical approach, devices are made up from different systems, representing the different forms of energy a device may consume, produce, or store. These devices in turn are subordinate to a smart house meter, elaborating a locally optimal solution. Figure 14.7 shows the resulting communication sequences. After registering with their respective device agent, one or several system interface energy agents report their current degrees of freedom to it. The device energy agent gathers the information received and communicates it in an aggregated form to the superior smart house meter agent.

In accordance with the integration levels (ILs) defined earlier, the system interface agent may be classified as an IL 1 component. This is due to the fact that it will only communicate the data measured, without further processing. The device agent may be seen as an IL 2 element, featuring minimum intelligence needed for data aggregation and processing, as well as two-way communications. Because it is able to interlink the received information and control the underlying elements based on some limited reasoning capabilities, the smart house meter is the highest-ranked component in this implementation, ranging into the IL 3 domain.

As the balancing algorithm implemented in this scenario is not the focus of the current research activities, it serves as a proof of concept only and is thus deliberately kept to a very rudimentary base. Figure 14.8 illustrates the decision tree, which is driven by the devices in the electricity and heat domain. After aggregating the electric load’s power demand and comparing it to the local electricity source’s capabilities, the supply and demand ratio in the heat domain is taken into account as well when configuring the local devices. Given the case where the electricity supply surpasses the electricity demand and the heat provided by renewable resources suffices to satisfy all heat sinks, the combined heat and power plant (CHP), as well as the gas heating system and the electrical heating system (EL heating), are switched off. Should there be a further need for heat and an abundance of electricity, it may be generated by switching on the electrical heating system though. In the case of a lack of electrical energy and a gas price that is lower than the price for externally bought electrical energy, it may be produced by powering up the local CHP. Regarding the less common situation, that there is a lack of electrical and thermal energy, and the electrical energy supplied form outside is the cheapest form of energy, the system may decide to cover all its energy needs with externally supplied electrical energy.

After elaborating on a possible solution by applying this algorithm, it is cross-checked against the operational boundaries and degrees of freedom as communicated by the connected devices. Should there be any interference, new setpoints are recalculated after assessing and adapting the input values used. Should the solution found satisfy all requirements posed, it is allocated to the affected devices. These in turn distribute it to the individual underlying power systems.

This way of allocating resources may be found as non-optimal or somewhat over simplified. Due to the fact, that the optimizer is not in the center of this work’s attention, the decision to make it small and simple was taken deliberately. The energy agent shall provide the means to implement any desired solution by finding an algorithm in a short time, while maintaining a common knowledge base for data storage, handling, and communications based upon a replicable development process.

14.4 Benefits and Assessment

The test runs revealed promising results. By means of abstract models and decision logic, the usability of the energy agent approach could be verified. Figure 14.9 shows the results divided into the three energy domains—electricity, gas, and heat—as well as the respective fuel costs for the first two. The energy-related Figures 14.9(a–c) are set in the same unit of Watts for reasons of comparability. The simulation calculates new states in discrete equidistant time steps of 15 minutes based on 345.600 quarter-second values per model, aggregating the energy conversion processes over time. The execution time of the simulation was optimized on a 4-Hz clocking. This is owed to the control’s 2-Hz work cycle and the Nyquist-Shannon sampling theorem. Thus, the simulation will calculate at least once the grid conditions resulting from the control’s input parameters and precedent node states before the control queries the respective datasets.

The electricity price is inversely proportional and linked to the energy production pattern of a wind power plant, which is not shown in the diagrams. When it drops below the costs for gas at about 05:00 AM, it can be observed that energy conversion processes in all three domains are affected immediately, emphasizing the high reactivity of the implemented optimizer. In this context, reaction times of less than a second can be observed on average. Due to the hierarchical system layout and the connected aggregation of information, a linear growth in complexity can be assumed for larger agent populations in this particular showcase. Nevertheless, this behavior cannot be declared as typical for energy agents because it is heavily dependent on the optimizer implemented. The energy agent concept allows each system developer to implement their individual solvers and have them tailored to their needs.

These design decisions based on the energy agent concept affect the real-time performance of the entire system in a similar fashion. Even though the energy agent concept features a time-model, which is the basic prerequisite for a real-time enabled system implementation, it was refrained from incorporating complex real-time mechanisms in the specific showcase presented here for reasons of operability. Because high performance of the system was observed in the context of the test runs performed, and the hard deadline imposed by the 2-Hz cycle times was met under all conditions, no further action was found to be necessary. Execution times for an entire information round-trip starting from the simulation stimulus at the system interface via the device agent to the smart home meter agent and back took 0.3 seconds on average. These results were obtained running a large part of the agent community, as well as the respective MATLAB simulation in parallel, on an Intel i7 1.8 GHz 64-bit Windows 7 virtual machine provided with 1.5 GB of main memory. Due to the Wi-Fi connection used, the real-life fridge equipped with a Raspberry Pi microcomputer accounted for most of the delay.

Because the electric generation of heat offers a higher efficiency than the gas heating system, and, as pointed out earlier, the price for electrical energy is very low, the control decides at 05:00 AM to cover all its heat demand by means of the electric heating system. Thus, the operation of the combined heat and power plant is throttled down to zero, considering plant specific ramp-down and ramp-up behavior. In cases such as this, the implemented energy agent community takes the different plants’ temporal behavior as well as their efficiency relations into account before switching from one form of energy to another. These efficiency values are communicated along with the different plants’ ratings and degrees of freedom by the particular energy agents connected to the individual energy-conversion systems. The power-operated combined heat and power plant can serve as an example for this fact. Because it features different energy conversion processes, such as gas to heat and gas to electricity, the respective efficiencies for the individual conversions may be expressed and interconnected. In this particular case, a 31% efficiency when converting gas to electricity may be perceived. In contrast to that, the gas-to-heat conversion offers a 52% efficiency. By adding up the values of different plants, relative efficiencies and efficiency threads may be considered in the decision-making process as well. This is done by adding the amount of extra energy resulting from a potential inefficiency to the needed input energy flow. When following the electricity demand, as depicted by a purple line in Figure 14.9(a), it produces heat in parallel, observable in Figure 14.9(b) while consuming gas, as shown in Figure 14.9(c). The periodic spikes of about 100 Watts in between 00:00 and 05:00 in the electricity domain caused by the fridge can be taken as a reference point for this observation. In the specified period, the gas price is low. Thus, the electricity production is ensured by the CHP, causing side-effects in all other energy domains. Due to the increased heat production accompanying the ramp-up of the plant, the radiator will take the chance and increase the room temperature within the given boundaries to prevent a dissipation of energy. These interconnections can be synthesized by means of the energy agents’ Option and Cost Model, allowing for the characterization of domain properties and the respective dependencies in intra- and inter-domain relations. This scalable Option and Cost Model is shared by all stakeholders with a strong focus on the information needed by the different agents. When analyzing the different energy converters located in the heat domain, it becomes apparent which way the requirements differ with regard to the model’s complexity. The least sophisticated model is required for integrating the heat generation by means of a solar-thermal plant introduced into the control system. As its heat output is uncontrollable and subject to external factors, it may be sufficient to only describe this single value. In parallel to the photovoltaic plant and the constant consumer in the electricity domain, it is thus assigned to Integration Level 1. In contrast to that, energy conversion processes allowing for more elaborate control will require more variables to be described in a flexible way. Regarding the radiator, which features an operation range dependent on the outside temperature and a flexibility in the targeted room temperature, adaptable demand response patterns allow for a more flexible control. It may consequently be allocated to Integration Level 3, as well as the electric- or gas-heating system and the implemented refrigerator. Other systems, featuring further degrees of complexity, such as the combined heat and power plant, can be included in Integration Level 4.

The combination of different plants, featuring distinct behaviors and control options, spanning multiple energy domains with specific properties and constraints, calls for a multidimensional, nonlinear optimization algorithm. For small-scale scenarios such as those presented here, an approach based on one central control unit still seems to be feasible. Nevertheless, as the problem becomes more complex, the solution space will grow in a disproportionate manner. In this case, the problem may be broken down into smaller fragments—possibly solvable by a small population of cooperative units as presented in this chapter. Individual solvers may be implemented on the base of the energy agent described to tackle this nontrivial problem.

In doing so, new challenges may arise, which might have been disregarded until now. In particular, concerns regarding control stability are commonly expressed when assessing a decentralized decision-making system. Most of these issues are caused by latencies found in communication and decision processes, leading to the feedback of already outdated information. Because the concept of a unified agent structure, as well as homogenous data handling, processing, and communication is inherent to the energy agent approach, such latencies are minimized to a great extent. Due to the eminent real-time behavior of the testbed presented here, no latency-related oscillations were induced by the control. Attention should be paid to the fact that the energy agent approach does not in itself prevent oscillations and instabilities, but is meant to support the system designers in their objective of creating a sound and resilient system.

14.5 Discussion

A cross-domain energy exchange, controlled by a decentralized agent-based network, in accordance with environmental stimuli, was established using the energy agent concept. It was therefore possible to seamlessly integrate different levels of sophistication in regard to system and associated control complexity in a diversified multi-domain environment. This task was accomplished by introducing a flexible option and cost model enabling the individual energy agent to scale its data storage and handle overhead according to the specific needs of the particular system under control. Furthermore, all communication between energy agents was based on this common data model, enabling interactions between heterogeneous systems. This internal data exchange was shown not to limit the agents in their interactions with systems under their control. For these, a lean plug-in architecture-based communications interface was realized, enabling the flexible integration of different communication standards and proprietary solutions. Using this interface, a serial connection to an Arduino microcontroller board was established, enabling the data acquisition and control of a real-life fridge. The respective JADE-based energy agent was running on a Raspberry PI microcomputer for this purpose. Due to this consequent pursuit of the flexible communication with external devices and a stringent adherence to the structure of the internal data model, an easy integration in simulated testbeds and adaption to real architectures can be observed. The preparation of the system for future upgrades benefits from this advantage as well.

While maintaining the flexibility of adapting to a high order of complexity induced by a rapidly increasing number of devices boasting ever-increasing degrees of freedom in regard to energy control and data acquisition, the energy agent concept may keep down the time needed for decisions by grouping concepts because they can be implemented by means of a holonic architecture.

This showcase of a hybrid testbed featuring simulated and real-life components working in different energy domains and being controlled by a distributed network of energy agents underlines the importance and applicability of multi-agent systems in the energy domain, and thus for electricity, gas and heat transport, distribution, storage, and conversion.

Nevertheless, the optimal control strategy for such complex systems is not achieved yet. The energy agent can only support the design, implementation, and utilization of such novel approaches, but does not carry in itself the answer on how to evaluate, optimally distribute, and run energy conversion processes.

14.6 Conclusions

In this chapter, an automated optimization of cross-domain energy exchange in a smart-house environment was realized using the novel energy agent concept. The used agent technology and the implemented decentralization of solution finding ensure a good scalability while maintaining redundant structures and a high degree of fault tolerance. Problems occurring with the diversification of systems and protocols used are encountered by introducing a unified systems architecture on the basis of energy agents. However, compatibility with other components is ensured by using a plug-in-based device-communication infrastructure. This approach allows for different and adaptable communication channels, as shown by incorporating a real-life fridge equipped with a Raspberry Pi–based JADE controller implementing an individual energy agent into the simulation environment.

The showcase presented here has demonstrated that a proper definition of optimization parameters is needed for every single entity, which still requires planning ahead. This factor may slow down the deployment of such decentralized solutions. Therefore, a semi-automated goal definition based on individual energy domains and other environmental factors has to be established. Furthermore, only a limited range of hardware meets the requirements for automated control in a household environment yet. Therefore, this approach—regarding current circumstances—is only suitable for factories and infrastructures featuring a deep penetration of measurement and control equipment.

The energy agent will therefore be continuously enhanced. This process will go hand in hand with the further elaboration of the respective development process, enabling the utilization in more restrictive environments, and requiring the demonstration of the controls’ functionality. The combination of design and validation techniques from computer science and automation technology, such as multi-agent-based simulations or rapid prototyping strategies, will enable a more structured approach for developing decentralized energy management systems in a unified manner.