Ontology-Based Semantic Multi-agent Framework for Micro-grids with Cyber Physical Concept

XU Shaolun（许少伦），FENG Donghan（冯冬涵）＊，SUN Tao（孙 弢），BAI Rongjing（白荣静），ZHAO Jingbo（赵静波），LEI Jinyong（雷金勇）

1 Key Laboratory of Control of Power Transmission and Conversion，Ministry of Education，Department of Electrical Engineering，Shanghai Jiao Tong University，Shanghai 200240，China

2 Electric Power Research Institute，Jiangsu Electric Power Company，Nanjing 210036，China

3 Electric Power Research Institute，China Southern Power Grid，Guangzhou510080，China

Introduction

With the increasing of environmental pressure caused by limited global resources，the public pays more attention to the environmental protection，energy conservation and sustainable development.The proportion of distributed power is increasing quickly in the whole power grid.A micro-grid，as a distribution subsystem，which is composed of the distributed power，load and battery storage，can blend with the power grid efficiently and has become an important direction for future development［1-2］.

In electric power systems，the current approach of using a central supervisory control and data acquisition（SCADA）system is no longer suitable for certain control operation.Therefore，current trends of controlling and monitoring the operation of electric power system are moving toward to the use of an automated agent technology，which is generally known as the multi-agent system （MAS）.The MAS can provide intelligent，fast，adaptable local decision making for flexible control of the modern power system，especially in micro-grid［3-4］.

By connecting computing and communication to physical processes and mediating interactions with the physical world，cyber physical system （CPS）is expected to play a major role in the design and development of future engineering systems with new capabilities.CPS has special characteristics of adaptability， autonomy， efficiency，functionality，reliability，safety and cyber security with higher level and is appropriate for the distributed，hierarchical and heterogeneous system［5-6］.

This paper aims to apply the CPS concept to the hybrid micro-grid control system based on MAS technology in order to propose a new control framework and a real-time simulation model of micro-grid.

1 Overview of Micro-grid System

1.1 Structure of micro-grid system

A micro-grid refers to the power system comprised by the low voltage distribution network and power generation assemblies with small modules and interconnected loads.It can be viewed as a small power grid based on distributed hybrid generation and the management of supply and demand［7］.A micro-grid generally uses the hybrid power generation methods which include the renewable energy generators.It is a classic hybrid system which means that the system contains discrete-event system， continuous-time system and the interactions between them.Its classic structure is shown in Fig.1.

Fig.1 Classic structure of micro-grid

The micro-grid in Fig.1 consists of three parallel distributed generators （DGs），the other network，the switch，the battery storage and some loads of different levels.The DGs mainly include the different types of distributed energy resources （DERs）such as the wind turbines （WTs），photovoltaics （PVs），combined cool heating and power（CCHP），and fuel cells（FCs）.

1.2 Operating mode and control method of micro-grid system

The micro-grid has two kinds of operating modes.One is the island operating mode and the other is the gridconnected mode.The island operating mode refers to the isolated micro-grid power that does not exchange the power flow with other power grids.The grid-connected mode refers to that it exchanges the power flow with other power grids.When the micro-grid operating in grid-connected mode，the micro-grid can not only accept the power flow from the other，but also output the power flow to other grids.The power flow direction depends on generating capacity and load capacity of micro-grid［8］.

In order to ensure the reliability and the economy of the micro-grid，micro-grid has to switch the mode between the grid-connected mode and the island operating mode.It has the ability to select a certain mode between the two modes intelligently in different situations to make sure a smooth transition when switching the two operating modes of microgrid［9］.

The existing micro-grid control methods are hierarchical control method，fully distributed control method and hybrid operation control method.The hybrid operation control method reduces the dependence that each controller relying on the central control unit and is more flexible to realize the local control and aggregation control.According to the difference of the function distribution and information processing mode，we can choose the appropriate control method.

2 Cyber Physical Control Concept Based on Ontology Semantic Agent

2.1 Hybrid system and networked CPSs

In a hybrid system，physical processes are modeled mathematically as dynamical systems and they interact with computation performed by various automata.The interaction is mostly in the form of control，which is performed by an automation that controls the evolutions of several dynamical systems in the typical situation.The evolution of every dynamical system is depicted as an open set in the Euclidean space called mode.A hybrid automation consists of a discrete controller that switches between different continuous dynamical systems via some control laws modeled as guards.The controller is represented as guarded transitions between modes.

CPSs are systems operating in physical environments and display rich interactions between physics and computation.These systems generalize the embedded and hybrid systems by allowing more flexible interactions between computing components and their physical environments.CPSs bring together the discrete and powerful logic of computing to monitor and control the continuous dynamics of physical and engineered systems.Therefore hybrid systems are prominent mathematical models for those CPSs that integrate discrete and continuous dynamics.The discrete dynamics typically comes from the discrete control decisions and digital controllers.The continuous dynamics comes from the physical process elements of the CPS［10］.In the study of cyber physical computing，hybrid systems seem to be a very versatile tool.

However，the hybrid systems do not apply to all aspects of CPS.For instance，distribution effects are neglected in hybrid systems models.Multi-agent hybrid systems which have multiple hybrid agents coevolved are not included in hybrid system models.More generally， these systems form distributed hybrid systems with discrete dynamics，continuous dynamics，and distributed dynamics［11］.We can use the ontology semantic to abstract the guards and transitions to realize the more flexible interactions between discrete and continuous dynamics.

2.2 Ontology semantic agent framework

As a modeling tool of conceptual model at the semantic knowledge level information system，ontology attracts the attention of many researchers and has been widely applied in many areas of the computer.

Ontologies describe entities and relations between them，and semantic interpretation is carried out on semantic streams，each of which is defined in a domain-specific ontology associated with the agent.Ontology-based interoperation provides a solution in environments with heterogeneous semantics.Agents use the ontology for the passing of information，formulating questions and requesting the execution of actions related to their specific domain.Based on the ontology，agents can share their common knowledge and work over common system problems because sharing knowledge is the natural way for single pieces of multi-agent software system to achieve their goals［12］.Figure 2illustrates how the agents use the information collected by sensors and the interpreted semantics based on the defined ontology.

Fig.2 Ontology semantic agent framework

Raw data is obtained from sensor networks，and since each agent is an independent entity in charge of a particular geographical region，sensors located in distributed areas are managed by different agents.The hybrid system can deal with its particular proprieties when event-driven part（hybrid automaton subsystem） and time-driven part（continuous state equations subsystem）can be corrected separately without disturbing the whole system.The ontological abstraction allows a sensing system to optimize its resource utilization in collecting，storing，and processing data.The benefit of abstracting services as semantic conversions on a common ontology includes the possibility of automatically composing services to satisfy a semantic query and the possibility of adapting to resource changes by switching to different services which produce the same output semantic streams.

A service component is an information-rich component used to define the semantic service for execution，extract information from the input，and produce new semantics at the output.For a semantic service component，the input semantic events are preconditions of the service.The processed output semantics are provided to the agents for further computing［13］.

3 MAS Architecture of Micro-grid with CPS Concept

3.1 Micro-grid structure with CPS concept

MAS can be viewed as a collection of autonomous and intelligent entities called agents，evolving in an environment where they can perceive and act on.MAS can be used for cyber physical micro-grid system because they have interesting characteristics such as flexibility，plugging ＆playing，fault-tolerance and suitability for distributed problems.

Figure 3shows the abstraction structure of the cyber physical micro-grid system.It can be abstracted into three layers including the physical layer，the network layer and the cyber layer which can also be called the agent layer.The physical layer represents the actual physical process including WT，PV，CCHP，FC，electric vehicle （EV），battery storage，load，transformer，circuit breaker，switch，etc.The network layer ensures the communication of the components.The cyber layer adopts the multi-agent architecture to specify the map relationship of the entities in the physical layer.In order to embody the fusion between the physical layer and the cyber layer，there are mapping relationships between the layers.That is to say，all the entities in the physical layer have their corresponding counterparts in the cyber layer.The fusion between the physical layer and the cyber layer is regarded as the entities in physical space mapping from the physical space into the counterparts in cyber space，interacting between each other and also with other entities that only belongs to cyber space inside the cyber space［14］.

Fig.3 Cyber physical micro-grid system structure

The cyber layer can be divided into the implement layer，the control layer and the distribution management layer according to the micro-grid operation status in detail.

3.2 Implement layer specification

The implement layer includes the different direct mapping agents of the physical devices such as WTA，PVA，BA，LA，TRA，CBA and SWA to realize the local control of the distributed generators，loads and the cooperate control in the micro-grid cell.

（1）DER agent.A DER agent is responsible for storing associated DER information，as well as monitoring，controlling DER power levels and the connect／disconnect status.The information to be stored may include DER identification number（owner name，address，etc.），type（WT，PVs，CCHP，FCs，etc.），power rating，local fuel availability as well as cost function or price at which users agree to sell.This agent can communicate with other agents to decide whether or not to connect the generator to the grid.

（2） Device agent.This agent is responsible for controlling the electronic devices such as circuit breakers and switches.For example when a device agent senses a failure in the electric power system from the abnormal status of a device，the device agent will communicate with other agents to take some safety precautions.

（3）Load agent.This agent is located at the user-side.And it can determine whether or not to shed the load corresponding to the state of the grid.

3.3 Control layer specification

The control layer mainly coordinates the different components and thus can also be considered as the control center of a micro-grid cell.

（1）User agent.A user agent is the gateway to connect a user to the rest of the system.It is responsible for retrieving real-time information and displaying all relevant information on an electronic console at the end-user's premises，and allows users to control the status of loads based on the priority predefined by the user.This information may include price，electricity consumption of each critical and non-critical load and duration（clock time）of electricity available for purchase，etc.

（2）Aggregation agent.An aggregation agent receives bid information and dispatch information from the bulletin board and will forward this information to the right DER agents.By accumulating available supply，it makes it much easier for end-user to purchase electricity.

（3）Bulletin board agent.A bulletin board agent is a special agent that represents a dynamic contact point through which user agents and the aggregation agent share and retrieve bid information.

（4）Database agent.A database agent is responsible for collecting and storing electricity price，demand information as well as the messages and data shared among agents in a database，and providing access to the database for other agents.Database agent also serves as a data access for other agents，as well as users.

（5）Control agent.A control agent contains software to perform generator synchronization， maintain operating frequency and voltage，sustain suitable real and reactive power，loading and unloading，and others.It puts forth responsibilities that include monitoring system voltage and frequency to detect contingency situations or grid failures，receiving electricity price signal from advanced metering infrastructure and conveying them to the rest of entities，and sending signals to the main circuit breaker to isolate the micro-grid from the utility when an upstream outage is detected.It can be viewed as the brain of a separated microgrid cell.

3.4 Distribution management layer specification

The distribution management layer can control multiple micro-grids on thedistribution network level.For example，the output voltage of micro-grid should track the grid voltage automatically to realize the cooperative control between the distribution management system and the micro-grid system.

3.5 MAS-based intelligent electronic device（IED）

IED is the important unit of micro-grid control.Generally speaking，every control unit has an IED such as the switch，circuit breaker and other DG controller.The function model of IED uses the CPS and agent technology.The classic function structure is shown in Fig.4.

Fig.4 MAS-based IED function structure

The IED based agent technology is similar with the traditional IED on the hardware，also including the input module（voltage and current acquisition，circuit breaker state acquisition）， output module （circuit breaker action command），communication module （communicating with other IED and central unit） and functional module（computing and data processing）.The main difference lies in the task of functional module.For example，the IED at the grid-connected switch needs to get more information than other IEDs because the grid-connected switch is the only way for micro-grid to connect the main power grid.The IED at the grid-connected switch can get the status information of the main power grid and micro-grid to complete the synchronization and disconnection functions.

4 Integrated Cyber Physical Simulation Framework

In order to operationalize and test the proposed cyber physical micro-grid concept，we also proposed an integrated simulation model including three key parts—physical part（resides in computer 1），cyber part（resides in computer 2）and middleware （resides in computer 1）.They can be accurately represented by different platforms respectively.The simulation framework is shown in Fig.5.

The cyber MAS is connected to the simulated physical micro-grid via TCP／IP connections.According to this setup，the micro-grid hardware simulation in Matlab can be controlled by the MAS from any remote location.Real-time measurements and control signals are exchanged between these two computers as follows.

Real-time measurements from computer 1，including voltages，currents，status of circuit breakers，power produced by the distributed generator and power consumption by critical and non-critical loads etc.，are sensed and monitored by the cyber MAS in computer 2via TCP／IP connection.The cyber MAS in computer 2issues real-time control signals to control the loads connected through the IP-enabled circuit breakers，as well as the power output from the distributed generator within the simulated micro-grid in computer 1，also via TCP／IP connection.

Fig.5 Cyber physical micro-grid simulation framework

4.1 Physical part—circuit components model

The physical part is related to modeling the hardware elements of a micro-grid.The SimPowerSystems toolbox of Matlab／Simulink platform has rich power elements and can be used to construct the hardware system quickly.For example，the toolbox can be used to create Simulink blocks representing devices （e.g.， switch gears， power conditioners，transformers，circuit breakers and relays），distributed energy sources（e.g.，WTs，PVs，CCHP，FCs and battery storage）and loads.

4.2 Cyber part—MAS model

The cyber part is the coding of agent functionalities to control the operation of DG units and the electricity transactions.To implement an MAS，there are a number of open-source agent platforms available in the literature that aid developers to build a complex agent system in a simplified fashion.It is very important to select an agent platform based on the IEEE standard on Foundation for Intelligent Physical Agents（FIPA），which is a well-known standard.

Java Agent Development Framework （JADE）is a software framework to make the development of multi-agent applications in compliance with the FIPA specifications easier.JADE agent platform tries to keep high the performance of a distributed agent system implemented with the Java language.It includes an agent live operating environment and some predefined agent models and tools to ease the development of systems［15］.It is very appropriate for the micro-grid MAS system construction and the cyber agent functionalities development.For example，the control agent can control the system voltage and frequency，perform generator synchronization and calculate real／reactive power flow；the DG agents can accept offers from DG owners（this agent can be given an objective to maximize the profit which is defined as the difference between revenue and cost）and forward the metadata to aggregation agent and control power output of DGs.

4.3 Middleware—TCP／IP connections between cyber part and physical part

The TCP／IP server is a middleware implemented in Matlab／Simulink to establish TCP／IP connection between the micro-grid in the simulated environment and the MAS residing in the other computer.The TCP／IP server is capable of conveying real-time control signals and forwarding unlimited number of real-time circuit measurements from and to the external MAS respectively［16］.So the middle server can help the multiple agents（i.e.，device agent，DER agent，load agent，control agent，user agent and other agents）to connect to the simulated circuit simultaneously.

5 Conclusions

This paper presents the cyber physical micro-grid system concept and proposes an abstract architecture based on the ontology and multi-agent technology.The specifications of each layer and the relationships of the heterogeneous components are illustrated in detail which uses the cyber physical control concept to realize the more flexible control.And an integrated cyber physical simulation framework is proposed to realize this idea.The modeling work presented in this paper is a preliminary step that will accomplish the broader goal of modeling cyber physical micro-grid system.Future extensions to this work will incorporate sophisticated decision support algorithms for the agents to realize the complex system control simulation.

［1］Lemmon M D，Venkataramanan G，Chapman P.Using Microgrids as a Path towards Smart Grids［C］.Joint National Workshop on New Research Directions for Future Cyber-Physical Energy Systems，Baltimore，Maryland，2009.

［2］Farhangi H.The Path of the Smart Grid［J］.IEEE Power and Energy Magazine，2010，8（1）：18-28.

［3］Stephen D J，Euan M，Victoria M，et al.Multi-agent Systems for Power Engineering Applications—Part I： Concepts，Approaches， and Technical Challenges ［J］.IEEE Transactions on Power Systems，2007，22（4）：1743-1752.

［4］Stephen D J，Euan M，Victoria M，et al.Catterson.Multiagent Systems for Power Engineering Applications—Part II：Technologies，Standards，and Tools for Building Multi-agent Systems［J］.IEEE Transactions on Power Systems，2007，22（4）：1753-1759.

［5］Bujorianu M，Barringer H.A Unifying Specification Logic for Cyber-Physical Systems［C］.17th Mediterranean Conference on Control and Automation，Thessaloniki，Greece，2009：1166-1171.

［6］Shi J H，Wan J F，Yan H H，et al.A Survey of Cyber-Physical Systems［C］.2011 International Conference on Wireless Communications and Signal Processing，Nanjing，China，2011：1-6.

［7］Kanellos F D，Tsouchnikas A I，Hatziargyriou N D.Microgrid Simulation during Grid-Connected and Island Models of Operation［C］.International Conference on Power Systems Transients，Monteral，Canada，2005：105-113.

［8］Katiraei F，Iaravani M R.Power Management Strategies for a Microgrid with Multiple Distributed Generation Units［J］.IEEE Transactions on Power Systems，2006，21（4）：1821-1831.

［9］Zhou X S，Gu Y，Ma Y J，et al.Hybrid Operation Control Method for Micro-grid Based on MAS［C］.IEEE International Conference on Progress in Informatics and Computing，Shanghai，China，2010：72-75.

［10］Henzinger T A.The Theory of Hybrid Automata［C］.The 11th Annual IEEE Symposium on Logic in Computer Science，Berlin，Germany，1996：265-292.

［11］Platzer A.Quantified Differential Dynamic Logic for Distributed Hybrid Systems［J］.Computer Science Logic，2010，6247：469-483.

［12］Toma I F.Contributions to the Study of Semantic Interoperability in Multi-agent Environments—an Ontology Based Approach［J］.International Journal of Computers Communications ＆Control，2010，2015（5）：946-952.

［13］Lin J，Sedigh S，Miller A.Modeling Cyber-physical Systems with Semantic Agents［C］.IEEE 34th Annual Computer Software and Applications Conference Workshops（COMPSACW 2010），Seoul，Korea，2010：13-18.

［14］Suo Y，Shi Y C.Towards Initiative Smart Space Model［C］.Third International Conference on Pervasive Computing and Applications（ICPCA 2008）， Alexandria， Egypt，2008：747-752.

［15］Bellifemine F，Poggi A，Rimassa G.Intelligent Agents VII Agent Theories Architectures and Languages［M］.Berlin Heidelberg：Springer，2001：89-103.

［16］Pipattanasomporn M，Feroze H，Rahman S.Multi-agent Systems in a Distributed Smart Grid： Design and Implementation ［C］.2009 IEEE／PES Power Systems Conference and Exposition（PSCE 2009），Seattle，America，2009：1-8.