Research progress of artificial psychology and artificial emotion in China

Zhiliang Wang*,Lun Xie,Ting Lu

School of Computer and Communication Engineering,University of Science and Technology Beijing,30 Xueyuan Road,Beijing 100083,PR China

Research progress of artificial psychology and artificial emotion in China

Zhiliang Wang*,Lun Xie,Ting Lu

School of Computer and Communication Engineering,University of Science and Technology Beijing,30 Xueyuan Road,Beijing 100083,PR China

Since the concept of artificial psychology and artificial emotion was first presented,it has become a topic of interest in academic circles and enterprises.In this article,we first briefly introduce the basic concepts and principles of artificial psychology and artificial emotion,analyzing the unified macro-model of the cross-disciplinary system architecture against the need-motivation-behavior framework.Second,we discuss the origin of artificial psychology and artificial emotion,its course of development,and its present situation in China.We also present a review of the published papers and research endeavors of Chinese universities and research institutions and the technical engineering applications of artificial psychology and artificial emotion.Finally,we summarize the challenges to the further development of artificial psychology and artificial emotion and our recommendations for improving the cognitive computing model of psychological states and developing reliable and accurate humanoid interaction and cooperation technology,robot platforms with emotions and humanoid interaction and cooperation capabilities,and humanoid robots for the elderly and the disabled in smart homes.We believe that,with intensive research,artificial psychology and artificial emotion may be developed further and may eventually reach maturity.

Artificial psychology;Artificial emotion;Intelligent interaction;Psychological assistive technology;Aging service

1.Introduction

In the field of information science,researchers have regarded the imitation of the human brain,human intelligence, and human behavior as important research directions and contents.A review of the developmental history of automation science and technology shows that more people use brain science,psychology,neuroscience or cybernetics(personification control),and theory of controlled source in studying control strategies and algorithms.Along this line of thought, we also draw from psychology and other related fields to explore,study,and attempt to develop a new intellectual discipline and a new field of information science.After several decades of research,artificial intelligence has reached a high level of personification.However,the objective of existing studies is to simulate human intelligence,including such thinking activities as judgment,reasoning,proving,identification,perception,understanding,design,thinking,planning, learning,and problem solving[1].The content of artificial intelligence research focuses on how to express,acquire,and use knowledge.Artificial intelligence,however,is only the preliminary stage in the field of personification research because human psychological activities include feeling, perception,memory,thinking,emotion,will,character,and creation.Among these activities,only feeling,perception, memory,and thinking are encompassed by artificial intelligence;thus,its scope is limited.Therefore,to study artificial psychology and artificial emotion and develop cutting-edge technology and industrial applications in these fields,artificial intelligence research(research results and research methods)must be used as theoretical basis;new theories and methods in psychology,brain science,neuroscience,information science,computer science,and automation science should be integrated;and psychological activities,particularly emotion,will,character,and creation,should be simulated[2].

2.System architecture of artificial psychology and artificial emotion

Artificial psychology and artificial emotion is a discipline that uses information science to achieve a more comprehensive coverage of human psychological activity with an artificial machine(e.g.,computer and algorithmic models).The application prospect of artificial psychology and artificial emotion is extensive,such as developing robots with emotion, consciousness,and intelligence;designing humanoid robots with real significance;and extending control theory to the control mode of the human brain.Currently,personification controltheory is mainly a combination of Wiener's“feedback”control theory and artificial intelligence.The existing studies on controland decision systems do not consider the emotional factors;by contrast,the control mode of the human brain includes feeling,perception,and emotional decision behavior.

Artificial psychology and artificial emotion as a discipline is the advanced stage of artificial intelligence,a new research field in automation and information science.Research on these concepts could significantly promote the progress of personification control theory,emotionalrobots,personalized product design,and market development and ultimately create a social environment where people and people,people and machine coexist harmoniously.Artificial psychology and artificial emotion is cross-disciplinary,as shown in Fig.1.Its theoretical foundation is based on brain science,psychology,physiology, moral philosophy,neuroscience,human engineering,Kansei engineering,linguistics,aesthetics,law,information science, computer science,automation science,and artificial intelligence.Its application scope mainly includes emotional robot technology support,humanoid machine,humanized product design,emotional market development,artificial psychology programming language,artificial creation technology,human emotion evaluation computer systems(virtual technology), human psychological database and mathematical model, interpersonal harmony environment technology,and human--machine harmony multichannel interface.

Nearly all human mental activities are associated with emotion,that is,emotion is associated with motivation, emotion with cognition,emotion with behavior,emotion with decision,emotion with evaluation,emotion with learning, emotion with character,emotion with memory,emotion with awareness,emotion with intelligence,emotion with desire, and emotion with reason.Thus,nearly all human psychological activities are subject to emotional influences and excitation.People are commonly known as emotional animals;thus, the core of artificial psychology and artificial emotion should be the study of the emotional stimulation of certain mental activities and the machine implementation issues between emotions and mental activities.In the past,emotions are usually regarded as affiliated mental processes or interference factors in human activities.At present,this viewpoint has changed considerably in the field of psychology,mostpsychologists believe that emotion has an internal,social behavior-adjusting function;it is an important excitation and organizer of cognition,action,social interaction,and development.This premise can serve as theoretical basis for the research on artificial psychology and artificial emotion.Selfconsciousness is the significance of the mental activities of a human being.Therefore,it should be one of the core ideas of artificial psychology and artificial emotion.

Fig.1.Cross-disciplinary framework of artificial psychology and artificial emotion.

The study of artificial psychology and artificial emotion has certain guiding principles to which adherence must be constant.The first guiding principle is self-adjustment:machine implementation should be completely aware of the human self, introspection,and consciousness.The second guiding principle is positive emotion:human feelings of pleasure should be achieved in the realization of machines.The third guiding principle is collaborative work:a mental activity is a product of the parts of the human brain working together;thus,complex and large system theory is a powerful tool for the study of artificial psychology and artificial emotion.The last guiding principle is energy balance:mental activities function from the inner power of human mental energy.

The objective of this research is to establish a unified model of artificial psychology and artificial emotion.This model can describe to a certain extent all human mental activities(intelligence,imagination,learning,memory,attention,consciousness,emotion,and so on)or describe some small models derived from big models.These small models can describe one of the aforementioned psychological activities or the cognition-emotion and motivation-emotion-decision relationships.Such models can function as synergistic,parallel, stratification,and decision models[3].

The overall goal of this study is to present small models (short-term goals)and big models(long-term goals),which when combined can serve as building blocks in understanding artificial psychology and artificial emotion.We use the theoretical knowledge of multiple disciplines,such as psychology, brain science,neuroscience,endocrinology,physiology,control theory,knowledge discovery,artificial intelligence, mathematics,complex system theory,nonlinear theory,system engineering,data structure,anthropology,and behavioral science in the application of the unified model.We initially propose a unified model of artificial psychology and artificial emotion(Fig.2).

This model can describe a person's will process,cognitive process,emotional process,and personality psychology,with the basic knowledge of brain science as basis and emotional arousal as the basic point.Its characteristic lies in the following premises:a small model obeys the unified model; modular models can be unified into a unified big model;a unified model can describe allhuman psychological activities; small models can describe a specific psychological activity individually;and the focus of the small model is the controlling process,whereas the focus of big model is the collaborative decision-making process of the small models.The relationship between the small models is collaborative,parallel,time shared,and coupling.In the modeling process,we should use computer knowledge and system structure as the technical methods and employ electronic circuit knowledge, system theory,control theory,and information theory as the basic methods.

Using Siminoff's cognitive theory,we further propose the need-motivation-behavior structural framework for artificial psychology and artificial emotion and study the generation mechanism of machine emotion against this framework.We use the concept of control system to discuss the implementation method for machine emotion generation.We intend to study emotional calculation by using the mental structural framework and considering the role and position of the proposed affective computing method in the anthropomorphic thinking process.We can then verify the correctness of the proposed need-motivation-behavior structural framework,the affective computing model,and the emotional transfer method,as shown in Fig.3.

The mechanism of machine emotion generation is described as follows:the sensors receive external stimulation; classify,analyze,and quantify the stimulus/stimuli;and then send the difference between the perceptron output and the demand to the mental model.The mental model and memory and learning constitute the machine's process of thinking and comparing,including the emotional calculation model and the intelligent reasoning model.In this process,the satisfaction degree of the machine desire and other corresponding emotions and behaviors are mainly analyzed.In a timely and active manner,the machine's needs are adjusted,and the mood and behavior controller regulates the performance of the machine's emotion and behavior.

In case of a stress response from the machine,the perceptrons obtain,classify,and then quantify the external stimulus/stimuli.Given that the priority level of the stimulus/ stimuli is high,the emotions and behavior produced by the mind model machine are sent directly to the mood and behavior controller to regulate the external performance of the machine's emotions and behaviors.The modules are interconnected but independent.The perceptrons receive,classify, and then analyze the stimuli;the demand is produced according to the machine's needs;the system and feedback is the thinking process of the machine,including the affective computing model and the intelligent reasoning model;the mood and behavior controller responds according to the satisfaction of the machine's desires and needs.However,some issues have to be addressed first.These issues include the establishment of a machine demand generation model,the quantification and classification of the external stimulus/ stimuli,the structure of the mental model and the state transfer of emotion,and the corresponding relationship between the satisfaction of the machine's needs and other emotions and behaviors.We hope to use mathematics and computer tools to address these issues with the use of emotional psychology as basis.

With the continuous development in affective computing and intelligent interaction,the requirements of people for machine intelligence have been increasing.A growing number of researchers have paid attention to artificial psychology and artificial emotion.In the future,an intelligent,sensitive,andfriendly personal computing system with capabilities to perceive,recognize,and solve human emotions can be designed,the gap between humans and computers can be narrowed,and a truly harmonious man-machine environment can be created.This goal may be achieved by using the existing artificial intelligence literature(research results and research method)[4];combining psychology,brain science, neuroscience,information science,computer science,automation science,and new theories and methods;simulating human psychological activities,particularly emotion,will,character,and creation;acquiring behavior and character signals caused by emotions;establishing an“emotion model.”

Fig.2.Unified model of artificial psychology and artificial emotion.

Fig.3.General structural model of machine emotion generation.

3.History and current situation of artificial psychology and artificial emotion

The research contents of artificial psychology and artificial emotional are similar to those of affective computing and Kansei engineering.Research on affective computing and Kansei engineering began in the 1990s.Kansei engineering is the technology of combining emotion and engineering.People's sensibility to the design and manufacture of products by analyzing people's feelings based on perceptual science.Itis a product design and technology science that aims to provide joy and satisfaction to people.Given that Kansei engineering can provide happiness and comfort,it is touted as a“happy and comfortable”science.The goods produced with Kansei engineering technology are called“Kansei commodities.”Japan has paved the way for the climax of the national Kansei engineering research.In 1996,by national funding,the Japanese Ministry of Education began to support“emotional information and psychology”as a major research topic.More than a dozen universities and research units participated in the project,whose main purpose is to study emotional information from the psychological perspective to the cross-disciplinary perspective of psychology,information science,and other related disciplines[5].A national conference on Japanese Kanseiengineering is held every year.Atthe same time,Japan made considerable progress in the industrialization of Kansei engineering.Japan's major companies race for the development,research,and production of personal robot product lines. Sony's Aibo robotic dog(6 million units have been produced, and USD 1 billion profithas been earned)and QRIO and SDR-4X emotional robots are typical representations of these personal robots.Clearly,Japan has led the world in artificial emotion technology.

Artificial psychology and artificial emotion has received widespread attention among academic circles and enterprises. Well-known international companies,such as IBM and British Telecom,established a special affective computing research group.Internationally renowned universities have likewise formed such a research group.Overall,affective computing research remains at the initial stage.At present,the Massachusetts Institute of Technology(MIT)focuses on the acquisition of emotional signals(by developing various types of sensors),the application of affective computing,and the development of wearable computers.Japanese scholars focus on Kansei information processing.In the affective computing research,the research of wearable computers has achieved rapid development in recent years.These computers can seamlessly become a part of daily wear(such as glasses, watches,necklaces,bracelets,belts,shoes,hats,and clothing), thereby enabling them to be inseparable from their wearers. Wearable computers can provide a good research platform in affective computing research,the ultimate form of affective computing may be a wearable computer or an implanted chip in the body.Thus,affective computing,mobile computing,and computer technology can be integrated for a truly personalized human-computer fusion.Although affective computing is a new subject,previous studies also provided cumulative knowledge.In terms of emotion,the studies on human psychology and physiology have provided a solid foundation for emotional calculation.In addition,studies on face and facial expression recognition,face and facial expression synthesis, sign language recognition and synthesis,and odor recognition have been topics of interest among academic circles.The development of different types of medical apparatus and instruments,such as a polygraph,can provide a certain basis for emotional calculation[6].

EU countries also actively research on emotional information processing technologies,such as facial expression recognition,measurement of the emotional information, wearable computers,and so on.Many European universities established research groups to study the relationship between emotion and intelligence.Among these research groups are the Emotional Research Laboratory of the University of Geneva led by Klaus Scherer,the emotional robot research group of the Free University of Brussels led by D.Canamero,and the Cognition and Affect Projectof the University of Birmingham, UK led by A.Sloman.Professor Sloman proposed the emotional three-layer architecture called CogAff agent architecture(cognitive affective system)to investigate the interaction between emotion and cognition.In terms of market applications,in Germany,Mehrdad Jalali-Soli proposed a multimodal shopping assistant based on EMBASSI in 2001. EMBASSI was funded by the German Ministry of Education and Research(BMBF),in which more than 20 universities and companies were involved.EMBASSI is a network electronic commerce system that considers the psychology of a consumer and the demand of the environment the research targets.For the EU countries'Humane ID plan,psychologists and information science experts from the United Kingdom, Switzerland,Germany,France,and Italy assembled to study the theoretical methods and technical problems related to artificial emotions.

In the computer science field,many international conferences on artificial emotion have been held with the goal of establishing harmonious human-computer interaction.Some famous conferences include the Affective Interactions Symposium in the 2005 International Conference on Intelligent User Interfaces;the biannual FGR of the IEEE Face Information Processing International Conference;Affective Dialogue Systems symposium in 2004;HCI 2004 Harmonious Human-Computer Interface Interaction Technology symposium;Emotionale 2000:Third International Design Conference;SAB2000 Sixth International Conference on Autonomous Agents;International Conference on Social Computing 2001;International Conference on Affective Human Factors Design 2001;Third International Conference on Design and Emotion 2002;and Design and Emotion 2004.

Importantly,the field of artificial intelligence is actively participating in artificial psychology and artificial emotion activities.Professor Marvin Minsky of MIT is one of the founders of artificial intelligence.In 1987,he first proposedthat emotional capability should be incorporated into computers in the monographThe Society of Mindand then published the monographEmotion Machine[7],which is about artificial emotion.Since then,lending emotional capability to computers and rendering the computer capable of understanding and expressing emotions have captured the interest of the computer community.In 1995,Professor R.Picard of MIT Media Lab proposed the concept of affective computing and published the monographAffective Computingin 1997[8].In her book,she said,“affective computing is a kind of computing which is associated with emotions,comes from emotions or can exert their influence on emotions.”The American Association for Artificial Intelligence(AAAI) organized artificial emotion seminars entitled Emotional and Intelligent(mood and the function body):The Tangled Knotof Cognition(AAAI Fall Symposium 1998),Psychological Models of Communication in Collaborative Systems(AAAI Fall Symposium 1999),Architectures for Modeling Emotion: Cross-Disciplinary Foundations(AAAI Spring Symposium 2004),and Exploring Attitude and Affect in Text:Theories and Applications(AAAI Spring Symposium 2004).The German Research Center for Artificial Intelligence(DAIR)is actively seeking international cooperation for the research and application of artificial emotion,such as introducing an emotion model and personality characteristics into three schemes from virtual theaters,virtual markets,and dialogue agent to enhance children's imagination and creativity,and introducing an artificial emotion model for e-learning systems to improve teaching efficiency[9].

The research department of China's Ministry of Science and Technology has also provided extensive support and guidance for the research on artificial emotion.Early in 1998,the National Natural Science Fund Committee listed“emotional calculation theory research for a harmonious human-machine environment”as the sixth theme of the information technology exploration of the National Natural Science Fund Project.In 2004,affective computing theory and method research was listed as the focus fund project.In 2005,pervasive computing was listed as the focus fund project.The office of the state 973 Program listed“research on harmonious human-computer interaction theory”as the key support direction for the last three years of“The Tenth Five-Year Plan.”

Research on artificial emotion in China began in the 1990s. A majority of the studies aimed at realizing artificial emotion unit theory and technology.Southwest University mainly studies the emotional information contained in different physiological signals[10].A physiological signal is induced in a laboratory,with a certain number of biosensors attached to the subjects.With a Kinect camera,physiological signals can be collected,and the emotional states of the subjects can be processed and analyzed.Currently,Southwest University is researching emotion analysis using skin signals,ECG signals, EMG signals,and respiration signals.The Institute of Software of the Chinese Academy of Sciences utilized a cloud computing platform and big data to study the influence of user's personalized cognitive and individual behavior.Psychological cognitive theory was integrated into the model to analyze the evaluation and optimization of the personalized features on the interactive intent of the users.Consequently, human-computer interaction could be enhanced in real time, the user experience could be improved,and human-computer interaction technology could be integrated into people's daily life and work as much as possible.The Institute has also conducted studies on the multichannel fusion of cognitive interaction[11].The Digital Media and Interpersonal Interaction Research Center of Hangzhou Normal University focuses on soma to sensory interaction,wearable devices,and virtual reality.A depth-sensing camera,such as Kinect,has been used to reconstruct a three-dimensional model of the human body,recognize gestures,track fingers,and identify the wrist and palm of the hand for a better human-computer interaction.The Research Center has also developed a virtual bodybuilding system and virtual dressing system.The Department of Computer Science and Technology of Tsinghua University specializes on emotional expression in speech, particularly the influences rhythm of speech,the tone of voice, and the semantic focus on emotional expression.The Department uses this technology to analyze the influences of the acoustic properties of the speech in communications.It has also conducted in-depth studies on voice conversion,speech synthesis,and speech visualization.The Institute of Automation of the Chinese Academy of Sciences mainly studies speech enhancement,speech synthesis,and the acoustic features for emotional speech recognition,including word stress and interrogative intonation.It has also conducted research on multimodal interaction in man-machine dialogue models [12].The Harbin Industrial University focuses on a multi perception machine,which combines artificial intelligence and parallel processing.It consists of an intelligent agent technology,combined digital and analog computing technology,parallel computing technology,real-time processing technology,voice recognition,expression recognition,face recognition,face detection and tracking,character recognition, sign language recognition,sign language synthesis,and expression synthesis of natural language technology.The University constructed a software and hardware platform that can research and develop human language,including vision and hearing.Beijing Industrial University conducts ongoing research on the integration of affective computing into a multi perception machine.The Computing Technology Research Institute of the Chinese Academy of Sciences is working on a virtual human with facial expression and body movement capabilities.The Institute of Psychology and Biology of the Chinese Academy of Sciences focuses on the relationship of emotional psychology and physiology.Chongqing University works on intelligence services,augmented reality,environment sensing,and smart watches,and the development of appropriate software.The Learning Science Research Center of the Southeast University pursues children's facial expressions and emotion speech information analysis,emotion and cognition,virtual reality and brain science visualization technology,and cognitive and biological information science as its major research directions.The University of Science and Technology of China researches on interactive emotionalimage retrieval based on content.The Beijing University of Science and Technology has long been engaged in the study of intelligent human-computer interaction and affective computing,ubiquitous sensing in service robots,and collaborative optimization,as well as in the research of the basic theories and application of the new pension scheme for the elders and disabled in a networking environment.The Beijing University of Science and Technology obtained valuable results from the studies on the organic integration of cognitive analysis and affective computing,self-controlled computing in a continuous state space,the topological structure of a nonlinear functional network under an infinite-dimensional continuous state,and nonlinear mapping and collaborative optimization theory of a heterogeneous multidimensional space.It used household companion robots as new and effective tools in the field of geriatric mental health services. The robots can help elders improve their cognitive psychological state,maintain their social interaction and communication ability,attain a perfect psychological(cognitive, emotional,self,interpersonal,and adaptation)state,retain short-term memory and associative memory,and ultimately achieve the goal of healthy aging;as a result,the robots can enable elders to have long-term capacity for independent living.The University combined advanced information science,such as controlling,artificial intelligence,and pattern recognition,with adjuvant therapy in developing a remote social adjuvant therapy robot for children with autism.The robot can improve the coordination and synchronous interaction abilities of children with autism and in develop their social interaction and communication skills.It can help children with autism to communicate naturally and harmoniously with other healthy people[13].

Clearly,universities and institutions in China have provided new and effective tools for children education and rehabilitation.They have also achieved the collaborative optimization of robotics and intelligent home environment,regulated the range of users' cognitive psychological activities based on the smart environment,and promoted the emotional communication between the user and the smart home environment(Table 1).

Artificial emotion researchers in China have conducted considerable research and academic activities.In December 2003,Beijing held the first Chinese affective computing and intelligent interaction conference.In October 2005,the China Association for Artificial Intelligence,Artificial Psychology, and Artificial Emotion Professional Committee was established,marking a new milestone for the country in this aspect. In October 2005,the first Affective Computing and Intelligent Interactive(ACII)International Conference was held,gathering world-renowned scholars in the fields of affective computing,artificial emotion,and artificial psychology.This event demonstrates that the artificial emotion and artificial psychology research in our country has gradually moved up to the internationallevel.On September 22,2015,the sixth ACII International Conference was hosted by the Northwestern Polytechnical University in Xi'an.The international conference brought together expert researchers from top universities and research institutions from all over the world.They discussed the recent advances in affective computing and intelligent interaction and other related fields,and then reported the latest research results.This event shows a huge leap in China's affective computing research.

Table 1 General research status of artificial psychology and artificial emotion in China(2013-2015).

Details of the development of artificial psychology as a course of study are as follows.In 1998,the concept of artificial psychology was first put forward[14].In 1999,artificial psychology was the topic of the keynote speeches in two in ternational conferences,thatis,IPMM99 and ICAI99.In 2000, the Study on the Theory of Artificial Psychology was funded by the National Natural Science Foundation.In 2004,artificial psychology experts were invited by the Ministry of Education to report(six expert reports in all)in the“Harmonious Human-Computer Interaction Technology”senior seminar.In 2005,artificial psychology was included in the Global Study on Artificial Science and Artificial Art Research List from Tokyo University.In 2005,the research progress of artificial psychology was selected as the theme of the conference report in the IEEE International Conference on Soft Computing (WSTST05).In 2005,the Study of Artificial Emotion Theory and Method based on State Space Description receivedsupport from the National Natural Science Foundation of China.In 2005,the first ACII Conference was held in Beijing, and the research results of artificial psychology were introduced into China's emotional computing.The Research Status and Progress of Artificial Psychology and Artificial Emotion in China was published in theScience Foundation of China, Vol.1,2013.Artificial psychology and artificial emotion theories,methods,and technology should be used in the field of psychology to aid facilitate psychological counseling and provide information technology support to psychiatrists.The development of modern modes of mental health care should be promoted in China.

The target of engineering is application,and the product of engineering is technological innovation.Thus,the highest stage of artificial psychology research and development is application.The main task is to consider the human mental and emotional components in artificial systems,and the goals are to enable an artificial control system and computer system adapt to human emotion,achieve interpersonal and human harmony,and accomplish the purposes of scientific human services.The study of artificial psychology mainly covers emotion modeling and formalization,emotional robots,virtual humans,personalized information systems,facial expression recognition,and image-processing technology to understand human behavior.Examples of projects in artificial psychology include network-based personalized intelligent commodity purchase system,computer vision based affective virtual human interaction technology,hidden Markov model(HMM) based expression recognition technology for emotional robots, artificial psychology modeling method and virtual human technology,Web-based face detection and facial information processing technology,artificial psychology modeling and virtual reality applications of emotional virtual human,integrated robot platform with emotional interaction ability, personalized shopping systems,multi-agent technology of an information service system in a pervasive environment,virtual human technology with emotion and voice interaction capabilities,image processing based facial feature extraction and expression recognition,artificial psychology technology and image understanding for an e-learning intelligent assistant system,control strategy and behavior modeling technology for biped walking robots,hardware platform for biped walking robots,artificial psychology based affective robot technology, computer vision-based gait feature extraction and identification,expression recognition,and affective computing modeling techniques.

4.Research and development trend of artificial psychology and artificial emotion

Artificial psychology and artificial emotion has numerous practical applications,including personalized services,humanoid machines,human-imitating controllers,emotionally aware and intelligent robots,smart homes,wearable devices, human-computer interaction,and virtual reality.In 2015, Pepper,the first affective humanoid robot,became available. Mass production of this first robotic product with human emotions has started mass production,and it is offered to ordinary consumers.Pepper is developed by Aldebaran and Soft bank Robotics.It can observe human expression by its visual system and identify human voice via its speech recognition system.It can also process facial expression and tone as well as analyze human emotions.Its emotion recognition function render it more flexible in coping with the prevailing situation and interact with humans.For example,when it detects that someone is approaching,it will look at the person and then stretch its hands and greet the person.When it detects that its master is in a bad mood,it will play music for the master and attempt to improve the mood of the master.It can communicate with the user.Pepper mainly plays the role of a family companion.With continuous improvement in its learning process and capability to cope with situations,this kind of humanoid robot could play an important role in the enterprise,retail,and customer service sectors.

Artificial psychology and artificial emotion has applications in the psychological service industry.Artificial psychology and artificial emotion as a discipline is an extension of artificial intelligence.With the progress in artificial intelligence, the psychological service industry is evolving as well.At present,the psychological services in China mainly include psychological counseling and psychological treatment.However,the development of psychological consultation industry in China is slow because of a number of problems.First,the existing consultation mode is unsuitable for China's national conditions.China's traditional culture affects Chinese people's values and way of thinking;thus,the Western face-to-face mode of psychological counseling is not entirely suitable for the Chinese implicit psychological characteristics.Second,the existing psychological consulting space is small;thus,the concept of psychological counseling is not entirely popular in China.Third,the current quality of psychological consultation staff in China is different from the expectations of patients [15].Thus,the development artificial intelligence opens a new direction for the traditional psychological consultation mode. People's thoughts and physiological characteristics may be acquired by man-to-machine dialogues or wearable devices. Subsequently,the acquired information or data are used to assess a person's mental health and prescribe appropriate treatment measures.Artificial intelligence aided psychological counseling can help reduce the possible conflicts and emotional concerns induced during psychological treatment; thus,artificial intelligence has a great advantage in psychological counseling.We believe that the application of artificial intelligence to the psychological counseling industry can significantly alleviate the existing problems,promote the development of psychological consultation industry,and stimulate a huge market demand[16].

Wearable devices have also gained popularity.Since Google introduced their intelligent glasses in 2012,wearable devices have used intelligence,health,and portability as their selling points,thereby quickly opening a new frontier in the information industry.Subsequently,wearable devices officially became commercial.At present,the smart bracelet is a mature product in the field of wearable devices.Smartbracelets mainly provide certain health functions,such as step counting,heart rate measurement,and sleep monitoring.Other wearable devices can be used to collect temperature,respiration,blood pressure,and other physiological parameters[17]. By putting on wearable devices,users can monitor their own health data in real time to understand their physical health.At present,the wearable devices in the market have not been used for measuring emotional responses;however,using a wearable device to measure people's psychological emotion is feasible. Human's emotions can be reflected in certain physiological data.Through the collection and analysis of physiological signs,the machine can better understand human's emotion; consequently,it can establish a better human-computer interaction and provide a more personalized service.

Theoretically,in artificial psychology and artificial emotion,machines could demonstrate psychological activities, such as feeling,character,and will.Their intelligence could even evolve,thereby allowing them to exhibit empathy consistent with humans.In the human-machine interaction process,the abilities of analyzing and regulating for both machines and human are necessary.Owing to the complexity and relativity of human cognition and emotion,cognitive affective computing develops gradually with research.Professor Fu Xiaolan of the Institute of the Chinese Academy of Sciences said that cognition and emotion are interdependent rather than separate.Thus,a new algorithm should be established to describe the relationship between cognition and emotion better based on the study of behavioral science and neuroscience.This comprehensive study on behavioral and psychological science yields a far-reaching effect in the field of computer science and artificial intelligence,particularly in the field of artificial psychology and artificial emotion theoretical research.

At present,cognitive-affective algorithms can divided into two types.The first type is the emotional algorithm based on cognitive appraisal.A representative of this type of algorithm is the Ortony-Clore-Collins(OCC)cognitive emotion generation model,which is a classical emotional computing model for cognitive psychology in the fields of information science and technology in Europe and the United States.The cognitive-affective algorithm does not rely on a set of basic emotions or a clear multidimensional space to express feelings;instead,it uses the consistency conditions for cognitive inference to express emotions.The emotion model based on cognitive appraisal generates emotions appropriate for a particular emotional situation(including the event,object,and agent).This method not only generates cognitive emotion but also triggers the subjective experience from other emotions. Professor Pan Zhigeng of the School of Computer Science of Zhejiang University proposed a comprehensive computational model of emotions based on Damasio's physiological mechanism and Ortony,Clore,and Collins' cognitive event assessment.Zhigeng's model optimized the cognitive model by introducing time estimation,integrating an emotional update mechanism to describe the dynamic change characteristics of emotions at different moments,establishing structures to describe the relationship between the events and emotions,and finally synthesizing a more specific and delicate emotional behavior.The OCC cognitive emotion generation model has laid a solid cognitive foundation for artificial psychology and artificial emotion.However,the migration process of emotion has yet to be perfected to make it more adaptive to human mental processes[18].

The second type is the cognitive-affective algorithm based on the transition of external stimulation.Robots extensively assess the external input stimulus/stimuli and the internal demand and subsequently adjust their behavior.This algorithm can be used to evaluate an emotional state and express the controlled interaction.Classical algorithms such as HMM by Professor Picard and the emotion model for the Robota robots. Professor Liu Shirong of the Institute of Automation of Hangzhou University of Electronic Science and Technology proposed an autonomous navigation control method for mobile robots based on the recognition of emotions and the environment.Liu's model used the Picard's HMM in a behavior-based robot controlling system.The system used an ART2 neural network to classify the environmental status, combining emotion,cognition,and behavior organically.According to the comprehensive evaluation results of emotion and cognition,the network can generate the internal reward from emotion and the external reward from cognition.Thus, the system can effectively strengthen the weights corresponding to the appropriate behavior and improve the learning capability of robots in a specific environment.Chung proposed an interaction-embedded HMM framework for human behavior understanding.By analyzing such parameters as the relative distance of subjects in the scene and duration for which this distance is maintained,the subjects' personal behavior and interactive behavior are obtained.Subsequently, the subjects' personal behavior and interactive behavior can be analyzed using HMM;the behavior understanding model can be established.One of the keys to establishing an interactive algorithm is to develop a continuous and controllable artificial psychology computing model under the conditions of meeting the decay of the state space,nonlinear time constant,and continuity in the process of human-machine interaction[19].

5.Summary and prospects

At present,studies on robot humanoid interaction and cooperation in China mainly focus on the integration of voice and images for interaction.However,research on new interaction techniques or humanoid cooperation theories and technology is limited.The problems and challenges facing the application of artificial psychology and artificial emotion are as follows:

The singular valley evaluation of humanoid robots and the evaluation of harmonious human-computer interaction and cooperation.Whether the robot's humanoid interaction and cooperation are influenced by the shape of the robot itself and the personification of interaction and cooperation remains unclear.First,an evaluation mechanism should be established, including related factors,such as project index and index weight,which influence the evaluation.On this basis,we canadjust the related factors against a certain standard index and then improve the naturalness of the humanoid interaction and cooperation.The former adjustment requires static evaluation, whereas the latter improvement requires dynamic evaluation. Singular valley evaluation theory can used to judge whether the shape of the robot is suitable.Subsequently,the humanoid robots can be produced by conducting psychological experiments and establishing the corresponding static evaluation database.For the personification of the robot's humanoid interaction and cooperation,a good robot platform should be created for the evaluation of human-computer interaction and cooperation.At the same time,a corresponding dynamic evaluation database should be constructed to guide the process of interaction and cooperation.Therefore,the singular valley evaluation of the humanoid robot and the evaluation of harmonious human-computer interaction and cooperation are the first problems to be solved in the process of humanoid robot interaction and cooperation.

The lack of in-depth research on the emotion model of the humanoid robot and human-machine interaction.Emotion should be integrated in the process of robot humanoid interaction and cooperation,for without it human-computer interaction is incomplete.Existing emotion models vary. Owing to the difference in the focus of modeling,the established models are also not the same.Although each of these models has contributed immensely to the study of simulating human emotion generation and change process,the research on the existing emotion models is not comprehensive.An in depth research is can increase the universal applicability and calculability of a model.For example,an emotion model described by infinite-dimensional delay differential equations can be studied based on an emotion model of a finite dimensional discrete Euclidean space to make the former more aligned with reality.For an emotion model based on the probability space,the individual emotional difference variation caused by the complex emotion simulation and parameter adjustment can be studied.Therefore,an in-depth research of emotion models is imperative in the process of humanoid robot emotion generation.

The need for a cognitive computing model of human mental states.The research on perception and vision in cognitive psychology mainly focuses on pattern recognition, particularly visual pattern recognition.Machine vision provides technical support to humanoid robot interaction and cooperation.The essence of humanoid interaction is the cognitive processes of human mental states.The technologies of cognitive psychological state recognition,such as human--machine interaction with intention understanding,problem analysis,and behavior or decision prediction are gradually becoming the research hot spots.However,at present,research is mostly concentrated on the recognition of specific areas and specific cognitive states.The cognitive computing model of mental states in human-computer interaction remains at the initial stage worldwide,and many theoretical research and implementation methods are not yet mature.Furthermore,the existing modes of human-computer interaction disregard the physiological characteristics of user's cognitive factors and cognitive states in system analysis.Therefore,research on physiological characteristics should be conducted using cognitive psychology as basis and combining a human cognitive mechanism,a pattern recognition method based on visual images,and a cognitive computing model of human psychological states.This research can promote the development and application of robot humanoid interaction and promote harmonious human-computer interaction.

The reliability and accuracy issues of humanoid interaction technology.We described the establishment of a dynamic evaluation database for harmonious human-computer interaction and cooperation.Reliable and accurate humanoid interaction and cooperation should be studied by using multichannel information,such as appearance,face state,head posture,and voice based on the theory of affective computing and cognitive computing.Research in this area will not only improve the capability of the robot to demonstrate anthropomorphic thinking and behavior but also constantly provide it with new knowledge via interaction with the environment.The robot can complete a variety of tasks beyond the imagination of its designers and adapt to an unstructured and dynamic environment by itself.A better understanding of humanoid interaction will also change the role of humans in human--computer interaction and cooperation;instead of having users adapt to the demands of the robot,the robot adapts to the users.As a result,the comfortderived from using robots could be improved,the technical support needed to popularize humanoid robots could be provided,and the large-scale application of humanoid robots in smart homes could be realized.

Development of a research platform with emotions and humanoid interaction and cooperation capabilities.In the research of humanoid interaction and cooperation,a humanoid robot highly similar to humans can serve as a research platform.As a solution to this problem,a corresponding static evaluation database can be constructed based on singular value theory,and the research results can guide the manufacture of the humanoid robot platform.In the process of humanoid robot interaction and cooperation,Ekman's facial action coding system only provides the mapping relationship between the facial muscle groups and the expression,but the mapping relationship among the inner emotion,expression,and facial action control of the humanoid robot is not provided.We can realize this mapping relationship via a PC software system of the robot platform.This mapping relationship is fundamental to the development of a robot platform that has emotions and humanoid interaction and cooperation capabilities.

The application of humanoid robots for the elderly and the disabled in smart homes.Application is essential for the development of humanoid robots.The importance of studying robot humanoid interaction and cooperation is to resolve application problems.Therefore,the practical applications of humanoid robots for the elderly and the disabled in smart homes need to be studied.Currently,China's population above 60 years old has exceeded 10%of the total population.The total of the elderly population in the country is expected to reach 200 million in 2015.The aging problem has become an unprecedented new challenge for China.In addition,theproportion of the disabled in the country ranked the highest in the world.These figures portend a major impending challenge to the country in nursing the elderly and the disabled.A large number of humanoid robots are necessary to provide services to the elders and the disabled.Thus,the possible application problems of humanoid robots for the elderly and the disabled in a smart home environment should be studied.

More than 10 years have passed since China first put forward the concept of artificial psychology and artificial emotion.Artificial psychology and artificial emotion research has progressed to a certain extent.However,owing to the diversity of emotional psychology theories and methods,the artificial emotion theories and methods have not reached maturity,thereby restricting the application of the technology. Numerous challenges to the research of artificial psychology and artificial emotion remain.However,with the constant enhancement of China's science and technology level and the constant growth of the influence of the country's scholars in the international academic arena,the country can gradually solve the existing research problems and realize the maturity and development of humanoid robot interaction techniques. Innovative research should be conducted to overcome the challenges to the study of robot humanoid interaction and hasten the progress of artificial psychology and artificial emotion in China.

Acknowledgments

This work is supported by the National Natural Science Foundation of China(Key Project No.61432004),National Natural Science Foundation of China(Normal Project No. 61672093),The Study of the Home Elderly Companion Robots' Affective State Migration with Micro-expression Cognition,National Key Research and Development Program(No.2016YFB1001404),and The Zhenjiang City's 331 Planning Project(2014-07-79).

[1]Q.M.Pan,W.J.Hu,Q.H.Li,Y.Shen,H.M.Xie,Z.Liu,Z.G.Pan, Adaptive virtual human perception modeling driven on the motivation and emotions,J.Comput.Aided Des.Comput.Graph.27(9)(2015) 1786-1794.

[2]Z.L.Wang,Artificial Psychology,Mechanical Industry Press,Beijing, 2007.

[3]H.D.Zhang,S.R.Liu,Autonomous navigation and control of mobile robot based on emotion and environment cognition,Control Theory Appl.6(25)(2008)995-1000.

[4]H.W.Yang,Z.G.Pan,G.D.Liu,An integrated approach to affective modeling,Comput.Res.Dev.45(4)(2008)579-587.

[5]H.Zhou,H.W.Yang,L.H.Cai,Changes of emotional speech based on SVR,J.Northwest Normal Univ.1(45)(2009)62-66.

[6]Y.Liu,Q.F.Fu,X.L.Fu,Interaction between cognition and emotion, Chin.Sci.Bull.18(54)(2009)2783-2796.

[7]Marvin Minsky,The Society of Mind,Simon&Schuster,1987.

[8]R.W.Picard,Affective Computing,MIT Press,London,1997.

[9]Y.L.Zeng,L.Cheng,T.N.Tan,Review and prospectof research on brain like intelligence,J.Comput.Sci.38(12)(2015)1-11.

[10]J.Cheng,G.Y.Liu,Research Progress on emotion recognition from the perspective of subject cross,Comput.Sci.39(5)(2012)19-24.

[11]L.Lu,A study of the multimodal cognition and interaction based on touch,audition and vision,J.Comput.Aided Des.Comput.Graph.4(26) (2014)654-661.

[12]M.H.Yang,Nature multimodal human-computer-interaction dialog system,Comput.Sci.10(41)(2014)12-35.

[13]Y.C.Huang,K.Y.Wu,Y.T.Liu,Future home design:an emotional communication channel approach to smart space,Personal Ubiquitous Comput.17(6)(2013)1281-1293.

[14]Z.L.Wang,Artificialpsychology:a scientific approach to the work ofthe human brain,J.Univ.Sci.Technol.Beijing 22(5)(2000)478-481.

[15]S.Y.Liu,An analysis of the current situation of the development of psychological consultation industry in China,Sci.Technol.Innov.Her. 36(2007)113-115.

[16]Y.Tao,T.M.Wang,S.L.Sun,The good care of the elderly life partner robot,Robot Technol.Appl.20(5)(2013)39-43.

[17]S.T.Feng,Developmentstatus and trend of wearable devices,Res.Dev.3 (2014)52-57.

[18]M.A.Salichs,I.P.Encinar,E.Salichs,A.Castro-Gonzalez,M.Malfaz, Study of scenarios and technical requirements of a social assistive robot for Alzheimer's disease patients and their caregivers,Int.J.Soc.Robot.8 (1)(2016)85-102.

[19]J.M.Beer,C.A.Smarr,A.D.Fisk,W.A.Rogers,Younger and oder users' recognition of virtual agent facial expressions,Int.J.Hum.Comput. Stud.75(2015)1-20.

Zhiliang Wangis a professor at the University of Science and Technology Beijing,China.He is a senior board member of China Artificial Society.He received his Ph.D.degree from Harbin Institute of Technology, China.His current research interests include artificial psychology and the Internet of Things.Corresponding author,e-mail:wzl@ustb.edu.cn

Lun Xieis a professor at the University of Science and Technology Beijing,China,where he received his Ph.D.degree.He is a board member of China Artificial Society.His current research interests include robotics research and affective computing.E-mail: xielun@ustb.edu.cn

Ting Luis currently a Master's student in the School of Computer and Communication Engineering,University of Science and Technology Beijing,China. Her research interests include human machine interaction,pattern recognition,and affective computing. E-mail:17801002160@163.com

Available online 12 November 2016

*Corresponding author.

E-mail address:wzl@ustb.edu.cn(Z.Wang).

Peer review under responsibility of Chongqing University of Technology.

http://dx.doi.org/10.1016/j.trit.2016.11.003

2468-2322/Copyright©2016,Production and hosting by Elsevier B.V.on behalf of Chongqing University of Technology.This is an open access article under the CC BY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

Copyright©2016,Production and hosting by Elsevier B.V.on behalf of Chongqing University of Technology.This is an open access article under the CC BY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).