A Theory of Semantic Information

Beijing University of Posts and Telecommunications, Haidian District, Beijing 100876, China

Editor: Tao Jiang

I.INTRODUCTION

Information has been recognized as one of the major resources that humans could effectively utilize for raising their standards of living in the age of information.The establishment of the theory of information has thus become one of the most urgent demands of the society.

It is well known that the really useful information for humans should be the trinity of its syntactic, semantic, and pragmatic information, enabling users of the information to know the form, the meaning, and the utility of the information and thus know how to use it.

Claude E.Shannon published a paper in 1948 in BSTJ entitled “A Mathematical Theory of Communication”[1]that successfully deals with such issues as the calculation of information amount produced by the source of communication system, the calculation of the information capacity of the channel, the coding and decoding for matching the source and the channel, etc., making great contributions to the communication theory.Considering that all the issues are related to information to some extent, the theory was later renamed as Information Theory.

In the case of communication engineering,however, only the form factor of signals (the carrier of syntactic information) needs to be considered and has nothing to do with both the semantic and pragmatic information.Hence,Shannon theory of information is indeed a statistical theory of syntactic information only,not an entire theory of information at all.This leads the fact that the theories of semantic and pragmatic information get more and more concerns.

In addition to Shannon theory of information, there have also been Combinatory Information Theory and Algorithm Information Theory in academic literate[2～4].But both of them are kinds of theory of syntactic information too and have not considered the theories of semantic and pragmatic information.So we will not mention them any more in the paper.

I.A BRIEF REVIEW OF THE CLASSICAL THEORY OF SEMANTIC INFORMATION

The earliest effort in attempting the theory of semantic information was made by R.Carnap and Y.Bar-Hillel.The framework of the theory that they proposed in 1950s[5]can very briefly be described as follows, which can also be found from the references[6,7].

First, they defined an ideal language model that containsn(a finite positive number)nouns andk(another finite positive number)adjectives as well as 1 verb “have, or be”.If“a” is a noun and “P” is an adjective, and then“Pa” is read as “ahasthe propertyP” or “aisP”.The model has 5 connects:

Based on the model above, it can produce a number of sentences which may true (Pa∨～Pa,for example), or false (Pa∧～Pa, for instance),or infinitive (Pa, for example) in logic.The amount of semantic information contained in a legal wordiis thus defined asa function of the number of sentences in the ideal model of language that the word i can imply.The more the number of sentences that a word can imply in the model of language, the larger the amount of semantic information the word has.

More specifically, a concept called “state descriptor Z” is defined as the conjunction of one noun and one adjective (positive, or negative, but not both).As result, there areknpossible such conjunctions and 2knpossible state descriptions.

Another concept called “range” is set up as follows: The range for a sentenceiis defined as a set of the state descriptors in which the sentenceiis valid and is denoted by R(i).Further, a measurement functionm(Z) is introduced with the following constraints:

(1) For eachZ, there is 0≤m(Z) ≤1,

(2) For allknstate descriptors, there is∑m(Z) ＝ 1,

(3) For any non-false sentencei, itsm(i) is the sum of all m(Z) withinR(i).

The amount of semantic information that sentenceicontains is defined as

It can be seen from the brief review of the classic theory of semantic information that the concept and measurement here are very much similar to that of Shannon theory of syntactic information.In fact, various kinds of the classic theory of semantic information existed have made most efforts in quantitative measurement of semantic information.Moreover, the basic idea for semantic information measurement of any sentence in classic theory is dependent on the number of sentences in the language model that can be excluded by the sentence.

There are, at least, two demerits existed in the classic theory of semantic information.One is the ideal model of language that is much too far from the natural language in reality and is completely unrealistic model of language.The other demerit is that the theory concerns only with the quantitative measure of the semantic information and yet has no concerns with the essence of semantic information, the meaning factor.

It is very important to point out the major difference between the theories of syntactic and semantic information.The theory of syntactic information needs to take serious concern with the numeric measurement of syntactic information because of the fact that the engineers of communication systems must be able to calculate the precise amount of communication resources consumed, such as the bandwidth of communication channel, the energy of transmission, etc.On the other hand,nevertheless, the most nucleus concern in the theory of semantic information is the meaning contained in the information.This is because of the fact that the users of semantic information should be able to understand the meaning factor of that information so as to be able to effectively use the information for solving problems.

Most of the researchers of classic theory of semantic information had not realized the radical difference between these two kinds of theories pointed out above.The researchers involving in the semantic information studies in later time[8～11]also did not mention the difference.

III.FUNDAMENTAL CONCEPTS RELATED TO SEMANTIC INFORMATION

The ‘root concepts’ of semantic information are the concept of information and that of semantics.Therefore, it is necessary to make clear the concept of information and that of semantics as well as the relationship between them before doing other things.

3.1 Classic concepts on semantics and information

The classic concept of semantics can be found from the studies on semiotics.Saussure,Peirce, and Morris, among many others, are the major contributors to semiotics.

Saussure proposed a dualistic notion of signs, relating the signifier as the form of the word or phrase uttered, to the signified as the mental concept[12].Peirce pointed out that a sign is something that stands to somebody for something in some respect or capacity.He considered further that the science of semiotics has three branches: pure grammar, logic proper, and pure rhetoric[13].More clearly,Morris defined semiotics as grouping the triad syntax, semantics, and pragmatics where syntax studies the interrelation of the signs, without regard to meaning, semantics studies the relation between the signs and the objects to which they apply, and pragmatics studies the relation between the sign system and its user[14].

It is recognized from semiotics that the terms of syntactic, semantic, and pragmatic have been used for expressing the formal features, the meaning, and the utility of the signs with respect to the user of the signs.Interestingly, the three terms form a comprehension for the signs.But semiotics did not give analysis on the mutual relations among the three, as we will do later.

On the other hand, there have been many works concerning the concept of information in history.The most representative ones include the followings:

-- N.Wiener announced that[15]information is information, neither matter, nor energy.

-- C.Shannon considers that［1］information is what can be used to remove uncertainty.

-- G.Bateson［16］: Information is the difference that makes difference.

-- V.Bertalanffy wrote that［17］information is a measure of system’s complexity.

More works in this area can be found from the reference［18］.As can be seen, the concept of information is still open.We will have more discussions on the concept in next sub-section.

3.2 Concepts related to semantic information in view of ecosystem

It is easy to see that the concept of semantic information not only has roots but also has its ecological system.Therefore, the understanding of the concept of semantic information must strictly rely on the understanding of its root concepts and its ecological system.Through the investigation of its root and ecological system, all the constraints that the concept of semantic information should observe could be clear.

What is the ecological system that semantic information belongs to? As is stated above, the root concept of semantic information is that of information while the ecological system of information is the full process of information-knowledge-intelligence conversion[19],which is shown in Fig.1.

In the model of Fig.1, the part at the bottom stands for the object in environment that presents the object information while the part on the top stands for the subject interacting with the object through the following processes: the object information is conversed to perceived one (perception), and the latter is conversed to knowledge (cognition), and further conversed to intelligent strategy (decision-making), and finally conversed to intelligent action (execution) that is applied on the object, forming the basic process of information ecosystem.

The most basic rule the theory of semantic information should observe is that the theory of semantic information must be able to meet with the constraints, or the requirements, imposed from the ecological process of information described above.

It is worth pointing out that, among all kinds of object information, only those participating in the process of subject-object interaction (see Fig.1) will be regarded as meaningful and will therefore be carefully studied.Others having not entered into the subject-object interaction will, naturally and thus reasonably,be neglected by the subject.In other words,the model of information ecosystem shown in Fig.1 is typical, necessary, and sufficient.

It is indicated from the model in Fig.1 that the most significant value that information could provide to humans (and human society)is not merely information itself, but even more its ecological products, the knowledge and intelligence.This is because of the fact that information itself is phenomenon about some things that can tell “what it is” while knowledge is the essence about some things that can tell “why it is so” and the intelligence is the strategy for dealing with some things that can tell “how to do it”.

So, it is not wise in any case if one concerns only with information itself without paying attentions to knowledge and intelligence.In other words, one who studies information should not stop at the level of information but should continue to do the study with the view of ecological process of information.This is an important understanding for information studies.

Now let us start to have a specific investigation on all the concepts related to the one of semantic information along with the line of ecological system of information shown in Fig.1.

Definition 1 Object Information / Ontological Information

The object information concerning an object is defined as “the set of states at which the object may stay and the pattern with which thestates vary” presented by the object itself.

Fig.1 Model of information ecosystem

The term “ontological information” is also adopted for “object information” because this information is only determined by the object and has nothing to do with the subject.The two names, object information and ontological information, are mutually equivalent but the term of “object information” will more frequently be used in the paper.

Referring back to Definition 1, the argument of object information can briefly be expressed as the “states-pattern”.For example,given an object whose possible set of states isXand the pattern with which the states vary isP, the corresponding object information can briefly be represented as {X,P}.

Definition 2 Perceived Information /Epistemological Information

The perceived information a subject possesses about an object,which is also termed as epistemological information,is defined as the trinity of the form (named the syntactic information),the meaning (the semantic information),and the utility (the pragmatic information),all of which are perceived by the subject from the object information.

The term “epistemological information” is adopted because of the fact that this information is determined not only by the factors of object (the “states-pattern”), but also by the factors of the subject (the subject’s knowledge and goal).

Fig.2 Relation of semantic information to syntactic and Pragmatic information

It is very clear by comparing Definition 1 and Definition 2 that the object information is the real source coming from the real world while the perceived information is the product of the object information via subject’s perception.As result, the perceived information should contain more abundant intensions than the object information does.

Note that the concept of “semantic information” has been defined in Definition 2, which is the meaning the subject perceived from the object information.Note that the term of semantic information here is well matched with that of semantics in semiotics.

Definition 3 Comprehensive Information

The trinity of syntactic,semantic,and pragmatic information is named comprehensive one.

So, the concept of “perceived information”,or of “epistemological information”, declares that it has three components: syntactic, semantic, and pragmatic information whereas the trinity of the three is specially named the“comprehensive information”.

Noted that although the concepts of syntactic, semantic, and pragmatic have been defined in semiotics, the mutual relationship among them has never been studied.As we can see later, the mutual relationship among them is the most important issues in information studies.

Definition 4 Mutual Relation among Syntactic,Semantic,and Pragmatic Information

The syntactic information is specific in nature and can directly be produced through subject’s sensing function while the pragmatic information is also specific in nature and candirectly be produced through subject’s experiencing.However,the semantic information is abstract in nature and thus cannot be produced via subject’s sensing organs and experiencing directly.The semantic information can only be produced based on both syntactic and pragmatic information just produced already,that is,by mapping the joint of syntactic and pragmatic information into the semantic information space and then naming it.

Further, the mutual relationship of the semantic information to the syntactic and pragmatic information can be expressed in Fig.2.Note that this will be explained more clearly via Fig.5 in section 4.1.

It is concluded from the discussions above thatthe semantic information can serve as the legal representative of the perceived information.This is why the semantic information possesses the highest importance compared with the syntactic and pragmatic information.

Compared with all definitions and discussions given above, we can conclude:

(1) The object/ontological information defined in Definition 1, the “states-pattern presented by object”, is neither matter nor energy and is thus in agreement with Wiener’s statement.However, the Definition 1 is more standardized than the statement.

(2) The concept of perceived/epistemological information defined in Definition 2 is just what can be used to remove the uncertainties concerning the “states-pattern”.It is easy to see that the concept of information in Shannon theory is the statistically syntactic information,only one component of the perceived information.So, the Definition 2 is more reasonable and more complete than Shannon’s understanding is.

(3) As for the Bateson’s statement, it is pointed out that the most fundamental “difference” among objects is their own “states-pattern” presented by the objects.Hence, the Definition 1 and Bateson’s statement is equivalent to each other.But Definition 1 is more regular.

(4) Bertalanffy regarded information as“complexity of system”, in fact the complexity of a system is just the complexity of its“states-pattern” presented.

All in all, the Definition 1 and 2 enjoy the advantages of more regular, more standardized, more reasonable, more universal, and more scientific compared with the statements given by Wiener, Shannon, Bateson, Bertalaffy, and others.So, we would like to use the definitions 1 and 2 for defining object/ontological information and perceived/epistemological information.

Up to the present, the concept of semantic information has been well defined.According to the view of information ecology, however,the discussion on fundamental concepts cannot stop here.It is necessary to investigate how the later processes of information ecology (see Fig.1) may exert influence and impose constraints on semantic information.

Definition 5 Knowledge

Knowledge that subjects have possessed in their minds concerning certain class of events is defined as the “set of states at which the class of events may stay and the common rule with which the states vary” that have been summed up from a sufficiently large set of samples of perceived information related to the class of events.

For brevity, knowledge can be described by using the phrase of “states-rule” just as perceived information is described by using the phrase of “states-pattern”.

Since knowledge is the products abstracted from perceived information, knowledge also has its own three components: formal, content,and value knowledge respectively corresponding to syntactic information, semantic information, and pragmatic information.Similar to the case of perceived information, the content knowledge can serve as the legal representative of the entire knowledge.It is obvious that semantic information plays a fundamental role in the studies of knowledge theory.

Fig.3 Internal Chain of Knowledge Ecology

Fig.4 External Chain of Knowledge Ecology

It is necessary to note that knowledge itself forms an ecological process: from the perceived information to the empirical knowledge via the process of induction, and then to the regular knowledge via the process of validation, and further to the commonsense knowledge via the process of precipitation, all based on the instinct knowledge.This is shown in Fig.3 and is called the internal chain of knowledge ecology.

It is easy to understand that the empirical knowledge is the knowledge in under-matured status, the regular knowledge is the one in the normal-matured status, and the commonsense knowledge is the one in the over-matured status.As is well accepted, from the under-matured status to the normal-matured status and further to the over-matured status, this is a typical ecological process.

On the other hand, there is an external chain of knowledge ecology that is from (perceived)information to knowledge via inductive operations and then from (perceived information and) knowledge to intelligence via deductive operations under the guidance of the system’s goal as is shown in Fig.4.

The internal chain and external chain of knowledge ecology, as well as the definition of semantic information (which is the representative of perceived information), are of great significance in the studies of artificial intelligence that will be seen in later stage of the paper.

Definition 6 Human Wisdom

Human wisdom is the special ability onlyfor humans.It is the ability for humans to discover,to define,and to solve the problem faced in environment by using his knowledge and his goal for a better life.When the old problem is solved,the new problem will be discovered,defined and solved.By virtual of this ability,the living standards for human kinds and the human ability itself are continuously improved and strengthened［20］.

It is clear from the definition 6 that human wisdom consists of two interactive parts:(1) the ability to discover and to define the problem that must be solved for better living,which is termed as the implicit wisdom, and(2) the ability to solve the problem that has been defined by implicit wisdom, which is named as the explicit wisdom.

The ability of implicit wisdom is supported by such mysterious factors as the subject’s goal for better living, the subject’s knowledge for problem solving, subject’s intuition, imagination, inspiration, and aesthetics, and the like whereas the ability of explicit wisdom is supported by such kinds of operational ability as information acquiring, information processing,information understanding, reasoning, and execution.Because of the difference between the implicit and the explicit wisdom, it is possible to understand and simulate the explicit wisdom on machines which is extremely difficult,if not impossible, to understand and simulate the implicit wisdom on machines.

Definition 7 Human Intelligence

The human explicit wisdom is particularly named as human intelligence that is obviously the subset of human wisdom.

Definition 8 Artificial Intelligence

The man-designed intelligence in machines inspired from human intelligence (explicit human wisdom) is named as artificial intelligence that evidently relies on the knowledge and therefore heavily relies on semantic information.

It is easy to know that there is very closed links between artificial intelligence and human wisdom (see the definitions 6, 7 and 8) on the one hand and there is also great difference between them on the other hand.As for the working performance like working speed,working precision, and strength of working load, etc., artificial intelligence may be much superior than human wisdom.However, as for the creative power is concerned, artificial intelligence can never compete with human wisdom because of the fact that machines are not living beings, they do not have their own goals for living, and thus do not have the ability to discover and to define the problems for solving.

Till now we could have the following conclusions drawn from the discussions carried above:

(1) Semantic information is the result of mapping the joint of syntactic and pragmatic information into the space of semantic information and naming.

(2) Semantic information is the legal representative of perceived/comprehensive information.

(3) Semantic information plays very important role in knowledge and intelligence research.

IV.NUCLEUS OF SEMANTIC INFORMATION THEORY[20]

As the necessary foundation of semantic information theory, the concepts related to semantic information have been thoroughly analyzed in section 3.In the following section of the paper, the generative theory, the representation and measurement theory, and the applications of semantic information will be discussed successively.

4.1 Semantic information genesis

Referring back to Definition 2, we realize that syntactic, semantic, and pragmatic information are three components of the perceived information, which is perceived by the subject from the object information related.So, object information is the real source of semantic information.

Looking back to Definition 4, we understand that different from the syntactic and pragmatic information that both of them are specific in nature and thus can directly be produced via subject’s sensing functions or experiencing function whereas semantic information is abstract in nature and thus cannot directly be produced via subject’s sensing and experiencing functions.Semantic information is a concept produced by subject’s abstract function.

Based on the definitions 2 and 4, the model for semantic information generation process can then be implemented as follows (see Fig.4 below).

First, let the object information be denoted by S while syntactic, semantic, and pragmatic information by X, Y, and Z respectively.And then we assume that there is a comprehensive knowledge base (representing the subject’s memory system) that has already stored a great set of pairwise knowledge: {Xn, Zn},n= 1, 2, …, N, where N is a very large positive integer number.In word expression, this means that then-th syntactic information has then-th pragmatic information associated with it.In other words, the subject has had abundance ofa prioriknowledge: what kind of syntactic information will have what kind of pragmatic information.XnandZn,n= 1, 2, …,N,are one-to-one correspondingly.

The specific process for generating Y from S can be described in the following steps as is shown in Fig.5 below.

Step 1 Generating X from S via Sensing Function

Mathematically, this is the mapping function from S to X.

This function can be realized via sensing organs for humans.

It can also be implemented via sensor systems for machine.

Step 2-1 Generating Z from X via Retrieval Operation

Using X, the just produced syntactic information in step 1, as the key word to search the set {Xn,Zn} in knowledge base for finding theXnthat is matched with X.If such an Xnis found, the Zncorresponding toXnin {Xn,Zn}is accepted as the pragmatic information Z.

This operation can be realized via recalling/memorizing function for humans.

It can also be implemented via information retrieval system for machine

Step 2-2 Generating Z from X via Calculation Function

If the Xnmatching with X cannot be found from {Xn, Zn}, this means that the system is facing a new object never seen before.In this case, the pragmatic information can be obtained via calculating the correlation between X and G, the system’s goal:

This function can be performed via utility evaluation for humans.

It can also be implemented via calculation for machine.

Step 3 Generating Y from X and Z via Mapping and Naming

Map (X, Z) into the space of semantic information and name the result:

This function can be performed via abstract and naming for humans.

It can also be implemented via mapping and naming for machine.

It is evident that both the model in Fig.5 and the steps 1-3 clearly describe the principle and the processes for semantic information(with the accompanied syntactic as well as pragmatic information) generation from the object (ontological) information via the system’s functions of mapping, retrieving (recalling), calculation (evaluation) and mapping and naming.All of the functions are theoretically valid while technically effective and feasible.

The equation (4.1.2) can also be expressed as

Fig.5 Model for semantic information generation

where X, Y, and Z stand respectively for the spaces of syntactic, semantic, and pragmatic information and λ the operations of mapping and naming.

The correctness of Eq.(4.1.2), or Eq.(4.13),can also be illustrated by a number of practical examples in reality.

Suppose a piece of syntactic information(the letter sequence) is given, the corresponding semantic information (y= meaning of the letter sequence) will then depend only onz,the pragmatic information associated according to Eq.(4.1.3).For instance, given thatx=apple, thenywill be different, depending on differentz:

If {x= apple,z= nutritious}, theny= fruit.

If {x= apple,z= useful for information processing}, theny= i-pad computer.

If {x= apple,z= information processor in pocket}, theny= i-phone.

The model in Fig.5 and the equations(4.1.2) and (4.1.3) have identified that the concept of semantic information stated in Definition 2 and the mutual interrelationship among syntactic, semantic, and pragmatic information described in Definitions 4 are appropriate and legitimate.It is also identified that semantic information so defined can practically be generated either by human subjects, or by intelligent systems.

From now on, we use semantic information to represent the comprehensive/epistemological information, or perceived information.

4.2 The representation and measurement of semantic information

As is emphasized above that different from Shannon theory of information (as a statistically syntactic information theory) where the measurement of information is the most concerned issue, here in the case of semantic information theory the most concerned issue is generally not its quantitative measurement but is its qualitative assessment - its ability to understand the meaning of the information and the ability in logic reasoning based on the meaning.The quantitative measurement is in the second place in semantic information theory.

Yet, no matter for quantitative measurement or for qualitative assessment, the representation of semantic information is a common need and an unavoidable foundation.Hence,the first issue before any others we need to discuss is the one of semantic information representation.

4.2.1 Representation of semantic information Suppose we have a piece of semantic informationy∈Y, which may be a word, a phrase or a sentence, even a paragraph of voice, or video,it can be represented by anN-dimensional vector, whereNis a finite and positive integer:

IfN= 1,yis the semantic information with a single element and is named atomic semantic information, the minimum unit of semantic information which can be judged as true or false, meaning that whether the corresponding syntactic informationxnis really associated with the pragmatic informationzn? IfN＞1,yis named the composite semantic information.

For atomic semantic informationyn, it can be expressed by using the following parameters: its logic meaningynand its logic truthtn,whereyncan be defined by Eq.(4.1.3) whiletncan be defined by

It is clear that the parameter is a kind of fuzzy quantity[21]:

Thus, for atomic semantic informationyn,n=1,2, …,N, the comprehensive expression is

where,ynis determined by Eq.(4.1.3)andtnby Eqs.(4.2.2) and (4.2.3).

For N-dimensional semantic information vectory, its truth vector is expressed as

Correspondingly, its comprehensive expression is

4.2.2 Measurement of semantic information Generally speaking, the quantitative measurement of semantic information is not as important issue as that in syntactic information theory.So this sub-section is treated in appendix.

V.MAJOR APPLICATIONS OF SEMANTIC INFORMATION

As stated many times above, the most important issue related to semantic information theory is its ability for expressing the meaning of information and for logic operation and reasoning rather than for its quantitative measurement.

More specifically, the most fundamental applications of semantic information include(1) knowledge organization based on semantic information, (2) learning and cognition based on semantic information, (3) strategy creation based on semantic information.

5.1 Knowledge base: knowledge organizing based on semantic information

The role that knowledge base plays in advanced information systems, like artificial intelligent systems, is almost the same as that the long-term memory system of brain plays in human information system.They are really fundamental in knowledge and intelligence systems.

In the fields of computing and artificial intelligence research, the concept of knowledge base is not new.Hence, we will not discuss the concept of knowledge base itself.But note that the approaches to knowledge organizing in those fields have all been based on the formal data (syntactic information).Here we will show how the knowledge is organized via semantic information that is the legal representative of comprehensive information,instead of syntactic information alone.In other words, the knowledge base based on semantic information will be more reasonable than that based only on syntactic one.

It has been pointed out in section 3 of the paper that the equation,y= λ(x,z),x∈X, y∈Y,z∈Z, can be regarded as the existential definition of semantic information and can also be seen as the constructive definition of semantic information.This indicates that what determines the semantic information,y, is exactly its accompanied 2-touple-array of syntactic and pragmatic information, (x,z).

For example, if given the 2-touple array is thatx= something with round shape like a small ball, red color, more or less 0.2 kilo weight;z= nutritious, good taste and good for health, the 2-touple array (x,z) can then be mapped into the space of semantic information and named as “y= fruit” (see Fig.5 and Fig.2).

Once again, the meaning of a piece of semantic information,y, can well be expressed by its associated 2-touple-array of syntactic and pragmatic information, (x,z).Therefore,no matter it is an object or is an action, its semantic information (meaning) can well be expressed by its associated 2-touple-array of syntactic and pragmatic information:

Name of an object= (Feature set of its form,feature set of its utility) (5.1.1)

Name of an action= (feature set of its form,feature set of its function) (5.1.2)

The more precise the description of the feature sets for the 2-touple-array is, the more precise the description of the semantic information will be.There must, of course, be a compromise between the requirement of preciseness and that of the briefness.

The method of knowledge representation based on semantic information is especially suitable for the description of “knowledge refining and abstraction” that is the real nucleus for human thinking and for organizing the large amount of knowledge into a large-scale of organic system of knowledge – the knowledge base.

Fig.6 shows the principle of knowledge organizing based on semantic information.There arenpieces of knowledge, name-1,name-2, … name-n, each of which has its form feature and utility feature.The form fea-ture in common among all form features and the utility feature in common among all utility features are abstracted to the new name - the class name.Evidently, the class name in higher level of the organization is more abstract than that in lower level.

It is easy to find numerous practical examples of implementing the principle of knowledge organizing based on semantic information, as shown in Fig.6, in academy and everyday life.

For example, ball-point-pen, brush-pen,and pencil respectively have their own form features and utility features, namely form feature (1) and utility feature (1); form feature(2) and utility feature (2); and form feature (3)and utility feature (3).Since they have certain form feature in common and certain utility feature in common, they have been named“pen” as their class name.The name of pen is more abstract than that of ball-point-pen,brush-pen, and pencil.

By utilizing the same principle of knowledge organizing based on semantic information, it is possible to organize great amount pieces of knowledge into a large scale and multi-level knowledge base.At the lower level of the base, the names are more specific while at the higher level of the base the names are more abstract, all the entities under their class name within the base will have certain form and utility features in common.This is the principle of building up the knowledge base and is an important function that semantic information plays.Compared with the one based on syntactic information, this principle is apparently much more reasonable and effective.

5.2 Cognition: learning and understanding based on semantic information

Learning, understanding, cognition, and evolution are a group of fundamental concepts that are mutually interlinked and are most important abilities for humans as well as for machines.Without these abilities, humans would not be able to adapt themselves to the environment and would not be able to make such great progresses as they have already made.

There have been many, and yet different, explanations on the concept of learning in history.In the earlier days, learning was explained as learning based on rule applications while in the modern time, learning is explained as a process of statistical processing based on large volume of data and high speed computing[22,23].

Fig.6 Principle of knowledge organizing based on semantic information

What we would like to propose here is the one that defines the concept of cognition as knowledge acquiring via learning and un-derstanding based on semantic information.This is a new definition of cognition because all definitions of cognition existed before are based only on syntactic information.

If referring back to the history of human learning development, the definition of cognition proposed here will become good choice against other definitions existed previously.There are three major stages of cognition development during the lifetime of human beings[24].

(1)Accepting and Remembering: the infants’ mode of learning and cognition

In the infant stage, he or she lives in a small world created by his or her parents and has no the ability of independent thinking.The parents in the small world play the role of absolute, and also reliable, authority for everything.As the result, what the parents taught would naturally be accepted and remembered with no doubtfulness.This is the really elementary mode of learning in the earliest stage of one’s life during which a certain number of the very basic terms and rules had been learnt and stored in his or her longer-term memories as the starting base for further learning in later stage.This is also the mode of memorization-based learning in machine.

(2)Following the Public: the teenagers’mode of learning and cognition

As youngsters, they begin to step in the society and live both inside and outside of the family.Parents have not been the only authorities for them.Instead, teachers, classmates, various kinds of public leaders could tell them many things that they never heard before within their family.Consequently, they could learn many more than what from their parents.On the other hand, however, they are still not matured enough for understanding the complex problems in the society.Therefore,a natural mode for them to adopt for learning is to follow the general trend from the public of the society.This is just the mode of statistic learning in machine.

(3)Autonomously Understanding: the adults’ mode of learning and cognition

As adults, they themselves have been matured enough and therefore able to analyze things via learning from normal education and various kinds of practical experiences.They have established the abilities of autonomous thinking and analysis and hence would like to accept something only based on their own understanding.It is this mode of thinking and learning that enable them to innovate many new technologies and create the new theories.This is the most powerful and useful mode of thinking and learning for human cognition.Unfortunately, there has not been such kind of mode of learning implemented in machine so far.

It is worth pointing out that the three stages of mode of learning discussed above are not independent to each other.In fact, through the mode of “accepting and remembering”,ones learn many of the commonsense knowledge and lay the very foundation for further learning; through the mode of “following the public”, ones learn many of the empirical and regular knowledge that are necessary for them to successfully live life in society; and finally,through the mode of “autonomously understanding”, ones could be able to make contributions to the social development.So, accepting and remembering, following the public,and autonomously understanding do form a kind of ecological process of human learning ability growth.In other words, autonomously understanding is the most advanced approach to learning.

Technologically, the mode of human learning based on “accepting and remembering” has been implemented as “mechanical learning” in the theory of machine learning whereas the mode of human learning based on“following the public” has been implemented as “statistic learning” in the theory of machine learning.However, due to lacking of semantic information theory in the past, there has never,till the present, been theoretical study as well as technical implementation for the most advanced, and hence the most useful, mode of learning based on the “autonomously understanding”.

Owing to the development of comput-ing and database technology, currently the statistical learning becomes the dominant approach to machine learning.However, the performance that statistical learning provides is far from satisfaction.This is because of the fact that statistical learning is based only on syntactic information and unable to use the semantic information.

As was emphasized many times above that semantic information is the legal representative of the comprehensive information.Therefore, having the semantic information implies knowing its form, meaning, and utility with respect to the subject and this is what we mean a complete understanding of a pierce of information.

The technological model of learning and cognition with autonomously understanding based on semantic information can be shown in Fig.7.

Fig.7 Cognition with autonomously Understanding based on Semantic Information

Fig.8 An Example of Inductive Cognition based on Semantic information

The input of the model in Fig.7 is not syntactic information as usual but is semantic information, the central algorithm performing the function of learning from the semantic information is the inductive-type, and the output is the knowledge, the result of cognition.The comprehensive knowledge base is the foundation of the model.It provides the system with various kinds of knowledge such as the algorithm for induction, the goal for learning,the set of {form feature; utility feature}, and other kinds of knowledge that are needed for inductive learning.The pre-processing and post-processing are units performing the auxiliary functions like information expression and knowledge expressions and the like.

The choice of induction as central algorithm of the model is natural because all kinds of information are facts and specific phenomena while all kinds of knowledge are rules and essences.Any rules and essences can be established only through the operation of induction and abstraction.Fig.8 is an example of cognition based on semantic information.

The inductive cognition based on semantic information expressed in Fig.8 consists of a number of steps necessary:

(1) Given a number of samples of semantic information (names): apple, pear, … and orange, each of them is described by the corresponding form features (the description of syntactic information) and utility features (the description of pragmatic information) as is seen at the lower part in Fig.8.

(2) The common features for the forms and utilities among all the names are found through the operation of induction.The joint of the common features for form and utility are then mapped into the knowledge space and named as “fruit”.It is found that all types of fruit has the same utility feature that is the“nutritious”.This can be seen in the higher part in Fig.8.

(3) Consequently, one of the knowledge that can be concluded from the step (2), operation of inductive cognition, is the sentence of“Fruits are nutritious”.This is because of the fact that there is a common feature of “nutritious” in the utility features of “fruit”.

It is not difficult to have many more examples showing the same principle of inductive cognition based on semantic information.

By comparing the three modes of cognition/learning, it is convinced that the inductive cognition with “autonomously understanding”based on semantic information is really an most advanced approach to learning and cognition among the three.This is true not only for human learning and cognition but also for machine learning and cognition.On the other hand, however, there is an organic relation among the three models of cognition/learning because the knowledge learnt via the “accepting and remembering” and “following the public” has been regarded as the fundamental knowledge stored in the comprehensive knowledge base in Fig.7.In other words, the three models of cognition/learning are very well cooperated to each other.

5.3 Strategy creation based on semantic information

Referring back to the model of information ecology shown in Fig.1 we can clearly see that intelligence (the major embodiment of intelligence here is the intelligent strategy) can only be created through the process of conversing the semantic information to strategy supported by the related knowledge and guided by the subject’s goal.This process can often be described by a conversion of “(semantic) information - knowledge – intelligence”.

The conversion of “(semantic) information– knowledge” has already been discussed in last subsection and the conversion of “(semantic) information – knowledge – intelligence”will therefore become the thematic topic of this subsection.The principle of the model for implementing this conversion, or the relationship between intelligent strategy and semantic information, can be shown in Fig.9 bellow.

As can be seen from Fig.8 and Fig.9 that the models for the two kinds of conversion look very much similar to each other.The major differences between them lie in that (1)the final product (output) is different: knowledge as the final product in Fig.8 while intelligent strategy as the final product in Fig.9;(2) the inductive-type algorithm as the central operation in Fig.8 while the deductive-type algorithm as the central operation in Fig.9;(3) the contents of the comprehensive knowledge base as well as the pre-processing and post-processing are also accordingly different between the two models.

The function for intelligent strategy to perform is to solve the problem the subject is facing.As is well known that inductive-type algorithms are suitable for refining knowledge from semantic information whereas deductive-type algorithms are suitable for creating intelligent strategy from knowledge, semantic information and the subject’s goal for problem solving.

Recalling to the problem solving in Artificial Intelligence, there is, for example, a wellknown algorithm called Production System,which can be described as follows[25]:

1.DATA← initial database

2.Until DATA satisfies the termination condition, do:

i) Begin

ii) Select some rules, R, in the set of rules that can be applied to DATA

iii) DATA← result of applying R to DATA

In our case, the initial database, DATA, the termination condition, and the set of rules (or rule base) in Production System are respectively corresponding to the followings:

(1) The semantic information stands for the subject’s understanding about the problem –the object information, which is the original state of the problem that the subject wants to solve and is the starting point for intelligent strategy to run.This is just the “initial database” in Production System in AI.

(2) The subject’s goal for the problem solving stands for the final state of the problem to be solved and is therefore the final point for intelligent strategy to run.This is equivalent to the termination condition in Production System in AI.Note that the expression for the subject’s goal should also have the format of{form feature of the goal, utility feature of the goal}.This means that the subject’s goal is regarded as a kind of semantic information.

Fig.9 Strategy Producing based on Semantic Information

(3) The knowledge supplied by the comprehensive knowledge base is the constraints given to the finding of the intelligent strategy:what it can do and what it cannot do within the space of strategy finding.This is equivalent to the “Rule Base” in product System in AI.Again, the expressions for the rule should also observe the format: If (form features, utility features) Then (form feature, utility feature).

It is seen that both the starting state and the final state are given from the problem description.The task for the strategy creation is to find a path going from the starting state to the final state subject to the constraints imposed by the knowledge stored in the knowledge base.The strategy can be implemented as a series of rule selection and application where the rules are written as “If … Then …” defined by the knowledge.And this is typical operation of deduction process.

Obviously, the strategy created for intelligently solving the problem given will be the result via performing some complex function of the semantic information, the knowledge,and the subject’s goal.Using the symbols,Isem,K, andGto denote the semantic information,knowledge, and the goal, we will have

Where in Eq.(5.3.1)Ststands for the strategy andffor the complex function, which may need the support from mathematical logic operations[26]and universal logic[27].

Furthermore, as long as the strategy is created and exerted to the problem, an error between the practical result of strategy exertion and the subject’s goal assigned beforehand may occur.This error should be considered as the new information for problem solving and should be fed back to the input of the system for learning more knowledge and optimizing the strategy.This kind of loop of error fed back, learning and optimizing may need to perform many times so as to make the error gradually decreased and finally becoming satisfactorily accepted.This can be seen again from the model of information ecology in Fig.1.

Naturally, the expression of error information here should have the format of {form feature, utility feature}, similar to the format of semantic information expression.This will be more convenient, and more helpful, for the learning and optimizing processes.

It can be summarized based on all discussions carried out above in the paper that the theory of semantic information does play the greatest role in the entire discipline of information science studies that cannot be replaced by any componential information theory, such as Shannon theory of information, or classic theory of semantic information.The theory of information science will be uncompleted if there is no proper theory of semantic information.This is the significance and value that the theory of semantic information deserves.

VI.CONCLUDING REMARKS

The most fundamental purpose for humans to acquire information is to utilize information for solving the problems they are facing.For this purpose, to understand the meaning of the information (semantic information), to evaluate the real value of the information (pragmatic information), and thus to create the strategy for solving the problem are absolutely necessary.However, the unique theory established so far is the mathematical theory of communication by Shannon in 1948, which is a statistical theory of syntactic information and has nothing to do with the theory of semantic and pragmatic information.

Having believed the truth that it should not be acceptable for humans to live in such a world that it has only form factors while has no meaning and no value factors, the author of the paper has made efforts to analyze the difficulties and setbacks that former researchers have encountered, and to investigate the methodologies suitable for information studies and then discovered the new one, which can be termed “methodology of ecological evolution in information discipline”, or more briefly, the methodology of information ecology.

Based on the methodology of information ecology, the concepts related to information and semantic information have been re-examined, the definitions of object information and perceived information have been re-clarified,the interrelationship among the syntactic,semantic and pragmatic information and the principle of genesis of semantic information have been discovered, the representation and measurement of semantic information have been set up, and the important applications of semantic information in the fields of knowledge expression and organizing (knowledge base architecture), learning, understanding,cognition, and strategy creation have been re-explored, and the framework of semantic information theory has been formulated.

On the other hand, there still exists some open issues.One is the better approach to the quantitative measurement of semantic information.The other is the logic theory needed for supporting the inference based on semantic information.The current theory of mathematic logic is not sufficient and the new and more powerful theory of logic, universal logic and the dialectic logic for instance, are eagerly expected.

ACKNOWLEDGEMENT

The studies on semantic information theory presented here have gained the financial support from the China National Natural Science Foundation.The author would like to express his sincere thanks to this foundation.

Appendix: Note on the Measurement of Semantic Information

In some cases people may concern such kind of problems as how many amount of semantic information is provided, or which one provides more amount of semantic information between two pieces of semantic information?

Evidently, the amount of a piece of semantic information would be closely related to its logic truth.For example, given an atomic semantic informationyn, the semantic information amount it provides would be minimum iftn= 0 whereas be maximum iftn= 1.The parametertnis a fuzzy quantity as was pointed out above.According to [18], the normalized amount of the atomic semantic information can be calculated by

In which

is assumed.The unit of semantic information amount is also “bit”.It is easy to calculate from Eq.(4.2.7) that I(yn) = 1, iftn= 1; I(yn)= -1, iftn= 0; I(yn) = 0, iftn= 1/2; I(yn) > 0, if 1/2

For N-dimensional semantic information vectory, we generally have

Wherefis a complex function whose form depends on the specific logic relation among the atomic semantic informationyn.

In one extreme case where the logic relation among theNpieces of atomic semantic informationyninyis independent to each other, we have

In another case whereNpieces atomic semantic informationynare strictly related to each other, we will have

The equations (4.2.7) ～(4.2.10) are acceptable but not necessary optimal though.

[1] Shannon C E.A Mathematical Theory of Communication, BSTJ, Vol.47, Part I, 379-423; Part II, 632-656; 1948

[2] Kolmogorov A N.Three Approaches to The Quantitative Definition of Information, Internat.J.Comput.Mach., 2, 157-168, 1968

[3] Kolmogorov A N.Logic Basis for Information Theory and Probability Theory.IEEE Trans.Information Theory, Vol.14, p.662-664, 1968

[4] Chaitin G J.Algorithmic Information Theory.Cambridge University Press, Cambridge, 1987

[5] Carnap R & Bar-Hillel Y.Semantic Information.Brit.J.Phil.Sci., 4, 147-157, 1953

[6] Brilluion A.Science and Information Theory.Academic Press, New york, 1956

[7] Bar-Hillel Y.Language ang Information.Read-ing, Mass, 1964

[8] Millikan R.G.Varieties of Meaning, Cambridge,MIT Press, 2002

[9] Stonier J.Informational Content: A Problem of Definition, The Journal of Philosophy, 63(8),201-211, 1966

[10] Gottinger H W.Qualitative Information and Comparative Informativeness, Kybernetik, 13,81, 1973

[11] Floridi L.The Philosophy of Information.Oxford,Oxford University Press, 2011

[12] Saussure F de.Course in General Linguistics,edited By Michael Ryan and Julie Rivkin.Blackwell Publishers, 2001

[13] Peirce C S.Collect Papers: Volume V.Pragmatism and Pragmaticism.Cambridge, MA, USA:Harvard University Press, 1934

[14] Morris C W.Writings on the general theory of signs.The Hague: Mouton, 1971

[15] Wiener N.Cybernetics.John-Wiley and sons,New York, 1948

[16] Bateson G.Steps towards an Ecology of Mind[M].Jason Aronson Inc., 1972

[17] Bertalanffy L V.General System Theory.George Braziller Inc., 1968

[18] Zhong Y X.Principles of Information Science.Beijing University of Posts and Telecommunications Press, 1988, 1996, 2002, 2005, 2013

[19] Zhong Y X.Information Conversion: The integrative Theory of Information, Knowledge, and intelligence, Science Bulletin, Vol.85, No.14,p.1300-1306, 2013

[20] Zhong Y X.Principles of Advanced Artificial Intelligence.Science Press, 2014

[21] Zadeh L A.Fuzzy Sets Theory.Information and Control.8, p.338-353, 1965

[22] Michalski R S, Carbonell J G and Mitchell T M.(Eds.) Machine Learning: An Artificial Intelligence Approach.Vol.1, Morgan Kaufmann, California, 1983

[23] Michalski R S, Carbonell J G and Mitchell T M.(Eds.) Machine Learning: An Artificial Intelligence Approach.Vol.2, Morgan Kaufmann, California, 1986

[24] Zhong Y X.Methodology for NLU based on Comprehensive Information.See: Some Issues in Chinese Information Processing.Ed.By Xu Bo, Science Press, 2013

[25] Nilsson N J.Principles of Artificial Intelligence.Springer-Verlag, New York, 1982

[26] Li W.Mathematical Logic: Basic Principles and Formal Calculus.Science Press, 2007

[27] He H C.Principles of Universal Logic.Science Press, 2001