Uncertain Bang-Bang optimal control for multi-stage uncertain linear quadratic systems

KANG Yujie

( School of Mathematics and Statistics, Zhoukou Normal University, Zhoukou 466001,China )



Uncertain Bang-Bang optimal control for multi-stage uncertain linear quadratic systems

KANG Yujie

( School of Mathematics and Statistics, Zhoukou Normal University, Zhoukou 466001,China )

Abstract:In a multi-stage system, when the state of the system at a stage is derived from the state of the former stage and disturbed by an uncertain variable, a multi-stage uncertain optimal control problem is proposed. The idea of considering optimizing the expected value of an uncertain objective is adopted in this paper. Based on Bellman’s principle of optimality, the Bang-Bang optimal controls for the uncertain optimal control problem with a quadratic objective function subject to an uncertain linear system can be obtained. At every stage, the boundary of the optimal objective values can only be gotten, so the upper limit value is taken as the approximation of the optimal objective values.

Key words:multi-stage system; uncertain optimal control; Bellman’s principle of optimality; Bang-Bang optimal control

An optimal control problem for a multi-stage system is to choose the best decision such that an objective function is optimized. And this problem has very important applications in practice. The study of optimal control initiated in 1970s such as in Merton for finance[1]. Some investigations on optimal control of Brownian motion or stochastic differential equations and applications in finance may be found in some books such as Fleming and Reshelf, Harrison and Karatzas[2-4]. One of the main methods to study optimal control is based on dynamic programming. The use of dynamic programming in optimization over Itos process is discussed such as in Dixit and Pindyck[5].

The study of Bang-Bang control initiated in 1960s such as the Pontryagin's Maximum Principle[6-7]of the minimum time problems and in Dorato[8]of linear stochastic systems. Some investigations on Bang-Bang optimal control of Brownian motion or stochastic differential equations and applications in time and flue problems may be found in some passages such as A.V.Balakrishnan, S.A.Vakhrameev and Johansen[9-11].

In this paper, we will introduce an uncertain Bang-Bang optimal control for multi-stage uncertain linear quadratic systems. In the first section, based on Bellman's principle of optimality, we will introduce and deal with an uncertain Bang-Bang optimal control problem for a multi-stage uncertain system. In next section, we will review some concepts such as uncertain measure, uncertainty space, uncertainty distribution, normal uncertainty distribution of normal variable, expected value of uncertain variable. In section 3, we will introduce an uncertain optimal control problem, a recurrence equation optimality which was presented by [12] and [13] for solving the problem by using dynamic programming. In section 4, we will obtain the uncertain Bang-Bang optimal controls and the optimal objective values for the multi-stage uncertain optimal control problem. In section 5, we will give an example and get its uncertain Bang-Bang optimal controls and the optimal values.

1Preliminary

In convenience, we give some useful concepts at first. Let Γ be a nonempty set, and L is a σ-algebra over Γ. Each element Λ∈ L is called an event.

Definition 1[14]An uncertain variable ξ is called normal if it has a normal uncertainty distribution

(1)

Definition 2[15]The uncertain variables ξ1,ξ2,…,ξmare said to be independent if

for any Borel sets B1,B2,…,Bmof real numbers.

Theorem 1 (Linearity of Expected Value Operator)[16]Let ξ and η be independent uncertain variables with finite expected values. Then for any real numbers a and b, we have

2Problem of Uncertain Optimal Control

An optimal control problem for a multi-stage (crisp) system is to choose the best decision such that an objective function is optimized subject to a multi-stage system. In this paper we will investigate the following uncertain expected value optimal control problem for a multi-stage uncertain system:

(2)

where x(j) is the state of the system at stage j, u(j) is the control variable at stage j, Ujis the constraint domain for the control variables u(j)for j=0,1,2, …,f is the objective function φ and σ are two functions, and x0is the initial state of the system. In addition, C1,C2,…,CNare some uncertain processes.

Applying Bellman’s principle of optimality, we obtain the following recurrence equation to solve the problem (2).

Theorem 2[12]For problem (2), we have the following recurrence equation

(3)

For k=N-1,N-2,…,1,0.

Theorem 2 tells us that to solve the problem (2) turns to solve the simpler problem (3) step by step from the last stage to the initial stage in reverse order.

Then for any real number a, we have

(4)

3Uncertain Linear Quadratic Optimal Control

By using the recurrence equation (3), we will obtain the exact solution for the following uncertain optimal control problem with a quadratic objective function subject to an uncertain linear system:

(5)

for all j. Then we get

(6)

The optimal controls are

|u*(N)|1,

And the optimal values are

where

PN=AN,

QN=0,

RN=0,

for k=N-1,N-2,…,1,0,

Proof

Denote the optimal control for the above problem by u*(0),u*(1),…,u*(N). By using the recurrence equation (3), we have

where

then

For k=N-1, by using the recurrence equation (3), we have

(7)

Denote

then

Since uncertain process CNis a canonical process. It follows from (4) and (6), we have

(8)

Substituting (8) into (7) yields that

and

Let

(9)

Then

(10)

(11)

Now we solve it as follow:

Let

(12)

It follows from

that

(13)

which is the minimum point of the function HN-1because

That is, if

So we have

Figure 1　Three types of functions H(u)

4Numerical example

In previous section, we studied an uncertain Bang-Bang optimal control problem for a quadratic objective function subject to an uncertain linear system. For that problem, we can get the Bang-Bang optimal controls and the optimal values with the state of the system at all stages. As an application, by using the result which had been showed in previous section, we consider the following example:

where coefficients are listed in Table 1. In addition, an uncertain process Cjis a canonical process whose uncertainty distribution is

for all j. Then

(14)

The optimal controls and optimal values are obtained by Theorem 4 and listed in Table 2. The data in the fourth column of Table 2 is the corresponding states which are derived from x(k+1)=akx(k)+bku*(k)+σk+1Ck+1for initial stage x(0)=1, where ck+1is the realization of uncertain variable Ck+1, and may be generated by ck+1=2rk+1-1 for a random number rk+1∈[0,1](k=0,1,…,9).

Table 2　The optimal results

5Conclusion

In this paper, we considered a multi-stage system, when the state of the system at a stage is derived from the state of the former stage and disturbed by an uncertain variable, a multi-stage uncertain optimal control problem is proposed. Based on Bellman's principle of optimality, the Bang-Bang optimal controls for the uncertain optimal control problem with a quadratic objective function subject to an uncertain linear system can be obtained. Finally, we gave an example and get its uncertain Bang-Bang optimal controls and the optimal values.

References：

[1] Merton R C. Optimal consumption and portfolio rules in a continuous time model [J]. Journal of Economic Theory, 1971, 3: 373-413.

[2] Fleming W H,Rishel R W. Deterministic and Stochastic Optimal Control [M]. New York: Springer-Verlag, 1986.

[3] Harrison J M. Brownian Motion and Stochastic Flow Systems[M]. New York: John Wiley & Sons, 1985.

[4] Karatzas I. Optimization problems in the theory of continuous trading[J]. SIAM Journal on Control and Optimization, 1989, 27(6): 1221-1259.

[5] Dixit A K, Pindyck R S. Investment under Uncertainty[M]. Princeton: Princeton University Press, 1994.

[6] M Athans,P L Falb. Optimal Control[M]. McGrawHifl, 1966.

[7] Xie Xueshu. The theory and application of Bang-Bang optimal control[M]. Princeton: qinghua University Press, 1986.

[8] Peter Dorato, Changming Hsieh, Prentiss N Roblnson. Optimal Bang-Bang Control of Linear Stochastic Systems with a Small Noise Parameter[J]. IEEE Transactions on Automatic Control, 1967, AC-12(6): 432-435.

[9] A V Balakrishnan. On Stochastic Bang-Bang Control[J]. Applied Mathematics and Optimization, 1980, 6: 91-96.

[10] S A Vakhrameev. A Bang-Bang Theorem With A Finite Number Of Swiches For Nonlinear Control System[J]. Journal of Mathematical Sciences, 1997, 85(3): 950-95.

[11] S Johansen. The Bang-Bang Problem for Stochastic Matrices[J]. z.Wahrscheinlichkeitsth eorie verw. Geb, 1973, 26(1): 191-195.

[12] Zhu Y. Fuzzy Optimal Control for Multi-stage Fuzzy Systems[J]. IEEE Transactions on Fuzzy Systems, Man and Cybernetics-Part B, 2011, 41(4): 964-975.

[13] Zhu Y. Uncertain Optimal Control with Application to a Portfolio Selection Model[J]. Cybernetics and Systems, 2010, 41(7): 535-547.

[14] Liu B. Uncertainty Theory, 2nd ed[M]. Berlin:Springer-Verlag, 2007.

[15] Liu B. Some research problems in uncertainty theory[J]. Journal of Uncertain Systems, 2009, 3(1): 3-10.

[16] Liu B. Uncertainty Theory: A Branch of Mathematics for Modeling Human Uncertainty[M]. Berlin:Springer-Verlag, 2010.

CLC number:O224

Document code:AArticle ID： 1671-9476(2016)02-0026-08

DOI：10.13450/j.cnki.jzknu.2016.02.006