Numerical Simulation Using GEM for the Optimization Problem as a System of FDEs

In this paper, we introduce a numerical treatment using generalized Euler method (GEM) for the non-linear programming problem which is governed by a system of fractional differential equations (FDEs). The appeared fractional derivatives in these equations are in the Caputo sense. We compare our numerical solutions with those numerical solutions using RK4 method. The obtained numerical results of the optimization problem model show the simplicity and the efficiency of the proposed scheme.

• Nonlinear Programming
• Penalty Function
• Dynamic System
• Caputo Fractional Derivative
• Generalized Euler Method
• RK4 Method

1. Introduction

Fractional differential equations (FDEs) have recently been applied in various areas of engineering, science, finance, applied mathematics, bio-engineering and others. However, many researchers remain unaware of this field [1] . Consequently, considerable attention has been given to the solutions of FDEs of physical interest. Most FDEs do not have exact solutions, so approximate and numerical techniques ( [2] , [3] ), must be used. Recently, several numerical methods to solve FDEs have been given, such as collocation method [4] .

Definition 1.

The Caputo fractional operator D ν of order ν is defined in the following form

D ν ψ ( x ) = 1 Γ ( m − ν )   ∫ 0 x ψ ( m ) ( t ) ( x − t ) ν − m + 1 d t ,     ν > 0 ,   m − 1 < ν ≤ m ,   m ∈ ℕ ,   x > 0

For more details on fractional derivatives definitions and its properties see [5] .

Optimization theory is aimed to find out the optimal solution of problems which are defined mathematically from a model arise in wide range of scientific and engineering disciplines. Many methods and algorithms have been developed for this purpose. The penalty function techniques are classical methods for solving non-linear programming (NLP) problem [6] . In this type of methods the optimization problem is formulated as a system of FDEs so that the equilibrium point of this system converges to the local minimum of the optimization problem ( [7] [8] [9] ).

2. Mathematical Formulation

In this section, we will reformulate the optimization problem as a system of FDEs. For achieve this propose, we will consider the non-linear programming problem with equality constraints defined by

  minimize       ψ ( x ) ,     subject     to       x ∈ D (1)

with

D = { x ∈ ℜ m : h ( x ) = 0 }

where ψ : ℜ m → ℜ and h = ( h 1 , h 2 , ⋯ , h p ) T : ℜ m → ℜ p ( p ≤ m ). It is assumed that the functions in the problem are at least twice continuously differentiable, that a solution exists, and that ∇ h ( x ) has full rank. To obtain a solution of (1), the penalty function method solves a sequence of unconstrained optimization problems. The well-known penalty function for this problem can be defined as follows

Ψ ( x ; μ ) = ψ ( x ) + μ 1 θ ∑ l = 1 p ( h l ( x ) ) θ (2)

for some constant θ > 0 and μ > 0 is an auxiliary penalty variable.

The corresponding unconstrained optimization problem of (2) is defined as follows

minimize       Ψ ( x ; μ ) ,     s .t .     x ∈ ℜ n (3)

Now, we consider the unconstrained optimization problem (3), an approach based on fractional dynamic system can be described by the following FDEs

D ν x ( t ) = − ∇ x Ψ ( x ; μ ) ,   0 < ν ≤ 1 (4)

with the initial conditions x ( t 0 ) = a i ,     i = 1 , 2 , ⋯ , m .

Note that, a point x e is called an equilibrium point of (4) if it satisfies the right hand side of the Equation (4). Also, we can rewrite the fractional dynamic system (4) in more general form as follows

D ν x i ( t ) = χ i ( t , μ ; x 1 , x 2 , ⋯ , x m ) ,   i = 1 , 2 , ⋯ , m (5)

The steady state solution of the non-linear system of FDEs (5) must be coincided with local optimal solution of the NLP problem (1). The existence and the uniqueness of this system of FDEs are studied in more papers [10] . For more details about NLP problem, see [11] .

3. The Proposed Methods

3.1. Generalized Taylor’s Formula

In this subsection, we give the generalization of Taylor’s formula that involves Caputo fractional derivatives [12] . Suppose that

D k ν ψ ( x ) ∈ C ( 0 , a ] ,       for     k = 0 , 1 , ⋯ , n + 1 ,       where     0 < ν ≤ 1

Then we have

ψ ( x ) = ∑ i = 0 n x i ν Γ ( i ν + 1 ) D i ν ψ ( 0 + ) + ( D ( n + 1 ) ν ψ ) ( ξ ) Γ ( ( n + 1 ) ν + 1 ) x ( n + 1 ) ν ,         0 ≤ ξ ≤ x ,   ∀ x ∈ ( 0 , a ] (6)

The generalized Taylor’s Formula (6) can be reduced to the classical Taylor’s formula in case of ν = 1 [13] .

3.2. Generalized Euler Method

In this subsection, we will introduce a generalization of the classical Euler’s method with Caputo derivatives. To achieve this aim, we consider the following IVP in its general form:

D ν ϒ ( t ) = ψ ( t , ϒ ( t ) ) ,     ϒ ( 0 ) = ϒ 0 ,     0 < ν ≤ 1 ,   0 < t < a (7)

In the proposed method we will not find a function ϒ ( t ) that satisfies IVP (7) but we will find a set of points ( t j , ϒ ( t j ) ) and use it for our approximation. For convenience we divide the interval [ 0, a ] into n subintervals [ t j , t j + 1 ] of equal width h = a / n by using the nodes t j = j h , for j = 0 , 1 , ⋯ , n . Assume that ϒ ( t ) , D ν ϒ ( t ) and D 2 ν ϒ ( t ) are continuous on [ 0, a ] and use the generalized Taylor’s Formula (6) to expand ϒ ( t ) about t = t 0 = 0 . For each value t there is a value c 1 so that

ϒ ( t ) = ϒ ( t 0 ) + D ν ϒ ( t 0 ) Γ ( ν + 1 ) t ν + D 2 ν ϒ ( c 1 ) Γ ( 2 ν + 1 ) t 2 ν (8)

Now, when D ν ϒ ( t 0 ) = ψ ( t 0 , ϒ ( t 0 ) ) and h = t 1 are substituted into Equation (8), the result is an expression for ϒ (t1)

ϒ ( t 1 ) = ϒ ( t 0 ) + ψ ( t 0 , ϒ ( t 0 ) ) h ν Γ ( ν + 1 ) + D 2 ν ϒ ( c 1 ) h 2 ν Γ ( 2 ν + 1 )

If the step size h is chosen small enough, then we may neglect the second-order term (involving h 2 ν ) and get

ϒ ( t 1 ) = ϒ ( t 0 ) + ψ ( t 0 , ϒ ( t 0 ) ) h ν Γ ( ν + 1 )

The process is repeated to generate a sequence of points that approximates the solution ϒ ( t ) . The general formula for GEM when t j + 1 = t j + h is given by:

ϒ ( t j + 1 ) = ϒ ( t j ) + ψ ( t j , ϒ ( t j ) ) h ν Γ ( ν + 1 ) ,   j = 0 , 1 , ⋯ , n − 1 (9)

Remark:

1) The generalized Euler’s method (9) is derived by Zaid and Momani for the numerical solution of IVPs with Caputo derivatives [12] .

2) The generalized Euler’s method (9) can be reduced to the classical Euler’s method in the case ν = 1 [12] .

4. Numerical Simulation

In this section, we illustrate the effectiveness of the proposed method and validate the solution scheme for solving the system of FDEs which generated from the non-linear programming problem. To achieve this propose, we consider the following two cases of optimization problems.

Example 1:

Consider the following non-linear programming problem (Optimization problem) ( [14] , [15] )

minimize           ψ ( x ) = 100 ( u 2 − v ) 2 + ( u − 1 ) 2 , subject   to         h ( x ) = u ( u − 4 ) − 2 v + 12 = 0 (10)

The optimal solution is x * = ( 2 , 4 ) , where x = ( u , v ) . For solving the above problem, we convert it to an unconstrained optimization problem with quadratic penalty function (2) for θ = 2 , then we have

Ψ ( x ; μ ) = 100 ( u 2 − v ) 2 + ( u − 1 ) 2 + 1 2 μ ( u ( u − 4 ) − 2 v + 12 ) 2

where μ ∈ ℜ + is an auxiliary penalty variable. The corresponding non-linear system of FDEs from (5) is defined as

D ν u ( t ) = − 400 ( u 2 − v ) u − 2 ( u − 1 ) − μ ( 2 u − 4 ) ( u 2 − 4 u − 2 v + 12 ) , D ν v ( t ) = 200 ( u 2 − v ) + 2 μ ( u 2 − 4 u − 2 v + 12 ) ,   0 < ν ≤ 1 (11)

The initial conditions are u ( 0 ) = 0 and v ( 0 ) = 0 .

Now, we solve numerically this system of non-linear FDEs using the GEM. In view of the GEM, the numerical scheme of the proposed model (11) is given in the following form

u ( t j + 1 ) = u ( t j ) + ψ 1 ( t j , u ( t j ) , v ( t j ) ) h ν Γ ( ν + 1 ) , v ( t j + 1 ) = v ( t j ) + ψ 2 ( t j , u ( t j ) , v ( t j ) ) h ν Γ ( ν + 1 ) (12)

where the quantities ψ 1 ( t j , u ( t j ) , v ( t j ) ) and ψ 2 ( t j , u ( t j ) , v ( t j ) ) are compu- ted from the following functions, respectively, at the points t j = j h ,     j = 0 , 1 , ⋯ , n .

ψ 1 ( t , u ( t ) , v ( t ) ) = − 400 ( u 2 − v ) u − 2 ( u − 1 ) − μ ( 2 u − 4 ) ( u 2 − 4 u − 2 v + 12 ) , ψ 2 ( t , u ( t ) , v ( t ) ) = 200 ( u 2 − v ) + 2 μ ( u 2 − 4 u − 2 v + 12 )

In Figure 1 and Figure 2, we presented the behavior of the numerical solution ( u ( t ) , v ( t ) ) of the problem (11) using GEM and RK4 method at ν = 1 , respectively. From Figure 1, we can see that for ν = 1 our solutions obtained using the proposed method are in excellent agreement with the solution using RK4 method and from Figure 2, we can note that the numerical solution takes the same behavior of the numerical solution as in the case ν = 1 . All computations in this paper are done using Matlab 8.0.

Example 2:

Consider the equality constrained optimization problem ( [14] , [15] )

minimize       ψ ( x ) = ( x 1 − 1 ) 2 + ( x 1 − x 2 ) 2 + ( x 2 − x 3 ) 2                                                   + ( x 3 − x 4 ) 4 + ( x 4 − x 5 ) 4 , subject   to           h 1 ( x ) = x 1 + x 2 2 + x 3 3 − 2 − 3 2 = 0 ,           h 2 ( x ) = x 2 − x 3 2 + x 4 + 2 − 2 2 = 0 ,           h 3 ( x ) = x 1 x 5 − 2 = 0 (13)

The solution of (13) is x * ≈ ( 1.191127,1.362603,1.472818,1.635017,1.679081 ) and this is not an exact solution. For solving the above problem, we convert it to an unconstrained optimization problem with quadratic penalty function (2) for θ = 2 , then we have

Ψ ( x ; μ ) = ψ ( x ) + 1 2 μ ∑ l = 1 3 ( h l ( x ) ) 2 (14)

where μ ∈ ℜ + is an auxiliary penalty variable. The corresponding non-linear system of FDEs from (5) is defined as

D ν x ( t ) = − ∇ ψ ( x ) − μ ∇ h ( x ) h ( x ) ,   0 < ν ≤ 1 (15)

The initial condition is x ( 0 ) = ( 2,2,2,2,2 ) T that is not feasible.

The numerical solution of the proposed optimization problem (13) can be obtained by pursuing the procedure stated in the previous example and solving the resulting nonlinear system of ODEs. The obtained numerical results from the proposed methods are presented in Tables 1-3. In Table 1 and Table 2, we presented the numerical solution x ( t ) = ( x 1 ( t ) , x 2 ( t ) , ⋯ , x 5 ( t ) ) using the GEM and the RK4 method at ν = 1 , respectively. But in Table 3, we presented the numerical solution of the same system (15) using the proposed method with ν = 0.9 . From these tables, we can conclude that our solutions of the proposed method are in excellent agreement with the RK4 method. In addition, we can see that the behavior of the solution is in agreement with the physical meaning of the proposed problem.

Table 1

The numerical solution of Example 2 using GEM at ν = 1

t	x 1 ( t )	x 2 ( t )	x 3 ( t )	x 4 ( t )	x 5 ( t )
0	2	2	2	2	2
2	1.191010	1.359541	1.474039	1.641529	1.679209
10	1.191082	1.362524	1.472778	1.634755	1.679140
15	1.191090	1.362530	1.472774	1.634738	1.679130
20	1.191090	1.362530	1.472774	1.634738	1.679130
30	1.191090	1.362530	1.472774	1.634738	1.679130

Table 2

The numerical solution of Example 2 using the RK4 method at ν = 1

t	x 1 ( t )	x 2 ( t )	x 3 ( t )	x 4 ( t )	x 5 ( t )
0	2	2	2	2	2
2	1.191010	1.359541	1.474039	1.641529	1.679209
10	1.191082	1.362524	1.472778	1.634755	1.679140
15	1.191090	1.362530	1.472774	1.634738	1.679130
20	1.191090	1.362530	1.472774	1.634738	1.679130
30	1.191090	1.362530	1.472774	1.634738	1.679130

Table 3

The numerical solution of Example 2 using GEM at ν = 0.9

t	x 1 ( t )	x 2 ( t )	x 3 ( t )	x 4 ( t )	x 5 ( t )
0	2	2	2	2	2
2	1.198931	1.369223	1.468744	1.616076	1.668076
10	1.191090	1.362530	1.472774	1.634738	1.679140
15	1.191090	1.362530	1.472774	1.634738	1.679130
20	1.191090	1.362530	1.472774	1.634738	1.679130
30	1.191090	1.362530	1.472774	1.634738	1.679130

5. Conclusion

We implemented the GEM for studying the numerical solution for the system of FDEs which described the NLP model. This work is devoted to introduce a study of the behavior of the numerical solution of the proposed problem for various fractional Brownian motions and also for standard motion ν = 1 . Also, we compared the obtained numerical solutions with those numerical solutions using RK4 method. From this study, we can see that the obtained numerical solution using the suggested method is in excellent agreement with the numerical solution using RK4 method and shows that this approach can be solved the problem effectively and illustrates the validity and the great potential of the proposed technique. Finally, the recent appearance of FDEs as models in some fields of applied mathematics makes it necessary to investigate analytical and numerical methods for such equations.

Acknowledgements

We thank the Editor and the Referees for their comments.

References

1. Podlubny, I. (1999) Fractional Differential Equations. Academic Press, New York.

2. Khader, M.M. and Adel, M. (2016) Numerical Solutions of Fractional Wave Equations Using an Efficient Class of FDM Based on Hermite Formula. Advances in Difference Equations, 2016, 1-10. https://doi.org/10.1186/s13662-015-0731-0

3. Sweilam, N.H. and Khader, M.M. (2010) A Chebyshev Pseudo-Spectral Method for Solving Fractional Integro-Differential Equations. ANZIAM, 51, 464-475. <br />https://doi.org/10.1017/S1446181110000830

4. Khader, M.M. (2011) On the Numerical Solutions for the Fractional Diffusion Equation. Communications in Nonlinear Science and Numerical Simulations, 16, 2535-2542. <br />https://doi.org/10.1016/j.cnsns.2010.09.007

5. Oldham, K.B. and Spanier, J. (1974) The Fractional Calculus. Academic Press, New York.

6. Luenberger, D.G. (1973) Introduction to Linear and Nonlinear Programming. Addison-Wesley, Boston.

7. ?zdemir, N. and Evirgen, F. (2010) A Dynamic System Approach to Quadratic Programming Problems with Penalty Method. Bulletin of the Malaysian Mathematical Sciences Society, 10, 79-91.

8. K.J. Arrow, Hurwicz L. and Uzawa, H. (1958) Studies in Linear and Non-Linear Programming. Stanford University Press, Redwood City.

9. Yamashita H. ,et al. (1976)Differential Equation Approach to Nonlinear Programming Mathematical Programming 1, 155-168.

10. Sun, W. and Yuan, Y. (2006) Optimization Theory and Methods: Nonlinear Programming. Springer-Verlag, New York.

11. Fiacco, A.V. and Mccormick, G.P. (1968) Nonlinear Programming: Sequential Unconstrained Minimization Techniques. John Wiley, New York.

12. Odibat, Z.M. and Shawagfeh, N.T. (2007) Generalized Taylor’s Formula. Applied Mathematics and Computation, 186, 286-293. https://doi.org/10.1016/j.amc.2006.07.102

13. Arafa, A.A.M., Rida, S.Z. and Khalil, M. (2012) Fractional Modeling Dynamics of HIV and CD4+ T-Cells during Primary Infection. Nonlinear Biomedical Physics, 6, 1-7. <br />https://doi.org/10.1186/1753-4631-6-1

14. Schittkowski, K. (1987) More Test Examples for Nonlinear Programming Codes. Springer-Verlag, Berlin.

15. Khader, M.M., Sweilam, N.H. and Mahdy, A.M.S. (2013) Numerical Study for the Fractional Differential Equations Generated by Optimization Problem Using Chebyshev Collocation Method and FDM. Applied Mathematics and Information Science, 7, 2011-2018.