Physical-Informed Neural Networks (PINNs) for Solving Shape Optimization Problems

In this paper, we use Physics-Informed Neural Networks (PINNs) to solve shape optimization problems. These problems are based on incompressible Navier-Stokes equations and phase-field equations. The phase-field function is used to describe the state of the fluids, and the optimal shape optimization is obtained by using the shape sensitivity analysis based on the phase-field function. The sharp interface is also presented by a continuous function between zero and one with a large gradient. To avoid the numerical solutions falling into the trivial solution, the hard boundary condition is implemented for our PINNs’ training. Finally, numerical results are given to prove the feasibility and effectiveness of the proposed numerical method.

• PINNs
• Phase-Field
• Shape Optimization
• Incompressible Navier-Stokes Equations

1. Introduction

Shape optimization for the fluid flows is widely interested in the optimum designs [1]-[4]. To obtain minimum drag wing profiles [5] or minimum pressure drop diffusers [6], when striving for a decreased drag or pressure drop, the feasible design modifications have concerned adjusting selected parts of the boundary to the fluid region. Borrvall and Petersson [7] optimized the dissipated power of the stokes flow. Allaire et al. [8] minimized the compliance of elastic structure.

Let u and p be the velocity and pressure of the incompressible flow which is governed by the incompressible Navier-Stokes equations in a domain Ω . To obtain optimal shape in Ω , the purpose is to find an optimal shape Ω * in certain admissible shape set Ω s e t ∈ ℝ * , d = 2 , 3 , satisfying

J ( Ω * ) = min Ω s e t J ( Ω ) , (1)

where J ( Ω ) = ∫ Ω j ( u , p ) d x is constantly referred to as objective functional or cost functional. A typical choice of J ( Ω ) used in this context is the total potential energy, which is given by j ( u , p ) = 0.5 μ ∇ u ⋅ ∇ u [4], which is used to represent the shear stress and influences the damage of fluid flows on the optimized shape surface [9]. In the framework of the model, the incompressible Navier-Stokes equations for the absence of body fluid forces is

{ − μ Δ u + ( u ⋅ ∇ ) u + ∇ p = 0 in   Ω , ∇ ⋅ u = 0 in   Ω , (2)

where the parameter μ is the viscosity of the fluid. The inflow boundary Γ i n and outflow boundary Γ o u t are fixed during the optimization, where inflow velocity u i n is specified. The no-slip or free-slip boundary ξ is the concerned part for an optimized shape.

There are many numerical methods for solving the shape optimization problem Equations (1), (2). Among those methods, the phase-field method is an outstanding approach for various free boundary problems [10] [11], topology optimization for elastic structure [12] [13], and photonic devices [14] [15]. The phase field method stands out in its treatment of the interface as a physically diffuse thin layer. The interface is sharp conceptually but regularized numerically by a continuous function between zero and one. Due to the generation of the interface, mixing energy is introduced to govern the dynamics of the diffusive interface. Wallin [13] and Garcke [4] use the Ginzburg-Landau energy as the mixing energy over the interfacial layer, but the mixing energy can be dated back to Van Der Waals energy W .

Neural networks are mathematical approximations that have been effectively applied in various fields, including speech recognition and classification tasks [16]-[18]. Neural networks have been applied to multiple scientific problems, such as turbulence model design from data [19] and for simulating heat transfer problems [20]. Among the methods that use deep learning for scientific computing, Physics-Informed Neural Networks (PINNs) [21] is a popular approach for simulating physical problems. Leverage all the information available about the problem, such as the underlying PDEs, boundary conditions, physical properties, and known measurement data, to train a neural network that best approximates the model. These networks transform boundary-value problems into optimization problems, providing flexibility to include any constraint in the problem. PINNs have been applied to a variety of forward, inverse, and parameterized problems [20] [22] [23], including fluid dynamics problems.

The remainder of this paper is organized as follows. In the next section, we introduce the phase-field model and its shape sensitivity analysis. The architecture of the PINNs and the loss functions used in training are presented in Section 3. We present the experimental results in Section 4. Finally, we provide the conclusions in Section 5.

2. Phase-Field Model for Fluid Shape Optimization

The fundamental idea of the phase-field model is to assume that the interface is a diffusive thin layer, as shown in the figure of Figure 1. We define the phase-field function ϕ over the whole design domain D to regularize numerically the sharp conceptually, which consists of three different parts of Figure 1: the material domain Ω with ϕ = 1 , the void domain D \ Ω with ϕ = 0 , and a fictitious diffused interface ξ with 0 < ϕ < 1 .

In the framework of the phase-field method, Equation (2) should be extended to the design domain D . The basic idea of a porous medium approach [4] [7] [9] is to add the term ϕ ( 1 − ϕ ) u to Equation (2), then Equation (2) ‘interpolate’ between the steady-state Navier?Stokes equations where ϕ ( x ) = 1 and a virtual Darcy flows through a porous medium where ϕ ( x ) = 0 . Thus we get

{ − μ Δ u + ( u ⋅ ∇ ) u + ∇ p + α ( 1 − ϕ ) = 0 in   D , ∇ ⋅ u = 0 in   D , (3)

where the parameter α penalizes the phase field. The boundary of D is also extended into three parts: Γ D = Γ i n + Γ o u t + Γ w a l l , where homogeneous boundary conditions are supposed for the fluid flow. The inflow boundary Γ i n and the outflow boundary Γ o u t are fixed during the optimization.

To ensure that the velocity vanishes in the void domain D \ Ω , an additional penalization term [4] is added to the objective function (1). Thus, the new objective function is rewritten as

J ( ϕ , u ) = ∫ D j ( ϕ , u ) d x = 1 2 ∫ D μ ∇ u : ∇ u d x + α 2 ∫ D ( 1 − ϕ ) | u | 2 d x . (4)

The phase-field model for fluid shape optimization is to find a ϕ * to minimize the Van Der Waals free energy, given by

W = ∫ D k 2 | Δ ϕ | 2 + f ( ϕ , u ) d x , (5)

with the constraints of state Equation (3). the k characters are the width of the diffusive thin layer and f ( ϕ , u ) is responsible for the moving direction of interfaces. Following the suggestion of Takezawa in [12], it is

f ( ϕ , u ) = 1 4 w ( ϕ ) + 30 ∂ J ( ϕ , u ) ∂ ϕ g ( ϕ ) , (6)

where w ( ϕ ) = ϕ 2 ( 1 − ϕ ) 2 and g ( ϕ ) = ϕ 3 ( 6 ϕ 2 − 15 ϕ + 10 ) are two sets of basis functions. To fulfill the movement of the interface with a diffusive thin layer in the direction indicated by ∂ J ( ϕ , u ) ∂ ϕ , w ( 0 ) = w ( 1 ) = w ′ ( 0 ) = w ′ ( 1 ) = 0 and g ( 0 ) = 0 , g ( 1 ) = 1 are the necessary conditions.

Authors presented a new objective function (4) in reference [24], Then the adjoint equation of (3) is rewritten as follows

{ − μ ( Δ w + Δ u ) + ( w ⋅ ∇ ) u − ( u ⋅ ∇ ) w − ∇ q + α ( 1 − ϕ ) ( w + | u | ) = 0 in   D , ∇ ⋅ w = 0 in   D , w = 0 on   Γ i n ∪ Γ w a l l , μ ∇ w ⋅ n + ( u ⋅ n ) w + q n = 0 on   Γ o u t , (7)

where w and q are the adjoint functions. n is the unit outward normal vector. The shape sensitivity of the objective function is given as follows

∂ J ( ϕ , u ) ∂ ϕ = − α 2 ∫ D | u | 2 d x + α ∫ D u ⋅ w d x . (8)

The gradient flow method ∂ ϕ ∂ t = − ∂ W ∂ ϕ is used to find the minimum of the Equations (4) and (5). From this, it can be obtained the phase-field equation

∂ ϕ ∂ t = k Δ ϕ + ϕ ( ϕ − 1 ) { ϕ − 1 2 − 900 ∂ J ( ϕ , u ) ∂ ϕ ( 1 − ϕ ) ϕ } . (9)

Moreover, we can simplify the model in the steady state. In this paper, we consider the problem with steady state, i.e. ∂ ϕ ∂ t = 0 . So we can get the phase-field model for fluid shape optimization as follows: Find [ u , p , w , q , ϕ ] satisfies

{ − μ Δ u + ( u ⋅ ∇ ) u + ∇ p + α ( 1 − ϕ ) = 0 in   D , − μ ( Δ w + Δ u ) + ( w ⋅ ∇ ) u − ( u ⋅ ∇ ) w − ∇ q + α ( 1 − ϕ ) ( w + | u | ) = 0 in   D , k Δ ϕ + ϕ ( ϕ − 1 ) { ϕ − 1 2 − 900 ∂ J ( ϕ , u ) ∂ ϕ ( 1 − ϕ ) ϕ } = 0 in   D , ∂ J ( ϕ , u ) ∂ ϕ = − α 2 ∫ D | u | 2 d x + α ∫ D u ⋅ w d x in   D , ∇ ⋅ u = 0 in   D , ∇ ⋅ w = 0 in   D , (10)

and boundary conditions

{ μ ∇ w ⋅ n + ( u ⋅ n ) w + q n = 0 on   Γ o u t , w = 0 on   Γ i n ∪ Γ w a l l . (11)

3. Numerical Methods

In this section, we present the numerical method for solving the shape optimization problem (10), (11). To gain a better understanding of neural networks, we consider a simple example of a one-layer neural network. If x is the input vector and y 1 is the output vector to have dimensions n and m , respectively, the output can be expressed as

y 1 = σ ( W x + b ) , (12)

where W is a weight matrix of dimension m × n and b is a bias vector of dimension m . The activation function σ ( z ) is user-defined and can take various forms, such as a cosine or a hyperbolic tangent. This function is applied to each component of a vector z . By stacking multiple layers, we can establish a feedforward neural network as illustrated in Figure 2. The output of the network is then expressed as

N θ ( x ) = σ l ( W l σ l − 1 ( W l − 1 σ l − 2 ( ⋯ ( σ 1 ( W 1 x + b 1 ) ) ⋯ ) ) ) , (13)

where l and θ represent the number of layers and all the parameters of the neural network, respectively. One can build a more complex neural network by changing the connections between the layers.

Denoted the differential operator about PDEs in regions D and ∂ D by F and B , the phase-field model can be expressed as

{ F [ U ( x ) ] = 0 x ∈ D , B [ U ( x ) ] = b ( x ) x ∈ ∂ D . (14)

Once the problem has been described, the next step core is using a formulation of the physics-informed loss function to determine the optimal values for the parameter θ . The loss function L ( x , θ ) , constructed by evaluating the residuals about the governing PDEs ( L f ( x , θ ) ) and boundary conditions ( L b ( x , θ ) ) , quantifies the network’s deviation from the physical laws encapsulated by the governing differential equations and are defined as follows

{ L f ( x ; θ ) = F [ N θ ( x ) ] − 0 x ∈ D , L b ( x ; θ ) = B [ N θ ( x ) ] − b ( x ) x ∈ ∂ D , (15)

The aggregated loss function for the PINNs, combining these residuals, is given by

L ( x ; θ ) = 1 N D ∑ i = 1 N D ‖ L f ( x i ; θ ) ‖ 2 + 1 N Γ ∑ j = 1 N Γ ‖ L b ( x i ; θ ) ‖ 2 , (16)

where N Ω , N Γ denote the number of collocation points in D and on ∂ D . And ‖   ⋅   ‖ is the discrete L 2 norm. With the PINNs, we approximate the solution of the shape optimization problem using a feedforward neural network.

While various optimization methods could address this minimization problem, in the context of this work, we will only consider gradient-based optimization methods. Gradient-based methods leverage the property that the gradient of a function points in the direction of the function’s steepest. To minimize the function, one would generally move in the direction opposite to the gradient, with a step size proportional to a user-defined learning rate. Popular gradient-based methods include the Stochastic Gradient Descent (SGD), the Limited-memory Broyden-Fletcher-Goldfarb-Shanno method(L-BFGS) [25], and the Adaptive Moment Estimation method (Adam) [26]. The gradient is calculated using the back-propagation algorithm, which employs the chain rule and automatic differentiation to compute the partial derivatives of the output concerning the weights, the biases, and the network’s input.

Since the problem includes integral terms, we incorporate Gaussian integration in the training of PINNs to approximate the calculation of shape sensitivity (8). During the training process, we implement hard boundary conditions in the PINNs’ boundary conditions. This can prevent excessive void domain where ϕ = 0 from causing trivial numerical solutions. The training flow chart of PINNs is shown in the following Figure 3.

4. Numerical Results

In this section, we first present an accuracy test of the algorithm by comparing the numerical results obtained from the PINNs with the exact solution. Our numerical algorithm exhibits excellent convergence properties for models with known exact solutions. Next, we conduct numerical simulations for practical problems, presenting the optimized forms of the pipeline under different boundary conditions.

4.1. Accuracy Test

In the two-dimensional shape optimization domain D = [ 0 , 1 ] 2 , we take the following exact solution

{ u e x a ( x ) = ( π s i n ( 2 π y ) s i n 2 ( π x ) , − π s i n ( 2 π x ) s i n 2 ( π y ) ) , w e x a ( x ) = ( π s i n ( 2 π y ) s i n 2 ( π x ) , − π s i n ( 2 π x ) s i n 2 ( π y ) ) , p e x a ( x ) = q e x a ( x ) = ϕ e x a ( x ) = cos ( π x ) sin ( π y ) . (17)

This will be used to evaluate the accuracy and convergence behavior of the shape optimization problem. To enhance the training effectiveness, we will randomly select 81 points within D to construct the loss functions for p and q , which will be incorporated into the experimental model:

L p q ( x ; θ ) = ∑ i = 1 81 ‖ p e x a ( x i ) − p N θ ( x i ) ‖ 2 + ∑ i = 1 81 ‖ q e x a ( x i ) − q N θ ( x i ) ‖ 2 . (18)

Substituting the numerical solution (17) into the model, we obtain the residual of the right-hand side term of the corresponding equation and set the parameters: α = 10 , k = 0.0006 , μ = 0.025 , u i n ( 0 , y ) = ( 0 , 0 ) . We selected 2601 training points and 10,000 Gaussian points in D and solved the experimental results using PINNs with four layers, each containing fifty neurons. The Adam and L-BFGS optimizers will be used for 10,000 and 700,000 iterations, respectively, with a learning rate of 0.5 × 10 − 3 .

Figure 4(a) and Figure 4(b) are exact solution images, and Figure 4(c) and Figure 4(d) are numerical solution images. By observing the error images in Figure 4(e), Figure 4(f), and the loss value images in Figure 5, we can see that the numerical solutions converge to the exact solution with high accuracy.

4.2. The Shape Optimization Problem for Pipes

Now, we consider the design of a bend, and the purpose of designing work is to find an optimized connection between the inlet and outlet [27]. The domain is shown in Figure 6(a). Set the parameters: α = 100 , k = 0.0006 , μ = 0.025 , u i n = ( 100 ( 0.9 − y ) ( y − 0.7 ) , 0 ) . We selected 2601 training points and 10,000 Gaussian points in D and solved the experimental results using a PINN with four layers, each containing fifty neurons. The Adam and L-BFGS optimizers will be used for 10,000 and 200,000 iterations, respectively, with a learning rate of 0.5 × 10 − 3 . To avoid converging to a trivial solution, boundary conditions are handled as hard boundaries.

We find that the shape and loss in Figure 6(b), Figure 6(c) are in the same direction as those reported in the literature (see [27] and references therein) and exhibit excellent convergence performance.

As shown in Figure 7(a), we set the inlet to be twice that of the pipe experiment and conducted another expansion experiment for prediction. Set the parameters: α = 10 , k = 0.0006 , μ = 0.025 , u i n = ( 100 ( 0.9 − y ) ( y − 0.5 ) , 0 ) . We select 2601 training points and 10,000 Gaussian points and use PINNs with four layers, each containing fifty neurons. The Adam and L-BFGS optimizers will be used for 10,000 and 200,000 iterations, respectively, with a learning rate of 0.5 × 10 − 3 . To avoid converging to a trivial solution, boundary conditions are handled as hard boundaries. Figure 7(b) and Figure 7(c) present our results, showing that the model exhibits excellent convergence.

5. Conclusion

In this paper, we use PINNs with hard boundary conditions to solve the phase-field model for fluid shape optimization. The accuracy test and the pipe experiment show that the established phase-field model aligns well with results from the literature, and the convergence achieved is also accurate. Moreover, we provide the expansion experiment result, demonstrating that using PINNs with hard boundaries to solve the phase-field model is effective and feasible for fluid shape optimization problems.

Funding

This work was supported by the NSF of China (Grant No. 11901100) and the Natural Science Foundation of Fujian Province of China (Grant No. 2020J05111).

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

1. Pironneau, O. (1982) Optimal Shape Design for Elliptic Systems. In: Drenick, R.F. and Kozin, F., Eds., System Modeling and Optimization, Springer, 42-66. https://doi.org/10.1007/bfb0006123

2. Mohammadi, B. and Pironneau, O. (2009) Applied Shape Optimization for Fluids. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199546909.001.0001

3. Guest, J.K. and Prévost, J.H. (2006) Topology Optimization of Creeping Fluid Flows Using a Darcy-Stokes Finite Element. International Journal for Numerical Methods in Engineering, 66, 461-484. https://doi.org/10.1002/nme.1560

4. Garcke, H., Hecht, C., Hinze, M. and Kahle, C. (2015) Numerical Approximation of Phase Field Based Shape and Topology Optimization for Fluids. SIAM Journal on Scientific Computing, 37, A1846-A1871. https://doi.org/10.1137/140969269

5. Pironneau, O. (1974) On Optimum Design in Fluid Mechanics. Journal of Fluid Mechanics, 64, 97-110. https://doi.org/10.1017/s0022112074002023

6. &#199;labuk, H. and Modi, V. (1992) Optimum Plane Diffusers in Laminar Flow. Journal of Fluid Mechanics, 237, 373-393. https://doi.org/10.1017/s0022112092003458

7. Borrvall, T. and Petersson, J. (2002) Topology Optimization of Fluids in Stokes Flow. International Journal for Numerical Methods in Fluids, 41, 77-107. https://doi.org/10.1002/fld.426

8. Allaire, G., Jouve, F. and Toader, A. (2004) Structural Optimization Using Sensitivity Analysis and a Level-Set Method. Journal of Computational Physics, 194, 363-393. https://doi.org/10.1016/j.jcp.2003.09.032

9. Zhang, B. and Liu, X. (2014) Topology Optimization Study of Arterial Bypass Configurations Using the Level Set Method. Structural and Multidisciplinary Optimization, 51, 773-798. https://doi.org/10.1007/s00158-014-1175-y

10. Shen, J. and Yang, X. (2009) An Efficient Moving Mesh Spectral Method for the Phase-Field Model of Two-Phase Flows. Journal of Computational Physics, 228, 2978-2992. https://doi.org/10.1016/j.jcp.2009.01.009

11. Sun, Y. and Beckermann, C. (2007) Sharp Interface Tracking Using the Phase-Field Equation. Journal of Computational Physics, 220, 626-653. https://doi.org/10.1016/j.jcp.2006.05.025

12. Takezawa, A., Nishiwaki, S. and Kitamura, M. (2010) Shape and Topology Optimization Based on the Phase Field Method and Sensitivity Analysis. Journal of Computational Physics, 229, 2697-2718. https://doi.org/10.1016/j.jcp.2009.12.017

13. Wallin, M., Ristinmaa, M. and Askfelt, H. (2011) Optimal Topologies Derived from a Phase-Field Method. Structural and Multidisciplinary Optimization, 45, 171-183. https://doi.org/10.1007/s00158-011-0688-x

14. Takezawa, A. and Kitamura, M. (2014) Phase Field Method to Optimize Dielectric Devices for Electromagnetic Wave Propagation. Journal of Computational Physics, 257, 216-240. https://doi.org/10.1016/j.jcp.2013.09.051

15. Wu, S., Hu, X. and Zhu, S. (2018) A Multi-Mesh Finite Element Method for Phase-Field Based Photonic Band Structure Optimization. Journal of Computational Physics, 357, 324-337. https://doi.org/10.1016/j.jcp.2017.12.031

16. Alam, M., Samad, M.D., Vidyaratne, L., Glandon, A. and Iftekharuddin, K.M. (2020) Survey on Deep Neural Networks in Speech and Vision Systems. Neurocomputing, 417, 302-321. https://doi.org/10.1016/j.neucom.2020.07.053

17. Chai, J., Zeng, H., Li, A. and Ngai, E.W.T. (2021) Deep Learning in Computer Vision: A Critical Review of Emerging Techniques and Application Scenarios. Machine Learning with Applications, 6, Article ID: 100134. https://doi.org/10.1016/j.mlwa.2021.100134

18. Nassif, A.B., Shahin, I., Attili, I., Azzeh, M. and Shaalan, K. (2019) Speech Recognition Using Deep Neural Networks: A Systematic Review. IEEE Access, 7, 19143-19165. https://doi.org/10.1109/access.2019.2896880

19. Beck, A., Flad, D. and Munz, C. (2019) Deep Neural Networks for Data-Driven LES Closure Models. Journal of Computational Physics, 398, Article ID: 108910. https://doi.org/10.1016/j.jcp.2019.108910

20. Cai, S., Wang, Z., Wang, S., Perdikaris, P. and Karniadakis, G.E. (2021) Physics-Informed Neural Networks for Heat Transfer Problems. Journal of Heat Transfer, 143, Article ID: 060801. https://doi.org/10.1115/1.4050542

21. Raissi M. ,et al. (2018)Deep Hidden Physics Models: Deep Learning of Non-Linear Partial Differential Equations Journal of Machine Learning Research 19, 1-24.

22. Jin, X., Cai, S., Li, H. and Karniadakis, G.E. (2021) NSFnets (Navier-Stokes Flow Nets): Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations. Journal of Computational Physics, 426, Article ID: 109951. https://doi.org/10.1016/j.jcp.2020.109951

23. Lim, J. and Psaltis, D. (2022) MaxwellNet: Physics-Driven Deep Neural Network Training Based on Maxwell’s Equations. APL Photonics, 7, Article ID: 011301. https://doi.org/10.1063/5.0071616

24. Li, F. and Hu, X. (2019) A Phase-Field Method for Shape Optimization of Incompressible Flows. Computers & Mathematics with Applications, 77, 1029-1041. https://doi.org/10.1016/j.camwa.2018.10.029

25. Bollapragada, R., Nocedal, J., Mudigere, D., Shi, H.J. and Tang, P.T.P. (2018) A Progressive Batching L-BFGS Method for Machine Learning. Proceedings of the 35th International Conference on Machine Learning, Stockholm, 10-15 July 2018, 620-629.

26. Kingma, D. and Ba, J. (2014) Adam: A Method for Stochastic Optimization. https://api.semanticscholar.org/CorpusID:6628106

27. Duan, X., Ma, Y. and Zhang, R. (2008) Shape-Topology Optimization for Navier-Stokes Problem Using Variational Level Set Method. Journal of Computational and Applied Mathematics, 222, 487-499. https://doi.org/10.1016/j.cam.2007.11.016