Grantee Research Project Results
1998 Progress Report: Parallel Least-Squares Finite Element Method for Large Eddy Simulation of Large Scale Environmental Flows and Transport Processes
EPA Grant Number: R825200Title: Parallel Least-Squares Finite Element Method for Large Eddy Simulation of Large Scale Environmental Flows and Transport Processes
Investigators: Tsang, Tate H. , Yost, Scott A. , Bai, Zhaojun
Institution: University of Kentucky
EPA Project Officer: Aja, Hayley
Project Period: November 1, 1996 through October 31, 1999 (Extended to November 14, 2000)
Project Period Covered by this Report: November 1, 1997 through October 31, 1998
Project Amount: $475,670
RFA: High Performance Computing (1996) RFA Text | Recipients Lists
Research Category: Human Health , Aquatic Ecosystems , Environmental Statistics
Objective:
The objective of this research is to develop highly parallel algorithms using domain decomposition method, least-squares finite element method and large eddy simulation technique for simulating turbulent dispersion of toxic chemicals around buildings and in Green Bay.Progress Summary:
We have successfully developed algorithms by combining the least-squares finite element method (LSFEM) with the large eddy simulation technique (LES). This allows us to simulate turbulent flows by using the finite element method in an implicit manner. We have explored different formulations of the least-squares finite element, and the CPU time for our new formulation is only 25% of the previous formulation. For example, we have simulated a shear flow problem using 1.05 million finite elements with over 7 million unknowns in a single processor. Therefore, we now have the capability to solve larger fluid flow and transport problems in a more efficient way. A dynamic subgrid-scale model has been successfully implemented in the least-squares finite element/ large eddy simulation code. We have also studied different preconditioners and iterative methods for solving the linear system of equations arising from the LSFEM/LES approach. We have successfully implemented and obtained preliminary evaluation of PETSC (Portable, Extensible Toolkit for Scientific Compuatation) for our project. This will naturally lend to our future effort to implement the domain decomposition technique to our LSFEM/LES codes. We have also developed finite element algorithms for the dispersion of pollutants in Green Bay.Accomplishments and Research Results:
Develop new and computationally efficient formulations of LSFEM for large scale fluid flows and transport processes. The new algorithm requires only 25% CPU time of the previous one. We have tested the new formulation with benchmark three-dimensional shear flow problems and natural convection problems.
Develop Large Eddy Simulation (LES) codes using the least-squares finite element method (LSFEM) and a dynamic subgrid scale model, which has the capability to resolve the turbulence within the wall layer of a surface. We have tested the LES/LSFEM code with benchmark three-dimensional shear driven cavity flows and turbulent channel flows.
Carry out an intensive study on different iterative methods and preconditioners for solving the linear system of equations arising from LSFEM/LES approaches. In particular, we used a three-dimensional lid-driven cavity flow problem for the study. We observed that the linear system of equations we need to solve are moderate ill-conditioning SPD (Symmetric Positive Definite) system. In some instances, we even observed indefinite linear system due to numerical discretization errors and roundoffs in finite precision arithmetic. We investigated the performance of different iterative methods, namely CG (Conjugate Gradient method), CGS, SYMMLQ, QMR and BiCGstab methods. These methods have the capacity to solve SPD, indefinite or general nonsymmetric systems. We also explored different preconditioners, such as Jacobi, SSOR and incomplete LU factorization. We have observed dramatic differences in the number of iterations (matrix-vector multiplications) required for different iterative methods and preconditioners. For example, CG method with Jacobi preconditioner needs 746 iterations for a 8x8x4 grid partition, whereas CG with SSOR preconditioner only requires 273 iterations. However, because of the cost of applying preconditioners, the CPU time for SSOR-CG is actually more than that for Jacobi-CG in sequential computing. We are in the process of investigating the performance in parallel computing environment. It clearly has to take into many factors into account under different computing environments.
Installation and preliminary performance evaluation of PETSC (portable, Extensible Toolkit for Scientific Computation) for our project. PETSC is an object-oriented software package, developed by researchers at Argonne National Laboratory. PETSC uses the standard linear algebra BLAS and LAPACK libraries, as well as the MPI (Message Passing Interface) standard for communication on multiprocessor computers. We have installed and run PETSC at a cluster of HP workstations and the HP Convex Examplar multiprocessor system in the computing center of the University of Kentucky. PETSC will provide a fundamental parallel computing environment for our project. We have successfully solved the model Poisson problem on the Examplar by using CG method with Jacobi and incomplete LU factorization preconditioners. For example, we have observed a factor of 4.6 speed-up on a 8 processor system with 160,000 unknowns.
Develop an algorithm to couple a multigid technique to the Galerkin Finite Element method for solving time dependent flow equations. As the focus of applying multigrid methods was confined to Finite-difference/Finite volume methods (for simulating steady state flow problems), the results of the present investigation indicate that the transient solution can be accelerated by a factor of 1.5-1.7, which is a significant amount. We propose to extend its applications for higher dimensional flows and check if any significant amount of acceleration factors can be obtained.
Future Activities:
Apply our LSFEM/LES code to convective boundary layer flows. This is a test against benchmark field experimental results and calculations.Develop domain-decomposition technique for the LSFEM/LES method. This is one of the central topics of our project. With the availability of the parallel computing environment of PETSC we have set up, we are now in a position to start applying domain-decomposition technique to the LSFEM/LES codes for time-dependent, three-dimensional turbulent flows. We will first divide the whole computational domain into non-overlapping subdomains and assign subdomains to a number of processors so that parallel computations can be carried out.
Investigate the scaling issue arising from the LSFEM formulation. The residuals in the least-squares functional may have very different scales. We propose to find a robust weighting matrix to balance the scaling of different components of the residual vector of the linear SPD system.
We are investigating the possibility of coupling the Flux corrected transport (FCT) techniques to the Finite element formulations. The advantage of such an approach lies in capturing the moving shock front with a high degree of resolution. For flow problems with strong shocks, the standard finite element formulations require a considerable amount of grid refinement and a careful selection of the upwinding parameter. Coupled to a FCT approach, the Finite element formulation becomes more robust. While we have encouraging results, we are in the process of developing a theoretical formulation for justifying the adopted procedure. Having completed that, we plan to extend it to larger scale higher dimensional flows.
Journal Articles on this Report : 6 Displayed | Download in RIS Format
Other project views: | All 16 publications | 8 publications in selected types | All 8 journal articles |
---|
Type | Citation | ||
---|---|---|---|
|
Ding X, Tsang TTH. Large eddy simulation of turbulent flows by a least-squares finite element method. International Journal for Numerical Methods in Fluids 2001;37(3):297-319. |
R825200 (1998) R825200 (1999) |
Exit |
|
Ding X, Tsang TTH. On first-order formulations of the least-squares finite element method for incompressible flows. International Journal of Computational Fluid Dynamics 2003;17(3):183-197. |
R825200 (1998) R825200 (1999) |
Exit |
|
Tang LQ, Wright JL, Tsang TTH. Simulations of 2D and 3D thermocapillary flows by a least-squares finite element method. International Journal for Numerical Methods in Fluids 1998;28(6):983-1007. |
R825200 (1998) R825200 (1999) |
Exit |
|
Wright JL, Chowdhury B, Tang LQ, Tsang TTH. Grid refinement tests of a least-squares finite element method for advective transport of reactive species. Environmental Modelling & Software 1997;12(4):289-299. |
R825200 (1998) R825200 (1999) |
Exit |
|
Yost SA, Rao PMSV. Flux-corrected transport technique for open channel flow. International Journal for Numerical Methods in Fluids 1999;29(8):951-973. |
R825200 (1998) R825200 (1999) |
Exit |
|
Yost SA, Rao PMSV. A non-oscillatory scheme for open channel flows. Advances in Water Resources 1998;22(2):133-143. |
R825200 (1998) R825200 (1999) |
Exit |
Supplemental Keywords:
RFA, Scientific Discipline, Geographic Area, Ecosystem Protection/Environmental Exposure & Risk, State, computing technology, Environmental Monitoring, Ecology and Ecosystems, air quality modeling, ecosystem modeling, fate and transport, large eddy simulation, least squares, finite element method, HPCC, three dimensional turbulant flow, computer science, numerical model, data analysis, information technology, parallel computing, convective transportRelevant Websites:
http://www.engr.uky.edu/cme/faculty/tsangProgress and Final Reports:
Original AbstractThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.