Grantee Research Project Results
1999 Progress Report: Parallel Least-Squares Finite Element Method for Large Eddy Simulation of Large Scale Environmental Flows and Transport Processes
EPA Grant Number: R825200Title: Parallel Least-Squares Finite Element Method for Large Eddy Simulation of Large Scale Environmental Flows and Transport Processes
Investigators: Tsang, Tate H. , Yost, Scott A. , Bai, Zhaojun
Institution: University of Kentucky
EPA Project Officer: Aja, Hayley
Project Period: November 1, 1996 through October 31, 1999 (Extended to November 14, 2000)
Project Period Covered by this Report: November 1, 1998 through October 31, 1999
Project Amount: $475,670
RFA: High Performance Computing (1996) RFA Text | Recipients Lists
Research Category: Human Health , Aquatic Ecosystems , Environmental Statistics
Objective:
The objective of this research is to develop highly parallel algorithms using the domain decomposition method, least-squares finite element method, and large eddy simulation (LES) technique to simulate turbulent dispersion of toxic chemicals around buildings and in Green Bay.Progress Summary:
We have explored different formulations of the least-squares finite element method (LSFEM), and the CPU time for our new formulations is only 25 percent of the previous formulation. We successfully developed algorithms by combining LSFEM with the LES technique. A dynamic subgrid-scale model has been implemented in the least-squares finite element/large eddy simulation code. This allows us to simulate turbulent flows by using the LSFEM in an implicit manner. Our LES/LSFEM results compare well with the experimental and simulation data for turbulent channel flows, lid-driven cavity flows, and turbulent thermal convection. More recently, we developed a domain-decomposition based, parallel LSFEM and obtained significant speedups for time-dependent, three-dimensional, shear-driven flow problems using 1 to 4 million finite elements with 2 to 29 million unknowns. We have successfully implemented and evaluated PETSc (Portable, Extensible Toolkit for Scientific Computation) for our project. This will naturally lend to our effort to implement the domain decomposition technique to our finite element codes. We also have developed finite element algorithms for the dispersion of pollutants in Green Bay.We have developed new and computationally efficient formulations of the LSFEM for large-scale fluid flows and transport processes. The new formulations require only 25 percent CPU time of the previous formulation. We have tested the new formulations with benchmark three-dimensional, shear-flow problems and natural convection problems.
We have developed LES codes using the LSFEM and a dynamic subgrid-scale model, which has the capability to resolve the turbulence within the wall layer of a surface. Our LES/LSFEM results compare well with benchmark turbulent channel flows, three-dimensional, shear-driven cavity flows and turbulent thermal convection. The LES/LSFEM can simulate the "Law of the Wall" without the use of damping or wall function. The eddy viscosity is predicted correctly without the use of any adjustable parameter.
We have developed a domain-decomposition based, parallel LSFEM, which has achieved high efficiency and speedups for fluid flows with 29 million unknowns and 4 million finite elements on 32 processes (EXEMPLAR SPP 2000). This shows that our parallel LSFEM has the potential to simulate large-scale environmental flow and transport processes.
The serial code for open channel flows has been optimized, to ensure maximum Mflops/processor before imbedding various high-performance strategies into it. The parallel code is based on domain decomposition principles and uses MPI for all interprocessor communication. The domain decomposition used is the data decomposition and was achieved by partitioning the domain that is homogeneous with respect to local governing flow equations.
The parallel code for open channel flows has been tested on both the HP Exemplar and Roadrunner Supercluster. Clusters that are considered to be the next generation high-performance machines are fast becoming popular among the research community, because of their low cost and reasonable performance. In line with this trend, we have tested the performance of the parallel code on the Roadrunner cluster, located at the University of New Mexico. While many studies have compared the cost benefits of a cluster, our investigation sheds some light on their performance details. The parallel code has been run up to 32 processor on both the machines, and high-performance data have been achieved. The code was parallelized using the PETSc modules being developed at the Argonne National Laboratory. The optimized version of PETSc libraries has been installed on the Roadrunner Supercluster.
Gaining synergy from the work we have been doing over the past 2 years, we are in the process of developing some guidelines by which any existing serial code can be parallelized. To illustrate this, we have implemented these ideas in the widely used RMA2 code. RMA2 is a popular code in the hydraulic modeling community that predicts the water surface elevation for two-dimensional free surface flows by solving the Reynolds form of the Navier Stokes equations for turbulent flows. The equations are solved by the finite element method using the Galerkin method of weighted residuals. Both steady and unsteady problems can be analyzed. The original RMA2 code was developed by the Water Resource Engineers in 1973, for the Walla Walla District, U.S. Army Corps of Engineers. Subsequent modifications were made by the researchers of Resource Management Associates (RMA) and the Waterways Experimentation Station (WES).
Future Activities:
We will develop a parallel LSFEM/LES code for convective boundary layer flows. This is a test against benchmark field experimental results and calculations.Apply the domain-decomposition based, parallel LES/LSFEM code to simulate turbulent flows around obstacles with complex geometry. This is one of the central applications of our project. We will use CAD software to generate the obstacles (solid modeling). Then, we will use a mesh generator to construct three-dimensional finite element mesh. We will use our parallel LES/LSFEM as the flow solver, and a visualization software to visualize the turbulent flow and pollutant dispersion around the obstacles.
We are investigating the possibility of coupling the flux corrected transport (FCT) techniques to the finite element formulations. The advantage of such an approach lies in capturing the moving shock front with a high degree of resolution. For flow problems with strong shocks, the standard finite element formulations require a considerable amount of grid refinement and a careful selection of the upwinding parameter. Coupled to a FCT approach, the finite element formulation becomes more robust. Although we have encouraging results, we are in the process of developing a theoretical formulation for justifying the adopted procedure. Having completed that, we plan to extend it to larger-scale, higher-dimensional flows.
Our relevant presentations made during this period have been well received by the modeling community. Based on their enthusiasm, we are planning to have a 2-day workshop at the University of Kentucky in March 2000. We expect attendees from across the country. Our idea is to share with them the concepts behind parallel programming and tools that can be used for running their serial codes in a parallel environment.
Journal Articles on this Report : 8 Displayed | Download in RIS Format
| Other project views: | All 16 publications | 8 publications in selected types | All 8 journal articles |
|---|
| Type | Citation | ||
|---|---|---|---|
|
|
Ding X, Tsang TTH. Large eddy simulation of turbulent flows by a least-squares finite element method. International Journal for Numerical Methods in Fluids 2001;37(3):297-319. |
R825200 (1998) R825200 (1999) |
Exit |
|
|
Ding X, Tsang TTH. On first-order formulations of the least-squares finite element method for incompressible flows. International Journal of Computational Fluid Dynamics 2003;17(3):183-197. |
R825200 (1998) R825200 (1999) |
Exit |
|
|
Tang LQ, Wright JL, Tsang TTH. Simulations of 2D and 3D thermocapillary flows by a least-squares finite element method. International Journal for Numerical Methods in Fluids 1998;28(6):983-1007. |
R825200 (1998) R825200 (1999) |
Exit |
|
|
Wright JL, Chowdhury B, Tang LQ, Tsang TTH. Grid refinement tests of a least-squares finite element method for advective transport of reactive species. Environmental Modelling & Software 1997;12(4):289-299. |
R825200 (1998) R825200 (1999) |
Exit |
|
|
Yost SA, Rao PMSV. Flux-corrected transport technique for open channel flow. International Journal for Numerical Methods in Fluids 1999;29(8):951-973. |
R825200 (1998) R825200 (1999) |
Exit |
|
|
Yost SA, Rao PMSV. A non-oscillatory scheme for open channel flows. Advances in Water Resources 1998;22(2):133-143. |
R825200 (1998) R825200 (1999) |
Exit |
|
|
Yost SA, Rao P, Brown RM. Absorbing boundary technique for open channel flows. International Journal for Numerical Methods in Fluids 2000;33(5):641-656. |
R825200 (1999) |
Exit |
|
|
Yost SA, Rao P. A multiple grid algorithm for one-dimensional transient open channel flows. Advances in Water Resources 2000;23(6):645-651. |
R825200 (1999) |
Exit Exit Exit |
Supplemental Keywords:
air, toxics, turbulence, modeling, large eddy simulation, least-squares finite element method, domain decomposition, parallel computing, open channel flows, Green Bay., RFA, Scientific Discipline, Geographic Area, Ecosystem Protection/Environmental Exposure & Risk, State, computing technology, Environmental Monitoring, Ecology and Ecosystems, air quality modeling, ecosystem modeling, fate and transport, large eddy simulation, least squares, finite element method, HPCC, three dimensional turbulant flow, computer science, numerical model, data analysis, information technology, parallel computing, convective transportRelevant Websites:
http://www.engr.uky.edu/cme/faculty/tsang
Progress and Final Reports:
Original AbstractThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.