Computational & Technology Resources
an online resource for computational,
engineering & technology publications
Civil-Comp Proceedings
ISSN 1759-3433
CCP: 84
PROCEEDINGS OF THE FIFTH INTERNATIONAL CONFERENCE ON ENGINEERING COMPUTATIONAL TECHNOLOGY
Edited by: B.H.V. Topping, G. Montero and R. Montenegro
Paper 99

Parallel Finite Volume Method Using Domain Decomposition, MPI and SSE Instructions: Applications

M.J. Castro, J.A. García, J.M. González and C. Parés

Department of Mathematical Analysis, University of Málaga, Spain

Full Bibliographic Reference for this paper
, "Parallel Finite Volume Method Using Domain Decomposition, MPI and SSE Instructions: Applications", in B.H.V. Topping, G. Montero, R. Montenegro, (Editors), "Proceedings of the Fifth International Conference on Engineering Computational Technology", Civil-Comp Press, Stirlingshire, UK, Paper 99, 2006. doi:10.4203/ccp.84.99
Keywords: finite volume methods, shallow water equations, parallelization, MPI, SSE instructions.

Summary
This paper deals with the development of efficient parallel implementations of finite volume solvers on unstructured grids to solve two-dimensional hyperbolic systems of conservation laws with source terms and nonconservative products. Problems of this nature arise, for example, in computational fluid dynamics.

We are concerned in particular with the simulation of free-surface waves in shallow layers of homogeneous fluids or internal waves in stratified fluids composed by two shallow layers of immiscible liquids. The motion of a layer of homogeneous fluid is supposed here to be governed by a shallow water system, formulated under the form of a conservation law with source terms or balance law. In the stratified case, the flow is supposed to be governed by a system composed by two coupled shallow water systems. The global system can be formulated in the form of two coupled balance laws, the coupling terms having the form of nonconservative products.

We are mainly interested in the application of these systems to geophysical flows: models based on shallow water systems are useful for the simulation of rivers, floodings, dambreak problems, etc. Stratified fluids are ubiquitous in nature: they appear in atmospheric flows, ocean currents, estuarine systems, etc. This is the situation, for instance, in the Straits of Gibraltar, where surface water from the Atlantic inflows over saltier westward-flowing Mediterranean water. Due to the huge spatial and temporal scales in which these phenomena take place, they require very long lasting simulations (in terms of CPU time). Extremely efficient implementations are required to be able to analyze these problems in small computational time.

In [1] an efficient implementation of the first order well-balanced numerical scheme for a general system of balance laws with nonconservative products, the generalisation of the one-dimensional solver presented in [2] for coupled systems of balance laws was carried out. The scheme was parallelized by means of domain decomposition techniques and the use of MPI in a PC cluster. Very good speedup results were obtained and the scheme was assessed with numerical and experimental data. Nevertheless, the nature of our target problem demands even more computing power: for example, two-dimensional tidal simulations of months or years in the Straits of Gibraltar can lead us to days or even weeks of simulation time on a PC cluster.

In order to increase the performance of the implementation, two main difficulties appear: on the one hand, PC clusters present an important bottleneck due to network latency very difficult to overcome just by using low cost components. On the other hand, the number of nodes has to be doubled in order to halve the CPU time, which becomes exponentially expensive and prohibitive. In this work we follow a different approach to improve the calculus time. The most costly part of the calculations performed at each processor consists of a huge number of small matrix and vector computations. Modern CPUs are provided with specific SIMD units devoted to carry out these kind of computations in a vectorial manner, for instance SSE ("Streaming SIMD Extensions") in Intel or AMD Processors. In this work we introduce a technique to develop a high level C++ small matrix library that takes advantage of multimedia SIMD instructions, present in modern processors, hiding the difficulties related to the use of very low level coding (mostly assembler) required to develop an efficient SIMD implementation.

First we introduce the general formulation for systems of balance laws with nonconservative products and source terms in two-dimensional domains and the particular cases corresponding to the one and two-layer shallow water systems. Next, the numerical scheme is presented. The domain decomposition, MPI implementation is described next and some performance tests are presented that show the efficiency of this implementation. After a brief description of the SSE instructions (registers), we describe in detail the high level C++ matrix library implementation. Some numerical experiments are presented to show the good performance of the matrix library developed and to compare the efficiency of the SSE implementation of the algorithm with the MPI implementation: in the case of one layer flows simulations, speedups of the order of 12 can be obtained. Finally, some numerical experiments are presented in order to validate the solver and to assess the performance of the one layer and the two-layer shallow water models.

References
1
M.J. Castro, J.A. García, J.M. González, C. Parés, "A parallel 2D finite volume scheme for solving systems of balance laws with nonconservative products: application to shallow flows", Computer Methods in Applied Mechanics and Engineering, 2006. doi:10.1016/j.cma.2005.07.007
2
M.J. Castro, J. Macías, C. Parés, "A Q-Scheme for a class of systems of coupled conservation laws with source term. Application to a two-layer 1-D Shallow Water system", ESAIM: M2AN, 35(1), 107-127, 2001. doi:10.1051/m2an:2001108

purchase the full-text of this paper (price £20)

go to the previous paper
go to the next paper
return to the table of contents
return to the book description
purchase this book (price £105 +P&P)