Keywords: parallel software, ScaLAPACK, Python interfaces, numerical experiments, performance.

The development of high performance engineering and scientific applications is an expensive process that often requires specialized support and adequate information about the available computational resources and software development tools. Researchers and engineers behind these applications have to spend a considerable amount of time efficiently developing and running codes in high performance computing environments. Developers would rather devote this time to using their computational applications and analyzing their output.

To alleviate this, we promote the reuse of robust software libraries like the ones in the ACTS Collection [3]. ScaLAPACK [1], is one of the tools in the ACTS Collection, and has been widely used in many high performance scientific applications. ScaLAPACK internally implements parallelism and scalability with the use of BLACS and PBLAS. In many instances, scientists and engineers find it difficult to understand the interfaces to these libraries because they carry arguments that are related to the parallel environment and performance in addition to arguments related to the problem at hand.

We work in the development of a collection of Python interfaces to the ACTS tools, and here we present the work done to the ScaLAPACK library. In order to implement the PyScaLAPACK interface, we have also implemented PyBLACS and PyPBLAS [2]. These libraries involve complex distributed data structures that our proposed interfaces, PyScaLAPACK, hide from the final user, making them easier to use.

In this abstract, Figure 1 presents an example of performance results obtained in a 320 dual-processor Opteron cluster, running a Linux operating system, named Jacquard; each processor runs at a clock speed of 2.2GHz, and has a theoretical peak performance of 4.4 GFlop/s. As it can be seen in the figures the overhead induced by the Python infrastructure is rather nominal. Thus, PyScaLAPACK does not hinder the performance deliverance of ScaLAPACK.

We introduce an example of a real scientific application code, whose functionality can be prototyped or extended with the use of PyScaLAPACK. PyClimate [4] is a Python based package that provides support to common tasks during the analysis of climate variability data. It provides a wide range of functions and uses functionality available in LAPACK. We use PyScaLAPACK to obtain a parallel version of some problems solved by PyClimate as, for example, a meteorological study by means of the EOF and SVD analysis.

1

L.S. Blackford, J. Choi, A. Cleary, E. D'Azevedo, J.W. Demmel, I. Dhillon, J.J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, R.C. Whaley, "ScaLAPACK User's Guide", SIAM, Philadelphia, Pennsylvania, 1997.

2

L.A. Drummond, V. Galiano, V. Migallón, J. Penadés, "Improving ease of use in BLACS and PBLAS with Python", in "Proceedings of Parallel Computing 2005 (ParCo 2005)", Málaga, Spain, 2005. To appear.

3

L.A. Drummond, V. Hernández, O. Marques, J.E. Román, V. Vidal, "A Study of Robust Scientific Libraries for the Advancement of Sciences and Engineering", Lecture Notes in Computer Science, 3402, 37-50, 2005.

4

J. Sáenz, J. Zubillaga, J. Fernández, "Geophysical data analysis using Python", Computers and Geosciences, 28/4, 457-465, 2002. doi:10.1016/S0098-3004(01)00086-3

purchase the full-text of this paper (price £20)

go to the previous paper
go to the next paper
return to the table of contents
return to the book description
purchase this book (price £105 +P&P)