by Santiago Lombeyda and Jim Pool, Center for Advanced Computing Research, California Institute of Technology, and Mahesh Rajan, Hewlett-Packard, Inc.

Imagine half-filling a glass with water and then adding an equal amount of a light oil. The two liquids do not mix (they are immiscible) and, if the oil is the less dense of the two, it remains above the water and the interface between the oil and the water is planar. Now imagine sealing the top of the glass (so that no liquid escapes) and turning it upside-down. If the glass is wide enough, gravity will cause the water to fall into the lower part of the glass and the oil will rise into the upper part. The interface becomes distorted by spikes, bubbles and other shapes, which is due to a phenomenon known as the Rayleigh-Taylor instability. This occurs whenever a light fluid is accelerated into a heavy one, and it has been studied widely because of its relationship to the onset of turbulence and mixing. More prosaically, it has also inspired a number of toys which involve watching the changing shape of the interface (one of the liquids is usually colored, to make it easier to see).

Some studies of the Rayleigh-Taylor instability have been done through real experiments, but more and more are being carried out on computers, especially as their increasing power and storage allows larger systems to be examined. In this type of simulation experiment, the equations describing the fluid flow are solved numerically at a series of time steps to describe the dynamic evolution of the system.

A recent simulation has been carried out on the ASCI Blue-Pacific machine at Lawrence Livermore National Laboratories by our colleague Dr Andy Cook. This machine is a massively parallel cluster of IBM RS/6000 processors which has a peak performance of almost four teraflops (about 15,000 times that of a typical desktop PC). The simulation was performed on a grid of dimensions 256 x 512 x 256, and run for more than three thousand time steps. The data for each time step occupies about 260 MB, making the total size of the output dataset from the simulation about a terabyte. The dataset is being analysed by our colleague Professor Paul Dimotakis, and we have been working on the production of a visualization from the data.

Making visualizations of datasets of such magnitude can exhibit problems which are just as complex as creating the simulations themselves, especially when there is a requirement to perform the visualization in (close to) real time. In this instance, we were interested in visualizing the results from this simulation by extracting and displaying the interface between the fluids as it evolves in time. A good way to display the interface is to calculate the density isosurface for a threshold value intermediate between the densities of the two fluids.

For our visualization system, we used IRIS Explorer, which includes a variety of modules for reading, sampling , visualizing and displaying data. One of these is an isosurface module, but, because of the size of our dataset and the number of time steps involved, it was not possible to extract and display isosurfaces dynamically in real time. So we decided to implement isosurface extraction in parallel to try to decrease processing time.

We decided to use IRIS Explorer as our platform-independent front-end interface running on a standard graphics workstation (HP, SGI, Windows NT, and others), while taking advantage of a multiprocessor machine to execute the isosurface extraction. To this end, we implemented a module within IRIS Explorer which was capable of communicating (via sockets) with the isosurface extraction code running on a 16 cpu Hewlett-Packard V-class Exemplar server.

To extract the isosurface we implemented a variation of the standard marching cubes algorithm which keeps track of visited cube edges (thus reducing calculation redundancy) and modified it for parallel execution using simple data decomposition. After some experimentation with various parallelization techniques on the Exemplar, we eventually settled on using p-threads. These could be easily invoked (using a simple one-line directive in the code), and had the additional advantage of working with sockets (which formed the basis of our communications framework).

Our performance results were encouraging. Compared to execution using a single thread, we achieved a speedup of about 7.8 when using eight threads, which indicates a high degree of parallelization. We were then capable of loading several frames of the dataset into the server, and through the use of our module within IRIS Explorer we were able to dynamically request isosurfaces across different density thresholds and through different time steps.

We think that this work illustrates the way in which IRIS Explorer can be utilized to drive large computations, taking advantage of parallel processing where necessary, while using a standard graphic workstations to display the results. Finally, embedding our parallel isosurface module within IRIS Explorer promotes its re-use amongst the user community who have different needs and access to a variety of hardware configurations.

We thank Andy Cook (LLNL) and Paul Dimotakis (Caltech) for giving us access to the Rayleigh-Taylor dataset, and Jeremy Walton (NAG Ltd) for helpful discussions about IRIS Explorer. More images from this work can be viewed at http://www.cacr.caltech.edu/~slombey/asci/asciGallery.html.