User's Guide to NCAR CCM3.6 Search Page

2. Using CCM3.6

    2.1 How to Build and Run CCM3.6.6

2.1.3 Running the Distributed Memory Implementation

Running CCM3 under MPI on the SUN and SGI architectures consists of two steps: building the MPI executable and starting the model using the appropriate command line arguments for MPI. Configuring CCM3.6 for MPI requires defining the SPMD token in the header file misc.h and including the directories $MODEL_SRCDIR/spmd and either $MODEL_SRCDIR/spmd/eul or $MODEL_SRCDIR/spmd/sld in Filepath. Additionally, the directories containing the MPI library and MPI include files must be specified as environment variables in the build script.

Most MPI implementations provide a startup script called mpirun, which accepts the MPI executable as a command line argument.   Additional command line arguments allow the user to specify details such as the various machine architectures or number of processes to use for the run.  For example the command used to run the MPI version using two processes might be:

     mpirun -np 2 atm < atm.parm

assuming that atm.parm corresponds to the input namelist file and atm to the executable.

Once MPI has created the specified number of processes, model execution will begin. The collection of active tasks will then compute locally and exchange messages with each other to integrate the model. A few caveats should be kept in mind when running the MPI version of the model.
 

• The user must specify the number of processes  to use. This is done via the -np parameter to mpirun.
• The user must specify an even number of processes.
• Since MPI creates remote processes, the user must take care that process limits for non-interactive shells are sufficient for model size.  The Sun default stack size of 8Mb is one such limit, which can cause higher resolution model runs to fail, usually with a segmentation fault.
• If the executable is not on a shared file system, you must copy the executable to each host manually.

 Search page

Questions on these pages can be sent to... erik@ucar.edu .