Subject: Re: mcuts
From: Jim Rosinski (rosinski@skyhawk.cgd.ucar.EDU)
Date: Wed Apr 16 1997 - 14:40:48 MDT
Regarding the recent email concerning the message-passing implementation of
CCM3, there are a few bugfixes which will be incorporated in the next
release of the model. One or more of these may address the problem mentioned
by Joel Malard in his message. I've included example mods in this mail
message for those who would like to make use of them right away.
> It looks like subroutine mcuts allocated spurious work to processes 43 to
> 64. What I did was to:
>
> a. change the lines in mcuts:
>
> mnstrt = nstart(begm(iam)) + 1
> mnend = nstart(endm(iam)) + nlen(endm(iam))
>
> to:
>
> if( iam.le.npessp ) then
> mnstrt = nstart(begm(iam)) + 1
> mnend = nstart(endm(iam)) + nlen(endm(iam))
> else
> mnstrt = 1
> mnend = 0
> end if
There should be no spurious work allocated to processes 43 through 64, due to
the code in mcuts which says:
begm(procid) = 0
endm(procid) = -1
for non-spectral processors. However, the original code segment above could
potentially result in spurious memory allocation for non-spectral processors
due to an out-of-bounds memory reference on the array nstart. The code fix
Joel has suggested is nearly correct, but the test should be on
if (iam.lt.npessp) then
rather than
if (iam.le.npessp) then
since the processors are numbered from 0 to npessp-1, not 1 to npessp. Given
this code change, the modification suggested by Mr. Malard for RESUME is
reasonable, or one could recode it as:
if (iam.lt.npessp) then
call getmem('RESUME ',len2*(plat/2),palpm)
call getmem('RESUME ',len2*(plat/2),pdalpm)
end if
In either case, this same change has to be made to subroutine INITAL for an
initial run, since RESUME is only called on a restart run.
Finally, there is a 1-word out-of-bounds memory reference error within the
routine BNDEXCH which may or may not be related to Joel's problem. The
modified code for bndexch.F (CCM3.2) which will be included in the next
release is included at the end of this message.
Thanks to Joel Malard for bringing up this problem. Hopefully these bugfixes
will help.
Jim Rosinski
CCM Core Group
-------------------------------------------------------------------------------
#include <params.h>
subroutine bndexch
C-----------------------------------------------------------------------
C Pack and Exchange initial prognostic information among all the
C processors
C---------------------------Code history--------------------------------
C
C Original version: CCM2
C Standardized: J. Rosinski, Oct 1995
C J. Truesdale, Feb. 1996
C-----------------------------------------------------------------------
c
c $Id: bndexch.F,v 1.1 1997/03/21 19:28:09 rosinski Exp $
c $Author: rosinski $
c
#include <implicit.h>
C----------------------------Parameters---------------------------------
integer msgtype
parameter ( msgtype = 6000 )
#include <pmgrid.h>
#include <pagrid.h>
integer j1m,siz
parameter (j1m = j1 - 1)
parameter (siz = (2 + pcnst)*plndlv)
C----------------------------Commons------------------------------------
#include <comspmd.h>
C-----------------------------------------------------------------------
#include <com3d.h>
C
C---------------------------Local workspace-----------------------------
C
integer neigh
integer side
integer inreg( 2 )
integer outreg( 2 )
integer iother ! Other node
integer bufidx, j,info,bufid ! index variables
integer bytes
integer msgtag
integer tid
integer numneigh
c
c Externals
c
external funpack
external fpack
C
C-----------------------------------------------------------------------
C
C For each partition (south and north) communicate boundaries
C on each side of partition among however many neighbors necessary
C
do side=1,2
if (side.eq.1) then
numneigh = neighs
else
numneigh = neighn
end if
do neigh=1,numneigh
if (side.eq.1) then
iother = iam - neigh
else
iother = iam + neigh
end if
c
c Intersection of my cuts and neighbor processor's extended
c cuts tells if this node needs to send data to neighbor
c On the other hand, intersection of neighbor cuts and this
c node's extended cut tells if this node recieves data from neighbor
c
c
c Initialize output and input regions to a null region
c
call intersct(cut(1,iam),cutex(1,iother),outreg)
call intersct(cut(1,iother),cutex(1,iam),inreg )
c write(6,*)'BNDEXCH: iam=',iam,' send lats ',outreg(1),
c $ '-',outreg(2),' recv lats ',inreg(1),
c $ '-',inreg(2),' to/fm other node=',iother
c write(6,*)'Input msgtype=',msgtype+iother,
c $ ' output msgtype=',msgtype+iam
bufidx = 0
do j=outreg(1),outreg(2)
call fpack(bufidx,plndlv,u3(1,1,j1m+j,n3))
call fpack(bufidx,plndlv,v3(1,1,j1m+j,n3))
call fpack(bufidx,pcnst*plndlv,q3(1,1,1,j1m+j,n3m1))
end do
call t_startf('comm')
call t_startf('send')
call pvmfsend( tids(iother), msgtype+iam, info )
call t_stopf('send')
call ckpvmerr("BNDEXCH: sending problem",info)
call t_startf('recv')
call pvmfrecv( tids(iother), msgtype+iother, bufid)
if (chkbufsz) then
call pvmfbufinfo(bufid,bytes,msgtag,tid,info)
if (info.lt.0) then
write(6,*)'iam=',iam,
$ ' Bad info returned from bufinfo =',info
call pvmfhalt(info)
else if (bytes.gt.maxrecv) then
maxrecv = bytes
write(6,*)'iam=',iam,' New max recvmsg size = ',maxrecv,
$ ' bytes'
end if
nrecv = nrecv + 1
comrcv = comrcv + bytes
nsend = nsend + 1
comsnd = comsnd + bufidx
end if
call t_stopf('recv')
call t_stopf('comm')
do j=inreg(1),inreg(2)
call funpack(plndlv,u3(1,1,j1m+j,n3))
call funpack(plndlv,v3(1,1,j1m+j,n3))
call funpack(pcnst*plndlv,q3(1,1,1,j1m+j,n3m1))
end do
end do
end do
return
end
This archive was generated by hypermail 2b27 : Thu Jun 01 2000 - 09:10:55 MDT