Changeset 622 for palm/trunk/SOURCE/init_pegrid.f90

Timestamp:

Dec 10, 2010 8:08:13 AM (13 years ago)

Author:

raasch

Message:

New:
---

Optional barriers included in order to speed up collective operations
MPI_ALLTOALL and MPI_ALLREDUCE. This feature is controlled with new initial
parameter collective_wait. Default is .FALSE, but .TRUE. on SGI-type
systems. (advec_particles, advec_s_bc, buoyancy, check_for_restart,
cpu_statistics, data_output_2d, data_output_ptseries, flow_statistics,
global_min_max, inflow_turbulence, init_3d_model, init_particles, init_pegrid,
init_slope, parin, pres, poismg, set_particle_attributes, timestep,
read_var_list, user_statistics, write_compressed, write_var_list)

Adjustments for Kyushu Univ. (lcrte, ibmku). Concerning hybrid
(MPI/openMP) runs, the number of openMP threads per MPI tasks can now
be given as an argument to mrun-option -O. (mbuild, mrun, subjob)

Changed:

---

Initialization of the module command changed for SGI-ICE/lcsgi (mbuild, subjob)

Errors:

---

File:

• palm/trunk/SOURCE/init_pegrid.f90 (modified) (4 diffs)

palm/trunk/SOURCE/init_pegrid.f90

@@ -4 +4 @@
 ! Current revisions:
 ! -----------------
-! 
+! optional barriers included in order to speed up collective operations
 ! ATTENTION: nnz_x undefined problem still has to be solved!!!!!!!!
 ! TEST OUTPUT (TO BE REMOVED) logging mpi2 ierr values
@@ -154 +154 @@
        CALL message( 'init_pegrid', 'PA0223', 1, 2, 0, 6, 0 )
     ENDIF
+
+!
+!-- For communication speedup, set barriers in front of collective
+!-- communications by default on SGI-type systems
+    IF ( host(3:5) == 'sgi' )  collective_wait = .TRUE.
 
 !
@@ -929 +934 @@
        id_inflow_l = 0
     ENDIF
+    IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
     CALL MPI_ALLREDUCE( id_inflow_l, id_inflow, 1, MPI_INTEGER, MPI_SUM, &
                         comm1dx, ierr )
@@ -935 +941 @@
 !-- Broadcast the id of the recycling plane
 !-- WARNING: needs to be adjusted in case of inflows other than from left side!
-    IF ( ( recycling_width / dx ) >= nxl  .AND.  ( recycling_width / dx ) <= nxr ) &
-    THEN
+    IF ( ( recycling_width / dx ) >= nxl  .AND. &
+         ( recycling_width / dx ) <= nxr )  THEN
        id_recycling_l = myidx
     ELSE
        id_recycling_l = 0
     ENDIF
+    IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
     CALL MPI_ALLREDUCE( id_recycling_l, id_recycling, 1, MPI_INTEGER, MPI_SUM, &
                         comm1dx, ierr )