Changeset 622 for palm/trunk/SOURCE/flow_statistics.f90

Timestamp:

Dec 10, 2010 8:08:13 AM (14 years ago)

Author:

raasch

Message:

New:
---

Optional barriers included in order to speed up collective operations
MPI_ALLTOALL and MPI_ALLREDUCE. This feature is controlled with new initial
parameter collective_wait. Default is .FALSE, but .TRUE. on SGI-type
systems. (advec_particles, advec_s_bc, buoyancy, check_for_restart,
cpu_statistics, data_output_2d, data_output_ptseries, flow_statistics,
global_min_max, inflow_turbulence, init_3d_model, init_particles, init_pegrid,
init_slope, parin, pres, poismg, set_particle_attributes, timestep,
read_var_list, user_statistics, write_compressed, write_var_list)

Adjustments for Kyushu Univ. (lcrte, ibmku). Concerning hybrid
(MPI/openMP) runs, the number of openMP threads per MPI tasks can now
be given as an argument to mrun-option -O. (mbuild, mrun, subjob)

Changed:

---

Initialization of the module command changed for SGI-ICE/lcsgi (mbuild, subjob)

Errors:

---

File:

• palm/trunk/SOURCE/flow_statistics.f90 (modified) (4 diffs)

palm/trunk/SOURCE/flow_statistics.f90

@@ -4 +4 @@
 ! Current revisions:
 ! -----------------
-! 
+! optional barriers included in order to speed up collective operations
 !
 ! Former revisions:
@@ -237 +237 @@
 !
 !--    Compute total sum from local sums
+       IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
        CALL MPI_ALLREDUCE( sums_l(nzb,1,0), sums(nzb,1), nzt+2-nzb, MPI_REAL, &
                            MPI_SUM, comm2d, ierr )
+       IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
        CALL MPI_ALLREDUCE( sums_l(nzb,2,0), sums(nzb,2), nzt+2-nzb, MPI_REAL, &
                            MPI_SUM, comm2d, ierr )
+       IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
        CALL MPI_ALLREDUCE( sums_l(nzb,4,0), sums(nzb,4), nzt+2-nzb, MPI_REAL, &
                            MPI_SUM, comm2d, ierr )
        IF ( ocean )  THEN
+          IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
           CALL MPI_ALLREDUCE( sums_l(nzb,23,0), sums(nzb,23), nzt+2-nzb, &
                               MPI_REAL, MPI_SUM, comm2d, ierr )
        ENDIF
        IF ( humidity ) THEN
+          IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
           CALL MPI_ALLREDUCE( sums_l(nzb,44,0), sums(nzb,44), nzt+2-nzb, &
                               MPI_REAL, MPI_SUM, comm2d, ierr )
+          IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
           CALL MPI_ALLREDUCE( sums_l(nzb,41,0), sums(nzb,41), nzt+2-nzb, &
                               MPI_REAL, MPI_SUM, comm2d, ierr )
           IF ( cloud_physics ) THEN
+             IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
              CALL MPI_ALLREDUCE( sums_l(nzb,42,0), sums(nzb,42), nzt+2-nzb, &
                                  MPI_REAL, MPI_SUM, comm2d, ierr )
+             IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
              CALL MPI_ALLREDUCE( sums_l(nzb,43,0), sums(nzb,43), nzt+2-nzb, &
                                  MPI_REAL, MPI_SUM, comm2d, ierr )
@@ -261 +269 @@
 
        IF ( passive_scalar )  THEN
+          IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
           CALL MPI_ALLREDUCE( sums_l(nzb,41,0), sums(nzb,41), nzt+2-nzb, &
                               MPI_REAL, MPI_SUM, comm2d, ierr )
@@ -796 +805 @@
 !
 !--    Compute total sum from local sums
+       IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
        CALL MPI_ALLREDUCE( sums_l(nzb,1,0), sums(nzb,1), ngp_sums, MPI_REAL, &
                            MPI_SUM, comm2d, ierr )