Ignore:
Timestamp:
Dec 23, 2010 12:06:00 PM (14 years ago)
Author:
suehring
Message:

summary:


Gryschka:

  • Coupling with different resolution and different numbers of PEs in ocean and atmosphere is available
  • Exchange of u and v from ocean surface to atmosphere surface
  • Mirror boundary condition for u and v at the bottom are replaced by dirichlet boundary conditions
  • Inflow turbulence is now defined by flucuations around spanwise mean
  • Bugfixes for cyclic_fill and constant_volume_flow

Suehring:

  • New advection added ( Wicker and Skamarock 5th order ), therefore:
    • New module advec_ws.f90
    • Modified exchange of ghost boundaries.
    • Modified evaluation of turbulent fluxes
    • New index bounds nxlg, nxrg, nysg, nyng

advec_ws.f90


Advection scheme for scalars and momentum using the flux formulation of
Wicker and Skamarock 5th order.
Additionally the module contains of a routine using for initialisation and
steering of the statical evaluation. The computation of turbulent fluxes takes
place inside the advection routines.
In case of vector architectures Dirichlet and Radiation boundary conditions are
outstanding and not available. Furthermore simulations within topography are
not possible so far. A further routine local_diss_ij is available and is used
if a control of dissipative fluxes is desired.

check_parameters.f90


Exchange of parameters between ocean and atmosphere via PE0
Check for illegal combination of ws-scheme and timestep scheme.
Check for topography and ws-scheme.
Check for not cyclic boundary conditions in combination with ws-scheme and
loop_optimization = 'vector'.
Check for call_psolver_at_all_substeps and ws-scheme for momentum_advec.

Different processor/grid topology in atmosphere and ocean is now allowed!
Bugfixes in checking for conserve_volume_flow_mode.

exchange_horiz.f90


Dynamic exchange of ghost points with nbgp_local to ensure that no useless
ghost points exchanged in case of multigrid. type_yz(0) and type_xz(0) used for
normal grid, the remaining types used for the several grid levels.
Exchange is done via MPI-Vectors with a dynamic value of ghost points which
depend on the advection scheme. Exchange of left and right PEs is 10% faster
with MPI-Vectors than without.

flow_statistics.f90


When advection is computed with ws-scheme, turbulent fluxes are already
computed in the respective advection routines and buffered in arrays
sums_xxxx_ws_l(). This is due to a consistent treatment of statistics
with the numerics and to avoid unphysical kinks near the surface. So some if-
requests has to be done to dicern between fluxes from ws-scheme other advection
schemes. Furthermore the computation of z_i is only done if the heat flux
exceeds a minimum value. This affects only simulations of a neutral boundary
layer and is due to reasons of computations in the advection scheme.

inflow_turbulence.f90


Using nbgp recycling planes for a better resolution of the turbulent flow near
the inflow.

init_grid.f90


Definition of new array bounds nxlg, nxrg, nysg, nyng on each PE.
Furthermore the allocation of arrays and steering of loops is done with these
parameters. Call of exchange_horiz are modified.
In case of dirichlet bounday condition at the bottom zu(0)=0.0
dzu_mg has to be set explicitly for a equally spaced grid near bottom.
ddzu_pres added to use a equally spaced grid near bottom.

init_pegrid.f90


Moved determination of target_id's from init_coupling
Determination of parameters needed for coupling (coupling_topology, ngp_a, ngp_o)
with different grid/processor-topology in ocean and atmosphere

Adaption of ngp_xy, ngp_y to a dynamic number of ghost points.
The maximum_grid_level changed from 1 to 0. 0 is the normal grid, 1 to
maximum_grid_level the grids for multigrid, in which 0 and 1 are normal grids.
This distinction is due to reasons of data exchange and performance for the
normal grid and grids in poismg.
The definition of MPI-Vectors adapted to a dynamic numer of ghost points.
New MPI-Vectors for data exchange between left and right boundaries added.
This is due to reasons of performance (10% faster).

ATTENTION: nnz_x undefined problem still has to be solved!!!!!!!!
TEST OUTPUT (TO BE REMOVED) logging mpi2 ierr values

parin.f90


Steering parameter dissipation_control added in inipar.

Makefile


Module advec_ws added.

Modules


Removed u_nzb_p1_for_vfc and v_nzb_p1_for_vfc

For coupling with different resolution in ocean and atmophere:
+nx_a, +nx_o, ny_a, +ny_o, ngp_a, ngp_o, +total_2d_o, +total_2d_a,
+coupling_topology

Buffer arrays for the left sided advective fluxes added in arrays_3d.
+flux_s_u, +flux_s_v, +flux_s_w, +diss_s_u, +diss_s_v, +diss_s_w,
+flux_s_pt, +diss_s_pt, +flux_s_e, +diss_s_e, +flux_s_q, +diss_s_q,
+flux_s_sa, +diss_s_sa
3d arrays for dissipation control added. (only necessary for vector arch.)
+var_x, +var_y, +var_z, +gamma_x, +gamma_y, +gamma_z
Default of momentum_advec and scalar_advec changed to 'ws-scheme' .
+exchange_mg added in control_parameters to steer the data exchange.
Parameters +nbgp, +nxlg, +nxrg, +nysg, +nyng added in indices.
flag array +boundary_flags added in indices to steer the degradation of order
of the advective fluxes when non-cyclic boundaries are used.
MPI-datatypes +type_y, +type_y_int and +type_yz for data_exchange added in
pegrid.
+sums_wsus_ws_l, +sums_wsvs_ws_l, +sums_us2_ws_l, +sums_vs2_ws_l,
+sums_ws2_ws_l, +sums_wspts_ws_l, +sums_wssas_ws_l, +sums_wsqs_ws_l
and +weight_substep added in statistics to steer the statistical evaluation
of turbulent fluxes in the advection routines.
LOGICALS +ws_scheme_sca and +ws_scheme_mom added to get a better performance
in prognostic_equations.
LOGICAL +dissipation_control control added to steer numerical dissipation
in ws-scheme.

Changed length of string run_description_header

pres.f90


New allocation of tend when ws-scheme and multigrid is used. This is due to
reasons of perforance of the data_exchange. The same is done with p after
poismg is called.
nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng when no
multigrid is used. Calls of exchange_horiz are modified.

bugfix: After pressure correction no volume flow correction in case of
non-cyclic boundary conditions
(has to be done only before pressure correction)

Call of SOR routine is referenced with ddzu_pres.

prognostic_equations.f90


Calls of the advection routines with WS5 added.
Calls of ws_statistics added to set the statistical arrays to zero after each
time step.

advec_particles.f90


Declaration of de_dx, de_dy, de_dz adapted to additional ghost points.
Furthermore the calls of exchange_horiz were modified.

asselin_filter.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

average_3d_data.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

boundary_conds.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng
Removed mirror boundary conditions for u and v at the bottom in case of
ibc_uv_b == 0. Instead, dirichelt boundary conditions (u=v=0) are set
in init_3d_model

calc_liquid_water_content.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

calc_spectra.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng for
allocation of tend.

check_open.f90


Output of total array size was adapted to nbgp.

data_output_2d.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng in loops and
allocation of arrays local_2d and total_2d.
Calls of exchange_horiz are modified.

data_output_2d.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng in loops and
allocation of arrays. Calls of exchange_horiz are modified.
Skip-value skip_do_avs changed to a dynamic adaption of ghost points.

data_output_mask.f90


Calls of exchange_horiz are modified.

diffusion_e.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

diffusion_s.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

diffusion_u.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

diffusion_v.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

diffusion_w.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

diffusivities.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

diffusivities.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.
Calls of exchange_horiz are modified.

exchange_horiz_2d.f90


Dynamic exchange of ghost points with nbgp, which depends on the advection
scheme. Exchange between left and right PEs is now done with MPI-vectors.

global_min_max.f90


Adapting of the index arrays, because MINLOC assumes lowerbound
at 1 and not at nbgp.

init_3d_model.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng in loops and
allocation of arrays. Calls of exchange_horiz are modified.
Call ws_init to initialize arrays needed for statistical evaluation and
optimization when ws-scheme is used.
Initial volume flow is now calculated by using the variable hom_sum.
Therefore the correction of initial volume flow for non-flat topography
removed (removed u_nzb_p1_for_vfc and v_nzb_p1_for_vfc)
Changed surface boundary conditions for u and v in case of ibc_uv_b == 0 from
mirror bc to dirichlet boundary conditions (u=v=0), so that k=nzb is
representative for the height z0

Bugfix: type conversion of '1' to 64bit for the MAX function (ngp_3d_inner)

init_coupling.f90


determination of target_id's moved to init_pegrid

init_pt_anomaly.f90


Call of exchange_horiz are modified.

init_rankine.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.
Calls of exchange_horiz are modified.

init_slope.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.

header.f90


Output of advection scheme.

poismg.f90


Calls of exchange_horiz are modified.

prandtl_fluxes.f90


Changed surface boundary conditions for u and v from mirror bc to dirichelt bc,
therefore u(uzb,:,:) and v(nzb,:,:) is now representative for the height z0
nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

production_e.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng

read_3d_binary.f90


+/- 1 replaced with +/- nbgp when swapping and allocating variables.

sor.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.
Call of exchange_horiz are modified.
bug removed in declaration of ddzw(), nz replaced by nzt+1

subsidence.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.

sum_up_3d_data.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.

surface_coupler.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng in
MPI_SEND() and MPI_RECV.
additional case for nonequivalent processor and grid topopolgy in ocean and
atmosphere added (coupling_topology = 1)

Added exchange of u and v from Ocean to Atmosphere

time_integration.f90


Calls of exchange_horiz are modified.
Adaption to slooping surface.

timestep.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.

user_3d_data_averaging.f90, user_data_output_2d.f90, user_data_output_3d.f90,
user_actions.f90, user_init.f90, user_init_plant_canopy.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.

user_read_restart_data.f90


Allocation with nbgp.

wall_fluxes.f90


nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng.

write_compressed.f90


Array bounds and nx, ny adapted with nbgp.

sor.f90


bug removed in declaration of ddzw(), nz replaced by nzt+1

Location:
palm/trunk/SOURCE
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • palm/trunk/SOURCE

    • Property svn:mergeinfo set to (toggle deleted branches)
      /palm/branches/suehring423-666
      /palm/branches/letzel/masked_output/SOURCE296-409
  • palm/trunk/SOURCE/data_output_2d.f90

    r623 r667  
    44! Current revisions:
    55! -----------------
    6 !
     6! nxl-1, nxr+1, nys-1, nyn+1 replaced by nxlg, nxrg, nysg, nyng in loops and
     7! allocation of arrays local_2d and total_2d.
     8! Calls of exchange_horiz are modiefied.
    79!
    810! Former revisions:
     
    112114
    113115       CASE ( 'xy' )
    114 
    115116          s = 1
    116           ALLOCATE( level_z(0:nzt+1), local_2d(nxl-1:nxr+1,nys-1:nyn+1) )
     117          ALLOCATE( level_z(nzb:nzt+1), local_2d(nxlg:nxrg,nysg:nyng) )
    117118
    118119!
     
    130131                IF ( iso2d_output )  CALL check_open( 21 )
    131132#if defined( __parallel )
    132                 ALLOCATE( total_2d(-1:nx+1,-1:ny+1) )
     133                ALLOCATE( total_2d(-nbgp:nx+nbgp,-nbgp:ny+nbgp) )
    133134#endif
    134135             ENDIF
     
    136137
    137138       CASE ( 'xz' )
    138 
    139139          s = 2
    140           ALLOCATE( local_2d(nxl-1:nxr+1,nzb:nzt+1) )
     140          ALLOCATE( local_2d(nxlg:nxrg,nzb:nzt+1) )
    141141
    142142!
     
    154154                IF ( iso2d_output )  CALL check_open( 22 )
    155155#if defined( __parallel )
    156                 ALLOCATE( total_2d(-1:nx+1,nzb:nzt+1) )
     156                ALLOCATE( total_2d(-nbgp:nx+nbgp,nzb:nzt+1) )
    157157#endif
    158158             ENDIF
     
    162162
    163163          s = 3
    164           ALLOCATE( local_2d(nys-1:nyn+1,nzb:nzt+1) )
     164          ALLOCATE( local_2d(nysg:nyng,nzb:nzt+1) )
    165165
    166166!
     
    178178                IF ( iso2d_output )  CALL check_open( 23 )
    179179#if defined( __parallel )
    180                 ALLOCATE( total_2d(-1:ny+1,nzb:nzt+1) )
     180                ALLOCATE( total_2d(-nbgp:ny+nbgp,nzb:nzt+1) )
    181181#endif
    182182             ENDIF
     
    192192!
    193193!-- Allocate a temporary array for resorting (kji -> ijk).
    194     ALLOCATE( local_pf(nxl-1:nxr+1,nys-1:nyn+1,nzb:nzt+1) )
     194    ALLOCATE( local_pf(nxlg:nxrg,nysg:nyng,nzb:nzt+1) )
    195195
    196196!
     
    219219             CASE ( 'lwp*_xy' )        ! 2d-array
    220220                IF ( av == 0 )  THEN
    221                    DO  i = nxl-1, nxr+1
    222                       DO  j = nys-1, nyn+1
     221                   DO  i = nxlg, nxrg
     222                      DO  j = nysg, nyng
    223223                         local_pf(i,j,nzb+1) = SUM( ql(nzb:nzt,j,i) * &
    224224                                                    dzw(1:nzt+1) )
     
    226226                   ENDDO
    227227                ELSE
    228                    DO  i = nxl-1, nxr+1
    229                       DO  j = nys-1, nyn+1
     228                   DO  i = nxlg, nxrg
     229                      DO  j = nysg, nyng
    230230                         local_pf(i,j,nzb+1) = lwp_av(j,i)
    231231                      ENDDO
     
    248248                   IF ( simulated_time >= particle_advection_start )  THEN
    249249                      tend = prt_count
    250                       CALL exchange_horiz( tend )
     250                      CALL exchange_horiz( tend, nbgp )
    251251                   ELSE
    252252                      tend = 0.0
    253253                   ENDIF
    254                    DO  i = nxl-1, nxr+1
    255                       DO  j = nys-1, nyn+1
     254                   DO  i = nxlg, nxrg
     255                      DO  j = nysg, nyng
    256256                         DO  k = nzb, nzt+1
    257257                            local_pf(i,j,k) = tend(k,j,i)
     
    261261                   resorted = .TRUE.
    262262                ELSE
    263                    CALL exchange_horiz( pc_av )
     263                   CALL exchange_horiz( pc_av, nbgp )
    264264                   to_be_resorted => pc_av
    265265                ENDIF
     
    287287                         ENDDO
    288288                      ENDDO
    289                       CALL exchange_horiz( tend )
     289                      CALL exchange_horiz( tend, nbgp )
    290290                   ELSE
    291291                      tend = 0.0
    292                    ENDIF
    293                    DO  i = nxl-1, nxr+1
    294                       DO  j = nys-1, nyn+1
     292                   END IF
     293                   DO  i = nxlg, nxrg
     294                      DO  j = nysg, nyng
    295295                         DO  k = nzb, nzt+1
    296296                            local_pf(i,j,k) = tend(k,j,i)
     
    300300                   resorted = .TRUE.
    301301                ELSE
    302                    CALL exchange_horiz( pr_av )
     302                   CALL exchange_horiz( pr_av, nbgp )
    303303                   to_be_resorted => pr_av
    304304                ENDIF
     
    306306             CASE ( 'pra*_xy' )        ! 2d-array / integral quantity => no av
    307307                CALL exchange_horiz_2d( precipitation_amount )
    308                 DO  i = nxl-1, nxr+1
    309                    DO  j = nys-1, nyn+1
     308                   DO  i = nxlg, nxrg
     309                      DO  j = nysg, nyng
    310310                      local_pf(i,j,nzb+1) =  precipitation_amount(j,i)
    311311                   ENDDO
     
    319319                IF ( av == 0 )  THEN
    320320                   CALL exchange_horiz_2d( precipitation_rate )
    321                    DO  i = nxl-1, nxr+1
    322                       DO  j = nys-1, nyn+1
     321                   DO  i = nxlg, nxrg
     322                      DO  j = nysg, nyng
    323323                         local_pf(i,j,nzb+1) =  precipitation_rate(j,i)
    324324                      ENDDO
     
    326326                ELSE
    327327                   CALL exchange_horiz_2d( precipitation_rate_av )
    328                    DO  i = nxl-1, nxr+1
    329                       DO  j = nys-1, nyn+1
     328                   DO  i = nxlg, nxrg
     329                      DO  j = nysg, nyng
    330330                         local_pf(i,j,nzb+1) =  precipitation_rate_av(j,i)
    331331                      ENDDO
     
    341341                      to_be_resorted => pt
    342342                   ELSE
    343                       DO  i = nxl-1, nxr+1
    344                          DO  j = nys-1, nyn+1
     343                   DO  i = nxlg, nxrg
     344                      DO  j = nysg, nyng
    345345                            DO  k = nzb, nzt+1
    346346                               local_pf(i,j,k) = pt(k,j,i) + l_d_cp *    &
     
    399399             CASE ( 'qsws*_xy' )        ! 2d-array
    400400                IF ( av == 0 ) THEN
    401                    DO  i = nxl-1, nxr+1
    402                       DO  j = nys-1, nyn+1
     401                   DO  i = nxlg, nxrg
     402                      DO  j = nysg, nyng
    403403                         local_pf(i,j,nzb+1) =  qsws(j,i)
    404404                      ENDDO
    405405                   ENDDO
    406406                ELSE
    407                    DO  i = nxl-1, nxr+1
    408                       DO  j = nys-1, nyn+1
     407                   DO  i = nxlg, nxrg
     408                      DO  j = nysg, nyng
    409409                         local_pf(i,j,nzb+1) =  qsws_av(j,i)
    410410                      ENDDO
     
    417417             CASE ( 'qv_xy', 'qv_xz', 'qv_yz' )
    418418                IF ( av == 0 )  THEN
    419                    DO  i = nxl-1, nxr+1
    420                       DO  j = nys-1, nyn+1
     419                   DO  i = nxlg, nxrg
     420                      DO  j = nysg, nyng
    421421                         DO  k = nzb, nzt+1
    422422                            local_pf(i,j,k) = q(k,j,i) - ql(k,j,i)
     
    453453             CASE ( 'shf*_xy' )        ! 2d-array
    454454                IF ( av == 0 ) THEN
    455                    DO  i = nxl-1, nxr+1
    456                       DO  j = nys-1, nyn+1
     455                   DO  i = nxlg, nxrg
     456                      DO  j = nysg, nyng
    457457                         local_pf(i,j,nzb+1) =  shf(j,i)
    458458                      ENDDO
    459459                   ENDDO
    460460                ELSE
    461                    DO  i = nxl-1, nxr+1
    462                       DO  j = nys-1, nyn+1
     461                   DO  i = nxlg, nxrg
     462                      DO  j = nysg, nyng
    463463                         local_pf(i,j,nzb+1) =  shf_av(j,i)
    464464                      ENDDO
     
    471471             CASE ( 't*_xy' )        ! 2d-array
    472472                IF ( av == 0 )  THEN
    473                    DO  i = nxl-1, nxr+1
    474                       DO  j = nys-1, nyn+1
     473                   DO  i = nxlg, nxrg
     474                      DO  j = nysg, nyng
    475475                         local_pf(i,j,nzb+1) = ts(j,i)
    476476                      ENDDO
    477477                   ENDDO
    478478                ELSE
    479                    DO  i = nxl-1, nxr+1
    480                       DO  j = nys-1, nyn+1
     479                   DO  i = nxlg, nxrg
     480                      DO  j = nysg, nyng
    481481                         local_pf(i,j,nzb+1) = ts_av(j,i)
    482482                      ENDDO
     
    503503             CASE ( 'u*_xy' )        ! 2d-array
    504504                IF ( av == 0 )  THEN
    505                    DO  i = nxl-1, nxr+1
    506                       DO  j = nys-1, nyn+1
     505                   DO  i = nxlg, nxrg
     506                      DO  j = nysg, nyng
    507507                         local_pf(i,j,nzb+1) = us(j,i)
    508508                      ENDDO
    509509                   ENDDO
    510510                ELSE
    511                    DO  i = nxl-1, nxr+1
    512                       DO  j = nys-1, nyn+1
     511                   DO  i = nxlg, nxrg
     512                      DO  j = nysg, nyng
    513513                         local_pf(i,j,nzb+1) = us_av(j,i)
    514514                      ENDDO
     
    551551             CASE ( 'z0*_xy' )        ! 2d-array
    552552                IF ( av == 0 ) THEN
    553                    DO  i = nxl-1, nxr+1
    554                       DO  j = nys-1, nyn+1
     553                   DO  i = nxlg, nxrg
     554                      DO  j = nysg, nyng
    555555                         local_pf(i,j,nzb+1) =  z0(j,i)
    556556                      ENDDO
    557557                   ENDDO
    558558                ELSE
    559                    DO  i = nxl-1, nxr+1
    560                       DO  j = nys-1, nyn+1
     559                   DO  i = nxlg, nxrg
     560                      DO  j = nysg, nyng
    561561                         local_pf(i,j,nzb+1) =  z0_av(j,i)
    562562                      ENDDO
     
    593593!--       Resort the array to be output, if not done above
    594594          IF ( .NOT. resorted )  THEN
    595              DO  i = nxl-1, nxr+1
    596                 DO  j = nys-1, nyn+1
     595             DO  i = nxlg, nxrg
     596                DO  j = nysg, nyng
    597597                   DO  k = nzb, nzt+1
    598598                      local_pf(i,j,k) = to_be_resorted(k,j,i)
     
    647647!--                   Carry out the averaging (all data are on the PE)
    648648                      DO  k = nzb, nzt+1
    649                          DO  j = nys-1, nyn+1
    650                             DO  i = nxl-1, nxr+1
     649                         DO  j = nysg, nyng
     650                            DO  i = nxlg, nxrg
    651651                               local_2d(i,j) = local_2d(i,j) + local_pf(i,j,k)
    652652                            ENDDO
     
    654654                      ENDDO
    655655
    656                       local_2d = local_2d / ( nzt -nzb + 2.0 )
     656                      local_2d = local_2d / ( nzt -nzb + 2.0)
    657657
    658658                   ELSE
     
    723723                         ENDIF
    724724#endif
    725                          WRITE ( 21 )  nxl-1, nxr+1, nys-1, nyn+1
     725                         WRITE ( 21 )  nxlg, nxrg, nysg, nyng
    726726                         WRITE ( 21 )  local_2d
    727727
     
    734734                         CALL MPI_BARRIER( comm2d, ierr )
    735735
    736                          ngp = ( nxr-nxl+3 ) * ( nyn-nys+3 )
     736                         ngp = ( nxrg-nxlg+1 ) * ( nyng-nysg+1 )
    737737                         IF ( myid == 0 )  THEN
    738738!
    739739!--                         Local array can be relocated directly.
    740                             total_2d(nxl-1:nxr+1,nys-1:nyn+1) = local_2d
     740                            total_2d(nxlg:nxrg,nysg:nyng) = local_2d
    741741!
    742742!--                         Receive data from all other PEs.
     
    760760!--                         Output of the total cross-section.
    761761                            IF ( iso2d_output )  THEN
    762                                WRITE (21)  total_2d(0:nx+1,0:ny+1)
     762                               WRITE (21)  total_2d(-nbgp:nx+nbgp,-nbgp:ny+nbgp)
    763763                            ENDIF
    764764!
    765765!--                         Relocate the local array for the next loop increment
    766766                            DEALLOCATE( local_2d )
    767                             ALLOCATE( local_2d(nxl-1:nxr+1,nys-1:nyn+1) )
     767                            ALLOCATE( local_2d(nxlg:nxrg,nysg:nyng) )
    768768
    769769#if defined( __netcdf )
     
    789789!
    790790!--                         First send the local index limits to PE0
    791                             ind(1) = nxl-1; ind(2) = nxr+1
    792                             ind(3) = nys-1; ind(4) = nyn+1
     791                            ind(1) = nxlg; ind(2) = nxrg
     792                            ind(3) = nysg; ind(4) = nyng
    793793                            CALL MPI_SEND( ind(1), 4, MPI_INTEGER, 0, 0, &
    794794                                           comm2d, ierr )
    795795!
    796796!--                         Send data to PE0
    797                             CALL MPI_SEND( local_2d(nxl-1,nys-1), ngp, &
     797                            CALL MPI_SEND( local_2d(nxlg,nysg), ngp, &
    798798                                           MPI_REAL, 0, 1, comm2d, ierr )
    799799                         ENDIF
     
    882882
    883883                   ENDIF
     884
    884885!
    885886!--                If required, carry out averaging along y
    886887                   IF ( section(is,s) == -1 )  THEN
    887888
    888                       ALLOCATE( local_2d_l(nxl-1:nxr+1,nzb:nzt+1) )
     889                      ALLOCATE( local_2d_l(nxlg:nxrg,nzb:nzt+1) )
    889890                      local_2d_l = 0.0
    890                       ngp = ( nxr-nxl+3 ) * ( nzt-nzb+2 )
     891                      ngp = ( nxrg-nxlg+1 ) * ( nzt-nzb+2 )
    891892!
    892893!--                   First local averaging on the PE
    893894                      DO  k = nzb, nzt+1
    894895                         DO  j = nys, nyn
    895                             DO  i = nxl-1, nxr+1
     896                            DO  i = nxlg, nxrg
    896897                               local_2d_l(i,k) = local_2d_l(i,k) + &
    897898                                                 local_pf(i,j,k)
     
    903904!--                   Now do the averaging over all PEs along y
    904905                      IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
    905                       CALL MPI_ALLREDUCE( local_2d_l(nxl-1,nzb),              &
    906                                           local_2d(nxl-1,nzb), ngp, MPI_REAL, &
     906                      CALL MPI_ALLREDUCE( local_2d_l(nxlg,nzb),              &
     907                                          local_2d(nxlg,nzb), ngp, MPI_REAL, &
    907908                                          MPI_SUM, comm1dy, ierr )
    908909#else
     
    936937!--                   BEGIN WORKAROUND---------------------------------------
    937938                      IF ( npey /= 1  .AND.  section(is,s) /= -1)  THEN
    938                          ALLOCATE( local_2d_l(nxl-1:nxr+1,nzb:nzt+1) )
     939                         ALLOCATE( local_2d_l(nxlg:nxrg,nzb:nzt+1) )
    939940                         local_2d_l = 0.0
    940941                         IF ( section(is,s) >= nys .AND. section(is,s) <= nyn )&
     
    945946!
    946947!--                      Distribute data over all PEs along y
    947                          ngp = ( nxr-nxl+3 ) * ( nzt-nzb+2 )
     948                         ngp = ( nxrg-nxlg+1 ) * ( nzt-nzb+2 )
    948949                         IF ( collective_wait ) CALL MPI_BARRIER( comm2d, ierr )
    949                          CALL MPI_ALLREDUCE( local_2d_l(nxl-1,nzb),            &
    950                                              local_2d(nxl-1,nzb), ngp,         &
     950                         CALL MPI_ALLREDUCE( local_2d_l(nxlg,nzb),            &
     951                                             local_2d(nxlg,nzb), ngp,         &
    951952                                             MPI_REAL, MPI_SUM, comm1dy, ierr )
    952953#else
     
    10221023                              ( section(is,s) == -1  .AND.  nys-1 == -1 ) )  &
    10231024                         THEN
    1024                             WRITE (22)  nxl-1, nxr+1, nzb, nzt+1
     1025                            WRITE (22)  nxlg, nxrg, nzb, nzt+1
    10251026                            WRITE (22)  local_2d
    10261027                         ELSE
     
    10361037                         CALL MPI_BARRIER( comm2d, ierr )
    10371038
    1038                          ngp = ( nxr-nxl+3 ) * ( nzt-nzb+2 )
     1039                         ngp = ( nxrg-nxlg+1 ) * ( nzt-nzb+2 )
    10391040                         IF ( myid == 0 )  THEN
    10401041!
     
    10441045                                 ( section(is,s) == -1  .AND.  nys-1 == -1 ) ) &
    10451046                            THEN
    1046                                total_2d(nxl-1:nxr+1,nzb:nzt+1) = local_2d
     1047                               total_2d(nxlg:nxrg,nzb:nzt+1) = local_2d
    10471048                            ENDIF
    10481049!
     
    10731074!--                         Output of the total cross-section.
    10741075                            IF ( iso2d_output )  THEN
    1075                                WRITE (22)  total_2d(0:nx+1,nzb:nzt+1)
     1076                               WRITE (22)  total_2d(-nbgp:nx+nbgp,nzb:nzt+1)
    10761077                            ENDIF
    10771078!
    10781079!--                         Relocate the local array for the next loop increment
    10791080                            DEALLOCATE( local_2d )
    1080                             ALLOCATE( local_2d(nxl-1:nxr+1,nzb:nzt+1) )
     1081                            ALLOCATE( local_2d(nxlg:nxrg,nzb:nzt+1) )
    10811082
    10821083#if defined( __netcdf )
     
    10991100                                 ( section(is,s) == -1  .AND.  nys-1 == -1 ) ) &
    11001101                            THEN
    1101                                ind(1) = nxl-1; ind(2) = nxr+1
     1102                               ind(1) = nxlg; ind(2) = nxrg
    11021103                               ind(3) = nzb;   ind(4) = nzt+1
    11031104                            ELSE
     
    11101111!--                         If applicable, send data to PE0.
    11111112                            IF ( ind(1) /= -9999 )  THEN
    1112                                CALL MPI_SEND( local_2d(nxl-1,nzb), ngp, &
     1113                               CALL MPI_SEND( local_2d(nxlg,nzb), ngp, &
    11131114                                              MPI_REAL, 0, 1, comm2d, ierr )
    11141115                            ENDIF
     
    11871188                   IF ( section(is,s) == -1 )  THEN
    11881189
    1189                       ALLOCATE( local_2d_l(nys-1:nyn+1,nzb:nzt+1) )
     1190                      ALLOCATE( local_2d_l(nysg:nyng,nzb:nzt+1) )
    11901191                      local_2d_l = 0.0
    1191                       ngp = ( nyn-nys+3 ) * ( nzt-nzb+2 )
     1192                      ngp = ( nyng-nysg+1 ) * ( nzt-nzb+2 )
    11921193!
    11931194!--                   First local averaging on the PE
    11941195                      DO  k = nzb, nzt+1
    1195                          DO  j = nys-1, nyn+1
     1196                         DO  j = nysg, nyng
    11961197                            DO  i = nxl, nxr
    11971198                               local_2d_l(j,k) = local_2d_l(j,k) + &
     
    12041205!--                   Now do the averaging over all PEs along x
    12051206                      IF ( collective_wait )  CALL MPI_BARRIER( comm2d, ierr )
    1206                       CALL MPI_ALLREDUCE( local_2d_l(nys-1,nzb),              &
    1207                                           local_2d(nys-1,nzb), ngp, MPI_REAL, &
     1207                      CALL MPI_ALLREDUCE( local_2d_l(nysg,nzb),              &
     1208                                          local_2d(nysg,nzb), ngp, MPI_REAL, &
    12081209                                          MPI_SUM, comm1dx, ierr )
    12091210#else
     
    12371238!--                   BEGIN WORKAROUND---------------------------------------
    12381239                      IF ( npex /= 1  .AND.  section(is,s) /= -1)  THEN
    1239                          ALLOCATE( local_2d_l(nys-1:nyn+1,nzb:nzt+1) )
     1240                         ALLOCATE( local_2d_l(nysg:nyng,nzb:nzt+1) )
    12401241                         local_2d_l = 0.0
    12411242                         IF ( section(is,s) >= nxl .AND. section(is,s) <= nxr )&
     
    12461247!
    12471248!--                      Distribute data over all PEs along x
    1248                          ngp = ( nyn-nys+3 ) * ( nzt-nzb+2 )
     1249                         ngp = ( nyng-nysg+1 ) * ( nzt-nzb + 2 )
    12491250                         IF ( collective_wait ) CALL MPI_BARRIER( comm2d, ierr )
    1250                          CALL MPI_ALLREDUCE( local_2d_l(nys-1,nzb),            &
    1251                                              local_2d(nys-1,nzb), ngp,         &
     1251                         CALL MPI_ALLREDUCE( local_2d_l(nysg,nzb),            &
     1252                                             local_2d(nysg,nzb), ngp,         &
    12521253                                             MPI_REAL, MPI_SUM, comm1dx, ierr )
    12531254#else
     
    13231324                              ( section(is,s) == -1  .AND.  nxl-1 == -1 ) )  &
    13241325                         THEN
    1325                             WRITE (23)  nys-1, nyn+1, nzb, nzt+1
     1326                            WRITE (23)  nysg, nyng, nzb, nzt+1
    13261327                            WRITE (23)  local_2d
    13271328                         ELSE
     
    13371338                         CALL MPI_BARRIER( comm2d, ierr )
    13381339
    1339                          ngp = ( nyn-nys+3 ) * ( nzt-nzb+2 )
     1340                         ngp = ( nyng-nysg+1 ) * ( nzt-nzb+2 )
    13401341                         IF ( myid == 0 )  THEN
    13411342!
     
    13451346                                 ( section(is,s) == -1  .AND.  nxl-1 == -1 ) ) &
    13461347                            THEN
    1347                                total_2d(nys-1:nyn+1,nzb:nzt+1) = local_2d
     1348                               total_2d(nysg:nyng,nzb:nzt+1) = local_2d
    13481349                            ENDIF
    13491350!
     
    13791380!--                         Relocate the local array for the next loop increment
    13801381                            DEALLOCATE( local_2d )
    1381                             ALLOCATE( local_2d(nys-1:nyn+1,nzb:nzt+1) )
     1382                            ALLOCATE( local_2d(nysg:nyng,nzb:nzt+1) )
    13821383
    13831384#if defined( __netcdf )
     
    14001401                                 ( section(is,s) == -1  .AND.  nxl-1 == -1 ) ) &
    14011402                            THEN
    1402                                ind(1) = nys-1; ind(2) = nyn+1
     1403                               ind(1) = nysg; ind(2) = nyng
    14031404                               ind(3) = nzb;   ind(4) = nzt+1
    14041405                            ELSE
     
    14111412!--                         If applicable, send data to PE0.
    14121413                            IF ( ind(1) /= -9999 )  THEN
    1413                                CALL MPI_SEND( local_2d(nys-1,nzb), ngp, &
     1414                               CALL MPI_SEND( local_2d(nysg,nzb), ngp, &
    14141415                                              MPI_REAL, 0, 1, comm2d, ierr )
    14151416                            ENDIF
Note: See TracChangeset for help on using the changeset viewer.