Changeset 1111 for palm/trunk/SOURCE/time_integration.f90

Timestamp:

Mar 8, 2013 11:54:10 PM (11 years ago)

Author:

raasch

Message:

New:
---

GPU porting of pres, swap_timelevel. Adjustments of openACC directives.
Further porting of poisfft, which now runs completely on GPU without any
host/device data transfer for serial an parallel runs (but parallel runs
require data transfer before and after the MPI transpositions).
GPU-porting of tridiagonal solver:
tridiagonal routines split into extermal subroutines (instead using CONTAINS),
no distinction between parallel/non-parallel in poisfft and tridia any more,
tridia routines moved to end of file because of probable bug in PGI compiler
(otherwise "invalid device function" is indicated during runtime).
(cuda_fft_interfaces, fft_xy, flow_statistics, init_3d_model, palm, poisfft, pres, prognostic_equations, swap_timelevel, time_integration, transpose)
output of accelerator board information. (header)

optimization of tridia routines: constant elements and coefficients of tri are
stored in seperate arrays ddzuw and tric, last dimension of tri reduced from 5 to 2,
(init_grid, init_3d_model, modules, palm, poisfft)

poisfft_init is now called internally from poisfft,
(Makefile, Makefile_check, init_pegrid, poisfft, poisfft_hybrid)

CPU-time per grid point and timestep is output to CPU_MEASURES file
(cpu_statistics, modules, time_integration)

Changed:

---

resorting from/to array work changed, work now has 4 dimensions instead of 1 (transpose)
array diss allocated only if required (init_3d_model)

pressure boundary condition "Neumann+inhomo" removed from the code
(check_parameters, header, poisfft, poisfft_hybrid, pres)

Errors:

---

bugfix: dependency added for cuda_fft_interfaces (Makefile)
bugfix: CUDA fft plans adjusted for domain decomposition (before they always
used total domain) (fft_xy)

File:

• palm/trunk/SOURCE/time_integration.f90 (modified) (9 diffs)

palm/trunk/SOURCE/time_integration.f90

@@ -20 +20 @@
 ! Current revisions:
 ! ------------------
-! 
+! +internal timestep counter for cpu statistics added,
+! openACC directives updated
 !
 ! Former revisions:
@@ -238 +239 @@
 !--       Exchange of ghost points (lateral boundary conditions)
           CALL cpu_log( log_point(26), 'exchange-horiz-progn', 'start' )
+          !$acc update host( e_p, pt_p, u_p, v_p, w_p )
           CALL exchange_horiz( u_p, nbgp )
           CALL exchange_horiz( v_p, nbgp )
@@ -272 +274 @@
 !
 !--       Swap the time levels in preparation for the next time step.
+          !$acc update device( e_p, pt_p, u_p, v_p, w_p )
           CALL swap_timelevel
 
@@ -298 +301 @@
              time_disturb = time_disturb + dt_3d
              IF ( time_disturb >= dt_disturb )  THEN
+                !$acc update host( u, v )
                 IF ( hom(nzb+5,1,pr_palm,0) < disturbance_energy_limit )  THEN
                    CALL disturb_field( nzb_u_inner, tend, u )
@@ -310 +314 @@
                    dist_range = 0
                 ENDIF
+                !$acc update device( u, v )
                 time_disturb = time_disturb - dt_disturb
              ENDIF
@@ -321 +326 @@
              CALL pres
           ENDIF
-!
-!--       Update device memory for calculating diffusion quantities and for next
-!--       timestep
-          !$acc update device( e, pt, u, v, w )
-          !$acc update device( q )  if ( allocated( q ) )
 
 !
@@ -351 +351 @@
                 CALL prandtl_fluxes
                 CALL cpu_log( log_point(19), 'prandtl_fluxes', 'stop' )
-!
-!++             Statistics still require updates on host
-                !$acc update host( qs, qsws, rif, shf, ts )
              ENDIF
 
@@ -369 +366 @@
              ENDIF
              CALL cpu_log( log_point(17), 'diffusivities', 'stop' )
-!
-!++          Statistics still require update of diffusivities on host
-             !$acc update host( kh, km )
 
           ENDIF
@@ -379 +373 @@
 !
 !--    Increase simulation time and output times
+       nr_timesteps_this_run      = nr_timesteps_this_run + 1
        current_timestep_number    = current_timestep_number + 1
        simulated_time             = simulated_time   + dt_3d