source: palm/trunk/SOURCE/time_integration.f90 @ 2233

Last change on this file since 2233 was 2233, checked in by suehring, 4 years ago

last commit documented

  • Property svn:keywords set to Id
File size: 41.7 KB
Line 
1!> @file time_integration.f90
2!------------------------------------------------------------------------------!
3! This file is part of PALM.
4!
5! PALM is free software: you can redistribute it and/or modify it under the
6! terms of the GNU General Public License as published by the Free Software
7! Foundation, either version 3 of the License, or (at your option) any later
8! version.
9!
10! PALM is distributed in the hope that it will be useful, but WITHOUT ANY
11! WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
12! A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
13!
14! You should have received a copy of the GNU General Public License along with
15! PALM. If not, see <http://www.gnu.org/licenses/>.
16!
17! Copyright 1997-2017 Leibniz Universitaet Hannover
18!------------------------------------------------------------------------------!
19!
20! Current revisions:
21! ------------------
22!
23!
24! Former revisions:
25! -----------------
26! $Id: time_integration.f90 2233 2017-05-30 18:08:54Z suehring $
27!
28! 2232 2017-05-30 17:47:52Z suehring
29! Adjustments to new topography and surface concept
30! Modify passed parameters for disturb_field
31!
32! 2178 2017-03-17 11:07:39Z hellstea
33! Setting perturbations at all times near inflow boundary is removed
34! in case of nested boundaries
35!
36! 2174 2017-03-13 08:18:57Z maronga
37! Added support for nesting with cloud microphysics
38!
39! 2118 2017-01-17 16:38:49Z raasch
40! OpenACC directives and related code removed
41!
42! 2050 2016-11-08 15:00:55Z gronemeier
43! Implement turbulent outflow condition
44!
45! 2031 2016-10-21 15:11:58Z knoop
46! renamed variable rho to rho_ocean
47!
48! 2011 2016-09-19 17:29:57Z kanani
49! Flag urban_surface is now defined in module control_parameters,
50! removed commented CALLs of global_min_max.
51!
52! 2007 2016-08-24 15:47:17Z kanani
53! Added CALLs for new urban surface model
54!
55! 2000 2016-08-20 18:09:15Z knoop
56! Forced header and separation lines into 80 columns
57!
58! 1976 2016-07-27 13:28:04Z maronga
59! Simplified calls to radiation model
60!
61! 1960 2016-07-12 16:34:24Z suehring
62! Separate humidity and passive scalar
63!
64! 1957 2016-07-07 10:43:48Z suehring
65! flight module added
66!
67! 1919 2016-05-27 14:51:23Z raasch
68! Initial version of purely vertical nesting introduced.
69!
70! 1918 2016-05-27 14:35:57Z raasch
71! determination of time step moved to the end of the time step loop,
72! the first time step is now always calculated before the time step loop (i.e.
73! also in case of restart runs)
74!
75! 1914 2016-05-26 14:44:07Z witha
76! Added call for wind turbine model
77!
78! 1878 2016-04-19 12:30:36Z hellstea
79! Synchronization for nested runs rewritten
80!
81! 1853 2016-04-11 09:00:35Z maronga
82! Adjusted for use with radiation_scheme = constant
83!
84! 1849 2016-04-08 11:33:18Z hoffmann
85! Adapted for modularization of microphysics
86!
87! 1833 2016-04-07 14:23:03Z raasch
88! spectrum renamed spectra_mod, spectra related variables moved to spectra_mod
89!
90! 1831 2016-04-07 13:15:51Z hoffmann
91! turbulence renamed collision_turbulence
92!
93! 1822 2016-04-07 07:49:42Z hoffmann
94! icloud_scheme replaced by microphysics_*
95!
96! 1808 2016-04-05 19:44:00Z raasch
97! output message in case unscheduled radiation calls removed
98!
99! 1797 2016-03-21 16:50:28Z raasch
100! introduction of different datatransfer modes
101!
102! 1791 2016-03-11 10:41:25Z raasch
103! call of pmci_update_new removed
104!
105! 1786 2016-03-08 05:49:27Z raasch
106! +module spectrum
107!
108! 1783 2016-03-06 18:36:17Z raasch
109! switch back of netcdf data format for mask output moved to the mask output
110! routine
111!
112! 1781 2016-03-03 15:12:23Z raasch
113! some pmc calls removed at the beginning (before timeloop),
114! pmc initialization moved to the main program
115!
116! 1764 2016-02-28 12:45:19Z raasch
117! PMC_ACTIVE flags removed,
118! bugfix: nest synchronization after first call of timestep
119!
120! 1762 2016-02-25 12:31:13Z hellstea
121! Introduction of nested domain feature
122!
123! 1736 2015-12-04 08:56:33Z raasch
124! no perturbations added to total domain if energy limit has been set zero
125!
126! 1691 2015-10-26 16:17:44Z maronga
127! Added option for spin-ups without land surface and radiation models. Moved calls
128! for radiation and lan surface schemes.
129!
130! 1682 2015-10-07 23:56:08Z knoop
131! Code annotations made doxygen readable
132!
133! 1671 2015-09-25 03:29:37Z raasch
134! bugfix: ghostpoint exchange for array diss in case that sgs velocities are used
135! for particles
136!
137! 1585 2015-04-30 07:05:52Z maronga
138! Moved call of radiation scheme. Added support for RRTM
139!
140! 1551 2015-03-03 14:18:16Z maronga
141! Added interface for different radiation schemes.
142!
143! 1496 2014-12-02 17:25:50Z maronga
144! Added calls for the land surface model and radiation scheme
145!
146! 1402 2014-05-09 14:25:13Z raasch
147! location messages modified
148!
149! 1384 2014-05-02 14:31:06Z raasch
150! location messages added
151!
152! 1380 2014-04-28 12:40:45Z heinze
153! CALL of nudge_ref added
154! bc_pt_t_val and bc_q_t_val are updated in case nudging is used
155!
156! 1365 2014-04-22 15:03:56Z boeske
157! Reset sums_ls_l to zero at each timestep
158! +sums_ls_l
159! Calculation of reference state (previously in subroutine calc_mean_profile)
160
161! 1342 2014-03-26 17:04:47Z kanani
162! REAL constants defined as wp-kind
163!
164! 1320 2014-03-20 08:40:49Z raasch
165! ONLY-attribute added to USE-statements,
166! kind-parameters added to all INTEGER and REAL declaration statements,
167! kinds are defined in new module kinds,
168! old module precision_kind is removed,
169! revision history before 2012 removed,
170! comment fields (!:) to be used for variable explanations added to
171! all variable declaration statements
172! 1318 2014-03-17 13:35:16Z raasch
173! module interfaces removed
174!
175! 1308 2014-03-13 14:58:42Z fricke
176! +netcdf_data_format_save
177! For masked data, parallel netcdf output is not tested so far, hence
178! netcdf_data_format is switched back to non-paralell output.
179!
180! 1276 2014-01-15 13:40:41Z heinze
181! Use LSF_DATA also in case of Dirichlet bottom boundary condition for scalars
182!
183! 1257 2013-11-08 15:18:40Z raasch
184! acc-update-host directive for timestep removed
185!
186! 1241 2013-10-30 11:36:58Z heinze
187! Generalize calc_mean_profile for wider use
188! Determine shf and qsws in dependence on data from LSF_DATA
189! Determine ug and vg in dependence on data from LSF_DATA
190! 1221 2013-09-10 08:59:13Z raasch
191! host update of arrays before timestep is called
192!
193! 1179 2013-06-14 05:57:58Z raasch
194! mean profiles for reference state are only calculated if required,
195! small bugfix for background communication
196!
197! 1171 2013-05-30 11:27:45Z raasch
198! split of prognostic_equations deactivated (comment lines), for the time being
199!
200! 1128 2013-04-12 06:19:32Z raasch
201! asynchronous transfer of ghost point data realized for acc-optimized version:
202! prognostic_equations are first called two times for those points required for
203! the left-right and north-south exchange, respectively, and then for the
204! remaining points,
205! those parts requiring global communication moved from prognostic_equations to
206! here
207!
208! 1115 2013-03-26 18:16:16Z hoffmann
209! calculation of qr and nr is restricted to precipitation
210!
211! 1113 2013-03-10 02:48:14Z raasch
212! GPU-porting of boundary conditions,
213! openACC directives updated
214! formal parameter removed from routine boundary_conds
215!
216! 1111 2013-03-08 23:54:10Z raasch
217! +internal timestep counter for cpu statistics added,
218! openACC directives updated
219!
220! 1092 2013-02-02 11:24:22Z raasch
221! unused variables removed
222!
223! 1065 2012-11-22 17:42:36Z hoffmann
224! exchange of diss (dissipation rate) in case of turbulence = .TRUE. added
225!
226! 1053 2012-11-13 17:11:03Z hoffmann
227! exchange of ghost points for nr, qr added
228!
229! 1036 2012-10-22 13:43:42Z raasch
230! code put under GPL (PALM 3.9)
231!
232! 1019 2012-09-28 06:46:45Z raasch
233! non-optimized version of prognostic_equations removed
234!
235! 1015 2012-09-27 09:23:24Z raasch
236! +call of prognostic_equations_acc
237!
238! 1001 2012-09-13 14:08:46Z raasch
239! all actions concerning leapfrog- and upstream-spline-scheme removed
240!
241! 849 2012-03-15 10:35:09Z raasch
242! advec_particles renamed lpm, first_call_advec_particles renamed first_call_lpm
243!
244! 825 2012-02-19 03:03:44Z raasch
245! wang_collision_kernel renamed wang_kernel
246!
247! Revision 1.1  1997/08/11 06:19:04  raasch
248! Initial revision
249!
250!
251! Description:
252! ------------
253!> Integration in time of the model equations, statistical analysis and graphic
254!> output
255!------------------------------------------------------------------------------!
256 SUBROUTINE time_integration
257 
258
259    USE advec_ws,                                                              &
260        ONLY:  ws_statistics
261
262    USE arrays_3d,                                                             &
263        ONLY:  diss, dzu, e, e_p, nr, nr_p, prho, pt, pt_p, pt_init, q_init, q,&
264               ql, ql_c, ql_v, ql_vp, qr, qr_p, q_p, ref_state, rho_ocean,     &
265               s, s_p, sa_p, tend, u, u_p, v, vpt, v_p, w, w_p
266
267    USE calc_mean_profile_mod,                                                 &
268        ONLY:  calc_mean_profile
269
270    USE control_parameters,                                                    &
271        ONLY:  advected_distance_x, advected_distance_y, average_count_3d,     &
272               averaging_interval, averaging_interval_pr,                      &
273               bc_lr_cyc, bc_ns_cyc, bc_pt_t_val,                              &
274               bc_q_t_val, call_psolver_at_all_substeps, cloud_droplets,       &
275               cloud_physics, constant_flux_layer, constant_heatflux,          &
276               create_disturbances, dopr_n, constant_diffusion, coupling_mode, &
277               coupling_start_time, current_timestep_number,                   &
278               disturbance_created, disturbance_energy_limit, dist_range,      &
279               do_sum, dt_3d, dt_averaging_input, dt_averaging_input_pr,       &
280               dt_coupling, dt_data_output_av, dt_disturb, dt_do2d_xy,         &
281               dt_do2d_xz, dt_do2d_yz, dt_do3d, dt_domask,dt_dopts, dt_dopr,   &
282               dt_dopr_listing, dt_dots, dt_dvrp, dt_run_control,              &
283               end_time, first_call_lpm, galilei_transformation, humidity,     &
284               intermediate_timestep_count, intermediate_timestep_count_max,   &
285               land_surface, large_scale_forcing,                              &
286               loop_optimization, lsf_surf, lsf_vert, masks,                   &
287               microphysics_seifert, mid, nest_domain,                         &
288               neutral, nr_timesteps_this_run, nudging,                        &
289               ocean, passive_scalar,                                          &
290               prho_reference, pt_reference, pt_slope_offset, random_heatflux, &
291               run_coupled, simulated_time, simulated_time_chr,                &
292               skip_time_do2d_xy, skip_time_do2d_xz, skip_time_do2d_yz,        &
293               skip_time_do3d, skip_time_domask, skip_time_dopr,               &
294               skip_time_data_output_av, sloping_surface,                      &
295               stop_dt, terminate_coupled, terminate_run, timestep_scheme,     &
296               time_coupling, time_do2d_xy, time_do2d_xz, time_do2d_yz,        &
297               time_do3d, time_domask, time_dopr, time_dopr_av,                &
298               time_dopr_listing, time_dopts, time_dosp, time_dosp_av,         &
299               time_dots, time_do_av, time_do_sla, time_disturb, time_dvrp,    &
300               time_run_control, time_since_reference_point,                   &
301               turbulent_inflow, turbulent_outflow, urban_surface,             &
302               use_initial_profile_as_reference,                               &
303               use_single_reference_value, u_gtrans, v_gtrans, virtual_flight, &
304               ws_scheme_mom, ws_scheme_sca
305
306    USE cpulog,                                                                &
307        ONLY:  cpu_log, log_point, log_point_s
308
309    USE flight_mod,                                                            &
310        ONLY:  flight_measurement
311
312
313    USE indices,                                                               &
314        ONLY:  nbgp, nx, nxl, nxlg, nxr, nxrg, nyn, nyng, nys, nysg, nzb, nzt
315
316    USE interaction_droplets_ptq_mod,                                          &
317        ONLY:  interaction_droplets_ptq
318
319    USE interfaces
320
321    USE kinds
322
323    USE land_surface_model_mod,                                                &
324        ONLY:  lsm_energy_balance, lsm_soil_model,                             &
325               skip_time_do_lsm
326
327    USE ls_forcing_mod,                                                        &
328        ONLY:  ls_forcing_surf, ls_forcing_vert
329
330    USE microphysics_mod,                                                      &
331        ONLY: collision_turbulence
332
333    USE nudge_mod,                                                             &
334        ONLY:  calc_tnudge, nudge_ref
335
336    USE particle_attributes,                                                   &
337        ONLY:  particle_advection, particle_advection_start,                   &
338               use_sgs_for_particles, wang_kernel
339
340    USE pegrid
341
342    USE pmc_interface,                                                         &
343        ONLY:  nested_run, nesting_mode, pmci_datatrans,                       &
344               pmci_ensure_nest_mass_conservation, pmci_synchronize
345
346    USE production_e_mod,                                                      &
347        ONLY:  production_e_init
348
349    USE progress_bar,                                                          &
350        ONLY:  finish_progress_bar, output_progress_bar
351
352    USE prognostic_equations_mod,                                              &
353        ONLY:  prognostic_equations_cache, prognostic_equations_vector
354
355    USE radiation_model_mod,                                                   &
356        ONLY: dt_radiation, force_radiation_call, radiation, radiation_control,&
357              skip_time_do_radiation, time_radiation
358
359    USE spectra_mod,                                                           &
360        ONLY: average_count_sp, averaging_interval_sp, calc_spectra, dt_dosp,  &
361              skip_time_dosp
362
363    USE statistics,                                                            &
364        ONLY:  flow_statistics_called, hom, pr_palm, sums_ls_l, u_max,         &
365               u_max_ijk, v_max, v_max_ijk, w_max, w_max_ijk
366
367    USE surface_layer_fluxes_mod,                                              &
368        ONLY:  surface_layer_fluxes
369
370    USE surface_mod,                                                           &
371        ONLY:  surf_def_h, surf_lsm_h, surf_usm_h
372
373    USE urban_surface_mod,                                                     &
374        ONLY:  usm_material_heat_model, usm_material_model,                    &
375               usm_radiation, usm_surface_energy_balance             
376
377    USE user_actions_mod,                                                      &
378        ONLY:  user_actions
379
380    USE wind_turbine_model_mod,                                                &
381        ONLY:  wind_turbine, wtm_forces
382
383    IMPLICIT NONE
384
385    CHARACTER (LEN=9) ::  time_to_string          !<
386
387    REAL(wp) ::  dt_3d_old  !< temporary storage of timestep to be used for
388                            !< steering of run control output interval
389
390!
391!-- At beginning determine the first time step
392    CALL timestep
393
394!
395!-- Synchronize the timestep in case of nested run.
396    IF ( nested_run )  THEN
397!
398!--    Synchronization by unifying the time step.
399!--    Global minimum of all time-steps is used for all.
400       CALL pmci_synchronize
401    ENDIF
402
403!
404!-- Determine and print out the run control quantities before the first time
405!-- step of this run. For the initial run, some statistics (e.g. divergence)
406!-- need to be determined first.
407    IF ( simulated_time == 0.0_wp )  CALL flow_statistics
408    CALL run_control
409
410!
411!-- Data exchange between coupled models in case that a call has been omitted
412!-- at the end of the previous run of a job chain.
413    IF ( coupling_mode /= 'uncoupled'  .AND.  run_coupled )  THEN
414!
415!--    In case of model termination initiated by the local model the coupler
416!--    must not be called because this would again cause an MPI hang.
417       DO WHILE ( time_coupling >= dt_coupling  .AND.  terminate_coupled == 0 )
418          CALL surface_coupler
419          time_coupling = time_coupling - dt_coupling
420       ENDDO
421       IF (time_coupling == 0.0_wp  .AND.                                      &
422           time_since_reference_point < dt_coupling )                          &
423       THEN
424          time_coupling = time_since_reference_point
425       ENDIF
426    ENDIF
427
428#if defined( __dvrp_graphics )
429!
430!-- Time measurement with dvrp software 
431    CALL DVRP_LOG_EVENT( 2, current_timestep_number )
432#endif
433
434    CALL location_message( 'start with time-stepping', .TRUE. )
435!
436!-- Start of the time loop
437    DO  WHILE ( simulated_time < end_time  .AND.  .NOT. stop_dt  .AND. &
438                .NOT. terminate_run )
439
440       CALL cpu_log( log_point_s(10), 'timesteps', 'start' )
441!
442!--    Determine ug, vg and w_subs in dependence on data from external file
443!--    LSF_DATA
444       IF ( large_scale_forcing .AND. lsf_vert )  THEN
445           CALL ls_forcing_vert ( simulated_time )
446           sums_ls_l = 0.0_wp
447       ENDIF
448
449!
450!--    Set pt_init and q_init to the current profiles taken from
451!--    NUDGING_DATA
452       IF ( nudging )  THEN
453           CALL nudge_ref ( simulated_time )
454!
455!--        Store temperature gradient at the top boundary for possible Neumann
456!--        boundary condition
457           bc_pt_t_val = ( pt_init(nzt+1) - pt_init(nzt) ) / dzu(nzt+1)
458           bc_q_t_val  = ( q_init(nzt+1) - q_init(nzt) ) / dzu(nzt+1)
459       ENDIF
460
461!
462!--    Execute the user-defined actions
463       CALL user_actions( 'before_timestep' )
464
465!
466!--    Calculate forces by wind turbines
467       IF ( wind_turbine )  THEN
468
469          CALL cpu_log( log_point(55), 'wind_turbine', 'start' )
470
471          CALL wtm_forces
472
473          CALL cpu_log( log_point(55), 'wind_turbine', 'stop' )
474
475       ENDIF       
476       
477!
478!--    Start of intermediate step loop
479       intermediate_timestep_count = 0
480       DO  WHILE ( intermediate_timestep_count < &
481                   intermediate_timestep_count_max )
482
483          intermediate_timestep_count = intermediate_timestep_count + 1
484
485!
486!--       Set the steering factors for the prognostic equations which depend
487!--       on the timestep scheme
488          CALL timestep_scheme_steering
489
490!
491!--       Calculate those variables needed in the tendency terms which need
492!--       global communication
493          IF ( .NOT. use_single_reference_value  .AND. &
494               .NOT. use_initial_profile_as_reference )  THEN
495!
496!--          Horizontally averaged profiles to be used as reference state in
497!--          buoyancy terms (WARNING: only the respective last call of
498!--          calc_mean_profile defines the reference state!)
499             IF ( .NOT. neutral )  THEN
500                CALL calc_mean_profile( pt, 4 )
501                ref_state(:)  = hom(:,1,4,0) ! this is used in the buoyancy term
502             ENDIF
503             IF ( ocean )  THEN
504                CALL calc_mean_profile( rho_ocean, 64 )
505                ref_state(:)  = hom(:,1,64,0)
506             ENDIF
507             IF ( humidity )  THEN
508                CALL calc_mean_profile( vpt, 44 )
509                ref_state(:)  = hom(:,1,44,0)
510             ENDIF
511
512          ENDIF
513
514          IF ( .NOT. constant_diffusion )  CALL production_e_init
515          IF ( ( ws_scheme_mom .OR. ws_scheme_sca )  .AND.  &
516               intermediate_timestep_count == 1 )  CALL ws_statistics
517!
518!--       In case of nudging calculate current nudging time scale and horizontal
519!--       means of u, v, pt and q
520          IF ( nudging )  THEN
521             CALL calc_tnudge( simulated_time )
522             CALL calc_mean_profile( u, 1 )
523             CALL calc_mean_profile( v, 2 )
524             CALL calc_mean_profile( pt, 4 )
525             CALL calc_mean_profile( q, 41 )
526          ENDIF
527
528!
529!--       Solve the prognostic equations. A fast cache optimized version with
530!--       only one single loop is used in case of Piascek-Williams advection
531!--       scheme. NEC vector machines use a different version, because
532!--       in the other versions a good vectorization is prohibited due to
533!--       inlining problems.
534          IF ( loop_optimization == 'cache' )  THEN
535             CALL prognostic_equations_cache
536          ELSEIF ( loop_optimization == 'vector' )  THEN
537             CALL prognostic_equations_vector
538          ENDIF
539
540!
541!--       Particle transport/physics with the Lagrangian particle model
542!--       (only once during intermediate steps, because it uses an Euler-step)
543!--       ### particle model should be moved before prognostic_equations, in order
544!--       to regard droplet interactions directly
545          IF ( particle_advection  .AND.                         &
546               simulated_time >= particle_advection_start  .AND. &
547               intermediate_timestep_count == 1 )  THEN
548             CALL lpm
549             first_call_lpm = .FALSE.
550          ENDIF
551
552!
553!--       Interaction of droplets with temperature and specific humidity.
554!--       Droplet condensation and evaporation is calculated within
555!--       advec_particles.
556          IF ( cloud_droplets  .AND.  &
557               intermediate_timestep_count == intermediate_timestep_count_max )&
558          THEN
559             CALL interaction_droplets_ptq
560          ENDIF
561
562!
563!--       Exchange of ghost points (lateral boundary conditions)
564          CALL cpu_log( log_point(26), 'exchange-horiz-progn', 'start' )
565
566          CALL exchange_horiz( u_p, nbgp )
567          CALL exchange_horiz( v_p, nbgp )
568          CALL exchange_horiz( w_p, nbgp )
569          CALL exchange_horiz( pt_p, nbgp )
570          IF ( .NOT. constant_diffusion )  CALL exchange_horiz( e_p, nbgp )
571          IF ( ocean )  THEN
572             CALL exchange_horiz( sa_p, nbgp )
573             CALL exchange_horiz( rho_ocean, nbgp )
574             CALL exchange_horiz( prho, nbgp )
575          ENDIF
576          IF ( humidity )  THEN
577             CALL exchange_horiz( q_p, nbgp )
578             IF ( cloud_physics .AND. microphysics_seifert )  THEN
579                CALL exchange_horiz( qr_p, nbgp )
580                CALL exchange_horiz( nr_p, nbgp )
581             ENDIF
582          ENDIF
583          IF ( cloud_droplets )  THEN
584             CALL exchange_horiz( ql, nbgp )
585             CALL exchange_horiz( ql_c, nbgp )
586             CALL exchange_horiz( ql_v, nbgp )
587             CALL exchange_horiz( ql_vp, nbgp )
588          ENDIF
589          IF ( wang_kernel  .OR.  collision_turbulence  .OR.                &
590               use_sgs_for_particles )  THEN
591             CALL exchange_horiz( diss, nbgp )
592          ENDIF
593          IF ( passive_scalar )  CALL exchange_horiz( s_p, nbgp )
594
595          CALL cpu_log( log_point(26), 'exchange-horiz-progn', 'stop' )
596
597!
598!--       Boundary conditions for the prognostic quantities (except of the
599!--       velocities at the outflow in case of a non-cyclic lateral wall)
600          CALL boundary_conds
601!
602!--       Swap the time levels in preparation for the next time step.
603          CALL swap_timelevel
604
605          IF ( nested_run )  THEN
606
607             CALL cpu_log( log_point(60), 'nesting', 'start' )
608!
609!--          Domain nesting. The data transfer subroutines pmci_parent_datatrans
610!--          and pmci_child_datatrans are called inside the wrapper
611!--          subroutine pmci_datatrans according to the control parameters
612!--          nesting_mode and nesting_datatransfer_mode.
613!--          TO_DO: why is nesting_mode given as a parameter here?
614             CALL pmci_datatrans( nesting_mode )
615
616             IF ( TRIM( nesting_mode ) == 'two-way' .OR.                       &
617                  nesting_mode == 'vertical' )  THEN
618!
619!--             Exchange_horiz is needed for all parent-domains after the
620!--             anterpolation
621                CALL exchange_horiz( u, nbgp )
622                CALL exchange_horiz( v, nbgp )
623                CALL exchange_horiz( w, nbgp )
624                IF ( .NOT. neutral )  CALL exchange_horiz( pt, nbgp )
625
626                IF ( humidity )  THEN
627
628                   CALL exchange_horiz( q, nbgp )
629
630                   IF ( cloud_physics  .AND.  microphysics_seifert )  THEN
631!                        CALL exchange_horiz( qc, nbgp )
632                       CALL exchange_horiz( qr, nbgp )
633!                        CALL exchange_horiz( nc, nbgp )
634                       CALL exchange_horiz( nr, nbgp )
635                   ENDIF
636
637                ENDIF
638
639                IF ( passive_scalar )  CALL exchange_horiz( s, nbgp )
640                IF ( .NOT. constant_diffusion )  CALL exchange_horiz( e, nbgp )
641             ENDIF
642!
643!--          Correct the w top-BC in nest domains to ensure mass conservation.
644!--          This action must never be done for the root domain. Vertical
645!--          nesting implies mass conservation.
646             IF ( nest_domain )  THEN
647                CALL pmci_ensure_nest_mass_conservation
648             ENDIF
649
650             CALL cpu_log( log_point(60), 'nesting', 'stop' )
651
652          ENDIF
653
654!
655!--       Temperature offset must be imposed at cyclic boundaries in x-direction
656!--       when a sloping surface is used
657          IF ( sloping_surface )  THEN
658             IF ( nxl ==  0 )  pt(:,:,nxlg:nxl-1) = pt(:,:,nxlg:nxl-1) - &
659                                                    pt_slope_offset
660             IF ( nxr == nx )  pt(:,:,nxr+1:nxrg) = pt(:,:,nxr+1:nxrg) + &
661                                                    pt_slope_offset
662          ENDIF
663
664!
665!--       Impose a turbulent inflow using the recycling method
666          IF ( turbulent_inflow )  CALL  inflow_turbulence
667
668!
669!--       Set values at outflow boundary using the special outflow condition
670          IF ( turbulent_outflow )  CALL  outflow_turbulence
671
672!
673!--       Impose a random perturbation on the horizontal velocity field
674          IF ( create_disturbances  .AND.                                      &
675               ( call_psolver_at_all_substeps  .AND.                           &
676               intermediate_timestep_count == intermediate_timestep_count_max )&
677          .OR. ( .NOT. call_psolver_at_all_substeps  .AND.                     &
678               intermediate_timestep_count == 1 ) )                            &
679          THEN
680             time_disturb = time_disturb + dt_3d
681             IF ( time_disturb >= dt_disturb )  THEN
682                IF ( disturbance_energy_limit /= 0.0_wp  .AND.                 &
683                     hom(nzb+5,1,pr_palm,0) < disturbance_energy_limit )  THEN
684                   CALL disturb_field( 'u', tend, u )
685                   CALL disturb_field( 'v', tend, v )
686                ELSEIF ( ( .NOT. bc_lr_cyc  .OR.  .NOT. bc_ns_cyc )            &
687                     .AND. .NOT. nest_domain )  THEN
688!
689!--                Runs with a non-cyclic lateral wall need perturbations
690!--                near the inflow throughout the whole simulation
691                   dist_range = 1
692                   CALL disturb_field( 'u', tend, u )
693                   CALL disturb_field( 'v', tend, v )
694                   dist_range = 0
695                ENDIF
696                time_disturb = time_disturb - dt_disturb
697             ENDIF
698          ENDIF
699
700!
701!--       Reduce the velocity divergence via the equation for perturbation
702!--       pressure.
703          IF ( intermediate_timestep_count == 1  .OR. &
704                call_psolver_at_all_substeps )  THEN
705             CALL pres
706          ENDIF
707
708!
709!--       If required, compute liquid water content
710          IF ( cloud_physics )  THEN
711             CALL calc_liquid_water_content
712          ENDIF
713!
714!--       If required, compute virtual potential temperature
715          IF ( humidity )  THEN
716             CALL compute_vpt
717          ENDIF
718
719!
720!--       Compute the diffusion quantities
721          IF ( .NOT. constant_diffusion )  THEN
722
723!
724!--          Determine surface fluxes shf and qsws and surface values
725!--          pt_surface and q_surface in dependence on data from external
726!--          file LSF_DATA respectively
727             IF ( ( large_scale_forcing .AND. lsf_surf ) .AND. &
728                 intermediate_timestep_count == intermediate_timestep_count_max )&
729             THEN
730                CALL ls_forcing_surf ( simulated_time )
731             ENDIF
732
733!
734!--          First the vertical (and horizontal) fluxes in the surface
735!--          (constant flux) layer are computed
736             IF ( constant_flux_layer )  THEN
737                CALL cpu_log( log_point(19), 'surface_layer_fluxes', 'start' )
738                CALL surface_layer_fluxes
739                CALL cpu_log( log_point(19), 'surface_layer_fluxes', 'stop' )
740             ENDIF
741
742!
743!--          If required, solve the energy balance for the surface and run soil
744!--          model. Call for horizontal as well as vertical surfaces
745             IF ( land_surface .AND. simulated_time > skip_time_do_lsm)  THEN
746
747                CALL cpu_log( log_point(54), 'land_surface', 'start' )
748!
749!--             Call for horizontal upward-facing surfaces
750                CALL lsm_energy_balance( .TRUE., -1 )
751                CALL lsm_soil_model( .TRUE., -1 )
752!
753!--             Call for northward-facing surfaces
754                CALL lsm_energy_balance( .FALSE., 0 )
755                CALL lsm_soil_model( .FALSE., 0 )
756!
757!--             Call for southward-facing surfaces
758                CALL lsm_energy_balance( .FALSE., 1 )
759                CALL lsm_soil_model( .FALSE., 1 )
760!
761!--             Call for eastward-facing surfaces
762                CALL lsm_energy_balance( .FALSE., 2 )
763                CALL lsm_soil_model( .FALSE., 2 )
764!
765!--             Call for westward-facing surfaces
766                CALL lsm_energy_balance( .FALSE., 3 )
767                CALL lsm_soil_model( .FALSE., 3 )
768
769                CALL cpu_log( log_point(54), 'land_surface', 'stop' )
770             ENDIF
771
772!
773!--          If required, solve the energy balance for urban surfaces and run
774!--          the material heat model
775             IF (urban_surface) THEN
776                CALL cpu_log( log_point(74), 'urban_surface', 'start' )
777                CALL usm_surface_energy_balance
778                IF ( usm_material_model )  THEN
779                   CALL usm_material_heat_model
780                ENDIF
781                CALL cpu_log( log_point(74), 'urban_surface', 'stop' )
782             ENDIF
783
784!
785!--          Compute the diffusion coefficients
786             CALL cpu_log( log_point(17), 'diffusivities', 'start' )
787             IF ( .NOT. humidity ) THEN
788                IF ( ocean )  THEN
789                   CALL diffusivities( prho, prho_reference )
790                ELSE
791                   CALL diffusivities( pt, pt_reference )
792                ENDIF
793             ELSE
794                CALL diffusivities( vpt, pt_reference )
795             ENDIF
796
797             CALL cpu_log( log_point(17), 'diffusivities', 'stop' )
798
799          ENDIF
800
801!
802!--       If required, calculate radiative fluxes and heating rates
803          IF ( radiation .AND. intermediate_timestep_count                     &
804               == intermediate_timestep_count_max .AND. simulated_time >    &
805               skip_time_do_radiation )  THEN
806
807               time_radiation = time_radiation + dt_3d
808
809             IF ( time_radiation >= dt_radiation .OR. force_radiation_call )   &
810             THEN
811
812                CALL cpu_log( log_point(50), 'radiation', 'start' )
813
814                IF ( .NOT. force_radiation_call )  THEN
815                   time_radiation = time_radiation - dt_radiation
816                ENDIF
817
818                CALL radiation_control
819
820                CALL cpu_log( log_point(50), 'radiation', 'stop' )
821
822                IF (urban_surface)  THEN
823                   CALL cpu_log( log_point(75), 'usm_radiation', 'start' )
824                   CALL usm_radiation
825                   CALL cpu_log( log_point(75), 'usm_radiation', 'stop' )
826                ENDIF
827
828             ENDIF
829          ENDIF
830
831       ENDDO   ! Intermediate step loop
832
833!
834!--    Increase simulation time and output times
835       nr_timesteps_this_run      = nr_timesteps_this_run + 1
836       current_timestep_number    = current_timestep_number + 1
837       simulated_time             = simulated_time   + dt_3d
838       simulated_time_chr         = time_to_string( simulated_time )
839       time_since_reference_point = simulated_time - coupling_start_time
840
841
842
843       IF ( simulated_time >= skip_time_data_output_av )  THEN
844          time_do_av         = time_do_av       + dt_3d
845       ENDIF
846       IF ( simulated_time >= skip_time_do2d_xy )  THEN
847          time_do2d_xy       = time_do2d_xy     + dt_3d
848       ENDIF
849       IF ( simulated_time >= skip_time_do2d_xz )  THEN
850          time_do2d_xz       = time_do2d_xz     + dt_3d
851       ENDIF
852       IF ( simulated_time >= skip_time_do2d_yz )  THEN
853          time_do2d_yz       = time_do2d_yz     + dt_3d
854       ENDIF
855       IF ( simulated_time >= skip_time_do3d    )  THEN
856          time_do3d          = time_do3d        + dt_3d
857       ENDIF
858       DO  mid = 1, masks
859          IF ( simulated_time >= skip_time_domask(mid) )  THEN
860             time_domask(mid)= time_domask(mid) + dt_3d
861          ENDIF
862       ENDDO
863       time_dvrp          = time_dvrp        + dt_3d
864       IF ( simulated_time >= skip_time_dosp )  THEN
865          time_dosp       = time_dosp        + dt_3d
866       ENDIF
867       time_dots          = time_dots        + dt_3d
868       IF ( .NOT. first_call_lpm )  THEN
869          time_dopts      = time_dopts       + dt_3d
870       ENDIF
871       IF ( simulated_time >= skip_time_dopr )  THEN
872          time_dopr       = time_dopr        + dt_3d
873       ENDIF
874       time_dopr_listing          = time_dopr_listing        + dt_3d
875       time_run_control   = time_run_control + dt_3d
876
877!
878!--    Data exchange between coupled models
879       IF ( coupling_mode /= 'uncoupled'  .AND.  run_coupled )  THEN
880          time_coupling = time_coupling + dt_3d
881
882!
883!--       In case of model termination initiated by the local model
884!--       (terminate_coupled > 0), the coupler must be skipped because it would
885!--       cause an MPI intercomminucation hang.
886!--       If necessary, the coupler will be called at the beginning of the
887!--       next restart run.
888          DO WHILE ( time_coupling >= dt_coupling .AND. terminate_coupled == 0 )
889             CALL surface_coupler
890             time_coupling = time_coupling - dt_coupling
891          ENDDO
892       ENDIF
893
894!
895!--    Execute user-defined actions
896       CALL user_actions( 'after_integration' )
897
898!
899!--    If Galilei transformation is used, determine the distance that the
900!--    model has moved so far
901       IF ( galilei_transformation )  THEN
902          advected_distance_x = advected_distance_x + u_gtrans * dt_3d
903          advected_distance_y = advected_distance_y + v_gtrans * dt_3d
904       ENDIF
905
906!
907!--    Check, if restart is necessary (because cpu-time is expiring or
908!--    because it is forced by user) and set stop flag
909!--    This call is skipped if the remote model has already initiated a restart.
910       IF ( .NOT. terminate_run )  CALL check_for_restart
911
912!
913!--    Carry out statistical analysis and output at the requested output times.
914!--    The MOD function is used for calculating the output time counters (like
915!--    time_dopr) in order to regard a possible decrease of the output time
916!--    interval in case of restart runs
917
918!
919!--    Set a flag indicating that so far no statistics have been created
920!--    for this time step
921       flow_statistics_called = .FALSE.
922
923!
924!--    If required, call flow_statistics for averaging in time
925       IF ( averaging_interval_pr /= 0.0_wp  .AND.  &
926            ( dt_dopr - time_dopr ) <= averaging_interval_pr  .AND.  &
927            simulated_time >= skip_time_dopr )  THEN
928          time_dopr_av = time_dopr_av + dt_3d
929          IF ( time_dopr_av >= dt_averaging_input_pr )  THEN
930             do_sum = .TRUE.
931             time_dopr_av = MOD( time_dopr_av, &
932                                    MAX( dt_averaging_input_pr, dt_3d ) )
933          ENDIF
934       ENDIF
935       IF ( do_sum )  CALL flow_statistics
936
937!
938!--    Sum-up 3d-arrays for later output of time-averaged 2d/3d/masked data
939       IF ( averaging_interval /= 0.0_wp  .AND.                                &
940            ( dt_data_output_av - time_do_av ) <= averaging_interval  .AND. &
941            simulated_time >= skip_time_data_output_av )                    &
942       THEN
943          time_do_sla = time_do_sla + dt_3d
944          IF ( time_do_sla >= dt_averaging_input )  THEN
945             CALL sum_up_3d_data
946             average_count_3d = average_count_3d + 1
947             time_do_sla = MOD( time_do_sla, MAX( dt_averaging_input, dt_3d ) )
948          ENDIF
949       ENDIF
950
951!
952!--    Calculate spectra for time averaging
953       IF ( averaging_interval_sp /= 0.0_wp  .AND.  &
954            ( dt_dosp - time_dosp ) <= averaging_interval_sp  .AND.  &
955            simulated_time >= skip_time_dosp )  THEN
956          time_dosp_av = time_dosp_av + dt_3d
957          IF ( time_dosp_av >= dt_averaging_input_pr )  THEN
958             CALL calc_spectra
959             time_dosp_av = MOD( time_dosp_av, &
960                                 MAX( dt_averaging_input_pr, dt_3d ) )
961          ENDIF
962       ENDIF
963
964!
965!--    Call flight module and output data
966       IF ( virtual_flight )  THEN
967          CALL flight_measurement
968          CALL data_output_flight
969       ENDIF
970
971!
972!--    Profile output (ASCII) on file
973       IF ( time_dopr_listing >= dt_dopr_listing )  THEN
974          CALL print_1d
975          time_dopr_listing = MOD( time_dopr_listing, MAX( dt_dopr_listing, &
976                                                           dt_3d ) )
977       ENDIF
978
979!
980!--    Graphic output for PROFIL
981       IF ( time_dopr >= dt_dopr )  THEN
982          IF ( dopr_n /= 0 )  CALL data_output_profiles
983          time_dopr = MOD( time_dopr, MAX( dt_dopr, dt_3d ) )
984          time_dopr_av = 0.0_wp    ! due to averaging (see above)
985       ENDIF
986
987!
988!--    Graphic output for time series
989       IF ( time_dots >= dt_dots )  THEN
990          CALL data_output_tseries
991          time_dots = MOD( time_dots, MAX( dt_dots, dt_3d ) )
992       ENDIF
993
994!
995!--    Output of spectra (formatted for use with PROFIL), in case of no
996!--    time averaging, spectra has to be calculated before
997       IF ( time_dosp >= dt_dosp )  THEN
998          IF ( average_count_sp == 0 )  CALL calc_spectra
999          CALL data_output_spectra
1000          time_dosp = MOD( time_dosp, MAX( dt_dosp, dt_3d ) )
1001       ENDIF
1002
1003!
1004!--    2d-data output (cross-sections)
1005       IF ( time_do2d_xy >= dt_do2d_xy )  THEN
1006          CALL data_output_2d( 'xy', 0 )
1007          time_do2d_xy = MOD( time_do2d_xy, MAX( dt_do2d_xy, dt_3d ) )
1008       ENDIF
1009       IF ( time_do2d_xz >= dt_do2d_xz )  THEN
1010          CALL data_output_2d( 'xz', 0 )
1011          time_do2d_xz = MOD( time_do2d_xz, MAX( dt_do2d_xz, dt_3d ) )
1012       ENDIF
1013       IF ( time_do2d_yz >= dt_do2d_yz )  THEN
1014          CALL data_output_2d( 'yz', 0 )
1015          time_do2d_yz = MOD( time_do2d_yz, MAX( dt_do2d_yz, dt_3d ) )
1016       ENDIF
1017
1018!
1019!--    3d-data output (volume data)
1020       IF ( time_do3d >= dt_do3d )  THEN
1021          CALL data_output_3d( 0 )
1022          time_do3d = MOD( time_do3d, MAX( dt_do3d, dt_3d ) )
1023       ENDIF
1024
1025!
1026!--    Masked data output
1027       DO  mid = 1, masks
1028          IF ( time_domask(mid) >= dt_domask(mid) )  THEN
1029             CALL data_output_mask( 0 )
1030             time_domask(mid) = MOD( time_domask(mid),  &
1031                                     MAX( dt_domask(mid), dt_3d ) )
1032          ENDIF
1033       ENDDO
1034
1035!
1036!--    Output of time-averaged 2d/3d/masked data
1037       IF ( time_do_av >= dt_data_output_av )  THEN
1038          CALL average_3d_data
1039          CALL data_output_2d( 'xy', 1 )
1040          CALL data_output_2d( 'xz', 1 )
1041          CALL data_output_2d( 'yz', 1 )
1042          CALL data_output_3d( 1 )
1043          DO  mid = 1, masks
1044             CALL data_output_mask( 1 )
1045          ENDDO
1046          time_do_av = MOD( time_do_av, MAX( dt_data_output_av, dt_3d ) )
1047       ENDIF
1048
1049!
1050!--    Output of particle time series
1051       IF ( particle_advection )  THEN
1052          IF ( time_dopts >= dt_dopts  .OR. &
1053               ( simulated_time >= particle_advection_start  .AND. &
1054                 first_call_lpm ) )  THEN
1055             CALL data_output_ptseries
1056             time_dopts = MOD( time_dopts, MAX( dt_dopts, dt_3d ) )
1057          ENDIF
1058       ENDIF
1059
1060!
1061!--    Output of dvrp-graphics (isosurface, particles, slicer)
1062#if defined( __dvrp_graphics )
1063       CALL DVRP_LOG_EVENT( -2, current_timestep_number-1 )
1064#endif
1065       IF ( time_dvrp >= dt_dvrp )  THEN
1066          CALL data_output_dvrp
1067          time_dvrp = MOD( time_dvrp, MAX( dt_dvrp, dt_3d ) )
1068       ENDIF
1069#if defined( __dvrp_graphics )
1070       CALL DVRP_LOG_EVENT( 2, current_timestep_number )
1071#endif
1072
1073!
1074!--    If required, set the heat flux for the next time step at a random value
1075       IF ( constant_heatflux  .AND.  random_heatflux )  THEN
1076          IF ( surf_def_h(0)%ns >= 1 )  CALL disturb_heatflux( surf_def_h(0) )
1077          IF ( surf_lsm_h%ns    >= 1 )  CALL disturb_heatflux( surf_lsm_h    )
1078          IF ( surf_usm_h%ns    >= 1 )  CALL disturb_heatflux( surf_usm_h    )
1079       ENDIF
1080
1081!
1082!--    Execute user-defined actions
1083       CALL user_actions( 'after_timestep' )
1084
1085!
1086!--    Determine size of next time step. Save timestep dt_3d because it is
1087!--    newly calculated in routine timestep, but required further below for
1088!--    steering the run control output interval
1089       dt_3d_old = dt_3d
1090       CALL timestep
1091
1092!
1093!--    Synchronize the timestep in case of nested run.
1094       IF ( nested_run )  THEN
1095!
1096!--       Synchronize by unifying the time step.
1097!--       Global minimum of all time-steps is used for all.
1098          CALL pmci_synchronize
1099       ENDIF
1100
1101!
1102!--    Computation and output of run control parameters.
1103!--    This is also done whenever perturbations have been imposed
1104       IF ( time_run_control >= dt_run_control  .OR.                     &
1105            timestep_scheme(1:5) /= 'runge'  .OR.  disturbance_created ) &
1106       THEN
1107          CALL run_control
1108          IF ( time_run_control >= dt_run_control )  THEN
1109             time_run_control = MOD( time_run_control, &
1110                                     MAX( dt_run_control, dt_3d_old ) )
1111          ENDIF
1112       ENDIF
1113
1114!
1115!--    Output elapsed simulated time in form of a progress bar on stdout
1116       IF ( myid == 0 )  CALL output_progress_bar
1117
1118       CALL cpu_log( log_point_s(10), 'timesteps', 'stop' )
1119
1120
1121    ENDDO   ! time loop
1122
1123    IF ( myid == 0 )  CALL finish_progress_bar
1124
1125#if defined( __dvrp_graphics )
1126    CALL DVRP_LOG_EVENT( -2, current_timestep_number )
1127#endif
1128
1129    CALL location_message( 'finished time-stepping', .TRUE. )
1130
1131 END SUBROUTINE time_integration
Note: See TracBrowser for help on using the repository browser.